After upgrade to SDX 13.1 27.59 or lower 13.1 version firmware, you may find SDX booted with SVM running, but VPX in Halted state.
Try launch the VM you get the following error:

"INTERNAL_ERROR xenopsd internal error: Unix.Unix_error(Unix.ENODEV. "write". "")

Checking mps_control.log on SVM, you may find the following error logs:
7 Aug 22 12:34:21.251 +0800 [Error] [Main] MPSControl::run_post_install_and_reboot::Error: File (postinst.sh) upload to Hypervisor failed
7 Aug 22 12:34:21.251 +0800 [Error] [Main] Exception in run_post_install_and_reboot: Exception: MPSControl::run_post_install_and_reboot::Error: File (postinst.sh) upload to Hypervisor failed
 

Once determined postinst.sh upload issue from log, you can manually run postinst.sh to continue post upgrade install, then SDX will recover.
Operation Guide:
Step 1. Check the mpscontrol.log on SVM to find what are the arguments that postinst.sh you need.

cd /var/mps/log
grep postinst mpscontrol.log*
//Find the shell execution log like following
31 May 22 21:11:36.970 +0800 [Debug] [Main] Executing "sh /postinst.sh 71000 450085 >/var/log/postinst.log 2>&1; echo $?" command on 169.254.0.1

//Note: Here we are trying to find the argument 450085, which is netscaler.sysid, it varies according to the platform, if you can not find old sh command logs, try grep netscaler.sysid in mps_control logs instead.

Step 2. Manually copy the postinst.sh from SVM to XenServer

scp /var/mps/generated_postinst.sh root@169.254.0.1:/postinst.sh

Step 3. Then go to XenServer shell, run the postinst.sh with proper arguments, reboot once script completes

##On SVM CLI
ssh root@169.254.0.1
##Into XenServer SSH
sh /postinst.sh 71000 450085 >/var/log/postinst.log 2>&1
reboot

After reboot, VPX state should go back to Running.
 

---

Problem Cause

A SDX firmware known issue NSHELP-31125 may cause this issue, where SCP postinst.sh may fail during upgrade for 13.1 firmware lower than 27.59.
Citrix ADC Engineering is already working on a fix for this issue.

A known SDX 13.1 firmware issue may cause upgrade failure, VPX in halted state. Manually running postinst.sh in XenServer can help resolve the issue.