For the ADC SDX single bundle upgrade to succeed, disk space used as reported by XenServer/Dom0 filesystem should be equal to or lower than 72%.

Before upgrading the SDX appliance, check the available space on XenServer and Dom0 partition. Run the df / command to check the amount of disk space used (Use%). If the disk space does not meet the requirement, don’t proceed with the upgrade until you’ve cleared the disk space. See the “How to clear disk space” section in this article about the steps to clear disk space.

Note that even if the system has the required disk space, sometimes the single bundle upgrade might fail, and after the failure disk space used is erroneously reported as 100%. See the “Recovery after a single bundle upgrade failure and when Dom0 shows 100%” section in this article for the solution.

Example 1: Failure scenario where disk use is greater than 72%

In the example below, disk space used by XenServer filesystem is reported as 75%. Amount of space used by logs in /var/log directory is reported over 400 Mb. This will cause the single bundle upgrade to fail.

[root@netscaler-sdx ~]# df /
Filesystem            1K-blocks      Used       Available     Use% Mounted on
/dev/sda2              4128448   2927040    991696       75% /
[root@netscaler-sdx ~]#
[root@netscaler-sdx ~]#
[root@netscaler-sdx ~]# cd /var/log/
[root@netscaler-sdx log]# du . -sh
483M    .

 
For SDX appliances in which RAID is deployed, disk reported disk partition is /dev/md_d0p2 instead of /dev/sda2.

Example 2: Working scenario where disk use is less than or equal to 72%

[root@netscaler-sdx]# df /
Filesystem                 1K-blocks      Used      Available   Use% Mounted on
/dev/md_d0p2           4128448   2821304   1097432   72% /
 
cd /var/log
 
[root@netscaler-sdx log]# df /
Filesystem                  1K-blocks      Used     Available   Use% Mounted on
/dev/md_d0p2           4128448   2821304   1097432   72% /
 
[root@netscaler-sdx ~]# cd /var/log/
[root@netscaler-sdx log]# du . -sh
215M

Recovery after a single bundle upgrade failure due insufficient disk space

In case of a single bundle upgrade failure, disk space used is reported as 100%. This condition is sticky, and Use% is always reported as 100%. Reboot the SDX appliance to bring the disk use percentage back to normal. Run the df / command to check the disk space used. If disk space use is still high, follow the steps mentioned in the following “How to clear disk space” section to clear the disk space used. Upgrade the appliance again after the disk space use percentage is equal to or lower than 72% on /dev/sda2 (or /dev/md_d0p2 ) partition.

How to clear disk space

The SDX appliance accumulates large number of log files over time, in the /var/log directory on the file system. To free space from the /var/log directory, follow these steps:

1. Run the df / command to check and see if available disk space Use% reported is 100%, if yes, reboot the SDX appliance first to bring the disk Use% percentage back to normal. Wait for reboot to complete, now run the df / command to check the disk space used.

2. Run the following command on XenServer / Dom0 console to cap the log rotate to 175 Mb instead of 500 Mb
   sed -i.bak 's/500/175/' /opt/xensource/bin/delete_old_logs_by_space

3. Wait for a minute for log rotate to run and do the necessary clean up to reduce the accumulated logs to around 175 Mb.

4. Run the df / command and verify that the XenServer / Dom0 available disk space Use% reported is less than 72% for the upgrade to succeed.

Note: A fix will be provided in the upcoming release where above steps will not be not necessary.

Refer to Citrix Documentation for detailed information:https://docs.citrix.com/en-us/citrix-adc/12-1/upgrade-downgrade-citrix-adc-appliance/troubleshooting.html

NetScaler SDX single bundle upgrade failure when upgrading from 11.1-58.13 or 12.0.57.24 to 12.0.58.X or 12.1.48.13 software release.