During new update push from Citrix Cloud, the connector continuously reboots in a never ending loop.
The reboot can be stopped temporarily if we disable the "Citrix Cloud Services Agent WatchDog" service.
But if the CC update was incomplete, the connector would stay in failed mode.

System event log on CC shows following entry for every reboot :

Information,3/3/2020 5:57:42 PM,User32,1074,None,"The process C:\Windows\System32\shutdown.exe (<hostname>) has initiated the restart of computer <hostname> on behalf of user NT AUTHORITY\SYSTEM for the following reason: Application: Installation (Planned)
 Reason Code: 0x80040002
 Shutdown Type: restart
 Comment: Citrix Cloud restarted the machine to begin the Connector update.

 

There are some locations on operating system where it stores flag for pending reboot:

Look for presence of 'PendingFileRenameOperations' inside following key:
HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\

Look for presence of key 'RebootPending' as below:
HKLM:\Software\Microsoft\Windows\CurrentVersion\Component Based Servicing\RebootPending

Look for presence of key 'RebootRequired' as below:
HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\RebootRequired

Most of these flags will be cleared with first reboot of operating system.
But in some special cases, 'PendingFileRenameOperations' would continue to remain inside HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\ even after reboot. 
The value 'PendingFileRenameOperations'  stores the names of files that the operating system will rename when it restarts. The key consists of pairs of file names. The operating system renames the file in the first item of the pair to match the second item of the pair after the reboot.

In such cases, we should check for the value data of 'PendingFileRenameOperations' from registry to identify the files that are pending for rename. Once we know the application responsible for those files (mostly a 3rd party application), follow below steps:

1. Disable 'Citrix Cloud Services Agent WatchDog' to stop the reboot loop temporarily.
2. Disable the identified applications service(s) OR uninstall the application completely.
3. Delete 'PendingFileRenameOperations' from HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\
4. Reboot the server manually and confirm 'PendingFileRenameOperations' doesn't appear any more inside HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\
5. Enable 'Citrix Cloud Services Agent WatchDog' to automatic and start this service to begin CC update.

The pending connector update will begin in few minutes.
Run a cloud connector health check after the upgrade to ensure everything is fine.

---

Problem Cause

The operating system has pending reboot flags on registry or some file rename actions is pending for reboot.