The Citrix Broker Service experiences memory usage spikes, with memory consumption increasing significantly over a short period. This may lead to degraded performance or instability on the Delivery Controller.

 

Affected Environments:

• CVAD environments with manually provisioned VDAs or PVS VDAs provisioned via the CVAD Setup Wizard.

• CVAD environments with these combinations of Delivery Controller and VDA versions:

  • Delivery Controller: 2311, 2402, 2402 CU1, 2407, 2411

  • Windows VDA: 2311, 2402, 2402 CU1, 2407
  • Linux VDA: 2311

Unaffected Environments:

• Non-AD Domain Joined VDAs

• VDAs bootstrapped using the WebSocket enrollment tool. 
• Mac VDAs.

The above mentioned sample code is provided to you as is with no representations, warranties or conditions of any kind. You may use, modify and distribute it at your own risk. CITRIX DISCLAIMS ALL WARRANTIES WHATSOEVER, EXPRESS, IMPLIED, WRITTEN, ORAL OR STATUTORY, INCLUDING WITHOUT LIMITATION WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NONINFRINGEMENT. Without limiting the generality of the foregoing, you acknowledge and agree that (a) the sample code may exhibit errors, design flaws or other problems, possibly resulting in loss of data or damage to property; (b) it may not be possible to make the sample code fully functional; and (c) Citrix may, without notice or liability to you, cease to make available the current version and/or any future versions of the sample code. In no event should the code be used to support ultra-hazardous activities, including but not limited to life support or blasting activities. NEITHER CITRIX NOR ITS AFFILIATES OR AGENTS WILL BE LIABLE, UNDER BREACH OF CONTRACT OR ANY OTHER THEORY OF LIABILITY, FOR ANY DAMAGES WHATSOEVER ARISING FROM USE OF THE SAMPLE CODE, INCLUDING WITHOUT LIMITATION DIRECT, SPECIAL, INCIDENTAL, PUNITIVE, CONSEQUENTIAL OR OTHER DAMAGES, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Although the copyright in the code belongs to Citrix, any distribution of the sample code should include only your own standard copyright attribution, and not that of Citrix. You agree to indemnify and defend Citrix against any and all claims arising from your use, modification or distribution of the sample code.

Upgrade Recommendation: Customers should upgrade their Delivery Controllers (DDCs) to version 2411 or higher and all VDAs to version 2411 or higher.

LTSR 2402 Customers: A solution will be included in a future Cumulative Update (CU) release.

In the meantime, you can implement the following steps as a workaround based on your deployment type. Please note that these steps are unnecessary if both the DDCs and VDAs are upgraded to the recommended versions.

The following three registry settings should be added to the affected VDAs:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket

1. Name: CustomerId, Type: REG_SZ, Value: CitrixOnPremises

2. Name: SiteAddress, Type: REG_SZ, Value: https://localhost

3. Name: VirtualSiteId, Type: REG_SZ, Value: 00000000-0000-0000-0000-000000000000

Step 1: Adding Registry Settings to Fix the Issue

To determine if a VDA is affected and requires a workaround, follow these steps:

• Check Unaffected Environments: If the VDA is listed in the "Unaffected Environments" section, the VDA is not impacted and no action is needed.

• Check Affected Environments and VDA/DDC Version: If the VDA and its corresponding DDC version are listed in the "Affected Environments" section, the VDA is affected and requires the following workaround:

Windows PVS VDAs: Update the registry in the vDisk

• Create a new version of the PVS vDisk in private mode

• Boot a target VM in maintenance mode, change the registry settings directly and shutdown

• Promote the new vDisk version to Production and reboot all VDAs using it.

Linux PVS VDAs: Update the setting in the vDisk

• Create a new version of the PVS vDisk in private mode

• Boot a target in maintenance mode and use ‘ctxreg’ to update the required settings and then shutdown:
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “CustomerId” -d “CitrixOnPremises” --force
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “SiteAddress” -d “https://localhost” --force
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “VirtualSiteId” -d “00000000-0000-0000-0000-000000000000” --force

• Promote the new vDisk version to Production and reboot all VDAs using it.

Manual Windows VDAs: use a GPO to update the required registry settings

• Open Group Policy Management Console (GPMC).

• Create a new GPO or edit an existing GPO.

• Add the required registry settings.

• Apply the GPO to the necessary organizational units (OUs) or directly to the impacted VDA machines.

• Allow time for the GPO to propagate and 1 more hour for the change to take effect (it is not required to reboot the VDA machine or to restart the Broker Agent service).

Manual Linux VDAs: Use `ctxreg` to update the required registry settings on every VDA:

• Run the following commands on every VDA:
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “CustomerId” -d “CitrixOnPremises” --force
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “SiteAddress” -d “https://localhost” --force
  sudo /opt/Citrix/VDA/bin/ctxreg create -k “HKLM\SYSTEM\CurrentControlSet\Services\CitrixBrokerAgent\WebSocket” -t “REG_SZ” -v “VirtualSiteId” -d “00000000-0000-0000-0000-000000000000” --force

• Allow 1 hour for the changes to take effect or restart the Broker Agent Service to make the changes effective immediately.

Step 2: Cleanup Excessive and Duplicate Data

Run the following SQL script to remove duplicate VDA service keys from the TrustSchema.ServiceKeys table:

---

Problem Cause

When the Broker Agent Service starts, it attempts to retrieve a registered VDA service key from its local storage. If it cannot find one, it registers a new VDA service key with the FMA Trust Service. However, due to an issue, the service fails to retain the key to its local storage. As a result, each time the service starts, the process repeats.

Over time, this leads to an accumulation of VDA service keys in the service key table of the FMA Trust Service. Compounding this issue, the table lacks an index on the InstanceId column. As the number of keys grows, searching the service key table places a significant load on the SQL database, slowing down database operations. This increased load eventually leads to database deadlocks, further degrading performance.

The overall impact is that most operations of the Delivery Controller (DDC) become extremely slow or fail to complete.