This article describes different methods used to troubleshoot load balancing issues.

• Qfarm
• QueryDS
• Logging
• Load Manager Monitor
• CDF Tracing
• Default Versus Advanced Load Evaluator
• Multiple Zones
• Intelligent Load Biasing (ILB)
• More Information

Qfarm

QFARM used without any switches retrieves information directly from the data store. Running QFarm without any switches is a good troubleshooting step to determine IMA connectivity to the data store. Use the qfarm utility with the respective switches to monitor server and application loads on all servers in the farm. Load balancing information is obtained from the Zone Data Collector(s) and the Local Host cache.
Issuing the qfarm /load command displays the load for all servers in the farm.

The qfarm /app command displays the load for all applications and servers in the farm. All Apps in the following screen shot show a load of 99999, as no load evaluator is assigned to any of the applications.

The following commands can also be used to monitor loads on all servers in a particular zone:
qfarm /zone
qfarm /zoneapp

Note: Logons must be enabled on a server for qfarm to report a load. If logons are disabled, qfarm does not return any information. Refer to the following screen shots.

After enabling logons on the server, qfarm /load reports a correct value.

QueryDS

Use queryds to view Dynamic Store tables. Queryds is not installed as part of the Citrix XenApp installation. This tool can be found on the Citrix XenApp CD-ROM in \\Support\debug\<OSPlatform>.
Compare queryds connected and disconnected tables against what is visible in the Citrix AppCenter Console.
Issue the queryds tables command to get a full list of available tables to query.

Issue the queryds /table:Conn_Sessions command to view all connected sessions.

Issue the queryds /table:Disc_Sessions to view all disconnected sessions. Another table to query is LMS_ServerLoadTable. This shows the real load along with information as reported to the Data Collector.

Based on this value, the Data Collector redirects the next session to the least loa​ded server.
Default Load Evaluator:
ProtocolMask-Bias:64HEX = 100 Decimal
RuleLoads:b:1; = Server user load
Load:64HEX = 100 Decimal
Advanced Load Evaluator:
ProtocolMask-Bias:c8HEX = 200 Decimal
RuleLoads:3:31;1:16;6:25; = Memory Usage, CPU Utilization, Page Swaps
Load:1473HEX = 5235 Decimal

RuleLoads: (For reference when using queryds /table:LMS_ServerLoadTable)

Server User load b
Application user load a
Context switches 2
CPU Utilization 1
Disk Data I/O 7
Disk operations 8
IP Range 9
Memory Usage 3
Page Faults  4
Page Swaps 6
Scheduling   5

Logging

Enable Logging in the Citrix XenApp Console. The following screen shot has the Enable Logging button highlighted.

Once enabled, logging records all connections to the farm.

Load Manager Monitor

Verify that the Load Manager Monitor tab displays the correct information.

Default Load Ev​aluator Set

Advanced Load Evaluator Set

CDF Tracing

Create a Citrix Diagnostics Facility (CDF) trace by tracing the following modules:

Load Management

Logon/Logoff

Default Vs Advanced Load Evaluator

Refer to the following articles for more information:
CTX111965 – Performance Monitor Counters Used by Load Manager
CTX115686 – XenApp Servers Return a Full Load when Advanced or Custom Load Evaluators are Assigned
CTX105449 – Load Manager Rules Explained
The Default Load Evaluator is based on the Load Throttling and Server User Load rules.

The Advanced Load Evaluator is based on the CPU Utilization, Load Throttling, Memory Usage, and Page Swaps rules.

If there is an issue with the Advanced or Custom Load Evaluator, test with the Default Load Evaluator. If loads are reported correctly with the Default Load Evaluator it might indicate that there is a problem with the system performance counters of the server.

To rebuild corrupt performance counters refer to the Microsoft article - How to manually rebuild Performance Counter Library values.

Multiple Zones

If there are multiple zones, ensure to configure the Data Collectors to share their load information. Use one of the following two methods:

• Select the Share load information across zones check box under <FarmName>\Properties\Zones in the Citrix XenApp Advanced Configuration Console.

-Or-

• Enable and configure the Zone Preference and Failover policy in Policies\PolicyName\Properties\UserWorkspace\Connections in the Citrix XenApp Advanced Configuration Console.

To reduce network traffic in large farms with multiple zones, Citrix recommends to use the Zone Preference and Failover policy rule to direct user requests for applications to preferred zones within the farm.

Intelligent Load Biasing (ILB)

Intelligent Load Biasing (or Load Throttling) is a new feature of the Load Balancing subsystem.
Incoming connections to a load balanced farm fail to fully connect. This occurs when a new server comes online during peak logon periods. At that time, the load balancer is sending all incoming connections to the new server, essentially overwhelming and preventing it from updating its server load completely.
With Load Throttling, logons are given a logarithmic load bias during connection time to limit the number of simultaneous logon requests. This biasing level is used in conjunction with the server's real load to route connections to the least-loaded server. This allows time for servers to gradually increment the number of connections in environments where the server load is much below the farm average load, which is often the case when a server is restarted.

How to Use Load Throttling

Load Throttling uses Intelligent Load Biasing (ILB). ILB works by giving logons a higher load bias. The default ILB algorithm assigns a bias of ½ the remaining load capacity. The default algorithm is: (New) Current Resolution load = (Old) Current Resolution Load + [(Max Load - Old Current Resolution Load) / 2 ]. The ILB adjusts itself back after pending logons are complete.
For example:

Note: All servers must have ILB enabled in the Default and/or Advanced Load Evaluator for the load to be shared evenly among all servers.

More Information

CTX112423 – Load Manager Administrator's Guide
CTX103242 – Servers Are Missing in the Load Balancing Usage Report
CTX104926 – Data Collector Resolves Load Balance Requests to a Single Server