Sometimes Citrix Gateway login page takes a long time to be presented on the client’s browsers. When this issue occurs, you might observe any of the following conditions.

• The number of established connections to Apache counter has hit the default configured limit of 30 (or a customized value, if modified). Along with this, pq_tot_waitTime counter has increased. The increase in pq_tot_waitTime of nshttpd-vpn-127.0.0.1-81 service indicates that Apache is taking a long time to respond (Only in 13.0-82.x and later builds).

root@ns# nsconmsg -d current -K newnslog -s disptime=1 | grep nshttpd-vpn-127.0.0.1-81
1226970       0             29         -1        0 si_cur_ConnEstablished server_svc_cfg_NSSVC_HTTP_127.0.0.1:81(nshttpd-vpn-127.0.0.1-81) Tue Oct 26 13:44:23 2021
1227880       0 50143902613118 1259491802 179927400 pq_tot_waitTime surgeQ_server_127.0.0.1:81(nshttpd-vpn-127.0.0.1-81) Tue Oct 26 13:44:23 2021

• The number of httpd process might have hit the maximum set limit that is indicated by the maxClients parameter in httpd.conf

• Server busy entry in the log file /var/log/httperror.log or /var/log/httperror-vpn.log

Example:
httperror.log: [Mon Mar 16 07:00:31 2020] [info] server seems busy, (you may need to increase StartServers, or Min/MaxSpareServers), spawning 8 children, there are 0 idle, and 19 total children

For releases 13.1, 13.0-82.x and above

From Citrix ADC 13.0-82.x and later, there is an optimization to limit the number of connections ADC can open to Apache. In case of a surge in traffic, ADC ensures that the number of connection requests to Apache does not exceed Apache’s maxClient limit, ensuring low response times from Apache.
To configure the limit on connections from ADC to Apache, maxClients parameter of service named nshttpd-vpn-127.0.0.1-81 can be customized. The default value of this parameter is 30.

To get optimal performance, MaxClients in httpd.conf and maxClients parameter of nshttpd-vpn-127.0.0.1-81 service must be tuned as mentioned below.

1. Set the maxClients parameter in /etc/httpd.conf

   1. Before changing the file, ensure to take a backup of original /etc/httpd.conf.

   2. In /etc/httpd.conf, search for “MaxClients” string and increase the value to 60. (This allows Apache to handle 60 connections in parallel)

   3. To make the change reboot persistent, run the following commands.

      1. cp /etc/httpd.conf /nsconfig/

      2. rm /etc/httpd.conf

      3. ln -s /nsconfig/httpd.conf /etc/httpd.conf

   4. Restart Apache using apachectl restart

2. Set the maxClients limit on the service (nshttpd-vpn-127.0.0.1-81) using the following NSCLI command and save the config.

Note: As the service nshttpd-vpn-127.0.0.1-81 is specific for gateway end-user portal traffic to Apache, it is recommended to have a slightly lower limit than the value configured in httpd.conf. This ensures that enough connections are allocated to Admin UI traffic even if there is a surge in end user traffic. A difference of at least 15 is recommended.
 

3. If the response times are high even with a limit of 60, a higher limit is needed. Repeat Step 1 and Step 2 to increase the maxClient values on the service as well as in httpd.conf in steps of 15 or 20 until a satisfactory response time is achieved. Kindly reach out to Citrix support if there is no performance improvement even on reaching a limit of 255.

Note: The value can be increased up to 255 but having a limit of more than required might result in a memory crunch. Higher values(>120) are not recommended unless found necessary while tuning the performance.

Example snippet of the default httpd.conf