When we try to configure NSWL we get the the below output in the log.conf:
"Login successful (nspe=192.168.235.123:user=nsroot:socket=5) 

RPC Signature is valid 
Received Invalid Ack for login command (2) 
Reconnect (nspe=192.168.235.123:user=nsroot) 
RPC Signature is valid 
Login successful (nspe=192.168.235.123:user=nsroot:socket=5)" 

 

To enable the client to use MSS and if it is already enabled, then increase the MSS value. 
 

---

Problem Cause

Customer was using NS running on 11.0 64.34nc release and the NSWL was installed on a windows server. 
The configuration from NS is just to enable the feature and on the client we need to add the netscaler ip address. 
You can refer to http://support.citrix.com/article/CTX110498 to understand the configuration part. 


When a TCP dump was performed and we filtered the traffic between the NSWL client and the NetScaler NSIP we could see that the TCP handshake was starting with MSS of 536. 
Since the client is not advertising the MSS in the handshake we are picking the least MSS value. 
We followed https://technet.microsoft.com/en-us/library/cc957539.aspx and checked EnablePMTUDiscovery registry key which was set to 0 which means that Windows server tries to use lowest MSS possible. Set it to 1 and rebooted the client, tested. 
After reboot we observed that Windows server now advertises MSS higher than 512 and both parties agree on that higher value. 

After this the NSWL started to work and we were no longer noticing the error messages.