After upgrading from older build of NetScaler to either NetScaler 11 or NetScaler 10.5 58.x or higher, secure LDAP does not work.

Running the aaad.debug we see failure with 4005.

To resolve this issue, complete one of the following procedures:

1. Enable TLS 1.1/1.2 support on the Windows 2003 server.

2. Force NetScaler to send TLS 1.0 by default on the back end.
   The only way to accomplish this is to create an intermediary load balancing VIP with SSL services defined with TLS 1.1 and TLS 1.2 disabled. The connection flow would be you connect to NetScaler Gateway VIP, authentication to LDAP first hits the load balancing VIP which has the LDAP server defined as an SSL service (which allows us to set the SSL negotiation).

Problem Cause

Change in the default behavior of the NetScaler around 10.5 57.X. In older builds (10.1 and earlier) the default SSL back end connection was using SSLv3. In later builds (10.5 and later) we have changed the default SSL back end to use TLSv1.2. 

In this case the environment has Windows Server 2003 in the back end which out of box does not provide TLS1.1/1.2 support. You can verify this by running the following commands:
openssl s_client -host 205.69.224.68 -port 636 -tls1 (CONNECTS)
openssl s_client -host 205.69.224.68 -port 636 -tls1_1 (FAILS)
openssl s_client -host 205.69.224.68 -port 636 -tls1_2 (FAILS)

Latest IIS servers with TLS 1.2 support needs “Signature Algorithms” extension to be sent from client hello to continue the TLS 1.2 Handshake.

The immediate workaround will be to disable only TLS 1.2 at back end service/service group:
set ssl service <service_name> -tls12 di
set ssl servicegroup <servicegroup_name> -tls12 di

The following is a trace with same client hello (NetScaler) sent to Apache/Windows back end servers:
nstrace1.pcap--------------àtcp.stream eq 52----------àApache server------------TLS1.2 Handshake passes with SHA256 certificate
nstrace1.pcap-------------àtcp.stream eq 43----------àWindows IIS server-----àHandshake fails

As per RFC 5246:
7.4.1.4.1.  Signature Algorithms

• If the negotiated key exchange algorithm is one of (RSA, DHE_RSA,

• DH_RSA, RSA_PSK, ECDH_RSA, ECDHE_RSA), behave as if client had sent the value {sha1,rsa}.

• If the negotiated key exchange algorithm is one of (DHE_DSS, DH_DSS), behave as if the client had sent the value {sha1,dsa}.

• If the negotiated key exchange algorithm is one of (ECDH_ECDSA, ECDHE_ECDSA), behave as if the client had sent value {sha1,ecdsa}.

Note: this is a change from TLS 1.1 where there are no explicit rules, but as a practical matter one can assume that the peer supports MD5 and SHA-1.