After upgrading the netscaler to any latest build customer might notice that they are not able to access Gateway Vservers or LB Vservers even when they are showing UP on Netscaler.

First make sure that Vserver status on Netscaler is showing UP.

Then took nstrace on netscaler and asked the user to generate the traffic and once user got "Page cannot be displayed" on browser stop the nstrace. This is to confirm if any packets are reaching netscaler or not.

From nstrace if we filter with client IP and Vserver IP we see that traffic is coming to Netscaler and Netscaler is replying back with SYN,ACK as shown below. But the handshake never completes after that and same pattern follows in the nstrace:



Further investigation shows that it seems like a asymmetrical routing issue where SYN packet is coming to Netscaler on one interface but Netscaler is sending SYN,ACK on a different interface as shown below:

For SYN packet Netscaler header shows below:



For SYN,ACK packet the Netscaler header shows below the different Nic no as compared to above screenshot:



So there can be below possible solutions:

a) Enable MBF (MAC Based Forwarding) feature on Netscaler to get immediate fix for the above issue so that netscaler makes sure that interface from which the request is coming will be used for replying back. Else firewalls will drop seeing the above behaviour.

b) Check if there are 2 static default routes on Netscaler as shown below:

        Network          Netmask          Gateway/OwnedIP  State   Traffic Domain  Type
        -------          -------          ---------------  -----   --------------  ----
1)      0.0.0.0          0.0.0.0          10.1.1.1       UP      0              STATIC
2)      0.0.0.0          0.0.0.0          159.168.1.1   UP      0              STATIC

The show arp output from Netscaler shows below:

2)      10.1.1.1       00:00:0c:07:ac:01  0/1   1     DYNAMIC    465    0

6)      159.1681.1    00:1b:17:00:01:16  1/1   1     DYNAMIC    466    0

Show interface output shows below:

Interface 0/1 (Gig Ethernet 10/100/1000 MBits) #8

Interface 1/1 (Gig Ethernet 10/100/1000 MBits) #7


There may be a possibility that before the reboot when everything was working netscaler will make use of correct route of the above two and will send the reply back through correct interface.

After the reboot Netscaler might start using a different route for which Netscaler learnt the ARP from different interface as shown above.

So keeping one static default route which is needed, should also fix this issue.

 

---

Problem Cause

The issue can be caused if there are 2 static default routes on Netscaler.

After upgrading the netscaler to any latest build customers might notice that they are not able to access Gateway Vservers or LB Vservers even when they are showing UP on Netscaler