VPX Interface Hangs on NetScaler VPX Over SDX

VPX Interface Hangs on NetScaler VPX Over SDX

book

Article ID: CTX201184

calendar_today

Updated On:

Description

The following are the symptoms of this issue:

  • VPX hangs

  • Unable to access VPX

  • Need to reboot VPX to regain access

Resolution

To resolve this issue upgrade to latest NetScaler 10.5 GA build. For more information refer to Fixed Issues in Previous 10.5 Builds.

The following is the snippet from the preceding link:

When the Netscaler ADC hits congestion with HA or LACP packets or continuous congestion in a single-PE environment, it cannot recover and packet transmission stops. This is applicable to the management ports on NetScaler SDX appliances and to all ports on NetScaler VPX instances running on XenServer.
[From Build 55.8] # 532316, 532045, 533018, 534634, 534671

Problem Cause

The VPX is hitting a known issue where, after the interface enters NIC congestion state it is not able to come out and the fact that this issue happens only on management interface 0/1 and 0/2 with a single PE system running on SDX confirms the known issue.

There are three priority queues for traffic (high, medium, and low) in ADC interfaces. After the traffic comes to the NIC, based on the packet type it is assigned to a priority queue to be processed accordingly. Now because of design, the queue specific to management interfaces (0/1 and 0/2) are not getting cleared and it results in missing heartbeats even though there is no real traffic. After the packet in the priority queue is processed, the queue should be ideally cleared which does not happen.

The following counters were verified:

reltime:mili second between two records Sun Apr 26 20:15:11 2015
  Index   rtime totalcount-val      delta rate/sec symbol-name&device-no&time
      0 2554044 2 ( interface(0/1) )          2        0 net_cur_congested  Sun Apr 26 20:15:11 2015
      1   49667 2 ( interface(0/1) )          2        0 net_cur_devq_congested  Sun Apr 26 20:16:00 2015

104       0              4          1        0 nic_info_congestion_entered interface(0/1) Sun Apr 26 20:06:08
106       0        6801812         13        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:06:15
108       0        6801822         10        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:06:22
254       0        6802642         10        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:14:57
256       0        6802657         15        2 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:15:04
257    7032 2 ( interface(0/1) )   2         0 net_cur_congested  Sun Apr 26 20:15:11
259       0            804        804      114 nic_err_tx_overflow interface(0/1) Sun Apr 26 20:15:11
260       0        6802670         13        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:15:11
261       0            769        769      109 nic_err_congested_pkts_dropped interface(0/1) Sun Apr 26 20:15:11
262       0              5          1        0 nic_info_congestion_entered interface(0/1) Sun Apr 26 20:15:11
264       0            888         84       11 nic_err_tx_overflow interface(0/1) Sun Apr 26 20:15:18
265       0        6802683         13        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:15:18
266       0            818         49        6 nic_err_congested_pkts_dropped interface(0/1) Sun Apr 26 20:15:18
268       0            965         77       10 nic_err_tx_overflow interface(0/1) Sun Apr 26 20:15:25
269       0        6802691          8        1 nic_err_dropped_pkts interface(0/1) Sun Apr 26 20:15:25
270       0            860         42        5 nic_err_congested_pkts_dropped interface(0/1) Sun Apr 26 20:15:25
272       0           1041         76       10 nic_err_tx_overflow interface(0/1) Sun Apr 26 20:15:32

Issue/Introduction

The VPX and management interface hangs once the management interface enters NIC congestion.