AppQoE HTTP DoS protection enabled, clients were not getting the HIC Response Challenge
Article ID: CTX250221
Updated On:
Description
Test / Validation Steps performed
1. AppQoE Policy and Action is Configured on LB with Dos Action as HICResponse.
2. AppQoE Parameter "DOS Attack Threshold" is set to 1.
3. Opened multiple sessions on the LB from different browsers but it's not challenging the users with the HIC Response although client connection count is more than 1
Resolution
The Dos action triggers when the SurgeQueue Build up happens.
https://docs.citrix.com/en-us/netscaler/12/appexpert/appqoe/appqoe-parameters.html
dosAtckThrsh
The denial-of-service attack threshold. The number of connections that must be waiting in queues before the ADC responds with DoS protection measures. Default value: 2000 Minimum value: 0 Maximum value: 4,294,967,294
To test the Dos Action:
Option-1 Generate excessive HTTP Load on LB that either the LB or Server cannot cope with
Or
Option-2: Bind a ping monitor to the Service bound on LB and Disable the web service on the server e.g. stop the Apache or IIS Service, this was ADC thinks the server is up but the requests will fail on back-end
Both of these action eventually will result in surge queue build up, which can be checked in realtime using the shell command "nsconmsg -d oldconmsg -s ConLb=2 -j <LB_NAME>
In the sample lab output below, 1.1.1.1 is the LB and 2.2.2.2 is the service and the SQ count is zero, once this value crosses the AppQoE Parameter "DOS Attack Threshold" the Dos action will trigger.
nsoot@NS# nsconmsg -d oldconmsg -s ConLb=2 -j "1.1.1.1"
Displaying debug performance information
NetScaler V20 Performance Data
NetScaler NS12.1: Build 49.23.nc, Date: Aug 24 2018, 21:04:47 (64-bit)
current time is Mon Apr 15 12:17:44 2019
-------------------------------------------------------
NATSession : Free(7280)A(7280)InUse(0)
NATSession: Cur(Tcp[0] Udp[0] Icmp[0] Other[0])
NATSession: Op/s(Tcp[0] Udp[0] Icmp[0] Other[0])
Session: A:0 F:0 IUse:0 SEs: SIP:0 C:0 SSL:0 Svr:0 UserId:0 SIPDIP:0 DIP:0 SO:0 FIXSESSION:0 USERSESSION : 0
SSF: Conn (Srvr 0 Clnt 0) U:0
CM: Conn (Srvr 0 Clnt 0) Sessions PCB 0 NATPCB 0
Z(SIP[0], C[0], SSL[0] Server[0] SIPDIP[0] DIP[0] SO[0])
Mon: Probes: 1106976, Failed: 147625
VIP(1.1.1.1:80:UP:LEASTCONNS): Hits(0, 0/sec) Mbps(0.00) Pers(OFF) Err(0) SO(0) LConn_Best [Idx:SubIdx] 0:0 PrimVserverDownBackupHits(0)
Pkt(0/sec, 0 bytes) actSvc(1) DefPol(RR) override(0) newlyUP(0)
Conn: Clt(0, 0/sec, OE[0]) Svr(0) SQ(Total: 0 OnVserver: 0 OnServices: 0)
slimit_SO: (Sothreshhold: 0 [Ex: 0] Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 0
S(2.2.2.2:UP) Hits(0, 0/sec, P[0, 0/sec]) ATr(0:0) Mbps(0.00) BWlmt(0 kbits) RspTime(0.00 ms) Load(0) LConn_Idx: (C:0; V:0,I:1, B:0, X:0, SI:0)
Other: Pkt(0/sec, 0 bytes) Wt(1) Wt(Reverse Polarity)(10000)
Conn: CSvr(0, 0/sec) MCSvr(0) OE(0) E(0) RP(0) SQ(0)
slimit_maxClient: (MaxClt: 0 [Ex: 0] Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 0)
newlyUP_mode: NO, Pending: 0, update: 0x0, incr_time: 0x0, incr_count: 0
-------------------------------------------------------
CPU:0.8% MEM:377769048 UP:04.09:55:23 since:Thu Apr 11 02:22:21 2019
https://docs.citrix.com/en-us/netscaler/12/appexpert/appqoe/appqoe-parameters.html
dosAtckThrsh
The denial-of-service attack threshold. The number of connections that must be waiting in queues before the ADC responds with DoS protection measures. Default value: 2000 Minimum value: 0 Maximum value: 4,294,967,294
To test the Dos Action:
Option-1 Generate excessive HTTP Load on LB that either the LB or Server cannot cope with
Or
Option-2: Bind a ping monitor to the Service bound on LB and Disable the web service on the server e.g. stop the Apache or IIS Service, this was ADC thinks the server is up but the requests will fail on back-end
Both of these action eventually will result in surge queue build up, which can be checked in realtime using the shell command "nsconmsg -d oldconmsg -s ConLb=2 -j <LB_NAME>
In the sample lab output below, 1.1.1.1 is the LB and 2.2.2.2 is the service and the SQ count is zero, once this value crosses the AppQoE Parameter "DOS Attack Threshold" the Dos action will trigger.
nsoot@NS# nsconmsg -d oldconmsg -s ConLb=2 -j "1.1.1.1"
Displaying debug performance information
NetScaler V20 Performance Data
NetScaler NS12.1: Build 49.23.nc, Date: Aug 24 2018, 21:04:47 (64-bit)
current time is Mon Apr 15 12:17:44 2019
-------------------------------------------------------
NATSession : Free(7280)A(7280)InUse(0)
NATSession: Cur(Tcp[0] Udp[0] Icmp[0] Other[0])
NATSession: Op/s(Tcp[0] Udp[0] Icmp[0] Other[0])
Session: A:0 F:0 IUse:0 SEs: SIP:0 C:0 SSL:0 Svr:0 UserId:0 SIPDIP:0 DIP:0 SO:0 FIXSESSION:0 USERSESSION : 0
SSF: Conn (Srvr 0 Clnt 0) U:0
CM: Conn (Srvr 0 Clnt 0) Sessions PCB 0 NATPCB 0
Z(SIP[0], C[0], SSL[0] Server[0] SIPDIP[0] DIP[0] SO[0])
Mon: Probes: 1106976, Failed: 147625
VIP(1.1.1.1:80:UP:LEASTCONNS): Hits(0, 0/sec) Mbps(0.00) Pers(OFF) Err(0) SO(0) LConn_Best [Idx:SubIdx] 0:0 PrimVserverDownBackupHits(0)
Pkt(0/sec, 0 bytes) actSvc(1) DefPol(RR) override(0) newlyUP(0)
Conn: Clt(0, 0/sec, OE[0]) Svr(0) SQ(Total: 0 OnVserver: 0 OnServices: 0)
slimit_SO: (Sothreshhold: 0 [Ex: 0] Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 0
S(2.2.2.2:UP) Hits(0, 0/sec, P[0, 0/sec]) ATr(0:0) Mbps(0.00) BWlmt(0 kbits) RspTime(0.00 ms) Load(0) LConn_Idx: (C:0; V:0,I:1, B:0, X:0, SI:0)
Other: Pkt(0/sec, 0 bytes) Wt(1) Wt(Reverse Polarity)(10000)
Conn: CSvr(0, 0/sec) MCSvr(0) OE(0) E(0) RP(0) SQ(0)
slimit_maxClient: (MaxClt: 0 [Ex: 0] Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 0] Available: 0)
newlyUP_mode: NO, Pending: 0, update: 0x0, incr_time: 0x0, incr_count: 0
-------------------------------------------------------
CPU:0.8% MEM:377769048 UP:04.09:55:23 since:Thu Apr 11 02:22:21 2019
Problem Cause
Normally accessing an LB was not introducing a Surge Queue Build UP in test environment
Was this article helpful?
Yes
No
Powered by
