Citrix ADC 12.x - Services are going down during a specific time

Citrix ADC 12.x - Services are going down during a specific time

book

Article ID: CTX238862

calendar_today

Updated On:

Description

  •  Services going down during a specific time and the application availability is completely lost.

  • Issue happens during peak business hours.

Mon: Probes: 285386068, Failed: 41677467
VIP(1.1.1.1:8443:UP:LEASTCONNS): Hits(27381848, 5/sec) Mbps(7.95) Pers(OFF) Err(0) SO(0) LConn_Best [Idx:SubIdx] 157:0 PrimVserverDownBackupHits(0)
        Pkt(1098/sec, 827 bytes) actSvc(2) DefPol(NONE) override(0) newlyUP(0)
        Conn: Clt(898, 5/sec, OE[626]) Svr(629) SQ(Total: 0 OnVserver: 0 OnServices: 0)
        slimit_SO: (Sothreshhold: 0 [Ex: 0]  Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 626] Available: 0
S(2.2.2.2:8443:UP) Hits(13364466, 2/sec, P[0, 0/sec]) ATr(313:314) Mbps(4.47) BWlmt(0 kbits) RspTime(30149.27 ms) Load(0) LConn_Idx: (C:157; V:157,I:1, B:156, X:2, SI:0)
        Other: Pkt(754/sec, 662 bytes) Wt(1) Wt(Reverse Polarity)(10000)
        Conn: CSvr(367, 4/sec) MCSvr(543) OE(351) E(351) RP(38) SQ(0)
        slimit_maxClient: (MaxClt: 0 [Ex: 0]  Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 351] Available: 0)
        newlyUP_mode: NO, Pending: 0, update: 0x0, incr_time: 0x0, incr_count: 0
S(3.3.3.3:8443:UP) Hits(14017382, 3/sec, P[0, 0/sec]) ATr(315:315) Mbps(6.39) BWlmt(0 kbits) RspTime(30706.15 ms) Load(0) LConn_Idx: (C:158; V:158,I:1, B:157, X:1, SI:1)
        Other: Pkt(1042/sec, 688 bytes) Wt(1) Wt(Reverse Polarity)(10000)
        Conn: CSvr(360, 4/sec) MCSvr(547) OE(351) E(351) RP(36) SQ(0)
        slimit_maxClient: (MaxClt: 0 [Ex: 0]  Consumed: [Ex: 0 Borrowed: 0 TotActiveConn: 351] Available: 0)
        newlyUP_mode: NO, Pending: 0, update: 0x0, incr_time: 0x0, incr_count:

Resolution

As long as latency/threshold issue is actually not present on the server end, the values can be modified
The ADC will wait for 60 seconds between two successive probes. And the response time out, 30 seconds for which the appliance will wait before it marks a probe as FAILED.
set lb monitor test-8443  HTTP -resptimeout 30
set lb monitor test-8443  HTTP -interval 60

Problem Cause

Monitor probe FAILED, causing service DOWN because the Server had huge response time ( 30+ seconds) , where as default is 5 seconds.