Citrix Virtual Apps and Desktops - VDA machines become unresponsive due to High CPU at the time of Scheduled reboot
Article ID: CTX694737
Updated On:
Description
Memory dump of the machine which was in the unresponsive state was collected for analysis. The analysis showed that the problem was likely due to the lsi_sas.sys module. This module is a part of VMware tools.
Cause
Possible issues due to corrupted vmware tools installation.
Resolution
Repairing VMware Tools resolved the problem.
If the problem persists even after repairing Vmware tools, customers will have to contact Vmware / Broadcom to investigate the issue further.
Issue/Introduction
Administrators might notice that during scheduled reboot, the VDA machines hosted on VMware start to show very high CPU utilization and become unresponsive. The machines then have to be restarted manually to get them back to working state. This problem has been reported with Windows Server 2019 and Windows Server 2022 machines with CVAD 2402 CU2 VDA installed.
Additional Information
DRIVER_POWER_STATE_FAILURE (9f)
A driver has failed to complete a power IRP within a specific time.
Arguments:
Arg1: 0000000000000003, A device object has been blocking an IRP for too long a time
Arg2: ffff838c2d3a3050, Physical Device Object of the stack
Arg3: ffff820a3063f290, nt!TRIAGE_9F_POWER on Win7 and higher, otherwise the Functional Device Object of the stack
Arg4: ffff838c6f287010, The blocked IRP
<Truncated for readability> <Truncated for readability> <Truncated for readability>
IMAGE_NAME: lsi_sas.sys
MODULE_NAME: lsi_sas
FAULTING_MODULE: fffff80244d30000 lsi_sas
BLACKBOXBSD: 1 (!blackboxbsd)
BLACKBOXPNP: 1 (!blackboxpnp)
PROCESS_NAME: System
STACK_TEXT:
ffff820a`3063f258 fffff802`3fd7f3a6 : 00000000`0000009f 00000000`00000003 ffff838c`2d3a3050 ffff820a`3063f290 : nt!KeBugCheckEx
ffff820a`3063f260 fffff802`3fd7f2b2 : ffff838c`e255e120 ffff820a`3063f2fc 00000000`00000043 ffffd880`f1ec0180 : nt!PopIrpWatchdogBugcheck+0xea
ffff820a`3063f2d0 fffff802`3fb41d19 : ffffd880`f1ed0100 00000000`000000ff ffff820a`3063f4d8 00000000`00000008 : nt!PopIrpWatchdog+0x22
ffff820a`3063f320 fffff802`3fb42c67 : 00000000`00000008 00000000`00989680 ffffd880`f1ed0100 00000000`00000019 : nt!KiProcessExpiredTimerList+0x159
ffff820a`3063f410 fffff802`3fc6ac9a : 00000000`00000000 ffffd880`f1ec0180 00000000`00000000 ffffd880`f1ed0100 : nt!KiRetireDpcList+0x4a7
ffff820a`3063f620 00000000`00000000 : ffff820a`30640000 ffff820a`30639000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x5a
STACK_COMMAND: .process /r /p 0xfffff80240007a40; .thread 0xffffd880f1ed0100 ; kb
FAILURE_BUCKET_ID: 0x9F_3_POWER_DOWN_disk_IMAGE_lsi_sas.sys
OS_VERSION: 10.0.17763.1
BUILDLAB_STR: rs5_release
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
FAILURE_ID_HASH: {8cb40927-f719-bf2e-8ad4-99432bebbead}
2: kd> !irp ffff838c6f287010
Irp is active with 5 stacks 3 is current (= 0xffff838c6f287170)
No Mdl: No System Buffer: Thread 00000000: Irp stack trace.
cmd flg cl Device File Completion-Context
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
>[IRP_MJ_POWER(16), IRP_MN_SET_POWER(2)]
0 e1 ffff838c2d867060 00000000 fffff80afcfe48c0-00000000 Success Error Cancel pending
\Driver\Disk partmgr!PmPowerCompletion
Args: 00012200 00000001 00000004 00000002
[IRP_MJ_POWER(16), IRP_MN_SET_POWER(2)]
0 e1 ffff838c2d46a040 00000000 fffff8023fc1a820-ffff838ce255e120 Success Error Cancel pending
\Driver\partmgr nt!PopRequestCompletion
Args: 00012200 00000001 00000004 00000002
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-ffff838ce255e120
Args: 00000000 00000000 00000000 00000000
2: kd> lmvm lsi_sas
Browse full module list
start end module name
fffff802`44d30000 fffff802`44d4f000 lsi_sas # (pdb symbols) C:\ProgramData\Dbg\sym\lsi_sas.pdb\9B13F4F8CB9842E4A052A00BE8DFBF4C1\lsi_sas.pdb
Loaded symbol image file: lsi_sas.sys
Mapped memory image file: C:\ProgramData\Dbg\sym\lsi_sas.sys\55130E501f000\lsi_sas.sys
Image path: \SystemRoot\System32\drivers\lsi_sas.sys
Image name: lsi_sas.sys
Browse all global symbols functions data Symbol Reload
Timestamp: Wed Mar 25 15:36:48 2015 (55130E50)
CheckSum: 0001F6A6
ImageSize: 0001F000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
