VMs are encountering hang or poor performance issue related with disk I/O. (xenvbd event ID 129 ). From VM event log:

++++++++++
xenvbd event ID 129 

\Device\RaidPort0 reset
++++++++++

- From XenServer messages log:

++++++++++
messages.1: 

Mar 8 14:11:35 MXVDIS01X tap-ctl[1481]: tap-err:tap_ctl_read_raw: failure reading data 0/544 
Mar 8 14:11:35 MXVDIS01X tap-ctl[1523]: tap-err:tap_ctl_read_raw: failure reading data 0/544 
Mar 8 14:11:45 MXVDIS01X tap-ctl[1481]: tap-err:tap_ctl_read_raw: failure reading data 0/544 
Mar 8 14:11:45 MXVDIS01X tap-ctl[1523]: tap-err:tap_ctl_read_raw: failure reading data 0/544 
++++++++++

1. Reschedule virus scan time to reduce Disk I/O rate
2. Apply the following hotfix to resolve tap-ctl hang or poor performance issues: CTX213494 - Hotfix XS65ESP1029 - For XenServer 6.5.0 Service Pack 1.

On rare occasions, when a VM hosts an application, the control domain (dom0) disk partition can fill up and cause the VM to become unresponsive.

---

Problem Cause

Event ID 129 means storage device is encountering timeout issue. And there're tap-ctl errors on Dom0, which indicated read request failed at the same time. Hence, this issue occurred due to too much Disk I/O read request has been performed at the same time, need performance tuning regarding disk I/O.

Microsoft - Understanding Storage Timeouts and Event 129 Errors

Windows I/O uses a layered architecture where device drivers are on a “device stack.”  In a basic model, the top of the stack is the file system.  Next is the volume manager, followed by the disk driver.  At the bottom of the device stack are the port and miniport drivers.  When an I/O request reaches the file system, it takes the block number of the file and translates it to an offset in the volume. The volume manager then translates the volume offset to a block number on the disk and passes the request to the disk driver. When the request reaches the disk driver it will create a Command Descriptor Block (CDB) that will be sent to the SCSI device.  The disk driver imbeds the CDB into a structure called the SCSI_REQUEST_BLOCK (SRB).  This SRB is sent to the port driver as part of the I/O request packet (IRP).
 

VMs are encountering hang or poor performance issue related with disk I/O. (xenvbd event ID 129 )