This article describes how to configure XenServer 6.0 with XenDesktop for HDX 3D Pro Graphics so that a host workstation with multiple graphics cards can be shared by an equal number of concurrent high-end professional graphics users (one user per GPU), thereby reducing the cost per user.

Hardware Requirements

XenServer Host

The Server CPU should be IOMMU enabled for AMD processors or VT-d enabled for Intel processors.

GPUs

Install the latest graphics driver from NVIDIA. AMD/ATI graphics cards are not supported by the Multi-GPU Passthrough feature.

Virtual CPUs

HDX 3D Pro Graphics utilizes one virtual CPU completely for extracting desktop surfaces, typically 12 to 30 times per second. If Aero is enabled on Windows this requires a second virtual CPU. In addition, at least two vCPU is required for the graphics applications. Graphics applications are often CPU intensive, even with hardware acceleration of graphics rendering provided by the GPU. Hence it is important to configure the virtual machine running HDX 3D Pro Graphics with at least four virtual CPUs. Following are the instructions for increasing the number of virtual CPUs on XenServer:

Refer to the following Knowledge Center articles for increasing vCPUs per virtual machine: 

• CTX126524 - How to Set Cores-Per-Socket Parameter for a Virtual Machine

• CTX125574 - XenServer 5.6 Multi-GPU Pass-through for XenDesktop HDX 3D Pro Graphics

Refer to the following Knowledge Center article for XenServer Host and GPUs support matrix for “HDX 3D Pro”:

CTX131385 - Citrix Tested Hardware for the HDX 3D Pro Feature in XenDesktop 5.5 or 5.6

For formal support of XenServer multi-GPU pass through solution, refer to the Hardware Compatibility List under GPU Pass-thru Support Matrix for XenServer: http://hcl.xensource.com/GPUPass-throughDeviceList.aspx

Software Requirements

XenServer Host

• Citrix XenServer 6.0.2 (final RTM release) with Platinum Edition License.

• Software on guest virtual machine, hosting XenDesktop HDX 3D Pro Graphics desktop

• XenDesktop 5.5 or later Virtual Desktop Agent , “recommended 5.6 Feature Pack 1”

Software on user device (endpoint)

HDX 3D Pro Graphics online plug-in for Windows 12.0.33 (includes special GPU codec to minimize bandwidth requirements), or standard 12.0.3 online plug-in if only the CPU codec is required
Or
11.100 Citrix Receiver for Linux (includes CPU codec only)

Refer to the XenServer 6.0 documentation for more details on XenServer configuration. Similarly for XenDesktop 5.5 and HDX 3D Pro Graphics deployment, refer to the relevant documentation (such as HDX 3D Pro Graphics CTX124443 - XenDesktop 4 HDX 3D for Professional Graphics 1.1 - Administrator's Guide).

Background

HDX 3D Pro Graphics, a feature of XenDesktop Enterprise and Platinum editions, enables the delivery of high-end 3D graphics applications, including OpenGL and DirectX based applications, such as CAD/CAM and medical imaging. Typically these applications are very computation-intensive and require hardware acceleration provided by a graphics processing unit (GPU). To support remoting of high-end 3D graphics based applications, Pro Graphics leverages a graphics card on the host workstation (usually located in the data center) for 3D rendering and optionally for deep compression of the data stream).
Because multi-user hypervisors prior to XenServer 5.6 did not support GPU access, HDX 3D Pro Graphics was previously recommended with physical machines only, whether physical desktop machines or blade or rack workstations.
With the Multi-GPU Pass-through feature of XenServer 5.6/6.0 it is now possible to map a physical GPU to a virtual machine; in fact, you can map multiple GPUs to an equal number of virtual machines, one to one. XenServer 6.0 with GPU mapping in the direct assign mode is the ideal solution for reducing hardware costs for high-end 3D graphics. Consider a server that has multiple GPUs or possibly (not yet tested) one that is connected to a multi-GPU appliance (such as NVIDIA Quadro Plex S4). In this case, each GPU can be mapped to one virtual machine, which can get access to the GPU and is powered by HDX 3D Pro Graphics. On a powerful multi-core server (such as a dual quad-core Nehalem processor server that would normally support 80 or more regular XenDesktop users running typical office applications), additional virtual machines could potentially run regular XenDesktop for normal use cases not requiring graphics hardware acceleration, subject to performance, thereby further reducing the cost per user.

 

---

Instructions

Mapping the GPU to a Virtual Machine

Mapping of XenServer 6.0 Virtual Machine with GPU can either be done from CLI or from GUI. It is preferred to use CLI, since it gives exact GPU cards data as compared to XenCenter GUI.

Notes:

• Citrix XenServer Platinum Licensing is required to enable GPU-Pass through in XenServer.

• Snapshot does not work with GPU Cards. It is highly recommended to create/delete/revert snapshot of VM when GPU card is not assigned to VM or else you may have garbage entries in XenServer and VM will not start properly.

• You should not take default console using XenCenter after assigning GPU Card and Installing NVIDIA Driver. Either take VNC console or take remote console.

Mapping GPU to Virtual Machine using CLI1

1. Attach Graphics cards to XenServer and get their list by running lspci | grep VGA, for example for ATI and nVIDIA attached cards run command, as displayed in the screen shot:

Note: If cards are not listed properly then they are not attached properly to the server, reattach them and again list using lspci. 

2. If Xenserver is not licensed with Platinum Edition License then get it Licensed.

3. Now check which all groups are created on XenServer. Note down GPU Groups UUID, for example, for ATI and nVIDIA attached cards run command, as displayed in the following screen shot: 

4. Now check which all cards are there in XenServer. For Example for ATI and nVIDIA attached cards run command, as displayed in the following screen shot: 

5. Assign GPU card to Virtual Machine. For this, you must note down UUID of the Virtual Machine, as displayed in the following screen shot: 

6. Now check if any GPU is already assigned to it. If yes, then shut down the Virtual Machine and destroy that assignment.

7. You can notice in XenCenter that the GPU is unassigned: 

8. Assign GPU to the Virtual Machine. You will get vgpu uuid as output: 

You can see same assignment is XenCenter, as displayed in the following screen shot: 

9. Now you can start the Virtual Machine. Install latest nVidia Driver and take VNC console to install HDX3DPro Setup.

Mapping GPU to Virtual Machine using XenCenter GUI (prone to errors)

Note: You must be very careful while performing operations with XenCenter GUI, since removed cards entries can be there in XenCenter.
In Xencenter, you must access the Virtual Machine Properties and in GPU section select the GPU type. Once this is done, you will have GPU assigned to the Virtual Machine. 
 

This article describes how to configure XenServer 6.0 with XenDesktop for HDX 3D Pro Graphics so that a host workstation with multiple graphics cards can be shared by an equal number of concurrent high-end professional graphics users (one user per GPU), thereby reducing the cost per user.

CTX126524 - How to Set Cores-Per-Socket Parameter for a Virtual Machine