How to Use SR-IOV Feature in XenServer

How to Use SR-IOV Feature in XenServer

book

Article ID: CTX235044

calendar_today

Updated On:

Description

This article describes following items.

  • How to configure SR-IOV Network
  • How to create SR-IOV VLAN network
  • How to assign SR-IOV Network to a VM
  • How to destroy SR-IOV Network
  • Additional steps for NICs that use a legacy driver

Instructions

Single Root I/O Virtualization (SR-IOV) is a virtualization technology that allows a single PCI device to appear as multiple PCI devices on the physical system. The actual physical device is known as a Physical Function (PF) while the others are known as Virtual Functions (VF). With this feature, the hypervisor can assign one or more VFs to a Virtual Machine so that the guest can then use the device as if it were directly assigned.
SR-IOV feature now can be supported in XenServer 7.6.

How to use SR-IOV with XenServer

In this article, we provide steps to use SR-IOV with XenServer. Before proceeding, ensure the following requirements are met:
  • NICs that are in XenServer Hardware Compatibility List website, http://hcl.xenserver.org
  • XenServer Enterprise Edition (or access to XenServer through a XenDesktop/XenApp entitlement)
  • NIC drivers are installed to XenServer and virtual machines.
  • Virtual machines must be HVM guest.
Supported OS list for Citrix version:
  • Ubuntu Xenial Xerus 16.04
  • Ubuntu Trusty Tahr 14.04
  • Debian Jessie 8.0
  • CentOS 7
  • Red Hat Enterprise Linux 7
  • Scientific Linux 7
  • Oracle Linux 7
  • Windows server 2008R2, 2012 and 2016
​SR-IOV feature can be enabled on both XenCenter and xe CLI, please refer to following sections for detailed steps.

Steps within XenCenter

  • Enable SR-IOV on XenCenter

  1. Add new network.
User-added image
  1. Select “SR-IOV Network”
User-added image
  1. Name the network.
User-added image
  1. Select the proper NIC to enable SR-IOV.
User-added image
  1. Click Finish button, read and click on “Create SR-IOV anyway” button on the warning popup.
User-added image
  1. Confirm SR-IOV Network is added.
User-added image
  1. Check if host reboot is required. If yes, reboot the host.
User-added image
  1. Remaining available VFs can be checked from NICs tab.
User-added image

  • Assign an SR-IOV network for a VM.

  1. Select the VM and add a new interface.
  • User-added image
  1. Select the SR-IOV network, and click Add.
User-added image
  1. Start the VM.
User-added image
 
Please note
  • SR-IOV VFs do not support hot-plugging. If SR-IOV network is assigned to a VM while the VM is running, please shutdown then start the VM to make this take effect. A restart operation will not activate the feature for the VM.
  • Performing live migration, suspend, and checkpoint is not supported on VMs using an SR-IOV VF.

  • Disable SR-IOV Network.

  1. ​Select the SR-IOV Network, and click Remove.
User-added image
  1. Select “Yes” on the popup.
User-added image

  • Create SR-IOV VLAN network

  1. Add External Network.
User-added image
  1. Name the SR-IOV VLAN network.
User-added image
  1. Check the check-box of “Create the VLAN on the SR-IOV network” and click finish.
User-added image
Note, VLAN network on top of SR-IOV Network belongs to SR-IOV Network.

Steps for xe CLI

  • Command to enable SR-IOV

  1. Create the network with the network-create command, which returns the UUID of the newly created network:
xe network-create name-label=<network_name>

Example:
xe network-create name-label=SRIOV
ee8ed573-edaa-21ca-5ce3-928a9a1e8eb3
  1. Determine the PIF uuid of the NIC on which SRIOV Network would be configured.
xe pif-list

Example:
xe pif-list
uuid ( RO)                  : 4762a246-1f11-7b93-44ca-06703affc149
                device ( RO): eth1
    currently-attached ( RO): true
                  VLAN ( RO): -1
          network-uuid ( RO): b37076a1-2309-9a04-ae32-446819732713
  1. Configure the network as a SR-IOV network, which returns the newly created SR-IOV Network UUID:
xe network-sriov-create network-uuid=<network_uuid> pif-uuid=<physical_pif_uuid>

Example:
xe network-sriov-create network-uuid=ee8ed573-edaa-21ca-5ce3-928a9a1e8eb3 pif-uuid=4762a246-1f11-7b93-44ca-06703affc149
ae70ddcb-fd2e-fdab-6612-91fe2e8eefcc  -->this is the uuid of SR-IOV Network you created.
 
  1. You may check if host reboot is required to make SR-IOV Network take effect, and the remaining capacity.
xe network-sriov-param-list uuid=<SR-IOV Network_uuid>
Example:

xe network-sriov-param-list uuid=ae70ddcb-fd2e-fdab-6612-91fe2e8eefcc
uuid ( RO)                  : ae70ddcb-fd2e-fdab-6612-91fe2e8eefcc
          physical-PIF ( RO): 4762a246-1f11-7b93-44ca-06703affc149
           logical-PIF ( RO): efb46271-fa28-11eb-92d9-7098f00704bf
       requires-reboot ( RO): false
    remaining-capacity ( RO): 63
 
 

  • Command Line to assign the SR-IOV network to a VM:

  1. Determine the vif mac address of the VM:
xe vm-vif-list vm=”vm name label”
Example:
xe vm-vif-list vm="Windows Server 2016"
uuid ( RO)                  : b22b444d-4d5b-a697-754f-5a7d42bbd990
         vm-name-label ( RO): Windows Server 2016
                device ( RO): 0
                   MAC ( RO): b2:61:fc:ae:00:1b
          network-uuid ( RO): 5f0bb332-da64-f95d-633e-570edee7ee33
network-name-label ( RO): Pool-wide network associated with eth0
  1. Assign SR-IOV Network to the VM, vif uuid of SR-IOV Network assigned to the VM is returned.
xe vif-create device=<device index> mac=<vf_mac_address> network-uuid=<sriov_network> vm-uuid=<vm_uuid>

Example:
xe vif-create device=1 mac=b2:61:fc:ae:00:1b network-uuid=ee8ed573-edaa-21ca-5ce3-928a9a1e8eb3 vm-uuid=91b7ec7c-07d7-1454-a229-48c07b2ff2a7
ae5ed118-00cd-9cba-3fbb-4732b2dbe99a
Please note, if a VM that is exported from a previous XenServer version and then imported into XenServer 7.6, and the guest OS of the VM supports SR-IOV Network, you assign a SR-IOV Network or SR-IOV VLAN Network to that VM via xe CLI only. For more details, please refer to How To Assign SR-IOV Network to a VM exported from a previous XenServer Version

  • Command to disable SR-IOV

xe network-sriov-destroy uuid=<network_sriov_uuid>
Example:
xe network-sriov-destroy uuid=ae70ddcb-fd2e-fdab-6612-91fe2e8eefcc

Additional steps for NICs that use a legacy driver

For some NICs that use legacy drivers (for example, Intel I350 family), the maximum number of VFs that the NIC can support is defined within the driver module configuration file and may need to be adjusted manually prior to enable SR-IOV.
To do this, open modprobe configuration in /etc/modprobe.d/<driver_name>.conf using an editor and change the line starting:
# VFs-maxvfs-by-user:
For example, to set the maximum VFs to 4 for the igb driver edit /etc/modprobe.d/igb.conf to read:
# VFs-param: max_vfs
# VFs-maxvfs-by-default: 7
# VFs-maxvfs-by-user: 4
options igb max_vfs=0

Notice
  • The value of VFs-maxvfs-by-user must be less than or equal to the value in the line VFs-maxvfs-by-default.
  •  Do not change any other line in the file.
  • The host must be rebooted to enable or disable SR-IOV on these devices.
  • If rebooting is again required even after host reboot, it indicates that the NIC is not able to enable SR-IOV.

Additional Information

How To Assign SR-IOV Network to a VM exported from a previous XenServer Version
How to troubleshoot common SR-IOV related issues 
SR-IOV support FAQ
Powered by Wolken Software