Citrix SD-WAN, formerly NetScaler SD-WAN

Table of Contents

• Introduction
• QOS Components
• Transmit Modes
• Queue Depth
• IP Rules
• Application QOS rules with release 9.3
• Basic Rules
• Basic guidance on Rules.
• Other Settings that affect Applications.
• Appendix A: Default Rules Breakdown

Introduction 

The NetScaler SD-WAN solution has one of the most sophisticated application QoS engines available in the market today. It has the broadest set of capabilities to assess not only the application traffic and prioritize it against other applications, but also to understand its needs with respect to the WAN network quality, and to pick a network path based on quality characteristics in real time. Configuring application QoS can be a little confusing, this guide is intended to explain the configuration of application QoS, and identify the important features that enables the NetScaler SD-WAN solution to perform flawlessly in most deployments.

Back to top

QOS Components

The following are three principle components define application QoS in the NetScaler SD-WAN product:

• IP and application rules*
• Virtual PATH QoS classes and settings
• WAN link and virtual path sensitivity settings

* NetScaler SD-WAN 9.3 release introduces Application QOS rules as an addition to the traditional IP Rules. 

We will discuss the common settings of each component, but first let us define some terminology and some application types.

Back to top

Transmit Modes and QoS Classes

In the SD-WAN environment, we think of applications as falling into one of the following three classes:

• Real-time –VoIP or VoIP like applications, such as Skype or ICA audio. In general, we refer to voice only applications that use small UDP packets, that are business critical. 
• Interactive – This is the broadest category, and refers to any application that has a high degree of user interaction. Some of these applications, for example video conferencing, is sensitive to latency, and requires high bandwidth. Other applications like HTTPS, may need less bandwidth, but are critical to the business. Interactive applications are typically transactional is nature. 
• Bulk – This is any application that does not need rich user experience but is more about moving data (i.e. FTP or backup/replication)

These terms are also used to describe the importance of the application as it refers to the QoS classes. Each Virtual PATH can have up to 17 QoS priority classes, although typically we use the 6 default classes. Each class is designated as Real-time, Interactive or Bulk. This, at a high level defines the overall priority of the class in terms of schedule time. By default: 

Real-time (RT) classes are given the highest priority and gets up to 50% of the overall scheduler time. Each class can be weighted with respect to the other RT classes, for example, we could have two RT classes one that weighted to 70% and the other to 30%. 

Interactive (INT) classes take the next priority and can consume the rest of the scheduler time as the traffic demands. Individual INT classes can be weighted and by default we have 4 weights (high, medium, low and very low) defined.

Bulk (BLK) classes takes the lowest priority and can be considered scavenge classes. They can be weighted but they can be completely starved of bandwidth if the INT/RT traffic is consuming all of the scheduler time.

It is important to note that this QoS is hierarchical, so at the MCN each Virtual PATH to each site combines its QoS when it gets down onto the WAN link. Lower priority traffic from one site should not starve off high priority traffic to another. 

The default classes are defined below:

Classes 0-3 are set for Citrix HDX traffic and are not user configurable.
Classes 4-9 not used by default but are user configurable.
Classes 10-16 are the default classes used by the default rules. 

Hint: Try to stick to the default classes. Ignore the initial share %.

You will notice each class depending on the type has a sustained share %. This is only relative to the same type of class. For example, we have 2 real-time classes, class 0 and class 10, Each has 30% sustained share. That means both classes get equal access to the overall schedule (50% for Real-time in total). It is the same for Interactive and Bulk.

Hint: Use Bulk classes for traffic we do not care about. It can easily be starved of all BW.

Back to top

Queue Depth 

The Queue depth (send buffer) is defined in the IP rule per application. It defines the amount of data to be queued on the sending side before dropping packets. This means you can have multiple applications all feeding into the same class with different buffer depth. All flows matching the same rule will share the same queue. 

The lower the application priority the larger the queue depth needs to be. For Real-time applications, this queue depth has not only a buffer size in bytes but also in time. The theory here is it is bad to buffer Real-time traffic for a long period of time because it can introduce jitter. That being said if the real-time class does not get serviced fast enough then you may have bigger problems.  Points to note:

• Assign only VoIP or VoIP like traffic to the real-time class.
• Set the queue depth to 80-125 ms and not more than 200 Kbytes
• If you have a lot of real-time traffic and you see packet drops in the Monitor > Classes view, increase the buffer.
• Set Duplicate Disable Delay to about 70% of the queue depth timer if you are doing packet duplication.

For all other types of classes queue depth is less critical but it is recommended to set these to be fairly large. You can use different depths to throw less important applications away by setting them slightly shorter. Again, it is good to look at the classes and see if any of the classes are dropping. Note, drops in the class view is not the same as packets lost in the WAN. We do not recover drops from the queue. We can retransmit packets lost in the WAN.

Back to top

IP Rules

The Default IP rules can be found on the Virtual PATH: 

The default IP rules cannot be edited. A breakdown on the default settings is shown in appendix A. 

To override the default rules, it is recommended to create a Virtual Path Default Set under the Global tab and then apply it to the relevant Virtual Paths:

Note:

• The rule set is applied in both directions, unless reverse is disabled.
• In general, most rules are created directionless so they will match in either direction (i.e. <any> <any> FTP etc.)
• You can change the classes but it is recommended to use the default class structure. You can change the class names in the global rules sets.
• You can create multiple rules sets for different types of Site (small branch, large DC etc.)
• When you clone a site, the rule set is also cloned.
• Dynamic virtual paths have their own rule set.

Back to top

Application QoS rules with release 9.3

With Release 9.3 we have introduced another layer to the application rules. Whilst you can create rules using just the IP settings, you can now take advantage of the application DPI Engine to identify specific applications or groups of applications as defined by custom application objects. The IP rules are still processed first and the app QoS rules are processed second but the same settings and elements still apply. The application QoS rules can be used to override the default IP rules also. 

Back to top

Basic Rules

When creating a new rule think about what the application is and how you want it to behave. Also, remember that other traffic may be using the same class so prioritize the application appropriately. There are benefits and drawbacks to the transmit modes and settings. Points to note: 

• Be as specific as you can with the match conditions to prevent other applications getting captured by the rule
• Never duplicate applications that have more than 100Kbps of throughput. Dup uses 2X the BW and should only be used for VoIP like applications.
• Retransmitting lost packets takes roughly 2.5 times the round trip. Consider if the added jitter is worth it.
• Reordering also takes time. Tradeoff is less lost packets but the longer the timer the worse the jitter.
• Video streaming is not a real-time application. Map it to an Interactive class with a fairly large queue depth.
• Video conferencing can be tricky to optimize for. Start with an interactive class and big buffer

Let’s look at some of the Rule settings: 

Transmit mode: 

There are 3 transmit modes on the WAN General tab: 

Load Balance Paths. In this mode, the application will select the best available path based on its overall class and the other application traffic currently transmitting. If the bandwidth demand for an individual flow exceeds a single path WAN link’s bandwidth, it will intelligently load balance the packets to multiple paths. It does this in such a way to minimize the re-ordering time on the receive side. Note, it will only load balance as needed, it is not simple round robin.

Persistent Path.  In this mode when the app flow starts the best path is selected. The application will stay on the path unless the latency of said path changes by 50ms (Default Persistent Impedance). It will then select the next best path. This means the flow generally will pin to a given path unless the quality changes dramatically. If the path goes BAD (die to loss) it will also move. If the application’s bandwidth demand exceeds the path it is on the app is allowed to flow across multiple paths (similar to Load Balanced transmit mode).

Duplicate Paths. This mode is used for VoIP applications. The flow is duplicated and sent to the two best and most diverse paths available based on Service Provider IDs. Use this for VoIP like applications only, as it consumes 2X the application bandwidth.

Retransmit Lost Packets can be used with all of the above modes except Persistent Path. If the receiving SD-WAN appliance detects a missing packet it can request that packet to be resent by the sending SD-WAN appliance. Note, this can take up to 2.5 times the round trip. The receiving appliance will give up if the hold timer (see WAN to LAN tab) is exceeded.

On the LAN to WAN Tab, you can select the send class and the Drop Depth (our queue buffer for this app). There are a lot of other settings here that can be mostly ignored.

On the WAN to LAN tab set the hold time (think at least 2.5 times RTT) and whether you want to reorder the packets.

Back to top

Basic guidance on Rules

1. Unless you really don’t care about the application only put them into the interactive classes. Bulk can be starved off entirely
2. Only put VoIP into Real-time.
3. Only use packet duplication for narrow band applications (i.e. VoIP)
4. Load-balance is the safest option.
5. Retransmit packets when the loss will have great affect (i.e. TCP apps)
6. Remember other traffic maybe classified into the same class – check the other rules.

Back to top

Other Settings that affect Applications

It is possible to change the behavior of the virtual paths using the settings under Auto-path groups, WAN links or on the virtual paths themselves. These settings include when a path will go BAD, when and how congestion is detected and, which WAN links are eligible for certain types of traffic. These are all advanced settings and it is recommended not to change them unless you are trying to change specific behavior. Consult with your Citrix support team to configure these settings.

Back to top

Appendix A: Default Rules Breakdown

Rule Name	MATCH			SET
	protocol	DSCP	port	class	transmit type	retransmit	queue depth (bytes)	reorder time	misc
VOIP	SIP	EF	*	10 RT	dup	no	15K/100ms	80	reassign at 600Byte to class13
ICA	ICA	*	*	11 int	LB	yes	30000/350ms	250	RED on
ICA CGP	ICACGP	*	*	11 int	LB	yes	30000/350ms	250	RED on
ICA UDP	ICAUDP	*	*	11 int	LB	yes	30000/350ms	250	RED off
ICA CGP UDP	ICACGPUDP	*	*	11 int	LB	yes	30000/350ms	250	RED off
ICMP	ICMP	*	*	12	Persist	no	30000/350ms	250
SSH	SSH	*	*	12	LB	yes	65000/350	900	RED on
telnet	telnet	*	*	12	LB	yes	65000/350	900	RED on
RDP	RDP	*	*	12	LB	yes	65000/350	900	RED on
RPC	RPC	*	*	12	LB	yes	65000/350	900	RED off
LDAP	LDAP	*	*	12	LB	yes	65000/350	900	RED off
HTTP	HTTP	*	*	14	LB	yes	100000/350	900	RED on
ALTHTTP	ALTHTTP	*	*	14	LB	yes	100000/350	900	RED on
HTTPS	HTTPS	*	*	14	LB	yes	100000/350	900	RED on
CIFS	CIFS	*	*	13	LB	yes	65000/350	900	RED on
POP3	POP3	*	*	13	LB	yes	30000/350ms	900	RED on
SMTP	SMTP	*	*	13	LB	yes	30000/350ms	900	RED on
IMAP	IMAP	*	*	13	LB	yes	30000/350ms	900	RED on
FTP	FTP	*	*	15	LB	yes	128000/50	900	RED on
IPERF	IPERF	*	*	15	LB	yes	128000/50	900	RED off
GRE	GRE	*	*	13	LB	yes	200000/350	250	RED off
DNS	DNS	*	*	13	LB	yes	128000/350	250	RED off
SNMP	SNMP	*	*	13	LB	no	128000/350	0	RED off
SNMP TRAP	SNMP TRAP	*	*	13	LB	no	128000/350	0	RED off
ALL EF	ALL EF	EF	*	10	dup	no	15K/100ms	80	reassign at 2000B to class 13
ALL AF11	ALL AF 11	AF11	*	12	persist	no	30000/350ms	250
UDP	UDP	*	*	*	persist	no	15000/100	250
TCP	TCP	*	*	13	LB	yes	300000/350	900
catch all	*	*	*	13	persist	no	200000/350	0

Back to top

The NetScaler SD-WAN solution has one of the most sophisticated application QoS engines available in the market today. It has the broadest set of capabilities to assess not only the application traffic and prioritize it against other applications, but also to understand its needs with respect to the WAN network quality, and to pick a network path based on quality characteristics in real time.