Summary
NetScalers have a feature called Path MTU Discovery, which is actually a common feature on most networking devices. Path MTU Discovery allows a networking device such as the NetScaler, or routers and switches made by companies such as Cisco and Juniper, to determine the largest packet size allowed along an arbitrary network path. This enables network traffic to flow correctly from one endpoint to another, without any of the traffic being dropped.
Background
Network protocols at the lower range of the Open System Interconnection (OSI) model have a maximum allowable size for the datagram unit of that layer, above which size the datagram is required to be reduced in size in order to pass through a given network device. This maximum size, expressed in numbers of bytes, is referred to as Maximum Transmission Unit (MTU), and is most often encountered in datagrams for Layer 2 (for example, Ethernet Frames) and Layer 3 (for example, IP packets). Ethernet’s MTU is 1500 bytes, but because that is the absolute maximum, most networking devices reduce the practical MTU to 1460 bytes.
Some network devices may have an MTU that is smaller than the usual 1460 bytes. This setting is generally applied to certain interfaces, rather than as a global setting on the device. The smaller MTU is usually due to a protocol in use on the device that needs to add additional information to the datagram such as address headers. A very common example of this is a VPN tunnel configured on a router interface. The VPN tunnel encapsulates traffic passing through it within an encrypted wrapper, which tends to add about 200 bytes to the size of the datagram. In this case, the MTU on the VPN tunnel is reduced to 1260 bytes to compensate for the VPN header data added to the datagram as it traverses the tunnel.
NetScaler Path MTU Discovery – version 6.0 and later
The IP protocol has a mechanism for signaling that datagrams are too large to pass through an interface on a network path – when a datagram is received on a router or Layer 3 switch interface that is larger than the interface’s MTU, the device sends an Internet Control Message Protocol (ICMP) message to the previous hop device indicating that the datagram needs to be fragmented in order to pass through that interface, as well as the MTU of the interface. The previous hop device breaks the datagram into pieces that are small enough to pass the next hop interface’s MTU. While this mechanism is usually sufficient to allow traffic to continue normally, it does have some drawbacks. The increase in the number of datagrams from fragmenting means Layer 3 routing has that much more work to do in routing decisions. There is always the possibility that another device further along the network path has an interface with an even smaller MTU and requires further fragmentation, and when the destination device for these datagrams eventually receives them, all datagram fragments must be received so it can be reassembled correctly. If all fragments are not received, the entire original datagram is dropped and must be retransmitted by the sending station (and will probably be fragmented in transit again).
These issues prompted the creation of a mechanism to avoid having to fragment datagrams at all – Path MTU Discovery. Path MTU Discovery works whereby a router or other device sets the “Don’t Fragment” flag on outgoing datagrams. If a datagram encounters an interface with a smaller MTU than the size of the datagram, that device sends an ICMP message back that indicates the datagram is “Too Large” and/or is marked “Don’t Fragment” along with the MTU of that interface. The sending device adjusts the size of the outgoing datagrams to accommodate the MTU of all the hops on the network path, eventually learning the absolute maximum size the datagram can be to successfully reach its destination without being fragmented. The “Don’t Fragment” bit can also be set by an application that is transmitting data across the network and doesn’t wish the datagrams to be fragmented before being received by the end station. Path MTU Discovery enables these datagrams to be received unfragmented as well.
Beginning in version 6.0, a NetScaler can participate in Path MTU Discovery with routers and other Layer 3 devices in a network. The NetScaler stores MTU values for network segments and reduces the size of datagrams sent along a path to fit within the MTU of the path. This is a global setting, configured using the CLI with the following command:
> enable ns mode PMTUD
A final caveat about Path MTU Discovery that is not specific to NetScalers, but to any device that can participate in it – the signals sent from upstream devices indicating “Needs Fragmentation” and other types of information are ICMP messages. Unfortunately, many networks restrict or block ICMP traffic, especially when it originates from outside the network edge. This practice is mostly to prevent denial of service attacks that exploit ping broadcasts, and so on. Obviously, Path MTU Discovery does not function correctly or at all in such a situation, as it relies on the receipt of the special ICMP messages to adjust datagram sizes. The easiest way to tell if Path MTU Discovery is being hampered by ICMP filtering is if transferring very small amounts of data succeeds along a given network path, but transfers of large amounts of data time out and fail. This indicates that when the sending device is filling datagrams to their limit with data (exceeding an upstream device’s MTU), the “Needs Fragmention” ICMP messages sent by the upstream device are not being received by the sending station. The culprit causing this can be anything from a router ACL to a firewall, but typically this is only a problem on a network’s edge and not within the core.
More Information
