What are Out-of-order packets? Out-of-order (OOO) packets are data packets that arrive in a different order from which they were sent.
What Causes Out-of-Order Packets?
Multiple Paths: Out-of-order packets can be caused by data streams following multiple paths through a network (such as traffic traveling through the Internet), or via parallel processing paths within network equipment that are not designed to ensure that packet ordering is preserved.
Queuing: OOO packets can also be caused by poorly configured queuing along a path or even asymmetric routing configurations. In the case of queuing along a path, OOO packets can be caused when the queuing device does not forward packets in a first-in/first-out (FIFO) order.
Link aggregation: Load balancing and Link aggregation can cause OOO packets if a round-robin (per-packet) based algorithm is used.
UDP Traffic: Out-of-order packets can also be caused by UDP traffic. This issue occurs primarily due to stateless connections and the lack of flow control mechanisms that exist within UDP protocol. One of the functions of TCP is to prevent the out-of-order delivery of data, either by reassembling packets into order or forcing retries of out-of-order packets.
Oversubscription: Oversubscribing of devices or links also causes OOO packets. Oversubscribed links and devices drop traffic causing retransmission, slowdowns and out-of-order packets.
Micro-bursting: In computer networking, micro-bursting is a behavior seen on fast packet-switched networks, where rapid bursts of data packets are sent in quick succession, leading to periods of full line-rate transmission that can overflow packet buffers of the network stack. A micro-burst “wave” comes across the network and gets chopped off because devices cannot handle the extra throughput. These packets get dropped causing retransmission, slowdowns and out-of-order packets. A micro burst does not show up on interface counters due to short length (100 milliseconds or less) of the burst.
How does the IPS process Out-of-Order Packets?
If the IPS is receiving significant amounts of OOO packets, the IPS will become less efficient (congested) in terms of inspection. This is because out-of-order packets need to be reassembled before packet inspection and trigger matching can occur. These reassembly and inspection functions are performed in Tier 3 and 4 of the Threat Suppression Engine. The three most process intensive operations are;
- IP Reassembly
- Threat verification
- TCP Packet reordering
Reducing Out of Order, Fragmented, & Small Packets
These days, most routers and switches can assist with traffic optimization if they are configured to do so. In some cases extensive use of link aggregation and load balancing can be part of the problem. If link aggregation is used with the IPS, traffic flow affinity must to be maintained. Use a flow based algorithm such as aggregating using the source IP. This will ensure that all fragments from any particular flow will go through the same segment.
WAN links are a common cause of fragmentation due to the necessity of the additional encapsulation that occurs. When IP Fragmentation occurs the datagrams are broken down into smaller pieces to create packets that will pass through the link. In these cases it is important to determine what kind of optimization, reassembly, and re-encapsulation the endpoints are doing before traffic is then forwarded out over the network. Additionally, it is important to determine (in the case of encrypted traffic or non-standard protocols) whether the IPS should be inspecting it at all.
Applications that rely on the UDP protocol are usually the reason you see an increase of small packets across the network. Be aware that many network management tools make extensive use SNMP which is normally UDP based (depending on the version). File sharing applications also make extensive use of the UDP protocol. Eliminating or funneling these types of traffic through specific network segments which can be inspected differently by the IPS can reduce the overall impact on the system.