Understanding and Using Selective Packet Discard

Summary

This document provides an overview of the operation, configuration, and monitoring of Selective Packet Discard (SPD). It builds on this fundamental information with actionable technical details that can help network engineers implement SPD in an environment composed of differing hardware platforms and IOS releases.

In today's complex internetworks, various types of network traffic compete for the finite resources of a router. These might include an interior routing protocol such as Open Shortest Path First (OSPF) or Enhanced Interior Gateway Routing Protocol (EIGRP), network management traffic, and Border Gateway Protocol (BGP). SPD was designed to help ensure that network stability was not undermined during periods of high CPU-bound traffic. Although the safeguards added by SPD are typically applied to the BGP reconvergence use case, they are relevant to security when viewed in the context of availability.

SPD achieves its goals using two techniques: the provisioning of additional queuing capacity for control plane traffic and the implementation of a simple congestion-control mechanism for the interface input queues.

Overview of Selective Packet Discard

All packets to be processed by the router CPU through process switching must be queued in the interface input queue. Without SPD there is no intelligent handling of packets in this queue. Packets are queued until the queue reaches its maximum length and then additional packets are indiscriminately tail dropped. Although administrators can configure the length of this queue per physical interface using the hold-queue value in interface configuration command, a global pool of buffers is used for temporary packet storage.1

SPD was created to provide preferential treatment during congestion of this interface-to-process-switching queue. At its most basic level, SPD provides extended buffering for control plane traffic. Known as the SPD headroom, this additional queue depth is typically reserved for traffic with IP Precedence equal to 6 (such as BGP), the Connectionless Network Service (CLNS) based routing protocol Intermediate System-to-Intermediate System (IS-IS), and Layer 2 keepalives.

Over time, it became clear that this simple method of protection was insufficient. The functionality of SPD was then expanded to include extended headroom, which is reserved for IP-based Interior Gateway Protocols (IGPs), Hot Standby Router Protocol (HSRP), IS-IS, and Layer 2 keepalives; all IP Precedence 6 traffic was specifically relegated to the headroom. This implementation gives IGPs, HSRP, and Layer 2 keepalives queuing priority over BGP, thereby helping to protect the stability of the IGPs and Layer 2 during BGP reconvergence.

Logically, the interface hold queue, headroom, and extended headroom can be viewed as a single, contiguous queue as illustrated in the following:

Interface Input Queue

Headroom

Extended Headroom

 

From left to right, each section of the total interface input queue has more stringent entry requirements. For example, although the hold queue may include SNMP requests or data-link switching (DLSw) packets with IP Precedence 0 bound for the CPU, these packets may not be queued to the headroom or extended headroom. However, IP Precedence 6 packets, while allowed into the headroom, are inherently allowed into the hold queue. The exact classification used by SPD may vary among Cisco IOS Software releases to meet changing needs; however, the classification cannot be changed by the end user. The following table shows the classification used for queuing enforcement.2

Queue Malformed Packets* Packets Allowed Examples
Interface hold queue (Standard mode) Allowed All packets destined to the CPU SNMP, SSH, BGP, EIGRP
Interface hold queue (Aggressive mode) Dropped All correctly formed packets destined to the CPU SNMP, SSH, BGP, EIGRP
SPD Headroom Dropped IP Precedence 6, ISIS, Specific Layer 2 packets BGP, OSPF, PPP/HDLC Keepalives
SPD Extended Headroom Dropped IGPs, Layer 2 Keepalives** IS-IS, HSRP, PPP/HDLC Keepalives
* Malformed is defined as any packet with an invalid checksum, incorrect version, incorrect header length, or incorrect packet length. Hardware forwarding platforms may have specific drop or rate-limit behaviors for malformed packets that are present regardless of the existence or configuration of SPD.
** Depending on the IOS version used, ARP and TDP/LDP may also be queued to extended headroom. Refer to "Functionality and Default Configuration by IOS Version."

 

There is also a lesser-known function of SPD: as well as providing additional queue depth for control plane traffic, SPD attempts to manage the interface input queue by proactively dropping packets that cannot be queued into headroom or extended headroom. This is implemented using two thresholds—min-threshold and max-threshold—that are configured or derived on a global basis but applied individually to the input queue of each interface.

SPD uses these thresholds to control a random drop congestion-control mechanism. This mechanism is known as the SPD state check and uses three states: normal, random drop, and full drop. The following table illustrates the queuing functionality provided by the SPD state check as well as the headroom and extended headroom. This table is limited to the interaction of these features within the context of a single interface.3

Behavior Relative to Queue Length of a Single Interface Selective Packet Discard State Queuing Allowed by Packet Classification
hold-queue Headroom Extended
Headroom
queue length <= min-threshold Normal    blue dot    blue dot blue dot
min-threshold < queue length <= max-threshold Random Drop    blue dot  * blue dot blue dot
max-threshold < queue length <= (hold-queue + headroom) Full Drop   blue dot blue dot
(hold-queue + headroom) <  queue length <= (hold-queue + headroom + extended headroom) Full Drop     blue dot
(hold-queue + headroom + extended headroom) < queue length Full Drop      
* Packets not eligible to be queued into headroom are subject to a random drop congestion-control mechanism.

 

It is important to note that, although the min-threshold and max-threshold are applied to a single input queue, the resulting SPD state is globally relevant. The following two tables further illustrate this point:

Interface Configured hold-queue min/max-threshold Packets in hold-queue Resulting SPD State Comments
Vlan 100 75 73/74 70 Normal All packets are queued.
Vlan 200 75 73/74 70

 

The preceding table shows that min-threshold and max-threshold are applied on a per-interface basis. The following table illustrates that the resultant SPD state is used on all interfaces regardless of their individual hold-queue depth:

Interface Configured hold-queue min/max-threshold Packets in hold-queue Resulting SPD State Comments
Vlan 100 75 73/74 75 Full Drop No more packets are allowed to be queued to hold queue on either interface; headroom and extended headroom are still available for eligible packets.
Vlan 200 75 73/74 0

 

It is important to note that the global value of min-threshold and max-threshold are automatically derived based on the smallest input queue size on the device. However, this derivation may change and it is possible that some future Cisco IOS trains may require the user to manually set min-threshold and max-threshold. In all SPD-enabled images, it is possible to manually set min-threshold and max-threshold using the global configuration command ip spd queue {min-threshold | max-threshold} length.4

That said, assuming that SPD is enabled and min-threshold and max-threshold are being derived automatically, altering the input queue length by using the hold-queue interface configuration command has no value for headroom-ineligible packets unless the input queue has been extended on all interfaces. Because many factors can influence automatically derived values for min-threshold and max-threshold, it is strongly recommended that these values be manually configured.

Cisco IOS does not enforce a similar logic when manually configuring min-threshold and max-threshold. For example, the following configuration is completely valid.

Interface SPD min-threshold and max-threshold hold-queue Description

TenGigabitEthernet1/1

min-threshold = 1498

and

max-threshold = 1499

1500

Backbone connection with high quantities of receive adjacency traffic not eligible for headroom or extended headroom

TenGigabitEthernet1/2

1500

Backbone connection with high quantities of receive adjacency traffic not eligible for headroom or extended headroom

Vlan900

200

Server VLAN where limited receive adjacency traffic is permitted

 

In this configuration, queuing to the input queue on Vlan900 is allowed until the limit of 200 is reached. After that, additional headroom-ineligible packets are simply tailed dropped. In these circumstances, the global SPD state would remain at NORMAL and queuing to all other interfaces on the device would remain unaffected.

The most common default and generally recommended values for hold-queue, headroom, extended headroom, min-threshold, and max-threshold are in the following table. The operating characteristics of networks and network devices vary widely depending on many factors, so it is important that these recommended values be used as a starting point for building a network-specific, operationally appropriate standard configuration.

Parameter Name Scope of Configuration Scope of Operation Default Value Recommended Value

hold-queue

Physical Interface

Physical Interface

75

1500

Headroom

Global

Physical Interface

1000

1000

Extended Headroom

Global

Physical Interface

10

1000

min-threshold

Global

Physical Interface

(Smallest hold-queue on device) – 2

(Largest hold-queue on device) – 2

max-threshold

Global

Physical Interface

(Smallest hold-queue on device) – 1

(Largest hold-queue on device) – 1

 

The operation of the SPD state check, as well as headroom and extended headroom, is illustrated in the following diagram.

 

operation of the SPD state check diagram

 

Functionality and Default Configuration by Cisco IOS Version

SPD has evolved separately on different Cisco IOS Software release trains. This divergence is a direct result of the platforms supported by each release train and the network scenarios to which each platform is targeted. The following table details the default configuration and available functionality of SPD on three primary Cisco IOS Software release trains.

Cisco IOS Version Feature Description
12.0S and 12.2S Trains 12.2SX Trains Mainline and T Trains

12.0(2)S

12.2(14)SX

11.3(1)

Selective Packet Discard

Selective Packet Discard is introduced.

12.0(2)S

12.2(14)SX

11.3(1)

SPD enabled by default

SPD is enabled by default.

12.0(2)S

12.2(14)SX

12.0(1)

Aggressive mode

Aggressive mode is added to drop all malformed packets during the SPD State Random Drop.

12.0(12)S1 12.2(18)S

12.2(18)SXD

12.3(1) 12.2(15)T

Extended Headroom for ISIS and L2 Keepalives

Extended Headroom is added; the default extended headroom is 10.

12.0(18)S 12.2(18)S

12.2(18)SXE

12.3(1) 12.3(2)T

IGPs and HSRP allowed into Extended Headroom

IGPs and HSRP allowed to be queued to the extended headroom.

12.0(22)S

12.2(18)SXE

Default headroom changed from 100 to 1000

Default headroom changed from 100 to 1000 for S and SX trains.

12.3(6) 12.3(6)T

Default extended headroom changed to 75

Default extended headroom changed to 75 for Mainline and T trains.

12.3(6) 12.3(6)T

ARP into extended headroom

ARP is allowed to queue to extended headroom.

12.0(28)S1 12.2(30)S

12.3(6) 12.3(6)T

TDP and LDP Queued to Extended Headroom

TDP and LDP are allowed to queue to extended headroom.

 

Platform-Specific Features, Limitations, and Caveats

Over time, SPD has evolved differently on various platforms based on customer need. Unfortunately, this has made understanding SPD complex. This section attempts to detail the known variations by platform.

Cisco Catalyst 6500 Series Switches and Cisco 7600 Series Routers

  • All Malformed Packets Dropped: As a hardware forwarding platform, all malformed packets are dropped in hardware. This makes SPD aggressive mode irrelevant for this platform.
  • Layer 2 Class of Service–Based Classification: This platform uses the Layer 2 Class of Service of a frame to determine eligibility for queuing to headroom or extended headroom. This inconsistency is scheduled to be remedied in the future. This will bring the Catalyst 6500 Series and the 7600 Series routers in line with other platforms.

Cisco 12000 Series Routers

  • All Malformed Packets Dropped: As a hardware forwarding platform, all malformed packets are dropped in hardware. This makes SPD aggressive mode irrelevant for this platform.
  • SPD Not Available on IOS-XR: Selective Packet Discard is not available on IOS-XR.

Selective Packet Discard as a Component of Multilayer Security

SPD is one of the many tools that must be used in concert to maximize the security and availability of a network infrastructure. Other crucial infrastructure security tools include the following:

  • Control Plane Policing and Protection: Control Plane Policing (CoPP) and Control Plane Protection (CPPr) are technologies that can be used to protect the router CPU from unnecessary and often malicious traffic. It allows users to configure a Quality of Service (QoS) policy that manages the traffic flow of control plane packets to protect the control plane of Cisco routers and switches. This function can help protect against such things as reconnaissance and denial of service (DoS) attacks, allowing the control plane to maintain packet forwarding and control protocol states despite an attack or heavy load on the router or switch.
  • Infrastructure ACLs: Infrastructure access control lists (ACLs) are designed to drop all traffic to the network infrastructure while allowing traffic to transit the same infrastructure. Exceptions to these ACLs are created for any communication that is required to the infrastructure, such as eBGP peering.
  • IP summarization: The use of IP addressing should be considered paramount in defining traffic classes to increase security. Restricting to known source IP addresses helps to limit the zone of trust, keeping in mind that anti-spoofing is often a requirement for complete trust.
  • Unicast RPF: Unicast RPF drops spoofed IP packets based on information in the local routing table. In one use case, if the IP source address of a packet is primarily reachable via an interface other than the interface that received the packet, the packet is dropped. There are several modes in which Unicast RPF can operate.
  • IP Options handling: All IP packets that include IP Options must be handled directly by the router CPU. To minimize the possible impact of a flood of these packets, it is important to devise a network-wide strategy for IP Options handling. Possible solutions may include a device-local configuration that forces the dropping or ignoring of IP Options, or possibly the wholesale dropping of packets with IP Options at the network edge.
  • TTL-based filtering: Packets entering a device with a TTL value of 0 or 1 must be processed by the CPU so that an ICMP Time Exceeded message can be generated. Administrators typically leverage rate limiting to mitigate the CPU load that could be caused by an onslaught of TTL 0 or 1 packets. However, if rate limiting is not possible given the configuration, placement, or capabilities of a device, another option exists. It is possible to use an ACL to filter packets based on the TTL value in a packet. Such an ACL makes it possible to filter at the edge of the network to protect devices inside the network that may not be capable of protecting themselves.
  • Quality of Service remarking: In periods of network congestion, certain control plane traffic must be given preferential queuing and forwarding treatment. QoS often facilitates this treatment. Administrators should remark all traffic at the network edge to protect this special treatment and simultaneously prevent users from receiving priority. The remarking should account for traffic that has been validly marked with an IP Precedence or DSCP greater than zero, such s eBGP.

Conclusion

Although rarely understood, when applied appropriately, SPD can help ensure the availability of a network. This paper aimed to give network engineers the tools to properly understand and implement SPD. This understanding includes the following:

  • The packet classification criteria used by SPD
  • The SPD state check and how the SPD state affects queuing globally
  • How min-threshold and max-threshold are derived or manually configured
  • The usefulness of headroom and the benefits of extended headroom
  • The evolution of SPD over time
  • How SPD is a tool in the arsenal of the network security engineer

Command Reference

hold-queue value in

Use the hold-queue interface configuration command to manually configure the length of the interface input queue. The default hold queue size is 75 packets.

[no] spd enable

This command is used to globally enable or disable SPD. SPD is enabled by default.

show ip spd

The show ip spd privileged EXEC command is used to display the current SPD state and configuration. The following is a sample of this command.

Router#sh ip spd

Current mode: init.

Queue min/max thresholds: 73/74, Headroom: 100, Extended Headroom: 75

IP normal queue: 0, priority queue: 0.

SPD special drop mode: none

ip spd queue min-threshold value

This global configuration command is used to manually configure the threshold at which SPD will transition into the Random Drop state. This value, known as min-threshold, is automatically derived on certain Cisco IOS versions to be equal to the smallest hold queue on the device minus 2. On IOS versions without automatic derivation of this value, min-threshold must be set manually using this command. On all IOS version, it is recommended to set this value manually. When manually configuring this value, it must be assigned to any value below max-threshold or 65535, whichever is lower.

ip spd queue max-threshold value

This global configuration command is used to manually configure the threshold at which SPD will transition into the Full Drop state. This value, known as max-threshold, is automatically derived on certain Cisco IOS versions to be equal to the smallest hold queue on the device minus 1. On IOS versions without automatic derivation of this value, max-threshold must be set manually using this command. On all IOS version it is recommended to set this value manually. When manually configuring this value, it must be assigned to any value above min-threshold and below 65535.

[no] ip spd mode aggressive

The global configuration command ip spd mode aggressive is used to enable the automatic filtering of malformed packets from the interface input queue. Malformed is defined as any packet with an invalid checksum, incorrect version, incorrect header length, or incorrect packet length. Hardware forwarding platforms always drop malformed packets regardless of the existence or configuration of SPD.

spd headroom value

The global configuration command spd headroom is used to specify the length of the SPD headroom. Valid values range from 0 to 65535.

spd extended value

The global configuration command spd extended is used to specify the length of the SPD extended headroom. Valid values range from 0 to 65535.

References

Understanding Selective Packet Discard (SPD)—Cisco 12000 Series Routers
http://www.cisco.com/en/US/partner/products/hw/routers/ps167/products_tech_note09186a008012fb87.shtml (for registered customers)

Classification ACLs, described in Characterizing and Tracing Packet Floods Using Cisco Routers
http://www.cisco.com/warp/public/707/22.html

Configuring Unicast Reverse Path Forwarding
http://www.cisco.com/univercd/cc/td/doc/product/software/ios121/121cgcr/secur_c/scprt5/scdrpf.htm

Protecting Your Core: Infrastructure Protection Access Control Lists
http://www.cisco.com/warp/public/707/iacl.html

Transit Access Control Lists: Filtering at Your Edge
http://www.cisco.com/warp/public/707/tacl.html

Deploying Control Plane Policing
http://www.cisco.com/en/US/products/sw/iosswrel/ps1838/products_feature_guide09186a008052446b.html

NetFlow
http://www.cisco.com/go/netflow

Cisco Security Intelligence Operations
http://www.cisco.com/security

Cisco Design Zone for Security
http://www.cisco.com/en/US/netsol/ns744/networking_solutions_program_home.html

Cisco Network Foundation Program
http://www.cisco.com/go/nfp

Notes

1. Information about memory and buffer management is available at http://www.cisco.com/warp/public/63/buffertuning.pdf.

2. Platform-specific variations are detailed in the "Platform-Specific Features, Limitations, and Caveats" section of this document.

3. This table also assumes that min-threshold and max-threshold have been set to a value less than the hold-queue length. Strictly speaking, this is not a requirement.

4. These global configuration commands are hidden. They will appear in the configuration and survive reboots, but will not be visible in the CLI help system.

This document is part of Cisco Security Intelligence Operations.

This document is provided on an "as is" basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness for a particular use. Your use of the information on the document or materials linked from the document is at your own risk. Cisco reserves the right to change or update this document at any time.

Back to Top

Cisco Security Intelligence Operations