Cisco - Buffers, Queues & Thresholds on Catalyst 6500 Ethernet Modules

With the wide variety of Ethernet modules currently available on the Catalyst 6500 series switches, determining the size of the port buffers, the number and type of the receive and transmit queues, and the number of tail-drop or weighted random early detect (WRED) thresholds can be a daunting task.

This document provides a brief discussion of how the port buffers, queues, and thresholds are used in the Catalyst 6500 series switch Ethernet modules, and then describes the buffer sizes, queue structures, and thresholds supported on each of the modules.

This document does not discuss the Cisco 7600 Optical Services Module (OSM) interfaces or the new SIP and SPA modules that are also supported in the Catalyst 6500 series switches.

Overview of Buffers, Queues, and Thresholds

All of the Catalyst 6500 series Ethernet modules implement some form of receive and transmit buffering. These buffers are used to store frames as forwarding decisions are made within the switch, or as packets are enqueued for transmission on a port at a rate greater than the physical medium can support.

In the Catalyst 6500 architecture, access into the switch fabric itself is almost never the bottleneck. Rather, on the transmit side, one or several ports are the likely destination for a majority of the packets entering the switch. As such, the receive-side port buffers on the Ethernet modules are relatively small compared to the transmit-side port buffers.

Without Quality of Service (QoS) features enabled, all packets have equal access to the port buffers, regardless of the type of traffic. For example, Voice over IP (VoIP) packets get placed in the same buffer as HTTP (web) or FTP (file download) data packets. Furthermore, in the event of congestion (that is, a port buffer overflows), all traffic is equally subject to discard. To summarize, packets are serviced in the order in which they were received, and if the buffer is full, all subsequent packets are dropped. This is known as First In, First Out (FIFO) queuing with tail-drop.

When you enable QoS on the switch, the port buffers are carved into one or more individual queues. Each queue has one or more drop thresholds associated with it. The combination of multiple queues within a buffer, and the drop thresholds associated with each queue, allow the switch to make intelligent decisions when faced with congestion. Traffic sensitive to jitter and delay variance, such as VoIP packets, can be moved to the head of the queue for transmission, while other less important or less sensitive traffic can be buffered or dropped.

Port Queue and Drop Threshold Types

When QoS is enabled, the multiple queues and drop thresholds on the Ethernet module switch ports are enabled. There are several different configurations of queue types and thresholds, depending on the model of the Ethernet module.

Ingress and egress scheduling are always based on the COS value associated with the frame. By default, higher COS values are mapped to higher queue numbers. COS 5 traffic, typically associated with VoIP traffic, is mapped to the strict priority queue, if present.

In addition to the different queues, each standard queue has one or more drop thresholds. There are two types of drop thresholds:

•

Tail-drop thresholds—On ports with tail-drop thresholds, frames of a given COS value are admitted to the queue until the drop threshold associated with that COS value is exceeded; subsequent frames of that COS value are discarded until the threshold is no longer exceeded.

For example, if COS 1 is assigned to queue 1, threshold 2, and the threshold 2 watermark is 60%, then frames with COS 1 will not be dropped until queue 1 is 60% full. All subsequent COS 1 frames are dropped until the queue is less than 60% full.

•

WRED drop thresholds—On ports with WRED drop thresholds, frames of a given COS value are admitted to the queue based on a random probability designed to avoid buffer congestion. The probability of a frame with a given COS being admitted to the queue or discarded depends on the weight and threshold assigned to that COS value.

For example, if COS 2 is assigned to queue 1, threshold 2, and the threshold 2 watermarks are 40% (low) and 80% (high), then frames with COS 2 will not be dropped until queue 1 is at least 40% full. As the queue depth approaches 80%, frames with COS 2 have an increasingly higher probability of being discarded rather than being admitted to the queue. Once the queue is over 80% full, all COS 2 frames are dropped until the queue is less than 80% full. The frames the switch discards when the queue level is between the low and high thresholds are picked out at random, rather than on a per-flow basis or in a FIFO manner. This method works well with protocols, such as TCP, capable of adjusting to periodic packet drops by backing off and adjusting their transmission window size.

When referring to the individual queues and thresholds on a port, a rather terse nomenclature is used. This terminology describes the number of strict priority queues (if present), the number of standard queues, and the number of tail-drop or WRED thresholds within each of the standard queues. The different queue and threshold types on the Catalyst 6500 Ethernet modules are shown in Table 1.

Buffer Size, Queues, and Thresholds by Ethernet Module

Table 2 provides the following information for each of the Catalyst 6500 series Ethernet modules:

Table 1 Receive and Transmit Port Queue and Drop Threshold Types on Catalyst 6500 Series Ethernet Modules
Port Queue and Drop Threshold Structure with QoS	Description
Receive Queues
1q2t	One standard queue with two tail-drop thresholds
1q4t	One standard queue with four tail-drop thresholds
1q8t	One standard queue with eight tail-drop thresholds
2q8t*	Two standard queues with eight tail-drop thresholds per queue
1p1q4t	One strict-priority queue, one standard queue with four tail-drop thresholds
1p1q0t	One strict-priority queue, one standard queue with one non-configurable (100%) tail-drop threshold
1p1q8t	One strict-priority queue, one standard queue with eight configurable WRED drop thresholds and one non-configurable (100%) tail-drop threshold
Transmit Queues
2q2t	Two standard queues with two tail-drop thresholds per queue
1p2q2t	One strict-priority queue, two standard queues with two WRED drop thresholds per queue
1p3q1t	One strict-priority queue, three standard queues with one WRED drop threshold and one non-configurable tail-drop threshold per queue
1p2q1t	One strict-priority queue, two standard queues with one WRED drop threshold and one non-configurable (100%) tail-drop threshold per queue
1p3q8t*	One strict-priority queue, three standard queues with eight WRED drop thresholds per queue
1p7q8t*	One strict-priority queue, seven standard queues with eight WRED drop thresholds per queue

Table 2 Buffer Size, Queues, and Thresholds by Ethernet Module
Module Model Name	Module Description	Total Buffer Size	Rx Buffer Size	Tx Buffer Size	Rx Port Type	Tx Port Type	Rx Queue Sizes	Tx Queue Sizes
Supervisor Engine Modules (Gigabit Ethernet and Ten Gigabit EthernetUplink Ports)
WS-SUP720 WS-SUP720-3B WS-SUP720-3BXL	All Supervisor Engine 720 uplink ports	512KB	80KB	432KB	1p1q4t	1p2q2t	SP—10KB Q1—70KB	SP—64KB Q2—64KB Q1—304KB
WS-SUP32-10GE	Supervisor 32-10GE TenGigabitEthernet uplink ports	193MB	105MB	88MB	2q8t	1p3q8t	Q2—1MB Q1—104MB	SP—14MB Q3—13MB Q2—17MB Q1—44MB
WS-SUP32-10GE	Supervisor 32-10GE GigabitEthernet Uplink port	17.7MB	9.6MB	8.1MB	2q8t	1p3q8t	Q2—1.9MB Q1—7.7MB	SP—1.2MB Q3—1.2MB Q2—1.6MB Q1—4.1MB
WS-SUP32-GE	All Supervisor Engine 32 uplink ports	10MB	5MB	5MB	2q8t	1p3q8t	Q2—1MB Q1—4MB	SP—715KB Q3—715KB Q2—954KB Q1—2.4MB
WS-X6K-S2U-MSFC2 WS-X6K-S2-MSFC2 WS-X6K-S2-PFC2	All Supervisor Engine 2 uplink ports	512KB	80KB	432KB	1p1q4t	1p2q2t	SP—10KB Q1—70KB	SP—64KB Q2—64KB Q1—304KB
WS-X6K-S1A-MSFC2 WS-X6K-SUP1A-MSFC WS-X6K-SUP1A-PFC WS-X6K-SUP1A-2GE	All Supervisor Engine 1A uplink ports	512KB	80KB	432KB	1p1q4t	1p2q2t	SP—10KB Q1—70KB	SP—64KB Q2—64KB Q1—304KB
WS-X6K-SUP1-2GE	Supervisor Engine 1 uplink ports	512KB	80KB	432KB	1q4t	2q2t	Q1—80KB	Q2—80KB Q1—352KB
Ethernet and Fast Ethernet Modules
WS-X6524-100FX-MM	24-port 100BaseFX fabric-enabled with MT-RJ connectors	1116KB	28KB	1088KB	1p1q0t	1p3q1t	SP—6KB Q1—22KB	SP—272KB Q3—272KB Q2—272KB Q1—272KB
WS-X6548-RJ-21	48-port 10/100BaseTX fabric-enabled with RJ-21 connectors	1116KB	28KB	1088KB	1p1q0t	1p3q1t	SP—6KB Q1—22KB	SP—272KB Q3—272KB Q2—272KB Q1—272KB
WS-X6548-RJ-45	48-port 10/100BaseTX fabric-enabled with RJ-45 connectors	1116KB	28KB	1088KB	1p1q0t	1p3q1t	SP—6KB Q1—22KB	SP—272KB Q3—272KB Q2—272KB Q1—272KB
WS-X6324-100FX-MM	24-port 100BaseFX with MT-RJ connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6324-100FX-SM	24-port 100BaseFX with MT-RJ connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6348-RJ-45 WS-X6348-RJ-45V	48-port 10/100BaseTX with RJ-45 connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6348-RJ21V	48-port 10/100BaseTX with RJ-21 connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6224-100FX-MT	24-port 100BaseFX with MT-RJ connectors	64KB	8KB	56KB	1q4t	2q2t	Q1—8KB	Q2—16KB Q1—40KB
WS-X6248-RJ-45	48-port 10/100BaseTX with RJ-45 connectors	64KB	8KB	56KB	1q4t	2q2t	Q1—8KB	Q2—16KB Q1—40KB
WS-X6248-TEL	48-port 10/100BaseTX with RJ-21 connectors	64KB	8KB	56KB	1q4t	2q2t	Q1—8KB	Q2—16KB Q1—40KB
WS-X6248A-TEL	48-port 10/100BaseTX with RJ-21 connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6196-RJ-21 WS-X6196-21AF	96-port 10/100BaseTX with RJ-21 connectors	1116KB	28KB	1088KB	1p1q0t	1p3q1t	SP—6KB Q1—22KB	SP—272KB Q3—272KB Q2—272KB Q1—272KB
WS-X6148A-RJ45 WS-X6148A-45AF	48-port 10/100BaseTX with RJ-45 connectors	5.3MB	60KB	5.3MB	1p1q4t	1p3a8t	SP—8KB Q1—52KB	SP—810KB Q3—810KB Q2—1.1MB Q1—2.6MB
WS-X6148-RJ-45 WS-X6148-RJ-45V WS-X6148-45AF	48-port 10/100BaseTX with RJ-45 connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6148-RJ-21 WS-X6148-RJ-21V WS-X6148-21AF	48-port 10/100BaseTX with RJ-21 connectors	128KB	16KB	112KB	1q4t	2q2t	Q1—16KB	Q2—22KB Q1—90KB
WS-X6148X2-RJ-45 WS-X6148X2-45AF	96-port 10/100BaseTX with RJ-45 connectors	1116KB	28KB	1088KB	1p1q0t	1p3q1t	SP—6KB Q1—22KB	SP—272KB Q3—272KB Q2—272KB Q1—272KB
WS-X6148-FE-SFP	48-port 100BaseFX with SFP transceivers	5.4MB	60KB	5.4MB	1p1q4t	1p3q8t	SP—8KB Q1—52KB	SP—810KB Q3—810KB Q2—1.1MB Q1—2.7MB
WS-X6024-10FL-MT	24-port 10Base-FL with MT-RJ connectors	64KB	8KB	56KB	1q4t	2q2t	Q1—8KB	Q2—16KB Q1—40KB
Gigabit Ethernet and 10/100/1000 Modules
WS-X6816-GBIC	16-port 1000BaseX dual-fabric with GBIC transceivers	512KB	80KB	432KB	1p1q4t	1p2q2t	SP—10KB Q1—70KB	SP—64KB Q2—64KB Q1—304KB
WS-X6748-GE-TX	48-port 10/100/1000T dual-fabric with RJ-45 connectors	1.3MB	166KB	1.2MB	2q8t (with DFC3)	1p3q8t	Q2—33KB Q1—133KB (with DFC3)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6748-GE-TX	48-port 10/100/1000T dual-fabric with RJ-45 connectors	1.3MB	166KB	1.2MB	1q8t (with CFC)	1p3q8t	Q1—166KB (with CFC)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6748-SFP	48-port 1000BaseX dual-fabric with SFP transceivers	1.3MB	166KB	1.2MB	2q8t (with DFC3)	1p3q8t	Q2—33KB Q1—133KB (with DFC3)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6748-SFP	48-port 1000BaseX dual-fabric with SFP transceivers	1.3MB	166KB	1.2MB	1q8t (with CFC)	1p3q8t	Q1—166KB (with CFC)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6724-SFP	24-port 1000BaseX single-fabric with SFP transceivers	1.3MB	166KB	1.2MB	2q8t (with DFC3)	1p3q8t	Q2—33KB Q1—133KB (with DFC3)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6724-SFP	24-port 1000BaseX single-fabric with SFP transceivers	1.3MB	166KB	1.2MB	1q8t (with CFC)	1p3q8t	Q1—166KB (with CFC)	SP—175KB Q3—175KB Q2—233KB Q1—583KB
WS-X6548-GE-TX WS-X6548V-GE-TX WS-X6548-GE-45AF	48-port 10/100/1000T fabric-enabled with RJ-45 connectors	1.4MB (shared among 8 ports)	185KB (shared among 8 ports)	1.2MB (shared among 8 ports)	1q2t (shared among 8 ports)	1p2q2t (shared among 8 ports)	Q1—185KB	SP—182KB Q2—182KB Q1—849KB
WS-X6516-GBIC	16-port 1000BaseX fabric-enabled with GBIC transceivers	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6516A-GBIC	16-port 1000BaseX fabric-enabled with GBIC transceivers	1MB	144KB	880KB	1p1q4t	1p2q2t	SP—26KB Q1—118KB	SP—128KB Q2—128KB Q1—624KB
WS-X6516-GE-TX	16-port 10/100/1000T fabric-enabled with RJ-45 connectors	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6408-GBIC	8-port 1000BaseX with GBIC transceivers	512KB	80KB	432KB	1q4t	2q2t	Q1—80KB	Q2—80KB Q1—352KB
WS-X6408A-GBIC	8-port 1000BaseX with GBIC transceivers	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6416-GBIC	16-port 1000BaseX with GBIC transceivers	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6416-GE-MT	16-port 1000BaseSX with MT-RJ connectors	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6316-GE-TX	16-port 1000BaseT with RJ-45 connectors	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB
WS-X6148A-GE-TX WS-X6148A-GE-45F	48-port 10/100/1000T with RJ-45 connections	5.5MB	120KB	5.4MB	1q2t	1p3q8t	Q1—120KB	SP—810KB Q3—810KB Q2—1.1MB Q1—2.7MB
WS-X6148-GE-TX WS-X6148V-GE-TX WS-X6148-GE-45AF	48-port 10/100/1000T with RJ-45 connectors	1.4MB (shared among 8 ports)	185KB (shared among 8 ports)	1.2MB (shared among 8 ports)	1q2t (shared among 8 ports)	1p2q2t (shared among 8 ports)	Q1—185KB	SP—182KB Q2—182KB Q1—849KB
Ten-Gigabit Ethernet Modules
WS-X6704-10GE	4-port 10GbE dual-fabric with XENPAK transceivers	16MB	2MB	14MB	8q8t (with DFC3)	1p7q8t	Q8—400KB Q7—0KB Q6—0KB Q5—0KB Q4—0KB Q3—0KB Q2—0KB Q1—1.6MB	SP—2.2MB Q7—0KB Q6—0KB Q5—0KB Q4—0KB Q3—2.2MB Q2—2.9MB Q1—7.2MB
WS-X6704-10GE	4-port 10GbE dual-fabric with XENPAK transceivers	16MB	2MB	14MB	1q8t (with CFC)	1p7q8t	Q1—2MB
WS-X6502-10GE	1-port 10GbE fabric-enabled with OIM connectors	64.2MB	256KB	64MB	1p1q8t	1p2q1t	SP—51KB Q1—205KB	SP—15.3MB Q2—17.9MB Q1—30.7MB
WS-X6501-10GEX4	1-port 10GbE fabric-enabled with SC connectors	64.2MB	256KB	64MB	1p1q8t	1p2q1t	SP—51KB Q1—205KB	SP—15.3MB Q2—17.9MB Q1—30.7MB
Optical Services Modules¹
OSM-*	All Optical Services Modules (L2 GbE fabric-enabled ports only)	512KB	73KB	439KB	1p1q4t	1p2q2t	SP—9KB Q1—64KB	SP—64KB Q2—64KB Q1—311KB

¹ This document only discusses the four "standard" Gigabit Ethernet ports present on most of the OSMs. It does not discuss the 4-port GE-WAN OSM or any other OSM-related interfaces.

Table Of Contents

WHITE PAPER

Buffers, Queues, and Thresholds on Catalyst 6500 Series Ethernet Modules

Overview of Buffers, Queues, and Thresholds

Port Queue and Drop Threshold Types

Buffer Size, Queues, and Thresholds by Ethernet Module