Guest

Cisco Catalyst 6800 Series Switches

Cisco Catalyst 6880-X - Innovation with Investment Protection for Tomorrow's Campus Backbone White Paper

  • Viewing Options

  • PDF (2.0 MB)
  • Feedback

Introduction. 3

Introducing the New Cisco Catalyst 6880-X Chassis. 3

What Is Innovation with Investment Protection?. 3

Does the Catalyst 6880-X Provide Innovation with Investment Protection?. 4

Chassis Overview.. 4

Chassis Highlights. 4

Chassis Design. 5

Chassis Backplane. 6

Number of Channels. 6

Clock Frequency. 7

Line Encoding. 8

Chassis Power. 8

Power Supply Units. 8

Power Inputs. 9

Chassis Cooling. 10

Enhanced Fan Tray. 10

Chassis Air Flow.. 11

Baseboard Details. 12

Baseboard Models. 12

Baseboard Components. 13

Port Card Details. 17

Port Card Models. 17

16-Port SFP Port Card Components. 17

Packet Walks. 20

Ingress Forwarding. 20

Egress Forwarding. 21

Conclusion. 22

For More Information. 22


Introduction

Exponential growth in network traffic has occurred over the past several years, and this trend is expected to continue into the foreseeable future. By 2016, there will be 19 billion networked devices, up from 10 billion in 2011. Forecasts predict that business IP traffic will reach 13.1 exabytes per month in 20161. Networks must be capable of scaling well beyond their capabilities today to deal with the traffic of tomorrow.

Figure 1. Cisco Catalyst 6880-X

Introducing the New Cisco Catalyst 6880-X Chassis

Cisco is pleased to announce the latest high-density, low-footprint “extensible fixed” switching platform, the Cisco® Catalyst® 6880-X Switch, designed for flexible deployment in enterprise campus core and distribution environments. This new 4-half-slot chassis with a fixed supervisor can support up to 220Gbps per slot, for a maximum system capacity of approximately 2 Tbps (up to approximately 4 Tbps in a virtual switching system [VSS]).

This whitepaper provides an architectural overview of the new Cisco Catalyst 6880-X chassis, including system design, power, cooling, and its support for both current and future modules. To the extent possible at the time of this writing, it provides all of the technical details necessary to fully understand this new chassis.

The new Cisco Catalyst 6880-X chassis is the “fixed” aspect of a new Cisco Catalyst 6800 Series family of multi-layer switching products, which combine significant technological “innovation” with unparalleled investment protection”, to support tomorrow’s Campus Backbone.

What Is Innovation with Investment Protection?

Many technology companies develop innovative new products that can boost the performance or capabilities of your network, but they usually require you to deploy an entirely new set of hardware and software. This undermines your previous investments and forces you to reinvest more money and relearn new products.

Why? It is easier and cheaper for other companies to simply create a completely new product. This approach eliminates the hardware or software development complexity of backward compatibility, but severely limits or prevents the use of earlier generations. That is, it is easier and cheaper for them, not for you.

Cisco Catalyst switches have been built (from the beginning) to maximize backward compatibility, along with an architectural foundation for next-generation hardware features and scalability. Since its introduction in 1999, the venerable Cisco Catalyst 6000 Series switches have supported at least two previous generations, working along side the current and next-generation products.

Does the Catalyst 6880-X Provide Innovation with Investment Protection?

Absolutely, as you will see during the course of this paper.

While the Catalyst 6880-X design itself is new, all of the same Catalyst 6500 Series hardware components (application-specific integrated circuits [ASICs], field-programmable gate arrays [FPGAs], and so on) along with all of the same Catalyst 6500 Series software features (infrastructure, protocols, and so on) have been used to build it. It’s simply a new package.

If you are already familiar with Supervisor Engine 2T and WS-X6904-40G hardware, and if you are familiar with 15.1SY Cisco IOS® Software, you are already familiar with the new Catalyst 6880-X system.

Chassis Overview

Chassis Highlights

This section briefly covers the highlights of the new Cisco Catalyst 6880-X chassis. Additional details can be found in later sections.

The Catalyst 6880-X is a 5-slot semi-fixed chassis, based on the DNA of the Catalyst 6500 Series.

This chassis includes the following highlights:

Support for light and heavy table scalability: Allows you to select a system that best fits your network scale needs, without breaking the bank.

High-performance backplane traces: Supports new clock frequencies that enable exponentially greater bandwidth for future modules.

New “extensible fixed” chassis design: Combines the flexibility of modularity with the economics of a fixed system.

Up to 80 multirate Small Form-factor Pluggable Plus (SFP+) ports: Fixed baseboard provides 16 SFP+ ports, and the system can be expanded with up to 4 additional 16 SFP+ port cards for a total of 80 ports.

Up to two “platinum-efficiency” 3000W power supplies: These support either 1:1 power redundancy or up to 6000W combined.

New fan tray with four high-efficiency 6000-rpm fans: Provide approximately 450 cfm, capable of cooling a fully loaded system with both current and future modules.

Smaller-footprint 4.75 rack unit (RU) form factor: Gives you an extra card slot and double the bandwidth but require seven less space than the Catalyst 6504-E (5RU).

Other Highlights

Supports 24 fabric channels

- 4 per card slot ([20]+ 4 Sup) x 7.5-GHz serializer/deserializer (SerDes)

Supports a new Multilayer Switch Feature Card [MSFC] route processor (RP) complex

- New Intel-based 2.0-GHz CPU and 4GB of DDR3 memory

Supports an improved Policy Feature Card (PFC) forwarding controller

- Based on PFC4 (EARL8) with enhancements and fixes

Supports a new Serial Gigabit Media Independent Interface (SGMII)-based switched Ethernet out-of-bound channel (EOBC)

- Dedicated programming channels (rather than bus based)

Supports a new custom chassis mounting system

- Designed for multiple two-post racking solutions

Chassis Design

This section briefly covers the high-level system design of the new Cisco Catalyst 6880-X chassis. Additional details can be found in later sections.

The chassis is only 4.75 RU high. That is smaller than the WS-C6504-E (5RU).

The chassis dimensions are (H x W x D): 17.35 x 8.5 x 23.0 inches (44.06 x 21.59 x 58.42 cm).

There are four module slots (also known as port card slots) and a single (fixed) supervisor slot.

In the past, a very popular platform was the Catalyst ME6524. This was a fixed 2RU chassis that was architecturally equivalent to a Supervisor Engine 32 combined with a WS-X6724 module. Customers expressed a need for the comprehensive software and hardware features of the Catalyst 6500 platform, but in a smaller (fixed) form factor.

This concept was the genesis for the new Catalyst 6880-X chassis design. At the same time, we noticed that there are certain drawbacks to a purely fixed design (notably the inability to increase port scale once the purchase and deployment have been completed). Hence, we also wanted to provide a certain level of modularity to allow customers to flexibly extend their “fixed” chassis.

This small form factor “extensible fixed” design approach, when combined with increased bandwidth per slot (and future port density), allows for more scale in less space.

There are two power supply unit (PSU) slots, and each PSU can handle AC or DC power inputs.

Each power supply can support up to 3000W. Many customers desire more than one power supply, for redundancy (for example, 1:1) configurations, as well as the ability to scale combined power up to 6000W.

This approach increases both the available power and redundancy while simultaneously reducing the cost of the equipment.

There is a front-serviceable fan tray with four variable-speed fans.

As the power requirements for new modules have increased, so have the cooling requirements. The enhanced fan tray provides enough cooling capacity to support a fully loaded chassis with all of the current modules as well as future modules (which will require even greater cooling capacity).

The fan tray supports hardware failure of up to one individual fan, and the remaining fans will automatically increase their rpm to compensate and maintain sufficient cooling. The fan tray also supports hardware online insertion and removal (OIR) for up to 120 seconds (a longer time is possible, depending on the ambient temperature) without service interruption.

Figure 2 shows a mechanical view of the Catalyst 6880-X.

Figure 2. Mechanical View of Catalyst 6880-X

Chassis Backplane

Three basic factors determine band width on a per-slot or per-system (total) basis in a modular hardware switching chassis:

Number of channels

Clock frequency

Line encoding

The sections that follow summarize each of these three factors.

Number of Channels

In traditional (external) networking, there are a finite number of physical approaches to connecting independent network nodes together. The overall bandwidth capacity and throughput (performance) of each approach is dictated by the characteristics of the medium and the software used to manage it.

In many ways, you can simply think of each module as an independent node, attached to the other modules via an internal network (in the chassis). This way of thinking helps provide a historical perspective on how chassis backplanes (and the attached modules) have evolved.

In external networking, one basic approach is to provide multiple data paths (also known as channels) to a single module. Using this approach, a modular chassis can increase its so-called “per-slot” bandwidth capacity, multiplied by the number of attached channels.

The new Catalyst 6880-X provides up to four channels for each module slot (Figure 3).

Note: Each channel further consists of eight parallel SerDes (serializer/deserializer) lanes. These eight parallel SerDes lanes are aggregated together to form a single logical fabric channel. The exact details of SerDes architecture and functions are outside the scope of this document. Please refer to the following link for additional information.
http://en.wikipedia.org/wiki/SerDes

Figure 3 shows the channel allocation of the Cisco Catalyst 6880-X.

Figure 3. Channel Allocation on the Catalyst 6880-X

For comparison, the Catalyst 6500-E Series chassis provides up to two channels per port card to each supervisor.

Thus, the new Catalyst 6880-X provides double the number of available channels.

Clock Frequency

All computer-based “data” is represented by binary “bits”, often referred to as “1’s” and “0’s”, which are simple representations of either the presence of (1) or absence of (0) an electrical or optical signal. The amount of time between the arrival of each individual “bit” is called its “frequency”, measured in Hertz (Hz), and controlled by a central “clock”.

What we commonly refer to as the “speed” of a piece of hardware is actually a reference to its clock frequency. The faster the frequency, the faster individual bits will arrive. However, these bits must be interpreted correctly in order for the higher functions to make sense of them. This requirement is the purpose of line encoding, discussed in the next section.

Note: The exact details of calculating and measuring clock frequency are outside the scope of this document. Please refer to the following link for additional information.
http://en.wikipedia.org/wiki/Clock_frequency

Note: Several lesser factors also influence the maximum clock frequency, such as the characteristics and quality (for example, signal integrity) of the physical medium, the design and operation of the components (such as the SerDes), the accuracy of the central clock, and others.

Note: These additional factors are also beyond the scope of this document, but it is important to point out that a chassis must also support them in order to support higher frequencies.

The clock frequencies supported by the Catalyst 6880-X are as follows:

6.25GHz: For up to 40Gbps per channel

7.50GHz: For up to 55Gbps per channel

Line Encoding

Along with clock frequency (which dictates the arrival rate of data “bits”), the transmitting and receiving equipment must both agree about how to interpret “good” data from “bad” (corrupt) data.

Furthermore, if data transmissions (bits) are simply the presence or absence of signal (1 or 0), how can the receiving equipment understand the difference between a “good” transmission of bits, or if it is actually experiencing interference, or if the transmission has completely ceased?

That dilemma explains the basic purpose of line encoding. It is actually much more complicated, including how to define what a “bit” is (based on the signal wavelength and frequency), but that is its essential function.

The transmitting equipment sends a fixed number of bits together in a sequence (encoding), and then the receiving equipment uses some portion of the total bits to determine if the transmission was successful (or not).

Note: The exact details of various line encoding schemes are outside the scope of this document. Please refer to the following link for additional information.
http://en.wikipedia.org/wiki/Line_code

The line encodings supported by the Catalyst 6880-X are as follows:

08/10b: For up to 40Gbps per channel

24/26b: For up to 55Gbps per channel

Along with clock frequency and number of channels, this is how the Catalyst 6880-X is capable of achieving up to 220Gbps (8 SerDes at 7.5GHz at 24/26b = 55Gbps per channel x 4 channels).

Note: The actual utilization of the available band width also depends on the exact supervisor(s) and module(s) and the associated ASICs and FPGAs (which actually process the data transmitted and received). This point will be discussed more in the “Baseboard Details” and “Port Card Details” sections.

Chassis Power

The Catalyst 6880-X supports up to two 3000W AC or DC power supply units (PSUs), for a total system capacity of up to 6000W.

The system supports either one or two PSUs operating in either combined or redundant mode.

Power Supply Units

The maximum power output per PSU is 3000W at 220V or 1400W at 110V (Figure 4).

Each PSU has been rated as “platinum efficient” for greater than 90 percent power efficiency at a 100 percent load.

Each PSU has a power hold-up time of approximately ~20 milliseconds (msec or ms) at a 100 percent load and fully supports OIR.

Each PSU supports dual redundant “front to back” variable-speed cooling fans.

Figure 4. Catalyst 6880-X 3000W AC PSU

Each PSU has a push-release lock and manual ejector lever, for simple but secure OIR.

Each PSU supports multiple LEDs to determine component and power input/output status (Figure 5).

Figure 5. Catalyst 6880-X AC PSU LEDs

Note: PSU fault conditions are:

5V out of range

Output stage OT

Fan fault

OR-ing fault (output is less than bus voltage)

OC shutdown

OT shutdown

OV shutdown

Input stage OT

Fault-induced shutdown occurred

Thermal sensor fault

Vout out of range

Boost Vbulk fault

Power Inputs

The Catalyst 6880-X supports fully OIR-capable small form-factor PSUs when using redundant power supplies. This allows the user to easily insert and remove PSUs without loss of service.

Two separate PSUs are located on the bottom center (facing) of the chassis, and each has its own on/off switch.

The PSUs are numbered (1 or 2) from left to right (Figure 6).

Figure 6. Numbering of Catalyst 6880-X Power Supplies

Note: The Catalyst 6880-X chassis will support both AC and DC power input (up to 16A).
http://en.wikipedia.org/wiki/AC_power
http://en.wikipedia.org/wiki/DC_power

Chassis Cooling

The Catalyst 6880-X supports a single front-serviceable fan tray, capable of providing up to 450 cfm in order to properly cool five 500W modules.

This is an enhanced fan-tray assembly with four redundant variable-speed 6000-rpm fans.

Enhanced Fan Tray

The maximum cooling capacity of the fan tray is approximately 500 CFM (approximately 100 to 150CFM per slot).

Each fan has been rated as high efficiency, for greater than 80 percent power efficiency at a 100 percent load.

Each fan has an acoustic noise rating of 67 dB (per ISO-7779) when operating at 14° to 149°F (-10° to 65°C).

Each fan supports four variable-speed operating modes between 3000 and 6000 RPM.

The fan tray supports up to one fan failure (the remaining fans will increase RPM/CFM).

The fan tray supports OIR for a minimum of ~120 seconds (depends on ambient temperature). Individual fans cannot be replaced (you must replace the fan tray).

Figure 7. Catalyst 6880-X Fan Tray

The fan tray has a built-in Multipoint Control-Unit (MCU) controller to receive (for example, fan speed) and send (for example, the temperature status) messages to the supervisor engine, through the Inter-Integrated Circuit (I2C) bus. The MCU monitors the thermistor mounted on the fan tray and sets fan speeds based on temperature ranges set by the MCU.

The fan tray uses a thumb-screw lock mechanism and fan-tray handle, for simple and secure OIR (Figure 7).

The fan tray supports multiple LEDs to determine fan-tray status, including a new blue ID LED.

Figure 8. Catalyst 6880-X Fan-Tray LEDs

Note: Fan-tray fault conditions are:

One or more fans stop spinning

42v hot-swap failures

3.3v goes away (from supervisor card)

+5v supply fails

PWM signal 0% duty cycle

Thermistor failure

System temperature out of range

Chassis Air Flow

The Catalyst 6880-X fan tray supports side-to-side (right-to-left) air flow (Figure 9). This was chosen as its optimal deployment in campus core, main distribution frame (MDF), and intermediate distribution frame (IDF) deployments. Simple airflow baffles or specialized data center racks can be used to redirect the airflow exhaust to the rear, if necessary.

To maintain proper airflow circulation, provide a minimum separation of 6 inches (15 cm) between a wall and the chassis air intake or air exhaust. Also allow for a minimum separation of 12 inches (30.5 cm) between the hot air exhaust on one chassis and the air intake of another chassis.

Figure 9. Catalyst 6880-X Fan-Tray Air Flow

Baseboard Details

The new Catalyst 6880-X chassis is based on a “fixed” platform philosophy. The fixed component of the chassis is the system baseboard (Figure 10). In many ways, it is the Supervisor Engine 2T of the system, but it is also a 16-port multirate SFP+ port card.

Figure 10. Catalyst 6880-X Baseboard

Baseboard Models

The Catalyst 6880-X supports two base board models that offer different levels of scalability (comparable to the two scalability models of the Supervisor Engine 2T), based on the PFC4-E forwarding engine present.

The two models are C6880-X (Heavy) and C6880-X-LE (Lite).

Table 1 outlines some high-level differences between these two models.

Table 1. Comparison of the Catalyst 6880-X Baseboards

Hardware Option

C6880-X-LE (Lite)

C6880-X (Heavy)

IPv4/v6 routing capability

256,000/128,000

2 million/1 million

Multicast routes (IPv6)

64,000

256,000

Number of adjacencies

256,000

1 million

MAC addresses

128,000

128,000

Layer 3 interfaces

128,000

128,000

Security and quality-of-service (QoS) access control lists (ACLs)

64,000

256,000

Flexible NetFlow

512,000

1 million

Microflow policers

512

512

Aggregate policers

8000

8000

Baseboard Components

The Catalyst 6880-X fixed supervisor slot has three main components:

RP complex

Fabric complex

Port complex

Figure 11. Catalyst 6880-X Baseboard

As noted earlier, the Catalyst 6880-X baseboard is architecturally equivalent to a Supervisor Engine 2T combined with a WS-X6904-40G module (operating in 10Gigabit Ethernet mode) (Figure 10).

RP Complex

The RP complex is the route processor of the system. It is based on the well-known Multilayer Switch Feature Card (MSFC) on the Supervisor 2T, along with several important new and improved components. As such, it has the same basic design and layout and serves the same basic functions (Figure 12).

Below is a summary of the highlights:

New 2.0-GHz X86 dual-core CPU

- A new Intel® Gladden dual-core CPU running at 2.0GHz

- Initial profiling suggests a nearly two fold increased in control-plane scale and performance

2 or 4GB of 1337-MHz DDR3 ECC SDRAM

- DDR3 can transfer data at twice the rate of DDR2:
http://en.wikipedia.org/wiki/DDR3

Support for USB Type A file system

- Supports standard USB thumb drives as a Cisco IOS file system (for example, copy, delete, format, andboot)

Support for USB Type B serial console

- Supports standard Type B (mini) serial console cables (in addition to RJ-45 serial)

Compact flash replaced with eUSB

- Internal Enhanced USB (eUSB) flash will support 8GB of storage

- The eUSB design provides single-level cell (SLC) NAND flash with a sequential read of up to 21 MBps, and write of up to 18 MBps

New switched EOBC interface

- Supports existing EOBC architecture but uses a point-to-point (P2P) switched design for dedicated bandwidth

Figure 12. RP Complex Block Diagram

The Gigabit Ethernet in-band MAC driver (on the baseboard and modular port cards) uses three queues per port: high priority, medium priority, and low priority. A weighted round robin scheme is used to service these queues.

Each of the queues has an independent buffer ring to minimize packet loss due to data storms. Priorities are defined as follows:

High priority: Incoming packets with an internal header with the Bridge Protocol Data Unit (BPDU) bit set, or with an internal header class-of-service (CoS) field in the range 5 to 7.

Medium priority: Incoming packets with an internal header CoS field in the range 2 to 4.

Low priority: All other packets (for example, with a CoS field in the range 0 to 1).

Fabric Complex

The fabric complex is the switch fabric of the system. It is based on the well-known 2-Tbps fabric ASIC on the Supervisor 2T. Refer to the “Chassis Backplane” section for more details.

Note: Recall that the Catalyst 6880-X chassis provides four fabric channels per slot (and four to the base board).

Port Complex

The port complex is essentially based on the well-known WS-X6904-40G module when operating in 10Gigabit Ethernet mode, along with several important new and improved components. As such, it has the same basic design and layout and serves the same basic functions (Figure 13).

Below is a summary of the highlights:

16 SFP+ (multirate) Ethernet ports

- The WS-X6904-40G supports four CFP-4SFP-10G (FourX) plus 16 SFP+ ports, capable of 1Gigabit Ethernet and 10Gigabit Ethernet.

- These are native SFP+ ports, capable of 10/100/1000, 1Gigabit Ethernet, and 10Gigabit Ethernet (they are 40Gigabit Ethernet capable via an adapter).

80-Gbps connection to switch fabric (2:1)

- This is the same as the WS-X6904-40G, with 80Gbps on the backplane and 160Gbps on the front panel; hence it is 2:1 oversubscribed if all ports are used at the line rate.

- It supports a “performance mode” configuration that will disable the second half (eight) of the ports, per-port ASIC, and enable 1:1 throughput.

Enhanced DFC4-E forwarding engine

- This is the first Catalyst 6800 module to use an enhanced DFC4-E (EARL8.5), which combines the previous Layer 2 and Layer 3 forwarding engine ASICs, improves read/write bandwidth, and includes various other fixes.

Improved 40-Gbps fabric interface and replication engines

- This is the first Catalyst 6800 module to use an enhanced Fabric Interface and Replication Engine (FIRE) ASIC, which combines the previous fabric and FIRE ASIC, improves read/write bandwidth, and includes various other fixes.

New 40+Gbps port interface MUX FPGA

- This is a new FPGA that will offload many of the traditional Port ASIC functions (such as packet buffering, links to the CPU, and links to the FIRE ASIC).

- It provides high-speed interfaces: Reduced-latency DRAM (RLDRAM3) buffer memory, four 16-Gbps uplinks to the FIRE ASIC, two 20-Gbps downlinks from the FIRE ASIC, one 40-Gbps downlink to the Port ASIC, and 1Gbps to the CPU.

- The new FPGA introduces a new capability to generate frame headers (encapsulation/decapsulation), such as BFD, NDE, and IPSLA, in hardware without the intervention of the CPU.

- The new FPGA also supports 256-bit AES encryption.

Gigabit Ethernet links to RP CPU and peer MUX FPGA

- These provide multiple 1-Gbps full-duplex control-plane paths to the RP CPU, as well as between separate MUX FPGAs.

- They are used to transmit data to and from the CPU, as well as for high-speed external control-plane programming, such as VSS and fabric extender (FEX).

New RLDRAM3 packet buffers on MUX FPGA

- Reduced-latency DRAM (RLDRAM) is faster than SRAM. RLDRAM3 is considered equivalent to or faster than DDR3 SDRAM: http://en.wikipedia.org/wiki/RLDRAM.

- The new MUX FPGA also supports increased read/write bandwidth to support external high-speed RLDRAM3 packet buffers.

192-MB (1536-Mb) packet buffers per MUX FPGA

- 24MB per 10Gigabit Ethernetport in 2:1 oversubscription mode.

- 48MB per 10Gigabit Ethernetport in 1:1 performance mode.

Figure 13. Port Complex Block Diagram

Port Card Details

The new Catalyst 6880-X chassis is technically on a fixed platform, but it has been classified as extensible due to its innovative semi-modular design. The extensible (or modular) aspect of the new chassis is the four separate port card slots and associated port cards.

Port Card Models

The Catalyst 6880-X supports two scalability models of port cards (comparable to the two scalability models of the Catalyst 6900 Series modules), based on the DFC4-E forwarding engine.

These are equivalent to the C6880-X (Heavy) and C6880-X-LE (Lite) scalability models of the baseboard.

Note: The Catalyst 6880-X uses the same interoperation model as the Catalyst 6500 and Supervisor 2T/PFC4 and LAN modules with DFC4. If both X and LE port cards are installed, the system will operate at the lowest common denominator (that is, LE).

16-Port SFP Port Card Components

The Catalyst 6880-X supports a 16-port SFP (multirate) port card (Figure 14).

The term “multirate” has been used here to describe the flexible SFP-based PHY component. This new PHY is capable of supporting 10/100/1000 BASE-TX copper, 1Gigabit Ethernet SFP, and 10Gigabit Ethernet SFP+ fiber (and later, 40Gigabit Ethernet via a special SFP-to-QSFP adapter).

Figure 14. Catalyst 6880-X Port Card

LCP Complex

This is the line card processor (LCP) of the port card. It is based on the LCP on the WS-X6904-40G module, along with several important new and improved components. As such, it has the same basic design and layout and serves the same basic functions.

Below is a summary of the highlights:

New 1.2-GHz X86 dual-core CPU

- A new Freescale P2010 CPU running at 1.2GHz.

1 or 2GB of 1337-MHz DDR3 ECC SDRAM

- DDR3 can transfer data at twice the rate of DDR2: http://en.wikipedia.org/wiki/DDR3.

Compact flash replaced with eUSB

- Internal eUSB flash will support 8GB of storage.

- The eUSB design provides SLC NAND flash with a sequential read of up to 21 MBps, and write of up to 18 MBps.

New switched EOBC interface

- Supports the existing EOBC architecture, but uses a P2P switched design for dedicated bandwidth.

Port Complex

The port complex is essentially based on the well-known WS-X6904-40G module when operating in 10Gigabit Ethernet mode, along with several important new and improved components. As such, it has the same basic design and layout and serves the same basic functions (Figure 14).

Below is a summary of the highlights:

16 SFP+ (multirate) Ethernet ports

- The WS-X6904-40G supports four CFP-4SFP-10G (FourX) plus 16 SFP+ ports, capable of 1Gigabit Ethernetand 10Gigabit Ethernet.

- These are native SFP+ ports, capable of 10/100/1000, 1Gigabit Ethernet, and 10Gigabit Ethernet (40Gigabit Ethernet capable via an adapter).

80-Gbps connection to switch fabric (2:1)

- This is the same as the WS-X6904-40G, with 80Gbps on the backplane and 160Gbps on the front panel; hence it is 2:1 oversubscribed if all ports are used at the line rate.

- Supports a “performance mode” configuration that will disable the second half (eight) of the ports, per the Port ASIC, and enable 1:1 throughput.

Enhanced DFC4-E forwarding engine

- This is the first Catalyst 6800 module to use an enhanced DFC4-E (EARL8.5), which combines the previous Layer 2 and Layer 3 forwarding engine ASICs, improves read/write bandwidth, and includes various other fixes.

Improved 40-Gbps fabric interface and replication engines

- This is the first Catalyst 6800 module to use an enhanced FIRE ASIC, which combines the previous fabric and FIRE ASIC, improves read/write bandwidth, and includes various other fixes.

New 40+Gbps port interface MUX FPGA

- This is a new FPGA that will offload many of the traditional Port ASIC functions (such as packet buffering, links to the CPU, and links to the FIRE ASIC).

- It provides high-speed interfaces: RLDRAM3 buffer memory, four 16-Gbps uplinks to the FIRE ASIC, two 20-Gbps downlinks from the FIRE ASIC, one 40Gigabit Ethernet downlink to the Port ASIC, and 1Gbps to the CPU.

- The new FPGA introduces a new capability to generate frame headers (encapsulation/decapsulation), such as BFD, NDE, and IPSLA, in hardware without the intervention of the CPU.

- The new FPGA also supports 256-bit AES encryption.

1Gigabit Ethernet links to the RP CPU and Peer MUX FPGA

- These provide multiple 1-Gbps full-duplex control-plane paths to the RP CPU, as well as between separate MUX FPGAs.

- They are used to transmit data to and from the CPU, as well as for high-speed external control-plane programming, such as VSS and FEX.

New RLDRAM3 packet buffers on MUX FPGA

- RLDRAM is faster than SRAM. RLDRAM3 is considered equivalent to or faster than DDR3 SDRAM: http://en.wikipedia.org/wiki/RLDRAM

- The new MUX FPGA also supports increased read/write bandwidth to support external high-speed RLDRAM3 packet buffers.

192-MB (1536-Mb) packet buffers per MUX FPGA

- 24MB per 10Gigabit Ethernetport in 2:1 oversubscription mode

- 48MB per 10Gigabit Ethernetport in 1:1 performance mode

Figure 15. 16-Port SFP Port Card Block Diagram

Packet Walks

This section provides a high-level overview of how packet forwarding is performed on the Catalyst 6880-X baseboard port complex and the 16-port SFP port card.

Since these are architecturally equivalent (excluding the RP and LCP complexes, and assuming the same fabric complex), we need to use only a single example.

Ingress Forwarding

The following is the basic sequence of events when packets enter the Catalyst 6880-X ports:

1. Packet arrives at ingress port.

2. PHY converts the signal and serializes the bits, and then sends the packet to the Port ASIC.

3. Port ASIC parses the packet to derive the VLAN, CoS, etc. and performs ingress QoS. Then it applies an internal header and sends it to the MUX ASIC.

4. MUX ASIC stores the packet in the local buffer and may perform special encapsulation/decapsulation. Then it sends it to one of two FIRE ASICs.

5. FIRE ASIC stores the data payload in the local buffer and then sends only the internal header to the forwarding engine (for lookup).

6. Inband FPGA parses the internal header and then sends it to the forwarding engine.

7. FIRE ASIC uses the new internal header to determine the fabric port mapped to the egress port, and converts the internal header to a fabric header. It then sends it to the fabric ASIC.

8. Forwarding Engine performs L2, L3, ACL and Netflow IFE & OFE processing and determines the Egress Port & Rewrite Info. Then it returns new Internal Header to FIRE ASIC (via Inband FPGA)

9. Fabric ASIC uses the fabric header to determine the egress fabric port and then sends the packet to the fabric complex.

Figure 16 shows a visual representation of the ingress packet-forwarding process.

Figure 16. 16-SFP Packet Walk at Ingress

Egress Forwarding

The following is the basic sequence of events when packets exit the Catalyst 6880-X ports:

1. Fabric complex transmits the frame to the egress fabric port, where it is received by the egress FIRE ASIC.

2. FIRE ASIC uses the fabric header to derive a new internal header, which it sends to the forwarding engine (egress lookup).

3. Inband FPGA parses the internal header and then sends it to the forwarding engine.

4. Forwarding engine performs an egress (Layer 2) lookup to learn the source MAC address. It then returns the internal header to the FIRE ASIC (via the inband FPGA).

5. FIRE ASIC uses the new internal header to determine the egress port and reassemble the packet, and then sends it to the MUX ASIC.

6. MUX ASIC uses the internal header to determine the egress port and perform egress QoS. It then sends it to the Port ASIC.

7. Port ASIC removes the internal header and rewrites VLAN, CoS, etc. It then sends it to PHY.

8. PHY serializes the bits and converts the signal and then transmits the packet.

9. Packet leaves at the egress port.

Figure 17 shows a visual representation of the egress packet forwarding process:

Figure 17. 16-Port SFP Packet Walk at Egress

Conclusion

Cisco is extending the scale, performance, and capabilities of the venerable Cisco Catalyst 6000 Series with the introduction of the new Catalyst 6800 Series. The new Catalyst 6880-X chassis provides extremely high levels of scalability and performance, with the size and economics of an innovative extensible fixed architecture.

The new Catalyst 6880-X chassis is capable of delivering up to 220Gbps of per-slot bandwidth. This translates to a system capacity of up to approximately 2 Tbps.

For More Information

Catalyst 6500 Supervisor 2T Architecture whitepaper

Catalyst 6500 WS-X6904-40G Architecture whitepaper

Catalyst 6500 Supervisor 2T data sheet

Catalyst 6500 WS-X6904-40G data sheet