Many hardware problems encountered during field installations or during
normal operation can be prevented by a thorough product overview ahead of time.
For Catalyst 5000 series switches, this means being able to identify the switch
chassis, Supervisor Engine and switching modules, and understanding system
specifications, cabling, and power requirements. Basic knowledge of switch
architecture is helpful, as is an understanding of software considerations.
Therefore, for those customers not already familiar with the Catalyst 5000
series, it is not required but highly
recommended that you complete this document before proceeding to
Troubleshooting for Catalyst 5500/5000/2926G/2926 Series
These major topics are covered:
Product Overview - an introduction to Catalyst
5500/5000/2926 series switches with links to detailed chassis, Supervisor
Engine, and switching module documentation.
Switch Management - establishing console and Telnet
connections and working with configuration files.
Software Considerations - software versions and
Basic Architecture Overview - understanding the
differences and similarities between various Catalyst switch architectures and
how frames are forwarded.
The following topics are not covered:
There are no specific requirements for this document.
This document is not restricted to specific software and hardware
For more information on document conventions, refer to the
Technical Tips Conventions.
The Catalyst 5000 series features five modular chassis: Catalyst 5000,
5002, 5505, 5509, and 5500 (13-slot chassis). The 2900/2926 series has six
non-modular or fixed-configuration switches: the 2901, 2902, 2926T, 2926F,
2926GS, and 2926GL, which are in the same switch series as the 5000/5500 series
and share the same architecture. A good reference for this subject is the
Catalyst 2900 Platforms section of the document
Catalyst 2900 and Catalyst 4000 Naming Conventions.
There are a number of Supervisor Engine, feature card, and uplink
module options in the Catalyst 5000 series. Some have internal, non-replaceable
power supplies, while others have external, removable power supplies. All
Catalyst 5000/5500/2900/2926 Supervisor Engines have a centralized processing
engine called the Encoded Address Recognition Logic (EARL) for learning and
forwarding packets based on MAC addresses, but each Supervisor Engine has a
different version of the EARL. For the 5000/5500 series, certain Supervisor
Engines can only be used in certain switch chassis, and there are requirements
for redundant Supervisor Engines. The information below helps you understand
your switch and its capabilities.
Note: Different versions of Catalyst 5000 Supervisor Engines have different
versions of the centralized processing engines. The discovery of an IEEE 802.1x
security vulnerability makes it important to understand which version of EARL
your Supervisor Engine has. Refer to the document
Catalyst 5000 EARL Version and Other Common EARL Questions for more
Refer to the document
5000 Family Installation Guide (May 2000). Cisco Technical Support
recommends reviewing at least the following:
Start by referring to the
Overview, which describes the features and functionality of the Catalyst
5000 series switches—specifically, which chassis supports which Supervisor
Planning, which talks about site power and environmental
which lists the Catalyst 5000 series system specifications in
Refer to the document
2900 User Guide. Cisco Technical Support recommends reviewing
at least the following:
Refer to the document
2926 and 2926G Series Installation Guide. Cisco Technical Support
recommends reviewing at least the following:
Start by referring to the
Overview, which describes the features and functionality of this
Description, which gives a more specific overview of system features and
Specifications, which lists Catalyst 2926 series system specifications
Refer to the document
5000 Supervisor Engine Install Guide (September 2000). Cisco Technical
Support recommends reviewing at least the
Start by referring to the
Overview, which describes the Catalyst 5000 series Supervisor Engine
models. Pay particular attention to the section Supervisor Engine
Redundancy, which lists the requirements for redundant Supervisor
for Installation, which talks about maximum distances for a variety of
the Supervisor Engine, which discusses proper insertion and removal of
Supervisor Engines, modules, and feature
Refer to the document
5000 Family Module Install Guide (January 2000). Cisco Technical Support
recommends reviewing at least the following:
5000 Family Installation & Configuration Notes. Cisco Technical
Support recommends you review the relevant information for your specific
Supervisor Engine module or line card, as time permits.
"Switch management" in this document describes configuration
management, flash file system management, syslogging, and Simple Network
Management Protocol (SNMP). This is one of the most critical elements in
preparing to troubleshoot hardware. Simply put: have a backup plan in place.
Back up your configuration on a regular basis, save passwords in a secure
location, and log system messages either manually or automatically to a server.
This principle applies to your network in general.
Take these steps in preparation for possible failures:
Cisco Technical Support highly recommends reading
Practices for Catalyst 4000, 5000 and 6000 Series Switch Configuration and
Management, which represents years of field experience from Cisco
engineers working with many of their largest customers and complex
Customers frequently have questions regarding software for Catalyst
4000, 5000, and 6500/6000 switches. Each Catalyst 4000, 5000, and 6500/6000
software release is a member of a release train (Catalyst 2900/2926 series
switches run Catalyst 5000 software, as well). Part of troubleshooting hardware
is understanding the software release numbering scheme, the software train life
cycle, train types and definitions, time frames, and guidelines for
The following documents are accessed from the
Literature page for the Catalyst 5000 and are essential for answering
questions of this type:
Notice from the links above that 5.5(7) is currently General Deployment
(GD) for Catalyst 4000, 5000, and 6500/6000 switches in the 5.x train, while
4.5(4) is GD for the 4.x train.
For new features, memory requirements, caveats (bugs), or upgrade paths
from early versions to later versions, always check the release notes.
Depending on the switch, the release notes may also contain a condensed list of
Supervisor Engine requirements and redundancy information.
For Catalyst 5000 series switches, refer to
Engine Module Software Release Notes. Check the release notes for the
version of software you are currently running or to which you plan to upgrade.
Pay particular attention to the sections Memory
Requirements, Software Upgrade Path and
Supervisor Engine Support Matrix.
For example, in
Notes for Catalyst 5000 Family Software Release 5.x, check the sections
Release 5.x Memory Requirements, Release 5.x
Software Upgrade Path and Release 5.x Supervisor Engine
Support Matrix. If you had a Supervisor Engine I and were planning
to upgrade, you would see that a Supervisor Engine I for the Catalyst 5000 does
not support 5.x. You could then check the
Notes for Catalyst 5000 Family Software Release 4.x and determine that a
Supervisor Engine I is supported in 4.x.
Note: The Catalyst 2900/2926 runs Catalyst 5000 software, as mentioned
above in Software Considerations. The latest
release supported by the Catalyst 2900/2926 series is 4.x. Only the Catalyst
2926Gs supports 5.x. None of the 2900/2926 series switches support 6.x.
The next section provides a basic overview of system architecture in
preparation for detailed troubleshooting of Catalyst 5000 series
The Catalyst 5000 series features five modular chassis: Catalyst 5002,
5000, 5505, 5500 (13-slot chassis), and 5509. The 2900/2926 series shares the
The Catalyst 5000 series architecture has three basic components: the
bus arbiter and EARL, the port interface, and the Network Management Processor
(NMP). The bus arbiter and EARL (used by all ports) govern access to the
data-switching bus and control packet transfer destination. Each Ethernet port
interface comprises a custom application-specific integrated circuit (ASIC)
called a SAINT (Synergy Advanced Interface and Network Termination) that has an
integrated 10/100 Mbps Ethernet MAC controller. Other media ports make use of a
second custom ASIC called a SAGE (Synergy Advanced Gate Array Engine) that does
not have an integrated MAC controller.
The switch uses central bus arbitration and address recognition logic.
Each port fully buffers each frame received from the network before issuing a
request to the central bus arbiter of the switch to access the switching bus to
transmit its stored frames. When given the go-ahead, the port forwards the
frame across the backplane, and all ports on all modules, as well as the
Supervisor Engine, receive a copy of it. The central EARL of the Supervisor
Engine determines the destination ports that should forward the frame and
instructs each port to either flush or forward the frame as appropriate.
Each frame traversing the switching bus may be destined to a single
port or to multiple ports. Ensuring that every port gets a copy of each frame,
and then using the "flush or forward" model, allows for high-speed multicast
and broadcast forwarding without the need for frame copies. Frames cross the
backplane just one time, thereby optimizing performance. The switching bus
operates at 1.2 Gbps and resides on the backplane with interfaces to each line
The Catalyst 5000 series switch architecture uses a store-and-forward
model for input and output. Each switch port maintains its own frame buffer
memory. Each frame is stored in a frame buffer before it is forwarded.
Using the bus arbiter, the bus supports a three-level priority request
scheme. The bus also allows each port to perform a local flush and maintains a
packet retry mechanism used during outbound port congestion. The Catalyst 5000
architecture allows media-rate performance for all traffic types (unicast,
broadcast, and multicast).
In addition to the backbone switching bus, the Catalyst switch
maintains the management bus and the index bus.
The management bus carries configuration information from the NMP to
each module and statistical information from each module to the NMP. The index
bus carries port-select information from the central EARL to the ports. This
information determines which ports forward the packet and which flush it from
The Supervisor Engine II can forward up to 1 million packets per second
(Mpps), and the Supervisor Engine III can forward up to 2 to 3 Mpps with the
NetFlow feature card. The NetFlow feature card is a modular feature card
upgrade to the Supervisor Engine that provides Layer 3 switching in hardware
for the Catalyst 5000 series switches.
The Supervisor Engine controls data access to the backplane,
prioritizes the interfaces, maintains up to 16,000 active MAC addresses in the
bridge lookup table, monitors system status, delivers configuration information
for all modules, gathers performance information, and updates operational
software information. The Supervisor Engine also supports VLANs on a
network-wide basis using any Fast Ethernet (with Inter-Switch Link Protocol
[ISL] or 802.1Q), FDDI (with 802.10), or ATM (with LAN Emulation [LANE])
interface between multiple Catalyst 5000 switches and Cisco routers.
The NMP on the Supervisor Engine uses system software and performs
system control and configuration, including the loading of run-time code to the
line modules and system diagnostics. The NMP executes a separate instance of
the Spanning Tree Protocol (STP) on each VLAN and runs a wide range of other
switch protocols, as well as VLAN Trunking Protocol (VTP), Cisco Discovery
Protocol (CDP), and others. The NMP is also responsible for controlling SNMP,
Telnet, and console access to the switch.
The primary function of the master communication processor (MCP) on the
Supervisor Engine is to communicate information between the NMP and the line
module communication processors (LCPs) distributed on Catalyst 5000 line
modules. This intermodule communication occurs across the management bus, which
is a serial bus.
Each LCP processes information sent to it by the MCP across the
management bus. The Serial Control Protocol (SCP) is an internal communications
protocol used for this communication between modules. The LCP runs diagnostics
on all the local module hardware upon power up or reset of the line
The switching bus is 48 bits wide and operates at 25 MHz, which gives a
data-transfer rate of 1.2 Gbps. A bus access arbitration scheme is implemented
on the Supervisor Engine. All line modules and the Supervisor Engine have
access to the switching bus. The Catalyst 5500 employs three such buses. The
Supervisor Engine II ties these buses together as a single data bus. The
Supervisor Engine III enables independent 1.2-Gbps input buses to be bridged
together by Phoenix ASICs, which are described below in the section
PHOENIX ASIC on Supervisor Engine III.
Switching bus encapsulations are added and removed by the SAINT and
Each dedicated Ethernet and Fast Ethernet switch port has its own SAINT
ASIC and 192 KB of dedicated frame buffer. ATM and FDDI MAC layers are not IEEE
802.3-compliant, so these interfaces use the SAGE (non-Ethernet) ASIC and 1 to
2 MB of additional buffering.
The SAINT ASIC is part of the port interface that also includes a
dual-channel direct memory access (DMA) controller for getting packets in and
out of the buffer and onto and off the switching backplane. The SAINT is a
highly integrated, high-performance, custom 10/100 Ethernet MAC-compliant (IEEE
802.3) ASIC. It has built-in support for 10-Mbps Ethernet, 4B/5B Fast Ethernet,
and the media-independent interface (MII). It permits the use of half- and
full-duplex Ethernet and two priority request levels. (A third priority, which
is hardware-initiated, is reserved to prevent per-port buffer overflow
conditions. The hardware-initiated priority has the highest priority and is set
when the use of the input buffer exceeds 90 percent). SAINT also performs ISL
and 802.1Q encapsulation and de-encapsulation on frames outbound from ports
configured as trunks.
The external packet buffer stores data coming from the network before
it is forwarded to the switching bus. Likewise, it is used to store data coming
from the switching bus before it is forwarded to the network. The packet buffer
is divided into two sections, one for packets going to the network (the output,
or transmit buffer), and the other for packets going to the switching bus (the
input, or receive buffer). The switching bus is very efficient in processing
the incoming packets. The output buffer, therefore, is larger than the input
buffer because it is more likely that the switch needs more output buffer than
As a frame is received from the network and stored in the frame buffer
of the port, ASICs on each port encapsulate Ethernet frames with 12 bytes of
information to indicate VLAN ID (color), the port of origin for the frame, and
a frame check sequence (FCS). This FCS is checked at the port receiving the
frame from the switching bus. The same ASIC on each port also strips off that
encapsulated information and extracts the Ethernet frame before sending it out
the destination ports. This encapsulation is different from the 30-byte ISL
encapsulation. Ports configured as ISL trunks also strip off the 12-byte
encapsulation used across the Catalyst 5000 series switch backplane before
forwarding the frame to the network. In addition, they add a 30-byte header on
the outbound frame.
The SAGE ASIC design is similar to that of the SAINT, but without the
10/100-Mbps Ethernet MAC-layer functionality. It is used for non-Ethernet
applications, such as the FDDI module, ATM LANE module, route switch module,
Token Ring, and the NMP on the Supervisor Engine.
The 48-port Ethernet switching module for the Catalyst 5000 series
switch uses a new ASIC called Bodega. Each Bodega ASIC supports 24 10-Mbps
Ethernet ports. The Bodega ASIC uses a shared-memory buffer, and buffer space
is allocated dynamically according to packet size. The Bodega ASIC helps reduce
the cost of high-performance, switched networks.
The SAMBA ASIC is located on both line modules and the Supervisor
Engines. It provides arbitration to the switching bus both among the ports and
among the line modules in the chassis. Its dual usage is accomplished by
strapping on external device pins.
The SAMBA can be strapped in either master mode or slave mode. The
SAMBA in master mode is located on the Supervisor Engine, and the SAMBA in
slave mode is on the line module and the Fast Ethernet ports on the Supervisor
Engine. A master can support up to 13 line cards and a slave can support up to
48 ports on a single device.
An arbitration process starts with ports on the line cards requesting
access to the bus through the slave SAMBA, which forwards the requests to the
master SAMBA on the Supervisor Engine. The slave SAMBA then waits for
permission from the master SAMBA before issuing grants to the ports.
Hardware broadcast/multicast suppression capability is available when
SAMBA is in slave mode. This capability allows SAMBA to monitor the number of
broadcast packets going through every port on the line card and removes the
entire broadcast packet if the total number of broadcast packet words reaches a
threshold within a given time period. The threshold number and the time period
are both initialized by the CPU.
The EARL is a custom Catalyst 5000 series switch component that is
similar in its function to the learning bridges or content-addressable memory
(CAM) used on other systems. The EARL automatically learns source MAC addresses
and associated VLANs, and saves them in a RAM address table with VLAN and port
information. You can also configure static entries in the EARL table. The EARL
stores up to 16,000 addresses.
The EARL then uses the learned entries to perform lookup operations on
destination addresses (DAs) to get VLAN and port information to be used to
direct the frame. It stores these addresses for 300 seconds (5 minutes) by
default, or 60 to 1200 seconds (1 to 20 minutes), if so configured by the
Local target logic (LTL) on each line module helps the EARL to find the
destination port(s) for each frame. Index values generated by the EARL can
select a single port, multiple ports, or all ports in a VLAN. LTL memory is
segmented to support unicast, multicast, and flood cases.
The color blocking logic (CBL) blocks traffic that is part of a VLAN
different from that configured on a port from coming into or going out of that
port and assists spanning tree in blocking ports to prevent loops.
The PHOENIX gate array allows you to continue to use all the Catalyst
5000 switch modules, while also providing a migration path to switching modules
that are capable of 3.6-Gbps switching. It provides intelligent forwarding of
packet data from one bus to another, which allows the Catalyst 5500 switch to
increase the bandwidth support to 3.6 Gbps by interconnecting three 1.2-Gbps
The PHOENIX ASICs on the Supervisor Engine III are a gigabit bridge
used to create the crossbar fabric interconnect between the three 1.2-Gbps
buses. Most Catalyst 5000 series switch modules connect only to one bus in the
fabric. However, some modules (such as the Supervisor Engine III itself)
connect to all three buses.
To understand how the major components that comprise the architecture
of the Catalyst 5000 series of switches work, you need to follow the path of an
Ethernet frame through the switch. When an Ethernet frame arrives at a port,
the port DMA controller stores it in its receive buffer. The port fully
receives, buffers, and checks the FCS for the frame, adds the 12-byte frame
header containing the port number and VLAN number, and increments the Ethernet
Management Information Base (MIB) and Remote Monitoring (RMON) counters before
storing the frame in the frame buffer and requesting access to the switching
bus to forward the frame.
After the entire frame is received and stored in the frame buffer, the
SAINT ASIC posts a request to the bus arbiter to transmit the frame across the
high-speed switching bus. The Supervisor Engine performs the necessary bus
arbitration and grants bus access to the line module as soon as it becomes
available, using a round-robin method. The local arbiter on the line module
then uses a round-robin method to grant each port permission to transmit one
frame at the priority level being serviced.
The bus arbiter issues a grant to the local arbiter, which, at the
appropriate time, signals the SAINT ASIC to initiate the data transfer. The
frame that is stored in the buffer is then transmitted across the high-speed
switching backplane by the DMA controller.
As the frame is transmitted across the switching backplane, all ports
receive the frame and store it in their frame output
As with all bus architectures, because all ports on the bus receive the
frame, there is no need for multiple copies of broadcast and multicast
The last step in the frame flow process varies, depending on these
Unicast and Multicast
When the EARL, in conjunction with LTL and CBL, identifies a specific
port or set of ports as the destination, it instructs the SAINT ASICs on the
destination ports through the control bus to continue to receive the frame and
subsequently transmit it out its port. The SAINT ASICs on the other ports flush
Broadcasts and Unknown Destination Addresses
If the destination address is unknown to the EARL, or is a broadcast
or multicast address (where multicasting is not configured), the frames are
forwarded to all ports on the same VLAN as the source
If the EARL determines that the frame does not require switching into
the system, it issues a flush to terminate frame transfer on the bus from the
source interface and causes all ports to flush their buffers. However, only a
few bits of the packet are stored in each buffer before the EARL orders the
flush. The EARL then flushes the buffer of only that packet.
If the destination interface has no buffer to accommodate a frame, it
issues a retry to the source. The source terminates the frame transfer and
retries the frame at a later time.
The EARL Plus (EARL+) is a logic based on the original EARL used for
Ethernet switching. The Plus part of EARL is the added logic required to
perform Token Ring switching in the Catalyst 5000. These EARL changes are
required by ISL for Token Ring and by the Token Ring card for the Catalyst
As data traverses the switching bus of the Catalyst 5000 backplane,
EARL+ monitors the type field of the switching bus header. If the packet is not
of type B"0000", then EARL performs its current Ethernet switching function. If
the packet type is B"0001," then EARL+ latches data from an additional
switching bus header word (48 bits) added for Token Ring.
The SAGE and SAINT ASICs take the 48-bit MAC address along with the 10
bits of VLAN information and encode them into 15 bits. The 15 bits are
presented to EARL and used to access a lookup table that contains the actual
MAC address and its associated port index and VLAN information.
Although Token Ring addresses are noncanonical and Ethernet addresses
are canonical, there can never be a match conflict within the EARL+ lookup
tables because a Token Ring address never has the same VLAN ID as an Ethernet
address. The VLAN ID acts as an additional qualifier for the EARL+