Catalyst 6500 Series Software Configuration Guide, 8.7
Configuring the Switch Fabric Module
Download this chapter
Configuring the Switch Fabric ModuleConfiguring the Switch Fabric Module
give_us_feedback

Table Of Contents

Configuring the Switch Fabric Modules

Understanding How the Integrated 720-Gbps Switch Fabric Works

Understanding How the External Switch Fabric Module Works

Forwarding Modes

Configuring and Monitoring the Integrated Switch Fabric and Switch Fabric Module on the Switch

Configuring a Fallback Option

Configuring the Switching Mode

Redundancy

Monitoring the Integrated Switch Fabric and Switch Fabric Module

Displaying the Module Information

Displaying the Fabric Channel Counters

Displaying the Fabric Channel Switching Mode and Channel Status

Displaying the Fabric Channel Utilization

Displaying the Fabric Errors

Displaying the Backplane Traffic and Fabric Channel Input and Output

Displaying the Switching Mode Configuration

Displaying the Integrated Switch Fabric Status

Configuring the LCD Banner


Configuring the Switch Fabric Modules

This chapter describes the integrated 720-Gbps switch fabric that is supported on Supervisor Engine 720 and the external Switch Fabric Module (WS-C6500-SFM) and Switch Fabric Module 2 (WS-X6500-SFM 2) that is supported on the Supervisor Engine 2 on the Catalyst 6500 series switches.

Note For complete syntax and usage information for the commands that are used in this chapter, refer to the Catalyst 6500 Series Switch Command Reference publication.

This chapter uses the following terminology:

•CEF720—Any module that has a part number that conforms to WS-X67xx-xxx (such as WS-X6724-SFP). These modules connect to the integrated 720-Gbps switch fabric on the Supervisor Engine 720.

•CEF256—Any module that has a part number that conforms to WS-X65xx-xxx (such as WS-X6548-GE-TX), the Optical Services Modules, the enhanced FlexWAN module, and most service modules, such as the Firewall Services Module (FWSM), the Secure Socket Layer Services Module (SSLM), the Virtual Private Network Services Module (VPNSM), the Network Analysis Module 1 (NAM-1), the NAM-2, the Intrusion Detection System Module (IDSM-2), the Content Services Gateway (CSG), and the Communications Media Module (CMM). These modules connect to either the integrated 720-Gbps switch fabric on the Supervisor Engine 720 or to the external 256-Gbps Switch Fabric Modules that are supported by the Supervisor Engine 2, and they connect to the 32-Gbps switching bus.

•Non-fabric-enabled—Any module that does not fall into the CEF720 or CEF256 categories. These modules have no fabric connections and connect only to the 32-Gbps switching bus.

This chapter consists of these sections:

•Understanding How the Integrated 720-Gbps Switch Fabric Works

•Understanding How the External Switch Fabric Module Works

•Forwarding Modes

•Configuring and Monitoring the Integrated Switch Fabric and Switch Fabric Module on the Switch

Note The WS-C6500-SFM is supported in the Catalyst 6500 series 6-and 9-slot chassis only. The WS-X6500-SFM 2 is supported in the Catalyst 6500 series 6-slot, 9-slot, and 13-slot chassis and in the Catalyst 6509-NEB chassis.

Understanding How the Integrated 720-Gbps Switch Fabric Works

Note The integrated 720-Gbps switch fabric is supported only on Supervisor Engine 720.

Note With software release 8.3(1) and later releases, in redundant systems, the integrated 720-Gbps switch fabric supports a high-availability failover to the standby switch fabric. High availability must be enabled for the failover to work (enter the set system highavailability enable command).

The integrated 720-Gbps switch fabric, which is built into Supervisor Engine 720, creates a dedicated connection between the CEF256 and CEF720 modules and provides an uninterrupted transmission of frames between these modules. In addition to the direct connection between the CEF256 and CEF720 modules that is provided by the integrated 720-Gbps switch fabric, these modules have a direct connection to the 32-Gbps switching bus.

Understanding How the External Switch Fabric Module Works

Note The external Switch Fabric Modules are supported only with Supervisor Engine 2 in the Catalyst 6500 series switch.

The external Switch Fabric Modules create a dedicated connection between the CEF256 modules and provide an uninterrupted transmission of frames between these modules. The external Switch Fabric Modules do not support the CEF720 modules because these modules do not work with a Supervisor Engine 2. In addition to using the Switch Fabric Module, the CEF256 modules connect to the 32-Gbps switching bus. Table 54-1 lists the switch fabric connection speed for each module.

Table 54-1 Switch Fabric Connection Speed by Module Type

Switch Fabric Type
Module Type1
 

CEF256

CEF720

Switch Fabric Module (256-Gbps)

1 x 16-Gbps

Not supported

Switch Fabric Module 2 (256-Gbps)

1 x 16-Gbps

Not supported

Integrated switch fabric on Supervisor Engine 720 (720-Gbps)

1 x 16-Gbps

1 x 40-Gbps:
WS-X6724-SFP

2 x 40-Gbps:
WS-X6748-GE-TX WS-X6748-SFP WS-X6704-10GE

1 The speeds shown are bidirectional.


Enter the set system crossbar-fallback bus-mode | none command to specify how the packets are handled if the Switch Fabric Module is removed or fails. If you specify the bus-mode keyword, the switching is done in flow-through mode. If you specify the none keyword, the switch ports are disabled and switching stops. For more detailed information, see the "Configuring a Fallback Option" section.

The external Switch Fabric Module does not have a console. A two-line LCD display on the front panel shows fabric utilization, software revision, and basic system information.

Note The term 9-slot switch refers to the WS-C6509-NEB and WS-C6509-NEB-A switches.

Install the WS-C6500-SFM in either slot 5 or 6 in the 6-slot and 9-slot switches. Install the WS-X6500-SFM 2 in either slot 7 or 8 in the 13-slot switches and slots 5 or 6 in the 6-slot and 9-slot switches. The Switch Fabric Module that is first installed functions as the primary module. For redundancy, you can install a standby Switch Fabric Module.

When you install two Switch Fabric Modules at the same time in a 6- or 9-slot chassis, insert the primary module into slot 5 and the backup module into slot 6. If you reset the module in slot 5, the module in slot 6 becomes active.

When you install two Switch Fabric Modules at the same time in a 13-slot chassis, insert the primary module into slot 7 and the backup module into slot 8. If you reset the module in slot 7, the module in slot 8 becomes active.

Forwarding Modes

The CEF256/CEF720 modules operate in one of three modes when using centralized forwarding:

•Compact mode—Operational mode when all modules in the system are CEF256 or CEF720 (no non-fabric-enabled modules can be present for this mode). In this mode, the CEF256/CEF720 modules send a "compact" 32-byte header for each frame to the supervisor engine over the switching bus. Once a forwarding decision is made, the CEF256/CEF720 modules send the entire frame through the switch fabric to the egress module.

•Truncated mode—Operational mode when at least one non-fabric-enabled module is present in the system. In this mode, the CEF256/CEF720 modules send the first 64 bytes of each frame to the supervisor engine over the switching bus. Once a forwarding decision is made, the CEF256/CEF720 modules send the entire frame through the switch fabric to the egress module.

•Flow-through mode—Operational mode for the CEF256 modules when there is no switch fabric present. In this mode, the CEF256 modules send the entire packet to the supervisor engine over the switching bus. This mode is not applicable for the CEF720 modules, which require the presence of the switch fabric.

Table 54-2 shows the switch modes that are used with the CEF256, CEF720, and non-fabric-enabled modules installed.

Table 54-2 Switching Modes with Switch Fabric Module Installed

Types of Modules Installed
Switching Modes

CEF256 or CEF720 modules (no non-fabric-enabled modules are installed)

Compact

CEF256 and/or CEF720 modules (non-fabric-enabled modules are installed)

Truncated

CEF256 and non-fabric-enabled modules

Flow-through

Non-fabric-enabled modules

Flow-through


Configuring and Monitoring the Integrated Switch Fabric and Switch Fabric Module on the Switch

The integrated 720-Gbps switch fabric and the Switch Fabric Modules do not require any user configuration but support a number of show commands for monitoring purposes. A fully automated startup sequence brings the module online and runs the connectivity diagnostics on the ports.

From the supervisor engine, you can reset the module by entering the reset module command, disable and enable the module by entering the set module enable | disable command, and power down the module by entering the set module powerdown module command.

These sections describe how to configure the integrated 720-Gbps switch fabric and Switch Fabric Modules:

•Configuring a Fallback Option

•Configuring the Switching Mode

•Redundancy

•Monitoring the Integrated Switch Fabric and Switch Fabric Module

•Configuring the LCD Banner

Configuring a Fallback Option

The set system crossbar-fallback {bus-mode | none} command allows you to configure a fallback option if the Switch Fabric Module connection fails.

If a switch is in compact mode, setting the crossbar-fallback to none prohibits the switch from running in any other switching mode. Assuming there are no non-fabric-enabled modules in the switch, the switching mode will be compact. Then if the crossbar-fallback is set to none and a non-fabric- enabled module is inserted, the module will not be powered up because that would force the switch out of compact mode. Also, if crossbar-fallback is set to none and the last Switch Fabric Module is removed, ports on all modules will be disabled. If a port fails to become disabled, the module on which it resides will be powered down. With no Switch Fabric Module, the switch cannot run in compact mode so the switch is effectively disabled since crossbar-fallback is set to none.

To configure a fallback option for the Switch Fabric Module, perform this task in privileged mode:

Task
Command

Configure a fallback option for the Switch Fabric Module.

set system crossbar-fallback {bus-mode | none}


This example shows how to configure a fallback option to bus-mode: 

Console> (enable) set system crossbar-fallback bus-mode

System crossbar-fallback set to bus-mode.

Console> (enable)

Configuring the Switching Mode

To improve performance, you can manually specify the switching mode that the system uses. If you have one or more non-fabric-enabled modules that are installed in the chassis, configure the switch to use flow-through mode. If you have only the CEF256 or CEF720 modules that are installed in the chassis, configure the switch to use compact mode.

Note Non-fabric-enabled modules do not support compact mode.

Note If there is a combination of a Supervisor Engine 720 with switch-fabric capability and CEF720 modules in the chassis, the bus-only operation is not permitted. The system stays in truncated mode.

To configure the switch to use flow-through mode if you have the non-fabric-enabled modules installed, perform this task:

Task
Command

Configure the switch to use flow-through mode.

set system switchmode allow bus-only


This example shows how to configure the switch to use flow-through mode: 

Console> (enable) set system switchmode allow bus-only


Console> (enable)

To configure the switch to use truncated mode when CEF256 and/or CEF720 and non-fabric-enabled modules are installed, perform this task:

Task
Command

Configure the switch to use truncated mode.

set system switchmode allow truncated


This example shows how to configure the switch to use truncated mode: 

Console> (enable) set system switchmode allow truncated


Console> (enable)

Redundancy

No configuration is required for the Switch Fabric Module redundancy. The module in slot 5 functions as the primary module, and a redundant Switch Fabric Module in slot 6 automatically takes over if the primary module fails. The Catalyst 6506 and 6509 switches support a mixed redundant configuration with a WS-C6500-SFM and a WS-X6500-SFM 2. The Catalyst 6513 switch supports a redundant configuration with the WS-C6500-SFM2s only.

No configuration is required for the integrated 720-Gbps switch fabric redundancy. The integrated switch fabric in the active Supervisor Engine 720 functions as the primary switch fabric. A supervisor engine switchover will also cause a switch fabric switchover.

Monitoring the Integrated Switch Fabric and Switch Fabric Module

This section describes how to monitor the integrated switch fabric and the Switch Fabric Module:

•Displaying the Module Information

•Displaying the Fabric Channel Counters

•Displaying the Fabric Channel Switching Mode and Channel Status

•Displaying the Fabric Channel Utilization

•Displaying the Fabric Errors

•Displaying the Backplane Traffic and Fabric Channel Input and Output

•Displaying the Switching Mode Configuration

•Displaying the Integrated Switch Fabric Status

Note Enter all the show commands that are supported by the integrated 720-Gbps switch fabric and the Switch Fabric Modules from the supervisor engine.

Displaying the Module Information

To display the module information, perform this task:

Task
Command

Display the module information.

show module mod1

1 The show module command is not supported for the integrated switch fabric.


This example shows how to display the module information: 

Console> show module

Mod Slot Ports Module-Type               Model               Sub Status

--- ---- ----- ------------------------- ------------------- --- --------

1   1    2     1000BaseX Supervisor      WS-X6K-SUP2-2GE     yes ok

4   4    24    100BaseFX MM Ethernet     WS-X6224-MM-MT      no  ok

5   5    0     Switch Fabric Module      WS-C6500-SFM        no  ok


Mod Module-Name         Serial-Num

--- ------------------- -----------

1                       Munish     

4                       SAD02390156

5                       SAD042818BR


Mod MAC-Address(es)                        Hw     Fw         Sw

--- -------------------------------------- ------ ---------- -----------------

1   00-40-0b-ff-00-00 to 00-40-0b-ff-00-01 0.219  6.1(0.146) 6.2(0.33-Eng)KEY

    00-50-3e-7e-71-56 to 00-50-3e-7e-71-57

    00-01-64-f8-ca-00 to 00-01-64-f8-cd-ff

4   00-10-7b-c2-3a-c0 to 00-10-7b-c2-3a-d7 0.204  4.2(0.24)V 6.2(0.14)KEY

5   00-40-0b-ff-00-00                      0.204  6.1(0.133) 6.2(0.14)KEY


Mod Sub-Type                Sub-Model           Sub-Serial  Sub-Hw

--- ----------------------- ------------------- ----------- ------

1   L3 Switching Engine II  WS-F6K-PFC2         SAD04110B5S 0.305

Console> (enable)

Displaying the Fabric Channel Counters

To display the fabric channel counters, perform this task:

Task
Command

Display the fabric channel counters.

show fabric channel counters {mod | all} [hex]


This example shows how to display the fabric channel counters: 

Console> show fabric channel counters 5

Channel 0 counters:

0  rxTotalPkts                        =                    0

1  txTotalPkts                        =                    0

2  rxGoodPkts                         =                    0

3  rxErrors                           =                    0

4  txErrors                           =                    0

5  txDropped                          =                    0

Displaying the Fabric Channel Switching Mode and Channel Status

To display the fabric channel switching mode and channel status, perform this task:

Task
Command

Display the fabric channel switching mode and channel status.

show fabric channel switchmode [mod]


This example shows how to display the fabric channel switching mode and channel status: 

Console> show fabric channel switchmode 

Global switching mode:truncated


Module Num Fab Chan Fab Chan Switch Mode  Channel Status

------ ------------ -------- ------------ --------------

     1            1   0, 0   flow through ok            

     4            0 n/a      n/a          n/a           

     5           18   0, 0   n/a          ok            

     5           18   1, 1   n/a          unused 

     5           18   2, 2   n/a          unused 

     5           18   3, 3   n/a          unused        

     5           18   4, 4   n/a          unused        

     5           18   5, 5   n/a          unused        

     5           18   6, 6   n/a          unused        

     5           18   7, 7   n/a          unused        

     5           18   8, 8   n/a          unused        

     5           18   9, 9   n/a          unused        

     5           18  10, 10  n/a          unused        

     5           18  11, 11  n/a          unused        

     5           18  12, 12  n/a          unused        

     5           18  13, 13  n/a          unused        

     5           18  14, 14  n/a          unused        

     5           18  15, 15  n/a          unused        

     5           18  16, 16  n/a          unused        

     5           18  17, 17  n/a          unused

In the show fabric channel switchmode command output, the Switch Mode field displays one of the following modes:

•Flow-through mode

•Truncated mode

•Compact mode

Note For definitions for the different modes, see the "Understanding How the External Switch Fabric Module Works" section.

Displaying the Fabric Channel Utilization

To display the fabric channel utilization, perform this task:

Task
Command

Display the fabric channel utilization.

show fabric channel utilization


This example shows how to display the fabric channel utilization: 

Console> show fabric channel utilization 

Fab Chan Input Output

-------- ----- ------

Fab Chan Speed Input Output

-------- ----- ----- ------

       0   n/a    0%     0%

       1   n/a    0%     0%

       2   n/a    0%     0%

       3   n/a    0%     0%

       4   20G    0%     0%

       5   n/a    0%     0%

       6   n/a    0%     0%

       7   20G    0%     0%

       8    8G    0%     0%

       9   n/a    0%     0%

      10   n/a    0%     0%

      11   n/a    0%     0%

      12   n/a    0%     0%

      13   n/a    0%     0%

      14   n/a    0%     0%

      15   n/a    0%     0%

      16   20G    0%     0%

      17   n/a    0%     0%

Displaying the Fabric Errors

To display the fabric errors of one or all modules, perform this task:

Task
Command

Display the fabric errors.

show fabric errors {mod | all}


This example shows how to display the fabric errors: 

Console> (enable) show fabric errors all

Module errors:

Slot  Channel    CRC        Hbeat      Sync       DDR sync  

----- ---------- ---------- ---------- ---------- ----------

2     0          0          0          0          0         


Fabric errors:

Slot  Channel    Sync       Buffer     Timeout   

----- ---------- ---------- ---------- ----------

2     0          0          0          0         


Console> (enable)

Displaying the Backplane Traffic and Fabric Channel Input and Output

To display the backplane traffic and fabric channel input and output, perform either of these tasks:

Task
Command

Display the system status including the backplane traffic and fabric channel input and output.

show system

Display the backplane traffic and fabric channel input and output.

show traffic


Note Supervisor Engine 720 with PFC3A does not support a hardware traffic meter. When you enter the show system and show traffic commands on this module, you do not receive backplane traffic information.

This example shows how to display the system status including backplane traffic and fabric channel input and output: 

Console> (enable) show system

PS1-Status PS2-Status 

---------- ---------- 

ok         none       


Fan-Status Temp-Alarm Sys-Status Uptime d,h:m:s Logout

---------- ---------- ---------- -------------- ---------

ok         off        ok         13,19:01:16    20 min


PS1-Type             PS2-Type

-------------------- --------------------

WS-CAC-1300W         none                


Modem   Baud  Backplane-Traffic Peak Peak-Time

------- ----- ----------------- ---- ------------------------

disable  9600   0%                0% Tue Oct 19 2004, 12:04:18


PS1 Capacity: 1153.32 Watts (27.46 Amps @42V)


System Name              System Location          System Contact           CC

------------------------ ------------------------ ------------------------ ---


Slot Channel Fab Chan Input Output

---- ------- -------- ----- ------

   2       0        1    0%     0%


Core Dump                Core File              

------------------------ -------------------------------------------

disabled                 slot0:crashdump


Crash Info               Crash Info File        

------------------------ -------------------------------------------

enabled                  bootflash:crashinfo


System Information Logging  Host              Interval

--------------------------  ----------------  --------

Disabled                    -                 1440    


System Information Log File

----------------------------------------------------------------------

tftp:sysinfo                                                          


Index        System Information Logging Commands

-----        ---------------------------------------------------------


Syslog Dump              Syslog File              

------------------------ -------------------------------------------

disabled                 bootflash:sysloginfo


No profile is configured for the system

Console> (enable)

This example shows how to display backplane traffic and fabric channel input and output: 

Console> (enable) show traffic

Threshold: 100%


Backplane-Traffic Peak Peak-Time

----------------- ---- -------------------------

  0%                0% Tue Oct 19 2004, 12:04:18


Slot Channel Fab Chan Input Output

---- ------- -------- ----- ------

   2       0        1    0%     0%


Console> (enable)

Displaying the Switching Mode Configuration

To display the switching mode configuration, perform this task:

Task
Command

Display the switching mode configuration.

show system switchmode


This example shows how to display the switching mode configuration: 

Console> show system switchmode

Switchmode allow:truncated

Switchmode threshold:2

Console> (enable)

Displaying the Integrated Switch Fabric Status

To display the integrated switch fabric status and forwarding speed, perform this task:

Task
Command

Display the integrated switch fabric status and speed.

show fabric status


This example shows how to display the integrated switch fabric status and speed: 

Console> show fabric status

Mod Speed  Fabric

           status

--- ----- -------

  5   20G  active

Console> (enable)

Configuring the LCD Banner

You can modify the LCD banner from the supervisor engine by entering the set banner lcd command to include the following information:

•Chassis serial number

•Switch IP address

•System name

•Supervisor engine version

•Multilayer Switch Feature Card (MSFC) version on active and standby supervisor engine

•System contact

After the LCD banner content is modified, this information is sent to the Switch Fabric Modules that are installed in the chassis and displayed in the LCDs.

Note The set banner lcd command is not supported in the systems with an integrated switch fabric.

To modify the LCD banner content, perform this task in privileged mode:

 
Task
Command

Step 1 

Modify the LCD banner content.

set banner lcd c [text] c

Step 2 

Verify the LCD banner change.

show banner

This example shows how to modify the LCD banner for the Switch Fabric Module: 

Console> (enable) set banner lcd &HelloWorld!&

LCD banner set

Console> (enable) show banner

MOTD banner:


LCD config:

Hello

World!