OIR of Modules in Catalyst Switches

Introduction

The Modular Cisco Catalyst switches, such as the 6500, 6000, 5500, 5000, 4500, and 4000 Series, support Online Insertion and Removal (OIR) or Hot Swap of all modules (power supplies, fan trays, Supervisor Modules and other Line and Service Modules). You can add, replace, or remove modules without interrupting the system power or causing other software or interfaces to shut down.

This document provides some simple checks you can carry out when you move modules to a different chassis or when you insert new modules into a chassis.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the Cisco Catalyst 6500 Series Switch with Supervisor Engine 720 and running Cisco IOS^® Software Release 12.2(18)SXD6.

The information in this document was created from the devices in a specific lab environment. If your network is live, make sure that you understand the potential impact of any command.

Related Products

This configuration can also be used with these Cisco Catalyst switches:

• Cisco Catalyst 6000 Series

• Cisco Catalyst 5500 Series

• Cisco Catalyst 5000 Series

• Cisco Catalyst 4500 Series

• Cisco Catalyst 4000 Series

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Background Information

The OIR feature was developed to enable you to replace faulty parts without affecting system operation. When a card is inserted, power is available on the card, and it initializes itself to start working.

When you remove or insert a module while the switch is powered on and operating, this is what the switch does:

• Determines if there is sufficient power for the module.

• Scans the backplane for configuration changes.

• Initializes all newly inserted modules, notes any removed modules, and places them in the administratively shutdown state.

• Places any previously configured interfaces on the module back to the state they were in when they were removed. Any newly inserted interfaces are put in the administratively shutdown state, as if they were present (but unconfigured) at boot time. If you insert a similar switching-module type into a slot, the ports are configured and brought online up to the port count of the original switching module.

Caution: When a module is inserted or removed, the switching bus can sometimes stall for about 3 seconds. This can disrupt the adjacencies in protocols such as Open Shortest Path First (OSPF), Border Gateway Protocol (BGP), or Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP) if their timers have been configured for fast convergence.

Note: Do not remove or install more than one module at a time. The switch can bring only an identical replacement module online. If the replacement module is different from the removed module, you must configure it before the switch can bring it online.

Online Insertion and Removal of Modules

Checklist for Online Insertion and Removal

In this section, you are presented with the list of items to be checked before you perform an online insertion and removal of modules:

• Verify if the module is supported by the supervisor engine of the destination switch.

• Verify if the module is supported by the release of OS (IOS or CatOS) that runs on the destination switch.

• Verify if the module can be placed in the choosen slot on the destination switch.

Move the Module to a Different Slot in a Same Switch

If you plan to move a blade to a different slot within the same chassis, you have to check the Release Notes for the Cisco IOS or CatOS version that the current supervisor runs in order to check if the module that is going to be moved can be inserted in any slot, or if that module needs to be inserted in some particular slots.

For example, module WS-X6748-SFP in a 13-slot chassis with a supervisor that runs Cisco IOS Software Release 12.2SX is only supported in slots 9 through 13 and does not power up in other slots. This information can be found in the Release Notes for Cisco IOS Release 12.2SX on the Supervisor Engine 720, Supervisor Engine 32, and Supervisor Engine 2.

Move the Module to a Different Switch

If you plan to move a module to a different chassis model, make sure the Cisco IOS or CatOS version that the supervisor engine runs, and the supervisor itself, support the module to be inserted. The Release Notes for the IOS or CatOS have to be checked before you move a module to a different chassis.

These are things to check before you move the module:

• Does the supervisor run CatOS or Cisco IOS?

• Verify if the CatOS or Cisco IOS version supports the module to be inserted.

• Verify if the supervisor supports the module that is going to be inserted.

• Verify if the module needs to be inserted in certain slots only.

In this example, there are two chassis:

• A 6506 chassis with:

  • WS-X6K-SUP1A-2GE that runs in Hybrid mode 6.4(19) + MSFC 12.(11b)

  • WS-X6408A-GBIC

• A 6509 chassis with:

  • WS-SUP32-GE-3B that runs in Native mode 12.2(18)SXF7

  • WS-X6516A-GIBIC

In this example, both GBIC modules will be swapped. This is how the configuration looks:

6506 with Supervisor Engine 1   <= WS-X6516A-GIBIC
6509 with Supervisor Engine 32  <= WS-X6408A-GIBIC

First, you need to check the Release Notes for Cisco IOS Software Release 12.2(18)SXF7 which is the version that Supervisor Engine 32 runs. You need to check if this IOS supports module WS-X6408A-GIBIC.

As seen in the Release Notes for Cisco IOS Release 12.2SX on the Supervisor Engine 720, Supervisor Engine 32, and Supervisor Engine 2, module WS-X6408A-GIBIC is supported for Cisco IOS Software Release 12.2SX.

Then, you need to take a look at which supervisors support module WS-X6408A-GIBIC. As you can see in the Release Notes, only Supervisor Engine 720, Supervisor Engine 32 and Supervisor Engine 2 support this module.

Finally, you need to check the minimum IOS each supervisor requires to support module WS-X6408A-GIBIC.

Supervisor	Minimum IOS
With Supervisor Engine 720	12.2(14)SX
With Supervisor Engine 32	12.2(18)SXF
With Supervisor Engine 2	12.2(17d)SXB

Note: Each supervisor requires a minimum IOS version in order to support a module.

Then, you need to check if the Supervisor Engine 1 that runs in Hybrid mode supports module WS-X6516A-GIBIC. Because the supervisor runs CatOS, you need to check the Release Notes for Catalyst 6000 Family Software Release 6.x.

If you search for module WS-X6516A-GBIC, you will see that "The WS-X6516A-GBIC version of this module is not supported in software release 6.x. The WS-X6516A-GBIC version is supported in software release 7.5(1)."

In this case, for Supervisor Engine 1 to support module WS-X6516A-GBIC, the supervisor needs to be upgraded to at least CatOS version 7.5(1).

Note: DRAM memory requirements need to be checked if you attempt a software upgrade.

Erase Configurations Related to a Module

• Before the Module is Removed

• After the Module is Removed

Before the Module is Removed

If a module is physically removed and the configuration is not needed anymore, then apply the module clear-config command from the global configuration mode before you physically remove the module.

Note: The module clear-config command is currently available only in Cisco Catalyst 6500/6000 Series Switches.

Note: The command works when applied before you remove the module.

This is an example of the command usage from the switch:

6509switch(config)#module ?
  ContentServicesGateway  Configure a CSG module
  ContentSwitchingModule  configure a CSM SLB module
  clear-config            To clear configuration when module is removed
  provision               Configure module provision status

Complete these steps:

1. Apply the module clear-config command in global configuration mode.

   6509switch(config)#module clear-config
   

2. Once the command is applied and the configuration is saved, check the output of the show run command to see if the command is there.

   6509switch#show run
   Building configuration...
   
   Current configuration : 6786 bytes
   !
   version 12.2
   service timestamps debug datetime
   service timestamps log datetime
   service password-encryption
   service counters max age 10
   !
   
   !--- Output supressed.
   
   no spanning-tree optimize bpdu transmission
   module clear-config
   fabric required
   fabric switching-mode allow truncated
   diagnostic bootup level com
   !
   
   !--- Output supressed.
   
   !
   6509switch#

3. After the changes have been saved, remove the module from the chassis.

   Once the module is physically removed from the chassis, the configuration will also be removed from the show run command output.

   Note: The side effect of this CLI is that all the configuration related to the removed module will be deleted. Also, when the card is re-inserted, all of the deleted configuration needs to be re-entered.

   Once the old configurations for the non-present modules have been cleared from the configuration, the SNMP MIB configuration for those non-present modules should be removed as well.

After the Module is Removed

After you physically remove a module from the chassis, the configuration for the module still appears. This is actually left in by design to allow for easier replacement. If the same type of module is inserted, it will use the already configured module configuration. If another type of module is inserted into the slot, the module configuration is cleared.

If the module clear-config command is not applied before you remove the module and is applied after you remove the module, then this command will only go into effect when you add modules from this point forward so it will not clear the current state. This means that the configuration for a non-present module will remain until a different model of module is inserted. As soon as a different model of module is inserted, then the configuration will be removed from the show run command output.

Verify

Use this section to confirm that your configuration works properly.

The Output Interpreter Tool (registered customers only) (OIT) supports certain show commands. Use the OIT to view an analysis of show command output.

• show module—Displays the module status and information. In the Mod Sub-Module fields, the show module command displays the supervisor engine number but appends the module type and information of the uplink daughter card.

Troubleshoot

Use this section to troubleshoot any issues with the newly inserted modules.

Module Status is Minor Error

After you insert a module into a slot, the module shows a status of Minor Error from the show module command output. This is probably due to a bad module, a bad slot, or a badly seated module.

Switch#show module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  3    8  8 port 1000mb GBIC Enhanced QoS        WS-X6408A-GBIC     SAL090603RA
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE     SAD09050DGP
  6   48  48 port 10/100/1000mb EtherModule      WS-X6148-GE-TX     SAL0850708A

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  3  0013.1a43.29f0 to 0013.1a43.29f7   3.1   5.4(2)       8.3(0.156)RO Ok
  5  0011.92e7.82cc to 0011.92e7.82cf   3.2   8.1(3)       12.2(18)SXD4 Ok
  6  0012.80f8.5030 to 0012.80f8.505f   6.1   7.2(1)       8.3(0.156)RO Ok

Mod Online Diag Status 
--- -------------------
  3 Pass
  5 Pass
  6 Minor Error

Perform these steps in order to recover the module. Schedule a maintenance window in case the switch is in production and perform these actions:

1. Turn on the diagnostics to a complete level, so when the switch is reloaded detailed information about the modules is displayed.

   Switch(config)#diagnostic bootup level complete
   Switch# show diagnostic mode all
   

2. Issue the hw-module module [module slot number] reset command in order to reset a particular module.

   Switch#hw-module module 4 reset
   Proceed with reload of module?[confirm]
   % reset issued for module 4
   Switch# 
   *Jun 18 19:31:58: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set off 
       (Reset)
   *Jun 18 19:32:43: %DIAG-SP-6-RUN_COMPLETE: Module 4: Running Complete 
       Diagnostics...
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/1, changed state 
       to down
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/2, changed state 
       to down
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/3, changed state 
       to down
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/4, changed state 
       to down
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/10, changed state 
       to down
   
   !--- Output suppressed.
   
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/47, changed state 
       to down
   *Jun 18 19:33:01: %LINK-3-UPDOWN: Interface FastEthernet4/48, changed state 
       to down
   *Jun 18 19:33:00: %DIAG-SP-6-DIAG_OK: Module 4: Passed Online Diagnostics
   *Jun 18 19:33:02: %OIR-SP-6-INSCARD: Card inserted in slot 4, interfaces are 
       now online
   Switch#

3. Enter the show environment command in order to check any possible alarms about the module. Enter the show diagnostic module [module slot number] command.

   If you still receive errors after you reset the module, then complete these steps:

   1. Reseat the module. Physically reseat the module.

   2. Check the output of the show environment command.

   3. Enter the show diagnostic module [module slot number] command.

      If the module still shows up with a minor error after these steps, then complete these next steps:

      1. Try the module in a different slot.

      2. Check the output of the show environment command.

      3. Enter the show diagnostic module [module slot number] command.

Module Status is Unknown / PwrDown

After a module was inserted, the status of this module shows up as Unknown in the show module command output.

This output shows the status of module WS-X6748-GE-TX as Unknown:

Switch#show module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAD09040FXH
  2   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAD09050BT8
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAD090406AF

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0011.bb2b.9b2c to 0011.bb2b.9b5b   2.1   Unknown      Unknown      PwrDown
  2  0011.93d0.acb0 to 0011.93d0.acdf   2.1   12.2(14r)S5  12.2(18)SXD3 Ok
  5  0011.21ba.b6c8 to 0011.21ba.b6cb   4.1   8.1(3)       12.2(18)SXD3 Ok

Mod Sub-Module                  Model              Serial        Hw     Status 
--- --------------------------- ------------------ ------------ ------- -------
  1 Centralized Forwarding Card WS-F6700-CFC       SAL09051F61   2.0    PwrDown
  2 Centralized Forwarding Card WS-F6700-CFC       SAL09051F5F   2.0    Ok
  5 Policy Feature Card 3       WS-F6K-PFC3B       SAD090407MW   1.1    Ok
  5 MSFC3 Daughterboard         WS-SUP720          SAD090306XN   2.2    Ok
          
Mod Online Diag Status 
--- -------------------
  1 Unknown
  2 Pass
  5 Pass
Switch#

When a module appears as Unknown on the show module command output, make sure you check these:

• The supervisor engine and the software it runs support the module.

• The specifications of the module. Make sure the module can be inserted in any slot or if it can only be inserted in specific slots.

Note: For both options, check the Release Notes of the software version that the supervisor engine runs.

Module Status is Unknown / PwrDeny

After you insert a module, the status of it is PwrDeny. If this is the case, check if there is enough power to turn on the module that appears as PwrDeny.

This output shows two modules with a status of Unknown / PwrDeny:

Switch#show module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1   48  48 port 10/100 mb RJ45                 WS-X6348-RJ-45     SAL062410XB
  2    6  Firewall Module                        WS-SVC-FWM-1       SAD0918068W
  3    6  Firewall Module                        WS-SVC-FWM-1       SAD090709TE
  5    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE     SAD090702NV
  6    2  Supervisor Engine 720 (Hot)            WS-SUP720-BASE     SAD085105XN
  7   48  CEF720 48 port 1000mb SFP              WS-X6748-SFP       SAL09148J7G
  9    8  Intrusion Detection System             WS-SVC-IDSM-2      SAD09180065

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0009.1279.5ef8 to 0009.1279.5f27   6.1   5.4(2)       8.3(0.110)TE Ok
  2  0013.c301.1a44 to 0013.c301.1a4b   3.0   7.2(1)       2.3(1)       Ok
  3  0003.e472.940c to 0003.e472.9413   3.0   7.2(1)       1.1(4)       Ok
  5  0011.92e7.8a60 to 0011.92e7.8a63   3.2   8.1(3)       12.2(17d)SXB Ok
  6  0011.21ba.9c4c to 0011.21ba.9c4f   3.2   8.1(3)       12.2(17d)SXB Ok
  7  0013.7f97.d210 to 0013.7f97.d23f   1.4   Unknown      Unknown      PwrDeny
  9  0013.8038.063c to 0013.8038.0643   5.0   Unknown      Unknown      PwrDeny

Mod Sub-Module                  Model              Serial        Hw     Status
--- --------------------------- ------------------ ------------ ------- -------
  1 Inline Power Module         WS-F6K-PWR                       1.0    Ok
  5 Policy Feature Card 3       WS-F6K-PFC3A       SAD0906076P   2.4    Ok
  5 MSFC3 Daughterboard         WS-SUP720          SAD0905052Z   2.4    Ok
  6 Policy Feature Card 3       WS-F6K-PFC3A       SAD08490B95   2.4    Ok
  6 MSFC3 Daughterboard         WS-SUP720          SAD0850062A   2.4    Ok
  7 Centralized Forwarding Card WS-F6700-CFC       SAL090607GH   2.0    PwrDeny

Mod Online Diag Status
--- -------------------
  1 Pass
  2 Pass
  3 Pass
  5 Pass
  6 Pass
  7 Unknown
  9 Unknown

If you have checked that the power supplies deliver enough power to turn on all of the modules, then enter the power enable module [module slot number] command in order to enable the power for the module that appears as PwrDeny:

Switch(config)#power enable module 4

If you are still unable to determine the problem, or if the error message is not present in the documentation, contact the Cisco Technical Support escalation center.

Related Information

• Online Insertion and Removal (OIR) Support in Routers
• Switches Product Support
• LAN Switching Technology Support
• Technical Support & Documentation - Cisco Systems