Guest

Cisco Catalyst 6500 Series Switches

Field Notice: FN - 62099 - Enhanced FlexWAN Fails to Boot with Supervisor Engine 32 - Upgrade Rommon


September 15, 2005


Products Affected

Product

Comments

Family

Part Number

7600

WS-X6582-2PA (=)

Rommon Version 1.0 or earlier

Problem Description

The Enhanced FlexWAN linecard with Rommon version 1.0 or less is unable to boot with Supervisor Engine 32 running IOS 12.2(18)SXF or later.

7600 Line Card

'show version' Rommon Version

'show hw-module all fpd' Rommon Version

Problem with Supervisor Engine 32?

Comments

Enhanced FlexWAN (WS-X6582-2PA)

12.2(17r)SXB3 or earlier

1.0

Yes

Refer to the How to Upgrade the Rommon Version section below for further instructions

Enhanced FlexWAN (WS-X6582-2PA)

12.2(17r)SXB3 and later

1.1 and higher

No

New Enhanced FlexWAN's received from Dec-04 are shipping with Rommon version 12.2(17r)SXB3 or later

SIP-200 (7600-SIP-200)

12.2(18r)SX3 and later

1.2 and higher

No

All SIP-200's are shipping with Rommon version 12.2(18r)SX3 or higher

SIP-400 (7600-SIP-400)

12.2(18r)SX3 and later

1.2 and higher

No

All SIP-400's are shipping with Rommon version 12.2(18r)SX3 or higher

Note: 12.2(17r)SXB3 appears as 1.1 and 12.2(18r)SX3 appears as 1.2 in the output of show hw-module all fpd .

To identify the version of Rommon, refer to the How to Identify the Rommon Version section. Note that all Cisco Service Depots are not affected by this problem. They are stocking Enhanced FlexWAN's with Rommon version 1.1 and later.

Background

The Supervisor Engine 720 and Supervisor Engine 32 use an internal ethernet connection called Ethernet Out of Band Channel (EOBC) to manage system operations. Enhanced FlexWAN linecards with Rommon version 1.0 may send EOBC frames of less than 64 bytes.

This behavior is compatible with the Supervisor Engine 720. The Supervisor Engine 32, however, requires at least 64-byte frames on the EOBC. As a result, an Enhanced FlexWAN linecard with Rommon version 1.0 will not boot in a Supervisor Engine 32 system until it receives an upgraded Rommon version of 1.1 or later, which pads EOBC frames to 64 bytes.

Problem Symptoms

Should the Enhanced FlexWAN linecard with Rommon version 1.0 attempt to boot with the Supervisor Engine 32 running 12.2(18)SXF or later, the Supervisor Engine 32 will power cycle the Enhanced FlexWAN linecard.

If system logging is enabled, you may see the following error messages in the Supervisor Engine 32 system logs:

%C6KPWR-SP-4-DISABLED: power to module in slot 3 set off (Module Failed SCP dnld) 
%C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (Reset) 
%CWAN_RP-3-BOOTFAIL: The WAN module in slot 3/1 failed to boot 
%CWAN_RP-3-POWER_CYCLE: Power cycling WAN module in slot 3/0. Find Master Timer expired 

Workaround/Solution

See the How to Upgrade the Rommon Version section below.

How To Identify the Rommon Version

There are two ways to identify an affected Enhanced FlexWAN Rommon version.

Use the CLI command show version to determine the Supervisor Engines Rommon version. In the example below, the Rommon version is 12.2(17r)S2.

Router#show version

Cisco Internetwork Operating System Software 

IOS (tm) s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXE2, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2005 by cisco Systems, Inc.

Compiled Mon 20-Jun-05 21:37 by kellythw

Image text-base: 0x40021040, data-base: 0x42B08000 

ROM: System Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)

BOOTLDR: s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXE2, RELEASE SOFTWARE (fc1)

[text omitted for legibility]

Router#

Issue the show hw-module all fpd command to verify the Enhanced FlexWAN linecard's Rommon version.

The Enhanced FlexWAN has one Rommon for each of the CPUs (bays). This is shown below with the Device ID's CPU0 and CPU1 Rommon.

Check the CPU0 and CPU1 current Rommon version.

If either of the Rommon versions is 1.0, then the Enhanced FlexWAN needs to be upgraded.

Proceed to the section titled How to Upgrade the Rommon Version and follow one of the outlined procedures to upgrade your Rommon version.

Note: If different versions of Rommon appear under CPU0 and CPU1, the code is designed to upgrade CPU0 and CPU1 Rommon separately. Therefore, if a CPU is already at the required version, it will not upgrade that CPU's Rommon.

7600-Router#show hw-module all fpd 

==== ====================== ====== =============================================

                             H/W   Field Programmable   Current   Min. Required

Slot Card Type               Ver.  Device: "ID-Name"    Version      Version

==== ====================== ====== ================== =========== ==============

   2 WS-X6582-2PA            1.3   1-I/O FPGA             0.37        0.37     

                                   2-EOS FPGA             0.28        0.28     

                                   3-CPU0 ROMMON          1.2         1.2      

                                   4-CPU1 ROMMON          1.2         1.2      

---- ---------------------- ------ ------------------ ----------- --------------

   3 WS-X6582-2PA            1.2   1-I/O FPGA             0.37        0.37     

                                   2-EOS FPGA             0.28        0.28     

                                   3-CPU0 ROMMON          1.2         1.2      

                                   4-CPU1 ROMMON          1.2         1.2      

---- ---------------------- ------ ------------------ ----------- --------------

   4 WS-X6582-2PA            1.5   1-I/O FPGA             0.37        0.37     

                                   2-EOS FPGA             0.29        0.28     

                                   3-CPU0 ROMMON          1.2         1.2      

                                   4-CPU1 ROMMON          1.2         1.2      

==== ====================== ====== =============================================

How To Upgrade the Rommon Version

To resolve this problem, the Enhanced FlexWAN Rommon version must be upgraded. There are two ways to do this:

  1. Upgrade the Rommon version with a Supervisor Engine 720 running IOS version 12.2(18)SXE3.

  2. Upgrade the Enhanced FlexWAN Rommon version with a Supervisor Engine 32 running IOS version 12.2(18)SXF or later.

As a prerequisite to attempting either of these options, verify that the Enhanced FlexWAN is running Rommon version 1.0. Refer to the How to Identify the Rommon Version section for further details.

1. Upgrading the Enhanced FlexWAN Rommon version using 12.2(18)SXE3 and a Supervisor Engine 720:

Follow these four steps to upgrade the Enhanced FlexWAN Rommon version with a Supervisor Engine 720:

  1. Install/Boot Enhanced FlexWAN with Rommon version 1.0 (Rommon version 12.2(17r)SXB3 or earlier) with Supervisor Engine 720 running IOS version 12.2(18)SXE3.

    Use the CLI command show version to verify the version of IOS the Supervisor Engine is running.

    7600-Router#show version
    
    Cisco Internetwork Operating System Software 
    
    IOS (tm) s72033_rp Software (s72033_rp-ADVENTERPRISEK9_WAN-M), Version 12.2(18)SXE3, RELEASE SOFTWARE (fc1)
    
    Technical Support: http://www.cisco.com/techsupport
    
    Copyright (c) 1986-2005 by cisco Systems, Inc.
    
    Compiled Mon 20-Jun-05 21:37 by kellythw
    
    Image text-base: 0x40021040, data-base: 0x42B08000
    
    [text omitted for legibility]
    
    65536K bytes of Flash internal SIMM (Sector size 512K).
    
    Configuration register is 0x2102
    
    7600-Router#
    
    
  2. Once the Enhanced FlexWAN line card has been inserted into the chassis with the Supervisor Engine 720 running 12.2(18)SXE3, the Enhanced FlexWAN Rommon version is automatically upgraded.

    Keep in mind that this will not cause any performance loss or negative side-effects to your system.

  3. Verify that the Enhanced FlexWAN's Rommon version has changed by issuing the CLI command show hw-module all fpd .

    Refer to CPU0 and CPU1 for the current Rommon version.

    Note: If different versions of Rommon appear under CPU0 and CPU1, the code is designed to upgrade CPU0 and CPU1 Rommon separately. Therefore, if a CPU is already at the required version, it will not upgrade that CPU's Rommon.

    7600-Router#show hw-module all fpd
    
    ==== ====================== ====== =============================================
    
                                 H/W   Field Programmable   Current   Min. Required
    
    Slot Card Type               Ver.  Device: "ID-Name"    Version      Version
    
    ==== ====================== ====== ================== =========== ==============
    
       2 WS-X6582-2PA            1.3   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.28        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ---- ---------------------- ------ ------------------ ----------- --------------
    
       3 WS-X6582-2PA            1.2   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.28        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ---- ---------------------- ------ ------------------ ----------- --------------
    
       4 WS-X6582-2PA            1.5   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.29        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ==== ====================== ====== =============================================
    
    
  4. If the Rommon version is now greater then 1.0, you can now safely install the Enhanced FlexWAN into a chassis running a Supervisor Engine 32 running IOS version 12.2(18)SXF or later.

2. Upgrading the Enhanced FlexWAN Rommon version with 12.2(18)SXF and a Supervisor Engine 32 :

Follow these eight steps to upgrade the Enhanced FlexWAN Rommon version with a Supervisor Engine 32:

  1. Ensure that the Supervisor Engine 32 is running IOS version 12.2(18)SXF. Issue the show version CLI command to verify the IOS version.

  2. Turn off the Supervisor Engine 32 EOBC error checking momentarily, using the CLI command remote command switch test platform eobc err-check disable

    If you are logged in thru the Switch-console, use test platform eobc err-check disable .

    Keep in mind that by issuing this command you will not cause any performance loss or negative side-effects to your system.

    SUP-32#remote command switch test platform eobc err-check disable
    Warning: Please re-enable error checking after upgrading the E-FlexWAN. Use the following command to re-enable,
    'remote command switch test platform eobc err-check enable'.
    
  3. Verify the EOBC error checking is disabled by issing the remote command switch show eobc CLI command .

    SUP32#remote command switch show eobc
    
    Interface information:
            Interface EOBC0/0 (idb = 0x433E2F98)
            Hardware is sb1250 EOBC
            Encap size         = 14         hardware status  = 0x210840
            IDB type           = 18         IDB state        = 4
            Encap type         = 0x1        Span encap size  = 0
            Error threshold    = 5000       Error count      = 0
            Error checking is disabled             ==> SUP-32 EOBC error checking is now disabled.
          
    Counters:
            rx_ring            = 0x435E2480 rx ring entries        = 512
            rx_head            = 145        rx_tail                = 0
            rx_count           = 652433     rx_bytes               = 990088374
            rx rate (bits/sec) = 22000      rx rate (packets/sec)  = 12
            rx_multicast_pkts  = 3          rx_bcast_pkts          = 55
            rx_good_pkts       = 5196387    rx_bad_pkts            = 5745
            rx_buf_unavail     = 0          rx_fcs_err_pkts        = 5745
            rx_runt_pkts       = 5742       rx_oversize_pkts       = 0
            rx_pkts_len_err    = 0          rx_pkts_dropped        = 0
            rx_bad_pkts_dropped= 32         rx_bad_len_pkts_dropped= 0
            rx input drops     = 0
            rxr eobc shadow    = 0x50D5E038
    
            tx ring            = 0x435E2480 tx ring entries        = 0x200
            tx_head            = 23         tx_tail                = 23
            tx_good_pkts       = 985111     tx_bad_pkts            = 0
            tx_bytes           = 364986086
            tx rate (bits/sec) = 62000      tx rate (packets/sec)  = 13
            tx_late_coll       = 0          tx_excessive_coll      = 0
            tx_crc_err         = 0          tx_aborted_pkts        = 0
            tx total drops     = 0
    
    
    
    
  4. Insert the Enhanced FlexWAN linecard into the chassis. As the Enhanced FlexWAN linecard powers up, the Rommon Version will be automatically upgraded. Keep in mind that that will not cause any performance loss or negative side-effects to your system.

  5. Once the Enhanced FlexWAN has successfully upgraded the Rommon version, the linecard will be power cycled by the Supervisor.

  6. Once the Enhanced FlexWAN has powered up, check the Rommon version by issuing the show hw-module all fpd CLI command .

    Note: If different versions of Rommon appear under CPU0 and CPU1, the code is designed to upgrade CPU0 and CPU1 Rommon separately. Therefore, if a CPU is already at the required version, it will not upgrade that CPU's Rommon.

    7600-Router#show hw-module all fpd 
    
    ==== ====================== ====== =============================================
    
                                 H/W   Field Programmable   Current   Min. Required
    
    Slot Card Type               Ver.  Device: "ID-Name"    Version      Version
    
    ==== ====================== ====== ================== =========== ==============
    
       2 WS-X6582-2PA            1.3   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.28        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ---- ---------------------- ------ ------------------ ----------- --------------
    
       3 WS-X6582-2PA            1.2   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.28        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ---- ---------------------- ------ ------------------ ----------- --------------
    
       4 WS-X6582-2PA            1.5   1-I/O FPGA             0.37        0.37     
    
                                       2-EOS FPGA             0.29        0.28     
    
                                       3-CPU0 ROMMON          1.2         1.2      
    
                                       4-CPU1 ROMMON          1.2         1.2      
    
    ==== ====================== ====== =============================================
    
  7. If the Enhanced FlexWAN has successfully upgraded the Rommon version, then enable the EOBC error checking by issuing the remote command switch test platform eobc err-check enable CLI command.

    Issue the test platform eobc err-check enable command if logged into the Switch-console.

    SUP-32#remote command switch test platform eobc err-check enable
    
  8. Verify that the EOBC error checking is once again enabled by issuing the remote command switch show eobc CLI command.

    SUP32#remote command switch show eobc
    
    Interface information:
            Interface EOBC0/0 (idb = 0x433E2F98)
            Hardware is sb1250 EOBC
            Encap size         = 14         hardware status  = 0x210840
            IDB type           = 18         IDB state        = 4
            Encap type         = 0x1        Span encap size  = 0
            Error threshold    = 5000       Error count      = 0
            Error checking is enabled              = = > > SUP-32 EOBC error checking is now enabled. 
          
    Counters:
            rx_ring            = 0x435E2480 rx ring entries        = 512
            rx_head            = 475        rx_tail                = 0
            rx_count           = 652763     rx_bytes               = 990789263
            rx rate (bits/sec) = 23000      rx rate (packets/sec)  = 13
            rx_multicast_pkts  = 3          rx_bcast_pkts          = 55
            rx_good_pkts       = 5200393    rx_bad_pkts            = 5748
            rx_buf_unavail     = 0          rx_fcs_err_pkts        = 5748
            rx_runt_pkts       = 5745       rx_oversize_pkts       = 0
            rx_pkts_len_err    = 0          rx_pkts_dropped        = 0
            rx_bad_pkts_dropped= 32         rx_bad_len_pkts_dropped= 0
            rx input drops     = 0
            rxr eobc shadow    = 0x50D5E038
    
            tx ring            = 0x435E2480 tx ring entries        = 0x200
            tx_head            = 357        tx_tail                = 357
            tx_good_pkts       = 985445     tx_bad_pkts            = 0
            tx_bytes           = 365151387
            tx rate (bits/sec) = 61000      tx rate (packets/sec)  = 13
            tx_late_coll       = 0          tx_excessive_coll      = 0
            tx_crc_err         = 0          tx_aborted_pkts        = 0
            tx total drops     = 0
    SUP32# 
    
    

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.