Guest

Cisco 7200 Series Routers

Field Notice: FN - 62514 - C7200-JC-PA - Certain Jacket Cards with PA installed on 7200VXR may have infrequent system crashes due to PCI Bus Error, Software Forced Crash, WDT Reset error - RMA required


Revised April 6, 2009

September 6, 2006

NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.


Revision History

Revision

Date

Comment

1.3 06-APR-2009 Removed Upgrade Program
1.2 09-JUN-2008 Added Upgrade Program and associated changes to Workaround/Solution section. Added identifiers in Products Affected section.

1.1

10-JAN-2008

Updated all sections including the title to address the failure independent of line card NPE-G1 and G2, and provide a Fix on Fail for a bad jacket card.

1.0

06-SEP-2006

Initial Public Release

    

Products Affected

Products Affected

Top Assembly

Printed Circuit Assembly

Comments

Hardware Revision

Part Number

Revision

Part Number

Revision

C7200-JC-PA

68-2619-04

A0

73-10416-04

A0

Units with TAN 68-2619-04 and lower are affected .

2.0 without deviations D086026, D086578, D093490

    

Problem Description

Infrequent failures have occurred on C7200-JC-PA which have resulted in system crashes due to following errors: PCI Bus Error, Software Forced Crash Error, and Watchdog Time Out (WDT) Reset.

Background

System crashes have been observed on 7200VXR systems when the PA is installed in a C7200-JC-PA port adaptor jacket card. The system crashes could occur when a bidirectional traffic stream is frequently started and stopped. The following errors: PCI Bus Errors, Software Forced Crash errors, and WDT errors are only seen at the onset of traffic.

Note: Port adaptors can be installed with or without jacket cards. Only port adaptors installed in jacket cards are affected by this issue.

Types of PAs supported with C7200-JC-PA on 7200VXR.

Problem Symptoms

The following errors : PCI Bus Errors, Software Crash Errors, and WDT Errors have been observed, which may lead to System crashes on 7200VXR platform with any PA installed in a Jacket Card.

  1. System Crash has been observed when the PA-MC-2T3+ is installed in the C7200-JC-PA with NPE-G2 as the result of a PCI bus errors.

  2. Software forced crashes have been observed when the PA-MC-STM1 is installed in the C7200-JC-PA with the NPE-G2.

  3. WDT Errors have been observed when the PA-POS-2OC3 is installed in the C7200-JC-PA with NPE-G1.

Examples of each system behavior can be seen below.

Example 1: PA-MC-2T3+, C7200-JC-PA and NPE-G2 System Crash due to PCI Bus Error

%ERR-1-SERR: PCI bus system/parity error
%ERR-1-FATAL: Fatal error interrupt, No reloading
 err_stat=0x0, err_enable=0xFF, mgmt_event=0x8


System bridge dump:
PCI B:3 D:0 F:0 Reg:0x00: device and vendor id           = 0x648511AB
PCI B:3 D:0 F:0 Reg:0x04: status and command             = 0x00000147
PCI B:3 D:0 F:0 Reg:0x08: class code and rev id          = 0x05800003
PCI B:3 D:0 F:0 Reg:0x0C: hdr type, lat timer and cls    = 0x80800010
PCI B:3 D:0 F:0 Reg:0x10: PCI CSN0 BAR (LOW)             = 0x3800000C
PCI B:3 D:0 F:0 Reg:0x14: PCI CSN0 BAR (HIGH)            = 0x00000000
PCI B:3 D:0 F:0 Reg:0x18: PCI CSN1 BAR (LOW)             = 0xF800000C
PCI B:3 D:0 F:0 Reg:0x1C: PCI CSN1 BAR (HIGH)            = 0x00000000
PCI B:3 D:0 F:0 Reg:0x20: Memory Mapped Base Address(l)  = 0x14000004
PCI B:3 D:0 F:0 Reg:0x24: Memory Mapped Base Address(h)  = 0x00000000
PCI B:3 D:0 F:0 Reg:0x2C: Subsystem VendorID             = 0x00000000
PCI B:3 D:0 F:0 Reg:0x30: Expansion ROM BAR              = 0xFF000000
PCI B:3 D:0 F:0 Reg:0x34: Capability List Pointer Reg    = 0x00000040
PCI B:3 D:0 F:0 Reg:0x3C: Interrupt pin line             = 0x00000100
PCI B:3 D:0 F:0 Reg:0x40: Power Management Control       = 0x7E0A4801
PCI B:3 D:0 F:0 Reg:0x44: Power Management Ctrl/Stat     = 0x00000000
PCI B:3 D:0 F:0 Reg:0x48: VPD Address                    = 0x00005003
PCI B:3 D:0 F:0 Reg:0x4C: VPD Data                       = 0x00000000
PCI B:3 D:0 F:0 Reg:0x50: MSI Message Control            = 0x00806005
PCI B:3 D:0 F:0 Reg:0x54: MSI Message Address            = 0x00000000
PCI B:3 D:0 F:0 Reg:0x58: MSI Message Upper Address      = 0x00000000
PCI B:3 D:0 F:0 Reg:0x5C: MSI Message Data               = 0x00000000
PCI B:3 D:0 F:0 Reg:0x60: PCI-X Command                  = 0x00306807
PCI B:3 D:0 F:0 Reg:0x64: PCI-X Status                   = 0x01930300
PCI B:3 D:0 F:0 Reg:0x68: CompactPCI Hot Swap            = 0x00000006
PCI B:3 D:0 F:1 Reg:0x10: PCI CSN2 BAR (LOW)             = 0x0100000C
PCI B:3 D:0 F:1 Reg:0x14: PCI CSN2 BAR (HIGH)            = 0x00000000
PCI B:3 D:0 F:1 Reg:0x18: PCI CSN3 BAR (LOW)             = 0x0180000C
PCI B:3 D:0 F:1 Reg:0x1C: PCI CSN3 BAR (HIGH)            = 0x00000000
PCI B:3 D:0 F:1 Reg:0x20: Integrated SRAM BAR (LOW)      = 0x4200000C
PCI B:3 D:0 F:1 Reg:0x24: Integrated SRAM BAR (HIGH)     = 0x00000000
PCI B:3 D:0 F:2 Reg:0x10: PCI DEVCS0 BAR (LOW)           = 0x1C000004
PCI B:3 D:0 F:2 Reg:0x14: PCI DEVCS0 BAR (HIGH)          = 0x00000000
PCI B:3 D:0 F:2 Reg:0x18: PCI DEVCS1 BAR (LOW)           = 0x1C800004
PCI B:3 D:0 F:2 Reg:0x1C: PCI DEVCS1 BAR (HIGH)          = 0x00000000
PCI B:3 D:0 F:2 Reg:0x20: PCI DEVCS2 BAR (LOW)           = 0x1D000004
PCI B:3 D:0 F:2 Reg:0x24: PCI DEVCS2 BAR (HIGH)          = 0x00000000
PCI B:3 D:0 F:3 Reg:0x10: PCI DEVCS3 BAR (LOW)           = 0xFF000004
PCI B:3 D:0 F:3 Reg:0x14: PCI DEVCS3 BAR (HIGH)          = 0x00000000
PCI B:3 D:0 F:3 Reg:0x18: PCI BOOTCS BAR (LOW)           = 0xFF800004
PCI B:3 D:0 F:3 Reg:0x1C: PCI BOOTCS BAR (HIGH)          = 0x00000000
PCI B:3 D:0 F:3 Reg:0x20: PCI CPU Base Address (LOW)     = 0x4000000C
PCI B:3 D:0 F:3 Reg:0x24: PCI CPU Base Address (HIGH)    = 0x00000000
PCI B:3 D:0 F:4 Reg:0x10: PCI P2PMEM0 BAR (LOW)          = 0x2200000C
PCI B:3 D:0 F:4 Reg:0x14: PCI P2PMEM0 BAR (HIGH)         = 0x00000000
PCI B:3 D:0 F:4 Reg:0x18: PCI P2PMEM1 BAR (LOW)          = 0x2400000C
PCI B:3 D:0 F:4 Reg:0x1C: PCI P2PMEM1 BAR (HIGH)         = 0x00000000
PCI B:3 D:0 F:4 Reg:0x20: PCI P2P I/O Base Address       = 0x20000001
PCI B:3 D:0 F:4 Reg:0x24: PCI I/O Mapped Base Address    = 0x14000001


PCI B:0 D:0 F:0 Reg:0x00: device and vendor id           = 0x648511AB
PCI B:0 D:0 F:0 Reg:0x04: status and command             = 0x40000147
PCI B:0 D:0 F:0 Reg:0x08: class code and rev id          = 0x05800003
PCI B:0 D:0 F:0 Reg:0x0C: hdr type, lat timer and cls    = 0x80800010
PCI B:0 D:0 F:0 Reg:0x10: PCI CSN0 BAR (LOW)             = 0x3800000C
PCI B:0 D:0 F:0 Reg:0x14: PCI CSN0 BAR (HIGH)            = 0x00000000
PCI B:0 D:0 F:0 Reg:0x18: PCI CSN1 BAR (LOW)             = 0xF800000C
PCI B:0 D:0 F:0 Reg:0x1C: PCI CSN1 BAR (HIGH)            = 0x00000000
PCI B:0 D:0 F:0 Reg:0x20: Memory Mapped Base Address(l)  = 0x14000004
PCI B:0 D:0 F:0 Reg:0x24: Memory Mapped Base Address(h)  = 0x00000000
PCI B:0 D:0 F:0 Reg:0x2C: Subsystem VendorID             = 0x00000000
PCI B:0 D:0 F:0 Reg:0x30: Expansion ROM BAR              = 0xFF000000
PCI B:0 D:0 F:0 Reg:0x34: Capability List Pointer Reg    = 0x00000040
PCI B:0 D:0 F:0 Reg:0x3C: Interrupt pin line             = 0x00000100
PCI B:0 D:0 F:0 Reg:0x40: Power Management Control       = 0x7E0A4801
PCI B:0 D:0 F:0 Reg:0x44: Power Management Ctrl/Stat     = 0x00000000
PCI B:0 D:0 F:0 Reg:0x48: VPD Address                    = 0x00005003
PCI B:0 D:0 F:0 Reg:0x4C: VPD Data                       = 0x00000000
PCI B:0 D:0 F:0 Reg:0x50: MSI Message Control            = 0x00806005
PCI B:0 D:0 F:0 Reg:0x54: MSI Message Address            = 0x00000000
PCI B:0 D:0 F:0 Reg:0x58: MSI Message Upper Address      = 0x00000000
PCI B:0 D:0 F:0 Reg:0x5C: MSI Message Data               = 0x00000000
PCI B:0 D:0 F:0 Reg:0x60: PCI-X Command                  = 0x00300007
PCI B:0 D:0 F:0 Reg:0x64: PCI-X Status                   = 0x21930000
PCI B:0 D:0 F:0 Reg:0x68: CompactPCI Hot Swap            = 0x00800006
PCI B:0 D:0 F:1 Reg:0x10: PCI CSN2 BAR (LOW)             = 0x0100000C
PCI B:0 D:0 F:1 Reg:0x14: PCI CSN2 BAR (HIGH)            = 0x00000000
PCI B:0 D:0 F:1 Reg:0x18: PCI CSN3 BAR (LOW)             = 0x0180000C
PCI B:0 D:0 F:1 Reg:0x1C: PCI CSN3 BAR (HIGH)            = 0x00000000
PCI B:0 D:0 F:1 Reg:0x20: Integrated SRAM BAR (LOW)      = 0x4200000C
PCI B:0 D:0 F:1 Reg:0x24: Integrated SRAM BAR (HIGH)     = 0x00000000
PCI B:0 D:0 F:2 Reg:0x10: PCI DEVCS0 BAR (LOW)           = 0x1C000004
PCI B:0 D:0 F:2 Reg:0x14: PCI DEVCS0 BAR (HIGH)          = 0x00000000
PCI B:0 D:0 F:2 Reg:0x18: PCI DEVCS1 BAR (LOW)           = 0x1C800004
PCI B:0 D:0 F:2 Reg:0x1C: PCI DEVCS1 BAR (HIGH)          = 0x00000000
PCI B:0 D:0 F:2 Reg:0x20: PCI DEVCS2 BAR (LOW)           = 0x1D000004
PCI B:0 D:0 F:2 Reg:0x24: PCI DEVCS2 BAR (HIGH)          = 0x00000000
PCI B:0 D:0 F:3 Reg:0x10: PCI DEVCS3 BAR (LOW)           = 0xFF000004
PCI B:0 D:0 F:3 Reg:0x14: PCI DEVCS3 BAR (HIGH)          = 0x00000000
PCI B:0 D:0 F:3 Reg:0x18: PCI BOOTCS BAR (LOW)           = 0xFF800004
PCI B:0 D:0 F:3 Reg:0x1C: PCI BOOTCS BAR (HIGH)          = 0x00000000
PCI B:0 D:0 F:3 Reg:0x20: PCI CPU Base Address (LOW)     = 0x4000000C
PCI B:0 D:0 F:3 Reg:0x24: PCI CPU Base Address (HIGH)    = 0x00000000
PCI B:0 D:0 F:4 Reg:0x10: PCI P2PMEM0 BAR (LOW)          = 0x1200000C
PCI B:0 D:0 F:4 Reg:0x14: PCI P2PMEM0 BAR (HIGH)         = 0x00000000
PCI B:0 D:0 F:4 Reg:0x18: PCI P2PMEM1 BAR (LOW)          = 0xF200000C
PCI B:0 D:0 F:4 Reg:0x1C: PCI P2PMEM1 BAR (HIGH)         = 0x00000000
PCI B:0 D:0 F:4 Reg:0x20: PCI P2P I/O Base Address       = 0x10000001
PCI B:0 D:0 F:4 Reg:0x24: PCI I/O Mapped Base Address    = 0x14000001



PLX Bridge 0, for PA Bay 0 (I/O Card, PCMCIA, Interfaces), handle=2
PLX PCI6520CB bridge, config=0x0
(0x00):device and vendor id           = 0x652010B5
(0x04):status and command             = 0x42B00147
        Signaled System Error
(0x08):class code and rev id          = 0x060400CB
(0x0C):hdr type, lat timer and cls    = 0x00012E10
(0x18):subord, sec and pri bus ids    = 0x18111000
(0x1C):sec status, IO limit and base  = 0x22A03101
        Received Master Abort
(0x20):memory limit and base          = 0xDEF0D800
(0x24):prefetch limit and base        = 0x0001FF01
(0x28):prefetch base upper 32 bits    = 0x00000000
(0x2C):prefetch limit upper 32 bits   = 0x00000000
(0x30):IO limit & base upper 16 bits  = 0x00000000
(0x3C):bridge control and intr line   = 0x04000000
(0x40):Chip, diag & drbiter control   = 0x02000000
(0x44):PFT, Timeout ctrl & misc opts  = 0x00100000
(0x48):Init & incr prefetch control   = 0x3C3C2E2E
(0x4C):Max prefetch, SFT & buf ctrl   = 0x00000000
(0x50):Internal arbiter control       = 0x00000000
(0x60):Timer conter and timer control = 0x00000000
(0x64):P_SERR and GPIO[3:0] control   = 0xF0000000
(0x68):Clk run & ctrl, P_SEER status  = 0x00557FFF
        Address Parity Error
        Posted Write Non-Delivery   
        Master Abort of Posted Write
        Delayed Read Failed
(0x9C):Read-Only reg & GPIO[7:4] ctrl = 0xF0000000
(0xF0):pci-x secondary status         = 0x0003AC07
(0xF4):pci-x bridge status            = 0x00070018
(0xF8):pci-x upstream split control   = 0x00200020
(0xFC):pci-x downstream split control = 0x00200020
PLX Bridge 1, for PA bay 1, 3 and 5, handle=3
PLX PCI6520CB bridge, config=0x0
(0x00):device and vendor id           = 0x652010B5
(0x04):status and command             = 0x02B00147
(0x08):class code and rev id          = 0x060400CB
(0x0C):hdr type, lat timer and cls    = 0x00012E10
(0x18):subord, sec and pri bus ids    = 0x18090403
(0x1C):sec status, IO limit and base  = 0x02A09141
(0x20):memory limit and base          = 0xE270E000
(0x24):prefetch limit and base        = 0x0001FF01
(0x28):prefetch base upper 32 bits    = 0x00000000
(0x2C):prefetch limit upper 32 bits   = 0x00000000
(0x30):IO limit & base upper 16 bits  = 0x00000000
(0x3C):bridge control and intr line   = 0x00000000
(0x40):Chip, diag & drbiter control   = 0x02000000
(0x44):PFT, Timeout ctrl & misc opts  = 0x00100000
(0x48):Init & incr prefetch control   = 0x3C3C2E2E
(0x4C):Max prefetch, SFT & buf ctrl   = 0x00000000
(0x50):Internal arbiter control       = 0x00200000
(0x60):Timer conter and timer control = 0x00000000
(0x64):P_SERR and GPIO[3:0] control   = 0xF0000000
(0x68):Clk run & ctrl, P_SEER status  = 0x00007FFF
(0x9C):Read-Only reg & GPIO[7:4] ctrl = 0xF0000000
(0xF0):pci-x secondary status         = 0x0003AC07
(0xF4):pci-x bridge status            = 0x00030308
(0xF8):pci-x upstream split control   = 0x00200020
(0xFC):pci-x downstream split control = 0x00200020


PLX Bridge 2, for PA bay 2, 4 and 6, handle=4
PLX PCI6520CB bridge, config=0x0
(0x00):device and vendor id           = 0x652010B5
(0x04):status and command             = 0x02B00147
(0x08):class code and rev id          = 0x060400CB
(0x0C):hdr type, lat timer and cls    = 0x00012E10
(0x18):subord, sec and pri bus ids    = 0x180F0A03
(0x1C):sec status, IO limit and base  = 0x02A0F1A1
(0x20):memory limit and base          = 0xE6F0E400
(0x24):prefetch limit and base        = 0x0001FF01
(0x28):prefetch base upper 32 bits    = 0x00000000
(0x2C):prefetch limit upper 32 bits   = 0x00000000
(0x30):IO limit & base upper 16 bits  = 0x00000000
(0x3C):bridge control and intr line   = 0x00000000
(0x40):Chip, diag & drbiter control   = 0x02000000
(0x44):PFT, Timeout ctrl & misc opts  = 0x00100000
(0x48):Init & incr prefetch control   = 0x3C3C2E2E
(0x4C):Max prefetch, SFT & buf ctrl   = 0x00000000
(0x50):Internal arbiter control       = 0x00200000
(0x60):Timer conter and timer control = 0x00000000
(0x64):P_SERR and GPIO[3:0] control   = 0xF0000000
(0x68):Clk run & ctrl, P_SEER status  = 0x00007FFF
(0x9C):Read-Only reg & GPIO[7:4] ctrl = 0xF0000000
(0xF0):pci-x secondary status         = 0x0003AC07
(0xF4):pci-x bridge status            = 0x00030310
(0xF8):pci-x upstream split control   = 0x00200020
(0xFC):pci-x downstream split control = 0x00200020


PA bridge dump:


Bridge 10, Port Adaptor 7, Handle=7
DEC21150 bridge chip, Primary Bus 16, Secondary Bus 17,config=0x0
(0x00):dev, vendor id       = 0x00231011
(0x04):status, command      = 0x42B00147
         Signaled System Error  on primary bus
(0x08):class code, revid    = 0x06040040
(0x0C):hdr, lat timer, cls  = 0x00012E10
(0x18):sec lat,cls & bus no = 0x18111110
(0x1C):sec status, io base  = 0x2AA03101
         Received Master Abort  on secondary bus
         Signaled Target Abort  on secondary bus
(0x20):mem base & limit     = 0xD870D800
(0x24):prefetch membase/lim = 0x0001FF01
(0x30):io base/lim upper16  = 0x00000000
(0x3C):bridge ctrl          = 0x07030000
(0x40):arb/serr, chip ctrl  = 0x02000000
(0x64):serr disable, gpio   = 0xB0000000
(0x68):sec
*** System received a System Error ***
signal= 0x16, code= 0x0, context= 0x5824d90


*** Unknown Exception ***
Exception vector = 0x0


PC     = 0x00000000   MSR    = 0x00000000   CR    = 0x00000000   CPUVER= 0x00000000
LR     = 0x00000000   CTR    = 0x00000000   XER   = 0x00000000   DEC   = 0x00000000
TBU    = 0x00000000   TBL    = 0x00000000   PVR   = 0x00000000   DAR   = 0x00000000
DSISR  = 0x00000000   HID0   = 0x00000000   HID1  = 0x00000000   PIR   = 0x00000000
SPRG0  = 0x00000000   SPRG1  = 0x00000000   SPRG2 = 0x00000000   SPRG3 = 0x00000000
MSSCR0 = 0x00000000   MSSSR0 = 0x00000000   PTEHI = 0x00000000   PTELO = 0x00000000


R0     = 0x00000000   R1     = 0x00000000   R2    = 0x00000000   R3    = 0x00000000
R4     = 0x00000000   R5     = 0x00000000   R6    = 0x00000000   R7    = 0x00000000
R8     = 0x00000000   R9     = 0x00000000   R10   = 0x00000000   R11   = 0x00000000
R12    = 0x00000000   R13    = 0x00000000   R14   = 0x00000000   R15   = 0x00000000
R16    = 0x00000000   R17    = 0x00000000   R18   = 0x00000000   R19   = 0x00000000
R20    = 0x00000000   R21    = 0x00000000   R22   = 0x00000000   R23   = 0x00000000
R24    = 0x00000000   R25    = 0x00000000   R26   = 0x00000000   R27   = 0x00000000
R28    = 0x00000000   R29    = 0x00000000   R30   = 0x00000000   R31   = 0x00000000


SR[0]  = 0x00000000   SR[1]  = 0x00000000   SR[2]  = 0x00000000  SR[3]  = 0x00000000
SR[4]  = 0x00000000   SR[5]  = 0x00000000   SR[6]  = 0x00000000  SR[7]  = 0x00000000
SR[8]  = 0x00000000   SR[9]  = 0x00000000   SR[10] = 0x00000000  SR[11] = 0x00000000
SR[12] = 0x00000000   SR[13] = 0x00000000   SR[14] = 0x00000000  SR[15] = 0x00000000
IBAT0L = 0x00000000   IBAT0U = 0x00000000   IBAT1L = 0x00000000  IBAT1U = 0x00000000
IBAT2L = 0x00000000   IBAT2U = 0x00000000   IBAT3L = 0x00000000  IBAT3U = 0x00000000
DBAT0L = 0x00000000   DBAT0U = 0x00000000   DBAT1L = 0x00000000  DBAT1U = 0x00000000
DBAT2L = 0x00000000   DBAT2U = 0x00000000   DBAT3L = 0x00000000  DBAT3U = 0x00000000
SDR1   = 0x00000000   TLBMISS= 0x00000000
clk

Example 2. PA-MC-STM1, C7200-JC-PA and NPE-G2 System Crash due to Software Forced Crash

Example 2a. (Watchdog not Cleared)

Some Output Omitted]


%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
%HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF


[Some Output Omitted]


%Software-forced reload

Preparing to dump core...
*Mar 29 20:11:02.823: HAL WatchDog not cleared, WatchDog = FFFFFFFF
*Mar 29 20:11:02.823: %HAL-2-HALFWCRASHED: HAL F/W crashed in bay 7: 0xFFFFFFFF - reset
*Mar 29 20:11:02.823: %HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
*Mar 29 20:11:02.823: %HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
*Mar 29 20:11:02.823: %HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF


[Some Output Omitted]


*Mar 29 20:11:02.859: %HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
*Mar 29 20:11:02.859: %HAL-2-HALFWCRASHEDINFO:  0xFFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF
No warm reboot Storage 
*** System received a Software forced crash ***
signal= 0x17, code= 0x700, context= 0x5822e30

*** Program Exception ***
Exception vector = 0x700

PC     = 0x006fb6ec   MSR    = 0x00029032   CR    = 0x40044082   CPUVER= 0x00008004
LR     = 0x006fb6ec   CTR    = 0x027eda60   XER   = 0x20000000   DEC   = 0x000287f8
TBU    = 0x0000003c   TBL    = 0x7d65320e   PVR   = 0x80040201   DAR   = 0x00000000
DSISR  = 0x00000000   HID0   = 0x00000000   HID1  = 0x00000000   PIR   = 0x00000000
SPRG0  = 0x00000000   SPRG1  = 0x00000000   SPRG2 = 0x00000000   SPRG3 = 0x00000000
MSSCR0 = 0x00000000   MSSSR0 = 0x00000000   PTEHI = 0x00000000   PTELO = 0x00000000

R0     = 0x006fb6ec   R1     = 0x06245efc   R2    = 0xfff3a9e0   R3    = 0x00000000
R4     = 0x027f1ae0   R5     = 0x00009032   R6    = 0x054c0000   R7    = 0x05430000
R8     = 0x05fd7bf8   R9     = 0x000017ba   R10   = 0x054c4020   R11   = 0x000001f0
R12    = 0x005f2444   R13    = 0xfff3bc40   R14   = 0x00000000   R15   = 0xffffffff
R16    = 0x00000020   R17    = 0x00000000   R18   = 0x00000000   R19   = 0xffffffff
R20    = 0x00000000   R21    = 0x00000000   R22   = 0xffffffff   R23   = 0x00000000
R24    = 0x00000010   R25    = 0x00000017   R26   = 0x00000000   R27   = 0x0000004f
R28    = 0x0582cb40   R29    = 0x00000000   R30   = 0x00000003   R31   = 0x00000000

SR[0]  = 0x00000000   SR[1]  = 0x00000000   SR[2]  = 0x00000000  SR[3]  = 0x00000000
SR[4]  = 0x00000000   SR[5]  = 0x00000000   SR[6]  = 0x00000000  SR[7]  = 0x00000000
SR[8]  = 0x00000000   SR[9]  = 0x00000000   SR[10] = 0x00000000  SR[11] = 0x00000000
SR[12] = 0x00000000   SR[13] = 0x00000000   SR[14] = 0x00000000  SR[15] = 0x00000000
IBAT0L = 0x00000000   IBAT0U = 0x00000000   IBAT1L = 0x00000000  IBAT1U = 0x00000000
IBAT2L = 0x00000000   IBAT2U = 0x00000000   IBAT3L = 0x00000000  IBAT3U = 0x00000000
DBAT0L = 0x00000000   DBAT0U = 0x00000000   DBAT1L = 0x00000000  DBAT1U = 0x00000000
DBAT2L = 0x00000000   DBAT2U = 0x00000000   DBAT3L = 0x00000000  DBAT3U = 0x00000000
SDR1   = 0x00000000   TLBMISS= 0x00000000
rommon 2 >

Example 2b. (CPU-Hog Error)

%SYS-3-CPUHOG: Task is running for (118004)msecs, more than (2000)msecs (73/37),process = hal_periodic 7.
-Traceback= 9C8974 3850F8 3150000 374A70 374CC4 
%SYS-3-CPUHOG: Task is running for (120004)msecs, more than (2000)msecs (74/37),process = hal_periodic 7.
-Traceback= 9C8974 3850F8 3150000 374A70 374CC4 
%SYS-3-CPUHOG: Task is running for (122004)msecs, more than (2000)msecs (75/37),process = hal_periodic 7.
-Traceback= 9C8974 38530C 3150000 374A50 374CC4 
%SYS-3-CPUHOG: Task is running for (124004)msecs, more than (2000)msecs (75/37),process = hal_periodic 7.
-Traceback= 9C8974 38530C 3150000 374A50 374CC4 
%SYS-3-CPUHOG: Task is running for (126004)msecs, more than (2000)msecs (76/37),process = hal_periodic 7.
-Traceback= 9C8974 38530C 3150000 374A50 374CC4 
%SYS-3-CPUHOG: Task is running f


%Software-forced reload


Preparing to dump core...


 20:09:27 UTC Wed Sep 27 2000: Unexpected exception to CPUvector 700, PC = 0xA08A98  , LR = 0xA08A98  
-Traceback= A08A98 A08A98 A019F8 A0CC88 9C678C 40CF90C 9CEFF0 9CEF4C 9CBD24 9CBD74 A76E4C 8CE044 8CB6FC 8CE4F0 8B37F4 8B4080 


CPU Register Context:
MSR = 0x00029032  CR  = 0x20044084  CTR = 0x009C883C  XER   = 0x00000000
R0  = 0x00A08A98  R1  = 0x040CF870  R2  = 0xFFE3AD80  R3    = 0x00000000
R4  = 0x009BAE70  R5  = 0x00009032  R6  = 0xE27DE7E8  R7    = 0x000002B9
R8  = 0x000E57E0  R9  = 0x000005FE  R10 = 0x00000000  R11   = 0x00000001
R12 = 0x000ED9AC  R13 = 0xFFE3BFE0  R14 = 0x00000000  R15   = 0x00000000
R16 = 0x00000000  R17 = 0x00000000  R18 = 0x00000000  R19   = 0x00000000
R20 = 0x00000000  R21 = 0x03E96504  R22 = 0x00000000  R23   = 0x00000000
R24 = 0x00000003  R25 = 0x00000012  R26 = 0xFFFFFFFE  R27   = 0x0000006F
R28 = 0x033BE5F8  R29 = 0x00000000  R30 = 0x00000003  R31   = 0x00000000


Writing crashinfo to bootflash:crashinfo_20000927-200927
*** System received a Software forced crash ***
signal= 0x17, code= 0x700, context= 0x31c30ec
  PC = 0x00a08a98, SP = 0x040cf870, LR = 0x00a08a98


*** Program Exception ***
Exception vector = 0x700


PC     = 0x00a08a98   MSR    = 0x00029032   CR    = 0x20044084   CPUVER= 0x00008003
LR     = 0x00a08a98   CTR    = 0x009c883c   XER   = 0x00000000   DEC   = 0x0002885f
TBU    = 0x00000009   TBL    = 0x710a9f87   PVR   = 0x80030101   DAR   = 0x00000000
DSISR  = 0x00000000   HID0   = 0x00000000   HID1  = 0x00000000   PIR   = 0x00000000
SPRG0  = 0x00000000   SPRG1  = 0x00000000   SPRG2 = 0x00000000   SPRG3 = 0x00000000
MSSCR0 = 0x00000000   MSSSR0 = 0x00000000   PTEHI = 0x00000000   PTELO = 0x00000000


R0     = 0x00a08a98   R1     = 0x040cf870   R2    = 0xffe3ad80   R3    = 0x00000000
R4     = 0x009bae70   R5     = 0x00009032   R6    = 0xe27de7e8   R7    = 0x000002b9
R8     = 0x000e57e0   R9     = 0x000005fe   R10   = 0x00000000   R11   = 0x00000001
R12    = 0x000ed9ac   R13    = 0xffe3bfe0   R14   = 0x00000000   R15   = 0x00000000
R16    = 0x00000000   R17    = 0x00000000   R18   = 0x00000000   R19   = 0x00000000
R20    = 0x00000000   R21    = 0x03e96504   R22   = 0x00000000   R23   = 0x00000000
R24    = 0x00000003   R25    = 0x00000012   R26   = 0xfffffffe   R27   = 0x0000006f
R28    = 0x033be5f8   R29    = 0x00000000   R30   = 0x00000003   R31   = 0x00000000


SR[0]  = 0x00000000   SR[1]  = 0x00000000   SR[2]  = 0x00000000  SR[3]  = 0x00000000
SR[4]  = 0x00000000   SR[5]  = 0x00000000   SR[6]  = 0x00000000  SR[7]  = 0x00000000
SR[8]  = 0x00000000   SR[9]  = 0x00000000   SR[10] = 0x00000000  SR[11] = 0x00000000
SR[12] = 0x00000000   SR[13] = 0x00000000   SR[14] = 0x00000000  SR[15] = 0x00000000
IBAT0L = 0x00000000   IBAT0U = 0x00000000   IBAT1L = 0x00000000  IBAT1U = 0x00000000
IBAT2L = 0x00000000   IBAT2U = 0x00000000   IBAT3L = 0x00000000  IBAT3U = 0x00000000
DBAT0L = 0x00000000   DBAT0U = 0x00000000   DBAT1L = 0x00000000  DBAT1U = 0x00000000
DBAT2L = 0x00000000   DBAT2U = 0x00000000   DBAT3L = 0x00000000  DBAT3U = 0x00000000
SDR1   = 0x00000000   TLBMISS= 0x00000000
rommon 2 > i

Example 3: Watchdog hard reset with NPE-G1 and C7200-JC-PA with PA-POS-2OC3

Router will reload due to watchdog hard reset without traceback or crashinfo file

----------------- 
System Bootstrap, Version 12.2(8r)B, RELEASE SOFTWARE (fc1) TAC Support: http://www.cisco.com/tac Copyright (c) 2002 by cisco Systems, Inc. 


ROM: Rebooted by watchdog hard reset 

C7200 platform with 262144 Kbytes of main memory 
------------------ 

Workaround/Solution

The solution is to replace the C7200-JC-PA jacket card through the RMA process when the problem occurs. Although an upgrade program was previously provided to replace a potentially affected but otherwise functional product, the upgrade program is now over, and Cisco now only replaces a product that has actually failed. The standard RMA process must be used in order to replace the failed product.

As of approximately November 12, 2007, new products that were manufactured under ECO : E086054 are guaranteed to be free of this problem. Refer to How to Identify Hardware Levels below for instructions on how to view the serial number of an in-service product.

Check the following set of information to confirm the status of the jacket card - Revision number, Deviation Number for NOT Affected units and TAN number, Revision number and Serial Number for Affected units. Each condition listed in the table below is exclusive.

Please see Table 1 to confirm the status of the unit and the associated action.

Table 1: C7200-JC-PA Screen and Action criteria

Status

Hardware

Action

Not Affected

Rev 2.1 and higher and Units with Deviations D086026, D086578, D093490 for Rev 2.0 or lower

No Action Required

Affected

Rev 1.0 and Rev 2.0 without deviations

Fix on fail

Conditionally Affected

Rev 1.1 with certain serial numbers : check on Serial Number Validation Tool

Fix on fail

    

Serial Number : Check if Serial Number is Affected through the Serial Number Validation Tool link below.

Cisco Serial Number Validation Tool

How To Identify Hardware Levels

Issuing the show c7200 and show diag CLI command on slot 0 of the suspect 7200VXR series router will confirm the revision level of the installed C7200-JC-PA port adaptor jacket card.

Example of an "Affected" Unit Using"Show Diag" CLI Command to Determine C7200-JC-PA Revision with Issue

nrouter#show diag 0
Slot 0:
        C7200 PA Jacket Card Port adapter, 1 port
        Port adapter is analyzed 
        Port adapter insertion time 00:00:22 ago
        EEPROM contents at hardware discovery:
        Controller Type          : 1297
        Hardware Revision        : 1.0       <--[HW Revision 1.0, 2.0 and some of 1.1 are affected]
    Top Assy. Part Number    : 68-2619-03    <--[TAN 68-2619-04 or lower are affected]
    PCB Part Number          : 73-10416-03   <--[PCB P/N 73-10416-04 or lower are affected]
    Board Revision           : A0
        PCB Serial Number        : JAEXXXXXXXX  <--[Check Serial Number Validation tool if unit is Affected] 
        RMA History              : 00
        Fab Version              : 03
        Fab Part Number          : 28-7642-03
        Product Identifier (PID) : C7200-JC-PA
        Deviation Number         : 00
    Version Identifier (VID) : V02 
        EEPROM format version 4
        EEPROM contents (hex):
          0x00: 04 FF 40 05 11 41 01 00 87 44 0A 3B 03 82 49 28
          0x10: B0 03 42 41 30 C1 8B 4A 41 45 31 30 31 36 5A 57
          0x20: 45 33 04 00 02 03 85 1C 1D DA 03 CB 8B 43 37 32
          0x30: 30 30 2D 4A 43 2D 50 41 88 00 01 45 C5 89 56 30
          0x40: 31 20 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x50: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Example of a NOT Affected Unit Using the "Show Diag" CLI Command to Determine C7200-JC-PA Revision

-router#show diag 0
Slot 0:
        C7200 PA Jacket Card Port adapter, 1 port
        Port adapter is analyzed 
        Port adapter insertion time 00:00:22 ago
        EEPROM contents at hardware discovery:
        Controller Type          : 1297
        Hardware Revision        : 2.1            <--[HW Revision 2.1 or greater is a Good Unit and some HW Revision 1.1 are conditionally Good]
        Top Assy. Part Number    : 68-2619-05     <--[TAN 68-2619095 and greater indicates a Good Unit]
        PCB Part Number          : 73-10416-05    <--[PCB P/N 73-10416-05 and greater indicates a Good Unit]
        Board Revision           : A0
        PCB Serial Number        : JAEXXXXXXXX    <--[Check with Serial Number Validation Tool]
        RMA History              : 00
        Fab Version              : 03
        Fab Part Number          : 28-7642-03
        Product Identifier (PID) : C7200-JC-PA
        Deviation Number         : D086026    <--[Units with Deviations D086026, D086578, D093490 indicate a Good Unit]
        Version Identifier (VID) : V02            <--[HW Version 2.1 is Good Unit]
        EEPROM format version 4
        EEPROM contents (hex):
          0x00: 04 FF 40 05 11 41 01 00 87 44 0A 3B 03 82 49 28
          0x10: B0 03 42 41 30 C1 8B 4A 41 45 31 30 31 36 5A 57
          0x20: 45 33 04 00 02 03 85 1C 1D DA 03 CB 8B 43 37 32
          0x30: 30 30 2D 4A 43 2D 50 41 88 00 01 45 C5 89 56 30
          0x40: 31 20 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x50: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x60: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
          0x70: FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.