Cisco Catalyst 8500 Series Campus Switch Routers

February 16, 2000

Products Affected

Problem Description

Device hangs on an attempt to reprogram ROMMON.

To follow the bug ID link below and see detailed bug information, you must be a registered user and you must be logged in.

Background

The ROMMON images on the Catalyst 8540 route processors (RPs) are designed to be upgraded using the reprogram command. In roughly 10 percent of the attempts to reprogram the ROMMON images on the RP, the system hangs after displaying the following message: Resetting in 3 seconds. This happens on a Catalyst 8540 CSR or MSR RP.

In rare circumstances, you can reset the CPU PIF. When this happens, the CPU (RP) no longer sends or receives traffic, although the switch may still pass traffic.

Problem Symptoms

The device hangs on an attempt to reprogram ROMMON.

In roughly 10 percent of the attempts to reprogram the ROMMON images on the RP, the system hangs after displaying the following message: Resetting in 3 seconds. The output from the switch follows:

gd83_m84_05#reprogram bootflash:RM.120.4.6.W5.13.srec rommon

WARNING: Old contents will be completely erased before new contents are programmed. Please stop everything else that is running and save any important information before proceeding. 

!!! Once programming is complete, system will be automatically reset and boot using new rommon code. 

Reprogram rommon flash? [yes/no]: yes 
Proceed with reload? [confirm]
Verifying file... Done. Records processed: 6117, Last address: 0x2FC50
Device in socket is 29F040, size = 524288
Flash initialized
Verifying sectors...
Erasing chip : 29F040... Done. Successfully erased 29F040 
Downloading srecords...............
Done
Resetting in 3 seconds...

The system hangs here.

CPU PIF Problem

The problem symptom is similar to having a port-stuck error on every port on the system, including ports that are shutdown or not being used. The following command is a quick and easy way to check:

show controller controller 0 (2 iterations)

Check if the CUBI TX (transmit) and RX (receive) are incrementing. If they aren't incrementing, the RP CPU PIF is hung or reset. An additional command you can use to check is:

show mmc pif_reg n 16

For a single CPU system,

RP in slot4 => n=0
RP in slot8 => n=4

For a redundant RP system,

RP in slot4(primary) => n=0
RP in slot8(primary) => n=4
RP in slot4(secondary) => n=0
RP in slot8(secondary) => n=4

Sample output follows:

NY31WE08R01#show mmc pif_reg 0 16
mod[0][16] = 0x60 <-----------****if its 0x0, then PIF reset is occurring, 0x60 is good.
vpi-mask:0 ts:0 te:1 re:1 pp:0 hg:0 hc:0 pl:0 

stat[0][16] = 0x200
stat-rclr[0][16] = 0x200
mask[0][16] = 0x4FF
rcc[0][16] = 0x5A53
rcc-rclr[0][16] = 0x5A53
tcc[0][16] = 0x985F
tcc-rclr[0][16] = 0x985F
rhec[0][16] = 0x0
rhec-rclr[0][16] = 0x0
tpec[0][16] = 0x0
tpec-rclr[0][16] = 0x0
pe[0][16] = 0xF000
pe-rclr[0][16] = 0xF000
tpae1[0][16] = 0x0
tpae0[0][16] = 0xF
rpae1[0][16] = 0x0
rpae0[0][16] = 0xF
pfsid[0][16] = 0x0
tfsp[0][16] = 0x0
rccx[0][16] = 0x0
rccx-shadow[0][16] = 0x2A7
tccx[0][16] = 0x0
tccx-shadow[0][16] = 0x162
irsr[0][16] = 0x0
cr[0][16] = 0xF03
ppec[0][16] = 0x0
ppec-rclr[0][16] = 0x0
cec[0][16] = 0x0
cec-rclr[0][16] = 0x0
smr[0][16] = 0x0
ds7:0 ole:0 vle:0 bme:0 16b:0 fe:0 rer:0 ul:0 
cgn:0 cck:0 ppe:0 mpe:0 
pvr[0][16] = 0x1822
nid7[0][16] = 0xFFFF
nid6[0][16] = 0xFFFF
nid5[0][16] = 0xFFFF
nid4[0][16] = 0xFFFF
nid3[0][16] = 0xFFFF
nid2[0][16] = 0xFFFF
nid1[0][16] = 0xFFFF
nid0[0][16] = 0xFFFF

NY31WE08R01#

If you see the failure, the show controller controller 0 and show epc queue output is as follows. Note that all cells are backed up towards the SRP side.

cs06_c84_06#show epc queue
INT X-INT VCI QCNT VCI QCNT 
Gi0/0/0 SRP 35 1792 437 0 
Gi0/0/0 SRP 36 494 438 0 
Gi0/0/1 SRP 35 1792 441 0 
Gi0/0/1 SRP 36 490 442 0 
Gi1/1/0 SRP 35 3574 439 0 
Gi1/1/0 SRP 36 301 440 0 
Gi1/1/1 SRP 35 3576 443 0 
Gi1/1/1 SRP 36 496 444 0 
Gi2/1/0 SRP 36 496 446 0 
Gi2/1/1 SRP 36 486 450 0 
Gi3/1/0 SRP 35 3572 447 0 
Gi3/1/0 SRP 36 490 448 0 
Gi3/1/1 SRP 35 3117 451 0 
Gi3/1/1 SRP 36 483 452 0 
Fa9/0/0 SRP 34 897 267 0 
Fa9/0/0 SRP 35 896 467 0 
Fa9/0/0 SRP 36 497 468 0 
Fa9/0/1 SRP 34 897 265 0 
Fa9/0/1 SRP 35 903 463 0 
Fa9/0/1 SRP 36 480 464 0 
Fa9/0/2 SRP 34 895 263 0 
Fa9/0/2 SRP 35 1798 459 0 
INT X-INT VCI QCNT VCI QCNT 
Fa9/0/2 SRP 36 480 460 0 
Fa9/0/3 SRP 34 895 261 0 
Fa9/0/3 SRP 35 896 453 0 
Fa9/0/3 SRP 36 480 454 0 
Fa9/0/4 SRP 34 897 275 0 
Fa9/0/4 SRP 35 1798 469 0 
Fa9/0/4 SRP 36 480 470 0 
Fa9/0/5 SRP 34 895 273 0 
Fa9/0/5 SRP 35 896 465 0 
Fa9/0/5 SRP 36 480 466 0 
Fa9/0/6 SRP 34 897 271 0 
Fa9/0/6 SRP 35 1801 461 0 
Fa9/0/6 SRP 36 480 462 0 
Fa9/0/7 SRP 34 897 269 0 
Fa9/0/7 SRP 35 896 457 0 
Fa9/0/7 SRP 36 480 458 0 
Fa9/0/8 SRP 34 897 283 0 
Fa9/0/8 SRP 35 1796 485 0 
Fa9/0/8 SRP 36 481 486 0 
Fa9/0/9 SRP 34 897 281 0 
Fa9/0/9 SRP 35 1840 481 0 
Fa9/0/9 SRP 36 480 482 0 
INT X-INT VCI QCNT VCI QCNT 
Fa9/0/10 SRP 34 897 279 0 
Fa9/0/10 SRP 35 895 477 0 
Fa9/0/10 SRP 36 489 478 0 
Fa9/0/11 SRP 34 895 277 0 
Fa9/0/11 SRP 35 1265 471 0 
Fa9/0/11 SRP 36 480 472 0 
Fa9/0/12 SRP 34 895 291 0 
Fa9/0/12 SRP 35 896 487 0 
Fa9/0/12 SRP 36 493 488 0 
Fa9/0/13 SRP 34 895 289 0 
Fa9/0/13 SRP 35 896 483 0 
Fa9/0/13 SRP 36 487 484 0 
Fa9/0/14 SRP 34 897 287 0 
Fa9/0/14 SRP 35 895 479 0 
Fa9/0/15 SRP 34 897 285 0 
Fa9/0/15 SRP 35 896 475 0 
Gi10/1/0 SRP 35 3574 455 0 
Gi10/1/0 SRP 36 480 456 0 
Gi10/1/1 SRP 35 3572 473 0 
Gi10/1/1 SRP 36 486 474 0 
Gi11/1/0 SRP 35 3571 489 0 
Gi11/1/0 SRP 36 486 490 0 
INT X-INT VCI QCNT VCI QCNT 
Gi11/1/1 SRP 35 3575 493 0 
Gi11/1/1 SRP 36 480 494 0 
Gi12/1/0 SRP 35 1795 491 0 
Gi12/1/0 SRP 36 479 492 0 
Gi12/1/1 SRP 35 3003 495 0 
Gi12/1/1 SRP 36 483 496 0 
cs06_c84_06# 

cs06_c84_06#show controller controller 0
printing the copy stats here... 
TxCopiedPkts : 113403678 
TxNonCopiedPkts :0 
RxCopiedPkts :112458960 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<
RxNonCopiedPkts :0 
Rx throttle_count :0 
cubi_busy :54181 
cubi_busy_max :6 
Island0: 611D4404 first Ctl address : 60D35660 
first blk address A80C6000(630)- A80FFE00(7FF) :total 1CF(464)
Island1: 611D4420 first Ctl address : 60D2EF20 
first blk address A8050000(280)- A80C3600(61B) :total 39B(924)
Rxcell count: 64, timer intrs: 0, latency: 100 us, mask: 0x6A800

cs06_c84_06#show controller c0
printing the copy stats here... 
TxCopiedPkts :113403678 
TxNonCopiedPkts :0 
RxCopiedPkts :112458960 <<<<<<<<<<<<<<<<<<<<<
RxNonCopiedPkts :0 
Rx throttle_count :0 
cubi_busy :54181 
cubi_busy_max :6 
Island0: 611D4404 first Ctl address : 60D35660 
first blk address A80C6000(630)- A80FFE00(7FF) :total 1CF(464)
Island1: 611D4420 first Ctl address : 60D2EF20 
first blk address A8050000(280)- A80C3600(61B) :total 39B(924)
Rxcell count: 64, timer intrs: 0, latency: 100 us, mask: 0x6A800  

Workaround/Solution

The workaround for CSCdp05103 is to reboot the switch. The solution to CSCdp05103 and CSCdp44579 is to upgrade the RP Field Programmable Gate Array (FPGA) Image to 4.7.

Get FFGA Image (registered customers only)

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email
• By telephone

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.