Field Notice: FN - 72541 - Some NCS 540 Routers Might Experience Multiple Reloads or Show No Console - Software Upgrade Recommended

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.1	01-Mar-23	Updated the Workaround/Solution Section
1.0	08-Feb-23	Initial Release

Products Affected

Affected Product ID	Comments
N540X-16Z4G8Q2C-D
N540X-16Z4G8Q2C-D=	Part Alternate
N540X-16Z4G8Q2C-A
N540X-16Z4G8Q2C-A=	Part Alternate
N540X-12Z16G-SYS-D
N540-12Z16G-SYS-D=	Part Alternate
N540X-12Z16G-SYS-A
N540-12Z16G-SYS-A=	Part Alternate
N540-28Z4C-SYS-D
N540-28Z4C-SYS-D=	Part Alternate
N540-28Z4C-M-D
N540-28Z4C-SYS-A
N540-28Z4C-SYS-A=	Part Alternate
N540-28Z4C-M-A
N540-12Z20G-SYS-D
N540-12Z20G-SYS-D=	Part Alternate
N540-12Z20G-SYS-A
N540-12Z20G-SYS-A=	Part Alternate
N540X-16Z8Q2C-D
N540X-16Z8Q2C-D=	Part Alternate

Defect Information

Defect ID	Headline
CSCvz19041	NCS540 Medium Density Router: Updated ADM ver 1.5 [Refer Summary/Enclosure for list of changes]
CSCvz15167	NCS540-Medium Density Router: Disable C-state for CPU C3708 & C3508 skew
CSCwd50606	NC540-Medium Desnisty Router: Bios - Disable clock gating of branch prediction blocks for all cores.

Problem Description

Some Network Convergence System (NCS) 540 Series routers might encounter no console or be subjected to multiple reloads with Cisco IOS^® XR Release 7.0.x, Release 7.1.x, Release 7.2.x, Release 7.3.x , and Release 7.4.x. Routers in this state will also display blinking green LED lights on the front panel.

Background

The root cause of the issue is an updated firmware requirement for the CPU. Routers currently shipped from manufacturing have the fix incorporated.

Problem Symptom

These symptoms might be exhibited with Cisco IOS XR Release 7.0.x, Release 7.1.x, Release 7.2.x, Release 7.3.x, and Release 7.4.x.

• A router with no reboot history might experience multiple reloads. - CSCvz15167
• A router might be subjected to an unexpected reload abruptly with a Kernel dump crash. - CSCwd50606
• The PWR and FAN LEDs, on the front panel of the unit, are blinking green. - CSCvz19041
• The router might also reboot continuously with a Sequenced Packet Protocol (SPP) initialization failure message. - CSCvz19041

Workaround/Solution

Routers with the issues must be upgraded to Cisco IOS XR Release 7.3.2 or later, where the fixes are incorporated.

The table listed provides the Software Maintenance Upgrades (SMUs) and base releases where the fixes are enabled for corresponding Defect IDs.

Cisco IOS XR Release	CSCvz19041	CSCvz15167	CSCwd50606
7.0.1	-	-	-
7.0.2	-	SMU- ncs540l-7.0.2.CSCvz15167.tar	-
7.1.1	-	-	-
7.1.2	-	SMU-ncs540l-7.1.2.CSCvz15167.tar	-
7.2.1	-	-	-
7.2.2	-	-	-
7.3.1	ncs540l-7.3.1.CSCvx66941.tar	SMU-ncs540l-7.3.1.CSCvz15167.tar	-
7.3.2	ncs540l-7.3.2.CSCvx66941.tar	NCS540-iosxr-7.3.2.tar	SMU-ncs540l-7.3.2.CSCwd50606.tar
7.4.1	-	SMU-ncs540l-7.4.1.CSCvz15167.tar	-
7.4.2	NCS540-iosxr-7.4.2.tar	NCS540-iosxr-7.4.2.tar	SMU- ncs540l-7.4.2.CSCwd50606.tar
7.5.1	NCS540-iosxr-7.5.1.tar	NCS540-iosxr-7.5.1.tar	-
7.5.2	NCS540-iosxr-7.5.2.tar	NCS540-iosxr-7.5.2.tar	SMU-ncs540l-7.5.2.CSCwd50606.tar
7.6.1	NCS540-iosxr-7.6.1.tar	NCS540-iosxr-7.6.1.tar	-
7.6.2	NCS540-iosxr-7.6.2.tar	NCS540-iosxr-7.6.2.tar	SMU-ncs540l-7.6.2.CSCwd50606.tar
7.7.1	NCS540-iosxr-7.7.1.tar	NCS540-iosxr-7.7.1.tar	-
7.7.2	NCS540-iosxr-7.7.2.tar	NCS540-iosxr-7.7.2.tar	SMU-ncs540l-7.7.2.CSCwd50606.tar
7.8.1	NCS540-iosxr-7.8.1.tar	NCS540-iosxr-7.8.1.tar	-

 

Cisco does not recommend a Return Material Authorization (RMA) for the problem symptoms described, as routers can resume normal operation once upgraded to the Cisco IOS XR release that has the fix.

Note: The fixes cannot be backported as a SMU for earlier releases. An upgrade is required.

How To Identify Affected Products

Affected products can be identified by these methods:

• On a router that experiences multiple reloads, enter the run pcimemread 85112000 4 command after a reload.

  The output of the command displays the value 0x02.

• The router continues to reload due to a SPP initialization failure. This error message is displayed:

  RP/0/RP0/CPU0:MMM DD HH:MM:SS UTC: spp[115]: %PKT_INFRA-spp-4-INIT_FAILED : Failed to initialize SPP: Failed to initialize device 0x7fe109b5

• The router crashes and these messages are displayed:

  [144065.132891] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[144065.141349] BUG: unable to handle kernel paging request at 00000000aa6cf4e9
[144065.149524] IP: [<00000000aa6cf4e9>] 0xffffa465c9067bc8
[144065.155641] PGD 27e59f067 PUD 27e5a4067 PMD 269267067 PTE 8000000269707063
[144065.164006] Oops: 0011 [#1] SMP
[144065.167687] Modules linked in: nf_tables_ipv6 nf_tables_ipv4 nf_tables nfnetlink exi_driver(O) ubifs bridge stp linux_user_bde(O) uio_dma_proxy m25p80 spi_nor cisco_fpga_shutdown_handler(O) cisco_fpga_qspi(O) pnd2_edac edac_core x86_pkg_temp_thermal coretemp crct10dif_pclmul crct10dif_common linux_kernel_bde(O) ixgbe(O) tpm_crb cisco_fpga_setup(O) uio_pci_proxy shmem_pg(O) nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack mtdoops jc42 i2c_imc fuse dm_crypt dm_thin_pool dm_persistent_data dm_bufio dm_bio_prison virtio_mmio virtio_blk virtio_pci virtio_ring virtio dm_mod
[144065.232831] CPU: 3 PID: 10584 Comm: cat Tainted: G           O    4.8.28-WR9.0.0.26_cgl #1
[144065.242143] Hardware name: Insyde Harrisonville/Type2 - Board Product Name1, BIOS 00.01.043 02/11/2022
[144065.252603] task: 000000000d67ae43 task.stack: 00000000a327f4d1
[144065.259338] RIP: 0010:[<ffffa465c9067bc8>]  [<00000000aa6cf4e9>] 0xffffa465c9067bc8
[144065.268132] RSP: 0018:ffffa465c9067bb8  EFLAGS: 00010292
[144065.274200] RAX: 0000000000000034 RBX: 000000006b010c16 RCX: 0000000000000001
[144065.282273] RDX: 0000000000000001 RSI: ffffffff8bc7006a RDI: 0000000000000000
[144065.290345] RBP: 000000000000a986 R08: 0000000000000020 R09: 0000000000000000
[144065.298419] R10: 0000000000ffff0a R11: 000000000000000f R12: ffffa11b30561500
[144065.306491] R13: ffffa11a7ef5d870 R14: 0000000000000001 R15: ffffa11a81639200
[144065.314564] FS:  00007f8ef4561480(0000) GS:ffffa11b3f6c0000(0000) knlGS:0000000000000000
[144065.323688] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[144065.330232] CR2: ffffa465c9067bc8 CR3: 00000001371f6000 CR4: 0000000000340670
[144065.338306] Stack:
[144065.340744]  ffffffff8b220b55 ffffa11b30561500 ffffa465c9067c28 ffffffff8b220bc3
[144065.349673]  ffffa11a00000030 ffffa465c9067c68 ffffa465c9067bf8 000000006b010c16
[144065.358608]  00007fbb57772000 ffffa465c9067c88 00007fbb57772000 00007fbb57774000
[144065.367537] Call Trace:
[144065.370460]  [<00000000f672b78e>] ? seq_vprintf+0x35/0x50
[144065.376624]  [<00000000f73cd189>] ? seq_printf+0x53/0x70
[144065.382692]  [<0000000088818067>] ? show_map_vma+0x116/0x260

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email or telephone

Receive Email Notification For New Field Notices

My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.