Field Notice: FN - 64127 - CRS-X Line Cards with New DRAM Fail to Boot to XR-Run State - Software Upgrade Recommended

Available Languages

Updated:June 4, 2018

Document ID:FN64127

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	12-Jan-17	Initial Release
10.0	14-Nov-17	Migration to new field notice system
10.1	04-Jun-18	Fixed Broken Image Links

Products Affected

Affected Product ID	Comments
CRS-FP-X
CRS-FP-X-L
CRS-FP200G
CRS-FP200G-L
CRS-FP200G=
CRS-FP400G
CRS-LSP-X
CRS-LSP400G
CRS-MSC-X
CRS-MSC-X-L
CRS-MSC200G=
CRS-MSC400G

Defect Information

Defect ID	Headline
CSCvf34445	There were no defects filed with this field notice at the time of publication.

Problem Description

CRS-X line cards (CRS-MSC-X, CRS-MSC400G, CRS-FP-X, CRS-FP400G=, CRS-FP400G, CRS-LSP-X, CRS-LSP400G, CRS-MSC-X-L, CRS-MSC200G=, CRS-FP-X-L, CRS-FP200G, CRS-FP200G-L, and CRS-FP200G=) built with new DRAM memory fail to boot to the XR-Run state in XR releases of 5.3.3, 5.1.4, 5.1.3, 5.3.1, 5.3.2, 5.1.1, 5.1.2 and 5.3.4.

Background

In the manufacture of line cards CRS-MSC-X, CRS-MSC400G, CRS-FP-X, CRS-FP400G=, CRS-FP400G, CRS-LSP-X, CRS-LSP400G, CRS-MSC-X-L, CRS-MSC200G=, CRS-FP-X-L, CRS-FP200G, CRS-FP200G-L, and CRS-FP200G=, it is required to use a new DRAM memory since the existing one is End of Life.

It was found that this new DRAM memory on the board fails to boot up due to Network Processor Unit (NPU) driver initialization errors. The unit requires a reload Software Maintenance Upgrade (SMU) in order to make the boards work properly.

Problem Symptom

The line card fails to boot to the XR-Run state. The unit goes for continuous reload and reaches the IN-Reset state.

The sample logs as the signature of this issue on XR 533 release are:

Reboot reason:

'NPU initialization failed'

The sample syslogs for issue symptoms are:

LC/0/2/CPU0:Mar 10 06:33:35.070 : npu_driver[271]: %L2-NPU-2-DRAM_INIT_FAILURE : NPU_INST 0: DRAM instance 10 initialization failed for memory type PBS at the phase control init LC/0/2/CPU0:Mar 10 06:33:46.417 : npu_driver[271]: %L2-NPU-2-DRAM_INIT_FAILURE : NPU_INST 0: DRAM instance 10 initialization failed for memory type PBS at the phase final LC/0/2/CPU0:Mar 10 06:33:51.144 : npu_driver[271]: %L2-NPU-3-ERROR_MSG : *** NPU0: FATAL error at {line:831}: failure in GIF block, rc=-1463657962, 'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument LC/0/2/CPU0:Mar 10 06:33:51.145 : npu_driver[271]: %L2-NPU-3-INIT_ERROR_MSG : NPU0: Init Failed. Caused by:'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument LC/0/2/CPU0:Mar 10 06:33:51.145 : npu_driver[271]: %L2-NPU-3-ERR_EXIT : Exit on error: Failed asic power-on reset init: Caused by 'NPU Driver' detected the 'fatal' condition 'at hw lib layer: Internal object': Invalid argument : pkg/bin/npu_driver : (PID=106555) : -Traceback= 420956c 420a14c 421071a df6996e cb84050 LC/0/2/CPU0:Mar 10 06:33:51.979 : npu_driver[271]: %L2-NPU-6-INFO_MSG : NPU0: *** Total Valid NPUs:2 (max=2) *** LC/0/2/CPU0:Mar 10 06:33:58.339 : sysmgr[82]: %OS-SYSMGR-2-MANAGED_REBOOT : reboot to be managed by process (platform_mgr_common) reason (ASIC pat instance 0 in critical alarm) LC/0/2/CPU0:Mar 10 06:33:58.522 : platform_mgr_common[286]: %PLATFORM-HFR_PM-3-COMMUNICATION_ERR : Device pse #0 has a fault=CRITICAL. action: Rebooting node

Workaround/Solution

If the listed line cards show symptoms of not booting up due to NPU driver initialization errors, then install the recommended and mandatory reload SMU which initializes the register settings properly for the new DRAM memory module during initial boot up of the line card.

Release 5.1.1

If you run release 5.1.1, upgrade with this SMU:

Description: Topaz LC with Atris 30nm memory failed to boot to XR state

Release: 5.1.1

Release Date: 04/25/16

File Name: hfr-px-5.1.1.CSCuy71498.pie

Release 5.1.2

If you run release 5.1.2, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.1.2

Release Date: 04/25/16

File Name: hfr-px-5.1.2.CSCuy71498.pie

Release 5.1.3

If you run release 5.1.3, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.1.3

Release Date: 04/14/16

File Name: hfr-px-5.1.3.CSCuy71498.pie

Release 5.1.4

If you run release 5.1.4, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.1.4

Release Date: 04/16/16

File Name: hfr-px-5.1.4.CSCuy71498.pie

Release 5.3.1

If you run release 5.3.1, upgrade with this SMU:

Description: Topaz LC with Atris 30nm memory failed to boot to XR state

Release:5.3.1

Release Date: 04/25/16

File Name: hfr-px-5.3.1.CSCuy71498.pie

Release 5.3.2

If you run release 5.3.2, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.3.2

Release Date: 04/25/16

File Name: hfr-px-5.3.2.CSCuy71498.pie

Release 5.3.3

If you run release 5.3.3, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.3.3

Release Date: 4/10/2016

File Name: hfr-px-5.3.3.CSCuy71498.pie

Release 5.3.4

If you run release 5.3.4, upgrade with this SMU:

Description: NCS6K/CRS-X LC with Atris 30nm memory failed to boot to XR-Run state

Release:5.3.4

Release Date: 12/23/2016

File Name: hfr-px-5.3.4.CSCuy71498.pie

How To Identify Affected Products

Complete one of these steps in order to determine if your product is the updated version, Version 03:

If the card boots up, enter the admin show diag command.

For CRS-X line card Product IDs (PIDs), mentioned previously, the new Version ID (VID) displayed is V03.

An example of the command output is shown here:

RP/0/RP0/CPU0:rtr(admin)#sh diag 0/1/CPU0
CARD 0/1/* : Cisco CRS Series Modular Services Card 400G
MAIN: board type 50006c
800-38514-05 rev A0
dev 139822
S/N CAT1744T0A5
PCA: 73-14750-09 rev A0
PID: CRS-MSC-X
VID: V03

On the face plate the PID/VID label displays the new Version ID - V03. This picture is a representation of the label.
Look for a special label on the packaging box which indicates this new board contains 30nm memory and a link to the Field Notice. This picture is a representation of the label.

For any assistance please contact Technical Assistance Center.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.

Was this Document Helpful?

Feedback

Contact Cisco

Open a Support Case
(Requires a Cisco Service Contract)