Field Notice: FN - 70536 - NCS4016-FC2-M Fabric Card Fails to Power On After Reload and/or Online Insertion and Removal - Software Upgrade Recommended

Available Languages

Updated:July 29, 2020
Document ID:FN70536

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision Publish Date Comments
1.0
22-Jul-20
Initial Release

Products Affected

Affected Product ID Comments
NCS4016-FC2-M
TAN: 800-45344-05 and higher are not impacted
NCS4016-FC2-M=
TAN: 800-45344-05 and higher are not impacted

Defect Information

Defect ID Headline
CSCvr61462 Enable Ancalgon FC LTC periodic refreshment in s/w to handle LTC device eeprom corruption
CSCvq63247 Ancalgon : Standby Powerzone failed to Powerup

Problem Description

A limited number of NCS4016-FC2-M Fabric Cards (FCs) might fail to boot/power on after a reload and/or Online Insertion and Removal (OIR) during the software upgrade to Cisco IOS XR Software Release 6.1.42, 6.5.26, or 6.5.28.

Background

This failure is observed due to an issue related to the EEPROM of a power device component in the unit.

Problem Symptom

This issue is observed only in chassis/nodes that have an NCS4016-FC2-M FC installed. If the NCS4016-FC2-M FC is functional, it will continue to work even with the improper EEPROM if the FC is not reloaded, no OIR is done, and the chassis is not reloaded.

The symptom is broadly classified as follows:

  • The affected FC might not work if the node or the FC is reloaded or an OIR is completed.
  • In some cases the EEPROM might be corrupted. In such cases, the affected FC cannot be recovered. The only recommendation is to request a replacement.

Workaround/Solution

There is no solution yet. Workarounds can be used to address these scenarios:

  • Scenario 1. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.1.42
  • Scenario 2. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.5.26 and Later Versions with Refresh Software Maintenance Upgrade (SMU) (ncs4k-sysadmin-6.5.26.CSCvr61462.tar)
  • Scenario 3. FC on Node/Chassis Already Runs Cisco IOS XR Software Release 6.5.26 Without Refresh SMU (ncs4k-sysadmin-6.5.26.CSCvr61462.tar)
  • Scenario 4. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.5.28 with Refresh SMU (ncs4k-sysadmin-6.5.28.CSCvr61462.tar)

Scenario 1. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.1.42

The objective of this scenario is to execute the workaround script that updates the firmware.

Note: Do not reload/OIR/power cycle the chassis or any cards during the upgrade operation in progress.

This procedure should be completed every six months.

It is mandatory that this procedure be completed during insertion of a new FC, Hardware Version ID -03 and lower, into the chassis/node that runs Cisco IOS XR Software Release 6.1.42.

Script Input Options:

  • The script expects to be run only on the 6142 version and all production SMUs to be installed.
  • The script expects all four FCs to be in the Operational state.

If the previous conditions are not met, the script will fail and exit.

In order to run the script, complete these steps:

  1. Enter admin on the console to go to the admin console.
  2. Enter the show controller ccc inventory summary command to get the master CCC RP slot.

    Example output:

    0/RP1    NCS4K-RP (master)       22    CAT1934B03K    0.1   CARD_READY

  3. Enter the attach location 0/RP<master ccc rp num> command to attach to the master CCC RP.
  4. Enter the run chvrf 0 bash command to go the the bash shell.
  5. Copy the script file to the bash shell of the master RP.
  6. Enter the chmod +x script_file_name command to change the mode of the script file.

    Example:

    chmod +x ccc_ltc_verify_upgrade_script.sh all
  7. Enter the ./<script_file_name> command to run the script.

    Example:

    ./ccc_ltc_verify_upgrade_script.sh all

After successful completion of the script, this message is displayed.

LTC MOP is completed

Scenario 2. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.5.26 and Later Versions with Refresh SMU (ncs4k-sysadmin-6.5.26.CSCvr61462.tar)

The objective of this scenario is to install the SMU that updates the firmware.

Software Download

Filename: ncs4k-sysadmin-6.5.26.CSCvr61462.tar

When an affected FC is plugged into the chassis, complete these steps in order to request a refresh:

  1. Enter admin on the console to go to the admin console.
  2. Enter the show controller ccc inventory summary command to get the master CCC RP slot.

    Example output:

    0/RP1    NCS4K-RP (master)       22    CAT1934B03K    0.1   CARD_READY

  3. Enter the attach location 0/RP<master ccc rp num> command to attach to the master CCC RP.
  4. Enter the run chvrf 0 bash command to go the the bash shell.
  5. Enter the /opt/cisco/calvados/sbin/ccc_driver_client command.
  6. Choose option 19] LTC refresh feature Disable/Enable/Refresh.
  7. Choose option 3] Refresh the LTC. After a successful refresh operation, this message is displayed. 
    LTC Refresh requested successfully
  8. Press CTRL+C to go back to the bash shell.

Scenario 3. FC on Node/Chassis Already Runs Cisco IOS XR Software Release 6.5.26 Without Refresh SMU (ncs4k-sysadmin-6.5.26.CSCvr61462.tar)

The objective of this scenario is to execute the workaround script that updates the firmware.

Notes:

  • The LTC Refresh Maintenance Operation Protocol (MOP) is to be run only on 6526 nodes that do not have the refresh SMU installed.
  • Do not reload/power cycle the chassis/RP/LC/FC cards when you perform a refresh MOP, otherwise the FC cards might fail permanently.
  • Run the script from the server which has Python. The server must be reachable to the network.

Run the python ltc_refresh.py script from the csh prompt.

Example:

bgl-ads-2147:130> python ltc_refresh.py

After successful completion of the script, this message is displayed.

Successfully completed LTC Refresh

Scenario 4. FC on Node/Chassis Runs Cisco IOS XR Software Release 6.5.28 with Refresh SMU (ncs4k-sysadmin-6.5.28.CSCvr61462.tar)

The objective of this scenario is to install the SMU that updates the firmware.

Software Download

Filename: ncs4k-sysadmin-6.5.28.CSCvr61462.tar

Prerequisite: Install the LTC refresh SMU.

Whenever an FC is plugged into the chassis, complete these steps in order to refresh the Ancalgon LTC devices:

  1. Enter admin on the console to go to the admin console.
  2. Enter the show controller ccc inventory summary command to get the master CCC RP slot.

    Example output:

    0/RP1    NCS4K-RP (master)       22    CAT1934B03K    0.1   CARD_READY

  3. Enter the attach location 0/RP<master ccc rp num> command to attach to the master CCC RP.
  4. Enter the run chvrf 0 bash command to go the the bash shell.
  5. Enter the /opt/cisco/calvados/sbin/ccc_driver_client command.
  6. Choose option 19] LTC refresh feature Disable/Enable/Refresh.
  7. Choose option 3] Refresh the LTC. After a successful refresh operation, this message is displayed. 
    LTC Refresh requested successfully
  8. Press CTRL+C to go back to the bash shell.

How To Identify Affected Products

Products can primarily be identified by the Top Assembly Number (TAN), and also by the Product ID/Version ID (PID/VID) and Common Language Equipment Identifier (CLEI) labels as shown in this image. Units with TAN 800-45344-05 and higher are not impacted.

The affected products can also be identified using the Serial Number Validation Tool provided in the Serial Number Validation section.

Serial Number Validation

This field notice provides the ability to determine if the serial number(s) of a device is impacted by this issue. In order to verify your serial number(s), enter it in the Serial Number Validation Tool.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.

Was this Document Helpful?

FeedbackFeedback

Contact Cisco