Field Notice: FN72187 - UCS 2408 IOM Upgrade/Downgrade Failure on Some Units Shipped in April 2021 - Replace on Failure
Available Languages
Notice
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Products Affected
| Affected Product Name | Description | Comments |
|---|---|---|
| UCS-IOM-2408 | UCS 2408 I/O Module (8 External 25Gb Ports, 32 Internal 10Gb | |
| UCS-IOM-2408= | UCS 2408 I/O Module (8 External 25Gb Ports, 32 Internal 10Gb | Part Alternate |
Defect Information
| Defect ID | Headline |
| CSCvy20701 | undefined |
Problem Description
Firmware upgrade or downgrade attempts on a subset of Cisco UCS-IOM-2408 Fabric Extender Expansion Modules shipped in April 2021 may fail and continuously repeat the firmware update process.
Background
A subset of UCS-IOM-2408 Fabric Extender Expansion Modules shipped in April 2021 will not allow a firmware upgrade or downgrade to complete successfully. This is a hardware issue.
Problem Symptom
Any upgrade or downgrade attempt of the 2408 I/O Module (IOM) firmware may fail and loop continuously. This will primarily manifest as a failure immediately after the initial installation to a Unified Computing System (UCS) domain.
If the IOM did not require upgrade or downgrade upon installation, the IOM will be functional until a firmware change is attempted. In this case, the Cisco Technical Assistance Center (TAC) will not be able to access the IOM’s debug kernel shell.
During Upgrade Failures
Cisco UCS Manager faults will show many occurrences of a message similar to "IOM 1/1 (A) is auto upgrading firmware".
The NX-OS CLI command show fex detail will repeat "Requesting satellite to download image".
The IOM /cmc/log/updated log will show:
NOCSN_updated-3-CMC 0:validate_image_signature:Image signature validation failed#012
NOCSN_updated-3-CMC 1067:check_magic:error reading key
NOCSN_updated-3-CMC 1067:check_magic:error reading key
Workaround/Solution
If an impacted UCS-IOM-2408 Fabric Extender Expansion Module fails, it should be replaced. Request a replacement by opening a case with Cisco TAC and providing the impacted SN.
How to Identify Affected Products
If installation has not been attempted, the UCS-IOM-2408 serial numbers can be collected from the Cisco Sales Order details when they were procured.
If installed, the UCS-IOM-2408 expansion module serial numbers can be collected from UCS Manager or from the NX-OS CLI.
UCS Manager - Click the Equipment tab and choose Chassis > Chassis X > IO Module Y for any UCS-IOM-2408 expansion module in question. The serial number will be listed under Properties.
NX-OS CLI - SSH to the Cluster Virtual IP address and enter this sequence of commands in order to show all IOM serial numbers in the system:
- connect nxos a
- show fex
- exit
- connect nxos b
- show fex
- exit
To determine if a serial number is affected, see the Serial Number Validation section of this field notice.
Serial Number Validation
Cisco provides the Serial Number Validation Tool to verify whether a device is impacted by this issue. To check the device, enter the serial number in the Serial Number Validation Tool.
Important: For security reasons, you must click the Serial Number Validation Tool link that is provided in this section. Do not copy and paste the link into a browser. Use of the Serial Number Validation Tool URL external to this field notice will fail.
Revision History
| Version | Description | Section | Date |
| 1.3 | Updated workaround/solution. | Workaround/Solution and How to Identify Affected Products | 2023-NOV-17 |
| 1.2 | Updated the Upgrade Program to Use Support Case Manager (SCM). | — | 2022-NOV-16 |
| 1.1 | Updated the Upgrade Program Replacement Tracking Number. | — | 2021-AUG-23 |
| 1.0 | Initial Release | — | 2021-JUN-08 |
For More Information
For further assistance or for more information about this field notice, contact the Cisco Technical Assistance Center (TAC) using one of the following methods:
Receive Email Notification About New Field Notices
To receive email updates about Field Notices (reliability and safety issues), Security Advisories (network security issues), and end-of-life announcements for specific Cisco products, set up a profile in My Notifications
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)