Field Notice: FN74064 - UCS X Series Server Intermittently Fails to Boot with UCSX-X10C-RAIDF-D / UCSX-X10C-RAIDF - Software Upgrade Recommended

The Cisco UCS X Series node fails to boot with UCSX-X10C-RAIDF-D or UCSX-X10C-RAIDF front mezzanines installed. This issue affects the following Cisco UCS X Series nodes with UCSX-X10C-RAIDF-D/UCSX-X10C-RAIDF if they are running affected older firmware:

• UCSX-210C-M6
• UCSX-210C-M7
• UCSX-410C-M7

The failure can occur if the server has not been powered up for more than 24 hours. The RAID controller that is installed in the X Series server has a Super Capacitor (SuperCap) that maintains power to the board and prevents data loss  in the event of a power off. The SuperCap inside UCSX-X10C-RAIDF-D slowly discharges while the server is powered off. This will cause the card power up to take longer than normal and create a fault. The server control circuit will not allow the server to power up with the UCSX-X10C-RAIDF/UCSX-X10C-RAIDF-D fault.

Upon initial insertion of an X210 or X410 server into a chassis, or if the server has been powered off for longer than 24 hours, the server may fail to power on with a critical power problem fault.

In the Intersight user interface, there will be a critical alarm similar to the one shown in the following image:

For Cisco UCS Managed servers, a critical F0868 fault like the following is generated: 

Severity: CriticalCode: F0868Last Transition Time: 2023-08-24T10:52:10.738ID: 280198Status: NoneDescription: Motherboard of server 1/7 (service profile: ) power: failedAffected Object: sys/chassis-1/blade-7/boardName: Compute Board Power FailCause: Power Problem

The following error may be seen in the System Event Log (SEL):

Platform alert PWR_SEQ_FAIL #0xa5 | Predictive Failure asserted | Asserted

Solution

To fix the issue that is described in this field notice, upgrade to a fixed software release, as shown in the following table:

Workaround

As a temporary workaround, power up the servers that failed using one of the following methods:

• Reboot the Cisco Integrated Management Controller (CIMC)
• Re-seat the server by removing the server, waiting for at least 20 seconds, and then re-inserting the server into the chassis.

Note: If a server that is running affected firmware is powered off for longer than 24 hours, it may once again be affected by this defect.

To reboot the CIMC from the Cisco UCS Manager (UCSM), do the following:

1. Choose the server that is affected.
2. Click Recover Server.
3. Choose Reset CIMC (Server Controller).

To reboot the CIMC from Intersight, do the following:

1. Click Actions.
2. Choose System.
3. Choose Reboot Management Controller.

Intersight-Managed Servers

To determine whether affected hardware is present, do the following:

1. On the affected blade, click Inventory.
2. Click Storage Controllers.
3. Choose Controller X (Raid).
4. Check for UCSX-X10C-RAIDF / UCSX-X10C-RAIDF-D.

To determine whether the hardware is running an affected firmware release, choose the affected server and scroll down on the left side to Firmware Version. Releases earlier than Release 5.2(0.230040) are affected.

UCSM-Managed Servers

To determine whether affected hardware is present, do the following:

1. Choose the affected server.
2. Click Inventory.
3. Click Storage.
4. Click Controller.
5. Choose Storage Controller SAS 1.
6. Check for the model number UCSX-X10C-RAIDF or UCSX-X10C-RAIDF-D.

To determine whether the server is running an affected firmware release, do the following:

1. Choose the affected server.
2. Click Installed Firmware.
3. Check if the CIMC controller is running a release earlier than Release 5.2(0.230040) (package version 4.32b B series bundle for UCSM on blades).

The specific power sequencer fault codes for M6 and M7 blades are different. The log locations are also different for Intersight-managed servers and UCS Managed servers. In a tech support bundle, the Power Failure log will have the following error messages:

X210 M6

**************************************************************************************************
UCD90320 Status Flags
**************************************************************************************************
FMEZZ_PWRGD :
TON_MAX Fault

X210 M7 and X410 M7

**************************************************************************************************
UCD90320 Status Flags
**************************************************************************************************
FMEZZ_BLD_CARD_PWRGD :
TON_MAX flag asserted

For an Intersight-managed server, the location of the logfile is:

intersight_ts.tgz\<date>_intersight_ts.tar\mnt\scratchpad\techsupport_pidXXXXX\tech_support-nvram.tar.gz\tech_support-nvram.tar\nv\etc\log\power_sys_logs\Fault_parsed.txt

For a UCSM-managed server, the logfile location is:

BCX_all\CIMCX_TechSupport.tar.gz\CIMCX_TechSupport.tar\mnt\scratchpad\techsupport_pidXXXX\CIMCX_TechSupport-nvram.tar.gz\CIMCX_TechSupport-nvram.tar\nv\etc\log\power_sys_logs\Fault_parsed.txt