Field Notice: FN74355 - Cisco UCS M7 or Cisco HCI M7 Host May Lock Up Due to Error-Correcting Code During Runtime - Software Upgrade Recommended

Cisco UCS M7 Server or Cisco Compute Hyperconverged (HCI) M7 Server can become unresponsive due to an incorrectly handled error-correcting code (ECC) error. This issue is only applicable to Intel-based platforms that are running the releases specified above. 

The server will need to be rebooted to recover, although if another ECC occurs, the host can again become unresponsive.

ECC memory errors are handled by BIOS using code that executes in System Management Mode (SMM). SMM is a special execution mode in the x86 architecture that operates independently of the operating system, which is unaware of its activity. 

Under certain conditions, the BIOS code that is responsible for handling ECC errors may encounter an exception while executing in SMM.

When this occurs, the processor halts in SMM and cannot return to normal operation or respond to other interrupts, such as non-maskable interrupts (NMIs). As a result, the server becomes unresponsive.

The following Cisco UCS M7 Server models may become unresponsive if they are running BIOS Release 4.3.6b, 4.3.6c, 6.0.1a, or 6.0.1b and an ECC memory error occurs: 

• UCSX-210C-M7
• UCSX-410C-M7
• UCSC-C220-M7
• UCSC-C240-M7

When this issue occurs, the server host becomes unresponsive. However, the Cisco Integrated Management Controller remains accessible, and the system can be reset remotely using management tools.

In some cases, such as with single-bit ECC errors, the errors may not be logged in the system event logs but can still cause the host to become unresponsive.

This is related to the DDR5 memory architecture, where the DIMM manages the error internally using on-die ECC. To recover, the host must be rebooted.

The server will remain operational until the next ECC error occurs.

There is no workaround available for this issue on affected firmware releases.

To resolve this issue on a system that is running an affected firmware release, upgrade to the fixed release indicated in the following table:

Download the latest release of Cisco UCS B-Series and C-Series bundles using the hyperlinks in the First Fixed Release column of the preceding table. 

For information on upgrading using Cisco Intersight, see https://intersight.com/help/saas/resources/Upgrading_Server_Firmware_Using_Firmware_Policy.

Identify Impacted Servers That Are Claimed in Cisco Intersight

Option 1

1. Log in to Cisco Intersight.
2. Click the Advisories icon (the loudspeaker icon located at the top right, near the Profile menu).
3. In the Advisories window, click the Field Notices tab.
4. Locate and select the Field Notice with the same name as this advisory.
5. Go to the Affected Devices section to view and identify servers that are impacted by this issue.

Option 2

From the navigation bar in Cisco Intersight, go to the Operate section, then choose the Servers menu. To identify servers that may be impacted based on their firmware release, use the Filters option to search for affected Server Model and Bundle Version. 

Click on a server, then scroll down in the left scroll window and search for Bundle Version to confirm there too. 

Identify Impacted Servers Using UCS Manager

If using a host firmware policy:

In the Equipment tab, choose Chassis > Servers > Server X. Click Installed Firmware. From there, check the package version for impact. 

If not using a host firmware policy:

In the Equipment tab, choose Chassis > Servers > Server X. Click Installed Firmware. If the Running Version and Startup Version of BIOS are 4.3.6b, 4.3.6c, 6.0.1a, 6.0.1b then the server is on impacted firmware. 

Identify Impacted Servers Using Standalone Cisco Integrated Management Controller

Upon login, on the Dashboard page, check the BIOS Version and Firmware Version. If the BIOS Version is 4.3.6b, 4.3.6c, 6.0.1a, or 6.0.1b or the Firmware Version is affected, the server could be impacted.