Field Notice: FN - 72272 - ADDDC VLS Does Not Trigger When Expected - BIOS/Firmware Upgrade Recommended

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	19-Nov-21	Initial Release
1.1	14-Jan-22	Updated the Background, Workaround/Solution, and How to Identify Affected Products Sections

Products Affected

Affected OS Type	Affected Software Product	Affected Release	Affected Release Number	Comments
NON-IOS	Unified Computing System (UCS) Server Firmware	4.1	4.1(3b)
NON-IOS	Unified Computing System (UCS) Server Software Bundle	4.1	4.1(3b), 4.1(3c)

Defect Information

Defect ID	Headline
CSCvx93920	ADDDC VLS does not trigger when expected

Problem Description

Intel-based Cisco Unified Computing System (UCS) M5 servers that run specific BIOS firmware versions will not trigger Adaptive Double Device Data Correction (ADDDC) Sparing or Virtual Lock Step (VLS) on memory modules that could potentially fail. As a result, a higher rate of runtime uncorrectable memory errors might be experienced when you run the impacted firmware versions.

Background

Intel Xeon Scalable processors and 2nd Gen Xeon Scalable processors implemented changes in Single Device Data Correction (SDDC). SDDC is a fundamental Intel Reliability, Availability, and Serviceability (RAS) feature available on all Cisco platforms. As a result of these architectural changes and dual in-line memory module (DIMM) errors, there is a difference in which errors will be corrected between the previous generation of processors and the Xeon Scalable processor family generation.

The latest Intel microcode and BIOS enhancements improve management of memory errors by enabling additional Memory RAS features such as ADDDC Sparing. ADDDC Sparing is now the default Memory RAS configuration on Cisco UCS M5 servers with Intel Xeon Scalable processors.

For more details on Cisco UCS Server M5 Memory features, see Cisco UCS HX M5 Memory Technical Overview - Memory RAS Features.

Problem Symptom

On impacted server BIOS firmware versions, the BIOS is not notified of each Correctable ECC error and cannot track error locations or trigger VLS for failing memory regions. As a result, a higher rate of runtime uncorrectable memory errors might be experienced when you run the impacted firmware versions.

All Intel-based M5 Server BIOS included in the 4.1(3b) and 4.1(3c) UCS Server firmware bundles are impacted. Furthermore, certain UCS Rack Server Standalone Firmware is also impacted. The exact BIOS versions bundled into the 4.1(3b) and 4.1(3c) firmware bundles are:

• ucs-b200-m5-bios.B200M5.4.1.3e.0.1210201721.bin
• ucs-b480-m5-bios.B480M5.4.1.3e.0.1210201720.bin
• ucs-c220-m5-bios.C220M5.4.1.3e.0.1210201720.bin
• ucs-c240-m5-bios.C240M5.4.1.3e.0.1210201753.bin
• ucs-c480-m5-bios.C480M5.4.1.3d.0.1210201819.bin
• ucs-s3260-m5-bios.S3X60M5.4.1.3d.0.1210201753.bin

These Cisco UCS Rack Server Standalone Firmware Host Upgrade Utilities (HUU) are impacted:

• ucs-c240m5-huu-4.1.3b.iso
• ucs-c220m5-huu-4.1.3b.iso
• ucs-c480m5-huu-4.1.3b.iso
• ucs-s3260-huu-4.1.3b.iso

This issue is specific to Intel-based M5 server BIOS firmware. Only the specific platforms listed in this section are impacted. Furthermore, this issue is specific to the BIOS code for the specific platforms (for example, if you run the 4.1(3b) code on your UCS Fabric Interconnect, this does not put your servers at risk for this defect).

Workaround/Solution

There is no workaround available on the impacted BIOS - a software upgrade is required. If you currently run an impacted Server Firmware Bundle, Cisco recommends to upgrade to a minimum of 4.1(3d) UCS Manager server bundle and 4.1(3c) Standalone Firmware HUU.

For more details on how to perform a server software upgrade, see the Cisco UCS Manager Firmware Management Guide.

How To Identify Affected Products

Intersight

In order to confirm the firmware version that your servers run in Intersight, search for the Firmware Version string. Values separated by a comma are treated as an "or" for search purposes. For example, navigate to the Operate > Servers page and create the filter "Firmware Version 4.1(3b),4.1(3c),4.1(33a),4.1(33b)".

Note: Intersight Management Module (IMM) servers display their Cisco Integrated Management Controller (CIMC) firmware version, while Unified Computing System Manager (UCSM) servers display their Firmware bundle version. When you search for an impacted server, ensure that you copy the string exactly. This example shows two servers that run the impacted 4.1(3b) release:

 

UCSM CLI

For UCSM-managed servers, enter this command from the root of the UCSM CLI in order to confirm the package version and BIOS version of your servers:

FI-6332-A# show server firmware
Server 4/1:
    CIMC:
        Running-Vers: 4.1(31e)
        Package-Vers: 4.1(1d)B   <--- 4.1(1d) Firmware Bundle, not impacted
        Update-Status: Ready
        Activate-Status: Ready
...
    BIOS:
        Running-Vers: B200M5.4.1.1d.0.0609200543   <---4.1.1d BIOS version, not impacted
        Package-Vers: 4.1(1d)B
        Update-Status: Ready
        Activate-Status: Ready

The BIOS version of a UCS Standalone Rack Server is displayed on the CIMC Chassis Summary screen. For example:

The impacted server firmware will read “Version: 4.1(3b)”

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email or telephone

Receive Email Notification For New Field Notices

My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.