Field Notice: FN - 70432 - Improved Memory RAS Features for UCS M5 Platforms - Software Upgrade Recommended

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	23-Aug-19	Initial Release
1.1	23-Sep-19	Updated the Products Affected and Workaround/Solution Sections
1.2	03-Oct-19	Updated the Workaround/Solution Section
1.3	18-Nov-19	Updated the Product Hierarchy Metatags Section
1.4	09-Mar-20	Updated the Products Affected, Problem Description, Background, and Problem Symptom Sections
1.5	15-May-20	Updated the Background and Workaround/Solution Sections
1.6	22-Jun-20	Updated the Products Affected Section
1.7	22-Jun-20	Updated the Products Affected Section
1.8	02-Oct-20	Updated the Products Affected, Background, and Workaround/Solution Sections
1.9	20-Feb-21	Updated the Problem Description, Background, Problem Symptom, and Workaround/Solution Sections
2.0	02-Apr-21	Updated the Workaround/Solution Section
2.1	19-Nov-21	Updated the Workaround/Solution Section

Products Affected

Affected OS Type	Affected Software Product	Affected Release	Affected Release Number	Comments
NON-IOS	Unified Computing System (UCS) Server Software Bundle	4.0	4.0(1a), 4.0(1b), 4.0(1c), 4.0(1d), 4.0(2a), 4.0(2b), 4.0(2d), 4.0(4a), 4.0(4b)	For FI-Attached (UCS Manager) Servers
NON-IOS	Unified Computing System (UCS) Server Software Bundle	3.2	3.2(1d), 3.2(2b), 3.2(2c), 3.2(2d), 3.2(2e), 3.2(2f), 3.2(3a), 3.2(3b), 3.2(3d), 3.2(3e), 3.2(3g), 3.2(3h), 3.2(3i), 3.2(3j), 3.2(3k), 3.2(3l), 3.2(3n), 3.2(3o)	For FI-Attached (UCS Manager) Servers
NON-IOS	Unified Computing System (UCS) Server Firmware	4.0	4.0(1a), 4.0(1b), 4.0(1c), 4.0(1d), 4.0(1e), 4.0(1g), 4.0(1h), 4.0(2c), 4.0(2d), 4.0(2f), 4.0(2g), 4.0(2h), 4.0(2i), 4.0(2k), 4.0(2l), 4.0(2m), 4.0(2n), 4.0(4b), 4.0(4c), 4.0(4d)	For Standalone (IMC SW) Servers
NON-IOS	Unified Computing System (UCS) Server Firmware	3.1	3.1(1d), 3.1(2b), 3.1(2c), 3.1(2d), 3.1(2e), 3.1(2g), 3.1(2i), 3.1(3a), 3.1(3b), 3.1(3c), 3.1(3d), 3.1(3g), 3.1(3h), 3.1(3i), 3.1(3j)	For Standalone (IMC SW) Servers

Defect Information

Defect ID	Headline
CSCvq38078	UCSM:Default option for "SelectMemory RAS configuration" changed to ADDDC sparing

Problem Description

Cisco Unified Computing System (UCS) M5 servers with certain Intel Xeon Scalable processors might experience a higher rate of runtime uncorrectable memory errors than previous generations with the default Single Device Data Correction (SDDC) Memory Reliability, Availability, and Serviceability (RAS) configuration.

Background

Intel Xeon Scalable processors and 2nd Gen Xeon Scalable processors implemented changes in SDDC. SDDC is a fundamental Intel RAS feature available on all Cisco platforms. As a result of these architectural changes and memory dual in-line memory module (DIMM) errors, there is a difference in which errors will be corrected between the previous generation of processors and the Xeon Scalable processor family generation.

The latest Intel microcode and BIOS enhancements improve management of memory errors by enabling additional Memory RAS features such as Adaptive Double Device Data Correction (ADDDC Sparing). ADDDC Sparing is now the default Memory RAS configuration on Cisco UCS M5 servers with Intel Xeon Scalable processors.

Problem Symptom

Due to memory DIMM errors and architectural changes in memory error handling on Intel Xeon Scalable processors (formerly code-named "Skylake Server") and 2nd Gen Intel Xeon Scalable processors (formerly code-named "Cascade Lake Server"), Cisco UCS M5 customers that experience memory DIMM errors might experience a higher rate of runtime uncorrectable memory errors than they experienced on previous generations with default SDDC Memory RAS mode.

Workaround/Solution

Cisco recommends that you upgrade to a Server Firmware Bundle that includes ADDDC Sparing to expand the memory error coverage. Refer to this table for supported and recommended firmware that includes ADDDC Sparing.

	Server Firmware That Supports ADDDC Sparing	Recommended Server Firmware
UCS M5 Blades and Integrated UCS M5 Rack Servers	3.2(3p) or later 4.0(4i) or later 4.1(1d) or later	4.1(3d) or later
Standalone UCS M5 Rack Servers	3.1(3k) or later 4.0(4l) or later 4.1(1g) or later	4.1(3c) or later

 

Updated Firmware Recommendation Note (November 2021): The original firmware versions that incorporated ADDDC Sparing include UCS Manager 4.0(4c) and 4.1(1a) and Cisco Integrated Management Controller (IMC) 4.0(4e) and 4.1(1c). However, additional enhancements and critical fixes have been added since the initial releases. Cisco now recommends at least 4.1(3d) for UCS M5 Blades and Integrated UCS M5 Rack Servers, and 4.1(3c) for Standalone UCS M5 Rack Servers due to the presence of these additional fixes and enhancements. At a minimuimm, Cisco recommends 4.1(1) or later for ADDDC event monitoring and Post Package Repair features. For both ADDDC/PPR features as well as BIOS Enhanced Memory Testing, Cisco recommends UCS firmware 4.1(3d) or later.

ADDDC Sparing tracks correctable memory errors and dynamically maps out failing regions by putting those banks or ranks into virtual lockstep mode. This can prevent correctable errors from accumulating and becoming uncorrectable. When initiated, ADDDC allows the system to continue operation, at a reduced performance, until maintenance can be scheduled to repair the DIMM.

ADDDC Sparing incurs a marginal performance impact when enabled. Cisco recommends ADDDC Sparing for the optimal balance of performance, memory capacity, and error resiliency.

For technical information regarding ADDDC Sparing and additional RAS features available on UCS M5 servers, see Cisco UCS HX M5 Memory Technical Overview - Memory RAS Features.

HyperFlex Server Support -  Consult the UCS firmware compatibility matrix in the Cisco HyperFlex Recommended Software Release and Requirements Guide.

HyperFlex Data Platform	UCS Server Firmware To Address This Field Notice
3.5(2g) or later 4.0(2a) or later 4.5(1a) or later	4.1(3d) or later

 

HyperFlex Edge Server Support - Consult the HyperFlex Edge firmware compatibility matrix in the Cisco HyperFlex Recommended Software Release and Requirements Guide. ADDDC with HyperFlex Edge is supported in Cisco IMC 4.0(4l) and later software releases.

Required BIOS Settings

For UCS Manager managed servers with the BIOS POLICY for RAS configuration set to "Platform Default", no changes are required for ADDDC Sparing to take effect.

For UCS Manager managed servers with the BIOS POLICY for RAS configuration NOT set to "Platform Default", the policy must be changed to ADDDC Sparing (or Platform Default) to take advantage of ADDDC.

For Standalone (non-UCS Manager managed) servers, no changes are required for ADDDC Sparing to take effect.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email
• By telephone

Receive Email Notification For New Field Notices

Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.