Guest

Cisco 7800 Series Media Convergence Servers

Field Notice: FN - 63255 - MCS-7825-I2 & IBM x306m, BIOS 1.42 & 1.43 error prevents a system from booting if rebooted after October 1, 2009

Field Notice: FN - 63255 - MCS-7825-I2 & IBM x306m, BIOS 1.42 & 1.43 error prevents a system from booting if rebooted after October 1, 2009

September 30, 2009


NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision Date Comment
1.2
30-SEP-2009
Updated Workaround/Solution section: added Unified Mobility (also known as MobilityManager) and MeetingPlace Express to the VOS-based server table; updated CUMA posting status for 7.0.2.10101-6; updated CER 7.x & 2.x posting COP File. Clarified FWUCD versions, from "2.3(1) or later" to "2.3(1) or later 2.X(X) versions.
1.1
23-SEP-2009
Updated All sections to include clarifications for both VOS and Windows based servers. Removed 7815-I2 from the Affected Product list.
1.0 03-SEP-2009 Initial Public Release.

Products Affected

Products Affected Comments
MCS-7825-I2-xxx 
All MCS-7825-I2 Bare Metal, Appliance Servers, and x306m Customer-provided servers 

Problem Description

An issue in the HostRAID BIOS 2106 (included with System BIOS version 1.42 and 1.43) will prevent a server from booting successfully if rebooted after October 1, 2009.

To determine the BIOS version of a server, please see the section called How to Identify Hardware Levels.

This is a Server BIOS issue, and therefore, it affects both Windows-based and VOS-based servers.

On VOS-based servers (see the Background section below for a definition of VOS), the BIOS version is included with the Application. The Application checks for the proper BIOS version each time the server boots, and it will re-flash the BIOS as necessary. Only certain VOS-based applications contain the affected BIOS versions. See the table below of Affected and Fixed VOS-Based Application Versions.

On Windows-based servers, the BIOS is included with the Operating System, not the Application. BIOS Version verification and reflashing are only executed during Operating System Installation. Only certain Cisco Windows versions contain the affected BIOS versions. See the table below of Affected and Fixed Cisco Windows OS Versions.

Background

This problem is caused by an error in the IBM BIOS and is documented in IBM RETAIN Tip: H182636.

The System BIOS for the affected servers contains the on-board HostRAID BIOS. BIOS version 1.42 and 1.43 were integrated with HostRAID BIOS 2106.

The HostRAID BIOS 2106 was erroneously compiled with the calendar limit flag set.

In this Notice, you will see references to VOS. VOS stands for Voice Operating System and is the name Cisco Systems, Inc. gives to the Linux-based Operation System that many of the Unified Communications applications use as the Operating System.

Problem Symptoms

Prior to October 1, 2009, servers listed in the affected product section will function normally and show no signs of failure.

On and following October 1, 2009, servers will function normally if they are not rebooted.

On and following October 1, 2009, if an affected server is rebooted or shut down, it will not be able to boot normally. The on-board HostRAID controller will not be properly detected while the server boots up and the 'CTRL+A' and 'F1' BIOS options will not be available.

****Critical Notes****
Avoid rebooting affected servers on or AFTER October 1, 2009 if the BIOS upgrade has NOT been applied.

Workaround/Solution

Do not request any Hardware Replacements (RMA) to resolve this issue. This issue is recoverable via SW upgrades in the field.

The permanent solution to this problem is to upgrade the BIOS to Version 1.45 or later. There are different steps for upgrading depending if the server is Windows-based or VOS-based.

Cisco Systems, Inc. recommends that you DO NOT apply the IBM Workaround by clearing CMOS and DO NOT apply the IBM solution documented in the IBM RETAIN Tip. Applying the IBM Workaround could have detrimental consequences because it removes critical information from the system. Applying the IBM Solution could result in firmware and BIOS settings being updated which have not been tested by Cisco Systems, Inc.

The following solutions can be applied to either MCS-branded or IBM equivalent affected servers.

Solution for Windows-Based Servers

For affected servers running Windows-based Applications from Cisco Systems, Inc., either upgrade to a Cisco Windows Version which contains the Fixed BIOS or use the Firmware Update CD (FWUCD), version 2.3(1) or later 2.X(X) versions.

Below is a table showing the Affected and Fixed Cisco Windows OS Versions:

Product Bug ID & Link to Bug Toolkit
(Registered cisco.com Users Only)
Affected Version Fixed Version
(Install This Version)
Availability
Cisco Windows 2000 Not Available 2000.4.6 and later FWUCD 2.3(1) (Note 1) Published on SW Download Site
Cisco Windows 2003 CSCta40816(registered customers only) 2003.1.3 and later
2003.1.4 and later
2003.1.5 and later
FWUCD 2.3(1) (Note 2)
2003.1.5 Target Oct 8
FWUCD 2.3(1) Published on SW Download Site

NOTES for Windows-based servers:

  1. Cisco Windows 2000 is currently beyond the End of Software Maintenance. The FWUCD 2.3(1) or later 2.X(X) versions can be used to upgrade the BIOS. See instructions below on using the FWUCD.
  2. On Cisco Windows 2003, customers are encouraged to upgrade to a fixed Cisco Windows 2003 version, but the FWUCD 2.3(1) or later 2.X(X) versions can be used as well.

The FWUCD can be applied before, on, or after October 1, 2009. The FWUCD can also be applied to a Windows-based server if it will not boot the Operating System because of this problem. The server will be able to boot the FWUCD, allowing the BIOS to be upgraded to restore functionality. The FWUCD can be used on any affected server which is running a Cisco Windows-based application. There is no need to install application-specific upgrades on Cisco Windows-based servers to resolve this problem.

The Latest version of the FWUCD for IBM Servers can be downloaded from the following location: Latest FWUCD Download (registered customers only).

Note: This download link points to the MCS-7825-I2 server, and this same file can be used for all affected servers. Meaning, there is not a separate image for each affected server. The image needs to be burned onto a CD/DVD in order to install it on a server.

Please see the Readme for important installation information. The Readme for the IBM FWUCD Version 2.3(1) image is available here: FWUCD 2.3(1) Readme.

Solution for VOS-Based Servers

VOS-based applications contain the BIOS upgrade, requiring an upgrade of the VOS-based application in order to upgrade the BIOS.

Please make every effort to upgrade the application before October 1, 2009.

The procedures differ depending on when the VOS-upgrades are applied:

Below is the Table of Affected and Fixed VOS-Based Application Versions:

Product Bug ID & Link to Bug Toolkit
(Registered cisco.com Users Only)
Affected Version Fixed Version
(Install This Version)
Availability
Cisco Unified Communications Manager CSCta43460(registered customers only) 5.x
6.x
7.x
5.1(3g) = 5.1.3.8000-4 and later
6.1(4) SU1 = 6.1.4.1190-3 and later
7.1(2b) SU1 = 7.1.2.31900-1 and later
All Published on SW Download Site
Cisco Unity Connection CSCta43460(registered customers only) 2.x
7.x
2.1(4) SU1 = 6.1.4.1190-3 and later
7.1(2b) SU1 = 7.1.2.31900-1 and later
All Published on SW Download Site
Cisco Unified Presence CSCtc00579(registered customers only) 1.x
6.x
7.x
6.0(6) MR
7.0(5) and later
6.0(6) MR Target Oct 16
7.0(5) Published on SW Download Site
Cisco Emergency Responder CSCtc00918(registered customers only) 2.0(x)
7.0(x)
Independent COP File
7.1(1) and later
2.0(x) COP File Published on Software Download site (ciscocm.cer2_x_CSCtc00918.cop.sgn)
7.0(x) COP File Published on Software Download site (ciscocm.cer7_x_CSCtc00918.cop.sgn)
7.1(1) Target Sept 30
Cisco Unified Mobility Advantage CSCtc03782 (7.0) (registered customers only)
CSCtc03794 (7.1) (registered customers only)
3.x
7.x
3.x (Note 1)
7.0.2.10101-6 and later
7.1(3) ES and later
7.0.2.10101-6 Published on SW Download site
7.1(3) ES Target Oct 8
Cisco Unified Mobility (also known as MobilityManager) Not Available All (Note 1) FWUCD Published on SW Download site
Cisco MeetingPlace Express Not Available All (Note 1) FWUCD Published on SW Download site

NOTES for VOS-based servers:

  1. The FWUCD 2.3(1) or later 2.X(X) versions can be used to upgrade the BIOS of a server running the indicated application or version. For both Windows-based and VOS-based servers, follow the procedure in the section called Solution for Windows-Based Servers.

Solution for Upgrading VOS-Based Applications BEFORE October 1, 2009

For affected servers running VOS-based Applications, please upgrade the Application to a fixed version as indicated in the table. The Application upgrade will include the proper BIOS upgrades. There is no need to use the FWUCD on a VOS-based server before October 1, 2009. Follow the Upgrade directions specific to the Application you are upgrading. No other action is required.

Solution for Recovering a Non-Booting Server that HAS BEEN Upgraded to a Fixed Version

If you upgraded to a fixed version on or after October 1, 2009, the server will not reboot successfully because the Application has not had a chance to upgrade the BIOS yet. BIOS re-flashes are only executed during bootup. Therefore, even though the Application contains the upgraded BIOS, the system board does not. To resolve this condition, the BIOS must be upgraded using the FWUCD.

  1. Download the FWUCD image (linked above), burn it onto a CD/DVD, and follow the instructions in the Readme (linked above) to apply the upgrades contained in the FWUCD.
  2. Insert the FWUCD and reboot the server allowing it to boot the FWUCD and install the updates. Once the upgrades are complete, remove the FWUCD.
  3. Allow the system to reboot normally.

Now, both the system board and Application share the same BIOS version information, and the server will be fully functional. No further action is required.

Solution for Recovering a Non-Booting Server That HAS NOT BEEN Upgraded to a Fixed Version

If an affected server is in a failed state without being upgraded to a fixed version already, restoring the server requires a combination of steps using both the FWUCD to recover the server, followed by upgrading to a fixed Application version, and repeat usage of the FWUCD.

1. Download the FWUCD image (linked above), burn it onto a CD/DVD, and follow the instructions in the Readme (linked above) to apply the upgrades contained in the FWUCD. Once the upgrades are complete, remove the FWUCD.

2. After the system reboots, press "F1" to boot into the BIOS Setup Menu. Also make sure that BIOS version shows up as "1.45" or later.

NOTE: If you missed out on opportunity to press "F1," IMMEDIATELY power cycle the system to restart, or else it will boot the hard disk and re-flash BIOS back to an affected BIOS version, returning the server to a failed state. If this occurs, you will need to restart the recovery process in Step 1 above.

3. Select Date and Time option in the BIOS Setup Menu. Change the System date to a date BEFORE October 1, 2009 date. Save settings and exit from BIOS setup. (The date is corrected later, either manually or by the NTP service.)

4. Allow the system to reboot, and the Application will re-flash the BIOS to an older version.

NOTE: This is allowable since the System date you changed in the step above is BEFORE October 1, 2009.

5. After the BIOS Re-flash completes, the system will reboot again. Allow the system to boot to the Application fully.

6. Upgrade the Application to a fixed version as indicated in the table, following the Upgrade steps specific to that application. Do not switch versions yet, though.

NOTE: You should select the option "DO NOT REBOOT AFTER UPGRADE." This will allow ample opportunity to re-insert the FWUCD again before you switch versions and reboot.

7. This step varies if you are using NTP. See Step 7A if you ARE using NTP. See Step 7B if you are NOT using NTP. If you do not know, follow the directions on Step 7A.

7A. If you are using NTP or do not know, before switching versions to the upgraded image and before allowing the server to reboot, insert the FWUCD again. The FWUCD must be used again because of two reasons:

  • The BIOS was downgraded as the affected version booted before the application upgrade occurred.

  • After the Affected version booted fully, the NTP service reset the System Date in the BIOS to the current Date.
Thus, the FWUCD must be used to again upgrade the BIOS, but another date change is not necessary. Once the FWUCD is inserted, allow the system to Switch Versions and reboot. The server will boot to the FWUCD. Once the BIOS update is complete, remove the FWUCD, and allow the system to reboot.

7B. If you are not using NTP, you should not need to re-apply the FWUCD. Allow the server to Switch Versions and Reboot. As the server reboots, it will upgrade the BIOS. AFTER the BIOS upgrade is complete, change the System Date back to the current date.

Now, both the system board and Application share the same BIOS version information, the System Date is current, and the server will be fully functional.

NOTE: Please be aware that if you ever switch versions back to an affected version, the BIOS will be downgraded, and the system will require a slightly modified recovery procedure. The only difference will be that instead of applying an upgrade image, you would just need to Switch Versions since the upgrade is already installed. The FWUCD would still need to be applied twice as noted in the above steps.

How To Identify Hardware Levels

To see what BIOS version is on a Windows-based system, you can check the following:

  • Reboot the server and hit "F1" to access the BIOS menu.
  • Optionally, you can insert the FWUCD and see if any updates are required as it detects them automatically.

To see what BIOS version is on a VOS-based system, you can check the following:

  • From the CLI, type 'show hardware' without the single quotes.
  • Reboot the server and hit "F1" to access the BIOS menu.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.