Guest

Cisco 7800 Series Media Convergence Servers

Field Notice: FN - 63374 - MCS-7816-I4, MCS-782x-I4, Hard Drive goes into Read Only mode, Hard Drive firmware upgrade required

Field Notice: FN - 63374 - MCS-7816-I4, MCS-782x-I4, Hard Drive goes into Read Only mode, Hard Drive firmware upgrade required

Revised July 30, 2012

December 13, 2010


NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision Date Comment
1.2
30-JUL-2012
Updated software links in the Workaround/Solution section.
1.1
28-JAN-2011
Added 74- part numbers in the How to Identify Hardware Levels section and clarified the implications for product shipped from Service Logistics. Added a note about how the Hard Drive Firmware upgrade media will ship with product from Service Logistics after April 1, 2011.
1.0
13-DEC-2010
Initial Public Release.

Products Affected

Products Affected Comments
MCS-7816-I4*
All Bare Metal Variants from IPCBU, UCBU, and CCBU
MCS7816I4-*
All Appliance Variants from IPCBU, UCBU, and CCBU
MCS-7825-I4-*
All Bare Metal Variants from IPCBU, UCBU, and CCBU
MCS7825I4-*
All Appliance Variants from IPCBU, UCBU, and CCBU
MCS7828I4-*
All Appliance Variants from IPCBU, UCBU, and CCBU

Engineering Change Order (ECO)

Engineering Change Order (ECO) Remarks
E102451
 

Problem Description

An internal failure of certain hard drives with firmware prior to version 3B06 used inside the servers that are listed in the Products Affected section can lead to an issue where the file system on the hard drive is marked 'Read-Only', preventing the software from being able to execute write operations.

Background

All products, as listed in the Products Affected section, manufactured prior to the referenced Engineering Change Order (ECO) & Deviation are at-risk for this issue.

Problem Symptoms

The file system will enter a read-only state. Services may stop, followed by the server being unreachable by SSH or web. The server may or may not respond to pings.

The following may be visible from the server's console:
EXT3-fs error (device sda6) in start_transaction: Jornal has aborted

If the server is reachable via an SSH login, the following may be visible:

java.io.FileNotFoundException: /var/log/active/platform/log/cli.bin (Read-only file system)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:212)
:
:
:
at org.apache.log4j.Category.info(Category.java:674)
at sdMain.main(sdMain.java:611)
log4j:ERROR No output stream or file set for the appender named [CLI_LOG]

Workaround/Solution

Upgrade the hard drive firmware to version 3B06 or later on all hard drives used in affected servers, including any spare hard drives in storage which may be used in affected servers.

Hard drive or server RMAs should not be requested for resolving this issue.

For at-risk hard drives that are currently operating or have previously operated in affected servers, resolving this issue is a two-step process:

  1. Run the Hard Drive Exerciser Utility.
  2. Apply the hard drive firmware upgrade.

For at-risk hard drives that have been in storage and never placed in-service, running the Hard Drive Firmware Exerciser Utility is not required. Resolving this issue is a one-step process:

  1. Apply the hard drive firmware upgrade.

Note: Installing a new at-risk spare hard drive, which had previously been in storage and never been placed in-service, into an affected server for the sole purpose of upgrading the firmware will not require the use of the Hard Drive Exerciser Utility. There is no harm in running the Hard Drive Exerciser Utility in this instance if the user chooses to do so.

Run the Hard Drive Exerciser Utility

For Windows-based systems, there is no need to run the Hard Drive Exerciser Utility. Please proceed directly to upgrading the firmware.

For Cisco's VOS (Linux) based systems, download and install the following COP file:
ciscocm.ibm-diskex-1.0.cop.sgn (registered customers only)

To install the COP File, please follow the instructions in the Readme, available here:
ciscocm.ibm-diskex-1.0.cop.sgn Readme (registered customers only)

Apply the hard drive firmware upgrade

The hard drive firmware upgrade is included in the Cisco-HDD-FWUpdate-3.0.1-I.ISO image, downloadable here:
Cisco-HDD-FWUpdate-3.0.1-I.ISO (registered customers only)

The Readme is available at the following location:
Cisco Standalone HDD Firmware Update Version 3.0 - IBM Servers Readme (registered customers only)

Please follow the instructions in the Readme for installation.

As of approximately December 1, 2010, new servers that were manufactured under Engineering Change Order (ECO) E102451 or Deviation D114051 are guaranteed to be free of this problem. Refer to How to Identify Hardware Levels below for instructions on how to view the version of an in-service product.

Replacement hard drives provided by IBM may have affected firmware installed. Make sure to verify the hard drive firmware version for any hard drive replacement provided by IBM.

Product shipped from Cisco Systems, Inc. Service Logistics may still exhibit this problem. If an RMA is requested, to ensure an RMA replacement is NOT affected by this problem before the above date, request an RMA coded as shown below:

Service Level: Mfg New - 3rd Bus Day NON BILLABLE
Failure Class: Administrative Request
Failure Code: Field Notice Alert

Note: Service level agreements do not apply to RMAs coded as shown above.

Additionally, after approximately April 1, 2011, product shipped from Service Logistics includes the Hard Drive Firmware Upgrade on a CD to be installed by the user.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered customer and you must be logged in.

DDTS Description
CSCti52867 (registered customers only)  

How To Identify Hardware Levels

Remember that both hard drives in-service and spare hard drives in storage should be examined.

Several methods exist for checking the hard drive firmware version

  • Cisco MCS Servers manufactured under the referenced Deviation and ECO will have upgraded firmware. The difference between servers built under the ECO and Deviation is as follows: Servers which carry the Deviation label are new servers in Cisco's supply chain which were manufactured and packaged (not yet shipped) prior to the ECO release. These servers were reworked to upgrade the hard drive firmware and then repackaged to await shipping. The Deviation label is attached to the server, located on the top of the front bezel near the 74- level part number label. Servers built after the ECO release were assembled with hard drives that had already been upgraded, and these servers carry a new 74- part number. The table below shows the part numbers and labels on both the server and packaging for both the ECO and the Deviation:



    See below for a picture of a system with the deviation label:


    The same Deviation label will be attached to the packaging:


  • Some hard drives, depending on where they were purchased, may have either a label with the firmware version or may have Cisco's Deviation label. If it does have a Deviation label, the label will be visible on both the hard drive packaging and the hard drive itself, as shown in the picture below:




  • On Unified Communications applications that support a Command Line Interface (CLI) via a direct connection or SSH, some versions support the show hardware command. The hard drive firmware version may be visible for each hard drive, but this function is not supported in all applications which support the show hardware command.
  • The hard drive firmware upgrade utility referenced above will automatically detect if hard drive firmware is at-risk, as described in the utility's readme.

Deviation

Deviation D114051

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.


d-of-sale issues for the Cisco products you specify.