Guest

Cisco MDS 9500 Series Multilayer Directors

Field Notice: FN - 63099 - MDS9000 - Generation 1 Modules May Reload During a Non-disruptive SAN-OS Upgrade Due to Compact Flash Being Unable to Partition for New Code - Issue Has Been Addressed for Upgrades from SAN-OS 3.3.1 or Higher Releases

Cisco - Field Notice: FN - 63099 - MDS9000 - Generation 1 Modules May Reload During a Non-disruptive SAN-OS Upgrade Due to Compact Flash Being Unable to Partition for New Code - Issue Has Been Addressed for Upgrades from SAN-OS 3.3.1 or Higher Releases

Revised July 18, 2008
April 14, 2008


NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision Date Comment
1.2 18-JUL-2008 Added flow chart and output examples, modified workaround to include updated instructions.
1.1 28-APR-2008 Minor wording updates to Title, Problem Description, and Workaround/Solution sections.
1.0 14-APR-2008 Initial Public Release

Products Affected

Products Affected
M9500 - DS-X9032-SSM
MDS9000 - DS-X9016
MDS9000 - DS-X9032
MDS9000 - DS-X9302-14K9

Problem Description

Generation 1 modules DS-X9016, DS-X9032,DS-X9032-SSM and DS-X9302-14K9 may reload during a non-disruptive SAN-OS upgrade, due to Compact Flash being unable to partition for new code.

This issue may be seen during an upgrade from SAN-OS 3.0.x, 3.1.x or 3.2.x.

This issue has been addressed for upgrades from SAN-OS 3.3.1 or higher releases. Therefore, it does not impact the customers who are already using SAN-OS release 3.3.1 when they upgrade to a higher SAN-OS release.

Background

During a SAN-OS upgrade, the installer attempts to download the new version of the software image on to the module's compact flash. The upgrade may be unsuccessful due to the module's compact flash being unable to partition for the new code. If this happens, the installer aborts and reloads the module.

Problem Symptoms

Generation 1 modules DS-X9016, DS-X9032, DS-X9032-SSM and DS-X9302-14K9 may reload during a SAN-OS upgrade, due to Compact Flash being unable to partition for new code. If this happens, the card will automatically reload and may cause the Install All to stop, causing the upgrade to be unsuccessful.

Customer may see error messages similar to the following:


Install has failed. Return code 0x40930020 (Non-disruptive upgrade of a module failed).
Please identify the cause of the failure, and try 'install all' again.


Module 2: Non-disruptive upgrading.
-- FAIL. Return code 0x40690009 (Error in downloading image for image upgrade).


To avoid this kind of unplanned disruption, follow steps in the Workaround/Solution section before proceeding with the SAN-OS upgrade.

Workaround/Solution

Please read instructions through in their entirety before executing any of these steps. A flow chart of this process is also included at the end of this document for your reference.

Generation 1 modules DS-X9016, DS-X9032,DS-X9032-SSM and DS-X9302-14K9 may reload during a non-disruptive SAN-OS upgrade, due to Compact Flash being unable to partition for new code.

This issue may be seen during an upgrade from SAN-OS 3.0.x, 3.1.x or 3.2.x.

This issue has been addressed for upgrades from SAN-OS 3.3.1 or higher releases. Therefore, it does not impact the customers who are already using SAN-OS release 3.3.1 when they upgrade to a higher SAN-OS release.

For customers who are planning to upgrade from SAN-OS 3.0.x, 3.1.x or 3.2.x, it is recommended that they use the following steps on the switch before the software upgrade:


STEP 1:

Issue the show system health statistics command and look for the following data points.

The output below is of a healthy LC and would be upgraded non-disruptively.

If any of the "Fail", "CFail" or "Errs" counters are non-zero for the Bootflash test, proceed to Step 4.

If "State" of the Bootflash test is not "Running", proceed to Step 2 regardless of the counters being zero.
Examples of outputs that require proceeding to Step 2 are listed below.


STEP 2:

Attach to the module in question and run the show kernel internal messages command and look
for any error conditions that state "I/O error".

If you see "I/O error" outputs, proceed to Step 4.

If the output of the show kernel internal messages is displayed correctly, then one additional step
is required. Proceed to Step 3 and continue.

Example:

MDS-switch#attach module 1

module-1# show kernel internal messages

end_request: I/O error, dev 16:000001 (hdc), sector 81930
end_request: I/O error, dev 16:000001 (hdc), sector 2
EXT2-fs error (device ide1(22,1)): ext2_write_inode: unable to read inode block - inode=9522, block=40965
end_request: I/O error, dev 16:000000 (hdc), sector 0

Step 3:

This step should only be run if the Bootflash Test State is in one of the following modes: "Failed", "Failing" or "Suspended" and no I/O Error's are seen in the show kernel internal messages log. See Step 2.

The Bootflash Test State must be cleared for a non-disruptive upgrade to succeed.

The command to clear the Bootflash Test State is:

MDS-switch# system health clear-errors module x bootflash

Where x = module with "Failed, Failing or Suspended" state.

This line card is now ready for a non disruptive upgrade. Refer to the Step 4 Caveat below.

Step 4:

Line cards identified by the steps above as having non operational Bootflash must be reloaded prior to an upgrade to avoid non-disruptive upgrades.

Caveat: This condition could recur over time even after the condition has been cleared.
Cisco recommends identifying problematic linecard bootflash prior to an upgrade window.
It is suggested the identified linecards then be reloaded during the upgrade maintenance window, just prior to an upgrade.
After the reload you must follow the above steps to verify linecard bootflash is ready for a non-disruptive upgrade.

DDTS

To follow the bug ID link below and see detailed bug information, you must be a registered customer and you must be logged in.

DDTS Description
CSCsm62295 (registered customers only) install fails with return code 0x40930020 - non-disruptive upgrade fails

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Product Alert Tool - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.