Guest

Cisco 10000 Series Routers

Field Notice: FN - 62762 - Updated - UMPIRE Archived -- ESR or uBR PRE2 Unable To Read PCMCIA ATA Flash Disk - Data Loss or System Boot Hang - Replace Flash Card


Revised April 28, 2010
April 26, 2007


NOTICE:

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Products Affected

Products Affected Top Assembly Printed Circuit Assembly Comments
Part Number Revision Part Number Revision
10000-PREMEMFD64
16-2733-01
-A0
NA
NA
-A0 or earlier
ESR-PRE-MEM-FD128
16-2117-01
-A0
NA
NA
-A0 or earlier
10000-PREMEMFD64
16-2733-01
-A0
NA
NA
-A0 or earlier
ESR-PRE-MEM-128M
16-2117-01
-A0
NA
NA
-A0 or earlier

Problem Description

The PCMCIA ATA Flash Disks used in specific Cisco products may contain a Manufacturing defect which reduces reliability of the Flash device. PRE2 cards used both in the Cisco ESR10000 and uBR10000 platforms that contain a suspect Flash device may not be able to access stored files on disk 0/1. The router may fail to reboot if it relies on an IOS image stored on the Flash device.

A portion of the PRE2 cards manufactured between May 2005 [s/n CAT0844026U] and October 2006 [s/n CAT1044521H] may have shipped with an affected PCMCIA ATA Flash disk from specific component lots. Refer to the How to Identify Hardware Levels section for Flash disk identification information.

PRE2 board Serial Number location is at the lower edge of the board in Quadrant 3. See the Serial Number Location Template at the end of the How to Identify Hardware Levels section.

The failure condition would be observed during a power cycle, reload event or during a write event such as a crash file dump. Non-redundant systems will be unable to pass traffic in the failed state. Redundant systems will switch to the backup PRE2. The chances of both flash cards being from the affected component lots are estimated as less than 1 in 10,000.

PRE2 does not allow ROMMON upgrades via software and therefore the affected Flash devices are recommended for replacement.

NOTE 1 - The PRE2 card itself is not defective; this issue is only related to the removable PCMCIA ATA Flash disk cards.

NOTE 2 - As customers may remove and re-use Flash Cards, the affected cards shipped with these products may have been re-used in different Cisco products. Please exercise caution when re-using Flash cards and visually check them against the Identification Section data provided.

NOTE 3 - PRE1 cards were not built with defective Flash disks.

Background

This issue affects specific subset of the PCMCIA ATA Disks and Compact Flash disks that were installed in specific Cisco routers manufactured between May 2005 and October 2006. Cards shipped individually as spares over that same time span may also contain one of the affected units.

This issue has been linked to a component manufacturing problem. The root cause was identified and appropriate steps taken to resolve the problem.

Product impact varies due to different flash memory access methodologies and use patterns. Some products may be able to mitigate the potential effect by re-writing affected memory areas at specific intervals. This feature does not work with all products, so please check the workaround section for availability and specific instructions.

Problem Symptoms

The problem symptom is typically seen during a system reload or cold start. Under normal circumstances a router's PCMCIA card or Compact Flash card could contain Cisco IOS software, configurations or other files necessary for system boot or operation. These files are accessed by the router during the boot sequence to reach an operating state.

In the problem state, the router is unable to access the contents of the Flash disk. When the problem occurs, the system is prevented from accessing directory information stored on the Flash disk. This type of error occurs in the controller of the Flash disk. Data stored on the Flash disk should be intact but may not be accessible. The router may indicate the state with a message log below. The router will be unable to read the contents of the Flash disk that were previously read or written to.

The output below shows the system log or terminal monitor message when the router is unable to read the Flash disk contents.

Error opening slot0:/ (No device available)

The router may stop the boot sequence and remain at a ROM Monitor (ROMmon) prompt as seen below.

Readonly ROMMON initialized
boot: cannot open "flash:"
boot: cannot determine first executable file name on device "flash:"
cxxxx processor with XXXXXX Kbytes of main memory Main memory is configured to 64 bit mode with ECC enabled
Readonly ROMMON initialized
rommon 1 >

NOTE 4 - The Router will be unaffected and continue to run with a failed Flash disk. Only read/write access such as a logfile write to the flash disk or a router reboot will cause the problem symptom to become evident.

Workaround/Solution

Workaround
There is no workaround available for Flash disks used in PRE2 processors.

Solution
Exchange a failed PCMCIA Flash disk with a known good disk and IOS image. Replacement of the Flash card at the next scheduled maintenance window is recommended to prevent a possible reboot hang on router restart.

Non failed Flash cards matching the replacement criteria should also be replaced to maximize system reliability at the next regular maintenance window.

Please use the regular RMA process for any future replacement, as of 4/20/2010.

For PCMCIA ATA Flash Disk installation in a PRE2 see Cisco 10000 Series Router Performance Routing Engine Installation.

How To Identify Hardware Levels

Identifying affected PCMCIA ATA FLASH disks. Please use the following flow chart as a trouble resolution guide for PCMCIA and CF disks.

NOTE #5 - Suspect boards were manufactured between May 2005 [s/n CAT0844026U] and October 2006 [s/n CAT1044521H].
NOTE #6 - See individual card sections [64 MB and 128 MB] for command information and disk information.
NOTE #7 - See individual identification card sections [64 MB and 128 MB] for pictures and code locations.
NOTE #8 - If removal of a system processor board to check serial number or Flash disk is required use ESD procedures noted in the installation link.

*** 10000-PREMEMFD64 ***********************************

Remote Identification of 64 MB PCMCIA Flash disks

For units inserted into operational systems:

Enter the CLI command: show disk[0,1]: filesys

(If the Disk is unable to be read, prioritize this system for card replacement at the next service window as the disk has failed. The router will continue to run until it reboots and cannot access the failed disk. Operation should be unaffected until a reboot or logfile write event to the failed disk.)

Look for the line: Number of Cylinders 490
If the number is 490, it may be an affected unit. Inspect visually and confirm card vendor type and date range at the next service window.

Visual Identification of 64 MB PCMCIA Flash disks

An affected unit is shown in the following figure:

Look for
1) Date code is earlier than 060728 (7/28/06)
2) Vendor Part Number starts with "CPC"
3) Cisco Part Number is 16-2733-01

*** ESR-PRE-MEM-FD128 ***********************************

Remote Identification of 128 MB PCMCIA disks

For 128 MB PCMCIA ATA flash disks inserted into operational systems:

Enter the CLI command: show disk[0,1]: filesys
(If the Disk is unable to be read, prioritize this system for card replacement at the next service window as the disk has failed. The router will continue to run until it reboots and cannot access the failed disk. Operation should be unaffected until a reboot or logfile write event to the failed disk.)

Look for the line: Number of Cylinders 980
If the number is 980, it may be an affected unit. Inspect visually and confirm card vendor type and date range at the next service window.


Visual Identification of 128 MB PCMCIA disks

An affected unit is shown in the following figure:

Look for
1) Date code is earlier than 060728 (7/28/06)
2) Vendor Part Number starts with "CPC"
3) Cisco Part Number is 16-2117-01

*****************************************************************************

Frequently Asked Questions

Q1 - Will formatting the Flash disk resolve this issue?

A1 - No, attempts to format the Flash disk will not resolve the problem. When the problem occurs data should be intact and re-formatting after applying the solution should not be necessary.

Q2 - I've applied the solution, but my router does not boot. What do I do?

A2 - There may be a separate issue preventing the router from booting. Further troubleshooting may be necessary.

Q3 - What do I do if my router is currently in ROMMON mode?

A3 - Reference the Cisco Tech Tip for instructions on booting the router using TFTP or use another Flash disk with a known good boot image.

Revision History

Revision Date Comment
1.2 28-APR-2010 Updated to remove the Umpire program. RMA process should now be used.
1.1 27-APR-2007 Corrected beginning S/N in Step 5 in How to Identify Hardware Levels Section
1.0 26-APR-2007 Initial Public Release

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

Receive Email Notification For New Field Notices

Cisco Notification Service - Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.