Guest

1600 Series Router Hardware Troubleshooting

Cisco - 1600 Series Router Hardware Troubleshooting

Document ID: 6346

Updated: Jun 20, 2008

   Print

Introduction

This document describes how to troubleshoot potential hardware issues with Cisco 1600 Series Routers and can help you identify which component might cause a hardware failure, depending on the type of error that the router is experiencing.

Note: This document does not describe software-related failures except for those that are often mistaken as hardware issues.

Prerequisites

Requirements

Readers of this document should be knowledgeable of the following:

Components Used

The information in this document is based on Cisco 1600 series routers.

The information presented in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If you are working in a live network, ensure that you understand the potential impact of any command before using it.

Hardware-Software Compatibility and Memory Requirements

Whenever you install a new card, module, or Cisco IOS® software image, it is important to verify that the router has enough memory, and that the hardware and software are compatible with the features you wish to use.

Perform the following recommended steps to check for hardware-software compatibility and memory requirements:

  1. Use the Software Advisor (registered customers only) tool to choose software for your network device.

    Tip: The Software Support for Features (registered customers only) section helps you determine the Cisco IOS software image needed by choosing the types of features you wish to implement.

  2. Use the Download Software Area (registered customers only) to check the minimum amount of memory (RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software image. To determine the amount of memory (RAM and Flash) installed on your router, see How to Choose a Cisco IOS Software Release - Memory Requirements.

    Tips:

    • If you want to keep the same features as the version that is currently running on your router, but don't know which feature set you are using, enter the show version command on your router and paste it into the Output Interpreter (registered customers only) tool to find out. It is important to check for feature support, especially if you plan to use recent software features.

    • If you need to upgrade the Cisco IOS software image to a new version or feature set, see How to Choose a Cisco IOS Software Release for more information.

  3. If you determine that a Cisco IOS software upgrade is required, follow the Software Installation and Upgrade Procedure for the Cisco 1600 Series Router.

    Tip: If your 1600 does not have a connection to the network or does not have a valid Cisco IOS software image, you may need to use the console port of the router to perform an xmodem software upgrade using ROMmon. This procedure does not require the use of a Trivial File Transfer Protocol (TFTP) server.

  4. If the router does not boot after a Cisco IOS software upgrade, go to the Cisco 1600 Does Not Boot section of this document for more information.

Error Messages

The Error Message Decoder (registered customers only) tool allows you to check the meaning of an error message. Error messages appear on the console of Cisco products, usually in the following form:

%XXX-n-YYYY : [text]

Here is an example error message:

Router# %SYS-2-MALLOCFAIL: Memory allocation of [dec] bytes failed from [hex], 
pool [chars], alignment [DEC]  

Some error messages are informational only, while others indicate hardware or software failures and require action. The Error Message Decoder (registered customers only) tool provides an explanation of the message, a recommended action (if needed), and if available, a link to a document that provides extensive troubleshooting information about that error message.

Conventions

For more information on document conventions, see the Cisco Technical Tips Conventions.

WAN Interface Cards (WICs)

The Cisco 1600 series consists of four models, each of which has an Ethernet port, a built-in WAN port, and a slot for an optional second WAN port:

Model Components
Cisco 1601 One Ethernet, one serial, one WAN interface card slot
Cisco 1602 One Ethernet, one serial with integrated 56-kbps data service unit/channel service unit (DSU/CSU), one WAN interface card slot
Cisco 1603 One Ethernet, one ISDN Basic Rate Interface (BRI) (S/T interface), one WAN interface card slot
Cisco 1604 One Ethernet, one ISDN BRI with integrated Network Termination 1 (NT1) (U interface), one S-bus port for ISDN phones, on WAN interface card slot

For WIC compatibility information, see the WIC Hardware Compatibility Matrix.

See Cisco 1600 Series and WAN Interface Cards for additional information.

Identifying the Issue

This section explains what to do to determine the cause of the potential hardware issue(s).

In order to determine the cause, the first step is to capture as much information about the problem as possible. The following information is essential for determining the cause of the problem:

  • Console logs - For more information, see Applying Correct Terminal Emulator Settings for Console Connections.

  • Syslog information - If the router is set up to send logs to a syslog server, you may be able to obtain information on what happened. For details, see How to Configure Cisco Devices for Syslog.

  • show technical-support command output - The show technical-support command is a compilation of many different commands including show version, show running-config, and show stacks. TAC engineers usually ask for this information to troubleshoot hardware issues. It is important to collect the show technical-support information before doing a reload or power-cycle as these actions can cause all information about the problem to be lost.

  • The complete bootup sequence if the router experiences boot errors.

If you have the output of a show command from your Cisco device (including show technical-support), you can use to display potential issues and fixes. To use , you must be a registered customer, be logged in, and have JavaScript enabled.

Cisco 1600 Does Not Boot

Capturing information from the console of the router is essential for troubleshooting this kind of problem. The console output should be logged in a file for later analysis or for the Cisco Technical Assistance Center (TAC), should a TAC Service Request be created.

Cisco1600 routers have a Bootstrap image and the Boot image embedded in the ROM. Even if there are no valid images on the Flash, the router should always be able to load these images. If you are not familiar with the booting process of Cisco routers, see "Figure 12: Booting Process" in the Rebooting Documentation. You can also have a look at Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.2.

The following table lists symptoms and recommended actions to take if you are encountering booting problems:

Symptom Recommended Action
No LEDs are on after powering on the router. If your router is a Cisco 1601-R, check the serial number against a known issue described in Cisco 1601-R Front Panel LEDs Will Not Illuminate. If your router is affected, it should still operate normally, despite the LEDs. Otherwise, check if the power supply is plugged in firmly. If that does not resolve the issue, replace the power supply. If the problem persists, replace the router.
LEDs are on after powering on the router, but there is nothing on the console. Verify that the baud rate is set to 9600 bps. If that doesn't help, verify that the equipment used for connecting to the console is operating properly. You can do this by connecting to a known good router to check your console equipment. If the equipment is successfully tested, but the problem remains, replace the router.
Router displays the System Bootstrap version and hangs at that point or falls into a booting loop:
System Bootstrap, Version 11.1(7)AX 
[kuong (7)AX], EARLY DEPLOYMENT RELEASE 
SOFTWARE (fc2)
Copyright (c) 1994-1996 by cisco Systems, Inc.
The memory may not be well seated. First try to reseat (remove and then reinsert) the single inline memory module (SIMM) memory. If the router still does not boot, replace the router. You can find information on the location of the Dynamic RAM (DRAM) SIMMs in Installing or Replacing the DRAM SIMM in Cisco 1600 Series Routers.
Router boots in ROMmon; no error messages on the console. Set the configuration register to 0x2102 and reload the router:
rommon 1 > confreg 0x2102

rommon 2 > reset
If the router remains in ROMmon, follow the procedure described in ROMmon Recovery for the Cisco 1600 Series Router.
Router boots in boot mode; no error messages on the console. Set the configuration register to 0x2102 and reload the router. There is no need to save the running configuration before reloading:
router(boot)#configure terminal
Enter configuration commands, one per line.  
End with CNTL/Z.
router(boot)(config)#config-register 0x2102
router(boot)(config)#end
router(boot)#reload
System configuration has been modified. 
Save? [yes/no]: no
Proceed with reload? [confirm]

Note: The config-register command is the only Cisco IOS software configuration command which doesn't get saved in non-volatile RAM (NVRAM). It changes the configuration register immediately, but this only takes effect during the next boot.

Router boots in boot mode, with the following messages on the console:
getdevnum warning: device "PCMCIA slot 1" 
has size of zero
getdevnum warning: device "PCMCIA slot 1" has 
size of zero
pen: read error...requested 0x4 bytes, got 0x0
trouble reading device magic number
boot: cannot open "flash:"
boot: cannot determine first file name on 
device "flash:"
The Flash card is removed or badly inserted. Verify the presence of the Flash card by running the show flash executable command:
router(boot)#show flash
%Device in REMOVED state
Insert or reseat the Flash. If the problem persists, replace the Flash card. If that doesn't resolve the problem, replace the router.
Router boots in boot mode, with the following messages on the console:
device does not contain a valid 
magic number
boot: cannot open "flash:"
boot: cannot determine first file name on 
device "flash:"
The Flash is empty or the filesystem is corrupted. Copy a valid image on the Flash, and while copying, you will be prompted to erase the old Flash (if one exists). Then reload the router. For more information on loading new software on the 1600 router, see Software Installation and Upgrade Procedure for the 1600 or Software Installation and Upgrade Procedure for the 1600-R.
Router boots in boot mode, with the following message on the console:
SYSTEM INIT: INSUFFICIENT MEMORY TO 
BOOT THE IMAGE!
This message only occurs when loading images that run from RAM (for example, c1600-y-mz.120-20). Since Cisco1601-1604 routers support images that run in RAM and images that run directly from Flash (for example, relocatable images), there are two solutions for the problem:
  • Upgrade the Dynamic RAM (DRAM)
  • Load a relocatable image (such as c1600-y-l.120-20). On Cisco1605 routers, the only option is to upgrade the DRAM.
For more information on image types, see the "Image Naming Conventions" section under Loading and Maintaining System Images. For information on memory requirements, see Memory Requirements (How to Choose a Cisco IOS Software Release). For upgrading the DRAM SIMMs, see Installing or Replacing the DRAM SIMM in Cisco 1600 Series Routers.
Router boots in boot mode, with the following messages on the console:
loadprog: error - Invalid image for 
platform 
The Cisco IOS software image is not appropriate for the 1600 router, or it is corrupted. Erase the Flash and download a new valid Cisco IOS software image. For more information on loading new software on the 1600 router, see Software Installation and Upgrade Procedure for the 1600 or Software Installation and Upgrade Procedure for the 1600-R.
Router boots correctly, but the WAN interface of the WAN interface card (WIC) is not available. Verify the presence of the WIC card by running the show diag command from enable mode. If the router doesn't detect the WIC card or if there are no WIC cards installed, the command has the following output:
router#show diag 
 No WAN daughter card present
Reseat the WIC card. Push it in firmly and tighten the screws. If that doesn't resolve the problem, replace the WIC. If the problem persists, replace the router. If the show diag command shows that there is a WIC card installed, but the information is not correct, use the Software Advisor (registered customers only) to verify that the WIC is supported in the running Cisco IOS software release. If the information is correct and the WIC is supported in the running Cisco IOS software, contact the Cisco Technical Assistance Center (TAC) for further troubleshooting.

Crashes

When we refer to a "system crash", we mean a situation where the system has detected an unrecoverable error and has restarted itself. A crash can be caused by software problems, hardware problems, or both. This section deals with hardware-caused crashes, and crashes that are software-related, but may be mistaken for hardware problems.

  • Processor memory parity errors - Parity errors are always hardware-related. There is no need to take any action after the first occurrence of a parity error; however, if the error occurs more than once during a 24-hour period, some action should be taken. The first step would be to replace the Dynamic RAM (DRAM). If that doesn't resolve the problem, the router should be replaced. For more information, see Increasing Network Availability and Processor Memory Parity Errors (PMPEs).

    Note: Not all Cisco1600 routers have a DRAM with enabled parity checking. Physically, DRAM is a combination of 2 MB of on-board non-parity chips, and one single inline memory module (SIMM) (with or without Parity). If SIMM is with Parity, on-board 2 MB is disabled.

  • Error interrupts - These errors most often indicate a hardware problem. Capture the output of the show technical-support command and use the TAC Service Request Tool (registered customers only) to open a TAC case and request a hardware replacement.

For all other types of crashes, consult Troubleshooting Router Crashes.

Packet Loss

Packet losses caused by hardware problems are fairly easy to identify. The following section uses the output of the show interfaces command to identify packet losses.

Cyclic Redundancy Check (CRC) and Frame Errors

If cyclic redundancy check errors or frame errors are constantly increasing on the interface, this usually indicates a hardware problem.

router#show interface ethernet 0 
Ethernet0 is up, line protocol is up 
... 

!--- Output suppressed

     121 input errors, 102 CRC, 19 frame, 0 overrun, 0 ignored 

An exception to this is when CRC and frame errors are found on channelized interfaces - they can also indicate clocking problems. The fault that is causing the errors can be anywhere between two connected interfaces - on cables, intermediate devices, or on interfaces themselves. Troubleshooting techniques differ slightly for different interface types:

Serial Interfaces

Consult the Troubleshooting Serial Line Input Errors section of the Troubleshooting Serial Line Problems document.

Integrated Services Digital Network (ISDN) Interfaces

For ISDN interfaces, the only troubleshooting that can be done without assistance from the telephone company is to try another ISDN interface on the router. On Cisco1600 routers, this means trying another WAN interface card (WIC) or another router.

Ethernet Interfaces

For Ethernet interfaces, troubleshooting differs between a shared environment (devices connected through a hub or with a coaxial cable) and a switched environment (devices connected to a switch).

In a switched environment, there are only three components that could cause the error:

  • cable

  • local interface (port)

  • remote interface (port)

Consequently, the troubleshooting steps are simple. For example, if a router is connected to a switch, the troubleshooting steps would be:

  1. Replace the cable.

  2. If this does not solve the problem, try another port on the switch.

  3. If the problem still persists, replace the router.

In a shared environment, the source of the problem is a lot harder to find. Every piece of hardware that makes up the shared segment can be the cause. All components (cables, connectors, and so on) have to be tested one by one.

Ignored Packets

router#show interface ethernet 0 
      Ethernet0 is up, line protocol is up 
      ... 
           21 input errors, 0 CRC, 0 frame, 0 overrun, 21 ignored

Packets are ignored if there are no free buffers to accept the new packet. This can happen if the router is overloaded with traffic, but can also happen if the interface is faulty. If "ignores" are present on all interfaces, then the router is probably overloaded with traffic, or doesn't have sufficient free buffers in the pool that matches the maximum transmission unit (MTU) on interfaces. In the latter case, an increment of the ignored counter is followed by an increment of the no buffer counter:

router#show interfaces serial 0 
... 
     1567 packets input, 0 bytes, 22 no buffer 
     22 input errors, 0 CRC, 0 frame, 0 overrun, 22 ignored, 0 abort

You may also see an increase in the buffer failures counter in the pool that matches the MTU size:

router#show buffers 
... 
Big buffers, 1524 bytes (total 50, permanent 50): 
     50 in free list (5 min, 150 max allowed) 
     3066 hits, 189 misses, 0 trims, 24 created 
     12 failures (0 no memory)

The number of preconfigured permanent, free, and maximum allowed buffers may not be completely compatible for every environment. You can read more about this and how to avoid it in the Buffer Tuning document.

If "ignores" are only increasing on one interface, are not followed by an increment of the no buffer counter, and the interface is not heavily loaded, then this interface could be faulty. In that case, capture the output of the show technical-support command and contact the TAC. The load on the interface can be viewed in the output of the show interfaces command:

router#show interfaces serial 0 
... 
     reliability 255/255, txload 100/255, rxload 122/255

Input and Output Queue Drops

Input queue drops are never caused by hardware problems. Output queue drops may be caused by a hardware problem only if the output queue is constantly full and no packets are being sent out of the interface. You can read more about these kinds of drops in Troubleshooting Input Queue Drops and Output Queue Drops.

If the issue you are having is not listed on this page, please read through the documents in the Related Information section. If you still can't solve your problem, please contact the Technical Assistance Center.

ROMmon Recovery

For information on how to recover a Cisco 1600 Series Router stuck in ROMmon (rommon > prompt), see ROMmon Recovery for the Cisco 1600 Series Router.

Troubleshooting Serial Interfaces

Here is a list of references to use for troubleshooting serial interfaces:

Troubleshooting ISDN Interfaces

Here are some references to use for troubleshooting ISDN interfaces:

Troubleshooting Router Hangs

A hang is when the router boots to a certain point and then no longer accepts any commands or keystrokes. In other words, the console screen hangs after a certain point. Hangs are not necessarily hardware issues and most of the time, they are a software issue. If your router is experiencing a router hang, see Troubleshooting Router Hangs.

Troubleshooting Bus Error Crashes

The system encounters a bus error when the processor tries to access a memory location that either does not exist (a software error) or does not respond properly (a hardware problem). A bus error can be identified by looking at the output of the show version command provided by the router if not power-cycled or manually reloaded.

Here are two examples of bus error crashes:

Router uptime is 2 days, 21 hours, 30 minutes
System restarted by bus error at PC 0x30EE546, address 0xBB4C4
System image file is "flash:igs-j-l.111-24.bin", booted via flash 
.........

At the console prompt, the following error message might also be seen during a bus error:

*** System received a Bus Error exception *** 
signal= 0xa, code= 0x8, context= 0x608c3a50
PC = 0x60368518, Cause = 0x20, Status Reg = 0x34008002

For more information regarding this issue, see Troubleshooting Bus Error Crashes.

Information to Collect if You Open a TAC Service Request

If you still need assistance after following the troubleshooting steps above and want to create a Service Request with the Cisco TAC, be sure to include the following information:
  • Console captures showing the error messages
  • Console captures showing the troubleshooting steps taken and the boot sequence during each step
  • The hardware component that failed and the serial number for the chassis
  • Troubleshooting logs
  • Output from the show technical-support command
You can attach information to your case by uploading it using the TAC Service Request Tool (registered customers only) . If you cannot access the TAC Service Request Tool, you can send the information in an email attachment to attach@cisco.com with your case number in the subject line of your message.

Note: Please do not manually reload or power-cycle the router before collecting the above information, unless required, as this can cause important information to be lost that is needed for determining the root cause of the problem.

Related Information

Updated: Jun 20, 2008
Document ID: 6346