This document describes how to troubleshoot potential hardware issues with Cisco 1600 Series Routers and can help you identify which component might cause a hardware failure, depending on the type of error that the router is experiencing.
Note: This document does not describe software-related failures except for those that are often mistaken as hardware issues.
Readers of this document should be knowledgeable of the following:
The information in this document is based on Cisco 1600 series routers.
The information presented in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If you are working in a live network, ensure that you understand the potential impact of any command before using it.
Whenever you install a new card, module, or Cisco IOS® software image, it is important to verify that the router has enough memory, and that the hardware and software are compatible with the features you wish to use.
Perform the following recommended steps to check for hardware-software compatibility and memory requirements:
Use the Download Software Area (registered customers only) to check the minimum amount of memory (RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software image. To determine the amount of memory (RAM and Flash) installed on your router, see How to Choose a Cisco IOS Software Release - Memory Requirements.
If you want to keep the same features as the version that is currently running on your router, but don't know which feature set you are using, enter the show version command on your router and paste it into the Output Interpreter (registered customers only) tool to find out. It is important to check for feature support, especially if you plan to use recent software features.
If you need to upgrade the Cisco IOS software image to a new version or feature set, see How to Choose a Cisco IOS Software Release for more information.
If you determine that a Cisco IOS software upgrade is required, follow the Software Installation and Upgrade Procedure for the Cisco 1600 Series Router.
Tip: If your 1600 does not have a connection to the network or does not have a valid Cisco IOS software image, you may need to use the console port of the router to perform an xmodem software upgrade using ROMmon. This procedure does not require the use of a Trivial File Transfer Protocol (TFTP) server.
If the router does not boot after a Cisco IOS software upgrade, go to the Cisco 1600 Does Not Boot section of this document for more information.
%XXX-n-YYYY : [text]
Here is an example error message:
Router# %SYS-2-MALLOCFAIL: Memory allocation of [dec] bytes failed from [hex], pool [chars], alignment [DEC]
Some error messages are informational only, while others indicate hardware or software failures and require action. The Error Message Decoder (registered customers only) tool provides an explanation of the message, a recommended action (if needed), and if available, a link to a document that provides extensive troubleshooting information about that error message.
For more information on document conventions, see the Cisco Technical Tips Conventions.
The Cisco 1600 series consists of four models, each of which has an Ethernet port, a built-in WAN port, and a slot for an optional second WAN port:
|Cisco 1601||One Ethernet, one serial, one WAN interface card slot|
|Cisco 1602||One Ethernet, one serial with integrated 56-kbps data service unit/channel service unit (DSU/CSU), one WAN interface card slot|
|Cisco 1603||One Ethernet, one ISDN Basic Rate Interface (BRI) (S/T interface), one WAN interface card slot|
|Cisco 1604||One Ethernet, one ISDN BRI with integrated Network Termination 1 (NT1) (U interface), one S-bus port for ISDN phones, on WAN interface card slot|
For WIC compatibility information, see the WIC Hardware Compatibility Matrix.
See Cisco 1600 Series and WAN Interface Cards for additional information.
This section explains what to do to determine the cause of the potential hardware issue(s).
In order to determine the cause, the first step is to capture as much information about the problem as possible. The following information is essential for determining the cause of the problem:
Console logs - For more information, see Applying Correct Terminal Emulator Settings for Console Connections.
Syslog information - If the router is set up to send logs to a syslog server, you may be able to obtain information on what happened. For details, see How to Configure Cisco Devices for Syslog.
show technical-support command output - The show technical-support command is a compilation of many different commands including show version, show running-config, and show stacks. TAC engineers usually ask for this information to troubleshoot hardware issues. It is important to collect the show technical-support information before doing a reload or power-cycle as these actions can cause all information about the problem to be lost.
The complete bootup sequence if the router experiences boot errors.
Capturing information from the console of the router is essential for troubleshooting this kind of problem. The console output should be logged in a file for later analysis or for the Cisco Technical Assistance Center (TAC), should a TAC Service Request be created.
Cisco1600 routers have a Bootstrap image and the Boot image embedded in the ROM. Even if there are no valid images on the Flash, the router should always be able to load these images. If you are not familiar with the booting process of Cisco routers, see "Figure 12: Booting Process" in the Rebooting Documentation. You can also have a look at Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.2.
The following table lists symptoms and recommended actions to take if you are encountering booting problems:
|No LEDs are on after powering on the router.||If your router is a Cisco 1601-R, check the serial number against a known issue described in Cisco 1601-R Front Panel LEDs Will Not Illuminate. If your router is affected, it should still operate normally, despite the LEDs. Otherwise, check if the power supply is plugged in firmly. If that does not resolve the issue, replace the power supply. If the problem persists, replace the router.|
|LEDs are on after powering on the router, but there is nothing on the console.||Verify that the baud rate is set to 9600 bps. If that doesn't help, verify that the equipment used for connecting to the console is operating properly. You can do this by connecting to a known good router to check your console equipment. If the equipment is successfully tested, but the problem remains, replace the router.|
Router displays the System Bootstrap version and hangs at that
point or falls into a booting loop:
System Bootstrap, Version 11.1(7)AX [kuong (7)AX], EARLY DEPLOYMENT RELEASE SOFTWARE (fc2) Copyright (c) 1994-1996 by cisco Systems, Inc.
|The memory may not be well seated. First try to reseat (remove and then reinsert) the single inline memory module (SIMM) memory. If the router still does not boot, replace the router. You can find information on the location of the Dynamic RAM (DRAM) SIMMs in Installing or Replacing the DRAM SIMM in Cisco 1600 Series Routers.|
|Router boots in ROMmon; no error messages on the console.||
Set the configuration register to 0x2102 and reload the router:
rommon 1 > confreg 0x2102
rommon 2 > resetIf the router remains in ROMmon, follow the procedure described in ROMmon Recovery for the Cisco 1600 Series Router.
|Router boots in boot mode; no error messages on the console.||
Set the configuration register to 0x2102 and reload the router.
There is no need to save the running configuration before reloading:
router(boot)#configure terminal Enter configuration commands, one per line. End with CNTL/Z. router(boot)(config)#config-register 0x2102 router(boot)(config)#end router(boot)#reload System configuration has been modified. Save? [yes/no]: no Proceed with reload? [confirm]
Note: The config-register command is the only Cisco IOS software configuration command which doesn't get saved in non-volatile RAM (NVRAM). It changes the configuration register immediately, but this only takes effect during the next boot.
Router boots in boot mode, with the following messages on the
getdevnum warning: device "PCMCIA slot 1" has size of zero getdevnum warning: device "PCMCIA slot 1" has size of zero pen: read error...requested 0x4 bytes, got 0x0 trouble reading device magic number boot: cannot open "flash:" boot: cannot determine first file name on device "flash:"
The Flash card is removed or badly inserted. Verify the
presence of the Flash card by running the show flash
router(boot)#show flash %Device in REMOVED stateInsert or reseat the Flash. If the problem persists, replace the Flash card. If that doesn't resolve the problem, replace the router.
Router boots in boot mode, with the following messages on the
device does not contain a valid magic number boot: cannot open "flash:" boot: cannot determine first file name on device "flash:"
|The Flash is empty or the filesystem is corrupted. Copy a valid image on the Flash, and while copying, you will be prompted to erase the old Flash (if one exists). Then reload the router. For more information on loading new software on the 1600 router, see Software Installation and Upgrade Procedure for the 1600 or Software Installation and Upgrade Procedure for the 1600-R.|
Router boots in boot mode, with the following message on the
SYSTEM INIT: INSUFFICIENT MEMORY TO BOOT THE IMAGE!
This message only occurs when loading images that run from RAM
(for example, c1600-y-mz.120-20). Since Cisco1601-1604 routers support images
that run in RAM and images that run directly from Flash (for example,
relocatable images), there are two solutions for the problem:
Router boots in boot mode, with the following messages on the
loadprog: error - Invalid image for platform
|The Cisco IOS software image is not appropriate for the 1600 router, or it is corrupted. Erase the Flash and download a new valid Cisco IOS software image. For more information on loading new software on the 1600 router, see Software Installation and Upgrade Procedure for the 1600 or Software Installation and Upgrade Procedure for the 1600-R.|
|Router boots correctly, but the WAN interface of the WAN interface card (WIC) is not available.||
Verify the presence of the WIC card by running the
show diag command from enable mode. If the router
doesn't detect the WIC card or if there are no WIC cards installed, the command
has the following output:
router#show diag No WAN daughter card presentReseat the WIC card. Push it in firmly and tighten the screws. If that doesn't resolve the problem, replace the WIC. If the problem persists, replace the router. If the show diag command shows that there is a WIC card installed, but the information is not correct, use the Software Advisor (registered customers only) to verify that the WIC is supported in the running Cisco IOS software release. If the information is correct and the WIC is supported in the running Cisco IOS software, contact the Cisco Technical Assistance Center (TAC) for further troubleshooting.
When we refer to a "system crash", we mean a situation where the system has detected an unrecoverable error and has restarted itself. A crash can be caused by software problems, hardware problems, or both. This section deals with hardware-caused crashes, and crashes that are software-related, but may be mistaken for hardware problems.
Processor memory parity errors - Parity errors are always hardware-related. There is no need to take any action after the first occurrence of a parity error; however, if the error occurs more than once during a 24-hour period, some action should be taken. The first step would be to replace the Dynamic RAM (DRAM). If that doesn't resolve the problem, the router should be replaced. For more information, see Increasing Network Availability and Processor Memory Parity Errors (PMPEs).
Note: Not all Cisco1600 routers have a DRAM with enabled parity checking. Physically, DRAM is a combination of 2 MB of on-board non-parity chips, and one single inline memory module (SIMM) (with or without Parity). If SIMM is with Parity, on-board 2 MB is disabled.
Error interrupts - These errors most often indicate a hardware problem. Capture the output of the show technical-support command and use the TAC Service Request Tool (registered customers only) to open a TAC case and request a hardware replacement.
For all other types of crashes, consult Troubleshooting Router Crashes.
Packet losses caused by hardware problems are fairly easy to identify. The following section uses the output of the show interfaces command to identify packet losses.
If cyclic redundancy check errors or frame errors are constantly increasing on the interface, this usually indicates a hardware problem.
router#show interface ethernet 0 Ethernet0 is up, line protocol is up ... !--- Output suppressed 121 input errors, 102 CRC, 19 frame, 0 overrun, 0 ignored
An exception to this is when CRC and frame errors are found on channelized interfaces - they can also indicate clocking problems. The fault that is causing the errors can be anywhere between two connected interfaces - on cables, intermediate devices, or on interfaces themselves. Troubleshooting techniques differ slightly for different interface types:
Integrated Services Digital Network (ISDN) Interfaces
For ISDN interfaces, the only troubleshooting that can be done without assistance from the telephone company is to try another ISDN interface on the router. On Cisco1600 routers, this means trying another WAN interface card (WIC) or another router.
For Ethernet interfaces, troubleshooting differs between a shared environment (devices connected through a hub or with a coaxial cable) and a switched environment (devices connected to a switch).
In a switched environment, there are only three components that could cause the error:
local interface (port)
remote interface (port)
Consequently, the troubleshooting steps are simple. For example, if a router is connected to a switch, the troubleshooting steps would be:
Replace the cable.
If this does not solve the problem, try another port on the switch.
If the problem still persists, replace the router.
In a shared environment, the source of the problem is a lot harder to find. Every piece of hardware that makes up the shared segment can be the cause. All components (cables, connectors, and so on) have to be tested one by one.
router#show interface ethernet 0 Ethernet0 is up, line protocol is up ... 21 input errors, 0 CRC, 0 frame, 0 overrun, 21 ignored
Packets are ignored if there are no free buffers to accept the new packet. This can happen if the router is overloaded with traffic, but can also happen if the interface is faulty. If "ignores" are present on all interfaces, then the router is probably overloaded with traffic, or doesn't have sufficient free buffers in the pool that matches the maximum transmission unit (MTU) on interfaces. In the latter case, an increment of the ignored counter is followed by an increment of the no buffer counter:
router#show interfaces serial 0 ... 1567 packets input, 0 bytes, 22 no buffer 22 input errors, 0 CRC, 0 frame, 0 overrun, 22 ignored, 0 abort
You may also see an increase in the buffer failures counter in the pool that matches the MTU size:
router#show buffers ... Big buffers, 1524 bytes (total 50, permanent 50): 50 in free list (5 min, 150 max allowed) 3066 hits, 189 misses, 0 trims, 24 created 12 failures (0 no memory)
The number of preconfigured permanent, free, and maximum allowed buffers may not be completely compatible for every environment. You can read more about this and how to avoid it in the Buffer Tuning document.
If "ignores" are only increasing on one interface, are not followed by an increment of the no buffer counter, and the interface is not heavily loaded, then this interface could be faulty. In that case, capture the output of the show technical-support command and contact the TAC. The load on the interface can be viewed in the output of the show interfaces command:
router#show interfaces serial 0 ... reliability 255/255, txload 100/255, rxload 122/255
Input queue drops are never caused by hardware problems. Output queue drops may be caused by a hardware problem only if the output queue is constantly full and no packets are being sent out of the interface. You can read more about these kinds of drops in Troubleshooting Input Queue Drops and Output Queue Drops.
If the issue you are having is not listed on this page, please read through the documents in the Related Information section. If you still can't solve your problem, please contact the Technical Assistance Center.
For information on how to recover a Cisco 1600 Series Router stuck in ROMmon (rommon > prompt), see ROMmon Recovery for the Cisco 1600 Series Router.
Here is a list of references to use for troubleshooting serial interfaces:
Here are some references to use for troubleshooting ISDN interfaces:
A hang is when the router boots to a certain point and then no longer accepts any commands or keystrokes. In other words, the console screen hangs after a certain point. Hangs are not necessarily hardware issues and most of the time, they are a software issue. If your router is experiencing a router hang, see Troubleshooting Router Hangs.
The system encounters a bus error when the processor tries to access a memory location that either does not exist (a software error) or does not respond properly (a hardware problem). A bus error can be identified by looking at the output of the show version command provided by the router if not power-cycled or manually reloaded.
Here are two examples of bus error crashes:
Router uptime is 2 days, 21 hours, 30 minutes System restarted by bus error at PC 0x30EE546, address 0xBB4C4 System image file is "flash:igs-j-l.111-24.bin", booted via flash .........
At the console prompt, the following error message might also be seen during a bus error:
*** System received a Bus Error exception *** signal= 0xa, code= 0x8, context= 0x608c3a50 PC = 0x60368518, Cause = 0x20, Status Reg = 0x34008002
For more information regarding this issue, see Troubleshooting Bus Error Crashes.
|If you still need assistance after following the troubleshooting steps above and want to create a Service Request with the Cisco TAC, be sure to include the following information:|
Note: Please do not manually reload or power-cycle the router before collecting the above information, unless required, as this can cause important information to be lost that is needed for determining the root cause of the problem.
- Cisco 1600 Series Routers
- Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.2
- Field Notice: Cisco 1601-R Front Panel LEDs Will Not Illuminate
- Software Installation and Upgrade Procedure for the Cisco 1600 Series Router
- Software Installation and Upgrade Procedure for the Cisco 1600-R
- Increasing Network Availability
- Buffer Tuning
- Troubleshooting Input Queue Drops and Output Queue Drops
- Technical Support & Documentation - Cisco Systems
The Cisco Support Community is a forum for you to ask and answer questions, share suggestions, and collaborate with your peers.
Refer to Cisco Technical Tips Conventions for information on conventions used in this document.