Guest

Cisco 4000 Series Routers

Hardware Troubleshooting for the Cisco 4000 Series Router

Cisco - Hardware Troubleshooting for the Cisco 4000 Series Router

Document ID: 17934

Updated: Jul 07, 2005

   Print

Introduction

Valuable time and resources are often wasted replacing hardware that actually functions properly. This document helps troubleshoot potential hardware issues with Cisco 4000 Series Routers, and can help you identify which component may be causing a hardware failure, depending on the type of error that the router is experiencing.

Note: This document does not cover any software-related failures except for those that are often mistaken as hardware issues.

Prerequisites

Requirements

Readers of this document should be knowledgeable of the following:

Components Used

The information in this document is based on the software and hardware versions below.

  • Cisco 4000 Series Routers including the following:

    • 4000/4000-M

    • 4500/4500-M

    • 4700/4700-M

  • All Cisco IOS® software versions that run on the Cisco 4000 Series Router

The information presented in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If you are working in a live network, ensure that you understand the potential impact of any command before using it.

Determining the Router Type

The different types of 4000 Series Routers are:

Product Type Revision Number Label
4000 Revision A0 440xxxxx C4000
4000M Revision B0, C 445xxxxx C4000 M+
4500 Revision 0x00 450xxxxx C4500
4500M Revision B, C, D, E 455xxxxx C4500 M+
4700 Revision B 470xxxxx C4700
4700M Revision C, D, E, F 475xxxxx C4700 M+

To determine what type of router you have, use the show version command. The output should look similar to this:

cisco 4500 (R4K) processor (revision 0x00) with 32768K/4096K bytes of memory.  

This identifies the router as a Cisco 4500.

Hardware-Software Compatibility and Memory Requirements

Whenever you install a new card, module, or Cisco IOS software image, it is important to verify that the router has enough memory, and that the hardware and software are compatible with the features you wish to use.

Perform the following recommended steps to check for hardware-software compatibility and memory requirements:

  1. Use the Software Advisor (registered customers only) tool to choose software for your network device.

    Tips:

  2. Use the Cisco Download Software Area (registered customers only) to check the minimum amount of memory (RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software image. To determine the amount of memory (RAM and Flash) installed on your router, refer to the Memory Requirements section of How to Choose a Cisco IOS Software Release.

    Tips:

    • If you want to keep the same features as the version that is currently running on your router, but don't know which feature set you are using, enter the show version command on your router and paste it into the Output Interpreter (registered customers only) tool to find out. It is important to check for feature support, especially if you plan to use recent software features.

    • If you need to upgrade the Cisco IOS software image to a new version or feature set, refer to How to Choose a Cisco IOS Software Release for more information.

  3. If you determine that a Cisco IOS software upgrade is required, follow the Software Installation and Upgrade Procedure for the Cisco 4000 Series Router.

    Tips:

Error Messages

The Error Message Decoder (registered customers only) tool allows you to check the meaning of an error message. Error messages appear on the console or in the show log output of Cisco products, usually in the following form:

%XXX-n-YYYY : [text]

Here is an example error message:

Router# %SYS-2-MALLOCFAIL: Memory allocation of [dec] bytes failed from [hex], 
pool [chars], alignment [DEC]  

Some error messages are informational only, while others indicate hardware or software failures and require action. The Error Message Decoder (registered customers only) tool provides an explanation of the message, a recommended action (if needed), and if available, a link to a document that provides extensive troubleshooting information about that error message.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Identifying the Issue

This section explains what to do to determine the cause of the potential hardware issue(s).

In order to determine the cause, the first step is to capture as much information about the problem as possible. The following information is essential for determining the cause of the problem:

  • Console logs - For more information, refer to Applying Correct Terminal Emulator Settings for Console Connections.

  • Syslog information - If the router is set up to send logs to a syslog server, you may be able to obtain information on what happened. For details, refer to the How to Configure Cisco Devices for Syslog section of Resource Manager Essentials and Syslog Analysis: How-To.

  • show technical-support command output - The show technical-support command is a compilation of many different commands including show version, show running-config, and show stacks. TAC engineers usually ask for this information to troubleshoot hardware issues. It is important to collect the show technical-support information before doing a reload or power-cycle as these actions can cause all information about the problem to be lost.

  • The complete bootup sequence if the router experiences boot errors.

If you have the output of a show command from your Cisco device (including show technical-support), you can use the Output Interpreter (registered customers only) to display potential issues and fixes. To use the Output Interpreter (registered customers only) , you must be logged in and have JavaScript enabled.

Recent Changes

If you have implemented any recent changes on the router such as a Cisco IOS software upgrade or a new hardware installation, the quickest solution may be to revert back to the original hardware or software. If you recently upgraded the Cisco IOS software, make sure there is enough Dynamic RAM (DRAM) for the image. Since the router has separate banks of memory for both the processor pool and the Input/Output (I/O) pool, you cannot add the main DRAM and the shared DRAM to get the total amount of memory on this platform.

You can determine how much main DRAM and shared DRAM is installed in the router by looking at the show version output:

Router#show version 
<snip>
cisco 4000 (68030) processor (revision 0xA0) with 16384K/4096K bytes 
of memory. 
!--- main DRAM is in bold

Processor board ID 5004788 
<snip>

Router#show version
<snip>
cisco 4000 (68030) processor (revision 0xA0) with 16384K/4096K bytes 
of memory.
!--- shared DRAM is in bold

Processor board ID 5004788
<snip>

See Figures 1, 2, and 3 below to find the location of the main and shared DRAM in your 4000 Series Router.

Figure 1: Cisco 4000, Cisco 4500, and Cisco 4700 SIMM Locations

4000_simm_locations.gif

Figure 2: Cisco 4000-M SIMM Locations

4000M_simm_locations.gif

Figure 3: Cisco 4500-M and Cisco 4700-M SIMM Locations

45_4700M_simm_locations.gif

Router Reboots

The router can reboot or reload due to various reasons. When the router reboots, it returns to a normal state (meaning that it is passing traffic and allows you access into the router); however, it may reboot again. The table below provides some common reasons why the router reboots, along with troubleshooting tips. If you are experiencing one of these issues, just click on the link and it takes you to the troubleshooting steps for that particular issue. To check why the router rebooted, issue the show version command and look at the output.

Reason for reboot Steps to take
Reload due to a router crash When we refer to a "system crash", we mean a situation where the system has detected an unrecoverable error and has restarted itself. A crash can be caused by software problems, hardware problems, or both. This section deals with hardware-caused crashes and crashes that are software-related, but may be mistaken for hardware problems. IMPORTANT: If the router is reloaded after the crash (for example, through a power-cycle or the reload command), important information about the crash will be lost, so try to collect show technical-support and show log output, as well as the crashinfo file (if possible) before reloading the router! Refer to Troubleshooting Router Crashes for more information regarding this issue
Reload due to a bus error crash The system encounters a bus error when the processor tries to access a memory location that either does not exist (a software error) or does not respond properly (a hardware problem). A bus error can be identified by looking at the output of the show version command provided by the router (if not power-cycled or manually reloaded). Here are two examples of bus error crashes:
Router uptime is 2 days, 21 hours, 30 minutes
System restarted by bus error at PC 0x30EE546, address 0xBB4C4
System image file is "flash:igs-j-l.111-24.bin", booted via flash 
.........
At the console prompt, the following error message might also be seen during a bus error:
*** System received a Bus Error exception *** 
signal= 0xa, code= 0x8, context= 0x608c3a50
PC = 0x60368518, Cause = 0x20, Status Reg = 0x34008002
For more information regarding this issue, refer to Troubleshooting Bus Error Crashes.
Reload due to a parity error At the first occurrence, simply monitor the router. At the second occurrence, replace the corresponding hardware as described in Processor Memory Parity Errors.
Reload due to a bus error Check the memory map of a show region versus the address of the bus error. If the address is valid, this is most likely a hardware issue. If the address is invalid, there is a problem with the version of Cisco IOS software that is running. Try using the Output Interpreter (registered customers only) tool to display potential issues and fixes. Refer to Troubleshooting Bus Error Crashes for more information regarding this issue.
Reload due to a software-forced crash This is almost always a software problem. Upgrade to the latest Cisco IOS software release in your release train.
Reload due to a SegV error SegV errors are always software-related problems. Upgrade to the latest Cisco IOS software release in your release train, or use the Output Interpreter (registered customers only) tool to display potential issues and fixes. You can also refer to SegV Exceptions for more information regarding this issue.
Reload due to watchdog timer expired Most of the time, these messages indicate a hardware issue. Replacing the CPU board usually solves the problem, unless other elements (for example, if a new module has been inserted, and it starts reloading) point to a specific piece of hardware. Refer to Troubleshooting Watchdog Timeouts for more information on how to troubleshoot this problem.
What Causes a Router To Be Restarted By the abort or trace trap Commands? If you don't power-cycle or manually reload the router, the show version output displays the following:
Router uptime is 1 minute 
System restarted by abort at PC 0x802737BC 
System image file is "flash:c2600-i-mz.120-4.T"
or
Router uptime is 2 minutes 
System restarted by trace trap at PC 0x3171310 
System image file is "flash:c2500-jos56i-l.120-9.bin" 
Why Does My Router Lose Its Configuration During Reboot? In most cases, this is the result of an improperly set configuration register. The configuration register is usually changed during password recovery to bypass the startup configuration upon reboot. Many times, the configuration register is not returned back to a normal setting.
Router stuck in ROMmon (rommon # > prompt) For information on how to recover a Cisco 4000 Series Router stuck in ROMmon (rommon # > prompt), refer to ROMmon Recovery for the Cisco 4000 Series Router or ROMmon Recovery for the Cisco 4500 and 4700 Series Routers.

For more information, please refer to Less Common Types of System Crashes.

Continuous/Boot Loop

If the router is stuck in a loop, you must have access to the console to do further troubleshooting:

  1. Get console access to the router.

  2. Send the break sequence to enter ROMmon.

  3. Verify that the router has enough memory to run the Cisco IOS software image in Flash by using the meminfo command.

  4. If the router does not have enough memory, set the router configuration register to 0x2101 to boot the boot image.

  5. Install a Cisco IOS software that loads properly. Instructions for this can be found in the Software Installation and Upgrade Procedure.

Troubleshooting

This section provides troubleshooting references, tips, and guidelines depending on the type of issue you may be experiencing.

Troubleshooting Serial Interfaces

Here is a list of references to use for troubleshooting serial interfaces:

Troubleshooting ISDN Interfaces

Here are some references to use for troubleshooting ISDN interfaces:

Troubleshooting Memory Problems

If a 4000 Series Router does not have enough memory, this can result in boot errors or other issues such as %SYS-2-MALLOCFAIL: Memory Allocation Failure errors.

Troubleshooting Router Hangs

A 4000 Series Router may experience a router hang. A hang is when the router boots to a certain point and then no longer accepts any commands or keystrokes. In other words, the console screen hangs after a certain point. Hangs are not necessarily hardware issues and most of the time, they are a software issue. If your router is experiencing a router hang, refer to Troubleshooting Router Hangs.

Troubleshooting the Blinking OK Light

When the OK LED light on the Cisco 4000 Series Router blinks, the number of times it blinks provides useful information. The light blinks n times, pauses, and then blinks n times again. Count what number n is, and look it up in the following list:

Note: These are only valid for the Cisco 4000 Series Router.

  • 3 = Initial system control register is bad

  • 4 = Write/read of system control register has failed

  • 5 = First write to main memory has failed

  • 6 = Memory test of low RAM has failed (this message is also printed)

  • 7 = Failed sizing main memory (this message is also printed)

  • 8 = Main memory parity error (this message is also printed)

  • 9 = Real time clock has failed (this message is also printed)

  • 10 = Shared memory parity error (this message is also printed)

Troubleshooting %SYS-3-CPUHOG Messages

For information regarding this issue, refer to What Causes %SYS-3-CPUHOG Messages?

Information to Collect if You Create a TAC Service Request

If you have identified a component that needs to be replaced, contact your Cisco partner or reseller to request a replacement for the hardware component that is causing the issue. If you have a support service agreement directly with Cisco, use the TAC Service Request Tool (registered customers only) to create a TAC service request for a hardware replacement. Make sure you attach the following information:
  • Console captures showing the error messages
  • Console captures showing the troubleshooting steps taken and the boot sequence during each step
  • The hardware component that failed and the serial number for the chassis
  • Troubleshooting logs
  • Output from the show technical-support command

Related Information

Updated: Jul 07, 2005
Document ID: 17934