Valuable time and resources are often wasted when you replace hardware that actually functions properly. This document helps troubleshoot potential hardware issues with the Cisco 7300 Series Router, and provides pointers to identify a hardware failure.
Note: This document does not cover any software-related failures except for those that are often mistaken as hardware issues.
Readers of this document should have knowledge of these topics:
The information in this document is based on these software and hardware versions:
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
For more information on document conventions, refer to the Cisco Technical Tips Conventions.
Whenever you install a new line card, module, or Cisco IOS software image, you must ensure that the router has enough memory. You must also check that the hardware and software are compatible with the features you wish to use.
Complete these steps to check for hardware-software compatibility and memory requirements:
Use the Software Advisor (registered customers only) tool to choose software for your network device.
Tip: The Software Support for Hardware (registered customers only) section helps you verify whether the chosen Cisco IOS software version supports the modules and cards installed on the router.
Use the Download Software Area (registered customers only) to check the minimum amount of memory (RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software image. In order to determine the amount of memory (RAM and Flash) installed on your router, see How to Choose a Cisco IOS Software Release - Memory Requirements.
If you want to keep the same features as your current version, but do not know which feature set you have, issue the show version command on your router. Paste the output into the Output Interpreter (registered customers only) tool to find out. Always check for feature support. If you plan to use recent software features, this is especially important.
If you need to upgrade the Cisco IOS software image to a new version or feature set, refer to How to Choose a Cisco IOS Software Release for more information.
If you determine that a Cisco IOS software upgrade is required, follow the Software Installation and Upgrade Procedure for the Cisco 7300 Series Router.
Tip: For information on how to recover a Cisco 7300 Series Router stuck in ROMmon (rommon # > prompt), see ROMmon Recovery Procedure for the Cisco 7300.
In order to determine the cause, the first step is to capture as much information about the problem as possible. This information is necessary to determine the cause of the problem:
Console logs —For more information, see Applying Correct Terminal Emulator Settings for Console Connections.
Syslog information—If you have set the router up to send logs to a syslog server, you may be able to obtain information on what happened. For details, see How to Configure Cisco Devices for Syslog.
show technical-support—The show technical-support command is a compilation of many different commands that includes show version, show running-config, and show stacks. When a router runs into problems, the Cisco Technical Assistance Center (TAC) engineer usually asks for this information to troubleshoot hardware issues. You must collect the show technical-support before you do a reload or power-cycle as these actions can cause all information about the problem to be lost.
Boot sequence information—The complete bootup sequence if the router experiences boot errors.
Crashinfo file (if available)—You can find information on how to obtain the crashinfo file at Retrieving Information from the Crashinfo File.
In order to troubleshoot this kind of problem, you must capture information from the console of the router. Log the console output in a file for later analysis, or for the Cisco Technical Assistance Center (TAC), if you want to open a TAC service request.
If you are not familiar with the boot process of Cisco routers, see Figure 12: Booting Process in the Rebooting Documentation.
If you encounter boot problems, check the symptoms and recommended actions in table 1.
Table 1 – Symptoms and Recommended Actions for Boot Problems
|| Recommended Action
| No light-emitting diodes (LEDs) on after you power on the router.
|| Ensure that the router is plugged firmly into a valid power source. After you power on the router, verify whether you see LED light changes on the power supply as described in Power Supply Overview. The power-up sequence LED for the Network Services Engine (NSE) is:
- On power-up, the hardware lights on the NSE LEDs are orange.
- When the router enters ROMmon, all LEDs are turned off.
- When the router successfully boots on Cisco IOS software, the LEDs turn green.
| Failed LED on Power Supply is red.
|| Check the on/standby switch. If the problem persists, re-seat and replace the power supply as outlined in Removing and Replacing a Power Supply.
| Input Ok LED on Power Supply is off.
|| Verify whether line input voltage is available and is within the proper range:
If the input voltage is OK, replace the power supply. If the input voltage is not OK, the problem is at the power source.
- AC > 90 Vrms (root mean square voltage)
- DC > 38 VDC and polarity is OK
| Appropriate Network Services Engine (NSE) and Power Supply LEDs are on after you power on the router, but the console does not respond.
|| Check your terminal settings.
| Router boots and starts in setup mode every time, although a configuration is stored in the nonvolatile RAM (NVRAM).
|| Verify whether your configuration register is set properly.
| Router boots and is stuck in ROMmon mode (rommon #> prompt).
|| Set the configuration register to 0x2102 and reload the router.
rommon 1 > confreg 0x2102
rommon 2 > reset If the 7300 Router is still in ROMmon mode, see the ROMmon recovery procedure.
| Router boots but is stuck in boot mode (Router (boot)> prompt).
|| If the router is stuck in boot mode, it is possible that the Cisco IOS software image is absent or corrupt. In order to resolve this issue, you may need to upgrade your Cisco IOS software image.
Software and hardware can both cause spontaneous router reboots/reloads or crashes. This document only covers hardware-related crashes. In order to determine whether the problem is hardware-related or software-related, you need the crashinfo and console logs.
Here are examples of crashes that hardware can cause:
Parity Errors—If a parity error occurs only once, it is considered a Single Event Upset (SEU). You do not need to take any action. You can find more information on Single Event Upsets at Increasing Network Availability. If the router reports multiple parity errors, then this is an indication of a hardware problem. See Processor Memory Parity Errors (PMPEs) for more information.
Bus Errors—Software or hardware can cause these types of crashes. See Troubleshooting Bus Errors to help determine whether hardware or software has caused the problem.
Router hangs are most frequently caused by software problems. See Troubleshooting Router Hangs for information on how to handle this issue.
When we refer to a "system crash", we mean a situation where the system has detected an unrecoverable error and has restarted itself. Software problems, hardware problems, or both, can cause a crash. This section deals with hardware-caused crashes, and crashes that are software-related, but can be mistaken for hardware problems.
IMPORTANT: If you reload the router after the crash (for example, through a power-cycle or the reload command), important information about the crash can be lost. Try to collect show technical-support and show log output, and also the crashinfo file (if possible) before you reload the router!
See Troubleshooting Router Crashes for more information on this issue.
In some cases, the processor tries to access a memory location that does not exist (a software error) or does not respond properly (a hardware problem). In these cases, the system encounters a bus error.
In order to identify a bus error, look at the show version output provided by the router (in routers that have not been power-cycled or manually reloaded).
Two examples of bus error crashes are shown here.
Router uptime is 2 days, 21 hours, 30 minutes
System restarted by bus error at PC 0x30EE546, address 0xBB4C4
System image file is "flash:igs-j-l.111-24.bin", booted via flash
You might see this error message at the console prompt during a bus error:
*** System received a Bus Error exception ***
signal= 0xa, code= 0x8, context= 0x608c3a50
PC = 0x60368518, Cause = 0x20, Status Reg = 0x34008002
For more information, see Troubleshooting Bus Error Crashes.
Table 2 lists symptoms and recommended actions for line card issues:
Table 2 – Symptoms and Recommended Actions for Line Card Issues
|| Recommended Action
| show diag output shows empty slot although card is inserted.
|| Verify whether the LEDs are visible. If the LEDs are not visible, remove and reinsert the line card, or try a different slot. If this does not work, it can be a hardware failure with the router or card. Contact the Cisco TAC for further assistance.
| Unknown line card. Error message such as:
00:00:06: %LC-3-LCCREATE: Unable to create driver for
Line Card type 630 in slot 2
| Verify whether the current Cisco IOS software version supports the line card. Use the Software Advisor (registered customers only) tool for this.
| show diag output shows that the line card is a deactivated line card.
|| If the show diag output shows that the line card is deactivated, verify whether the current Cisco IOS software version supports the line card. Use theSoftware Advisor (registered customers only) tool to do this.
By default, the Parallel eXpress Forwarding (PXF) processors are enabled. If, however, you are unsure, or experience problems with PXF, verify whether Cisco Express Forwarding and PXF are enabled. In order to use PXF processing, you must have IP Cisco Express Forwarding switching enabled. In order to verify this, review the output of the show running-config command. If Cisco Express Forwarding is enabled, you see "ip cef" in the configuration output. If PXF is disabled, you see "no ip pxf" in the configuration output. If you do not see "no ip pxf," PXF is enabled.
Enter the show c7300 pxf interface all command to see whether packets received from that interface are PXF-processed or discarded.
Router# show c7300 pxf int all
PXF-If: Y 00001 Gi0/0 (Up, Processing Input)
!--- Processing input => PXF processed
Features: in=CEF [0x208], out=None [0x0] qstatus=XON
In order to troubleshoot PXF further, review the output of the show c7300 pxf accounting command to verify which packets enter and exit the PXF processors.
The 7300 platform introduces a Command Line Interface-based preparation mechanism for Online Insertion and Removal (OIR) of a line card. You can stop particular line card traffic, shut down all interfaces, and deactivate the line card through the hw-module slot slot-number stop command.
While a line card is in the process of deactivation, wait until the OIR LED is green before you issue any commands related to the line card. Also, if the line card is in the process of activation, wait until the OIR LED is out before you issue any commands related to the card.
In order to remove a line card from the Cisco 7304 router without disrupting data flow, use the hw-module slot slot-number stop command. This command stops traffic, turns on the green OIR LED, and shuts down all line card interfaces . Do not remove line cards while there is active traffic.
The stop keyword stops traffic through line card interfaces and deactivates the line card. When the OIR LED turns green, the line card has been deactivated and can be physically removed.
The hw-module slot slot-number start command restarts a line card and shuts off the OIR LED, and puts the card back online. If you used the hw-module slot slot-number stop command, use the hw-module slot slot-number start command to reactivate the line card. You can also use the hw-module slot slot-number start command to restore a line card that has been deactivated due to some failure. You can also reactivate a line card if you physically remove and reinsert the card without the hw-module slot slot-number start command.
Note: Line cards are automatically initialized when you insert them or after a system bootup. You do not need to issue the hw-module slot slot-number start command.
Table 3 presents common error messages on the 7300 and their reasons:
Table 3 – Common Error Messages on 7300 Routers
| Error Message
Line card activation is in progress.
Please retry the command later.
| If you have entered the hw-module slot slot-number start command, any additional configuration of the hw-module slot slot-number start command is ignored.
Line card deactivation is in progress.
Please retry the command later.
| You can use the hw-module slot slot-number stop command to deactivate a line card. However, if you issue the hw-module slot slot-number start command before the OIR LED turns green and the deactivation process is complete, you see this message.
Command cannot be executed.
Line card status is deactivated.
| If a line card is already deactivated, you see this message. The hw-module slot slot-number stop command is ignored.
| If you still need assistance after you follow the troubleshooting steps above you can open a service request (registered customers only) with the Cisco TAC. Be sure to include the information listed here:
Please attach the collected data to your service request in non-zipped, plain text format (.txt). You can upload information to your service request with the TAC Service Request Tool (registered customers only) . If you cannot access the Service Request tool, you can send the information in an email attachment to email@example.com. Include your service request number in the subject line of your message to attach the relevant information to your service request.
- Console captures that show the error messages.
- Console captures that show the steps you took to troubleshoot the problem and the boot sequence during each step.
- The hardware component that failed and the serial number for the chassis.
- Troubleshooting logs.
- Output from the show technical-support command.
Note: Please do not manually reload or power-cycle the router before you collect the above information, unless absolutely necessary. This can cause important information needed to determine the root cause of the problem to be lost.