| |
Troubleshoot the Device Prior to request for Return Material Authorization (RMA)
|
|
|
|
Introduction
This document provides instructions to a customer on how to
troubleshoot a common problem with SMB devices prior to its request for
RMA.
The basis of this document is, when a customer opens a case and wants a
replacement for a device, the customer can perform few steps to troubleshoot to
ensure that it is a hardware issue. If the customer believes that, the device
needs a RMA, along the way to create the case for RMA, customer can give their
delivery address, contact name, contact person details so that
SMB
TAC can process the RMA much faster.
Back to Top
Requirements
To perform the steps described in this document, you need to have these
items:
-
A PC with a serial port. The PC must be close enough to connect
directly to the Cisco device that you want to access.
-
A console cable or rolled cable with an adapter. Refer to
Cable
Descriptions for further information.
-
Terminal-emulation software such as HyperTerminal on your
PC
Back to Top
Basic Troubleshooting Steps
If your device experiences some hardware or software issue, check these
items to help resolve the issue before you contact
SMB
TAC.
Note: Some of the steps to troubleshoot this document is not applicable to
few SMB devices.
Troubleshoot the Power and Cooling Systems
Follow these steps to troubleshoot power related issues on your
device:
-
Connect the AC power source to the power connector of the device as
indicated in the document that shipped with your device.
-
Press the power switch to the on ( | ) position if it is available
in the device.
-
When the device powers on, it begins the power-on self test (POST),
a series of automatic tests that confirm proper operation. POST lasts
approximately for 1-2 minutes. When POST starts, the LEDs blink and go out.
When POST completes, the PWR or SYST LEDs turn solid
green.
-
With the power switch on, does the PWR or SYST LED stay on or
blink?
-
If the LED is green, the device receives the power and it is
functional.
-
If the LED is off, check the power source and power cable or use
a different Power outlet and cord that has worked with some other
device.
-
With the power switch on and the PWR or SYST LEDs on, do the fans
operate?
-
With the power switch on and the system LED off, do the fans
operate?
-
If yes, the device receives the power. The fans are connected
directly to the DC outputs of the power supply.
-
If no, check the power source and power cable or use a different
Power outlet and cord that has worked with some other
device.
-
For Cisco Aironet Access Points, if the Ethernet, Status, and Radio
LEDs on the top of the AP turns Red or Amber it is a Boot Loader error. Contact
the
SMB TAC for assistance.
Troubleshoot Device Console Port
If you have problems to access your device console port, follow these
steps:
Create a HyperTerminal Connection to your Device:
-
Connect the console or rolled cable to the serial port of your
PC.
-
Connect the cable to the console port of the Cisco device that you
want to access. This example uses a Cisco 800 series
router.
-
To open HyperTerminal, go to Start >
Programs > Accessories >
Communications > HyperTerminal.
Note: The menu path to HyperTerminal differs slightly on your
PC.
-
In the Connection Description window, enter a name
for the connection and click OK.
-
In the Connect To window, select the
communications port on your computer that you want to use to create the
HyperTerminal connection. This port is COM1 or COM2 on many
PCs.
Click OK.
-
Enter the port settings. Change the default values to these
values:
-
Be sure that your PC is connected to the device, and click
OK to initiate the connection.
-
Press Enter. The HyperTerminal window displays a
command-line prompt.
Note: Cisco devices display different output if there is no
configuration in memory or if the device starts to boot.
-
If the session shows that you are connected, but if you see a
blank page displayed in the HyperTerminal window, follow these steps:
-
Press Enter several times to refresh the output
on the screen.
-
Ensure that the console cable is properly connected to the
computer and to the device.
-
Ensure that you do not use a faulty cable. Use a console cable
that has previously worked with some other device.
-
Ensure that you selected the correct connection
settings.
-
Ensure that the PC serial port connected to the cable matches the
serial port that you selected in the menu dialogue.
-
Ensure that the console or rolled cable is attached to the blue
console port of the Cisco device.
-
To identify a RJ-45 to DB-9 female console cable or rolled cable
with an adapter, refer to
Cable
Descriptions.
-
If you are still unable to connect to the device console, the
console port is faulty. Contact the
SMB
TAC for assistance.
Troubleshoot the Device Boot Problem
Here are a few common problems the device encounters while it
boots:
-
Power on the device and login into its command line interface
through console connection. For more information on how to connect to the
console, refer Create a
HyperTerminal Connection to your Device
-
If the device continuously reboots, follow these steps:
-
Boot the device without any cards and modules inserted. If you
have added any new module which is damaged the device does not boot
properly.
-
Break into ROMMON and press CTRL+Break keys when the device
reboots.
-
If the device goes into the ROMMON mode it seems that the device
does not find an IOS software image to load. Follow these instructions to find
a valid IOS and to boot the device properly:
rommon 1 >dir flash:
program load complete, entry point: 0x8000f000, size: 0xc0c0
Initializing ATA monitor library.......
Directory of flash:
2 36673864 -rw c3825-advipservicesk9-mz.124-12.bin
4424 1649 -rw- sdmconfig-38xx.cfg
4425 4052480 -rw- sdm.tar
5415 812032 -rw- es.tar
5614 1007616 -rw- common.tar
5860 1038 -rw- home.shtml
5861 113152 -rw- home.tar
5889 756288 -rw- 256MB.sdf
!--- Output truncated.
-
Use the command boot flash: <file name>.
This command Boots the device with the IOS specified. If you have a switch, use
the commands Flash_init and Boot_helper
before you enter the boot flash: <file name>
command.
-
Once you are able to boot the device properly, kindly make sure
that the device boots from flash by the command config-register
0x2102 in the device global configuration mode.
Note: The command config-register 0x2102 is not
supported for all the products. Contact the
SMB
TAC for further assistance.
-
If the device still does not boot the IOS image, check for Device
DRAM memory size.
-
The IOS image needs more memory than the one the device has, if
the device is unable to boot properly.
-
Use the command meminfo from the rommon mode and
check the main memory size of the device.
rommon 2 >meminfo
------------------------------------------
Current Memory configuration is:
DIMM0: Size = 256 MB : Start Phy Addr = 0x00000000
-------------------------------------------------
Main memory size: 256 MB in 72/0(DIMM0/DIMM1) bit mode.
Available main memory starts at 0xa0015000, size 262060KB
Smart Init is enabled.
NVRAM size: 480KB
!--- Output truncated.
-
The Device performed POST, but the SYST LED turned amber instead of
green. The Device failed POST. POST failures are usually fatal. Capture all the
error messages that occur while the device bootup and contact the
SMB
TAC for further assistance.
Troubleshoot Modules and Interface Cards
If your Router interfaces and modules are not recognized, follow these
simple steps to help identify and resolve the module issues:
-
To verify hardware information for the network device, use the
show diag command in EXEC or privileged EXEC mode.
Router#show diag
3725 Backplane EEPROM:
PCB Serial Number : JAE0821JA9S
Processor type : 61
Top Assy. Part Number : 800-16147-02
Board Revision : D0
Fab Part Number : 28-4226-06
Deviation Number : 65535-65535
Manufacturing Test Data : FF FF FF FF FF FF FF FF
RMA Number : 255-255-255-255
RMA Test History : FF
RMA History : FF
Chassis Serial Number : JMX0836L10L
Chassis MAC Address : 0011.218d.dd30
MAC Address block size : 48
Field Diagnostics Data : FF FF FF FF FF FF FF FF
Hardware Revision : 0.1
Number of Slots : 2
EEPROM format version 4
EEPROM contents (hex):
0x00: 04 FF C1 8B 4A 41 45 30 38 32 31 4A 41 39 53 09
0x10: 61 40 02 59 C0 46 03 20 00 3F 13 02 42 44 30 85
Slot 0:
C3725 Mother board 2FE(TX)-3W Port adapter, 4 ports
Port adapter is analyzed
Port adapter insertion time unknown
EEPROM contents at hardware discovery:
PCB Serial Number : JAE0821JA9S
Processor type : 61
Top Assy. Part Number : 800-16147-02
Board Revision : D0
Fab Part Number : 28-4226-06
Deviation Number : 65535-65535
Manufacturing Test Data : FF FF FF FF FF FF FF FF
RMA Number : 255-255-255-255
RMA Test History : FF
RMA History : FF
Chassis Serial Number : JMX0836L10L
Chassis MAC Address : 0011.218d.dd30
MAC Address block size : 48
Field Diagnostics Data : FF FF FF FF FF FF FF FF
Hardware Revision : 0.1
Number of Slots : 2
Product (FRU) Number : C3725-2FE
EEPROM format version 4
EEPROM contents (hex):
0x00: 04 FF C1 8B 4A 41 45 30 38 32 31 4A 41 39 53 09
0x10: 61 40 02 59 C0 46 03 20 00 3F 13 02 42 44 30 85
WIC Slot 1:
Unknown WAN daughter card
WIC module not supported/disabled in this slot
Hardware Revision : 3.0
Part Number : 73-5797-03
Board Revision : A0
Deviation Number : 0-0
Fab Version : 02
PCB Serial Number : FOC07160WZ2
RMA Test History : 00
RMA Number : 0-0-0-0
RMA History : 00
Top Assy. Part Number : 800-09311-03
Connector Type : 01
Chassis MAC Address : 0004.dd0d.798c
MAC Address block size : 1
Product (FRU) Number : PA-A2-4T1C-T3ATM=
EEPROM format version 4
EEPROM contents (hex):
0x00: 04 FF 40 00 39 41 03 00 82 49 16 A5 03 42 41 30
0x10: 80 00 00 00 00 02 02 C1 8B 46 4F 43 30 37 31 36
WIC Slot 2:
T1 (2 port) Multi-Flex Trunk (Drop&Insert) WAN daughter card
Hardware revision 1.0 Board revision B0
Serial number 22688207 Part number 800-04614-02
FRU Part Number VWIC-2MFT-T1-DI=
Test history 0x0 RMA number 00-00-00
Connector type PCI
EEPROM format version 1
EEPROM contents (hex):
0x20: 01 24 01 00 01 5A 31 CF 50 12 06 02 00 00 00 00
0x30: 58 00 00 00 01 01 18 00 FF FF FF FF FF FF FF FF
!--- Output truncated.
-
If the module's hardware information is not shown in the
show diag, make sure that the module is firmly seated in its
slot.
-
Install the module in a different slot or a different router. Some
modules are not hot swappable; reload the router after you install the
module.
-
If the module's hardware information is shown in the show
diag but, you are unable to see the module interfaces in the
show run or show IP interface brief output, check the
hardware installation documentation. Some modules need router configuration
after the installation.
-
In the show diag output, note the error message
under the WIC Slot 1. It is displayed as "Unknown WAN
daughter card" "WIC module not supported/disabled in this
slot." This is due to an incompatible hardware issue. Check the
supported IOS version for that module and check the device hardware
installation documentation. Contact the
SMB
TAC for further assistance.
-
When you reboot the router, Cisco recommends that you wait for at
least 20 seconds before you turn on the device.
Troubleshoot Device Ethernet Connectivity
Follow these steps to troubleshoot the device ethernet
connectivity:
Verify the Ethernet State on the Cisco Device
You can use the command-line interface (CLI) to verify the state of
your device Ethernet interface and the current configuration. To verify the
interface configuration on the Cisco device, follow these
steps:
-
Verify that the LINK LED on the Cisco device is lit. If the LINK
LED is not lit, verify that the cable between the PC and the Cisco device is
connected properly.
-
Ensure that you do not use a faulty cable. Use a cable which has
worked in a different device. Ensure that the LINK LED is illuminated before
you proceed to the next section.
-
Type show interfaces or show interface
FastEthernet0 on the device command line interface and press
Enter to show the current status of your Ethernet
interface(s).
Note: The name of your Ethernet interface varies across Cisco devices.
Review this table to determine the name of your first Ethernet interface. For
example, if your Ethernet interface is named FastEthernet0/0, you need to enter
the command show interface FastEthernet0/0. If you have a
device with multiple Ethernet interfaces such as a switch, you can enter a
higher number as the interface number. For example, the last Ethernet port on a
24 port switch is named FastEthernet0/24.
Cisco Device Model
|
First Ethernet Interface
|
Catalyst Switch
|
FastEthernet0/1
|
Aironet 1200 Access Point
|
FastEthernet0
Note: The Aironet BVI1 interface must also be enabled for you
to connect to the Ethernet port.
|
SOHO Router
|
When you attempt to access PDM, the message "the page
cannot be displayed" appears in Internet Explorer or the message "network
connection was refused by the server" appears in Netscape
Communicator.
|
800 Series Router
|
Ethernet0
|
1760 Router
|
FastEthernet0/0
|
1700 Series Router
|
FastEthernet0
|
1800 Router
|
FastEthernet0/0
|
2600 Series Router
|
FastEthernet0/0
|
2800 Router
|
FastEthernet0/0
|
Cisco PIX
|
Ethernet1 (inside interface)
|
The output shows the state of the Ethernet interface and the IP
address assigned to the interface.
Cisco#show interface FastEthernet0
FastEthernet0 is up, line protocol is up
Hardware is PQUICC_FEC, address is 000f.23f5.5d64 (bia 000f.23f5.5d64)
Description: $ETH-LAN$$ETH-SW-LAUNCH$
Internet address is 192.168.10.1/24
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
!--- Output truncated.
-
Compare the first line of output from the Cisco device to the items
in this table. The table gives you instructions on how to troubleshoot each
interface state.
Interface State
|
Action
|
Catalyst Switch
|
The status "FastEthernet0 is up" indicates that the
interface is properly activated/enabled. The status "line protocol is up"
indicates that the interface is properly connected to your PC. If you are
unable to connect to the Cisco device, verify that the IP addresses are in the
same range. For more information about how to assign an IP address, refer to
Configure
an IP Address on Your PC.
|
FastEthernet0 is up, line protocol is up
|
The status "FastEthernet0 is administratively down"
indicates that the interface is turned off in the Cisco device configuration.
To bring the interface up, see
Configure
the Ethernet Interface on the Cisco Device.
|
FastEthernet0 is administratively down, line protocol is
down
|
The status "FastEthernet0 is up" indicates that the
interface is properly activated/enabled but the interface is unable to
communicate with the PC. Check the cable between the Cisco device and the PC to
ensure that, you use a crossover cable, and verify the status of the PC. For
more information on how to identify a cable, refer to
Cable
Descriptions.
|
Configure the Ethernet Interface on the Cisco Device
If your Cisco device Ethernet interface does not have an IP address or
is shut down, you need to configure your interface. To configure the Cisco
device Ethernet interface, follow these steps:
-
At the Cisco# prompt, type configure terminal and
press Enter.
Cisco#configure terminal
Enter the configuration commands, one per line. End with CNTL/Z.
Cisco(config)#
-
Type interface fastethernet0 and press
Enter.
Note: The name of your Ethernet interface varies across Cisco
devices.
Cisco(config)#interface fastethernet0
Cisco(config-if)#
-
Type no shutdown and press
Enter.
Cisco(config-if)#no shutdown
Note: If you have a PIX Security Appliance, type interface
ethernet1 100full and press Enter.
-
Ensure that there is no Duplex/Speed mismatch at the both end of
the ports. Configure the ports duplex and speed settings to auto-negotiate mode
in the interface configuration mode.
Router(config-if)#duplex auto
Router(config-if)#speed auto
-
Type ip address ip-address subnet-mask to assign
an IP address to the Cisco device. If you need to determine the correct IP
address for the Cisco device, refer to the
Site
Survey.
-
Type end and press Enter.
Cisco(config-if)#end
Cisco#
Note: If you have a PIX Security Appliance, type quit
and press Enter.
-
Type write memory and press Enter
to save your configuration.
Cisco#write memory
-
See Verify the Ethernet State on the Cisco
Device to confirm whether your Ethernet interface is now
active.
-
If the interfaces or ports still have problems, capture
Show Tech and Show Syslog output from the
device and contact the
SMB
TAC for further assistance.
Dead On Arrival (DOA) Devices
DOA replacements are provided by the partner or reseller from where you
purchased the equipment. Please contact your partner or reseller to get a DOA
replacement.
If you have further questions regarding this, please contact our
Service Relations team via the local toll free number for your country listed
in this link:
http://www.cisco.com/web/siteassets/contacts/index.html
Useful information to note prior to requesting a Device RMA
You can note down and send these information prior to contact
SMB
TAC for Device troubleshooting or RMA.
-
When did the problem start?
-
Did the device work before?
-
Was any hardware/software change performed on the device prior to
encounter the problem?
-
What initial steps have you done so far to troubleshoot this
device?
-
Do you receive any error?
-
Do you get the output of a Show Tech, Show Post and Show Syslog of
the device though you do not receive the error always?
-
Is the device in production or in the test
area?
-
How many users are affected?
-
What is the potential impact if this issue is not
resolved?
Customer Details
Customer can give these information, when the customer creates the case
for RMA, so that
SMB
TAC can process the RMA much faster.
(On-Site) Contact Name:
(On-Site) Contact Phone Number:
Contract Number(s):
Company Name:
Delivery Address:
Serial Numbers:
Part model Number(s) to be replaced:
Shipping address:
Name and contact information of the person that will receive the units:
TAC Service Request Tool
Please note that the TAC Service Request Tool allows you to do more
than open your service requests online. You are also able to:
-
Monitor the status of your service request
-
Upload additional files or notes to your open service
request
-
Upload configuration information or documentation in regard to your
case
-
Download software updates
-
Use online troubleshooting and configuration
tools
The TAC Service Request Tool is available around the clock and is
accessible from the TAC Web Site at
http://tools.cisco.com/ServiceRequestTool/query/
Troubleshooting High CPU Problems in Routers
Follow these steps to troubleshoot high CPU problems on your
Routers:
Symptoms of High CPU Utilization
This list describes common symptoms of high CPU utilization. If you
notice any of these symptoms, follow the troubleshooting steps in this document
to alleviate the problem.
-
High percentages in the show processes cpu command
output.
-
Slow performance
-
Services on the router fail to respond, for instance:
-
Slow response in Telnet or unable to Telnet to the
router
-
Slow response on the console
-
Slow or no response to ping
-
Router does not send routing updates to other
routers
Determining Causes and Solving the CPU Problem
Once you notice any of the above symptoms follow these steps to
troubleshoot the CPU problem:
-
Check for a possible security issue. Commonly, high CPU utilization
is caused by a security issue, such as a worm or virus operating in your
network. This is especially likely to be the cause if there have not been
recent changes to the network. Usually, a configuration change, such as adding
additional lines to your access lists can mitigate the effects of this problem.
Cisco Product Security Advisories and Notices contains information on detection
of the most likely causes and specific workarounds.
For additional information, refer to:
100
Questions and Answers about Internet Threats
Cisco Product Security
Advisories and Notices
Cisco
Threat Defense System
-
Make sure all debugging commands in your router are turned off. In
the privilege mode of the router type, undebug all or
no debug all command and press Enter.
Router#undebug all
-
Are you able to issue show commands on the router? If yes, start
collecting more information immediately, with these commands and capture the
output for further analysis.
Note: You can capture the above show outputs and send it to
SMB
TAC for further assistance.
-
Is the router inaccessible? Can you reproduce this problem? If yes,
power-cycle the router and, before reproducing the problem, configure the
scheduler interval time on your Cisco router. In the global configuration mode
of the router type, scheduler interval 500 command and press
Enter.
Router#config terminal
Router(config)#scheduler interval 500
Note: This schedules low priority processes to run every 500
milliseconds, which provides time for you to run some commands, even if CPU
usage is at 100 percent.
-
Does the router show symptoms of high CPU utilization at brief and
unpredictable intervals? If yes, periodically collect the output of the
show processes cpu command, which shows if the high CPU
utilization is caused by interrupts or by a certain process. Use this UNIX
script and, based on the first findings, modify the script to collect data
needed for further investigation of the issue.
Use the show processes cpu command to check if CPU
utilization is high due to interrupts or processes. In the privilege mode of
the router type, show process cpu command and press
Enter.
Router_2610XM#show process cpu
CPU utilization for five seconds: 65%/0%; one minute: 56%; five minutes: 57%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
1 4 2 2000 0.00% 0.00% 0.00% 0 Chunk Manager
2 89228 1904011 46 0.00% 0.00% 0.00% 0 Load Meter
3 156532306 204807909 764 15.43% 12.06% 13.04% 62 Exec
4 6593599 1127012 5850 0.00% 0.07% 0.05% 0 Check heaps
5 12 17 705 0.00% 0.00% 0.00% 0 Pool Manager
6 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
7 0 2 0 0.00% 0.00% 0.00% 0 Timers
8 13450 317310 42 0.00% 0.00% 0.00% 0 Environmental mo
9 559447 1135037 492 0.00% 0.00% 0.00% 0 ARP Input
10 0 2 0 0.00% 0.00% 0.00% 0 ATM Idle Timer
11 156279 951813 164 0.00% 0.00% 0.00% 0 Net Background
12 8 37 216 0.00% 0.00% 0.00% 0 Logger
13 698374 9493627 73 0.00% 0.01% 0.00% 0 TTY Background
---Output Truncated for brevity---
This is an example of the header of the show processes cpu command:
CPU utilization for five seconds: X%/Y%; one minute: Z%; five minutes: W%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
This table describes the fields in the header:
Feild
|
Description
|
X
|
Average total utilization during last five seconds
(interrupts + processes)
|
Y
|
Average utilization due to interrupts, during last five
seconds1
|
Z
|
Average total utilization during last minute2
|
W
|
Average total utilization during last five
minutes2
|
PID
|
Process ID
|
Runtime
|
CPU time the process has used (in milliseconds)
|
Invoked
|
Number of times a process has been called
|
uSecs
|
Microseconds of CPU time for each invocation
|
5Sec
|
CPU utilization by task in the last five
seconds
|
1Min
|
CPU utilization by task in the last minute2
|
5Min
|
CPU utilization by task in the last five
minutes2
|
TTY
|
Terminal that controls the process
|
Process
|
Name of process
|
1CPU utilization on process level = X - Y
2Values do not represent an arithmetical average, but an
exponentially decayed average. Thus, more recent values have more influence on
the calculated average.
Note: : Total CPU utilization must not be used as a measure of the
ability of the router to switch more packets.
-
High CPU Utilization due to Processes:
Check which process is loading the CPU. Unusual activity related to
a process results in an error message in the log. Therefore, the output of the
show logging exec command must be checked first for any errors
related to the process which consumes lots of CPU cycles. In the privilege mode
of the router type, show logging command and press
Enter.
Router_2610XM#show logging
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes,
0 overruns, xml disabled)
Console logging: level debugging, 22 messages logged, xml disabled
Monitor logging: level debugging, 0 messages logged, xml disabled
Buffer logging: disabled, xml disabled
Logging Exception size (4096 bytes)
Count and timestamp logging messages: disabled
Trap logging: level informational, 26 message lines logged
Debugging can also be of great help in troubleshooting high CPU utilization in
processes. Nevertheless, debugging must be carried out with extreme caution
because it may raise the CPU utilization even more. These prerequisites must be
met in order to make debugging safe and useful:
-
All logging destinations except buffer logging must be either
disabled or their logging severity level should be lowered from 7 (debugging)
to 6 (informational) or less, using the global configuration command
logging trap informational. In the global configuration mode
of the router type, logging trap informational command and
press Enter.
Router#config terminal
Router(config)#logging trap informational
Note: To see which logging destinations and corresponding levels are
enabled, read the header lines of the show logging exec
command.
Back to Top
Next Step
You have completed basic steps to troubleshoot your Cisco
device.
To configure other devices in your network, refer to the
Configuration
Overview Page.
Back to Top
Related Information
|
|
