Table Of Contents
Troubleshooting the Installation
Troubleshooting Overview
Troubleshooting Using a Subsystem Approach
Normal Router Startup Sequence
Identifying Startup Issues
Troubleshooting the Power Subsystem
Troubleshooting the AC-Input Power Subsystem
Troubleshooting the DC-Input Power Subsystem
Troubleshooting a DC Power Module
Additional Power Subsystem Troubleshooting Information
Hardware and Software Identification
Obtaining Temperature and Environmental Information
Troubleshooting the Power Distribution System
Troubleshooting the Route Processor Subsystem
Route Processor Overview
RSP and RP Front Panel Indicators
Compact Flash and Status LEDs
Ethernet Ports and Status LEDs
Auxiliary and Console Ports
Alphanumeric Message Displays
Flash Memory
Troubleshooting RSP and RP Cards
FC Card Front Panel Indicator
Troubleshooting Line Cards and Modular Port Adapters
Monitoring Critical, Major, and Minor Alarm Status
Troubleshooting the Cooling Subsystem
Fan Tray Operation
Power Module Fans
Overtemperature Conditions
Isolating Cooling Subsystem Problems
Troubleshooting the Installation
This chapter contains general troubleshooting information to help isolate the cause of any difficulties you might encounter during the installation and initial startup of the system.
Although an overtemperature condition is unlikely at initial startup, environmental monitoring functions are included in this chapter because they also monitor internal voltages.
Troubleshooting the installation is presented in the following sections:
•
Troubleshooting Overview
•Troubleshooting the Power Subsystem
•Troubleshooting the Route Processor Subsystem
•Troubleshooting the Cooling Subsystem
Troubleshooting Overview
This section describes the methods used in troubleshooting the router. The troubleshooting methods are organized according to the major subsystems in the router.
If you are unable to solve a problem on your own, you can contact a Cisco customer service representative for assistance. Cisco customer service and technical support can be reached at http://www.cisco.com/en/US/support/tsd_cisco_worldwide_contacts.html
When you call, please have the following information ready:
•Date you received the router and the chassis serial number (located on a label on the back of the chassis).
•Installed line cards and Cisco software release number:
–Use the show version command to determine which line cards are installed and the Cisco software release number, if possible.
•Brief description of the symptoms and steps you have taken to isolate and solve the issue.
•Maintenance agreement or warranty information.
Troubleshooting Using a Subsystem Approach
To solve a system problem, try to isolate the problem to a specific subsystem. Compare current router behavior with expected router behavior. Because a startup issue is usually attributable to one component, it is most efficient to examine each subsystem, rather than trying to troubleshoot each router component.
For troubleshooting purposes in this chapter, the router consists of the following subsystems:
•Power subsystem—Router chassis is shipped with up to six AC-input or DC-input power supply modules installed in the Cisco ASR 9010 Router chassis, up to three AC-input or DC-input power supply modules installed in the Cisco ASR 9006 Router chassis, or up to 16 AC-input or DC-input power supply modules installed in the Cisco ASR 9922 Router chassis.
•Chassis backplane power distribution—System transfers -54 VDC power from the power modules to the chassis backplane and distributes it to all the cards through the backplane connectors.
•Processor subsystem in the Cisco ASR 9010 Router and Cisco ASR 9006 Router—Includes the active Route Switch Processor (RSP) card (and optional, redundant RSP card, if installed) and up to eight line cards (in the Cisco ASR 9010 Router) or four line cards (in the Cisco ASR 9006 Router). The RSP and line cards are equipped with onboard processors. The RSP downloads a copy of the Cisco software image to each line card processor. The system uses an alphanumeric display on the active RSP to display status and error messages, which can help in troubleshooting.
Note There are two types of image files, -P PIE files, and x86-based -PX PIE files. The -P PIE files are for use on Cisco ASR 9000 Series Aggregation Services Routers with RSP route switch processors (RSP-4G and RSP-8G) while the -PX PIE files are for use on Cisco ASR 9000 Series Aggregation Services Routers with RSP-440 route switch processors and the ASR 9922 route processors.
•Processor subsystem in the Cisco ASR 9922 Router—Includes the active Route Processor (RP) card, standby RP card, and up to 20 line cards. The RP and line cards are equipped with onboard processors. The RP downloads a copy of the Cisco software image to each line card processor. The system uses an alphanumeric display on the active RP to display status and error messages, which can help in troubleshooting.
•Cooling subsystem in the Cisco ASR 9010 Router and Cisco ASR 9006 Router—Consists of one or two fan trays, which circulate air through the card cage to cool the cards, and two fans in each of the power modules, which circulate cooling air through the power module.
•Cooling subsystem in the Cisco ASR 9922 Router—Consists of four fan trays, which circulate air through the top and bottom line card cages to cool the cards, and two fans in each of the power modules, which circulate cooling air through the power module.
Normal Router Startup Sequence
You can generally determine when and where the router failed during the startup sequence by checking the status LEDs on the power modules, and the alphanumeric displays on the RSP, RP, and line cards.
In a normal router startup sequence, the following sequence of events and conditions occur:
1. The fan in each power module receives power and begins drawing air through the power supply.
The power module input power and output power indicators are on.
2. The fans in the fan tray receive power and begin drawing air through the chassis.
The fan tray OK indicator is on.
3. As the power-on and boot process progresses for the RSP/RP, the status of the RSP/RP appears on the alphanumeric display on the front panel of the card.
Identifying Startup Issues
Table 4-1 shows the contents of the alphanumeric displays on the RSP/RP card, as well as the normal LED states on the power modules (AC or DC) and the fan tray after a successful system startup.
Note For the RSP/RP card to communicate properly to a power module in a power tray, the appropriate input power should be present.
Table 4-1 Alphanumeric Displays and LEDs at System Startup
Component
|
Type of Indicator
|
Display Contents/LED Status and Meaning
|
RSP card
|
Alphanumeric display
|
INIT—Card is inserted and microcontroller is initialized
BOOT—Board is powered on and CPU is booting
IMEM—Start initializing memory
IGEN—Start initializing the board
ICBC—Start initializing communication with the microcontroller
PDxy—Loading programmable devices
(x = FPGA, y = ROMMON)
PSTx—Power on self test x
RMN—All tests are finished and ROMMON is ready for commands
LOAD—Downloading Minimum Boot Image (MBI) image to CPU
MBI—Starting execution of MBI
IOXR—Cisco IOS XR software is starting execution
ACTV—RSP role is determined to be active RSP
STBY—RSP role is determined to be standby RSP
PREP—Preparing disk boot
|
Line Cards
|
Status LED
|
Green: The line card is enabled and ready for use.
|
RSP-440 card
|
Alphanumeric display
|
INIT—Card is inserted and microcontroller is initialized
BOOT—Board is powered on and CPU is booting
IMEM—Start initializing memory
IGEN—Start initializing the board
ICBC—Start initializing communication with the microcontroller
SCPI—Board is not plugged in properly
STID—CBC was unable to read slot ID pins correctly
PSEQ—CBC detected power sequencer failure
DBPO—CBC detected an issue during board power up
KPWR—CBC detected an issue during board power up
LGNP—CBC detected an issue during board power up
LGNI—CBC detected an issue during board power up
RMN—All tests are finished and ROMMON is ready for commands
LOAD—Downloading MBI image to CPU
RRST—ROMMON is performing a soft reset after 5 consecutive MBI validation requests timed out
MVB—ROMMON trying MBI validation boot
MBI—Starting execution of MBI
IOXR—Cisco IOS XR software is starting execution
LDG—The RSP is loading (MBI started and card preparing for activity)
INCP—The software or configuration is incompatible with the RSP
OOSM—The RSP is in Out of Service, Maintenance mode
ACTV—RSP role is determined to be active RSP
STBY—RSP role is determined to be standby RSP
PREP—Preparing disk boot
|
RP card
|
Alphanumeric display
|
INIT—Card is inserted and microcontroller is initialized
BOOT—Board is powered on and CPU is booting
IMEM—Start initializing memory
IGEN—Start initializing the board
ICBC—Start initializing communication with the microcontroller
SCPI—Board is not plugged in properly
STID—CBC was unable to read slot ID pins correctly
PSEQ—CBC detected power sequencer failure
DBPO—CBC detected an issue during board power up
KPWR—CBC detected an issue during board power up
LGNP—CBC detected an issue during board power up
LGNI—CBC detected an issue during board power up
RMN—All tests are finished and ROMMON is ready for commands
LOAD—Downloading MBI image to CPU
RRST—ROMMON is performing a soft reset after 5 consecutive MBI validation requests timed out
MVB—ROMMON trying MBI validation boot
MBI—Starting execution of MBI
IOXR—Cisco IOS XR software is starting execution
LDG—The RP is loading (MBI started and card preparing for activity)
INCP—The software or configuration is incompatible with the RP
OOSM—The RP is in Out of Service, Maintenance mode
ACTV—RP role is determined to be active RP
STBY—RP role is determined to be standby RP
PREP—Preparing disk boot
|
AC Power Modules
|
Power status LEDs
|
Input power indicator on (green): Input AC power OK.
Output power indicator on (green): Output DC power OK.
Fault LED off (red): No fault is present.
The correct power module voltages are present and no faults have been detected.
|
DC Power Modules
|
Power status LEDs
|
Input power indicator on (green): Input DC power OK.
On the DC power tray, the Power Input LED is lit solid green if both DC feeds are valid and blinks green if only a single DC feed is valid.
Output power indicator on (green): Output DC power OK.
Fault LED off (red): No fault is present.
The correct power module voltages are present and no faults have been detected.
|
Fan Trays
|
Fan tray status LED
|
Green LED on: Fan Tray OK.
The fan tray fans are operating correctly.
|
Troubleshooting the Power Subsystem
This section contains information to troubleshoot the power subsystems:
•Troubleshooting the AC-Input Power Subsystem
•Troubleshooting the DC-Input Power Subsystem
•Troubleshooting the Power Distribution System
Note For the RSP/RP card to communicate properly to a power module in a power tray, input power to at least one of the three power modules in the power tray should be present.
Troubleshooting the AC-Input Power Subsystem
AC-input power modules are monitored for internal temperature, voltage, and current load by the RSP/RP. If the router detects an extreme condition, it generates an alarm and logs the appropriate warning messages on the console.
Figure 4-1 shows the status indicators for the version 1 power module and Figure 4-2 shows the status indicators for the version 2 power module. The indicator definitions follow the two figures.
Figure 4-1 Version 1 Power Module Status Indicators
1
|
Input power LED
|
ON continuously when the input voltage is present and within the correct range
BLINKING when the input voltage is out of acceptable range
On the DC power tray, the Power Input LED is lit solid green if both DC feeds are valid and blinks green if only a single DC feed is valid.
OFF when no input voltage is present
|
2
|
Output power LED
|
ON when the power module output voltage is present
BLINKING when the power module is in a power limit or overcurrent condition
|
3
|
Fault LED
|
ON to indicate that a power module failure has occurred
|
Figure 4-2 Version 2 Power Module Status Indicators
1
|
Input power LED
|
ON continuously when the input voltage is present and within the correct range
BLINKING when the input voltage is out of acceptable range
On the DC power tray, the Power Input LED is lit solid green if both DC feeds are valid and blinks green if only a single DC feed is valid.
OFF when no input voltage is present
|
2
|
Output power LED
|
ON when the power module output voltage is present
BLINKING when the power module is in a power limit or overcurrent condition
|
3
|
Fault LED
|
ON to indicate that a power module failure has occurred
|
Follow these steps to troubleshoot the AC power module if it is not operating properly:
Step 1 Make sure the power module is seated properly by ejecting and reseating the power module. Check that:
•Latch on the door/ejector lever is locked securely.
•Power switch on the power tray is set to the ON (1) position.
Step 2 Make sure the router is powered on and that all power cords are connected properly. Check that the:
•Power cords plugged into the power tray receptacles are secured in place with their retention clips.
•Power cords at the power source end are securely plugged into their own AC power outlets.
•Source AC circuit breaker is switched on.
Step 3 Check the power supply status LED indicators:
•Input power LED (green)—Indicates that the AC power input is operating normally, and the source AC input voltage of 200 to 240 VAC is within the nominal operating range.
If the input power LED is blinking, the input voltage is out of acceptable range. Verify that each AC power source is operating in the nominal range of 200 to 240 VAC.
Note On the DC power tray, the Power Input LED is lit solid green if both DC feeds are valid and blinks green if only a single DC feed is valid.
•Output power LED (green)—Indicates that the DC power output is operating normally and the -54 VDC output voltage to the backplane are within the nominal operating range. This indicator lights only when the power switch at the rear of the power tray is set to the ON (1) position.
–If the Output power LED remains off after checking all the power sources, replace the power supply with a spare. If the spare power module does not work, troubleshoot the power tray in which the module is plugged.
–If the output power LED is blinking, the power module is in a power limit or overcurrent condition. Make sure that each power cord is connected to a dedicated AC power source. Verify that each AC power source is operating in the nominal range of 200 to 240 VAC and is supplying a minimum service of 20 A, North America (or 13 A, international).
•Fault LED (red)—Indicates that the system has detected a fault within the power supply. This indicator remains off during normal operation. If the fault LED is on:
–If your system has more than one power tray (Cisco ASR 9010 Router and Cisco ASR 9922 Router) with power modules installed for redundancy, you can toggle the power switch at the rear of the first power tray off and then on. If the fault LED remains on after several attempts to power it on, replace the power module with a spare.
–If the spare power module also fails, the problem could be a faulty power tray backplane connector. Power off the router and contact a Cisco service representative for assistance.
–Verify that the power module fans are operating properly.
–Verify that the fan tray is operating properly.
If the power module fans and the fan trays are operating properly, replace the existing power module with a spare.
Caution Because the AC-input power subsystems use redundant power modules, a problem with the DC output voltage to the backplane from only one power module should not affect router operation. When the router is equipped with two AC power supplies, it powers on and operates even if one power supply fails. However, complete router functionality may be affected depending on the system load.
Troubleshooting the DC-Input Power Subsystem
DC-input power supplies are monitored for internal temperature, voltage, and current load by the RSP/RP. If the router detects an extreme condition, it generates an alarm and logs the appropriate warning messages on the console.
Figure 4-1 shows the status indicators for the version 1 power module and Figure 4-2 shows the status indicators for the version 2 power module. The indicator definitions follow the two figures.
Troubleshooting a DC Power Module
Follow these steps to troubleshoot a DC power module if it is not operating properly.
Step 1 Make sure the power module is seated properly by ejecting and reseating the power module. Check that:
•Latch on the door/ejector lever is locked securely.
•Power switch on the power tray is set to the ON (1) position.
Step 2 Make sure the router is powered on and that all power cords are connected properly. Check that the:
•Power cables are securely attached to their power module terminal studs.
•Power cables are securely attached at the DC source end.
•Source DC circuit breaker is switched on.
Step 3 Check the power supply status LED indicators:
•Input power LED (green)—Indicates that the DC power input is operating normally, and the source DC input voltage is within the nominal operating range of -40 to -72 VDC.
–If the input power LED is blinking, input connections to the power module are loose or not connected, or the input voltage is below the minimum. Verify that DC power from the source is operating in the nominal range of -40 to -72 VDC.
Note On the DC power tray, the Power Input LED is lit solid green if both DC feeds are valid and blinks green if only a single DC feed is valid. Check the input connections to the power module.
–If the indicator is still flashing after you perform the above checks, replace the power module.
•Output power LED (green)—Indicates that the DC power output is operating normally, and the -54 VDC output voltage to the backplane is within the nominal operating range. This indicator lights only when the power switch at the rear of the power tray is set to the ON (1) position.
–If the output power LED remains off after checking all the power sources, replace the power module with a spare. If the spare power module does not work, troubleshoot the power tray in which the module is plugged.
–If the output power LED is blinking, the power module is in a power limit or overcurrent condition. Make sure that each power cable is connected to a dedicated DC power source. Verify that each DC power source is operating in the nominal range of -40 to -72 VDC.
•Fault LED (red)—Indicates that the system has detected a fault within the power supply. This indicator remains off during normal operation. If the fault LED is on, check the following:
–If your system has more than one power tray (Cisco ASR 9010 Router and Cisco ASR 9922 Router) with power modules installed for redundancy, you can toggle the power switch at the rear of the first power tray off and then on. If the fault LED is still on, eject and reseat the power module. If the fault LED remains on after several attempts to power it on, replace the power module with a spare.
–If the spare power module also fails, the problem could be a faulty power tray backplane connector. Power off the router and contact a Cisco service representative for assistance.
–Verify that the power module fans are operating properly.
–Verify that the fan tray is operating properly.
–If the power module fans and the fan trays are operating properly, replace the faulty power module with a spare.
Caution Because there are redundant power modules, a problem with the DC output voltage to the backplane from only one power module should not affect router operation. When the router is equipped with two DC power supplies, it powers on even if one power supply fails. However, complete router functionality may be affected depending on the system load.
Additional Power Subsystem Troubleshooting Information
This section contains additional troubleshooting information to help you isolate the cause of a power problem.
Hardware and Software Identification
The power modules have software IDs that differ from the hardware ID labels on the chassis. Table 4-2 is a table for converting power module hardware IDs to software IDs.
Table 4-2 Power Module Hardware and Software IDs
Hardware ID
|
Software ID
|
PS0 M0
|
PM0
|
PS0 M1
|
PM1
|
PS0 M2
|
PM2
|
PS0 M3
|
PM3
|
PS1 M0
|
PM4
|
PS1 M1
|
PM5
|
PS1 M2
|
PM6
|
PS1 M3
|
PM7
|
PS2 M0
|
PM8
|
PS2 M1
|
PM9
|
PS2 M2
|
PM10
|
PS2 M3
|
PM11
|
PS3 M0
|
PM12
|
PS3 M1
|
PM13
|
PS3 M2
|
PM14
|
PS3 M3
|
PM15
|
Obtaining Temperature and Environmental Information
If both the RSP/RP and fan trays are operating, all internal correct DC voltages are present.
Enter the show environment command at the router admin prompt to display temperature and voltage information for each installed card, fan tray, and power module as shown in this example:
RP/0/RSP0/CPU0:router(admin)#show environment
---------------------------------------------
R/S/I Modules Inlet Hotspot
---------------------------------------------
R/S/I Modules Sensor (mV) Margin
host 1.2V_LDO_BRG0 1195 n/a
host 1.2V_LDO_BRG1 1196 n/a
host 1.1V(1.05V_CPU) 1051 n/a
host 0.75VB_0.75VC 754 n/a
host 1.0V_Bridge_LDO 995 n/a
host 0.75VD_and_0.75VE 755 n/a
host ZARLINK_3.3V 3284 n/a
host ZARLINK_1.8V 1810 n/a
host 5.0V_XFP_DB 5034 n/a
host 1.2V_LDO_BRG0 1191 n/a
host 1.2V_LDO_BRG1 1194 n/a
host 1.1V(1.05V_CPU) 1054 n/a
host 0.75VB_0.75VC 755 n/a
host 1.0V_Bridge_LDO 995 n/a
host 0.75VD_and_0.75VE 753 n/a
host ZARLINK_3.3V 3272 n/a
host ZARLINK_1.8V 1811 n/a
---------------------------------------------
---------------------------------------------
FAN0 FAN1 FAN2 FAN3 FAN4 FAN5
7080 7020 6990 7020 6960 6900
6900 6900 7110 6960 6900 7020
---------------------------------------------
R/S/I Modules Sensor Watts Status
Total Power Capacity: 3000W
Usable Power Capacity: 3000W
Supply Failure Protected Capacity: 0W
Worst Case Power Used: 1910W
0/RSP1/CPU0 250 (default)
Worst Case Power Available: 1090W
Supply Protected Capacity Available: Not Protected
Troubleshooting the Power Distribution System
The power distribution system consists of:
•AC or DC power modules, which supply -54 VDC to the backplane.
•Chassis backplane, which carries voltage to chassis components.
•DC-to-DC converters, which convert -54 VDC from the backplane to the correct voltages required by the line cards.
Follow these steps to troubleshoot the power distribution system:
Step 1 Check each power module to make sure that:
•Power module door is fully closed and properly secured by its latch.
•Green Input Power LED is on.
•Green Output Power LED is on.
•Red Fault LED is off.
If the power modules meet the above criteria, then the correct source power is present and within tolerance and output DC power is present. The power modules are functioning properly.
Step 2 Make sure the fan trays are operating:
•If the fan trays are functioning, then the -54 VDC from the chassis backplane and the cables from the backplane to the fan trays are functioning properly.
•If one or both fan trays are not functioning, there may be a problem with either the fan trays themselves, or the -54 VDC power supplied to the fan trays. Eject and reseat the fan trays.
•If a fan tray is still not operating, there could be a problem with the fan tray controller card or cable. Replace the fan tray.
•Contact your Cisco representative if replacing a fan tray or both fan trays does not fix the problem.
Troubleshooting the Route Processor Subsystem
The router processor subsystem consists of the route processor located on the RSP card. The RSP and the line cards each have the same onboard CPU serving as the main processor. The Controller Area Network (CAN) microcontroller processor monitors the environment and controls the onboard DC-to-DC converters.
Note A minimally configured router must have an RSP/RP installed in RSP slot 0 or RP slot 0 of the card cage to operate. If the router is equipped with a redundant RSP/RP, the redundant RSP/RP must be installed in RSP slot 1 or RP slot 1 of the card cage.
This section contains information to troubleshoot the route processor subsystem, including:
•Route Processor Overview
•RSP and RP Front Panel Indicators
•Troubleshooting RSP and RP Cards
•FC Card Front Panel Indicator
•Troubleshooting Line Cards and Modular Port Adapters
Route Processor Overview
The CPU on the RSP/RP card provides chassis control and management, boot media functionality, telecom timing and precision clock synchronization, communication to the line cards through the backplane Ethernet network, and power control through the CAN bus. In addition, the CPU on the RSP/RP card also runs the routing protocols.
Figure 4-3 identifies the ports and LEDs on the RSP card front panel.
Figure 4-4 identifies the ports and LEDs on the RSP-440 card front panel.
Figure 4-5 identifies the ports and LEDs on the RP card front panel.
Figure 4-3 RSP Card Front Panel
1
|
Management LAN ports
|
5
|
Compact Flash type I/II
|
2
|
Console and Auxiliary (AUX) ports
|
6
|
Alarm Cutoff (ACO) and Lamp Test push buttons
|
3
|
Sync (BITS and J.211) ports
|
7
|
Eight discrete LED indicators
|
4
|
Alarm Out DB9 Connector
|
8
|
LED matrix display
|
Figure 4-4 RSP-440 Card Front Panel
1
|
SYNC (BITS/J.211) ports
|
7
|
External USB port
|
2
|
SFP/SFP+ ports
|
8
|
Management LAN ports
|
3
|
Service LAN port
|
9
|
Console and Auxiliary (AUX) ports
|
4
|
ToD port
|
10
|
Alarm Cutoff (ACO) and Lamp Test push buttons
|
5
|
10 MHz and 1 PPS indicators
|
11
|
Eight discrete LED indicators
|
6
|
Alarm Out DB9 connector
|
12
|
LED matrix display
|
Figure 4-5 RP Card Front Panel
1
|
SYNC (BITS/J.211) ports
|
8
|
External USB port
|
2
|
SFP/SFP+ ports
|
9
|
Management LAN ports
|
3
|
IEEE 1588 port
|
10
|
CONSOLE and AUX ports
|
4
|
Inter-chassis nv Sync0
|
11
|
Alarm Cutoff (ACO) and Lamp Test push buttons
|
5
|
Inter-chassis nv Sync1 GPS ToD
|
12
|
Nine discrete LED indicators
|
6
|
10 MHz and 1 PPS indicators
|
13
|
LED matrix display
|
7
|
Alarm Out DB9 connector
|
|
|
RSP and RP Front Panel Indicators
The RSP card has eight discrete LED indicators and an LED dot-matrix display for system information. The RSP-440 adds three USB-specific LEDs. The RP has nine discrete LED indicators and an LED dot-matrix display for system information.
Table 4-3 lists the display definitions of the eight discrete LEDs on the RSP and RSP-440 front panels and the three RSP-440 specific USB LEDs.
Table 4-3 RSP and RSP-440 Discrete LED Display Definitions
Indicator (Label)
|
Color
|
Description
|
Power Fail (FAIL)
|
Red
|
Standby Power Fail LED. The LED is turned off by the CAN bus controller after it is up and running.
|
Off
|
Standby power is normal.
|
Critical Alarm (CRIT)
|
Red
|
Critical Alarm LED. A critical alarm has occurred.
|
Off
(Default after reset)
|
No critical alarm has occurred.
|
Major Alarm (MAJ)
|
Red
|
Major alarm LED. A major alarm has occurred.
|
Off
(Default after reset)
|
No major alarm has occurred.
|
Minor Alarm (MIN)
|
Amber
|
Minor alarm LED. A minor alarm has occurred.
|
Off
(Default after reset)
|
No minor alarm has occurred.
|
Synchronization (SYNC)
|
Green
|
System timing is synchronized to an external timing source.
|
Amber
|
System timing is free running.
|
Off
|
LED never turns off.
|
Internal Hard Disk Drive (HDD)
|
Green
|
Hard Disk Drive is busy/active. The LED is driven by the SAS controller.
|
Off
(Default after reset)
|
Hard Disk Drive is not busy/active
|
External Compact Flash (CF)
|
Green
|
Compact Flash is busy/active.
|
Off
(Default after reset)
|
Compact Flash is not busy/active.
|
Alarm Cutoff (ACO)
|
Amber
|
Alarm Cutoff has been enabled. The ACO push button was pressed after at least one alarm has occurred.
|
Off
(Default after reset)
|
Alarm Cutoff is not enabled.
|
External USB 2.0
[RSP-440]
|
Green
|
External USB is busy/active.
|
Off
(Default after reset)
|
External USB is not busy/active.
|
Internal USB 2.0 A
[RSP-440]
|
Green
|
Internal USB is busy/active.
|
Off
(Default after reset)
|
Internal USB is not busy/active.
|
Internal USB 2.0 B
[RSP-440]
|
Green
|
Internal USB is busy/active.
|
Off
(Default after reset)
|
Internal USB is not busy/active.
|
Table 4-4 lists the display definitions of the nine discrete LEDs on the RP front panel.
Table 4-4 RP Discrete LED Display Definitions
Indicator (Label)
|
Color
|
Description
|
Power Fail (FAIL)
|
Red
(Default after power on)
|
Standby Power Fail LED. The LED is turned off by the CAN bus controller after it is up and running.
|
Off
|
Standby power is normal.
|
Critical Alarm (CRIT)
|
Red
|
Critical Alarm LED. A critical alarm has occurred.
|
Off
(Default after reset)
|
No critical alarm has occurred.
|
Major Alarm (MAJ)
|
Red
|
Major alarm LED. A major alarm has occurred.
|
Off
(Default after reset)
|
No major alarm has occurred.
|
Minor Alarm (MIN)
|
Amber
|
Minor alarm LED. A minor alarm has occurred.
|
Off
(Default after reset)
|
No minor alarm has occurred.
|
Alarm Cutoff (ACO)
|
Amber
|
Alarm Cutoff has been enabled. The ACO push button was pressed after at least one alarm has occurred.
|
Off
(Default after reset)
|
Alarm Cutoff is not enabled.
|
Synchronization (SYNC)
|
Green
|
System timing is synchronized to an external timing source including
IEEE 1588.
|
Amber
|
System timing is free running.
|
Off
(Default after reset)
|
LED never turns off.
|
Internal Solid State Hard Disk Drive (SSD)
|
Green
|
Internal Solid State Hard Disk Drive (SSD0) is busy/active. The LED is driven by the SSD controller.
|
Off
(Default after reset)
|
Internal Solid State Hard Disk Drive is not busy/active.
|
FC Fault
|
Amber
|
A fault has occurred on any or all of the FC cards installed. This LED will be on during the boot phase of the FC.
|
Off
(Default after reset)
|
FC cards are booted up and ready.
|
GPS
|
Green
|
GPS interface provisioned and ports are turned on. ToD, 1 PPS, 10 Mhz are all valid.
|
Off
(Default after reset)
|
Either the interface is not provisioned, or the ports are not turned on. ToD, 1 PPS, 10 Mhz are not valid.
|
Compact Flash and Status LEDs
One compact flash slot provides the RSP with additional flash memory capacity. All combinations of different flash devices are supported by the RSP. You can use ATA flash disks, Type 1 or Type 2 linear flash memory cards, or a combination of the two.
Note The RSP only supports +5.2 VDC flash memory devices. It does not support +3.3 VDC PCMCIA devices.
Note The RSP-440 and RP do not have a compact flash slot.
The slot has an eject button (located behind the cover) to remove a flash card from the slot.
Ethernet Ports and Status LEDs
The RSP/RP has two 8-pin media-dependent interface (MDI) RJ45 Management LAN ports for 100 Mbps and 1000Mbps Ethernet connections. These ports are labeled MGT LAN 0 and MGT LAN 1.
The transmission speed of the Ethernet port is not user-configurable. You set the speed through an autosensing scheme on the RSP/RP, which determines the speed by the network that the Ethernet port is connected to. However, even at an autosensed data transmission rate of 100 Mbps, the Ethernet port can only provide a usable bandwidth of substantially less than 100 Mbps. You can expect a maximum usable bandwidth of approximately 12 Mbps when using an Ethernet connection.
The following LEDs on the front panel indicate traffic status and port selection (see Figure 4-6):
•LINK—Indicates link activity.
•ACT—Indicates which Ethernet port is selected (ETH 0 or ETH 1).
Note Because both ports are supported on the RSP/RP card, MGT LAN 0 is always on. MGT LAN 0 lights when it is selected.
Figure 4-6 Management LAN Port Activity LEDs
Auxiliary and Console Ports
The auxiliary and console ports on the RSP/RP are EIA/TIA-232 (also known as RS-232) asynchronous serial ports connect external devices to monitor and manage the system:
•Auxiliary port—RJ45 interface that supports flow control and is often used to connect a modem, a channel service unit (CSU), or other optional equipment for Telnet management.
•Console port—Receptacle (female) that provides a RJ45 interface for connecting a console terminal.
Alphanumeric Message Displays
The alphanumeric message displays are organized in one row of four LED characters (see callout 8 in Figure 4-3 for the RSP, callout 12 in Figure 4-4 for the RSP-440, and callout 13 in Figure 4-5 for the RP).
The alphanumeric message displays show router status messages during the boot process and after the boot process is complete:
•During the boot process, the message displays are controlled directly by the CAN microcontroller.
•After the boot process, the message displays are controlled by Cisco IOS XR software (through the CAN bus).
The alphanumeric message displays also provide information about different levels of system operation, including the status of the RSP/RP, router error messages, and user-defined status and error messages.
Note A list of all system and error messages appears in the Cisco IOS XR System Error Messages publication.
Flash Memory
You can use the flash memory on the RSP card to store multiple Cisco IOS XR software and microcode images that you can use to operate the router. You can download new images to flash memory over the network (or from a local server) to replace an existing image or to add it as an additional image. The router can be booted (manually or automatically) from any of the stored images in flash memory.
Flash memory also functions as a Trivial File Transfer Protocol (TFTP) server to allow other servers to boot remotely from the stored images, or to copy them into their own flash memory.
The system uses two types of flash memory:
•Onboard flash memory (bootflash)—Contains the Cisco IOS XR boot image
•Compact flash memory disks (or cards)—Contains the Cisco IOS XR software images
Note The RSP-440 and RP do not have a compact flash slot.
Table 4-5 lists supported compact flash disk sizes and Cisco part numbers.
Table 4-5 Supported Compact Flash Disk Sizes
Compact Flash Disk Size
|
Part Number
|
1 GB
|
16-3204-01
|
Troubleshooting RSP and RP Cards
When the router is powered on, the alphanumeric display on the RSP indicates the following sequence:
INIT—Card is inserted and microcontroller is initialized.
BOOT—Board is powered on and CPU is booting.
IMEM—Start initializing memory.
IGEN—Start initializing the board.
ICBC—Start initializing communication with the microcontroller.
PDxy—Loading programmable devices (x = FPGA, y = ROMMON).
PSTx—Power on self test x.
RMN—All tests are finished and ROMMON is ready for commands.
LOAD—Downloading Minimum Boot Image (MBI) image to CPU.
MBI—Starting execution of MBI.
IOXR—Cisco IOS XR software is starting execution.
ACTV—RSP role is determined to be active RSP.
STBY—RSP role is determined to be standby RSP.
PREP—Preparing disk boot.
When the router is powered on, the alphanumeric display on the RSP-440 and RP indicates the following sequence:
INIT—Card is inserted and microcontroller is initialized
BOOT—Board is powered on and CPU is booting
IMEM—Start initializing memory
IGEN—Start initializing the board
ICBC—Start initializing communication with the microcontroller
SCPI—Board is not plugged in properly
STID—CBC was unable to read slot ID pins correctly
PSEQ—CBC detected power sequencer failure
DBPO—CBC detected an issue during board power up
KPWR—CBC detected an issue during board power up
LGNP—CBC detected an issue during board power up
LGNI—CBC detected an issue during board power up
RMN—All tests are finished and ROMMON is ready for commands
LOAD—Downloading MBI image to CPU
RRST—ROMMON is performing a soft reset after 5 consecutive MBI validation requests timed out
MVB—ROMMON trying MBI validation boot
MBI—Starting execution of MBI
IOXR—Cisco IOS XR software is starting execution
LDG—The RSP/RP is loading (MBI started and card preparing for activity)
INCP—The software or configuration is incompatible with the RSP/RP
OOSM—The RSP/RP is in Out of Service, Maintenance mode
ACTV—RSP/RP role is determined to be active RSP/RP
STBY—RSP/RP role is determined to be standby RSP/RP
PREP—Preparing disk boot
You can use the alphanumeric display to isolate a problem with the RSP:
•Alphanumeric display is powered directly from the CAN microcontroller on the RSP card through the chassis backplane:
–If the alphanumeric display is not operating, the CAN microcontroller may be malfunctioning.
–If the CAN microcontroller is operating, the alphanumeric display could be on even if the RSP failed to power on.
•If the alphanumeric display is not operating, but the power modules and the fan trays are operational, the RSP may not be installed properly, or the +5 VDC output from the chassis backplane may be faulty:
–Make sure that the system is powered on.
–Initialize the RSP card by ejecting it from the chassis backplane and then reseating it.
•If the alphanumeric display is operating, check the meaning of the messages (see Table 4-6 for the RSP).
When the DC-to-DC converters are powered on by the CAN microcontroller, the begins the RSP boot process and displays various status messages. Some messages appear briefly; while others appear for several seconds. If the messages appear to stop at a particular point, the boot process may be halted:
–Make a note of the message.
–Turn off power to the router, then turn on the power again to reset the router and start the boot process. If the router halts again, replace the RSP (see the "Removing and Replacing Cards from the Chassis" section).
Table 4-6 Troubleshooting Using the RSP Alphanumeric Display Messages
Message
|
Description
|
PST1
|
Failed DDR RAM memory test
|
PST2
|
Failed FPGA image CRC check
|
PST3
|
Failed board type and slot ID verification
|
FC Card Front Panel Indicator
The front panel of the FC card has one tri-color LED indicator for system information.
Table 4-7 lists the display definitions of the discrete LED on the FC card front panel.
Table 4-7 FC Card LED Display Definitions
Indicator (Label)
|
Color
|
Description
|
Power Fail (FAIL)
|
Green
|
FC card powered on and FPGA is programmed.
|
Red
|
Malfunction in FC card power up or FPGA programming.
|
Amber
|
FC card powered on but fabric not active.
|
Off
(Default after reset)
|
FC card powered off by CLI.
|
Troubleshooting Line Cards and Modular Port Adapters
To troubleshoot line cards and modular port adapters (MPAs), see the Cisco ASR 9000 Series Aggregation Services Router Ethernet Line Card Installation Guide.
Monitoring Critical, Major, and Minor Alarm Status
The alarms can warn of:
•Overtemperature condition on a component in the card cage
•Fan failure in a fan tray
•Overcurrent condition in a power supply
•Out-of-tolerance voltage on one of the cards
•Insertion count for an RSP, RP, FC, or line card has reached a specified threshold. For more information on OIR insertion counts, see "OIR Monitoring" section.
The alarm LEDs are controlled by the CAN microcontroller software, which sets the threshold levels for triggering the different stages of alarms.
The RSP/RP card continuously polls the system for temperature, voltage, current, and fan speed values. If a threshold value is exceeded, the RSP/RP sets the appropriate alarm severity level on the alarm card, which lights the corresponding LED, and energizes the appropriate alarm display relays to activate any external audible or visual alarms wired to the alarm display. The RSP/RP also logs a message about the threshold violation on the system console.
Note If one or more of the alarm LEDs is on, check the system console for messages describing the alarm.
Troubleshooting the Cooling Subsystem
You may need to troubleshoot the cooling subsystem if an overtemperature condition occurs. The cooling subsystem of the router consists of a fan tray in the chassis and a fan in each of the power supplies. The fan tray and the power supply fans circulate air to maintain acceptable operating temperatures within the router.
Caution When troubleshooting the fan trays, never unplug all the fan trays at the same time.
This section contains information to troubleshooting the cooling subsystem and includes:
•Fan Tray Operation
•Power Module Fans
•Overtemperature Conditions
•Isolating Cooling Subsystem Problems
Fan Tray Operation
The fan trays maintain acceptable operating temperatures for the internal components by drawing cooling air through a replaceable air filter into the switch fabric and alarm card cage and then through the line card and RSP card cage (see Figure 1-8 for the Cisco ASR 9010 Router cooling path, and Figure 1-9 for the Cisco ASR 9006 Router cooling path). In the Cisco ASR 9922 Router, the fan trays draw cooling air from the front of the chassis into the middle card cage and then through each of the two line card cages top and bottom (see Figure 1-10). The fan tray receives power from the chassis backplane through a wiring harness.
The fan tray contains 12 fans (Cisco ASR 9010 Router and Cisco ASR 9922 Router) or six fans (Cisco ASR 9006 Router), a controller card, and one front panel status LED indicator:
•Green—Fan tray is functioning properly.
•Red—There is a fault detected in the fan tray.
If the air temperature inside the chassis rises, blower speed increases to provide additional cooling air to the internal components. If the internal air temperature continues to rise beyond the specified threshold, the system environmental monitor shuts down all internal power to prevent equipment damage because of excessive heat.
If the system detects that one or more of the fans in the fan tray has failed, it displays a warning message on the system console. In addition, the remaining fans go to full speed to compensate for the loss of the failed fan.
Note Due to air leakage, the chassis should not be operated with any of the fan trays completely missing. Replace any missing fan tray within five minutes. Any fan tray replacement should be performed when the chassis is back to room temperature.
Power Module Fans
Each AC or DC power module is equipped with two fans that draw cooling air in through the front of the power module and force warm air out the back of the power tray:
•If the power source is within the required range, the power supply fan remains on.
•If a fan fails:
–Power module detects an internal overtemperature condition.
–Fault and Temp indicators light.
–Power module sends an overtemperature warning to the system and then shuts down the system.
For additional power supply troubleshooting information, see the "Troubleshooting the Power Subsystem" section.
Note For the RSP/RP to communicate properly to a power module in a power tray, input power to at least one of the three power modules in the power tray should be present.
Overtemperature Conditions
The following console error message indicates that the system has detected an overtemperature condition or out-of-tolerance power value inside the system:
%ENVM-1-SHUTDOWN: Environmental Monitor initiated shutdown
The preceding message could also indicate a faulty component or temperature sensor. Enter the show environment command or the show environment all command at the user EXEC prompt to display information about the internal system environment. The information generated by these commands includes:
•Voltage measurements on each card from the DC-to-DC converter
•The +5 VDC for the I2C module
•Operating voltage for the fan trays
•Temperature measurements received by two sensors on each card (one for inlet air temperature and one for the card's hot-spot temperature) as well as temperature measurements from sensors located in each power module
If an environmental shutdown results from an overtemperature or out-of-tolerance condition, the Fault indicator on the power supply lights before the system shuts down.
Although an overtemperature condition is unlikely at initial system startup, make sure that:
•Heated exhaust air from other equipment in the immediate environment is not entering the chassis card cage vents.
•You allow sufficient air flow by maintaining a minimum of 6 inches (15.24 cm) of clearance at both the inlet and exhaust openings on the chassis and the power modules to allow cool air to enter freely and hot air to be expelled from the chassis.
Isolating Cooling Subsystem Problems
Follow these steps to isolate a problem with the chassis cooling system if you have an overtemperature condition:
Step 1 Make sure the fan trays are operating properly when you power on the system. To determine if a fan tray is operating, check the LED indicator on each fan tray front panel:
•OK (green)—Fan tray is functioning properly and receiving -48 VDC power, indicating that the cables from the chassis backplane to the fan tray are good.
•Fail (red)—Fault is detected in the fan tray. Replace the fan tray.
•If neither indicator is on and the blower is not operating, there may be a problem with either the fan tray or the -48 VDC power supplied to the fan tray. Go to Step 2.
Caution Never unplug all the fan trays at the same time.
Step 2 Eject and reseat the fan tray making sure the captive screws are securely tightened to a torque of 10 +/-1 in-lb.
If the fan tray still does not function, go to Step 3.
Step 3 Check for -48 VDC power by looking at the LED indicators on each power module:
•If the Pwr OK indicator is on and the Fault indicator is off on each power module, it indicates that the fan trays are receiving -48 VDC:
–If a fan tray is still not functioning, there could be a problem with the fan tray controller card or an undetected problem in the fan tray cable. Replace the fan tray.
–If the new fan tray does not function, contact a Cisco customer service representative for assistance.
•If the Fault indicator is on, the power supply is faulty. Replace the power supply.
•If the Temp and Fault indicators are on, an overtemperature condition exists:
–Verify that the power supply fan is operating properly.
–If the fan is not operating, replace the power supply.
–Contact your Cisco representative if replacing the power supply does not fix the problem.
Feedback
