Cisco 12008 Gigabit Switch Router Installation and Configuration Guide
Troubleshooting the Installation
Downloads: This chapterpdf (PDF - 147.0KB) The complete bookPDF (PDF - 5.95MB) | Feedback

Troubleshooting the Installation

Table Of Contents

Troubleshooting the Installation

Troubleshooting Overview

Normal System Status at Startup

Problem Solving Using a Subsystem Approach

Identifying Startup Problems

Normal System Startup Sequence

Power Supply Status LEDs

Troubleshooting the Power Subsystem

Troubleshooting the Processor Subsystem

Troubleshooting the RP

Troubleshooting the Line Cards

CSC Alarm Functions

Troubleshooting the Cooling Subsystem

Additional Troubleshooting Reference Information


Troubleshooting the Installation


Your Cisco 12008 was subjected to extensive testing and burn-in before being shipped from the factory. However, if you encounter problems starting up the router, the information in this chapter will help you to isolate the probable cause.

This chapter contains the following sections:

Troubleshooting Overview

Troubleshooting the Power Subsystem

Troubleshooting the Processor Subsystem

Troubleshooting the Cooling Subsystem

Additional Troubleshooting Reference Information

In this chapter, it is assumed that you are troubleshooting the initial Cisco 12008 system startup, and that the system is in the original factory configuration. If you have removed or replaced components or changed any default settings, the recommendations in this chapter might not apply.

At initial system startup, you should verify the following:

External power cables are connected, and proper source power is being supplied.

The card cage fan tray and the power supply fan tray are operating.

The system software boots successfully.

The RP and the line cards are properly installed in their slots; each card initializes (is enabled by system software) without problems.

If you cannot solve a problem, contact a customer service representative for assistance. When you call, have the following information at hand:

Date you received the router and the chassis serial number

Line card configuration (the line cards installed in your system)

The version number of the Cisco IOS software running on your system

Brief description of the problem being experienced and the steps you have already taken to isolate and resolve the problem

Maintenance agreement or warranty information

Troubleshooting Overview

This section identifies the normal status of the system at startup. It also describes the methods used in troubleshooting the Cisco 12008. To enable efficient problem solving, the troubleshooting methods are presented organized according to the router's major subsystems.

Normal System Status at Startup

shows the contents of the alphanumeric displays on the RP and the line cards at system startup; this table also shows the normal, expected state of the LEDs on the faceplate of the CSC(s) at system startup.

For the layout of the status LEDs on the CSC faceplate, refer to .

Table 5-1 Status of Alphanumeric Displays and LEDs at System Startup

Card
LED Function
Display Contents/LED Status/Meaning

RP

LED alphanumeric displays

MSTR (top)

RP (bottom)

The RP is enabled and recognized by the system; a valid Cisco IOS software image is running.

Line card

LED alphanumeric displays

IOS (top)

RUN (bottom)

The line card is enabled and ready for use.

CSC(s)

System alarm LEDs

CRITICAL: OFF

MAJOR: OFF

MINOR: OFF

No system alarm conditions exist.

CSC Status LEDs

FAIL: OFF

ENABLED: ON

The CSC is functional.

FAN FAIL Status LEDs

LINECARD (left LED): OFF

PWR SPLY (right LED): OFF

The fan trays are operational.

SFC Status LEDs

FAIL (top): OFF

ENABLED (bottom): ON

All the SFCs are functional.

AC-input power supply

Power status LEDs

AC INPUT OK (top): ON

OUTPUT FAIL (bottom): OFF

The power supply voltages are present and within tolerance.

DC-input power supply

Power status LEDs

INPUT OK (top): ON

OUTPUT FAIL (bottom): OFF

The power supply voltages are present and within tolerance.


Problem Solving Using a Subsystem Approach

The key to solving system problems is to try to isolate the difficulty to a specific subsystem. The first step in solving startup problems, for example, is to compare what the system is doing to what it should be doing.

Since a startup problem is usually attributable to a single component, it is more effective to isolate the problem to a specific subsystem, rather than trying to troubleshoot each system component.

For purposes of the troubleshooting procedures in this chapter, the Cisco 12008 will be regarded as consisting of the following subsystems:

Power subsystem—The power subsystem consists of the following components:

AC-input power supplies or DC-input power supplies—The Cisco 12008 can be configured to operate with either source AC power or source DC power.


Note   An AC-input power supply and a DC-input power supply cannot be used together in the same Cisco 12008 chassis.


A Cisco 12008 can be configured to operate with either a single AC-input power supply or a single DC-input power supply. Adding a second power supply of the same type provides redundancy (backup power capability) and enables the power supplies to share the current load for the router.

DC-DC converters—A DC-DC converter is incorporated into each of the cards that you can install in the upper card cage (the RP, CSCs, and line cards) or the lower card cage (the SFCs). The DC-DC converter on each card operates under control of its onboard MBus module.

The converter takes the -48 VDC supplied to the card through the backplane from the power supply(ies) and converts it into the +3.3 VDC and +5  VDC required by the card's circuitry.

Cooling subsystem—The cooling subsystem consists of two fan trays: one for the card cage and one for the power supply bays.

Each fan tray incorporates individual fans that draw +24 VDC from a DC-DC converter on the CSC that operates under control of its onboard MBus module. Both fan trays should begin to operate about two seconds after application of system power.

The fan trays incorporate a variable-speed feature, enabling the fans to run at a slower speed (when the internal chassis temperature remains within the normal operating range) or to run at a higher speed (when the internal temperature of the router exceeds a specified threshold). In the latter case, the voltage being delivered to the fans is increased, causing them to run at maximum speed to force a greater volume of air through the router.

In a noisy, air-conditioned environment, it may be difficult to hear whether or not the fan trays are operating. The first indication of a fan failure comes from the side-by-side fan tray status LEDs on the CSC faceplate (see ).

If the LED on the left is on (amber), a fan failure has occurred in the card cage fan tray. If the LED on the right is on (amber), a fan failure has occurred in the power supply fan tray.

Each fan in a fan tray is monitored separately for failure. A failed fan is not shut off in the usual sense; instead, a current-limiting feature in the faulty fan prevents it from interfering with the operation of other fans. The most common fan failure is that the fan just stops running, but continues to draw current.

If a fan in the card cage fan tray or the power supply fan tray fails, the CSC increases the voltage being delivered to the fans, causing them to run at maximum speed to compensate for the failed fan.

You can make a gross assessment of fan tray operation by placing your hand near the exhaust vents at the top rear of the router enclosure. A reduced flow of air from the card cage exhaust vent or the power supply exhaust vent may indicate that one or more fans on a given fan tray have failed. However, this method of checking fan operation is neither definitive nor reliable.

If you determine that a fan tray is not operating properly, you should immediately consider replacing the faulty fan tray to guard against an overheating condition in the router that could approach the shutdown threshold.

No operating adjustments can be made for either fan tray. An individual fan is either operating normally or it is not operating at all.

Processor subsystem—The processor subsystem includes the RP and all installed line cards. Each line card has an onboard processor, to which the RP downloads a copy of the Cisco IOS operating image.

A line card or RP that is not firmly seated in the backplane might cause the system to hang and crash. Two 4-character alphanumeric displays at the bottom of the RP and each line card provide status and error messages that can be an aid in troubleshooting.

Identifying Startup Problems

Startup problems are commonly due to problems with source power or to a card that is not properly seated in the backplane.

Normal System Startup Sequence

Each card installed in the system incorporates an MBus module and at least one DC-DC converter. The MBus module on each card provides an interface to the RP via the system's maintenance bus (MBus); the MBus module on each card also controls its own onboard DC-DC converter.

When you first start up the router, the following sequence of events occurs:

Each MBus module receives +5 VDC directly from the power supply(ies) through the backplane. When the power supply power switches are turned on, each MBus module powers up. As it does so, the MBus module processor boots from its onboard EEPROM.

As part of the boot process, each MBus module processor reads a set of ID pins on the card. These pins, whose meaning is determined at the time of card manufacture, identify the kind of card on which the MBus module is mounted.

In the case of the MBus modules on the CSC and the RP, the CSC MBus module processor immediately enables the card's DC-DC converter, followed by that of the RP. (The CSC takes precedence in the system power-up sequence, because it supplies master clocking services to the entire system.)

If the onboard MBus module determines that it is mounted on a line card or an SFC, the MBus module remains idle for the time being, leaving the card powered down until it receives power up instructions from the master MBus module on the RP.

The MBus module on the RP monitors the progress of the CSC power-up sequence. When the CSC is powered up, the MBus module on the RP turns on its DC-DC converter to power its own electronics.

The master MBus module on the RP then sends instructions to each line card and each SFC to power up; the MBus module on each card then turns on its own onboard DC-DC converter.

The MBus module on each line card monitors its progress during the power-up sequence. During this process, the line card processor performs its own boot sequence to load line card operating software. When the line card boot process is complete, the card communicates its status to the master MBus module on the RP.

The MBus module on each SFC also monitors its own progress during the power-up sequence. When the SFC boot process is complete, the cards likewise communicate their status to the master MBus module on the RP.

As the power on and boot process progresses for the RP and each installed line card, the status of each card is given in the alphanumeric displays near the bottom of its faceplate.

The top display is powered by the DC-DC converter on the card; the bottom display is powered by the +5 VDC provided through the backplane by the AC-input or DC-input power supply(ies).

By checking the status LEDs on the power supply(ies) and the alphanumeric displays on the RP and each line card, you can generally determine when and where the system failed during the startup sequence.

The following section describes what you should expect to see in the power supply LEDs on system startup.

Power Supply Status LEDs

When you start up the system by turning on the rotary power switch on the power supply faceplate, the following should occur:

The green INPUT OK LED on each DC-input power supply or the green AC INPUT OK LED on each AC-input power supply should go on immediately and stay on as long as source power is applied to the system.

If this LED does not go on, or if it goes off while system power is still applied, there could be a problem with either the source power itself or the DC output voltages being delivered to the backplane (+5 VDC and -48 VDC). There could also just be a problem with the MBus controller inside the power supply.

The AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) stays on when all of the following conditions are satisfied:

The power supplies are on and receiving normal source power.

The source power specifications for the AC-input power supply are 180 VAC to 264 VDC, 50 to 60 Hz, single phase.

The source power specifications for the DC-input power supply are -40.5 VDC to -70 VDC.

The power supplies are providing the nominal +5.2 VDC and -48 VDC operating voltages to the backplane.

If the AC or DC source power or the DC operating voltages being supplied to the backplane fall outside the allowable tolerances, the OUTPUT FAIL LED on the power supply goes on, because a problem with either of the DC operating voltages being supplied to the backplane prevents the system from starting up or continuing normal operations.

For example, if a problem occurs with the -48 VDC line that supplies power to the router's internal components (see ), the system fails during the boot sequence.

Depending on when the OUTPUT FAIL LED goes on, proceed as follows:

If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) stays off when system power is applied, there is a problem with either the AC or the DC source power or the connection to the power supply. In this case proceed to the section entitled "Troubleshooting the Power Subsystem."

If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) goes on temporarily and then goes off within 30 seconds, the system is probably shutting itself down because it detected an out-of-tolerance power or temperature condition within a power supply. In this case proceed to the section entitled "Troubleshooting the Power Subsystem."

If the AC INPUT OK LED (on the AC-input power supply) or the INPUT OK LED (on the DC-input power supply) goes on, and the system starts up as expected but then displays the following message and shuts down after 2 minutes, there may be a problem with one of the fan trays.

%ENVM-2-FAN: Fan has failed, shutdown in 2 minutes

In this case, proceed to the section entitled "Troubleshooting the Cooling Subsystem."

If the red OUTPUT FAIL LED goes on (when a failure occurs with a power supply of either type), yet the system starts up correctly, displays the preceding message, and shuts down after about 2 minutes, there may a problem with the -48 VDC being supplied to the chassis. In this case, proceed to the section entitled "Troubleshooting the Power Subsystem."

When you turn on the rotary power switch, you should hear the fan trays come up to normal rotational speed after about two seconds.

If background noise prevents you from hearing fan tray operation, place your hand at the top rear of the router enclosure to determine if air is being exhausted from the fan tray vents. You should feel a steady volume of air coming from these vents.

If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) is on, but the fan trays are not operating, there might be a problem with the fan trays (or, possibly, the CSC, which powers the fan trays).

To check the operating voltage of the fan trays, issue either the show environment or the show environment all command. (Either command displays information about the system's internal environment.) If the fan tray voltage is okay, the fan trays themselves may be faulty. If the fan tray voltage is bad, there may be a problem with the CSC (which supplies the fan tray operating voltages).

If the system detects that it is overheating due to a fan tray failure, it shuts itself down. In this case, proceed to the section entitled "Troubleshooting the Cooling Subsystem."

When you turn on system power, the alphanumeric displays on the RP indicate the following:

The top display indicates which component is running.

The bottom display indicates the current stage of the boot process.

Troubleshooting the Power Subsystem

The power subsystem in the Cisco 12008 consists of the following:

AC-input or DC-input power supply(ies)

MBus modules on the individual cards

DC-DC converters on the individual cards

Power distribution system in the Cisco 12008 (see )

Each power supply provides +5.2 VDC and -48 VDC output voltages to the backplane. The +5.2 VDC output powers the MBus modules on each card in the system. The MBus modules, in turn, control the DC-DC converters on each card. The DC-DC converter takes the -48 VDC from the power supply and converts it into the various voltages required to operate card circuitry.

To begin checking the power subsystem, examine the status of the two LEDs on the power supply faceplate at initial system startup for the following conditions:

The AC INPUT OK LED on the AC-input power supply or the INPUT OK LED on the DC-input power supply goes on when the rotary power switch on the power supply faceplate is turned on (|) and the unit is receiving source AC or DC power.

The OUTPUT FAIL LED is normally off, but it goes on if the power supply output voltage is not within tolerance.

In systems with a single AC-input power supply or a single DC-input power supply, and in systems with redundant power when both power supplies are being shut down, the OUTPUT FAIL LED goes on momentarily as the system shuts down, but goes off and remains so when the power supply completely shuts down.

Each AC-input or DC-input power supply is monitored by its own onboard MBus module, as well as by the master MBus module on the RP. Each power supply is monitored for internal temperature conditions, overvoltage conditions, and overcurrent conditions.

Continue checking the power subsystem by assessing the following:

Is the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) on?

If the answer is yes, source power is present and within tolerance; the power supplies are functional.

If the answer is no, but the fan trays are operating and the alphanumeric displays on the RP and line cards are functional, the likely suspect is a faulty power supply LED.

The MBus modules driving the alphanumeric displays are powered by +5.2 VDC from the power supply.

The fan trays are powered by +24 VDC from the DC-DC converter on the CSC; hence, if the RP and the fan trays are operating normally, all internal DC voltages are within tolerance.

You can issue the show environment command at the user EXEC mode prompt to display temperature and voltage information for each installed card in the system.

If the answer is no and there is no other obvious system activity, verify that the rotary power switch on the power supply is in the fully clockwise ON (|) position. If it is not, turn the power switch clockwise until you hear an audible click, ensuring that the switch is in the full ON position and that the latching mechanism has been engaged.

If the rotary power switch is set correctly and the AC INPUT OK LED (on the AC-input power supply) or the INPUT OK LED (on the DC-input power supply) still remains off, examine the source AC or source DC power cable.

If the internal circuit breaker of the power supply has been tripped, you can reset it by turning the rotary power supply switch OFF then ON again. However, note that the tripping of the internal circuit breaker is a likely indication of an electrical problem in the chassis.

Turn the rotary power switch OFF (O); check the state of the source AC or DC circuit breaker, as appropriate. Verify that the breaker is ON and that it has not been tripped. Verify that the source circuit breaker has the proper current rating.


Note   Each power supply in the Cisco 12008 should be attached to a separate power source.


In systems powered by source AC, there might be an uninterruptable power supply (UPS) for each installed AC-input power supply. Check that the UPS is functioning correctly.

In systems powered by source DC, check the cable connections to the terminal studs on the DC-input power supply faceplate to ensure that the correct polarity (+ and -) has been observed in making the connections.

Is the power cord (AC) or the power cable (DC) from the power source to the router in good condition and not damaged? If insulation on the cord or the cable appears cracked or broken, or if the AC plug or the DC terminal lugs appear to be loose, do not use the item. Immediately replace it with a new AC power cord or a new DC power cable.

If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) still fails to go on after you connect the power supply to a new power source, swap the existing power cord or cable with a replacement unit.

Turn the rotary power switch back ON. If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) goes on, return the first power cable for replacement.

If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) still fails to go on when the router is connected to a different power source with a new power cable, the power supply itself is probably faulty.

If a spare power supply is available, replace the existing power supply with the spare unit and restart the system. If the AC INPUT OK LED (AC-input power supply) or the INPUT OK LED (DC-input power supply) goes on, return the faulty power supply for replacement.

If you are unable to resolve the problem, or if you determine that either the power supply or the power cord (AC) or power cable (DC) is faulty, contact your Cisco service representative for assistance.

Troubleshooting the Processor Subsystem

The Cisco 12008 processor subsystem consists of the RP and the line cards.

The RP and the line cards each have two onboard processors. One processor serves as the card's main processor, and the other serves as the MBus module processor. The latter monitors the card's environment and controls the card's onboard DC-DC converter.

The MBus module processor begins to operate as soon as power is applied to the system. The MBus processor determines what type of card it is mounted on and whether it should turn on the card's DC-DC converter. The master MBus module on the RP turns on card power after a brief delay; the MBus modules on the line cards delay turning on power until they receive a command to do so from the RP master MBus module.

To operate at all, a minimally configured Cisco 12008 must have a RP. However, the system can operate without any line cards, as long as no line card is in partial contact with the backplane pins.

A line card that is only partially inserted into the backplane sends incomplete signals to the RP, which can cause the system to hang. First ensure that the RP is installed properly and that the system software has initialized successfully. Then, if necessary, you can troubleshoot the individual line cards.

A power-on self-test (POST) runs immediately at power-on to determine the condition of RP memory. The results of this test appear in the alphanumeric display as a pass/fail message.

Troubleshooting the RP

To isolate a problem with the RP, assess the following:

Are both RP alphanumeric displays on?

If both displays are off, the RP may not be fully seated in the backplane, there may be a problem with the MBus module on the RP, or the system power supply may be off.

The two alphanumeric displays are powered separately. The top display receives power from the onboard DC-DC converter of the RP. The bottom display is powered directly from the MBus by means of the power supply(ies). Thus, even if the RP has not powered up, the bottom display could be on.

If both displays are on, check the message being displayed.

As soon as the DC-DC converter is turned on by the MBus module, the RP processor begins the boot process. Status messages are displayed as the boot process continues (see ).

Some messages appear briefly (millisecond duration), and others last several seconds. If one of the messages appears frozen, the boot process could be hung. Note the message being displayed on a piece of paper. Turn off the system power supply power switches, then turn them back on to reset the system and start the boot process anew. If the system hangs again, the RP could be faulty and in need of replacement.

Table 5-2

Top Display
Bottom Display
Message

LMEM

TEST

Low memory test running

LCAH

INIT

Lower 15K cache initialization

BSS

INIT

Initialize main memory for ROM

NVRAM

INIT

Initialize NVRAM

EXPT

INIT

Initialize interrupt handlers

TLB

INIT

Initialize TLB

CACH

INIT

Initialize CPU data and instruction cache

CACH

PARY

Enable CPU cache parity

MEM

INIT

Initialize main memory

NVRAM

SIZE

Size the NVRAM

PCMC

INIT

Initialize the PCMCIA

EXIT

INIT

Exit the initialization sequence

IOS

UP

The Cisco IOS is up and running


RP Alphanumeric Display Messages

If the power supplies and fan trays are operational, but none of the RP LEDs or displays are on, the RP might not be properly installed, or the +5.2 VDC output from the power supply(ies) might be faulty. Turn the rotary power supply switch(es) off, and then loosen the two captive screws at the top and bottom of the RP faceplate; use the ejector levers on the RP to eject it and reseat it. Tighten the two captive screws, and power up the system by turning the rotary power switch(es) to the on (|) position.

Is a critical, major, or minor alarm LED on the CSC(s) on?

If any one of the three system alarm LEDs is on, a fault has been detected in the system. Check the console for messages indicating the source of the problem.

A false error indication may be originating from the RP. Reseat or replace the RP.


Caution   
The RP reset switch resets the RP and the entire system. To prevent system errors and problems, use the RP reset switch only at the direction of your Cisco-certified service representative.

Troubleshooting the Line Cards

Line cards can be installed in any slot in the upper card cage, except for the two slots in the middle of the card cage, which are reserved for the CSCs. Also, by convention, the left-most slot in the upper card cage is occupied by the RP. However, you need not follow this recommendation.

As each line card powers up in response to a command issued by the RP across the MBus, a power-on self-test (POST) is performed on line card memory. A full set of field diagnostics can also be run on a line card from the system console. The diagnostics provide a pass/fail message in the line card alphanumeric displays, as well as on the system console.

To isolate a problem with a line card, make the following assessment:

Are both line card alphanumeric displays on?

If both alphanumeric displays are off, the line card may not be fully inserted into the backplane, there may be a problem with the MBus module on the line card, or the system power supply may be off.

The two line card alphanumeric displays are powered separately. The top display receives power from the DC-DC converter on the line card, and the bottom display receives power from the MBus. Thus, even if the line card has not powered up, the bottom display could be on.

If both displays are on, check the message being displayed. As soon as the DC-DC converter is turned on by the MBus module, the processor on the line card begins the boot process. Status messages appear in the alphanumeric displays as the boot process continues on the line card. lists the messages that appear in the line card alphanumeric display. Some messages are displayed for only a few milliseconds, and others are displayed for several seconds.

Table 5-3 Line Card Alphanumeric Display Messages

Top Display
Bottom Display
Message

MEM

TEST

POST memory test running

LROM

RUN

After POST memory test

BSS

INIT

Initialize main memory for ROM

RST

SAVE

Save reset reason register

IO

RST

Reset the I/O system on the card

EXPT

INIT

Initialize interrupt handlers

TLB

INIT

Initialize TLB

CACH

INIT

Initialize CPU data and instruction cache

MEM

INIT

Initialize main memory

LROM

RDY

Ready to access download

ROMI

GET

Getting ROM images

FABL

WAIT

Wait for load of fabric downloader

FABL

DNLD

Loading fabric downloader

FABL

STRT

Launching fabric downloader

FABL

RUN

Fabric downloader launch complete

IOS

DNLD

Downloading the IOS

IOS

STRT

Launching the IOS

IOS

UP

IOS is running

IOS

RUN

Line card enabled


CSC Alarm Functions

The CSC incorporates the following system and component alarm functions:

Primary system alarm functions—The system's primary alarm LEDs are incorporated into the CSC faceplate (see ). From top to bottom, these LEDs correspond to three levels of severity for system alarm conditions: critical, major, and minor.

External alarm monitoring facility—The CSC faceplate has a DB-25 connector for the attachment of a site-wide external monitoring system. Such a system can provide both visible and audible alarms.

A 25-pin D-sub connector on the CSC faceplate is connected directly to the critical, major, and minor alarm relays incorporated into the CSC.


Note   Only safety extra-low voltage (SELV) external alarm circuits can be attached to the DB-25 connector on the CSC faceplate.


A manual reset switch is also incorporated into the CSC faceplate that you can push to reset (silence) an audible alarm. The visible alarm, however, can be reset only by the system's alarm monitoring software.

Alarm status for other installed components—The CSC provides a visible indication of the status of other cards and components installed in the system. Two LEDs indicate the status of the CSC itself; two side-by-side LEDs indicate the status of the card cage fan tray and the power supply fan tray; finally, two LEDs at the bottom of the CSC faceplate indicate the status of the SFCs installed in the lower card cage.

Check the CSC for an indication of a critical, major, or minor alarm. If any one of the three system alarm LEDs is on, check the system console for messages describing the fault condition.

Troubleshooting the Cooling Subsystem

The Cisco 12008 incorporates two fan trays (see ) that provide cooling air for other system components:

Card cage fan tray—This fan tray is mounted in the lower card cage behind the air filter assembly.

Power supply fan tray—This fan tray is mounted in the lower right corner of the router enclosure.

Each fan tray operates from +24 VDC that is distributed to it through the backplane from a DC-DC converter on the CSC. A recessed, blind-mating connector in the back of each fan tray provides connectivity to the backplane.

Both fan trays are essential; they provide sufficient cooling air for the router's components and electronic circuitry. The card cage fan tray incorporates six variable-speed fans; the power supply fan tray incorporates four variable-speed fans.

To isolate a problem with the router's cooling subsystem, make the following assessments:

When the system is started up, do the fan trays begin to come up to normal rotational after about two seconds?


Note   To determine if the fan trays are operating, first listen for the hum of the fans. In a noisy environment, you might have to place your hand at the top rear of the router enclosure to feel the air being exhausted from the fan tray vents.


If both fan trays operate as expected, the +24 VDC power from the DC-DC converter on the CSC is confirmed as present and operational.

If one fan tray or the other does not operate as expected, there could be a problem with the voltage being supplied to the fan tray. In this case, check the status of the power supply LEDs on the CSC faceplate (see ).

If the left fan tray LED on the CSC faceplate is on (amber), reseat the card cage fan tray in the backplane, as follows:

1 Remove the air filter assembly from the chassis.

2 Loosen the two captive installation screws securing the fan tray to the chassis frame.

3 Grasp the fan tray carrier by its extraction/insertion ring and pull the unit part way out of the slot.

4 Firmly reseat the fan tray in the slot.

5 Tighten the fan tray captive installation screws.

6 Reinstall the air filter assembly.

If the left fan tray LED remains on after reseating the card cage fan tray, the unit is faulty and should be replaced.

If the right fan tray LED on the CSC faceplate is on (amber), reseat the power supply fan tray in the backplane, as follows:

1 Loosen the captive installation screw on the fan tray faceplate.

2 Grasp the fan tray by the captive installation screw and partially withdraw the unit from the slot.

3 Firmly reseat the fan tray in the slot.

4 Tighten the fan tray captive installation screw.

If the right fan tray LED remains on after reseating the power supply fan tray, the unit is faulty and should be replaced.

If a fan tray still fails to operate as expected, a problem may exist with the DC-DC converter on the CSC that powers the fan trays.

On system startup, note that the fans operate at maximum speed until IOS is fully booted or if any fans fail or are absent.

Fan speed is determined by temperature sensors on the cards in the upper and lower card cages and the power supplies. If an air temperature above the normal operating limit is detected within the router, the MBus module on the CSC increases the fan tray operating voltage. This higher voltage results in a corresponding increase in fan speed, forcing more cooling air through the entire router enclosure.

The following message indicates that the system has detected an overtemperature condition or an out-of-tolerance power condition inside the router:

Queued messages:
%ENVM-1-SHUTDOWN: Environmental Monitor initiated shutdown

The preceding message could also indicate a faulty component or a faulty temperature sensor. Before the system shuts down, issue the show environment command or the show environment all command. Either command displays information about the internal system environment, including voltage measurements on each card for the +3.3 VDC and +5 VDC from the DC-DC converter, the +5.2 VDC for the MBus module, and the operating voltage for the fan trays. Each command also displays the temperature measurements made by two sensors on each card (one for inlet air temperature and one for the card's hot-spot temperature), as well as a temperature measurement made by a sensor in the power supply(ies).

If an environmental shutdown results from an out-of-tolerance power condition, the OUTPUT FAIL LED on the power supply goes on before the system shuts down. In this case, refer to the section entitled "Troubleshooting the Power Subsystem."

Although an overtemperature condition is unlikely at initial system startup, ensure that heated exhaust air from other equipment in the immediate environment is not entering the air filter assembly; in addition, ensure sufficient clearance — at least 12 inches (30.5 cm) — in the lower front and top rear of the chassis to enable cooling air to freely enter and be expelled from the chassis.

Check the condition of the air filter. If it appears dirty, remove the filter and clean it, or replace it altogether.

There are no field-replaceable components in the fan trays. If a fan tray is faulty, you must replace the entire unit.

If you are still unable to resolve a problem with the router's cooling subsystem, contact a Cisco service representative for assistance.

Additional Troubleshooting Reference Information

The following additional reference materials are available for troubleshooting your Cisco 12008 installation:

The various configuration notes for the system's major components

Troubleshooting Internetworking Systems

Debug Command Reference

System Error Messages