Document ID: 18899
Updated: Jul 14, 2005
Contents
Introduction
Valuable time and resources are often wasted replacing hardware that actually functions properly. This document helps you to troubleshoot common hardware issues on Cisco AS5800 Series Routers, and provides pointers for identifying whether or not there is a hardware fault.
Note: This document does not cover any software-related failures.
Prerequisites
Requirements
Readers of this document should have knowledge of these topics:
-
It is recommended that you read Troubleshooting Router Crashes before proceeding with this document.
-
You should be familiar with the various hardware parts on the AS5800. For more information on the components of the AS5800, refer to the document AS5800 Introduction. The Cisco AS5800 Product Overview also includes diagrams for the various components.
-
You should also be knowledgeable of using Cisco IOS® software on the AS5800, and executing commands on the router-shelf (RS) and dial-shelf (DS).
Components Used
The information in this document is based on these software and hardware versions:
-
Cisco AS5800 Series Routers
-
Cisco IOS Software (all versions)
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
For more information on document conventions, refer to the Cisco Technical Tips Conventions.
Background Information
Whenever you install a new card/module or Cisco IOS software image, it is important to verify that the access server has enough memory, and that the hardware and software are compatible with the features you wish to use.
Perform these recommended steps to check for hardware-software compatibility and memory requirements:
Note: To use the tools referenced in this document, you must be a registered customer, and you must be logged in.
-
Use the Software Advisor tool (registered customers only) to choose software for your network device.
Tips:
-
The Software Support for Features section (registered customers only) helps you determine the Cisco IOS software image needed by choosing the types of features you wish to implement.
-
-
Use the Download Software Area (registered customers only) to check the minimum amount of memory (RAM and Flash) required by the Cisco IOS software, and/or download the Cisco IOS software image. To determine the amount of memory (RAM and Flash) installed on your router, refer to How to Choose a Cisco IOS Software Release - Memory Requirements.
Tips:
-
If you want to keep the same features as the version that is currently running on your router, but do not know which feature set you are using, issue the show version command on your router and paste the output in the Output Interpreter tool (registered customers only) to find out. It is important to check for feature support, especially if you plan to use recent software features.
-
If you need to upgrade the Cisco IOS software image to a new version or feature set, refer to How to Choose a Cisco IOS Software Release for more information.
-
-
If you determine that a Cisco IOS software upgrade is required, see Upgrading IOS Software on Cisco AS5800 Access Servers - Practical Example.
Troubleshooting Flowchart
This document includes some flowcharts to assist in troubleshooting. Click on a particular box to go to the section describing the procedure. Also, proceed to the next flowchart by clicking on the numbers.
To Verify System Health Using the Onboard LEDs
The LEDs on various parts of the AS5800 platform can provide a wealth of information regarding the status of the different components. This section guides you through the procedure of using the LEDs to verify system health.
This diagram shows the various components of the AS5800:
Power-On the Router
Power-on these components in the order described below:
-
AC-Input Power Shelf: If you are using the AC-input power shelf, power-on ( | ) the two power switches located on the AC-input power shelf front panel. Four green power OK LEDs should light.
-
Dial-Shelf: Power-on ( | ) the power switches located on each dial shelf Power Entry Module (PEM) front panel. A green power LED on each PEM should light. You should hear the fans on the dial shelf power up.
Note: Always power-on the dial shelf before the router shelf. The Dial Shelf Controller (DSC) card checks the dial shelf's inventory, which requires extra time. After two minutes, power-on the router shelf. The router shelf depends on the DSC card for the dial shelf's inventory report.
-
Router-Shelf: Power-on ( | ) the two power switches located on the rear of the router shelf (7200). Both green power supply OK LEDs should light. You should hear the fans on the router shelf power up.
For more information on this procedure, or for diagrams identifying the various power switches described above, refer to the section Powering On the Cisco AS5800 in the document Powering On the Cisco AS5800 and Observing Initial Startup Conditions.
To Identify Hardware Subsystem Failures Using LEDs
You can use the various LEDs to determine whether any of the hardware subsystems failed during the boot sequence. The general flow of reasoning looks for the following (answering "Yes" advances to the next question):
-
Do the green Power OK LEDs illuminate on the AC-Input Power Shelf, Dial-Shelf and Router-Shelf?
To identify the various LEDs, refer to Powering On the Cisco AS5800. If any of the green Power OK LEDs do not come on, refer to the Troubleshooting the Power Subsystems section of the AS5800 Hardware Troubleshooting document.
-
Are the fans operating?
If the fans do not operate, refer to the Troubleshooting the Cooling Subsystems section of the AS5800 Hardware Troubleshooting document.
-
Do the Input/Output (I/O) LEDs on the router shelf (7206) light in the proper sequence?
The I/O LEDs should light in the following sequence. If they do not light as described, proceed to the Troubleshooting the Processor Subsystems section of the AS5800 Hardware Troubleshooting document.
-
The I/O power OK LED lights up immediately and indicates that the I/O controller card is receiving DC power from the router midplane. This LED lights up during a successful router boot and remains green during normal operation of the router.
-
The enabled LED lights up after the I/O power OK LED and indicates that the network processor card and the I/O controller card are enabled for operation by the system; however, that the enabled LED lights up does not mean that the Fast Ethernet port on the I/O controller card is functional or enabled. This LED lights up during a successful router boot and remains green during normal router operation.
-
The FE enabled LED lights up after the enabled LED and indicates that the Fast Ethernet port on the I/O controller card is initialized and enabled for operation by the system. This LED lights up during a successful router boot and remains green during normal router operation.
-
The FE link LED lights green only when the Fast Ethernet port on the I/O controller card is receiving a carrier signal from the network. (This LED remains off during normal router operation unless there is an incoming carrier signal and does not indicate startup problems.)
-
The slot 1 and slot 2 LEDs light up after the enabled LED and indicate which PCMCIA slot is in use.
During a successful router boot, these LEDs light up and remain on if a Flash memory card is present in the slot. These LEDs also blink when either slot is being accessed by the system. These LEDs do not indicate startup problems.
If the LEDs do not light in the above sequence, try reseating the I/O Controller Card and restarting. If this does not help, refer to the Troubleshooting the Processor Subsystems section of the AS5800 Hardware Troubleshooting document.
-
-
Do the port adapter-enabled LEDs (on the router shelf) come on?
The port adapter-enabled LEDs each light up when the network processor card completes its initialization of the port adapter for operation. The enabled LED indicates that the port adapter is receiving power and has been recognized by the network processor card; it does not indicate the state of the individual port adapter interfaces.
If an enabled LED does not light up, first try reseating the port adapters. If this does not help, refer to the Troubleshooting the Port Adapters section of the AS5800 Hardware Troubleshooting document.
-
Does the Dial Shelf Interconnect (DSI) port adapter Link LED (on the router-shelf) stay on?
The link status LED indicates an active connection to the dial shelf and lights up when the interconnect port adapter is receiving a carrier signal from the dial shelf controller card. This LED should light up and remain green when the access server is operating.
If the interconnect port adapter link LED does not light up, refer to the Troubleshooting the Dial Shelf Interconnect Port Adapter section of the AS5800 Hardware Troubleshooting document.
-
Do the LEDs on the Dial-Shelf light in the proper sequence?
The LEDs on the dial shelf controller card, modem cards, and trunk cards should light as follows:
-
Dial shelf controller card—Powers on first, followed by remaining logic cards. The PWR and MBus LEDs should light green. The remaining four alarm LEDs should all briefly flash on and then turn off.
If the MBus LED remains off, but the power LED is green, there might be a problem with the MBus power supply on the card. Replace the card.
-
Trunk card—At initial power on, all LEDs light up while the system runs a series of diagnostics. After the system passes initial diagnostics, all LEDs go off. The power, host CPU, and framer CPU LEDs then light green to indicate that the trunk card is operating properly.
-
Modem card—During normal operation, all five LEDs light at system power up. When the modem card CPU software image starts running, it shuts off the modem LED. The power, host CPU, and modem LEDs then light green to indicate that the modem card has passed diagnostics and is operating properly.
If the dial shelf processor component LEDs do not light as described, refer to theTroubleshooting the Dial Shelf Processor Subsystem section of the AS5800 Hardware Troubleshooting document.
-
The above steps address most common AS5800 hardware issues that are identifiable using LEDs. Continue to the next section for further troubleshooting.
Troubleshooting Flowchart (continued)
To Verify that the Dial-Shelf and Router-Shelf are running the same Cisco IOS Software Release
The dial shelf (DS) and router shelf (RS) run separate Cisco IOS images:
-
Both images MUST be from the same Cisco IOS software release. They must match.
-
The RS image is located in the Flash memory of the Cisco 7206. The filename begins with "c5800". The DS image is in the DSC card. The filename begins with "dsc."
Perform a show version on the RS to determine the Cisco IOS software version currently running. On the DS, you can also issue the show version command to check the Cisco IOS software version; however, if you do not have a physical console connection to the DS, issue the command execute-on slot [12 | 13] show version on the RS. The DSC can be in slot 12 or 13. Verify that the IOS versions on the DS and RS are the same. For more information, refer to section 1.2, "Matching the Cisco IOS Images", in the document Commissioning the Cisco AS5800 Hardware.
If the RS and DS run different image versions, some of the cards on the DS may not come up properly, or the RS may completely lose connectivity with the DS.
This example demonstrates a situation where the Cisco IOS software versions on the DS and RS are different. The router shelf is running Cisco IOS Software Release 12.2(23), and the dial shelf is running Cisco IOS Software Release 12.1(8). Begin by checking the Cisco IOS software version running on the Router-Shelf. Connect to the router shelf (using Console, Telnet, and so on) and issue these commands:
AS5800#show version Cisco Internetwork Operating System Software IOS (tm) 5800 Software (C5800-P4-M), Version 12.2(23), RELEASE SOFTWARE (fc2) !--- The router-shelf is running IOS Version 12.2(23) TAC:Home:SW:IOS:Specials for info Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Tue 04-Sep-01 17:45 by samuel Image text-base: 0x60008960, data-base: 0x61B40000 ROM: System Bootstrap, Version 12.0(19990210:195103) [12.0XE 105], DEVELOPMENT SOFTWARE BOOTFLASH: 7200 Software (C7200-BOOT-M), Version 12.0(5)T1, RELEASE SOFTWARE (fc1) c5800 uptime is 15 minutes System returned to ROM by power-on System image file is "slot0:c5800-p4-mz.122-23.bin" !--- The router-shelf image filename. Note that the filename begins with c5800. cisco 7206VXR (NPE300) processor (revision B) with 122880K/40960K bytes of memory. Processor board ID 13251981 R7000 CPU at 262Mhz, Implementation 39, Rev 1.0, 256KB L2, 2048KB L3 Cache 6 slot VXR midplane, Version 2.0 !--- Output suppressed
Determine whether the Dial-Shelf is located in slot 12 or 13. Issue the show dial-shelf command on the router-shelf.
AS5800#show dial-shelf
Slot Board CPU DRAM I/O Memory State Elapsed
Type Util Total (free) Total (free) Time
0 CT1 0( 0%) 0( 0%) Unknown 00:15:19
1 CT3 0( 0%) 0( 0%) Unknown 00:15:19
2 CT1 0( 0%) 0( 0%) Unknown 00:15:19
3 Modem(HMM) 0( 0%) 0( 0%) Unknown 00:15:19
5 Modem(DMM) 0( 0%) 0( 0%) Unknown 00:15:19
11 UP324 0( 0%) 0( 0%) Unknown 00:15:19
12 DSC 0%/0% 18470752( 82%) 8388608( 66%) Up 00:15:03
!--- The Dial Shelf Controller is in slot 12
AS5800#
Issue the show version command on the DS. You can either connect directly to the console port on the DS or you can issue the command execute-on slot <dsc_slot_number> show version on the RS. In this case, you have previously determined that the DSC is in slot 12.
c5800#execute-on slot 12 show version User Access Verification Password: Cisco Internetwork Operating System Software IOS (tm) 5800 Software (C5800-DSC-M), Version 12.1(8), RELEASE SOFTWARE (fc1) !--- The Dial-Shelf is running IOS Version 12.1(8) Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Tue 17-Apr-01 03:45 by kellythw Image text-base: 0x60008940, data-base: 0x605AC000 ROM: System Bootstrap, Version 11.2(19980113:222207) [tkam-nitro_ios_rommon_011398 555], INTERIM SOFTWARE ROM: 5800 Software (C5800-DSC-M), Version 11.3(2)AA, EARLY DEPLOYMENT, RELEASE SOFTWARE (fc1) DA-Slot12 uptime is 16 minutes System returned to ROM by power-on System image file is "slot0:dsc-c5800-mz.121-8" !--- The Dial-Shelf image filename. Note that the filename begins with dsc. !--- Output suppressed
Note: The image versions on the RS and the DS are not the same.
Now issue the show dsip version command on the router-shelf to check whether the router-shelf has proper connectivity to the dial-shelf using DSIP.
AS5800#show dsip version DSIP version information: ------------------------ Local DSIP major version = 5, minor version = 2 All feature boards are running DSIP versions compatible with router shelf Local clients registered versions: ------------------------------------ Client Name Major Version Minor Version !--- Output suppressed DSIP Test No version No version Mismatched remote client versions: !--- Remote client version mismatches exist ----------------------------------- Client Name Slot Major Version Minor Version Split DS 12 No version No version NextPort 12 No version No version
Note: The show dsip version command indicates that there are mismatched remote client versions. Ideally, you should not see any mismatches.
To Upgrade the IOS Image on the Router-Shelf, Dial-Shelf, or Both
You can upgrade the Cisco IOS image on the RS, the DS, or both. Follow the procedure described in these documents:
To Troubleshoot the Dial-Shelf Controller and Feature Boards
From the router-shelf, issue the show dial-shelf command to determine whether the DSC and associated feature boards are Up. In the example shown here, all feature boards are Up.
Router#show dial-shelf
Slot Board CPU DRAM I/O Memory State Elapsed
Type Util Total (free) Total (free) Time
0 CT3 0%/0% 21598976( 81%) 8388608( 41%) Up 00:01:35
2 Modem(DMM) 20%/20% 46764800( 86%) 16777216( 74%) Up 00:01:35
3 Modem(DMM) 0%/0% 46764800( 86%) 16777216( 74%) Up 00:01:35
4 Modem(DMM) 20%/20% 46764800( 86%) 16777216( 74%) Up 00:01:35
5 Modem(DMM) 20%/20% 46764800( 86%) 16777216( 74%) Up 00:01:35
6 Modem(DMM) 40%/40% 46764800( 86%) 16777216( 74%) Up 00:01:35
7 Modem(DMM) 40%/40% 46764800( 86%) 16777216( 74%) Up 00:01:35
8 Modem(DMM) 35%/35% 46764800( 86%) 16777216( 74%) Up 00:01:35
9 Modem(DMM) 0%/0% 46764800( 86%) 16777216( 74%) Up 00:01:35
10 Modem(DMM) 20%/20% 46764800( 86%) 16777216( 74%) Up 00:01:34
12 DSC 0%/0% 19097792( 79%) 8388608( 66%) Up 00:02:49
Dial shelf set for auto boot
Router#
If all feature boards are Up, then any hardware issue is not the result of a bootup problem. Proceed to the section Common Hardware Error Messages.
-
Dial-Shelf Controller is not Up
-
One or more Dial-Shelf feature boards are not Up
-
MICA/NextPort feature board is not Up
-
T1/E1/T3/E3 trunk line card is not coming Up
-
The Dial Shelf Controller is Not Up
Note: Follow the procedure described in the section Troubleshooting the Dial Shelf Controller Card in the document AS5800 Hardware Troubleshooting before continuing with this section.
If the process described in that document does not solve the problem, continue below:
-
Reload the DSC by issuing the command execute-on slot {12 | 13} reload.
-
Look for LED lights on the DSC card. If the lights are off, try re-seating the card. Refer to the section Troubleshooting the Dial Shelf Controller Card in the document AS5800 Hardware Troubleshooting for more information. Also, if the other DSC slot (12 or 13) is available, then move the DSC to the free slot.
-
Verify that the DSI port adapter on the router-shelf is inserted correctly. Refer to the section Identifying Hardware Subsystem Failures Using LEDs for more information.
-
Verify that the cable between the DSI port adapter and the DSC card is connected correctly. You can try replacing the DSI cable as well. For more information on attaching the DSI cable refer to the section Attaching the Dial Shelf Interconnect Cable in the document Cisco AS5800 Universal Access Server Dial Shelf Interconnect Port Adapter Installation and Replacement.
-
From the router-shelf (Cisco 7206), verify that the DSI-Fast Ethernet interface and line protocol are up:
AS5800>show dsi DSI-Fastethernet0/2/0 is up, line protocol is up Hardware is DEC21140A, address is 0030.f2f5.1438 (bia 0030.f2f5.1438) !--- Output suppressed
-
For advanced troubleshooting measures after the DSC card is up, open a virtual-console session to the DSC card (DA-Slot12) from the router shelf by issuing the command dsip console slave 12|13 . To end the session, press Ctrl+C three times:
AS5800#dsip console slave 12 Trying Dial shelf slot 12 ... Entering CONSOLE for slot 12 Type "^C^C^C" to end this session DA-Slot12> DA-Slot12# Terminate NIP IO session? [confirm] [Connection to Dial shelf slot 12 closed by local host] Router#
-
You can also try a dsip ping from the router shelf and see if the DSC responds. Issue the dsip ping destination slot_number command:
AS5800#dsip ping destination ? <0-13> remote slot number rs destination is RS AS5800#dsip ping destination 12 AS5800#!!!!!!!!
Tip: Once a dsip ping has been activated, you cannot stop it as you would a normal ICMP ping. Since the router prompt does not appear, you must blindly type dsip ping off to turn it off (as displayed in the example below):
AS5800#!!!!!!!d!!si!!p!! ping !!off
One or More Feature Boards are not Up
If one or more feature boards on the dial-shelf do not come up during the boot sequence, then you must delve into the specific module for further troubleshooting. However, you can perform some troubleshooting procedures regardless of the type of feature board not coming up. Follow the procedure below:
-
Verify that each feature board's MAC addresses are registered by DSIP. Unregistered boards cannot communicate with the system. Shelf 0 is the router shelf (master). Shelf 1 is the default dial shelf (slave). Issue the show dsip transport command to verify that the boards are registered by the system. For example:
c5800# show dsip transport DSIP transport statistics: IPC : input msgs=2753859, bytes=406835215; output msgs=2753733, bytes=156251079 total consumed ipc msgs=15316; total freed ipc msgs = 15316 transmit contexts in use = 18, free = 238, zombie = 0, invalid = 0 ipc getmsg failures = 0, ipc timeouts=0 core getbuffer failures=0, api getbuffer failures=0 dsip test msgs rcvd = 53, sent = 53 CNTL : input msgs=1040705, bytes=50803786; output msgs=679316, bytes=64974836 getbuffer failures=0 DATA : input msgs=27811, bytes=891245; output msgs=43985, bytes=675187 DSIP Private Buffer Pool Hits = 0 DSIP registered addresses: Shelf0 : Master: 00d0.06d3.0870, Status=local Shelf1 : Slot0 : 0010.7b03.a700, Status=remote Shelf1 : Slot1 : 0010.7b03.a708, Status=remote Shelf1 : Slot2 : 0010.7b03.a710, Status=remote Shelf1 : Slot3 : 0010.7b03.a718, Status=remote Shelf1 : Slot5 : 0010.7b03.a728, Status=remote Shelf1 : Slot11: 0010.7b03.a758, Status=remote Shelf1 : Slot12: 0010.7b03.a760, Status=remote c5800#In the above example, note that dial-shelf slots 0,1,2,3,5,11, and 12 have been registered by DSIP.
-
Perform an Online Insertion and Removal (OIR)/Hotswap using the command line. For more information, use the OIR procedure described in the section %DSIPPF-5-DS_KEEPALIVE_LOSS: DSIP Keepalive Loss from shelf <x> slot <y>.
-
Reseat the card or try the same card in another slot. If the card comes up in another slot, then the card is fine and the slot has gone bad.
-
Verify that all feature boards (cards) are running DSIP versions that are compatible with the router shelf. Issue the show dsip version command to determine this.
-
If the card does not come up for a long period, activate these debugs: debug dsip transport, debug dsip trace and debug dsip boot . The debug dsip transport command shows the registered MAC address sent from each feature board. The debug dsip trace command displays detailed DSIP hello and keepalive messages. The debug dsip boot command shows if the router shelf is sending the boot image to the feature boards.
-
Proceed to the troubleshooting section corresponding to the feature board in question.
MICA/NextPort Feature Board is not Up
If the MICA/NextPort feature board does not come up for an extended period of time, then follow the procedure described below:
-
Make sure you are running a Cisco IOS software version that supports Nextport or MICA cards. Use the Software Advisor tool (registered customers only) located on the Support Tools & Resources page to verify this.
-
The MICA HMM Modem (144 Modems) feature card must have at least 64 MB of DRAM if running Cisco IOS Software Release 12.0 or later. For more information, see Field Notice: AS5800 - 64MB DMM DRAM is Required for Running all Cisco IOS 12.0-Based Releases.
MICA HMM feature cards (72 Modems) require 32 MB of DRAM for Cisco IOS Software Release 12.0 and later. This is described in Field Notice: AS5800 - 32MB HMM DRAM Required to Run all Cisco IOS 12.0- Based Releases.
-
Perform OIR/Hotswap through the command line. Use the OIR procedure described in the section %DSIPPF-5-DS_KEEPALIVE_LOSS: DSIP Keepalive Loss from shelf <x> slot <y> for more information.
Issues Related to the MICA or NextPort Modem Card
If the MICA or NextPort Modem card is not recognized by the dial-shelf (using the command show dial-shelf) then the problem could be on the Modem Card or it could be a software or configuration issue related to the Modem Card. Consult these documents depending on your needs:
-
Managing Port Services on the Cisco AS5800 Universal Access Server
-
Troubleshooting MICA Modem Hardware issues on AS5200/AS5300
Note: Even though this document is not meant for the AS5800, many of the procedures can be used on this platform.
Note: Ensure that MICA modems are running portware version 2.7.3.0. If you are not running 2.7.3.0, upgrade the modem code.
T1/E1/T3 Trunk Cards are not Booting Up
Verify that the trunk card LEDs operate properly by observing these LED states:
-
The power LED is ON.
If the power LED remains OFF, verify that the card is seated properly.
If the power LED lights up on other trunk cards in the dial shelf, try inserting the trunk card in a different slot. If none of the power LEDs light up, check your dial shelf power connections, power entry modules, and AC-input power supplies (if present). Perform an OIR.
-
The HCPU LED is ON.
If the Host CPU (HCPU) LED is OFF but the power LED is ON, the software image might have failed to load onto the card. The dial shelf controller attempts to reload the software automatically. After a programmed number of attempts to reload the software image fails, the dial shelf controller powers OFF the trunk card and all LEDs shut off.
If this happens, assume that the failure is due to defective hardware. Contact Cisco Technical Support for further troubleshooting.
-
The FCPU LED is ON.
If the Framer CPU (FCPU) LED is OFF while the HCPU LED is ON, either the hardware is defective or the framer processor software has crashed. To determine if the failure is software related, wait while the auto-reload feature on the dial shelf controller card attempts to reload the software image. If the software fails to reload after the programmed number of times, assume that the failure is due to defective hardware. Contact the Cisco Technical Support for further troubleshooting.
References:
Issues Related to the T1/E1/T3 Trunk Card
If the T1/E1/T3 trunk line card is not recognized by the dial-shelf (using the show dial-shelf command), then the problem could be on the trunk card, or it could be a software or configuration issue related to the trunk card. Consult these documents depending on your needs:
Common Hardware Error Messages
This section provides information on common error messages associated with hardware issues. A troubleshooting procedure for each error is also included.
The Error Message Decoder tool (registered customers only) allows you to check the meaning of an error message. Error messages appear on the console of Cisco products, usually in this form:
%XXX-n-YYYY : [text]
This is an example error message:
%TRUNK-3-NOMSGS: Cannot send message to framer; no more messages available
Some error messages are informational only, while others indicate hardware or software failures and require action. The Error Message Decoder tool (registered customers only) provides an explanation of the message, a recommended action (if needed), and if available, a link to a document that provides extensive troubleshooting information about that error message.
For a comprehensive list of error messages, refer to the document System Error Messages for the Cisco AS5800 Universal Access Server.
%DEC21140-5-NOCARR: DSI-Fastethernet 0/2/0 Cable/Connector problem?
This error message indicates that the router-shelf has lost connectivity to the dial-shelf, through the Dial Shelf Interconnect (DSI) cable, connected to port 0/2/0 on the router-shelf. Issue the show dsi command to verify that the connection between the router-shelf and dial-shelf is up. Verify that both the interface and the line protocol are in an UP state. For example:
AS5800#show dsi DSI-Fastethernet0/2/0 is up, line protocol is up !--- Interface is UP/UP Hardware is DEC21140A, address is 00d0.d342.4c38 (bia 00d0.d342.4c38) !--- Output suppressed
Perform these steps if the connection between the router-shelf and dial-shelf is not in the UP state:
-
If the DSI-Fastethernet interface is not UP, perform a no shutdown on the interface.
-
Check whether the dial-shelf interconnect port adapter-enabled LED is on.
If it is not, verify that the interconnect port adapter is seated properly and that the port adapter levers are in the locked position. You can also try to reseat the port adapter.
If power is supplied to the router-shelf, but the enabled LED remains off, the system has detected a hardware failure. You need to replace the interconnect port adapter.
-
Verify that the link LED on the DSI Port Adapter is on.
The link LED indicates an active connection to the dial-shelf. This LED lights when the dial-shelf interconnect port adapter is receiving a carrier signal from the dial-shelf.
If the link LED remains off, check the Dial-Shelf Interconnect Cable (DSIC) connection and tighten the jack screws at both ends of the cable.
-
Replace the DSI Cable (Part number CAB-DSIC-5= or CAB-DSIC-20=). Make sure that the new cable is properly secured.
The dial-shelf interconnect port adapter contains a single RJ-45 receptacle, which is used to connect the router-shelf to the dial-shelf. The cable used for this connection is a Cisco proprietary cable, customized with jackscrews to secure the connection. You must use this specially designed cable that shipped with your dial-shelf interconnect port adapter to connect the dial-shelf to the router-shelf.
For more information on attaching the DSI Cable, refer to the section Attaching the Dial Shelf Interconnect Cable in the document Cisco AS5800 Universal Access Server Dial Shelf Interconnect Port Adapter Installation and Replacement.
-
Troubleshoot the dial-shelf. Proceed to the section Troubleshooting the Dial Shelf Controller Card in the document Hardware Troubleshooting.
%DSCCLOCK-3-FAIL: The System Primary Clock is Down. Moving to HOLDOVER STATE and Waiting to See if it Comes Up
The DSC Clock Synchronization Circuit (DCSC), normally locks on to a clock derived from one of the incoming trunk ports either based on user configuration or using a default mechanism. This selected clock acts as the TDM primary clock. If this primary clock fails, the DCSC moves from NORMAL to HOLDOVER state.
In HOLDOVER mode, DCSC still provides timing and synchronization signals, which are not locked to an external reference signal, but are based on storage techniques. The storage value is determined while the device is in Normal mode and locked to an external reference signal. Note that the accuracy of the TDM clock derived while the DCSC is in HOLDOVER mode decreases, and slips occur eventually if the problem is not rectified.
Normally, on the AS5800, with two trunks installed and at least two ports connected to an external network, there exist one primary clock and three backup clocks. So, when a primary fails, the DCSC immediately selects a backup as a new primary.
The DSC primary can enter holdover mode for one of these reasons:
-
The trunk card providing the primary has crashed.
-
The port providing the primary, has gone bad or cable has been removed.
-
The clock that is being derived from the trunk port has gone bad.
-
When the router-shelf reloads, it reloads all feature boards, and a message appears on the DSC console indicating that the primary clock has gone into holdover mode.
Use this procedure to isolate the issue:
-
Issue the show controllers {t1 | e1} command to check whether the T1s are experiencing errors. If the T1 is not clean, refer to the T1 Troubleshooting Flowchart. You may also have to contact your telco for further T1 line troubleshooting. Ensure that the T1 lines are clean and stable before proceeding further.
-
Reseat the CE1/T1/T3 trunk cards (feature boards) on the Dial-Shelf. Use the procedure described in the document Replacing or Installing Dial Shelf Cards.
Warning: This causes all calls through that card to be dropped. -
Try moving the T1/E1 line in question to a different port on the same card. This can help isolate problems associated with a specific port on the T1/E1 trunk card.
-
Follow the instructions below to configure the AS5800 to set the primary clock source as a different T1/E1 line. This can help to overcome clocking and slip problems, commonly seen when using T1s from multiple vendors into the same unit. Instructions
For more information on clocking, refer to the document Clock Synchronization for AS5xxx Network Access Servers.
-
Issue the show dial-shelf clocks command to determine the current primary and secondary clock sources. This output shows a good Primary and Secondary (Backup) Clock Source:
AS5800# show dial-shelf clocks Primary Clock: -------------- Slot 12: System primary is 1/3/1 of priority 3 !--- The primary clock is T1 1 in slot 3 TDM Bus Master Clock Generator State = NORMAL !--- Clock state is NORMAL. This is the desired state Backup clocks: Source Slot Port Priority Status State ------------------------------------------------------- Trunk 1 2 10 Good Configured !--- The Backup clock source is T1 2 in slot 1 Status of trunk clocks: ----------------------- Slot Type 11 10 9 8 7 6 5 4 3 2 1 0 1 T1 B B B B B B B B B G B B 3 T1 B B B B B B B B B B G B AS5800#
This output shows a bad Primary Clock Source. A Secondary (Backup) clock source is not configured:
AS5800# show dial-shelf clocks Slot 12: System primary is 6/76/0 of priority 76 TDM Bus Master Clock Generator State = HOLDOVER !--- Clock state is HOLDOVER. This is NOT Desired. Backup clocks: Source Slot Port Priority Status State ------------------------------------------------------- !--- No Backup clocks are configured Slot Type 11 10 9 8 7 6 5 4 3 2 1 0 0 E1 B B B B B B B B B B B B
-
Issue the dial-tdm-clock command to configure a T1 as the primary clock source. Refer to the section Configuring Clocking in the document Configuring Your Cisco AS5800 Universal Access Server.
-
%DSIPPF-5-DS_KEEPALIVE_LOSS: DSIP Keepalive Loss From Shelf <x> slot <y>
This indicates that the router-shelf has not received a scheduled keepalive message from slot y on the dial-shelf x. The default value for x is 1 if there is only one DSC. Perform these steps to troubleshoot:
-
Issue the show dial-shelf command to verify that the feature board (card) in slot y is UP. If the Board in slot y is up, then this error could be caused by a faulty DSIC cable. Replace the cable (Part number CAB-DSIC-5= or CAB-DSIC-20=) and check if the error message disappears. Make sure that the new cable is properly secured.
For more information on attaching the DSI Cable refer to the section Attaching the Dial Shelf Interconnect Cable in the document Cisco AS5800 Universal Access Server Dial Shelf Interconnect Port Adapter Installation and Replacement.
-
Perform an OIR procedure on the affected feature board in slot y. This effectively resets the affected feature board.
-
Execute the show dial-shelf slot slot_number command from the enable prompt on the RS and note down the numeric value associated with the type of the FB (in slot y) you are trying to reset. In this sample output, the Board Type for slot 3 is 259:
AS5800#show dial-shelf slot 3 Slot: 3, Type: Channelised T3 (259) !--- Output suppressed
-
Open a virtual connection to the DSC using the command dsip console slave dsc_slot_number. The value for dsc_slot_number should be either 12 or 13 depending on the slot number of the DSC that the affected feature board belongs to. For example:
AS5800#dsip console slave 12 Trying Dial shelf slot 12 ... Entering CONSOLE for slot 12 Type "^C^C^C" to end this session -
Issue the command oir testport enable to enable OIR simulation.
AS5800-Slot12#oir testport enable
-
Issue the command oir slot slot_number remove to remove the card using the software. The value for slot_number should be the slot number of the feature board you want to OIR (slot y in the error message above).
AS5800-Slot12#oir slot 3 remove AS5800-Slot12# 08:45:53: %OIR-6-REMCARD: Card removed from slot 3, interfaces disabled -
To re-insert the card back, issue the command oir slot slot_number insert <board type from step (a)> .
AS5800-Slot12#oir slot 3 insert 259 AS5800-Slot12# 08:46:00: %OIR-6-INSCARD: Card inserted in slot 3, interfaces administratively shut down -
To check the OIR events, issue the oir log command:
AS5800-Slot12#oir log Slot 3: CT3 card inserted
-
-
Physically remove and reseat the feature board that is experiencing errors. Refer to the document Replacing or Installing Dial Shelf Cards for more information.
-
Remove and insert the feature board in another empty slot. Keep in mind that T3/T1/E3/E1 trunk cards can only be installed in slots 0 though 5 (slots 0 and 1 are recommended). Modem feature boards can only be installed in slots 0 through 11.
%ENVM-4-ENVWARN chassis outlet 3 measured at 50C/122F
This error message indicates that the router has detected an abnormally high chassis temperature. Verify that all three fans are operational and that ventilation in the facility is adequate. Refer to Troubleshooting the Cooling Subsystems for more information on troubleshooting the fans. You should consider powering down the AS5800 to prevent hardware damage.
%DSC_ENV-1-CRITICAL: Slot 24, 48V supply outside critical range Value=79 V; Power supply may shut itself off
This error message indicates a problem with the power supply subsystem. Verify that the input cable from the AC to DC Supply Shelf is wired correctly. Consult a qualified electrician, if necessary.
You should also check whether the bus voltage at the connector is -48Volts. If the input voltage is not 48Volts, then there is a power supply issue. You can measure the voltage using a Digital Volt Meter at the PEM input connector.
Tip: You can use the show environment all command to verify that there are no critical grounding, heating, or power problems.
This example shows a normal operating environment:
AS5800>show environment all
Power Supplies:
Power supply 1 is empty.
Power supply 2 is Zytek AC Power Supply. Unit is on.
Temperature readings:
chassis inlet measured at 25C/77F
chassis outlet 1 measured at 27C/80F
chassis outlet 2 measured at 33C/91F
chassis outlet 3 measured at 41C/105F
Voltage readings:
+3.45 V measured at +3.49 V
+5.15 V measured at +5.21 V
+12.15 measured at +12.34 V
-11.95 measured at -11.81 V
Envm stats saved 1 time(s) since reload
AS5800>
Refer to the Troubleshooting the Power Subsystems in the document Hardware Troubleshooting for more information.
%DSC_ENV-0-CRITICAL_BLOWER: Slot 28, Fans in the blower unit have failed. Operation of unit may lead to cards overheating and being shutdown
The above error message can be generated due to one of these reasons:
-
The fan actually has failed inside the fan blower unit, which means that the fan tray has to be replaced. Refer to the section Replacing the Blower Assembly in the document AS5800 Maintenance for more information on the replacement procedure.
-
The Mbus module that resides in the EMI filter module is defective and needs to be replaced.
As far as cooling goes, if you start to see warning messages from each card stating that the card is going overtemp, then the cards automatically power down in minutes if proper cooling is not restored.
Try this procedure:
-
Swap the fan tray, and see if the problem follows the tray on another system.
-
If the error message does not move with the fan tray, then the Filter module must be pulled and replaced.
Summary
Once you are able to identify the piece of hardware that is causing the issue, open a service request online with the Cisco Technical Support and request a Return Materials Authorization (RMA) for the part that is causing the issue if the router is covered by a valid warranty or service contract. Make sure you follow the procedure described in the document Cisco IOS Software Commands for Cisco AS5800 Hardware Inspection before calling the Cisco Technical Support.
If you were not successful in identifying which piece of hardware is causing the issue, open a service request with the Cisco Technical Support, attach all relevant information such as troubleshooting logs and troubleshooting steps taken, and an engineer can assist you in further troubleshooting the issue.
Related Information
- Field Notice: AS5800 - 32MB HMM DRAM Required to Run all Cisco IOS 12.0-Based Releases
- Field Notice: AS5800 - 64MB DMM DRAM is Required for Running all Cisco IOS 12.0-Based Releases
- Cisco AS5800 Operations, Administration, Maintenance, and Provisioning Guide
- Powering On the Cisco AS5800 and Observing Initial Startup Conditions
- Configuring Your Cisco AS5800 Universal Access Server
- AS5800 Hardware Troubleshooting
- Cisco AS5800 Product Overview
- Hardware Troubleshooting for the Cisco 7200 Series Router
- Technical Support & Documentation - Cisco Systems
Open a Support Case 
(Requires a Cisco Service Contract.)
Related Cisco Support Community Discussions
The Cisco Support Community is a forum for you to ask and answer questions, share suggestions, and collaborate with your peers.
Refer to Cisco Technical Tips Conventions for information on conventions used in this document.