Cisco 6400 Series Broadband Aggregators

Document ID: 17822

Contents

Introduction
Prerequisites
      Requirements
      Components Used
      Conventions
Identifying Cisco 6400 Hardware Issues - Chassis
      Blower Module
      Power Entry Module
Identifying 6400 Hardware Issues - Node Switch Processor
      Useful Information
      NSP Bootup Issues
      NSP Reboot/Reload
      NSP Stops Responding
Identifying 6400 Hardware Issues - Node Route Processor
      NRP-2SV Module
      Methods Available for Gaining Access to the NRP-2
      NRP-1 Module
      Useful Information
      NRP Bootup Issues
      NRP Reboot/Reload
      NRP Stops Responding
Identifying 6400 Hardware Issues - Node Line Cards
Related Information

Introduction

The Cisco 6400 is a broadband concentrator that features Cisco end-to-end Asynchronous Transfer Mode (ATM) services, Point-to-Point Protocol (PPP) termination, and tunneling. The Cisco 6400 combines Cisco ATM switching and routing capabilities in a modular, scalable, redundant chassis. This document does not discuss any architectural overview of the 6400 chassis or its parts. It discusses basic dispalyed LED indications, how to troubleshoot IOS reboot problems, and console hangs; but not hardware architecture details. If you want a more detailed discussion about 6400 (configuration, installation, release notes, command line), please refer to Cisco 6400 Carrier-Class Broadband Aggregator . Also For quick referencing for error messages and their meaning on 6400, please read Error Messages on the Cisco 6400 UAC Descriptions and Resolutions. For more detailed descriptions and technical discussions, see the useful links below under Related Information section.

The Cisco 6400 includes the following hardware components:

• Chassis

  • Module compartment

  • Blower compartment

  • Power Entry Module (PEM) compartment

  • Blackplane

• Node Switch Processor (NSP)

• Node Route Processor (NRP)

• Node Line Card (NLC)

The following diagram shows a fully-loaded chassis with Blower compartment, PEMs, NSPs, NRPs and NLCs. The Module compartment in the chassis has ten slots—two central slots that are reserved for NSP cards and eight slots that can accommodate a mixture of full-height NRP cards and full-height and half-height NLCs.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the Cisco 6400 Universal Access Gateway.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

Identifying Cisco 6400 Hardware Issues - Chassis

Blower Module

The Blower module is located on the top of the chassis and is connected to the backplane. The module provides airflow throughout the system. The air intake is located at the bottom of the chassis, and there is an exhaust in the top rear of the chassis through the Blower module.

The Blower module supports hot-swapping and can be replaced without interrupting system operation. It is important to note that the system will shut down after two minutes if the Blower module is removed and not replaced.

Use the show environment command on the NSP to display information about fan operation and temperature levels.

Switch#show environment
Temperature normal: chassis inlet measured at 27C/80F
Temperature normal: chassis core measured at 33C/91F
Fan:                                            OK
AC Power Entry Module 0 status:                 OK
Switch#

When a single fan fails, replacing the Blower module is recommended. If several fans fail, replacing the Blower module is required. The following table shows blower module LEDs.

Table 1

Power Entry Module

The DC PEM shown above provides filtering and supplies DC power to the chassis electronics. Two PEM bays are in the chassis but only one PEM is required. An additional PEM can be used for redundancy. The PEMs are located on the left side of the chassis and are installed from the front. Input source power (-48 VDC from a building centralized power source) enters the chassis through screw terminals on the rear of the chassis and is supplied to the PEMs when they are plugged into the backplane.

The AC PEM shown above provides power conversion directly from the facility VAC input power to the -48 VDC used internally in the Cisco 6400 chassis. AC power goes into the AC PEM through a power cord attached to the front faceplate.

Issue the show environment command on the NSP to display information about the PEM operation.

Switch#show environment
Temperature normal: chassis inlet measured at 27C/80F
Temperature normal: chassis core measured at 33C/91F
Fan:                                            OK
AC Power Entry Module 0 status:                 OK
Switch#

When a PEM fails and a redundant PEM is available in the chassis, the redundant PEM will start and continuously provide power to the Cisco 6400. Replacing the PEM is recommended. If there is not a redundant PEM, replacing the PEM is required.

Table 2 shows DC power entry module LEDs.

Table 2

Table 3 shows AC power entry module LEDs.

Table 3

Identifying 6400 Hardware Issues - Node Switch Processor

The NSP is the main system processor card in the Cisco 6400 and resides in chassis slot 0A or 0B. The NSP contains the ATM switch engine and processor, and most memory components. The NSP runs the system software that maintains and executes the management functions that control the system.

Table 4 shows NSP indicators.

Table 4

Useful Information

The following information is essential for determining the cause of most NSP problems.

• Outputs of the show tech-support command.

• Console logs.

• Syslog information.

• Crashinfo file.

• Complete bootup sequence if the router experiences boot errors.

Capturing information from the console of the NSP is essential for troubleshooting NSP problems. Log the console output in a file for later analysis or for use by the Cisco Technical Assistance Center (TAC), if a TAC case is opened.

NSP Bootup Issues

If the STATUS or the FAIL LED on the NSP remains yellow (does not change to green), the first step in determining the cause of the problem is to connect to the console and see if you can get a router prompt. For information about terminal settings of a console connection, see the Troubleshooting section of the Cisco 6400 UAC Hardware Installation and Maintenance Guide.

If the NSP is locked in ROM monitor (ROMMON) mode with a rommon prompt, set the configuration register to 0x2102 and reload the NSP.

rommon 1>confreg 0x2102
rommon 2>reset

If the NSP is still in ROMMON mode after it was reloaded, use the following recovery procedure.

1. Issue the dev command to determine which devices are available on the NSP.

   rommon1>dev 
   Devices in device table: 
            id  name 
    bootflash:  boot flash 
        slot0:  PCMCIA slot 0
        slot1:  PCMCIA slot 1
        disk0:  PCMCIA slot 0
        disk1:  PCMCIA slot 1
       eprom:   eprom
    rommon 2>

2. Issue the dir <device_id> command to search for a valid Cisco IOS software image.

   rommon 2>dir disk0:
   Directory of disk0:
   
   3     0         drw-     images
   4     0         drw-     slot1
   5     0         drw-     slot2
   6     0         drw-     slot3
   7     0         drw-     slot4
   8     0         drw-     slot5
   9     0         drw-     slot6
   10    0         drw-     slot7
   11    0         drw-     slot8
   8283  5025140   -rw-     c6400s-wp-mz_121-5_DB.bin
   13298 5289460   -rw-     c6400s-wp-mz.122-2.B1.bin
   4287  0         drw-     non-mirror
   rommon 3>

3. Boot from the image you selected.

   rommon3>boot disk0:c6400s-wp-mz.122-2.B1.bin 
   Self decompressing the image : ################################################ 
   ###############################################################################
   ###############################################################################
   ############################################################################### 
   ############################################################################### 
   ###############################################################################
   ############################################################################### 
   [OK]

NSP Reboot/Reload

Spontaneous NSP reboots/reloads or failures can be caused by either software or hardware. Only failures caused by hardware are discussed in this document. The crash info and console logs are required to understand whether the problem is hardware or software related.

Parity errors and bus errors are examples of failures that can be caused by hardware:

• Parity Errors – These can be caused by hardware failures or single event upsets. Please review Processor Memory Parity Errors (PMPEs) to help determine whether the problem is caused by a single event upset or a hardware failure.

• Bus Errors – These can be caused by software or hardware. Please review Troubleshooting Bus Error Crashes to help determine whether the problem is caused by hardware or software.

NSP Stops Responding

The console will not respond if the NSP stops responding. This can be caused by a failure in hardware, memory, software, or by a high CPU utilization. Please see Troubleshooting Router Hangs for more information.

Identifying 6400 Hardware Issues - Node Route Processor

The NRP sends and receives traffic between one or more NSP ATM switch ports. The Cisco 6400 can contain multiple NRP modules configured to operate independently or as redundant pairs (1+1). You can insert NRPs into slots 1 through 8 in the Cisco 6400 chassis.

The Cisco 6400 supports two NRPs, designated as NRP-2SV and NRP-1. These can be used together in any combination in a single Cisco 6400 chassis.

NRP-2SV Module

The NRP-2SV provides a Gigabit Ethernet interface and sufficient processing capability for handling an OC-12 rate of user traffic.

Table 5 shows NRP-2SV LED indicators.

Table 5

Methods Available for Gaining Access to the NRP-2

There are two methods available for accessing the NRP-2:

• Accessing the NRP-2 Console Through the NSP

• Using Telnet to Connect to the NRP-2 from the NSP

You can also configure the NRP-2 with the Cisco 6400 Service Connection Manager, Release 2.2(1) and later. For more information, refer to Cisco 6400 SCM .

Accessing the NRP-2 Console Through the NSP

The NSP is equipped with an internal communication server for accessing the NRP-2 console line. To access the NRP-2 console line, use Telnet to connect to the NSP as a communication server, using the port numbers shown in Table 6 below to select the NRP-2.

Table 6

To exit the NRP-2 console line without closing the console connection, use the escape sequence Ctrl-Shift-6. To close the NRP-2 console line connection, use the exit command.

The example procedure below illustrates this further.

1. Suppose the NSP in your Cisco 6400 system has the management IP address 10.1.5.4. To access the console line of the NRP-2 in slot 6 of the same Cisco 6400 chassis, use the telnet command from another router:

   device#telnet 10.1.5.4 2006 
   Trying 10.1.5.4, 2006 ... Open 
   NRP-2# 

2. To return to the device prompt without closing the NRP-2 console line connection, enter the escape sequence Ctrl-Shift-6 at the NRP-2 prompt. Notice that the full escape sequence does not appear as you enter it in the command-line interface (CLI):

   NRP-2# Ctrl^ x 
   device# 
   

3. To return to the connected NRP-2 console line, hit Enter at the device prompt.

   device# 
   [Resuming connection 1 to 10.1.5.4 ... ] 
   NRP-2# 

4. To close the NRP-2 console line connection, use the escape sequence to return to the device prompt, and then use the exit command.

   NRP-2#Ctrl^ 
   device#exit 
   (You have open connections) [confirm] 
   Closing:10.1.5.4 ! 
   device con0 is now available 
   Press RETURN to get started. 
   device> 

Using Telnet to Connect to the NRP-2 from the NSP

The NSP is equipped with command aliases for using Telnet to connect to an NRP-2 in the same Cisco 6400 chassis. To use Telnet to connect to the NRP-2, use the following NSP command alias in EXEC mode.

Switch#nrpsslot 

Note: Set the enable password for the NSP before you use Telnet to connect to the NRP-2.

To exit the NRP-2 VTY line without closing the Telnet session, use the escape sequence Ctrl-Shift-6. To close the NRP-2 Telnet session, use the exit command.

The example procedure below illustrates this further.

1. Suppose you want to use Telnet to connect to the NRP-2 from a device outside your Cisco 6400 system, and the NSP in the Cisco 6400 has the management IP address 10.1.5.4. To use Telnet to connect to the NRP-2, first connect to the NSP, and then use the nrps command alias to connect to the NRP-2.

   device#telnet 10.1.5.4 
   Trying 10.1.5.4 ... Open 
   User Access Verification 
   Password: 
   NSP> 
   NSP>nrps6 
   Trying 10.6.0.2 ... Open 
   NRP-2> 

2. To close the Telnet session to the NRP-2 and return to the NSP prompt, use the exit command.

   NRP-2>exit 
   [Connection to 10.6.0.2 closed by foreign host] 
   NSP>

NRP-1 Module

The NRP-1 incorporates a 100 Mbps Fast Ethernet interface for connecting into an IP network and has processing capability for an OC-3 rate of user traffic.

Table 7 shows NRP-1 LED indicators.

Table 7

Useful Information

The following information is essential for determining the cause of most NRP problems.

• Outputs of the show tech-support command.

• Console logs.

• Syslog information.

• Crashinfo file.

• Complete bootup sequence if the router experiences boot errors.

Capturing information from the console of the NRP is essential for troubleshooting NRP problems. Log the console output in a file for later analysis or for use by the Cisco TAC, if a TAC case is opened.

NRP Bootup Issues

If the STATUS or the FAIL LED on the NSP remains yellow (does not change to green), the first step in determining the cause of the problem is to connect to the console and see if you can get a router prompt. For information about terminal settings of a console connection, see the Troubleshooting section of the Cisco 6400 UAC Hardware Installation and Maintenance Guide.

If the NRP is locked in ROMMON mode with a rommon prompt, set the configuration register to 0x2102 and reload the NRP.

rommon 1>confreg 0x2102
rommon 2>reset

If the NRP is still in ROMMON mode after it has been reloaded, use the following recovery procedure.

1. Issue the dev command to determine which devices are available on the NRP.

   rommon 1 >dev
   Devices in device table:
           id  name
       flash:  flash 
   bootflash:  boot flash
       eprom:  eprom
   rommon 2>

2. Issue the dir <device_id> command to look for a valid Cisco IOS software image.

   rommon 2>dir flash:
           File size           Checksum      File name
   5545532 bytes (0x549e3c)    0xf1cbd97e    c6400r-g4p5-mz.121-5.DC1
   rommon 3>

3. Boot from the image you selected.

   rommon 3>boot flash:c6400r-g4p5-mz.121-5.DC1 
   Self decompressing the image : ################################################ 
   ###############################################################################
   ###############################################################################
   ############################################################################### 
   ############################################################################### 
   ###############################################################################
   ############################################################################### 
   [OK]

NRP Reboot/Reload

Spontaneous NRP reboots/reloads or failures can be caused by either software or hardware. Only failures caused by hardware are discussed in this document. The crashinfo and console logs are required to understand whether the problem is hardware or software related.

Parity errors and bus errors are examples of failures that can be caused by hardware.

• Parity Errors – These can be caused by hardware failures or single event upsets. Please review Processor Memory Parity Errors (PMPEs) to help determine whether the problem is caused by a single event upset or a hardware failure.

• Bus Errors – These can be caused by software or hardware. Please review Troubleshooting Bus Error Crashes to help determine whether the problem is caused by hardware or software.

NRP Stops Responding

The console will not respond if the NRP stops responding. This can be caused by a failure in hardware, memory, software, or by a high CPU utilization. Please see Troubleshooting Router Hangs for more information.

Identifying 6400 Hardware Issues - Node Line Cards

The OC-3/STM-1 SM half-height NLC has two 155 Mbps fiber-optic ports for a single-mode intermediate reach connection of uplink and downlink interfaces.

The OC-3/STM-1 MM half-height NLC has two 155 Mbps fiber-optic ports for a multimode connection on the front of each NLC.

The DS3 half-height NLC has two 45 Mbps bidirectional ports for a connection to network services using coaxial cable.

Table 8 shows OC-3 and DS3 NLC LED indicators.

Table 8

The OC-12/STM-4 full height NLC has one 622 Mbps fiber-optic port for the connection of uplink and downlink interfaces; single mode, intermediate reach.

Table 9 shows OC-12 NLC LED indicators.

Table 9

The Cisco 6400 can contain multiple NLC modules, configured to operate independently or as redundant pairs. NLCs can be inserted into slots 1 through 8 (subslots 0 and 1) in the Cisco 6400 chassis.

To show the NLC modules in the Cisco 6400, issue the show hardware command on the NSP.

Switch#show hardware
6400 named Switch, Date: 12:41:06 UTC Wed Dec 19 2001
Feature Card's FPGA Download Version: 0
Slot  Ctrlr-Type    Part No.   Rev  Ser No  Mfg Date   RMA No.  Hw Vrs Tst EEP
----- ------------  ---------- --  -------- ---------  -------- ------ --- ---
1/0  NRP            73-3082-02 ??  09694065 Jan 00 00  00-00-00   2.20  FF   0
2/0  NRP            73-2342-03 F0  11667429 Feb 26 99  00-00-00   1.0    0   2
3/0  NRP            73-2342-03 F0  11667428 Feb 27 99  00-00-00   1.0    0   2
4/0  NRP2           73-4902-02 C0  25389496 Feb 28 01  00-00-00   1.0    0   2
5/0 155SM NLC       73-2892-03 01  09156343 Sep 09 98  00-00-00   0.0    0  FF
6/0 155SM NLC       73-2892-03 01  09156410 Sep 09 98  00-00-00   3.1    0  FF
7/0 155MM NLC       73-5824-02 A0  25388608 Feb 24 01  00-00-00   1.0    0   2
8/0 622SM NLC       73-3868-02 A0  14342212 Jul 15 99  00-00-00   1.0    0   2
5/1 DS3 NLC         73-3696-02 02  11555742 Feb 04 99  00-00-00   1.0    0   2
6/1 DS3 NLC         73-3696-02 A0  16088970 Dec 23 99  00-00-00   1.0    0   2
7/1 155SM NLC       73-2892-03 01  11671065 Feb 09 99  00-00-00   1.0    0   2
0A/PC NSP-PC        73-2996-06 A0  15792098 Dec 30 99  00-00-00   1.0    0   2
0A/FC FC-PFQ        73-2281-04 B0  15792035 Dec 30 99  00-00-00   4.1    0   2
0A/SC NSP-SC        73-2997-06 A0  15791346 Dec 30 99  00-00-00   1.0    0   2
0B/PC NSP-PC        73-2996-06 A0  15792204 Dec 28 99  00-00-00   1.0    0   2
0B/FC FC-PFQ        73-2281-04 B0  15791847 Dec 28 99  00-00-00   4.1    0   2
0B/SC NSP-SC        73-2997-06 A0  15791341 Dec 29 99  00-00-00   1.0    0   2

Primary NSP:Slot 0A

DS1201 Backplane EEPROM:

 Model   Ver.  Serial  MAC-Address  MAC-Size  RMA  RMA-Number   MFG-Date
------- ----  -------- ------------ --------  ---  ----------  -----------
C6400    2    10036146 00107BB9BD00   128      0   0           Sep 16 1998

Switch#

To determine if a NLC module is operational, issue the show hw-module subslot <slot>/<subslot> command on the NSP.

Switch#show hw-module subslot 8/0

Slot  Ctrlr-Type            Comment
----  ----------  ---------------------------------
8/0    622SM NLC   State = Operational

Switch#

Related Information

• Cisco 6400 Carrier-Class Broadband Aggregator
• Aggregation / DSLAM (DSL Access Multiplexer)
• Cisco 6400 Feature Guide
• Cisco 6400 NRP Configuration and Troubleshooting
• Error Messages on the Cisco 6400 UAC:Descriptions and Resolutions
• Cisco 6400 Universal Access Concentrator Frequently Asked Questions
• Cisco 6400 Node Line Card Frequently Asked Questions
• Cisco 6400 Node Route Processor Frequently Asked Questions
• Cisco 6400 Node Switch Processor Frequently Asked Questions
• Cisco 6400 Carrier-Class Broadband Aggregator Release Notes
• Cisco IOS Error Message Decoder (registered customers only)
• Cisco IOS Output Interpreter (registered customers only)
• DSL Technical Support
• Technical Support - Cisco Systems