Cisco IOS XR Troubleshooting Guide for the Cisco ASR 9000 Aggregation Services Router - General Troubleshooting [Cisco IOS XR Software (End-of-Sale)]

Table Of Contents

General Troubleshooting Procedures

Prerequisite Documentation for Troubleshooting

Verifying and Troubleshooting CLI Access

General CLI Access Information

User Access Privileges

Cisco-support Task ID

CLI Access Through a Console Port

CLI Access Through a Terminal Server

CLI Access Through the Management Ethernet Interface

Validating and Troubleshooting Installation of the Cisco IOS XR Software Package

Verifying the Software Version

Validating the Installation

install verify Command

show install active Command

show install committed Command

Validating and Troubleshooting Cisco IOS XR Software Configuration

Local and Global Configurations

Collecting Configuration Information

Verifying the Running Configuration

Using the show configuration failed Command

Startup Failed Configuration

Commit Configuration Failed

Verifying the System

Troubleshooting the Backplane Ethernet Control System

Basic Cisco IOS XR Verification and Troubleshooting Commands

man Command

describe Command

show platform Command

top Command

show context Command

show users Command

show history Command

show configuration Command

Displaying ASIC Errors

Using Trace Commands

MIB Location

Gathering Information Before You Call Cisco TAC

Gathering Information about Crashes and Core Dumps

Capturing Logs

Using Debug Commands

Using Diagnostic Commands

Commands Used to Display Process and Thread Details

General Troubleshooting Procedures

This chapter describes general troubleshooting techniques you can use to troubleshoot the Cisco ASR 9000 Aggregation Series Router. It includes the following sections:

•Prerequisite Documentation for Troubleshooting

•Verifying and Troubleshooting CLI Access

•Validating and Troubleshooting Installation of the Cisco IOS XR Software Package

•Validating and Troubleshooting Cisco IOS XR Software Configuration

•Verifying the System

•Troubleshooting the Backplane Ethernet Control System

•Basic Cisco IOS XR Verification and Troubleshooting Commands

•Displaying ASIC Errors

•Using Trace Commands

•MIB Location

•Gathering Information Before You Call Cisco TAC

Prerequisite Documentation for Troubleshooting

As a starting point for troubleshooting, we strongly recommend that you have a system of maintaining and accessing detailed information about your network and ASR 9000 router. This should include:

•Current documentation about the system, including chassis numbers, serial numbers, installed cards, and location of chassis details.

•Diagrams illustrating the connectivity of the router control plane Ethernet network.

•Detailed documentation about the network, including the following:

–Up-to-date internetwork map that outlines the physical location of all the devices on the network and how they are connected, as well as a logical map of interfaces, network addresses, network numbers, subnetworks, and so on

–List of all network protocols implemented in your network; and for each of the protocols implemented, a list of the network numbers, subnetworks, zones, areas, and so on that are associated with them

–All points of contact to external networks

–Routing protocol for each external network connection

–Established baseline for your network, that is, the normal network behavior and performance at different times of the day so that you can compare any problems with a baseline

–Name of the device that is the spanning tree root bridge for the system control plane Ethernet network

•Captured output of all commands

Verifying and Troubleshooting CLI Access

Ensure that the system has been booted. If the system has not booted, see Cisco IOS XR Getting Started Guide for the Cisco ASR 9000 Aggregation Services Router for information on booting a router running Cisco IOS XR software. The following CLI access troubleshooting information is provided:

•General CLI Access Information

•User Access Privileges

•Cisco-support Task ID

•CLI Access Through a Console Port

•CLI Access Through a Terminal Server

•CLI Access Through the Management Ethernet Interface

General CLI Access Information

The following CLI access information applies to a console port, terminal server, and management Ethernet interface connections.

Once the terminal emulation software is started and you press Enter, a router prompt should appear. If no prompt appears, verify the physical connection to the console port and press Enter again. If the prompt still does not appear, contact Cisco Technical Support. See the "Obtaining Documentation and Submitting a Service Request" section for Cisco Technical Support contact information.

If a prompt appears, indicating that the CLI is accessible, but your login username and password are invalid, you are prevented from accessing the router. Verify that you have the correct username and password. If you have the correct username and password, but are locked out of the router, you may need to perform password recovery to access the system again. See Cisco ASR 9000 Aggregation Services Router ROM Monitor Guide for password recovery procedures.

User Access Privileges

When you log on to the router, use a username that is associated with a valid user group that has the authorization to execute the required commands. If you suspect user group assignment is preventing you from using a command, contact your AAA administrator for assistance.

See Cisco ASR 9000 Aggregation Services Router System Security Command Reference and Cisco ASR 9000 Aggregation Services Router System Security Configuration Guide for information on users, usernames, and user groups.

Cisco-support Task ID

Many of the troubleshooting commands can be performed only by users who are assigned to a user group that includes the cisco-support task ID. Users without the cisco-support task ID receive a "This command is not authorized" response if they attempt to use those commands. The cisco-support commands are normally reserved for use by Cisco Technical Support personnel, because there is some risk that they may cause performance or other issues.

Caution These Cisco support commands are normally reserved for use by Cisco Technical Support personnel only. There is some risk that they may cause performance or other issues that impact products without proper usage, and we highly recommend that you contact Cisco Technical Support prior to using any of these commands. See the "Obtaining Documentation and Submitting a Service Request" section for information on contacting Cisco TAC.

CLI Access Through a Console Port

The first time a router is started, you must use a direct connection to the console port to connect to the router and enter the initial configuration. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on connecting to the router through a console port. When you use a direct connection to the Console port, CLI commands are entered at a terminal or at a computer running terminal emulation software. A direct Console port connection is useful for entering initial configurations and performing some debugging tasks.

CLI Access Through a Terminal Server

A terminal server connection provides a way to access the Console port from a remote location. A terminal server connection is used when you need to perform tasks that require Console port access from a remote location.

Connecting to a router through a terminal server is similar to directly connecting through the Console port. For both connection types, the physical connection takes place through the Console port. The difference is that the terminal server connects directly to the Console port, and you must use a Telnet session to establish communications through the terminal server to the router.

If you are unable to access the CLI through a terminal server, perform the following procedure.

Step 1 Disable flow control (XON/XOFF) on the Terminal Server.

Step 2 Disable local echo mode on the Terminal Server.

Step 3 Verify the router name configured using the hostname command.

Step 4 Check whether the port address is configured correctly.

Step 5 Verify whether the address (interface) used for the reverse Telnet is up/up. The output of the show interfaces brief command provides this information. Cisco recommends you to use loopbacks because they are always up.

Step 6 Ensure that you have the correct type of cabling. For example, you must not use a crossover cable to extend the length.

Step 7 Establish a Telnet connection to the IP address port to test direct connectivity. You must Telnet from both an external device and the terminal server. For example, telnet 172.21.1.1 2003.

Step 8 Ensure that you have the transport input telnet command under the line for the target device. The target device is the device that is connected to the terminal server.

Step 9 Use a PC/dumb terminal to connect directly to the console of the target router. The target router is the device connected to the terminal server. This step helps you identify the presence of a port issue.

Step 10 If you are disconnected, check timeouts. You can remove or adjust timeouts.

Note If you encounter authentication failures, remember that the terminal server performs the first authentication (if configured), while the device to which you try to connect performs the second authentication (if configured). Verify whether AAA is configured correctly on both the terminal server and the connecting device.

Step 11 Contact Cisco Technical Support. See the "Obtaining Documentation and Submitting a Service Request" section for Cisco Technical Support contact information.

CLI Access Through the Management Ethernet Interface

The Management Ethernet interface allows you to manage the router using a network connection. Before you can use the Management Ethernet interface, the interface must be configured. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on configuring the interface.

Once configured, the network connection takes place between client software on a workstation computer and a server process within the router. The type of client software you use depends on the server process you use. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on the client and server services supported by the Cisco IOS XR software.

If you are unable to access the CLI through a management Ethernet interface, perform the following procedure.

SUMMARY STEPS

1. show interface MgmtEth interface-instance

2. show arp MgmtEth interface-instance

3. show ipv4 interface type instance

4. ping

5. Contact Cisco Technical Support if the problem is not resolved

DETAILED STEPS

Command or Action

Purpose

Step 1

show interfaces MgmtEth interface-instance
Example:

RP/0/RSP0/CPU0:router# show interfaces MgmtEth 0/RSP0/CPU0/0

Displays statistics for all interfaces configured on the router.

Check the following:

•MgmtEth interface is up

•Line protocol (state of the Layer 2 line protocol) is up

•Number of input and output errors

If an interface is administratively down, use the no shutdown command to enable the interface.

If an interface is down (operationally down), input or output errors are not within an acceptable range, the management Ethernet interface is not enabled when the no shutdown command is used, or the line protocol is down, see Chapter 2 "Verifying and Troubleshooting Interface Status," for detailed information on troubleshooting interfaces.

If the interface is up and the input and output errors are within an acceptable range, proceed to Step 2.

Step 2

show arp MgmtEth interface-instance
Example:

RP/0/RSP0/CPU0:router# show arp MgmtEth 0/RSP0/CPU0/0

Displays the Address Resolution Protocol (ARP) table for the management Ethernet interface.

Ensure that the expected ARP entries exist for the management Ethernet interface.

If the expected ARP entries exist, proceed to Step 3.

If the expected ARP entries do not exist, verify the physical layer Ethernet interface connectivity. Use the show arp trace command to display the ARP entries in the buffer. See the Chapter 2 "Verifying and Troubleshooting Interface Status," for more information on troubleshooting interfaces.

Step 3

show ipv4 interface type instance
Example:

RP/0/RSP0/CPU0:router# show ipv4 interface MgmtEth 0/RSP0/CPU0/0

Displays the usability status of interfaces configured for IPv4.

If the interface is in the expected state, proceed to Step 4.

If the status of the interface is not as expected, see Chapter 2 "Verifying and Troubleshooting Interface Status," for more information on troubleshooting interfaces.

Step 4

ping
Example:

RP/0/RSP0/CPU0:router# ping

Checks host reachability and network connectivity on the IP network.

Note Enter a specific IP address or follow the prompts to send the ping message to the target address.

If no problems are detected, proceed to Step 5.

Step 5

Contact Cisco Technical Support.

If the problem is not resolved, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section.

Examples

The output from the show interfaces MgmtEth command displays the status of the management Ethernet interface. In the following example, the management Ethernet interface is up, and there are 20 input errors and 8 output errors.
RP/0/RSP0/CPU0:router# show interface MgmtEth 0/RSP0/CPU0/0 
Tue Sep 14 14:21:07.496 DST
MgmtEth0/RSP0/CPU0/0 is up, line protocol is up 
  Interface state transitions: 1
  Hardware is Management Ethernet, address is 001b.53ff.4a62 (bia 001b.53ff.4a62)
  Description: Connected to Lab LAN
  Internet address is 172.29.52.137/24
  MTU 1514 bytes, BW 100000 Kbit (Max: 100000 Kbit)
     reliability 73/255, txload 0/255, rxload 0/255
  Encapsulation ARPA,
  Half-duplex, 100Mb/s, THD, link type is autonegotiation
  output flow control is off, input flow control is off
  loopback not set,
  ARP type ARPA, ARP timeout 04:00:00
  Last input 00:00:00, output 00:00:00
  Last clearing of "show interface" counters never
  5 minute input rate 2000 bits/sec, 3 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     373082 packets input, 51028824 bytes, 239105 total input drops
     62028 drops for unrecognized upper-level protocol
     Received 2601 broadcast packets, 194653 multicast packets
              10 runts, 0 giants, 0 throttles, 0 parity
     20 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     45232 packets output, 3042775 bytes, 0 total output drops
     Output 24 broadcast packets, 0 multicast packets
     8 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     1 carrier transitions
The output from the show arp MgmtEth 0/RSP0/CPU0/0 command displays the ARP table for the management Ethernet interface. Use the output from this command to verify that there are dynamic ARP addresses in the table and that ARP is functioning over the interface. The output shows that ARP is functioning over the management Ethernet interface 0/RSP0/CPU0/0.
RP/0/RSP0/CPU0:router# show arp MgmtEth 0/RSP0/CPU0/0 
Tue Sep 14 14:24:03.962 DST
-------------------------------------------------------------------------------
0/RSP0/CPU0
-------------------------------------------------------------------------------
Address         Age        Hardware Addr   State      Type  Interface
172.29.52.1     01:44:00   0000.0c07.ac01  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.13    01:16:59   0010.79e9.6038  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.21    01:40:25   0022.0d5a.a6c4  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.27    02:18:16   0012.7fd6.ba08  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.28    02:05:29   0012.7fd6.ba09  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.32    01:42:16   0022.0d26.3bc5  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.36    02:39:34   0026.527c.5341  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.46    01:36:50   0012.7fd6.b9aa  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.47    01:36:39   0012.7fd6.b9ab  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.60    01:35:20   0003.a099.8000  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.69    00:00:00   001b.7852.4bd1  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.70    01:23:38   0011.93ef.e8e6  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.71    02:00:47   0011.93ef.e8fe  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.75    01:44:59   5a59.0000.0202  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.76    01:41:10   0011.93ef.e8ea  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.81    00:15:35   001a.6c40.d89c  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.83    00:21:05   001a.6c40.d89c  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.127   01:43:38   0013.c4cb.a200  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.134   01:15:53   001f.6c26.7fc0  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.135   01:01:46   001f.6c25.c480  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.136   00:43:39   0022.5560.8840  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.137   -          001b.53ff.4a62  Interface  ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.138   -          001b.53ff.4a62  Interface  ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.161   01:32:12   0019.aaa3.3d48  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.171   00:16:12   001c.5838.5b28  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.172   00:17:47   001c.5838.5b29  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.173   01:57:04   0015.c75f.09f8  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.180   01:26:50   0015.c75f.0800  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.217   01:16:21   0019.aaa3.b5ff  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.226   01:28:32   0010.f60e.8400  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
172.29.52.243   01:48:25   001e.79c1.e0c1  Dynamic    ARPA  MgmtEth0/RSP0/CPU0/0
The ping command checks to see if the neighbor is reachable.
RP/0/RSP0/CPU0:router# ping 172.16.52.28 count 10 
Tue Sep 14 14:36:52.441 DST
Type escape sequence to abort.
Sending 10, 100-byte ICMP Echos to 172.16.52.28, timeout is 2 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 1/1/2 ms
Validating and Troubleshooting Installation of the Cisco IOS XR Software Package

The Cisco IOS XR software is divided into software packages allowing you to select which features run on your router. Each package contains the components to perform a specific set of router functions, such as routing, security, or Modular Services Card (MSC) support. Bundles are groups of packages that can be downloaded as a set. For example, the Unicast Routing Core Bundle provides six packages for use on every router.

This section provides information on how to validate and troubleshoot the Cisco IOS XR software package installation. The following sections are provided:

•Verifying the Software Version

•Validating the Installation

Verifying the Software Version

To verify the Cisco IOS XR software version, perform the following procedure.

SUMMARY STEPS

1. show version

2. show install

DETAILED STEPS

Command or Action

Purpose

Step 1

show version
Example:

RP/0/RSP0/CPU0:router# show version

Displays a variety of system information, including hardware and software version, router uptime, boot settings (configuration register), and active software.

Determine if all expected packages are installed and the current software versions are the expected versions.

If the expected packages are not installed or are not the expected version, install the correct package. See Cisco ASR 9000 Aggregation Series Router Getting Started Guide for information on installing and upgrading Cisco IOS XR software packages.

Step 2

show install
Example:

RP/0/RSP0/CPU0:router# show install

Displays a list of all installed and active packages on each node.

Determine if the expected packages are installed on each node.

If the software or active package versions are not as expected for a node, the package is not compatible with the node for which it is being activated, or the package being activated is not compatible with the current active software set, install the correct software or package on the node. See Cisco ASR 9000 Aggregation Series Router Getting Started Guide for information on installing and upgrading Cisco IOS XR software packages.

The following example shows that the Cisco IOS XR software and active packages are version 4.0.0.
RP/0/RSP0/CPU0:router# show version
Cisco IOS XR Software, Version 4.0.0
Copyright (c) 2010 by cisco Systems, Inc.
ROM: System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],  
router uptime is 1 day, 18 hours, 34 minutes
System image file is "bootflash:disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm"
cisco ASR9K Series (MPC8641D) processor with 4194304K bytes of memory.
MPC8641D processor at 1333MHz, Revision 2.2
2 Management Ethernet
12 DWDM controller(s)
12 TenGigE
40 GigabitEthernet
2 SONET/SDH
2 Packet over SONET/SDH
219k bytes of non-volatile configuration memory.
975M bytes of compact flash card.
33994M bytes of hard disk.
1605616k bytes of disk0: (Sector size 512 bytes).
1605616k bytes of disk1: (Sector size 512 bytes).
Configuration register on node 0/RSP0/CPU0 is 0x0
Boot device on node 0/RSP0/CPU0 is disk0:
Package active on node 0/RSP0/CPU0:
asr9k-optics-supp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-optics-supp-4.0.0
    Built on Wed Sep  8 16:17:30 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-fwding, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-fwding-4.0.0
    Built on Wed Sep  8 16:12:40 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-cpp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-cpp-4.0.0
    Built on Wed Sep  8 16:13:28 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9K-doc-supp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9K-doc-supp-4.0.0
    Built on Wed Sep  8 16:16:57 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-scfclient, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-scfclient-4.0.0
    Built on Wed Sep  8 16:13:26 DST 2010
 --More-- 
.
.
.
The following example shows that the Cisco IOS XR software and active packages are version 4.0.0. If there is an expected package missing or an active package is not an expected package, install and activate the missing package or upgrade the unexpected package to the appropriate package. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for details on installing, activating, and upgrading software packages.
RP/0/RSP0/CPU0:router# show install 
Node 0/RSP0/CPU0 [RP] [SDR: Owner]
    Boot Device: disk0:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-doc-p-4.0.0
      disk0:asr9k-k9sec-p-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mgbl-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/1/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/2/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/4/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/6/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
Validating the Installation

Validate the Cisco IOS XR software package installation to ensure the packages were installed correctly. The following commands are used to validate the currently installed software packages:

•install verify Command

•show install active Command

•show install committed Command

install verify Command

Use the install verify command to verify the consistency of a previously installed software set with the package file from which it originated.

This command can be used as a debugging tool to verify the validity of the files that constitute the packages to determine if there are any corrupted files. The command is also used to check that the install infrastructure is up and running and to determine if all files are expected. If there are corrupted files, see Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on deactivating and removing software packages and adding and activating software packages.

Note The install verify command can take up to two minutes per package to process.

Note The install verify command ignores secure domain router (SDR) boundaries and performs the operation in global scope.

The following example shows the output of the install verify command. The output is used to verify the consistency of a previously installed software set with the package file from which it originated.
RP/0/RSP0/CPU0:router(admin)# install verify 
Sat Sep 25 08:18:14.077 DST
Install operation 3 '(admin) install verify packages' started by user_A
'dwolman-r' via CLI at 08:18:14 DST Sat Sep 25 2010.
The install operation will continue asynchronously.
RP/0/RSP0/CPU0:router(admin)#Info:     This operation can take up to 2 minutes per package 
being verified.
Info:     Please be patient.
Info:     0/0/CPU0 [LC] [SDR: Owner]
Info:         meta-data: [SUCCESS] Verification Successful.
Info:         /install/asr9k-optics-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-fwding-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-cpp-4.0.0: [SUCCESS] Verification Successful.
Info:         /install/asr9k-scfclient-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-video-adv-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-mpls-4.0.0: [SUCCESS] Verification Successful.
Info:         /install/iosxr-mcast-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-routing-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-infra-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-fwding-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-diags-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-adv-video-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-diags-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-mcast-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-base-4.0.0: [SUCCESS] Verification Successful.
Info:     0/6/CPU0 [LC] [SDR: Owner]
Info:         meta-data: [SUCCESS] Verification Successful.
Info:         /install/asr9k-optics-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-fwding-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-cpp-4.0.0: [SUCCESS] Verification Successful.
Info:         /install/asr9k-scfclient-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-video-adv-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-mpls-4.0.0: [SUCCESS] Verification Successful.
Info:         /install/iosxr-mcast-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-routing-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-infra-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-fwding-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/iosxr-diags-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-adv-video-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-diags-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-mcast-supp-4.0.0: [SUCCESS] Verification
Info:     Successful.
Info:         /install/asr9k-base-4.0.0: [SUCCESS] Verification Successful.
Info:     0/5/CPU0 [LC] [SDR: Owner]
.
.
.
Info:     Verification Summary:
Info:         0/0/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/6/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/5/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/7/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/1/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/4/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/2/CPU0: SUCCESSFUL. No anomalies found.
Info:         0/RSP0/CPU0: SUCCESSFUL. No anomalies found.
Info:         The system needs no repair.
Install operation 3 completed successfully at 08:19:48 DST Sat Sep 25 2010.
show install active Command

Use the show install active command to display active software packages. Verify that the command output matches the output of the show install committed command. If the output does not match, when you reload the router, the software displayed in the show install committed command output is the software that will be loaded. For example, the following output shows two different software package versions, one is the active version and the other is the committed version, so when the router reloads, the 3.9.1 version will be loaded even though 4.0.0 is the currently active version on 0/RSP0/CPU0.
RP/0/RSP0/CPU0:router(admin)# show install active location 0/RSP0/cpu0   
Node 0/RSP0/CPU0 [RP] [SDR: Owner]
    Boot Device: disk0:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm << 4.0.0 is active, not committed
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-doc-p-4.0.0
      disk0:asr9k-k9sec-p-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mgbl-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
RP/0/RSP0/CPU0:router(admin)# show install committed location 0/RSP0/cpu0    
Node 0/RSP0/CPU0 [RP] [SDR: Owner]
    Boot Device: disk0:
    Boot Image: /disk0/asr9k-os-mbi-3.9.1/mbiasr9k-rp.vm<< 3.9.1 is committed
    Committed Packages: 
      disk0:asr9k-mini-p-3.9.1
      disk0:asr9k-optic-3.9.1
      disk0:asr9k-doc-p-3.9.1
      disk0:asr9k-k9sec-p-3.9.1
      disk0:asr9k-video-p-3.9.1
      disk0:asr9k-mpls-p-3.9.1
      disk0:asr9k-mgbl-p-3.9.1
      disk0:asr9k-mcast-p-3.9.1
If the expected active software packages are not displayed, install the packages (if required) and activate the packages. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on installing and activating Cisco IOS XR software packages. The following example output shows the active packages for all cards in a router. The output displays the disk on which each package is located.
RP/0/RSP0/CPU0:router# show install active
Node 0/RSP0/CPU0 [RP] [SDR: Owner]
    Boot Device: disk0:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-doc-p-4.0.0
      disk0:asr9k-k9sec-p-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mgbl-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/0/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/1/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/2/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/4/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/5/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/6/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/7/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Active Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
The output shows the name of the disk on which the packages are located. In the above example, the active packages for each node are on disk0, and for all nodes, the composite package asr9k-os-mbi-4.0.0 is active. Additional packages shown are optional packages that have been activated after the initial loading of the Cisco ASR 9000 Aggregation Series Router Unicast Routing Core Bundle.

show install committed Command

Use the show install committed command to display committed software packages. The committed software packages are the software packages that will be booted on a router reload.

Committed packages are the packages that are persistent across router reloads. If you install and activate a package, it remains active until the next router reload. If you commit a package set, all packages in that set remain active across router reloads until the package set is replaced with another committed package set. The show install committed command is useful to ensure software is installed and committed after a router reload. If the expected software is not installed and committed, see Cisco ASR 9000 Aggregation Services Router Getting Started Guide for information on installing and committing Cisco IOS XR software packages.

The following command output shows the committed software packages on all cards in the router. The output displays the disk on which each package is located.
RP/0/RSP0/CPU0:router# show install committed 
Node 0/RSP0/CPU0 [RP] [SDR: Owner]
    Boot Device: disk0:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-doc-p-4.0.0
      disk0:asr9k-k9sec-p-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mgbl-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/0/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
Node 0/1/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
Node 0/2/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/4/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/5/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/6/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
  Node 0/7/CPU0 [LC] [SDR: Owner]
    Boot Device: mem:
    Boot Image: /disk0/asr9k-os-mbi-4.0.0/lc/mbiasr9k-lc.vm
    Committed Packages: 
      disk0:asr9k-mini-p-4.0.0
      disk0:asr9k-optic-4.0.0
      disk0:asr9k-video-p-4.0.0
      disk0:asr9k-mpls-p-4.0.0
      disk0:asr9k-mcast-p-4.0.0
The output shows the name of the disk on which the packages are located. In the above example, the committed packages for each node are on disk0, and for all nodes, the composite package asr9k-os-mbi-4.0.0 is committed. Additional packages shown are optional packages that have been committed after the initial loading of the Cisco ASR 9000 Aggregation Series Router Unicast Routing Core Bundle.

Validating and Troubleshooting Cisco IOS XR Software Configuration

Validating the Cisco IOS XR software configuration includes collecting configuration information on the router to determine configuration changes and verifying the current running configuration. When a configuration fails during a commit, the failed configuration can be viewed to help determine why the configuration was not committed.

The following sections are provided:

•Local and Global Configurations

•Collecting Configuration Information

•Verifying the Running Configuration

•Using the show configuration failed Command

Local and Global Configurations

To troubleshoot configurations, you need to determine whether the problem is in the local configuration or the shared (global) configuration.

•The local configuration is specific to the individual LC or RP to which it belongs. Every LC and RP has a data store containing the local data for that node, including configuration and operational data for the local interfaces. An example of a local configuration is the port designations on a particular LC.

•The shared (global) configuration applies to the entire router, and is shared with all of the LCs and RPs. An example of a shared configuration is the routing protocol parameters.

To view the local configuration, use the show running-config interface * command. The output displays all the configured interfaces on the node.
RP/0/RSP0/CPU0:router# show running-config interface * 
interface Bundle-Ether16
 description Connect to router-S Port-Ch 16
 mtu 9216
 bundle maximum-active links 1
!
interface Bundle-Ether16.160 l2transport
 description Connect to router-S Port-Ch 16 Service Instance 160
 encapsulation dot1q 160
!
.
.
.
!
interface Loopback0
ipv4 address 10.144.144.144 255.255.255.255
!
interface tunnel-ip10
!
interface tunnel-te44190
 description Primary GE Tunnel from router-S to router-T
 ipv4 unnumbered Loopback0
 priority 0 0
 autoroute announce
 signalled-bandwidth 100000
 destination 10.19.19.19
 fast-reroute
 record-route
 path-option 1 explicit name Primary_GE_Path_to_router-T ospf 100 area 0
!
.
.
.
interface MgmtEth0/RSP0/CPU0/0
 description Connected to LAN
 ipv4 address 172.29.52.137 255.255.255.0
!
interface MgmtEth0/RSP0/CPU0/1
 shutdown
!         
interface GigabitEthernet0/0/0/0
 shutdown
!
interface GigabitEthernet0/0/0/1
 shutdown
!
interface GigabitEthernet0/0/0/2
 shutdown
!
.
.
.
interface TenGigE0/7/0/1
 shutdown
!
interface TenGigE0/7/0/2
 shutdown
!
interface TenGigE0/7/0/3
 shutdown
!
interface POS0/2/0/0
 description Connected to PE_router-2 POS 0/2/0/0
!
interface POS0/2/0/1
 description Connected to PE_router-3 POS 0/2/0/1
!
controller SONET0/2/0/0
 clock source internal
!
controller SONET0/2/0/1
 clock source internal
!         
Use the show sysdb trace commands to display the contents of the system database after a configuration change. The trace information includes a history of any changes to the running configuration. You can specify either a local node or the shared plane.

The following example output shows the contents of the local database, that is, for a specific location (node):
RP/0/RSP0/CPU0:router# show sysdb trace verification location 0/5/cpu0 reverse 
.
.
.
Timestamp                   nid      reqid      jid      tid   reg_hndl connid     action                         
path
432 wrapping entries (8192 possible, 158 filtered, 590 total)
Sep 23 04:35:39.969      0/RSP0/CPU0  8168      354       1     94       4483      apply 
reply                    '--'
Sep 23 04:35:39.960      0/RSP0/CPU0  8168      354       1     94       4505      Apply 
called                   'cfg/if/act/tunnel-ip10/v'
Sep 23 04:35:39.960      0/RSP0/CPU0  8168      354       1     94       4505      verify 
reply: accept           '--'
Sep 23 04:35:39.685      0/RSP0/CPU0  8168      354       1     94       4505      Verify 
called                  'cfg/if/act/tunnel-ip10/v'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/mtu/tunnel-ip'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/im/bw'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/tunl_gre/keepalive'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/tunl_gre/dfbit_disable'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/tunl_gre/ttl'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/tunl_gre/tos'
Sep 23 04:35:39.678      0/RSP0/CPU0  0         354       1     94       4505      
register                       'cfg/if/act/tunnel-ip[0-9]*/tunl_gre/mode'
.
.
.
The following example output shows the contents of the shared database, that is, the configuration data that is shared with all LC and RP in the router:
RP/0/RSP0/CPU0:router# show sysdb trace verification shared-plane reverse 
Config Shared Server
====================
Timestamp                   nid      reqid      jid      tid   reg_hndl connid     action                         
path
2259 wrapping entries (4096 possible, 0 filtered, 2259 total)
Sep 23 19:34:40.202      0/3/CPU0     0         241       8     384      1430      
unregister                     'from-'
Sep 23 19:34:40.197      0/3/CPU0     0         241       15    385      1434      
unregister                     'from-'
Sep 23 19:34:40.196      0/3/CPU0     0         163       1     386      1440      
unregister                     'from-'
Sep 23 19:14:45.076      0/3/CPU0     0         163       1     386      1440      
register                       'cfg/gl/ipv4/cef/hardware/forwarding/update/synchronous'
Sep 23 19:14:41.679      0/3/CPU0     0         241       15    385      1434      
register                       'cfg/gl/dbgtrace/node/831/'
Sep 23 19:14:41.593      0/3/CPU0     0         241       8     384      1430      
register                       'cfg/gl/dbgtrace/node/831/'
Sep 23 19:12:36.472      0/3/CPU0     0         241       8     381      1375      
unregister                     'from-'
Sep 23 19:12:36.471      0/3/CPU0     0         241       15    382      1378      
unregister                     'from-'
Sep 23 19:12:36.470      0/3/CPU0     0         163       1     383      1383      
unregister                     'from-'
Sep 23 19:07:56.914      0/3/CPU0     0         163       1     383      1383      
register                       'cfg/gl/ipv4/cef/hardware/forwarding/update/synchronous'
.
.
.
The show processes location node-id | include sysdb command displays all active database processes for a specified node.
RP/0/RSP0/CPU0:router# show process location 0/1/CPU0 | include sysdb 
Thu Nov  4 14:06:30.191 DST
279    1    0   56K  10 Sigwaitinfo  739:28:22:0145    0:00:00:0057 sysdb_svr_local
279    2    1   56K  10 Receive        0:00:00:0779    0:00:02:0459 sysdb_svr_local
279    3    1   56K  10 Receive        0:03:34:0474    0:00:03:0285 sysdb_svr_local
279    4    1   56K  10 Receive        0:05:03:0006    0:00:02:0368 sysdb_svr_local
277    1    0   64K  10 Sigwaitinfo  739:28:21:0305    0:00:00:0046 sysdb_mc
277    2    0   64K  10 Receive      739:28:21:0274    0:00:00:0003 sysdb_mc
277    3    1   64K  10 Receive      166:59:14:0698    0:00:00:0038 sysdb_mc
277    4    1   64K  10 Receive        0:01:49:0941    0:00:00:0106 sysdb_mc
277    6    1   64K  10 Receive      739:15:22:0734    0:00:00:0058 sysdb_mc
See Chapter 1 "General Troubleshooting Procedures" for additional information on troubleshooting processes.

Collecting Configuration Information

Collecting configuration information allows you to determine if changes to the system have occurred. It also allows you to determine if these changes could impact the system. The following commands allow you to determine if there was an unknown commit, if there was a commit that overwrote a previous configuration, or there are configuration changes that should be removed from the running configuration.

•show configuration commit changes {[since] commit-id | last number-of-commits} [diff]—the command output displays changes made to the running configuration by previous configuration commits.
RP/0/RSP0/CPU0:router# show configuration commit changes since 1000000319 
Wed May 17 09:30:27.877 UTC 
Building configuration...
no logging console
no domain ipv4 host ce1
no domain ipv4 host ce2
domain ipv4 host ce6 172.29.52.73
domain ipv4 host ce7 172.29.52.78
no domain ipv4 host pe1
no domain ipv4 host pe2
domain ipv4 host pe6 172.29.52.128
domain ipv4 host pe7 172.29.52.182
interface GigabitEthernet0/1/5/1
 no negotiation
!
end
•show configuration commit list [number-of-commits] [detail]—the command output displays a list of the commit IDs (up to 100) available for rollback.
RP/0/RSP0/CPU0:router# show configuration commit list 
Wed May 17 09:31:21.727 UTC 
SNo. Label/ID    User      Line        Client      Time Stamp
~~~~ ~~~~~~~~    ~~~~      ~~~~        ~~~~~~      ~~~~~~~~~~
1    1000000324  userA     vty0        CLI         16:50:33 UTC Wed May 10 2006
2    1000000323  userA     vty0        CLI         16:49:51 UTC Wed May 10 2006
3    1000000322  userB     vty0        CLI         16:48:05 UTC Wed May 10 2006
4    1000000321  userC     vty2        CLI         19:11:26 UTC Wed May 03 2006
5    1000000320  userA     vty2        CLI         19:10:45 UTC Wed May 03 2006
6    1000000319  userB     vty2        CLI         18:03:01 UTC Wed May 03 2006
7    1000000318  userB     vty2        CLI         18:02:43 UTC Wed May 03 2006
8    1000000317  userB     vty2        CLI         18:02:38 UTC Wed May 03 2006
9    1000000316  userC     vty2        CLI         17:59:16 UTC Wed May 03 2006
10   1000000315  userC     vty2        CLI         17:46:38 UTC Wed May 03 2006
11   1000000314  userA     vty2        CLI         15:40:04 UTC Wed May 03 2006
12   1000000313  userA     vty2        CLI         13:05:09 UTC Wed May 03 2006
13   1000000312  userD     con0_RSP0_C  CLI         13:49:31 UTC Mon May 01 2006
•commit confirmed minutes (executed from config mode)—This command commits the configuration on a trial basis for a minimum of 30 seconds and a maximum of 300 seconds (5 minutes). During the trial configuration period, enter commit to confirm the configuration. If commit is not entered, then the system will revert to the previous configuration when the trial time period expires.

Verifying the Running Configuration

To verify the running configuration, perform the following procedure.

SUMMARY STEPS

1. configure

2. show running-config

3. describe hostname hostname

4. end

5. show sysdb trace verification shared-plane | include path

6. show sysdb trace verification location node-id

7. show cfgmgr trace

8. show configuration history commit

9. show configuration commit changes {last | since | commit-id}

10. show config failed startup

11. cfs check

DETAILED STEPS
Command or Action

Purpose

Step 1

configure
Example:

RP/0/RSP0/CPU0:router# configure

Enters global configuration mode.

Step 2

show running-config
Example:

RP/0/RSP0/CPU0:router(config)# show running-config

Displays the contents of the running configuration.

Verify that the running configuration is as expected.

Step 3

describe hostname hostname
Example:

RP/0/RSP0/CPU0:router(config)# describe hostname router_A

Determines the path.
Step 4

end
Example:

RP/0/RSP0/CPU0:router(config)# end
Saves configuration changes.

•When you issue the end command, the system prompts you to commit changes:
Uncommitted changes found, commit them before 
exiting(yes/no/cancel)? 
[cancel]:
–Entering yes saves configuration changes to the running configuration file, exits the configuration session, and returns the router to EXEC mode.

–Entering no exits the configuration session and returns the router to EXEC mode without committing the configuration changes.

–Entering cancel leaves the router in the current configuration session without exiting or committing the configuration changes.
Step 5

show sysdb trace verification shared-plane | include path
Example:

RP/0/RSP0/CPU0:router# show sysdb trace verification shared-plane | include gl/a/hostname

Displays details of recent verification sysDB transactions and changes on the shared plane allowing you to verify whether the configuration was verified correctly.

Specifying the path filters the data to display only the sysDB path for the router.

Verify that changes to the SysDB were verified and accepted.

Step 6

show sysdb trace verification location node-id
Example:

RP/0/RSP0/CPU0:router# show sysdb trace verification location 0/3/CPU0

Displays details of recent verification sysDB transactions and changes on local plane configurations.

Verify that changes to the SysDB were verified and accepted.

Step 7

show cfgmgr trace
Example:

RP/0/RSP0/CPU0:router# show cfgmgr trace

Displays cfgmgr trace information.

Step 8

show configuration history commit
Example:

RP/0/RSP0/CPU0:router# show configuration history commit

Displays a list of historical changes to the configuration.

Verify that the timeline of changes is as expected.

Step 9

show configuration commit changes {last | since | commit-id}
Example:

RP/0/RSP0/CPU0:router# show configuration commit changes last 15

Displays detailed committed configuration history information.

Verify that the history information is as expected.

Step 10

show configuration failed startup
Example:

RP/0/RSP0/CPU0:router# show configuration failed startup

Displays information on any configurations that failed during startup.

Step 11

cfs check
Example:

RP/0/RSP0/CPU0:router# cfs check

Checks the current configuration to see if there are any missing configurations.
Examples

The following example shows the output of the show running-config command:
RP/0/RSP0/CPU0:router# show running-config  
Building configuration...
!! IOS XR Configuration 4.0.1.10I
!! Last configuration change at Thu Sep 23 04:35:38 2010 by user_A
!
hostname router
.
.
.
logging suppress duplicates
telnet vrf default ipv4 server max-servers 100
domain name cisco.com
domain lookup disable
taskgroup default
!
.
.
.
interface preconfigure GigabitEthernet0/3/0/7
 shutdown
!
interface preconfigure GigabitEthernet0/3/0/8
 shutdown
!
interface preconfigure GigabitEthernet0/3/0/9
 shutdown
!
interface preconfigure GigabitEthernet0/3/0/10
 shutdown
!
.
.
.
The output is used to determine if the configuration is as expected.

In the following example, the path to SysDB where the configuration is stored in the database is displayed.
RP/0/RSP0/CPU0:router(config)# describe hostname router 
Package:
    iosxr-infra
        iosxr-infra V4.0.0 IOS-XR Infra Package Definition
        Vendor : Cisco Systems
        Desc   : IOS-XR Infra Package Definition
        Build  : Built on Wed Sep  8 16:07:48 DST 2010
        Source : By sc-g-01 in /auto/ioxbuild8/production/4.0.0/asr9k/workspace for pie
        Card(s): RP, NP24-4x10GE, NP24-40x1GE, NP40-40x1GE, NP40-4x10GE, NP40-8x10GE, 
NP40-2_20_COMBO, NP80-8x10GE, NP80-16x10GE, A9K-SIP-700, A9K-SIP-500
        Restart information:
          Default:
            parallel impacted processes restart
        Size Compressed/Uncompressed: 38MB/85MB (44%)
Component:
    shellutil V[ci-401/7]  Common shell utility applications
User needs ALL of the following taskids:
        host-services (READ) or root-lr (READ WRITE) 
It will take the following actions:
  Create/Set the configuration item:
         Path: gl/a/hostname
        Value: router
Using the show configuration failed Command

Use the show configuration failed command to browse a failed configuration. The configuration can be classified as failed during startup or during a configuration commit.

•Startup Failed Configuration

•Commit Configuration Failed

Startup Failed Configuration

A configuration can be classified as failed during startup for three reasons:

•Syntax errors

Syntax errors are generated by the parser and usually indicate that there is an incompatibility with the CLI commands. Correct the syntax errors and reapply the configuration. A syntax error can be an invalid CLI entry or a CLI syntax change. See the "Obtaining Documentation and Submitting a Service Request" section in the Preface for information on obtaining Cisco IOS XR software CLI documentation.

•Semantic errors

Semantic errors are generated by the backend components when the configuration is being restored by the configuration manager during startup of the router. Semantic errors include logical problems (invalid logic).

•Apply errors

Apply errors are generated when a configuration has been successfully verified and accepted as part of running configuration but the backend component is not able to update its operational state. The configuration shows both as the running configuration (since it was correctly verified) and as a failed configuration because of the backend operational error. To find the component apply owner, use the describe on the CLI that failed to be applied.

Note You may browse startup failed configurations for up to the previous four router reloads.

Use the show configuration failed startup command and the load configuration failed startup command to browse and reapply any failed configuration. The load configuration failed startup command can be used in configuration mode to load the failed startup configuration into the target configuration session, then the configuration can be modified and committed. See Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide for information on committing a configuration.
RP/0/RSP0/CPU0:router# show configuration failed startup 
!! CONFIGURATION FAILED DUE TO SYNTAX/AUTHORIZATION ERRORS 
telnet vrf default ipv4 
server max-servers 5 interface POS0/7/0/3 router static 
address-family ipv4 unicast 
  0.0.0.0/0 172.18.189.1 
!! CONFIGURATION FAILED DUE TO SEMANTIC ERRORS 
router bgp 217 
!!% Process did not respond to sysmgr ! 
RP/0/RSP0/CPU0:router# 
RP/0/RSP0/CPU0:router# config 
RP/0/RSP0/CPU0:router(config)# load config failed startup noerror 
Loading. 263 bytes parsed in 1 sec (259)bytes/sec 
RP/0/RSP0/CPU0:mike3(config-bgp)#show configuration 
Building configuration... 
telnet vrf default ipv4 server max-servers 5 router static 
address-family ipv4 unicast 
  0.0.0.0/0 172.18.189.1 
  ! 
!
router bgp 217 
! 
end 
The failed configuration is loaded into the target configuration, minus the errors that caused the startup configuration to fail.
RP/0/RSP0/CPU0:router(config-bgp)# commit 
Use the show configuration failed command to display failed items in the last configuration commit, including reasons for the error.

In any mode, the configuration failures from the most recent commit operation are displayed.

The show configuration failed command can be used in EXEC mode and configuration mode. The command is used in EXEC mode when the configuration does not load during startup. The command is used in configuration mode to display information when a commit fails.

The following example shows the show configuration failed command.
RP/0/RSP0/CPU0:router(config)# interface pos 0/6/0/4 
RP/0/RSP0/CPU0:router(config-if)# no vrf 
RP/0/RSP0/CPU0:router(config-if)# commit
% Failed to commit one or more configuration items during an atomic operation, no changes 
have been made. Please use 'show configuration failed' to view the errors
RP/0/RSP0/CPU0:router(config-if)# exit
RP/0/RSP0/CPU0:router(config)# show configuration failed 
Wed May  2 13:14:08.426 EST EDT 
!! CONFIGURATION FAILED DUE TO SEMANTIC ERRORS interface POS0/6/0/4 no vrf !!
% The interface's numbered and unnumbered IPv4/IPv6 addresses must be removed prior to 
changing or deleting the VRF !
Note The show configuration failed command in configuration mode only exists as long as the configuration session is active. Once you exit configuration mode, the command cannot be used to display the failed configuration.

Commit Configuration Failed

The following example shows an invalid task ID configuration that fails to commit. The show configuration failed command provides information on why the configuration failed.
RP/0/RSP0/CPU0:router(config)# taskgroup isis  
RP/0/RSP0/CPU0:router(config-tg)# commit
% Failed to commit one or more configuration items during an atomic operation, s
RP/0/RSP0/CPU0:router(config-tg)# show configuration failed
!! CONFIGURATION FAILED DUE TO SEMANTIC ERRORS
taskgroup isis
!!% Usergroup/Taskgroup names cannot be taskid names
!
If a configuration commit fails, do not exit configuration mode (return to EXEC mode) as you will not be able to view the failed configuration.
RP/0/RSP0/CPU0:router# configure
RP/0/RSP0/CPU0:router(config)# taskgroup bgp 
RP/0/RSP0/CPU0:router(config-tg)# end
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:y
% Failed to commit one or more configuration items during an atomic operation, s
RP/0/RSP0/CPU0:router(config)# exit
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:n
RP/0/RSP0/CPU0:router# show configuration failed 
RP/0/RSP0/CPU0:router# 
Verifying the System

To verify the general status and state of a router using Cisco IOS XR software, perform the following procedure.

SUMMARY STEPS

1. admin

2. show platform [node-id]

3. show version

4. show running-config

5. show logging

6. show environment

7. show context

8. exit

9. show context

10. show memory summary detail location all

11. show memory heap summary {job-id | all}

12. top processes

13. show running-config

14. show system verify start
show system verify report

15. show {ipv4 | ipv6} interface brief

DETAILED STEPS

Command or Action

Purpose

Step 1

admin
Example:

RP/0/RSP0/CPU0:router# admin

Enters administration mode.

Step 2

show platform [node-id]
Example:

RP/0/RSP0/CPU0:router(admin)# show platform

Displays information about the status of cards and modules installed in the router.

•Some cards support a CPU module and service processor (SP) module. Other cards support only a single module.

•A card module is also called a node. When all nodes are working properly, the status of each node displayed in the State column is IOS-XR RUN.

•If you run the command without a node-id (show platform as shown in the example), the output will include all nodes in the system.

•Type the show platform node-id command to display information for a specific node. Replace node-id with a node name from the show platform command Node column.

Step 3

show version
Example:

RP/0/RSP0/CPU0:router(admin)# show version

Displays information about the router, including image names, uptime, and other system information.

Verify that the expected software version and images are installed.

Step 4

show running-config
Example:

RP/0/RSP0/CPU0:router(admin)# show running-config

Displays all of the nondefault commands currently running, including hardware module power status, secure domain router (SDR) configuration, and fabric configuration. The output also displays the users defined in administration mode with root-system access.

Verify that the serial numbers for the nodes in the current running configuration are what you expected. The expected rack numbers and serial numbers should be listed in the current system documentation. See the "Prerequisite Documentation for Troubleshooting" section.

Also verify that the hardware module power status is as expected and the SDR and fabric configurations are as expected.

Step 5

show logging
Example:

RP/0/RSP0/CPU0:router(admin)# show logging

Displays all syslog messages stored in the buffer. The command output displays the device operation history from a system perspective.

Analyze the logged events and their order of happening. Check for anything out of the ordinary such as errors, tracebacks, or crashes. Also check for any Severity 1 or Severity 2 errors.

Step 6

show environment
Example:

RP/0/RSP0/CPU0:router(admin)# show environment

Displays environmental monitor parameters for the system.

Verify that the parameters are as expected.

Step 7

show context
Example:

RP/0/RSP0/CPU0:router(admin)# show context

Displays core dump context information on fabric cards, alarm modules, fan controllers, and service processors (system-owned cards). See the "show context Command" section for more information on the show context command output.

Step 8

exit
Example:

RP/0/RSP0/CPU0:router(admin)# exit

Exits administration mode.

Step 9

show context
Example:

RP/0/RSP0/CPU0:router# show context

Displays core dump context information on CPUs responsible for routing and Cisco Express Forwarding (CEF). See the "show context Command" section for more information on the show context command output.

Step 10

show memory summary detail location all
Example:

RP/0/RSP0/CPU0:router# show memory summary detail location all

Displays information about the memory available on the router after the system image decompresses and loads.

Verify that the expected memory is available or installed. Ensure that all memory regions have adequate free space available.

Step 11

show memory heap summary {job-id | all}
Example:

RP/0/RSP0/CPU0:router# show memory heap summary all

Displays a summary of the information about the heap space. The output displays each process and the amount of memory allocated for each process.

Note The job-id is the output of the show processes command.

Verify if there are any processes using a large amount of memory.

Step 12

top processes
Example:

RP/0/RSP0/CPU0:router# top processes
Press q to exit the command.

Provides a live update of process resource consumption.

Press `M' to sort by memory usage.

Verify that the resource consumption is as expected.

Step 13

show running-config
Example:

RP/0/RSP0/CPU0:router# show running-config

Displays the contents of the currently running configuration.

Verify that the contents of the current running configuration are what you expected.

Step 14

show system verify start
show system verify report
Example:

RP/0/RSP0/CPU0:router# show system verify start
RP/0/RSP0/CPU0:router# show system verify report

A two-step command that produces system reports.

•show system verify start—Starts the system verify process (creates the initial baseline file)

•show system verify report—Generates a report for the system verification process (report of the current status)

The output of the show system verify report command provides a comparison of the system at the time of the show system verify start snapshot and the show system verify report snapshot. The output provides a sanity check of the system provided the show system verify start system snapshot was taken when the system was healthy or before an event.

Verify that the system parameters are as expected.

Step 15

show (ipv4 | ipv6} interface brief
Example:

RP/0/RSP0/CPU0:router# show ipv4 interface brief

Displays the usability status of interfaces.

Verify that all expected interfaces are listed, that they have the correct assigned address, and that they are in the expected states.

Examples

The output from the show platform command indicates that all expected nodes are in the run state. If all nodes in the system are active, the cards should be in the IOS XR RUN and the SPAs should be in the OK state. The example output shows that all expected nodes are in the run state.
RP/0/RSP0/CPU0:router(admin)# show platform
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON
0/FT0/SP        FAN TRAY                  READY            
0/FT1/SP        FAN TRAY                  READY            
0/1/CPU0        A9K-40GE-B                IOS XR RUN       PWR,NSHUT,MON
0/2/CPU0        A9K-SIP-700               IOS XR RUN       PWR,NSHUT,MON
0/2/0           SPA-2XOC48POS/RPR         OK               PWR,NSHUT,MON
0/3/CPU0        A9K-2T20GE-B              IN-RESET         PWR,NSHUT,MON
0/4/CPU0        A9K-8T/4-B                IOS XR RUN       PWR,NSHUT,MON
0/6/CPU0        A9K-4T-B                  IOS XR RUN       PWR,NSHUT,MON
0/PM0/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
0/PM1/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
0/PM2/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
The output from the show version command indicates the version of software being run on the nodes and from which location (disk or network). Check that the expected software version and images are installed. The example output shows that the Cisco IOS XR software version is 4.0.0 and that the installed pie versions are also 4.0.0.
RP/0/RSP0/CPU0:router(admin)# show version
Cisco IOS XR Software, Version 4.0.0[Default]
Copyright (c) 2010 by Cisco Systems, Inc.
ROM: System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],  
router uptime is 1 day, 15 hours, 53 minutes
System image file is "bootflash:disk0/asr9k-os-mbi-4.0.0/mbiasr9k-rp.vm"
cisco ASR9K Series (MPC8641D) processor with 4194304K bytes of memory.
MPC8641D processor at 1333MHz, Revision 2.2
2 Management Ethernet
12 DWDM controller(s)
12 TenGigE
40 GigabitEthernet
2 SONET/SDH
2 Packet over SONET/SDH
219k bytes of non-volatile configuration memory.
975M bytes of compact flash card.
33994M bytes of hard disk.
1605616k bytes of disk0: (Sector size 512 bytes).
1605616k bytes of disk1: (Sector size 512 bytes).
Configuration register on node 0/RSP0/CPU0 is 0x102
Boot device on node 0/RSP0/CPU0 is disk0:
Package active on node 0/RSP0/CPU0:
asr9k-optics-supp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-optics-supp-4.0.0
    Built on Wed Sep  8 16:17:30 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-fwding, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-fwding-4.0.0
    Built on Wed Sep  8 16:12:40 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-cpp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-cpp-4.0.0
    Built on Wed Sep  8 16:13:28 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9K-doc-supp, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9K-doc-supp-4.0.0
    Built on Wed Sep  8 16:16:57 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
asr9k-scfclient, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:asr9k-scfclient-4.0.0
    Built on Wed Sep  8 16:13:26 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
iosxr-security, V 4.0.0[DT_IMAGE], Cisco Systems, at disk0:iosxr-security-4.0.0
    Built on Wed Sep  8 16:16:48 DST 2010
    By sjc5-gf-021 in /auto/ioxbuild8/production/4.0.0.DT_IMAGE/asr9k/workspace for pie
 --More-- 
The output from the show running-config command displays the current running configuration, that is, all of the nondefault commands currently active. Verify that the contents of the current running configuration are as expected.

Tip The output of this command in exec mode is different from the output in admin mode. You should run the command from each of these modes to locate all of the configuration information.
RP/0/RSP0/CPU0:router# show running-config
Building configuration...
!! Last configuration change at 18:56:31 UTC Tue Feb 28 2006 by user_A
!
hostname router
clock timezone PST 8.
logging console informational
telnet vrf default ipv4 server max-servers 100
domain name cisco.com
domain lookup disable
taskgroup default
!
taskgroup igpadmin
 task write rib
 task write isis
 task write ospf
!
taskgroup routeadmin
 task write bgp
 inherit taskgroup igpadmin
!
usergroup default
 taskgroup default
!
usergroup igp_admin
 taskgroup igpadmin
!
usergroup route_admin
 taskgroup routeadmin
!
tacacs-server host 172.29.52.69 port 49
 key 7 060506324F41
aaa group server tacacs+ DOC_LAB_TACACS+
 server 172.29.52.69
 server 172.29.52.68
!
aaa authorization exec LAB_AAA group DOC_LAB_TACACS+ local
aaa authorization exec CONSOLE_AAA group DOC_LAB_TACACS+ none
aaa authorization commands LAB_AAA group DOC_LAB_TACACS+ none
aaa authorization commands CONSOLE_AAA group DOC_LAB_TACACS+ none
aaa authentication login default group DOC_LAB_TACACS+ local
aaa authentication login CONSOLE_AAA group DOC_LAB_TACACS+ local
aaa default-taskgroup default
explicit-path name Primary_GE_Path_to_P19
 index 1 next-address strict ipv4 unicast 10.114.4.44
 index 2 next-address strict ipv4 unicast 10.114.4.11
 index 3 next-address strict ipv4 unicast 10.119.4.11
 index 4 next-address strict ipv4 unicast 10.119.4.19
 index 5 next-address strict ipv4 unicast 10.19.19.19
!
line console
 accounting exec CONSOLE_AAA
 accounting commands CONSOLE_AAA
 authorization exec CONSOLE_AAA
 authorization commands CONSOLE_AAA
 login authentication CONSOLE_AAA
 exec-timeout 600 0
 session-timeout 600
!
line default
 exec-timeout 600 0
 session-timeout 600
!
.
.
.
interface preconfigure GigabitEthernet0/3/0/18
 shutdown
!
interface preconfigure GigabitEthernet0/3/0/19
 shutdown
!
interface preconfigure TenGigE0/3/0/0
 shutdown
!
interface preconfigure TenGigE0/3/0/1
 shutdown
!
router static
 address-family ipv4 unicast
  0.0.0.0/0 172.29.52.1 200
 !
!
router isis 100
 is-type level-2-only
 net 49.0001.0000.0000.0044.00
 nsf cisco
 address-family ipv4 unicast
  metric-style wide
  mpls traffic-eng level-2-only
  mpls traffic-eng router-id Loopback0
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
  !
 !
 interface TenGigE0/4/0/0
  bfd minimum-interval 50
  bfd multiplier 3
  bfd fast-detect ipv4
  address-family ipv4 unicast
  !
 !
!
router ospf 100
 nsr
 router-id 10.144.144.144
 bfd minimum-interval 50
 bfd multiplier 3
 mpls ldp sync
 nsf cisco
 area 0
  mpls ldp sync-igp-shortcuts
  mpls traffic-eng
  interface Loopback0
   passive enable
  !
.
.
.
http server
ssh server vrf default
igmp snooping profile default
 system-ip-address 10.144.144.144
 minimum-version 2
 internal-querier
 tcn query solicit
 ttl-check disable
 router-alert-check disable
!
igmp snooping profile mrouter
 router-guard
 mrouter
!
router pim
 address-family ipv4
  mofrr mofrr-acl
  rp-address 10.11.11.11
  rp-address 10.144.144.144 bidir-acl bidir
  spt-threshold infinity
 !
!
end
RP/0/RSP0/CPU0:router# admin
Wed Oct 27 14:52:07.000 DST
RP/0/RSP0/CPU0:router(admin)# show running-config 
Wed Oct 27 14:52:12.766 DST
Building configuration...
!! IOS XR Admin Configuration 4.0.0
username doclabuser-c
 group root-system
 group cisco-support
 secret 5 $1$RJVQ$6w7saUHgk16v5HXRWEp6m/
!
username doclabuser-r
 group root-system
 secret 5 $1$.uOF$O9N0aRRk.V1qe250IavLw1
!
alias cr copy run disk0a:/usr/base_config_admin                      
alias sa show alias                      
alias sc show config commit list                      
alias sd show diag                      
alias si show install req                      
alias sl show led                      
alias sp show platform                      
alias sr show run                      
alias sv show version                      
alias nda no debug all
end
The output from the show logging command displays the contents of the logging buffer. The output displays details on syslog historical events. Analyze the logged events and the order in which they happened. Check for anything out of the ordinary such as errors, tracebacks, or crashes. Also check for any Severity 1 or Severity 2 errors.
RP/0/RSP0/CPU0:router(admin)# show logging
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
    Console logging: level informational, 693 messages logged
    Monitor logging: level debugging, 0 messages logged
    Trap logging: level informational, 0 messages logged
    Buffer logging: level debugging, 4467 messages logged
Log Buffer (307200 bytes):
LC/0/3/CPU0:Sep 13 23:58:03.272 : pfm_node_lc[230]: %PLATFORM-NP-0-NP_INIT_FAILURE : 
Set|prm_server[110670]|Network Processor Unit(0x1007000)|Persistent Initialization 
Failure.  
LC/0/3/CPU0:Sep 13 23:58:03.276 : pfm_node_lc[230]: %PLATFORM-PFM-0-CARD_RESET_REQ : 
pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 110670 
(prm_server), Fault Sev: 0, Target node: 0/3/CPU0, CompId: 0x1f, Device Handle: 0x1007000, 
CondID: 1027, Fault Reason: Persistent Initialization Failure.  
LC/0/3/CPU0:Sep 13 23:58:03.276 : syslog_dev[85]: pfm_node_lc[230]: Request Graceful 
Reboot via Sysmgr: Reason: pfm_dev_sm_perform_recovery_action, Card reset requested by: 
Process ID: 110670 (prm_server), Fault Sev: 0, Target node: 0/3/CPU0, CompId: 0x1f, Device 
Handle: 0x1007000, CondID: 1027, Fault Reason: Persistent Initialization Failure.  
LC/0/3/CPU0:Sep 13 23:58:03.277 : sysmgr[87]: %OS-SYSMGR-2-REBOOT : reboot required, 
process (pfm_node_lc) reason (pfm_dev_sm_perform_recovery_action, Card reset requested by: 
Process ID: 110670 (prm_server), Fault Sev: 0, Target node: 0/3/CPU0, CompId: 0x1f, Device 
Handle: 0x1007000, CondID: 1027, Fault Reason: Persistent Initialization Failure. ) 
LC/0/3/CPU0:Sep 13 23:58:03.467 : sysmgr[87]: %OS-SYSMGR-3-ERROR : 
sysmgr_shutdown_cleanup_handler: shutdown script execution timed-out! Node will reset  
LC/0/3/CPU0:Sep 13 23:58:03.467 : sysmgr[87]: %OS-SYSMGR-7-DEBUG : 
sysmgr_shutdown_cleanup_handler: shutdown script execution timed-out! Node will reset  
LC/0/3/CPU0:Sep 13 23:58:03.468 : sysmgr[87]: %OS-SYSMGR-3-ERROR : 
sysmgr_shutdown_cleanup_handler: shutdown triggered by (pfm_node_lc) did not complete in 
45 seconds, shutting down  
RP/0/RSP0/CPU0:Sep 13 23:58:16.859 : shelfmgr[299]: 
%PLATFORM-SHELFMGR-0-MAX_RESET_BRINGDOWN : Can not boot node 0/3/CPU0 A9K-2T20GE-B due to 
multiple resets, putting it IN_RESET state. The probable cause is an unexpected event on 
the node or a failure in communication with the node. Please refer to the Cisco ASR 9000 
System Error Message Reference Guide for further information if needed. 
 --More-- 
The output from the show environment command displays environmental monitor parameters for the system. Verify that the environment parameters are as expected. Environment parameter anomalies are logged in the syslog, so if an environment parameter displayed in the show environment command output is not as expected, check the syslog using the show logging command. The syslog provides details on any logged problems.
RP/0/RSP0/CPU0:router(admin)# show environment 
Wed Sep 15 09:48:27.178 DST
Temperature Information
---------------------------------------------
R/S/I   Modules Sensor                  (deg C)
0/1/*
        host    Inlet0                  36.4
        host    Hotspot0                46.7
0/2/*
        spa0    InletTemp               35.5
        spa0    Hotspot                 35.5
        host    Inlet0                  34.5
        host    Hotspot0                61.0
0/3/*
        host    Inlet0                  31.1
        host    Hotspot0                32.5
0/RSP0/*
        host    Inlet0                  31.3
        host    Hotspot0                42.0
0/4/*
        host    Inlet0                  34.9
        host    Hotspot0                45.2
0/6/*
        host    Inlet0                  38.0
        host    Hotspot0                48.9
0/FT0/*
        host    Inlet0                  31.9
        host    Hotspot0                33.5
0/FT1/*
        host    Inlet0                  32.3
        host    Hotspot0                35.3
Voltage Information
---------------------------------------------
R/S/I   Modules Sensor                   (mV)   Margin
0/1/*
        host    IBV                     10592   n/a
        host    5.0V                     4925   n/a
        host    VP3P3_CAN                3289   n/a
        host    3.3V                     3302   n/a
        host    2.5V                     2516   n/a
        host    1.8VB                    1812   n/a
        host    1.2VB                    1193   n/a
        host    1.8VA                    1806   n/a
        host    0.9VB                     886   n/a
        host    1.2V_LDO_BRG0            1193   n/a
        host    1.2V_LDO_BRG1            1195   n/a
        host    1.8VC                    1811   n/a
        host    1.5VB                    1505   n/a
        host    1.5VA                    1503   n/a
        host    1.1V(1.05V_CPU)          1053   n/a
        host    0.75VA                    752   n/a
        host    0.75VB_0.75VC             754   n/a
        host    1.1VB                    1103   n/a
        host    1.2V_TCAM0               1003   n/a
        host    1.2V_TCAM1               1000   n/a
        host    1.0V_Bridge_LDO           999   n/a
        host    1.0VB                    1042   n/a
        host    0.75VD_and_0.75VE         752   n/a
        host    1.2V_TCAM2               1006   n/a
        host    1.2V_TCAM3               1002   n/a
        host    1.5VC                    1504   n/a
        host    1.8VD                    1804   n/a
        host    1.1VC                    1100   n/a
        host    ZARLINK_3.3V             3272   n/a
        host    ZARLINK_1.8V             1807   n/a
        host    1.2V_DB                  1195   n/a
        host    3.3V_DB                  3318   n/a
        host    2.5V_DB                  2535   n/a
        host    1.5V_DB                  1509   n/a
.
.
.
LED Information
---------------------------------------------
R/S/I   Modules LED             Status
0/RSP0/*
        host    Critical-Alarm  Off
        host    Major-Alarm     Off
        host    Minor-Alarm     Off
        host    ACO             Off
        host    Fail            Off
Fan Information
---------------------------------------------
  Fan speed (rpm):
        FAN0    FAN1    FAN2    FAN3    FAN4    FAN5    FAN6    FAN7    FAN8    FAN9    
FAN10   FAN11
0/FT0/*   
        3510    3480    3510    3570    3540    3510    3510    3480    3540    3540    
3480    3480
0/FT1/*
        3510    3510    3510    3540    3480    3510    3390    3510    3510    3540    
3450    3480
Power Supply Information
---------------------------------------------
R/S/I   Modules         Capacity        Status
                        (W)
0/PM0/*
        host    PM      3000            Ok
0/PM1/*
        host    PM      3000            Ok
0/PM2/*
        host    PM      3000            Ok
R/S/I   Power Draw      Voltage         Current
        (W)             (V)             (A)
0/PM0/* 622.9           53.7            11.6
0/PM1/* 774.7           53.8            14.4
0/PM2/* 0.0             54.1            0.0
--------------
Total:  1397.6
Power Shelves Type: AC
Total Power Capacity:                           9000W
Usable Power Capacity:                          9000W
Supply Failure Protected Capacity:              6000W
Feed Failure Protected Capacity:                3000W
Worst Case Power Used:                          3170W
 Slot                                                      Max Watts
 ----                                                      ---------
 0/1/CPU0                                                        350
 0/2/CPU0                                                        450
 0/RSP0/CPU0                                                     235
 0/RSP1/CPU0                                                     235  (default)
 0/4/CPU0                                                        350
 0/6/CPU0                                                        350
 0/FT0/SP                                                        600
 0/FT1/SP                                                        600
Worst Case Power Available:             5830W
Supply Protected Capacity Available:    2830W
Feed Protected Capacity Available:      Not Protected
The output from the show context command displays core dump context information. See the "show context Command" section for more information on the show context command output.
RP/0/RSP0/CPU0:router# show context
node:      node0_1_CPU0
------------------------------------------------------------------
Crashed pid = 61524 (pkg/bin/tcam_mgr)
Crashed tid = 1
Crash time: Wed Apr 05, 2006: 18:27:26
Core for process at harddisk:/dumper/first.tcam_mgr.abort.node0_1_CPU0.ppc.Z
    Stack Trace
#0 0xfc1d3fa0
#1 0xfc1c6340
#2 0xfc1c5364
#3 0xfc1c542c
#4 0x48210930
#5 0x482110b8
#6 0x48212ba4
#7 0x48203dd8
#8 0x4820c61c
#9 0xfc1557ec
#10 0xfc15573c
#11 0xfc152fb8
#12 0x4820d140
                Registers info 
            r0       r1       r2       r3
  R0   00000000  481ff7b0  4824a55c  00000000  
            r4       r5       r6       r7
  R4   0000f054  00000001  00000006  00000000  
            r8       r9      r10      r11
  R8   00000000  fc220000  481fffc0  00000000  
           r12      r13      r14      r15
  R12  4823be90  4824a4a0  48230000  00000000  
           r16      r17      r18      r19
  R16  00000048  00000001  00000019  48256520  
           r20      r21      r22      r23
  R20  00000000  00000000  00000003  00000045  
           r24      r25      r26      r27
  R24  00000003  00000000  00000003  4825dc34  
           r28      r29      r30      r31
  R28  00000006  0000f054  48254064  481ff810  
           cnt       lr      msr       pc
  R32  00000000  fc1c6340  0000d932  fc1d3fa0  
           cnd      xer
  R36  28004024  00000008  
                      DLL Info 
DLL path    Text addr.  Text size   Data addr. Data size   Version
/hfr-os-3.3.90/lib/libinfra.dll  0xfc142000  0x00034200  0xfc1343b8  0x00000bbc 
       0
/lib/libc.dll  0xfc1a8000  0x00079dd8  0xfc222000  0x00002000        0
    Crash Package Infomation
Package: hfr-mgbl, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-mcast, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-mpls, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-rout, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-k9sec, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-lc, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/works
pace for c2.95.3-p8
Package: hfr-fwdg, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-admin, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-base, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-os-mbi, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/w
orkspace for c2.95.3-p8
node:      node0_6_CPU0
------------------------------------------------------------------
node:      node0_RSP0_CPU0
------------------------------------------------------------------
node:      node0_RSP1_CPU0
------------------------------------------------------------------
The example output shows that the pkg/bin/tcam_mgr process crashed.

The output from the show memory command displays information about the memory available on the router after the system image decompresses and loads. Verify that the expected memory is available or installed. Ensure that all memory regions have adequate free space available. The example output shows that there is 2.003 gigabits of application memory available.
RP/0/RSP0/CPU0:router# show memory summary detail location all
Physical Memory: 4.000G total (2.003G available)
 Application Memory : 3.826G (2.003G available)
 Image: 48.725M (bootram: 48.725M)
 Reserved: 128.000M, IOMem: 1.980G, flashfsys: 0
 Shared window mfwdv6: 449.910K
 Shared window mfwd_info: 701.910K
 Shared window soasync-app: 242.402K
 Shared window soasync: 242.402K
 Shared window li: 3.359K
 Shared window ipv4_fib: 1.003M
 Shared window l2fib: 2.425M
 Shared window statsd_db: 67.386K
 Shared window mgid: 587.390K
 Shared window ifc-protomax: 1.290M
 Shared window ifc-mpls: 7.981M
 Shared window ifc-ipv6: 7.212M
 Shared window ifc-ipv4: 11.286M
 Shared window infra_statsd: 3.402K
 Shared window im_rd: 1.104M
 Shared window im_db: 1.204M
 Shared window infra_ital: 67.316K
 Shared window netio_fwd: 292
 Shared window vkg_bmp_adj: 211.371K
 Shared window aib: 623.375K
 Shared window rspp_ma: 3.351K
 Shared window im_rules: 293.308K
 Shared window aaa: 67.382K
 Shared window pfm_node: 131.304K
 Shared window atc_cache: 35.359K
 Shared window spp: 619.312K
 Shared window qad: 134.707K
 Shared window pcie-server: 43.246K
 Total shared window: 37.931M
 Allocated Memory: 378.742M
 Program Text: 46.175M
 Program Data: 37.796M
 Program Stack: 16.539M
The show system verify start command starts the system verification process and the show system verify report generates the output from the system verification process. The output allows you to verify that the system parameters are as expected.
RP/0/RSP0/CPU0:router# show system verify start
Storing initial router status ... 
done.
The example output compares the system from the time the show system verify start command took the first snapshot to the snapshot taken of the system when the show system verify report command took the second snapshot and generated the comparison. If there are no changes, [OK] is displayed. If there are changes between the first and second snapshot, the specific change is noted and marked with [WARNING] or [FAIL].

RP/0/RSP0/CPU0:router# show system verify report
Getting current router status ... 
System Verification Report
==========================
- Verifying Memory Usage
- Verified Memory Usage                                 : [OK]     
- Verifying CPU Usage
- Verified CPU Usage                                    : [OK]     
- Verifying Blocked Processes
- Verified Blocked Processes                            : [OK]     
- Verifying Aborted Processes
- Verified Aborted Processes                            : [OK]     
- Verifying Crashed Processes
- Verified Crashed Processes                            : [OK]     
- Verifying LC Status
- Verified LC Status                                    : [OK]     
- Verifying QNET Status
Unable to get current LC status info
- Verified QNET Status                                  : [FAIL]   
- Verifying GSP Fabric Status
- Verified GSP Fabric Status                            : [OK]     
- Verifying GSP Ethernet Status
- Verified GSP Ethernet Status                          : [OK]     
- Verifying POS interface Status
- Verified POS interface Status                         : [OK]     
- Verifying TenGigE interface Status
- Verified TenGigE interface Status                     : [OK]     
- Verifying TCP statistics
- Verified TCP statistics                               : [OK]     
- Verifying UDP statistics
  tcp_udp_raw WARNING messages for router 
  UDP Packets sent has not increased during this period.
- Verified UDP statistics                               : [WARNING]
- Verifying RAW statistics
- Verified RAW statistics                               : [OK]     
- Verifying RIB Status
- Verified RIB Status                                   : [OK]     
- Verifying CEF Status
- Verified CEF Status                                   : [OK]     
- Verifying CEF Consistency Status
- Verified CEF Consistency Status                       : [OK]     
- Verifying BGP Status
- Verified BGP Status                                   : [OK]     
- Verifying ISIS Status
- Verified ISIS Status                                  : [OK]     
- Verifying OSPF Status
- Verified OSPF Status                                  : [OK]     
- Verifying Syslog Messages
- Verified Syslog Messages                              : [OK]     
System may not be stable. Please look into WARNING messages.
The show interface brief command displays the usability status of the configured interfaces. Verify that all expected interfaces are listed. For an interface to be usable, both the interface hardware (Status) and line protocol must be up. The protocol is Up if the interface can provide two-way communication. The example output displays IP addresses, status, and protocol status for each interface. The output shows that all assigned interfaces (interfaces that are configured with IP addresses) have an interface hardware status and line protocol status of Up.
RP/0/RSP0/CPU0:router# show ipv4 interface brief 
Interface                      IP-Address      Status                Protocol
Bundle-Ether16                 unassigned      Up                    Up      
Bundle-Ether16.160             unassigned      Up                    Up      
Bundle-Ether16.161             unassigned      Up                    Up      
Bundle-Ether16.162             10.194.8.44     Up                    Up      
Bundle-Ether16.163             10.194.12.44    Up                    Up      
Loopback0                      10.144.144.144  Up                    Up      
tunnel-te44190                 10.144.144.144  Up                    Up      
tunnel-te44192                 10.144.144.144  Up                    Up      
tunnel-te44194                 10.144.144.144  Up                    Up      
tunnel-te44196                 10.144.144.144  Up                    Up      
MgmtEth0/RSP0/CPU0/0           172.29.52.137   Up                    Up      
MgmtEth0/RSP0/CPU0/1           unassigned      Shutdown              Down    
GigabitEthernet0/1/0/0         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/1         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/2         10.147.4.44     Up                    Up      
GigabitEthernet0/1/0/3         unassigned      Up                    Up      
GigabitEthernet0/1/0/3.160     unassigned      Up                    Up      
GigabitEthernet0/1/0/3.161     unassigned      Up                    Up      
GigabitEthernet0/1/0/3.185     unassigned      Up                    Up      
GigabitEthernet0/1/0/3.189     unassigned      Up                    Up      
GigabitEthernet0/1/0/3.215     unassigned      Up                    Up      
GigabitEthernet0/1/0/4         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/5         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/6         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/7         unassigned      Up                    Up      
GigabitEthernet0/1/0/7.185     unassigned      Up                    Up      
GigabitEthernet0/1/0/7.187     unassigned      Up                    Up      
GigabitEthernet0/1/0/7.189     unassigned      Up                    Up      
GigabitEthernet0/1/0/7.210     unassigned      Up                    Up      
GigabitEthernet0/1/0/7.211     unassigned      Up                    Up      
GigabitEthernet0/1/0/7.215     unassigned      Up                    Up      
GigabitEthernet0/1/0/8         10.146.4.44     Up                    Up      
GigabitEthernet0/1/0/9         unassigned      Shutdown              Down    
GigabitEthernet0/1/0/10        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/11        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/12        10.194.16.44    Up                    Up      
GigabitEthernet0/1/0/13        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/14        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/15        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/16        unassigned      Up                    Up      
GigabitEthernet0/1/0/17        unassigned      Up                    Up      
GigabitEthernet0/1/0/18        10.194.4.44     Up                    Up      
GigabitEthernet0/1/0/19        unassigned      Up                    Up      
GigabitEthernet0/1/0/19.2127   unassigned      Up                    Up      
GigabitEthernet0/1/0/19.2130   unassigned      Up                    Up      
GigabitEthernet0/1/0/20        unassigned      Up                    Up      
GigabitEthernet0/1/0/20.2125   unassigned      Up                    Up      
GigabitEthernet0/1/0/21        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/22        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/23        10.114.4.44     Up                    Up      
GigabitEthernet0/1/0/24        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/25        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/26        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/27        10.145.4.44     Up                    Up      
GigabitEthernet0/1/0/28        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/29        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/30        unassigned      Up                    Up      
GigabitEthernet0/1/0/30.215    unassigned      Up                    Up      
GigabitEthernet0/1/0/31        unassigned      Up                    Up      
GigabitEthernet0/1/0/32        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/33        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/34        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/35        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/36        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/37        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/38        unassigned      Shutdown              Down    
GigabitEthernet0/1/0/39        unassigned      Shutdown              Down    
POS0/2/0/0                     unassigned      Up                    Up      
POS0/2/0/1                     unassigned      Up                    Up      
TenGigE0/4/0/0                 10.114.8.44     Up                    Up      
TenGigE0/4/0/1                 unassigned      Shutdown              Down    
TenGigE0/4/0/2                 unassigned      Shutdown              Down    
TenGigE0/4/0/3                 unassigned      Shutdown              Down    
TenGigE0/4/0/4                 unassigned      Shutdown              Down    
TenGigE0/4/0/5                 unassigned      Shutdown              Down    
TenGigE0/4/0/6                 unassigned      Shutdown              Down    
TenGigE0/4/0/7                 unassigned      Shutdown              Down    
TenGigE0/6/0/0                 unassigned      Shutdown              Down    
TenGigE0/6/0/1                 unassigned      Shutdown              Down    
TenGigE0/6/0/2                 unassigned      Shutdown              Down    
TenGigE0/6/0/3                 unassigned      Shutdown              Down 
Troubleshooting the Backplane Ethernet Control System

This section describes techniques that you can use to troubleshoot the control plane Ethernet network on routers using Cisco IOS XR software. The system control plane Ethernet network is used for processes on different devices to communicate for functions such as system device discovery, image transfers, heartbeat messages, alarms, and configuration management.

All devices in a system using Cisco IOS XR software connect to the system control plane Ethernet network, also called the Ethernet over backplane channel (EOBC). The control plane is provided using Gigabit Ethernet (GE) links between nodes. The GE links are internal to the chassis and cannot be removed.

Figure 1-1 shows the control plane Ethernet network (the dotted line in the drawing).

Figure 1-1 Cisco ASR 9000 Aggregation Services Router Control Ethernet Topology

To verify and troubleshoot booting of the system control plane Ethernet network, perform the following procedure.

SUMMARY STEPS

1. show platform

2. show controllers backplane ethernet clients all location node-id

3. show controllers backplane ethernet clients 18 statistics location node-id

4. Contact Cisco Technical Support if the problem is not resolved.

DETAILED STEPS

Command or Action

Purpose

Step 1

show platform
Example:

RP/0/RSP0/CPU0:router# show platform

Displays information about the status of cards and modules installed in the router.

Verify that the expected nodes display IOS XR RUN under the State column of the command output.

Step 2

show controllers backplane ethernet clients all location node-id
Example:

RP/0/RSP0/CPU0:router# show controllers backplane ethernet clients all location 0/RSP0/CPU0

Displays information about all local client applications. Each row contains the client Ethernet server ID and the client process ID (PID).

The system allows client processes to send and receive packets over the control Ethernet. It uses client IDs to demultiplex packets that arrive at the node.

Two client IDs in the output are important for troubleshooting boot problems:

•Client Ethernet server ID 18—used for boot requests

•Client Ethernet server ID 22—used for heartbeats

Step 3

show controllers backplane ethernet clients 18 statistics location node-id
Example:

RP/0/RSP0/CPU0:router# show controllers backplane ethernet clients 18 statistics location 0/RSP0/CPU0

Displays a list of client statistics for the specified client ID.

Check the values for:

•Packets input

•Packets delivered

If they contain values other than 0, boot requests have been received and replies have been sent (packets output).

If they contain values of 0, check the system control plane Ethernet network physical connectivity.

If there are no problems with the physical connectivity, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section.

Step 4

Contact Cisco Technical Support if the problem is not resolved.

If the problem is not resolved, contact Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section.

Examples

The output from the show platform command indicates that all expected nodes are in the run state. If all nodes in the system are active, the cards should be in the IOS XR RUN and the SPAs should be in the OK state. The example output shows that all expected nodes are in the run state.
RP/0/RSP0/CPU0:router# show platform
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON
0/1/CPU0        A9K-40GE-B                IOS XR RUN       PWR,NSHUT,MON
0/2/CPU0        A9K-SIP-700               IOS XR RUN       PWR,NSHUT,MON
0/2/0           SPA-2XOC48POS/RPR         OK               PWR,NSHUT,MON
0/3/CPU0        A9K-2T20GE-B              IN-RESET         PWR,NSHUT,MON
0/4/CPU0        A9K-8T/4-B                IOS XR RUN       PWR,NSHUT,MON
0/6/CPU0        A9K-4T-B                  IOS XR RUN       PWR,NSHUT,MON
The following example shows the current state of each Ethernet server client.
RP/0/RSP0/CPU0:router# show controllers backplane ethernet clients all location 
0/RSP0/CPU0
    Intf           Client ethernet         Client              Description
    Name              server id            Process Id
------------------------------------------------------------------------------
GE0_RSP0_CPU0               1                110639          QNX network manager
                           2                221272               Group services
                           3                     0          Reserved for Attach
                           4                221274            Plugin controller
                           5                     0                Designated SC
                           6                     0              ASR9K H/W diags
                           7                221279            IP packet handler
                           8                217149        Redundancy controller
                           9                     0        ASR9K Virtual console
                          10                110638       ASR9K Virtual terminal
                          11                 49196        Control ethernet echo
                          12                     0       Control eth echo reply
                          13                221274  Card Configuration Protocol
                          14                     0          Reserved for Attach
                          15                     0           Chassis controller
                          16                     0            Forwarding driver
                          17                     0                    MBI hello
                          18                110640       MBI Boot Server Source
                          19                     0                HSR ES client
                          20                     0  Packets for ethernet server
                          21                     0         For Diag application
                          22                233589            heartbeat request
                          23                     0              heartbeat reply
                          24                221275             Async IPC client
                          25                     0           Test application 1
                          26                     0           Test application 2
                          27                     0      Test client out-of-band
The following example shows that there are 18 nodes in the run state, which means that 12 boot requests have been received by eth_server and 12 replies have been sent:
RP/0/RSP0/CPU0:router# show controllers backplane ethernet clients 18 statistics location 
0/RSP1/CPU0
  Client ShelfMgr, ES Client Id 18, PID 53338 running on FastEthernet0_RSP0_0 
    12 packets input, 8676 bytes
    12 packets delivered, 8676 bytes
    0 packets discarded (0 bytes) in garbage collection
    0 (0 bytes) unicast packets filtered
    0 (0 bytes) multicast packets filtered
    0 (0 bytes) buffer mgmt policy discards
    0 (0 bytes) locking error discards
    12 packets output, 8676 bytes, 0 could not be transmitted
Basic Cisco IOS XR Verification and Troubleshooting Commands

The following commands are used to collect information to aid in verifying the system and troubleshooting problems:

•man Command

•describe Command

•show platform Command

•top Command

•show context Command

•show users Command

•show history Command

•show configuration Command

man Command

The man command provides online help for standard Cisco IOS XR CLI commands using manual (man) pages. The command is used to display the manual pages for a specific command based on the command name, a feature, or a keyword. Each man page contains the command name, syntax, command mode, usage, examples, and related commands.

Note To run the man command, you must have the Cisco IOS XR Documentation Package, "asr9k-doc.pie-4.0.0, .man pages for Cisco IOS XR software on the Cisco ASR 9000 Series Router chassis," loaded. If you are running a release later than 4.0.0, the package installation envelope (PIE) name might be different. For the appropriate PIE name and an explanation of PIE installation, see the "Upgrading Cisco IOS XR Software" section of the Release Notes document for the IOS XR version you are running

The following example shows the output from the man command show users command.
RP/0/RSP0/CPU0:router# man command show users
Tue Sep 14 14:39:16.409 DST
Building index table...
Total Number of Command Entries:2726 
[OK]
DESCRIPTION
Displays information about the active lines on the router.
  To display information about the active lines on the router, use the show users
  command in EXEC mode.
  show users
SYNTAX DESCRIPTION
  This command has no arguments or keywords.
COMMAND DEFAULT
  No default behavior or values
COMMAND MODES
  EXEC
COMMAND HISTORY
  Release 
          Modification
  Release 3.7.2
          This command was introduced.
USAGE GUIDELINES
  To use this command, you must be in a user group associated with a task group
  that includes the proper task IDs. If you suspect user group assignment is
  preventing you from using a command, contact your AAA administrator for
  assistance.
  Use the show users command to display the line number, connection name, idle
  time, hosts, and terminal location. An asterisk (*) indicates the current
  terminal session.
===================================================================================
Note:
  To display all user groups and task IDs associated with the currently logged-in
  user, use the show user command in EXEC mode. See the Authentication,
  Authorization, and Accounting Commands on Cisco IOS XR Software module in
  Cisco^B^`ASR^B^`9000 Series Aggregation Services Router System Security Command
  Reference.
===================================================================================
TASK ID
  Task ID
          Operations
  tty-access
          read
EXAMPLES
  The following example shows sample output identifying an active vty terminal
  session:
 * * * * * * * * * * * * * * START OF LISTING * * * * * * * * * * * * * * 
RP/0/RSP0/CPU0:router# show users
     Line            User                 Service  Conns   Idle        Location
     con0_RSP0_CPU0   cisco                hardware     0  18:33:48     
     vty0            cisco                telnet       0  00:30:36     10.33.54.132
  *  vty1            cisco                telnet       0  00:00:00     10.33.54.132  
 * * * * * * * * * * * * * *  END OF LISTING * * * * * * * * * * * * * * *  
  Table 1 describes the significant fields shown in the display.
show users Field Descriptions
  Field
          Description
  Line
          All current connections. An asterisk (*) indicates the active connection.
  User
          Username of the user logged into the line.
  Service
          Physical or remote login service used.
  Conns
          Number of outgoing connections.
  Idle
          Interval (in hours:minutes:seconds) since last keystroke.
  Location
          IP address of remote login host. For local (physical) terminal
          connections, this field is blank.
RELATED COMMANDS
  Command
          Description
  show line Displays the parameters of terminal lines.
          Displays the parameters of a terminal line.
  show user
          Displays all user groups and task IDs associated with the currently
          logged-in user.
describe Command

The describe command provides a preview of a command without actually implementing it. This command lists information about the package, component, and task ID for a specific command. You must be in the appropriate configuration mode for the specific command. For example, to display the package, component, and task ID information for the router bgp 1 command, you must be in global configuration mode.

The following example shows the output from the describe router bgp 1 command.
RP/0/RSP0/CPU0:router(config)#describe router bgp 1
Package:
    iosxr-routing
        iosxr-routing V4.0.0[DT_IMAGE]  IOS-XR Routing Package Definition
        Vendor : Cisco Systems
        Desc   : IOS-XR Routing Package Definition
        Build  : Built on Wed Sep  8 16:10:14 DST 2010
        Source : By router-021 in /files/4.0.0.DT_IMAGE/asr9k/workspace fo8
        Card(s): RP, NP24-4x10GE, NP24-40x1GE, NP40-40x1GE, NP40-4x10GE, NP40-8x10GE, 
NP40-2_20_COMBO, NP80-8x10GE, NP80-16x10GE, A9K-SIP-700, A9K-SIP-500
        Restart information:
          Default:
            parallel impacted processes restart
        Size Compressed/Uncompressed: 8556KB/22MB (37%)
Component:
    ipv4-bgp V[ci-401/18]  IPv4 Border Gateway Protocol (BGP)
User needs ALL of the following taskids:
        bgp (READ WRITE) 
show platform Command

The show platform command displays a high level overview of the entire physical system. Use the show platform command in administration mode to display a summary of the nodes in the system, including node type and status.

The following example shows the output from the show platform command in administration mode.
RP/0/RSP0/CPU0:router(admin)#show platform
Tue Sep 14 14:52:52.558 DST
Node            Type                      State            Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0     A9K-RSP-4G(Active)        IOS XR RUN       PWR,NSHUT,MON
0/FT0/SP        FAN TRAY                  READY            
0/FT1/SP        FAN TRAY                  READY            
0/1/CPU0        A9K-40GE-B                IOS XR RUN       PWR,NSHUT,MON
0/2/CPU0        A9K-SIP-700               IOS XR RUN       PWR,NSHUT,MON
0/2/0           SPA-2XOC48POS/RPR         OK               PWR,NSHUT,MON
0/3/CPU0        A9K-2T20GE-B              IN-RESET         PWR,NSHUT,MON
0/4/CPU0        A9K-8T/4-B                IOS XR RUN       PWR,NSHUT,MON
0/6/CPU0        A9K-4T-B                  IOS XR RUN       PWR,NSHUT,MON
0/PM0/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
0/PM1/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
0/PM2/SP        A9K-3KW-AC                READY            PWR,NSHUT,MON
top Command

The top command is used to monitor CPU usage on the system through interactive process statistics.

The following example show the output from the top command.
RP/0/RSP0/CPU0:router# top 
Computing times...
224 processes; 803 threads;
CPU states: 93.6% idle, 3.6% user, 2.7% kernel
Memory: 4096M total, 3504M avail, page size 4K
JID   TID LAST_CPU PRI STATE  HH:MM:SS      CPU  COMMAND
    91     1   0      10 Rcv     0:16:29     2.25% spp
   256    10   0      10 Rcv     0:19:43     2.16% netio
   340    10   0      10 Rcv     0:03:21     0.13% udp
   294     1   1      10 Rcv     0:03:24     0.09% sc
     1    12   1      10 Rcv     0:00:03     0.09% procnto-600-smp-instr
 65816     1   0      10 Rply    0:00:00     0.07% top
     1    11   1      10 Run     0:00:19     0.05% procnto-600-smp-instr
    60     5   1      10 Rcv     0:00:43     0.02% eth_server
   256    11   1      10 Rcv     0:00:15     0.02% netio
   340    14   1      10 Rcv     0:01:06     0.01% udp
Press q to exit the command.

show context Command

The show context command displays core dump context information for the last ten core dumps. The command output is used for post-analysis in the debugging of processes (determine if any process crashes have occurred).

If there are no crashed processes, the show context command displays no output for each node. The following example shows the output of the show context command with no crashed processes.
RP/0/RSP1/CPU0:router# show context 
node:      node0_1_CPU0
------------------------------------------------------------------
node:      node0_6_CPU0
------------------------------------------------------------------
node:      node0_RSP0_CPU0
------------------------------------------------------------------
node:      node0_RSP1_CPU0
------------------------------------------------------------------
The following example shows the output from the show context command where there is a crashed process.
RP/0/RSP1/CPU0:router# show context 
node:      node0_1_CPU0
------------------------------------------------------------------
Crashed pid = 61524 (pkg/bin/tcam_mgr)
Crashed tid = 1
Crash time: Wed Apr 05, 2006: 18:27:26
Core for process at harddisk:/dumper/first.tcam_mgr.abort.node0_1_CPU0.ppc.Z
    Stack Trace
#0 0xfc1d3fa0
#1 0xfc1c6340
#2 0xfc1c5364
#3 0xfc1c542c
#4 0x48210930
#5 0x482110b8
#6 0x48212ba4
#7 0x48203dd8
#8 0x4820c61c
#9 0xfc1557ec
#10 0xfc15573c
#11 0xfc152fb8
#12 0x4820d140
                Registers info 
            r0       r1       r2       r3
  R0   00000000  481ff7b0  4824a55c  00000000  
            r4       r5       r6       r7
  R4   0000f054  00000001  00000006  00000000  
            r8       r9      r10      r11
  R8   00000000  fc220000  481fffc0  00000000  
           r12      r13      r14      r15
  R12  4823be90  4824a4a0  48230000  00000000  
           r16      r17      r18      r19
  R16  00000048  00000001  00000019  48256520  
           r20      r21      r22      r23
  R20  00000000  00000000  00000003  00000045  
           r24      r25      r26      r27
  R24  00000003  00000000  00000003  4825dc34  
           r28      r29      r30      r31
  R28  00000006  0000f054  48254064  481ff810  
           cnt       lr      msr       pc
  R32  00000000  fc1c6340  0000d932  fc1d3fa0  
           cnd      xer
  R36  28004024  00000008  
                      DLL Info 
DLL path    Text addr.  Text size   Data addr. Data size   Version
/hfr-os-3.3.90/lib/libinfra.dll  0xfc142000  0x00034200  0xfc1343b8  0x00000bbc 
       0
/lib/libc.dll  0xfc1a8000  0x00079dd8  0xfc222000  0x00002000        0
    Crash Package Infomation
Package: hfr-mgbl, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-mcast, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-mpls, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-rout, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-k9sec, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-lc, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/works
pace for c2.95.3-p8
Package: hfr-fwdg, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-admin, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wo
rkspace for c2.95.3-p8
Package: hfr-base, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/wor
kspace for c2.95.3-p8
Package: hfr-os-mbi, Source: By edde-bld1 in /vws/aga/production/3.3.90.1I/hfr/w
orkspace for c2.95.3-p8
node:      node0_6_CPU0
------------------------------------------------------------------
node:      node0_RSP0_CPU0
------------------------------------------------------------------
node:      node0_RSP1_CPU0
------------------------------------------------------------------
Use the show context command to locate the core dump file path. For example, the core dump file path shown in the command output is: harddisk:/dumper/first.tcam_mgr.abort.node0_1_CPU0.ppc.Z. The command output shows a crash on a node. The process is pkg/bin/tcam_mgr.

Collect the following information and send it to Cisco Technical Support. For Cisco Technical Support contact information, see the "Obtaining Documentation and Submitting a Service Request" section.

•ppc.Z file—This file contains the binary core dump information. Use the path listed in the command output to copy the contents of the ppc.Z file. The path shown in the command output is: harddisk:/dumper/first.tcam_mgr.abort.node0_1_CPU0.ppc.Z

•ppc.txt file—This file contains content on the core dump similar to the show context command output. Use the path listed in the command output to copy the contents of the ppc.txt file. The path shown in the command output is: harddisk:/dumper/first.tcam_mgr.abort.node0_1_CPU0.ppc.txt

•Collect the show version or show install active command output.

show users Command

The show users command displays information on active lines on the router including the line number, user, service, number of connections, idle time, and remote terminal location. An asterisk (*) indicates the current terminal session.

The following example shows the output from the show users command.
RP/0/RSP0/CPU0:router# show users
   Line            User                 Service  Conns   Idle        Location
*  vty0            User_A               telnet       0  00:00:00     161.44.1925
   vty1            User-B               telnet       0  00:00:03     161.44.1929
show history Command

The show history command displays a history of the command entered for the current command mode. You can enter the show history command to display a history of commands entered in EXEC, ADMIN, or CONFIG mode.

Examples
RP/0/RSP0/CPU0:router# show history
RP/0/RSP0/CPU0:router(admin)# show history
RP/0/RSP0/CPU0:router(config)# show history
The following example shows the output from the show history command in EXEC mode:
RP/0/RSP0/CPU0:router# show history
RP/0/RSP0/CPU0:router# show history 
Thu Oct 28 14:20:50.328 DST
  show vrrp interface gigabitEthernet 0/1/0/0
  show vrrp interface brief
  show vrrp brief
  show vrrp detail
  show vrrp interface gigabitEthernet 0/1/0/1 
  show vrrp interface gigabitEthernet 0/1/0/1 statistics all
The detailed history provides a timestamp also:
RP/0/RSP0/CPU0:router# show history detail 
Thu Oct 28 14:26:06.199 DST
     1 Thu Oct 28 14:02:25.310 show vrrp interface gigabitEthernet 0/1/0/0
     2 Thu Oct 28 14:03:34.854 show vrrp interface brief
     3 Thu Oct 28 14:04:02.042 show vrrp brief
     4 Thu Oct 28 14:04:08.167 show vrrp detail
     5 Thu Oct 28 14:08:25.180 show vrrp interface gigabitEthernet 0/1/0/1 
     6 Thu Oct 28 14:09:03.402 show vrrp interface gigabitEthernet 0/1/0/1 statistics all
show configuration Command

The show configuration command displays details on uncommitted configuration changes, that is, the commands you are about to commit. You can enter the show configuration command to display the changes in EXEC, ADMIN, or CONFIG mode.

Use the show configuration command with the running keyword to display the running (active) configuration.

Prior to committing the target configuration, use the show configuration command with the merge keyword from any configuration mode to display the result of merging the target configuration with the running configuration.

Examples
RP/0/RSP0/CPU0:router# show configuration 
RP/0/RSP0/CPU0:router# show configuration running-config
RP/0/RSP0/CPU0:router(admin)# show configuration running
RP/0/RSP0/CPU0:router(config)# show configuration 
RP/0/RSP0/CPU0:router(config)# show configuration running
RP/0/RSP0/CPU0:router(config)# show configuration merge
In this example, the show configuration command displays uncommitted changes made during a configuration session:
RP/0/RSP0/CPU0:router# configure 
RP/0/RSP0/CPU0:router(config)# interface tengige0/3/0/3 
RP/0/RSP0/CPU0:router(config-if)# description faq 
RP/0/RSP0/CPU0:router(config-if)# ipv4 address 10.10.11.20 255.0.0.0 
RP/0/RSP0/CPU0:router(config-if)# show configuration 
Building configuration... 
interface TenGigE0/3/0/3 
 description faq 
 ipv4 address 10.10.11.20 255.0.0.0 
 end  
The following example shows sample output from the show configuration command with the optional merge keyword. The command is entered during a configuration session. The output displays the result of merging the target and running configuration, without committing the changes.
RP/0/RSP0/CPU0:router# configure 
RP/0/RSP0/CPU0:router(config)# interface tengige0/3/0/3
RP/0/RSP0/CPU0:router(config-if)# description faq 
RP/0/RSP0/CPU0:router(config-if)# ipv4 address 10.10.11.20 255.0.0.0 
RP/0/RSP0/CPU0:router(config-if)# show configuration merge 
Building configuration... 
hostname router 
interface TenGigE0/0/0/0 
 ipv4 address 10.2.3.4 255.0.0.0 
 exit
interface TenGigE0/3/0/3 
 description faq 
 ipv4 address 10.1.1.1 255.0.0.0 
 shutdown 
 end
Displaying ASIC Errors

The following example shows how to display ASIC errors for each ASIC in a SIP-700 optical LC. If an error is displayed, dump the individual ASIC instance number to obtain details on the ASIC error.
RP/0/RSP0/CPU0:router# show asic-errors all location 0/6/CPU0
Thu Oct 21 19:00:54.178 DST
************************************************************
*                  Fia ASIC Error Summary                  *
************************************************************
Instance            : 0
Number of nodes     : 0
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
************************************************************
*                 Mace ASIC Error Summary                  *
************************************************************
Instance            : 0
Number of nodes     : 0
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
************************************************************
*                Prm_np ASIC Error Summary                 *
************************************************************
Instance            : 0
Number of nodes     : 3
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
Instance            : 1
Number of nodes     : 3
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
Instance            : 2
Number of nodes     : 3
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
Instance            : 3
Number of nodes     : 3
SBE error count     : 0
MBE error count     : 0
Parity error count  : 0
CRC error count     : 0
Generic error count : 0
Reset error count   : 0
--------------------
The following ASIC error types are supported:

•FIA (Fabric Interface ASIC)

•Mace ASIC

•Prm_np ASIC

The following ASIC error classifications are supported:

•Single Bit Errors (SBE)—Correctable ECC protected single bit errors in external or internal memory.

Not reported to PM on each occurrence and reported to the platform manager (PM) as Minor when software threshold rate is exceeded. Report alarm using Alarm Logging, and Debugging Event Management System (ALDEMS).

Error data:

–Address—Address that encountered the SBE

–Syndrome—Syndrome if available

•Multiple Bit Errors—Uncorrectable multiple bit error in memory.

Reported to PM as Major and ALDEMS for each occurrence.

Error data:

–Address—Address that encountered the SBE.

–Data—Actual error data.

•PARITY Errors—Parity error in all applicable memory.

Reported to PM as Major.

•Cyclic redundancy check (CRC) Errors—CRC errors in EIO other links.

Not reported for each occurrence. When the threshold is reached it is reported as Major to the PM.

•GENERIC Errors—Errors that do not fall under any of the other classifications.

Threshold and alarm reporting is done.

•RESET Errors—Logged for each reset instance of the ASIC.

Reported to PM when threshold is exceeded.

Error data:

–Interrupt status—Interrupt status bits due to ASIC reset.

–Halt status—Halt status bits.

–Reset node key—Key for the error node that causes the reset.

–Time—Reset time.

The following ASIC error fault severities are supported:

•Critical—Affected component is unusable or card is reset if no redundant card exists.

•Major—Partially service affecting fault, causing the card to run in degraded mode. For redundant cards, consider performing a switchover.

•Minor—Non-service affecting fault.

•OK—No fault.

Using Trace Commands

Trace commands provide an `always on' debug feature. Many major functions in Cisco IOS XR software have "trace" functionality to show the last actions it conducted allowing you to analyze function events. Use the show trace commands to display the trace data for a specific feature or process. Use the ? in the CLI to determine if a command has the trace keyword. The following example shows that the show arp command has the trace keyword.
RP/0/RSP0/CPU0:router# show arp ? 
  A.B.C.D          IP address or hostname of ARP entry
  BVI              Bridge-Group Virtual Interface
  Bundle-Ether     Aggregated Ethernet interface(s)
  GigabitEthernet  GigabitEthernet/IEEE 802.3 interface(s)
  H.H.H            48-bit hardware address of ARP entry
  MgmtEth          Ethernet/IEEE 802.3 interface(s)
  TenGigE          TenGigabitEthernet/IEEE 802.3 interface(s)
  api-stats        Show ARP API statistics data
  client           ARP Client show commands
  dagr             Show Direct Attached Gateway Redundancy group information
  idb              Show the internal ARP interface data block
  location         specify a node name
  resolution       Show the ARP resolution history
  trace            Show trace data for the ARP component
  traffic          ARP traffic statistics
  vrf              Specify a VRF
  |                Output Modifiers
  <cr> 
The following example shows the last 20 events in the address resolution protocol (ARP) table.
RP/0/RSP0/CPU0:router# show arp trace tailf last 20 
1349 wrapping entries (2048 possible, 0 filtered, 1349 total)
Apr 19 09:52:29.857 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: creating incomplete entry for 
address: 172.18.105.255
Apr 19 09:52:34.501 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
Apr 19 09:52:41.856 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 09:52:46.324 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
Apr 19 09:52:59.979 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: entry 172.18.105.255: deleted 
from table
Apr 19 09:59:37.463 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 09:59:37.463 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: creating incomplete entry for 
address: 172.18.105.255
Apr 19 09:59:39.515 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 09:59:42.082 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
Apr 19 09:59:45.007 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: entry 172.18.105.255: deleted 
from table
Apr 19 09:59:50.101 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 09:59:50.101 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: creating incomplete entry for 
address: 172.18.105.255
Apr 19 09:59:54.820 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
Apr 19 10:00:00.008 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: entry 172.18.105.255: deleted 
from table
Apr 19 10:04:11.675 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 10:04:11.675 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: creating incomplete entry for 
address: 172.18.105.255
Apr 19 10:04:16.272 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
Apr 19 10:04:30.028 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: entry 172.18.105.255: deleted 
from table
Apr 19 10:04:44.097 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: received address resolution 
request for 172.18.105.255
Apr 19 10:04:44.097 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: creating incomplete entry for 
address: 172.18.105.255
Apr 19 10:04:48.810 ipv4_arp/arp 0/RSP0/CPU0 t1  ARP-TABLE: address resolution failed for 
172.18.105.255
MIB Location

To locate and download MIBs, use the Cisco MIB Locator found at the following URL and choose a platform under the Cisco Access Products menu: http://cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml.

Gathering Information Before You Call Cisco TAC

We recommend that you have a system of maintaining and accessing detailed information about your network and ASR 9000 router, including system hardware and software, network diagrams, and captured output from commands. For additional details, see the "Prerequisite Documentation for Troubleshooting" section.

Before calling the Cisco Technical Assistance Center (TAC), you should gather the information described in the following sections, if possible. This information will be helpful for troubleshooting.

Caution We strongy recommend that, if possible, you gather the information described in this section before you reset any cards. If you reset cards before you gather information, the system erases the information and it will be more difficult to diagnose and repair the problem.

•Gathering Information about Crashes and Core Dumps

•Capturing Logs

•Using Debug Commands

•Using Diagnostic Commands

•Commands Used to Display Process and Thread Details

Timesaver Before contacting Cisco Technical Support, review the information provided at the following URL: http://www.cisco.com/web/services/ts/access/index.html.

For information on contacting Cisco Technical Support, see the "Obtaining Documentation and Submitting a Service Request" section.

Gathering Information about Crashes and Core Dumps

Gather system information with the following commands:

•show install active summary

•show version

•show run

•show context

•show log

•show inventory

•show diagnostics

Upload any core dumps that were written to disk0, disk1, or harddisk directories.

Capturing Logs

See the "Prerequisite Documentation for Troubleshooting" section in Chapter 1 "General Troubleshooting Procedures," for information on collecting current system information.

Collect system information using the following commands:

•show tech-support—Displays system information for Cisco Technical Support and includes a traditional dump of the configuration and show command outputs.

Note Some tech-support commands require the user to be assigned the cisco-support task ID. For a mapping of commands to task IDs and allowed operations, see Cisco IOS XR Task ID Reference Guide.

•show logging—Displays the contents of the logging buffers

•show system verify—Displays system verification information

Using Debug Commands

For details on using debug commands, see Cisco IOS XR Using Debug Guide.

Using Diagnostic Commands

The Cisco ASR 9000 Aggregation Series Router diagnostic tests verify control Ethernet and fabric data paths. If a diagnostic tests fails, it indicates a bad data path. The integrity of the covered data paths is verified when the diagnostic tests pass.

The diagnostic tests generally test data paths between multiple nodes, therefore error reports need to be analyzed to narrow down the possible points of failure in a system.

All diagnostic tests run within the 1 second to 1 minute range.

Note On the Cisco ASR 9000 Aggregation Services Router, only online diagnostics are supported.

To run a specified on-demand diagnostic test or series of tests, use the diagnostic start location command.

Example:
RP/0/RSP0/CPU0:router(admin)# diagnostic start location 0/RSP1/CPU0 test 1
RP/0/RSP0/CPU0:router(admin)# diagnostic stop location 0/RSP1/CPU0 
For details on the diagnostic commands, see Cisco ASR 9000 Series Aggregation Services Router Interface and Hardware Component Command Reference.

Commands Used to Display Process and Thread Details

For details on processes and threads, see the "Understanding Processes and Threads" section in Cisco ASR 9000 Aggregation Services Router Router Getting Started Guide.