Cisco 7600-ES20 Ethernet Line Card Configuration Guide - Troubleshooting the Cisco 7600 Series Ethernet Services 20G Line Card [Support]

Table Of Contents

Troubleshooting the Cisco 7600 Series Ethernet Services 20G Line Card

General Troubleshooting Information

Interpreting Console Error Messages

Using debug Commands

Using show Commands

Using the Cisco IOS Event Tracer to Troubleshoot Problems

Troubleshooting SFP Issues

Preparing for Online Insertion and Removal of Cisco 7600 Series Ethernet Services 20G Line Card

Preparing for Online Removal of a Cisco 7600 Series Ethernet Services 20G Line Card

Deactivating a Cisco 7600 Series Ethernet Services 20G Line Card

Reactivating a Cisco 7600 Series Ethernet Services 20G Line Card

Verifying Deactivation and Activation of a Cisco 7600 Series Ethernet Services 20G Line Card

Deactivation and Activation Configuration Examples

Deactivation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

Activation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

Line Card Online Diagnostics

Onboard Failure Logging

Troubleshooting the Cisco 7600 Series Ethernet Services 20G Line Card

This chapter describes techniques that you can use to troubleshoot the operation of your Cisco 7600 Series Ethernet Services 20G (ES20) line card.

It includes the following sections:

•General Troubleshooting Information

•Using the Cisco IOS Event Tracer to Troubleshoot Problems

•Troubleshooting SFP Issues

•Preparing for Online Insertion and Removal of Cisco 7600 Series Ethernet Services 20G Line Card

•Line Card Online Diagnostics

•Onboard Failure Logging

The first section provides information about basic interface troubleshooting. If you are having a problem with your SFP and XFP modules, use the steps in the "Using the Cisco IOS Event Tracer to Troubleshoot Problems" section to begin your investigation of a possible interface configuration problem.

General Troubleshooting Information

This section describes general information for troubleshooting the ES20 line card. It includes the following sections:

•Interpreting Console Error Messages

•Using debug Commands

•Using show Commands

Interpreting Console Error Messages

To view the explanations and recommended actions for Cisco 7600 series router error messages, including messages related to Cisco 7600 series router ES20 line card, refer to the Cisco 7600 Series Cisco IOS System Message Guide, 12.2SR at http://www.cisco.com/en/US/docs/ios/system/messages/guide/consol_smg.html

System error messages are organized in the documentation according to the particular system facility that produces the messages. The ES20 line card error messages use the following facility names:

•7600-ES20-2X10G—ESM20

•7600-ES20-20XG—ESM20

Using debug Commands

Along with the other debug commands supported on the Cisco 7600 series router, you can obtain specific debug information for the ES20 line card on the Cisco 7600 series router using the debug hw-module privileged EXEC command.

The debug hw-module command is intended for use by Cisco technical support personnel.

Caution Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions with Cisco technical support staff. Moreover, it is best to use debug commands during periods of lower network traffic and fewer users. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use.

For information about other debug commands supported on the Cisco 7600 series routers, refer to the Cisco IOS Debug Command Reference, Release 12.2 SR at http://www.cisco.com/en/US/docs/ios/12_2/debug/command/reference/dbfser.html

Using show Commands

There are several show commands that you can use to monitor and troubleshoot the ES20 line card on the Cisco 7600 series routers. See Chapter "Command Summary for the Cisco 7600 Series Ethernet Services 20G Line Card".

Using the Cisco IOS Event Tracer to Troubleshoot Problems

Note The Event Tracer feature is intended for use as a software diagnostic tool and should be configured only under the direction of a Cisco Technical Assistance Center (TAC) representative.

The Event Tracer feature provides a binary trace facility for troubleshooting Cisco IOS software. This feature gives Cisco service representatives additional insight into the operation of the Cisco IOS software and can be useful in helping to diagnose problems in the unlikely event of an operating system malfunction or, in the case of redundant systems, Route Processor switchover.

Event tracing works by reading informational messages from specific Cisco IOS software subsystem components that have been preprogrammed to work with event tracing, and by logging messages from those components into system memory. Trace messages stored in memory can be displayed on the screen or saved to a file for later analysis.

For more information about using the Event Tracer feature, refer to the following URL:

www.cisco.com/en/US/docs/ios/netmgmt/configuration/guide/nm_event_tracer_ps6441_TSD_Products_Configuration_Guide_Chapter.html

Troubleshooting SFP Issues

Use the following steps when troubleshooting an small form-factor pluggable (SFP) issue:

Note The ES20 line card uses a slot/bay/port numbering scheme. The slot refers to whichever slot the line card occupies in the router. The bay number is always 0. The port number is either 0 or 1 in the 2-port card or 0 through 19 in the 20-port card.

Step 1 Connect to the card with the problematic SFP module using the execute-on command:
Router# execute-on 1 show tech
Step 2 Use the test sfp command to display information about the problematic SFP module on line card slot 1, port 0.
# test interfaces gigabitEthernet 1/0/0
Step 3 Use the debug commands to provide additional information. Use the following command to debug all interfaces:
# debug ethernet-interface
Use the following command to debug a specific interface in slot 1, port 0:
# debug interface gigabitEthernet 1/0/0
Step 4 Use the show controller command to view additional information:
Router# show controller gigabitethernet 2 1 sfp 
SFP disabled or link problems (0x32) 
Preparing for Online Insertion and Removal of Cisco 7600 Series Ethernet Services 20G Line Card

The Cisco 7600 series router supports online insertion and removal (OIR) of the ES20 line card, in addition to each of the small form-factor pluggable (SFP or XFP) optical transceivers.

Therefore, you can remove an ES20 line card with its optical transceivers still intact, or you can remove an optical transceiver independently from the ES20 line card, leaving the ES20 line card installed in the router.

This section includes the following topics on OIR support:

•Preparing for Online Removal of a Cisco 7600 Series Ethernet Services 20G Line Card

•Verifying Deactivation and Activation of a Cisco 7600 Series Ethernet Services 20G Line Card

•Deactivation and Activation Configuration Examples

Preparing for Online Removal of a Cisco 7600 Series Ethernet Services 20G Line Card

The Cisco 7600 series router supports OIR of the ES20 line card. To do this, you can power down an ES20 line card (which automatically deactivates any installed optical transceivers) and remove the ES20 line card still intact.

Although graceful deactivation of an ES20 line card is preferred using the no power enable module command, the Cisco 7600 series router does support removal of the ES20 line card without deactivating it first. If you plan to remove an ES20 line card, you can deactivate the ES20 line card first, using the no power enable module global configuration command. When you deactivate an ES20 line card using this command, it automatically deactivates each of the optical transceivers that are installed in that ES20 line card. Therefore, it is not necessary to deactivate each of the optical transceivers prior to deactivating the ES20 line card.

Either a blank filler plate or a functional optical transceiver should reside in every subslot of an ES20 line card during normal operation.

For more information about the recommended procedures for physical removal of the ES20 line card, refer to the Cisco 7600 Series Ethernet Services 20G Line Card Hardware Installation Guide.

Deactivating a Cisco 7600 Series Ethernet Services 20G Line Card

To deactivate an ES20 line card and its installed optical transceivers prior to removal of the line card, use the following command in global configuration mode:

Command

Purpose

Router(config)# no power enable module slot

Shuts down any installed interfaces, and deactivates the ES20 line card in the specified slot, where:

•slot—Specifies the chassis slot number where the line card is installed.

For information on how to specify the physical locations of an ES20 line card on the Cisco 7600 series routers, see Identifying Slots and Subslots for the Cisco 7600 Series Ethernet Services 20G Line Card.

Reactivating a Cisco 7600 Series Ethernet Services 20G Line Card

Once you deactivate an ES20 line card, whether or not you have performed an OIR, you must use the power enable module global configuration command to reactivate the ES20 line card.

If you did not issue a command to deactivate the optical transceivers installed in an ES20 line card, but you did deactivate the ES20 line card using the no power enable module command, then you do not need to reactivate the optical transceivers after an OIR of the ES20 line card. The installed optical transceivers automatically reactivate upon reactivation of the ES20 line card in the router.

For example, consider the case in which you remove an ES20 line card from the router to replace it with another ES20 line card. You reinstall the same optical transceivers into the new ES20 line card. When you enter the power enable module command on the router, the optical transceivers will automatically reactivate with the new ES20 line card.

To activate an ES20 line card and its installed optical transceivers after the ES20 line card has been deactivated, use the following command in global configuration mode:

Command

Purpose

Router(config)# power enable module slot

Activates the ES20 line card in the specified slot and its installed optical transceivers, where:

•slot—Specifies the chassis slot number where the ES20 line card is installed.

For information on how to specify the physical locations of an ES20 line card on the Cisco 7600 series routers, see Identifying Slots and Subslots for the Cisco 7600 Series Ethernet Services 20G Line Card.

Verifying Deactivation and Activation of a Cisco 7600 Series Ethernet Services 20G Line Card

To verify the deactivation of an ES20 line card, enter the show module command in privileged EXEC configuration mode. Observe the Status field associated with the ES20 line card that you want to verify.

The following example shows that the ES20 line card located in slot 2 is deactivated. This is indicated by its "PwrDown" status.
Router# show module 2
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  2    0  ESM20G                                 7600-ES20-BASE     JAB1030007C
Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  2  00e0.aabb.cc00 to 00e0.aabb.cc00   1.0   12.2(2006032 12.2(2006110 PwrDown
Mod  Sub-Module                  Model              Serial       Hw     Status 
---- --------------------------- ------------------ ----------- ------- -------
  2  ESM20G/PFC3C Distributed Fo 7600-ES20-D3C      JAB1030008H  1.0    PwrDown
Mod  Online Diag Status 
---- -------------------
  2  Not Applicable
Router#
To verify activation and proper operation of an ES20 line card, enter the show module command and observe "Ok" in the Status field as shown in the following example:
Router# show module 4
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  4    2  ESM20G                                 7600-ES20-BASE     JAB10230687
Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  4  00e0.aabb.cc00 to 00e0.aabb.cc00   1.0   12.2(2006032 12.2(nightly Ok
Mod  Sub-Module                  Model              Serial       Hw     Status 
---- --------------------------- ------------------ ----------- ------- -------
  4  ESM20G Distributed Forwardi 7600-ES20-D3CXL    JAB10230672  1.0    Ok
 4/0 1x10GE XFP Port             7600-ES20-2X10G    JAB1023069L  1.0    Ok
 4/1 1x10GE XFP Port             7600-ES20-2X10G    JAB1023069L  1.0    Ok
Mod  Online Diag Status 
---- -------------------
  4  Bypass
 4/0 Bypass
 4/1 Bypass
Router#
For information on how to specify the physical locations of an ES20 line card on the Cisco 7600 series routers, see Identifying Slots and Subslots for the Cisco 7600 Series Ethernet Services 20G Line Card.

Deactivation and Activation Configuration Examples

This section provides the following examples of deactivating and activating an ES20 line card and optical transceivers:

•Deactivation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

•Activation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

Deactivation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

Deactivate an ES20 line card when you want to perform OIR of the ES20 line card. The following example deactivates the ES20 line card that is installed in slot 5 of the router, its optical transceivers, and all of the interfaces. The corresponding console messages are shown:
Router# configure terminal
Router(config)# no power enable module 5
1w4d: %OIR-6-REMCARD: Card removed from slot 5, interfaces disabled
1w4d: %C6KPWR-SP-4-DISABLED: power to module in slot 5 set off (admin request)
Activation of a Cisco 7600 Series Ethernet Services 20G Line Card Configuration Example

Activate an ES20 line card if you have previously deactivated it. If you did not deactivate the optical transceivers, the optical transceivers automatically reactivate with reactivation of the ES20 line card.
The following example activates the ES20 line card that is installed in slot 5 of the 
router, its optical transceivers, and all of the interfaces (as long as the hw-module 
subslot shutdown command was not issued to also deactivate the optical transceivers):
Router# configure terminal
Router(config)# power enable module 5
Notice that there are no corresponding console messages shown with activation. If you 
re-enter the power enable module command, a message is displayed indicating that the 
module is already enabled:
Router(config)# power enable module 5
% module is already enabled
Line Card Online Diagnostics

Note Output from this procedure will vary slightly depending on which line card you are using, but the basic information will be the same.

Line card field diagnostic software is bundled with the main Cisco IOS software to enable you to test whether a suspect line card is faulty. For information on running online diagnostics, see Configuring Online Diagnostics in the Cisco 7600 Series Cisco IOS Software Configuration Guide, 12.2SR at http://www.cisco.com/en/US/docs/routers/7600/ios/12.2SR/configuration/guide/swcg.html

Onboard Failure Logging

The onboard failure logging (OBFL) feature gathers boot, environmental, and critical hardware failure data for field-replaceable units (FRUs), and stores the information in the nonvolatile memory of the FRU. This information is used for troubleshooting, testing, and diagnosis if a failure or other error occurs.

Because OBFL is on by default, data is collected and stored as soon as the card is installed. If a problem occurs, the data can provide information about historical environmental conditions, uptime, downtime, errors, and other operating conditions.

Caution OBFL is activated by default in all cards and should not be deactivated. OBFL is used to diagnose problems in FRUs and to display a history of FRU data.

For information on configuring OBFL, see Onboard Failure Logging at http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/12sobfl.html