Cisco 7600-ES20 Ethernet Line Cards Hardware Installation Guide
Troubleshooting the Installation
Downloads: This chapterpdf (PDF - 219.0KB) The complete bookPDF (PDF - 3.49MB) | Feedback

Table of Contents

Troubleshooting

Troubleshooting Installation Issues

Troubleshooting Line Card Power Issues

Miscellaneous Line Card Issues

Troubleshooting ES20 Line Card States

Using debug Commands

Packing a Cisco 7600 ES20 Line Card for Shipment

Troubleshooting

This chapter describes how to troubleshoot the installation of the Cisco 7600 Series Ethernet Services 20G (ES20) line card on the Cisco 7600 Series Routers. This chapter contains the following sections:

Troubleshooting Installation Issues

This section describes troubleshooting the installation of the Cisco 7600 ES20 line card. Possible problems, observations and comments, and solutions are indicated for the following troubleshooting symptoms:

  • Cisco 7600 ES20 line card transitions repeatedly from on to off.
  • Cisco 7600 ES20 line card is deactivated.

 

Cisco 7600 ES20 Line Card Transitions Repeatedly From On to Off
Possible Problem
Observations and Comments
Solutions

Cisco 7600 ES20 line card is booting up; this is normal operation

Cisco 7600 ES20 line card STATUS LED alternates green, amber, or off

Wait 30 seconds until the boot process completes and the STATUS LED stays on.

Cisco 7600 ES20 line card does not go beyond the bootup stage

Cisco 7600 ES20 line card STATUS LED transitions continue and alternates green, amber, or off

Follow the recommended action for the displayed error message.

Cisco 7600 ES20 line card is not up to date

During Cisco 7600 ES20 line card initalization, the need to update the FPGA is automatically detected

Follow the system prompts to update the FPGA image.

If the Cisco 7600 ES20 line card is cycling because of an FPD problem, the most likely cause is a FPD failure or that the FPD package file is not present.

For more information about performing FPD upgrades, refer to the “Upgrading Field-Programmable Devices” chapter in the Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide .

 

Cisco 7600 ES20 Line Card Is Deactivated
Possible Problem
Observations and Comments
Solutions

Cisco 7600 ES20 line card is not fully seated in the chassis slot

Output of the show diag slot command

STATUS LED is off

Follow this procedure:

  • Remove the Cisco 7600 ES20 line card from the slot.
  • Inspect the Cisco 7600 ES20 line card. Verify there are no bent pins or parts and that there is nothing that could prevent a good connection.
  • Insert the Cisco 7600 ES20 line card into the chassis slot.

Cisco 7600 ES20 line card is not at the minimum hardware revision level

Error message indicating the Cisco 7600 ES20 line card is not at the minimum FPGA revision level

Output of the show hw-module subslot fpd command

Output of the show diag slot command

STATUS LED is off

Follow the FPD upgrade process to update the FPGA.

For more information about performing FPD upgrades, refer to the “Upgrading Field-Programmable Devices” chapter in the Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide from the following URL:

http://www.cisco.com/en/US/docs/interfaces_modules/shared_port_adapters/configuration/7600series/sipspasw.html

Digital Optical Monitoring (DOM) alarm conditions

No change in the LED status unless the link is down due to high receiver power.

To remove the alarm, ensure that appropriate attenuators are incrementally used. However, when the received power exceeds a lower threshold, a lower db attenuator should be used.The supported attenuators in a lab scenario are:

  • 5 db
  • 10 db
  • 15 db
  • 20 db

Troubleshooting Line Card Power Issues

Table 7-1 provides solutions for troubleshooting line card issues.

Table 7-1 Troubleshooting Line Card Power Issues

Problem
Solution

If a line card fails to operate or power up

1. Ensure that the card is seated firmly in the slot.

2. Ensure that you have placed the gold edge of the PCB in the guide rails of the chassis. This prevents damage to the connector placed at the rear side the board.

3. Check whether the ejector levers are latched and that the captive screws are fastened properly. If you’re uncertain, unlatch the levers, loosen the screws, and attempt to reset the supervisor again.

4. Examine the power supply to see whether the complete chassis is receiving power.

5. Use the show power command to confirm if the chassis has sufficient power to initiate the new inserted line card.

6. Use the status LED on the line card to verify the correct installation of the card. If the card is installed correctly, the Status LED turns green.

7. Ensure that you have installed the line cards in their designated slots.

Note All line cards can be inserted in all the slots of chassis; however 7613 chassis has exceptions. Dual fabric channel line cards such as (ES20 and ES+ cards) can be inserted only in 9 to13 slots of 7613 chassis.

Miscellaneous Line Card Issues

Table 7-2 provides solutions for troubleshooting line card issues.

Table 7-2 Troubleshooting Miscellaneous Line Card Issues

Problem
Solution

Insufficient power

  • When multiple line cards are inserted in the 7600 chassis, some cards may not be able to power up due to insufficient power. If the power required by the line card exceeds the power available in the system, the line card is denied power and is shut down. In such a case, use the show power command to verify how much power is used and available.
  • Execute the command in the configuration mode to deny power to a line card.
Router# configure terminal
Router(config)# no power enable module <slot_number>

On doing this, the power consumed by the line card before turning it OFF, is returned to the total power available in the system. This is handy in bringing up another line card. If you have manually disabled power to a line card, you can enable it with the following config mode command:

Router(config)# power enable module <slot_number>

Line card image not bundled in the IOS image

  • For a line card to bootup, the line card image has to be bundled in the IOS image. Not all line cards are supported with every image. A line card does not bootup if its image is not bundled with the system image. Execute the show power and dir sup-microcode commands to determine if the line card image is bundled with the IOS image.

Find Master Timer expiry

  • This problem occurs when the RP does not process the Find Master Messages in time, or line card crashes or is unable to download the image from the supervisor.
  • If one of the CPUs is not able to load the image from the SP, try with a different image and if the issue persists, replace the flexwan. If any of the CPUs has crashed, collect the crash information data to analyze further.

Faulty hardware


Note Bad hardware can also be a reason for the line card not booting up. There are no specific pointers which indicate that the line card hardware is bad. But if none of the above reasons match the failure of the line card, it could a bad hardware issue.


  • This following message indicates that the module software cannot read the module hardware version. This condition indicates a hardware error.
00:02:51: %CWPA-2-BAD_HWVERS: Unable to determine board hardware version

SCP Keepalive Failure

  • The SP and the MSFC periodically poll each individual LC in the chassis for its health via SCP keepalives. Each LC in turn, must respond within a certain amount of time (dictated by the SCP keepalive timer) to indicate its health. If the LC fails to respond to the keepalives within the stipulated period of time, the MSFC assumes that there is an issue with the LC and power cycles it. When the LC comes back online, the SCP keepalives are resumed.
  • A line card might be unable to respond to the scp keepalives under situations of extreme stress when the CPU is busy with higher priority tasks, such as packet switching, interrupt processing etc.
  • Execute the show scp status and show scp line cards command to gather detailed information about the SCP process and health of each LC. A show diagbus indicates any errors with the LC.
  • Execute the test scp line card keepalive disable exec command on the RP console to disable the SCP keepalive timer and debug LC issues. To prevent the LC from being power cycled and losing its information, ensure that you disable the SCP keepalives for some time.

Line Card in ROM monitor

  • Linecards have a ROMMON variable DL_DISABLE depending on which line card auto boots or remains in ROMMON waiting for manual boot. If DL_DISABLE is set to 0, line card does not boot automatically but waits for the user to manually boot with respective line card image (from line card console). Line card auto boots only if DL_DISABLE =1. When DL_DISABLE =0 and line card doe not initialize, SCP Keepalive Failure messages appear on RP console.

SCP Download Failures

  • A line card is reset by the SP with messages that SCP download failed or module reloaded during reset. This occurs at the time of bootup with a heavy configuration and a many line cards. Execute the debug cwan oir command to check the RP status. When a line card sends a Find Master to the RP, some of the messages are sent to SP to confirm if the module is online. If the message fails to reach the SP, then SP resets the module with message that SCP download has failed. If the line card, does not receive the response in the time, the line card resets itself.
  • Reload the line card and connect the LC console and check the status. If there is a problem with ROMMON download of the image from the SP, then the ROMMON of the line card should then be upgraded to the latest version. If a bay crashes, collect the stack trace to identify problem.

Troubleshooting ES20 Line Card States

Table 7-3 provides solutions for troubleshooting line card issues.

Table 7-3 Troubleshooting Line Card States

Problem
Solution

Troubleshooting Line Card States

  • LED on the line card front panel is green indicating that the that line card is up and running with IOS.
  • The show module command shows the status of line card slot as Ok and the status of the Distributed Forwarding submodule as Ok indicating that supervisor recognizes the line card and if the line card is powered up and online in the C7600 system or not.
  • LED on the SPA front panel is green.
  • The show hw-module subslot x/y oir [ internal ] command indicates that the SPA operation status is Ok.
  • The show running-config command shows interfaces for the line card and if the controllers for the installed SPAs (if any) are installed.
  • The show ip interface brief command shows the configured interfaces for the line card if they are ready for configuration.

Using debug Commands

The debug hw-module subslot command is intended for use by Cisco Systems technical support personnel. For more information about the debug hw-module subslot command, see the Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide .


Caution Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions with Cisco technical support staff. Moreover, it is best to use debug commands during periods of lower network traffic and fewer users. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use.

For information about other debug commands supported on the Cisco 7600 ES20 line card and to view the explanations and recommended actions for the Cisco 7600 series router error messages, including messages related to the Cisco 7600 ES20 line card, refer to the following documents:

Packing a Cisco 7600 ES20 Line Card for Shipment

This section provides step-by-step instructions for packing a Cisco 7600 ES20 line card for shipment. Before beginning this procedure, you should have the following original Cisco Systems packaging materials:

  • Static shielding bag
  • Smaller inner carton
  • Larger exterior carton
  • Two foam packing cushions

Caution The Cisco Systems original packaging is to be used for the shipment of all Cisco 7600 ES20 line cards. Failure to properly use Cisco Systems packaging can result in damage or loss of product.


Warning During this procedure, wear grounding wrist straps to avoid ESD damage to the card. Do not directly touch the backplane with your hand or any metal tool, or you could shock yourself. Statement 93.



Note These instructions assume that the Cisco 7600 ES20 line card has been removed from the router according to the recommended procedures specified in this guide.


To pack a Cisco 7600 ES20 line card for shipment, perform the following steps:


Step 1 Insert the Cisco 7600 ES20 line card into the static shielding bag.

Step 2 Insert the bagged Cisco 7600 ES20 line card into the smaller inner carton. Be careful to position the Cisco 7600 ES20 line card so that the bottom motherboard lip is held by the packaging cutout.

Step 3 Close the smaller inner carton and tape the sides closed.

Step 4 Place the sealed smaller inner carton containing the Cisco 7600 ES20 line card into the two foam packing cushions (they only fit one way).

Step 5 Place the sealed smaller inner carton and packing cushions into the larger exterior carton, and seal the larger exterior carton with tape for shipment.