Cisco 10000 Series Router Troubleshooting Guide
Cisco 10000 Series Router Alarms and Error Messages
Downloads: This chapterpdf (PDF - 210.0KB) The complete bookPDF (PDF - 0.98MB) | Feedback

Cisco 10000 Series Router Alarms and Error Messages

Table Of Contents

Cisco 10000 Series Router Alarms and Error Messages

OC-12 POS Line Card Alarms and Error Messages

OC-12 POS Line Card SONET Alarms

OC-12 POS Line Card Bit Error Rate Alarms

OC-12 POS Line Card Error Messages

Channelized T3 Line Card Alarms and Error Messages

Channelized T3 Line Card Alarms

Channelized T3 Line Card Error Messages

Gigabit Ethernet Line Card Error Messages


Cisco 10000 Series Router Alarms and Error Messages


This chapter describes the alarms and error messages that can be raised on the Cisco 10000 series router.

The Cisco 10000 series router interface, environment, and system alarms provide feedback on the functional status of a chassis, PRE, and line cards. Furthermore, alarms and error messages alert users to less serious conditions that can degrade the performance of networks.

The following topics are addressed in this chapter:

OC-12 POS Line Card Alarms and Error Messages

Channelized T3 Line Card Alarms and Error Messages

Gigabit Ethernet Line Card Error Messages

OC-12 POS Line Card Alarms and Error Messages

This section describes alarm and error messages associated with the OC-12 POS line card. Where possible, the source of the error message is explained and a corrective action is suggested.

OC-12 POS Line Card SONET Alarms

The OC-12 POS line card provides standard SONET alarms to monitor the health of OC-12 POS trunk links. Table 5-1 describes these alarms and suggests responses to each.

Table 5-1 SONET Alarms 

Alarm Type and Severity
Alarm Symptoms
Recommendation

SLOS

Section Loss of Signal

Critical

1. Alarm messages appear in the CLI and logs.

2. Service on the SONET line card is lost.

3. If APS 1+1 redundancy is enabled, trunk service cuts over to the alternate line card.

1. Check the fiber optic cable to make sure it is plugged in.

2. Verify that the local fiber optic cable is not damaged.

3. Make sure that the remote end of the fiber optic cable is connected, undamaged and that the remote port is configured properly.

SLOF

Section Loss of Frame

Critical

1. Alarm messages appear in the CLI and logs.

2. Service on the SONET trunk is lost.

3. If APS is enabled, trunk service cuts over to the alternate line card.

1. Check the fiber optic cable to make sure it is plugged in.

2. Verify that the fiber is not damaged.

3. Make sure that the remote end of the fiber optic cable is connected and the router properly configured.

LAIS

Alarm Indicate Signal - Line

Major

1. This alarm is indicative of a remote problem.

2. Alarm messages appear in the CLI and logs.

1. Verify that the remote configuration is correct.

2. Check the line status the remote end of the link.

LRDI

Remote Defect Indication - Line

Major

Alarm messages appear in the CLI and logs.

RDI - Line problems arise from the remote interface.

Check the remote site for alarm conditions.

PAIS

Alarm Indicate Signal - Path

Minor

1. Alarm messages appear in the CLI and logs.

2. This is sent by a site that has received LAIS.

This is sent by a site that has received LAIS. This is a minor warning, and no action needs to be taken except to monitor the far end.

If the alarms are persistent, verify the interface configurations on both ends of the trunk.

PLOP

Loss of Pointer - Path

Minor

Alarm messages appear in the CLI and logs.

This is an error in the data. As with the PAIS, you should monitor for more of the same messages.

If the alarms are persistent, verify the interface configurations on both ends of the trunk.

PRDI

Remote Defect Indication - Path

Minor

Alarm messages appear in the CLI and logs.

A PRDI alarm usually indicates a problem two sites away. If the alarm is persistent, check the alarm status of neighboring sites, beginning with the nearest neighbor.


OC-12 POS Line Card Bit Error Rate Alarms

Table 5-2 describes a separate set of alarm messages raised when the bit error rate (BER) thresholds exceed specified limits. The BER monitors help identify signal degrading BER levels.

Table 5-2 BER Alarms 

Alarm Type and Severity
Alarm Symptoms
Recommendation

TCA_B1

Threshold crossing alarm - B1

Minor

For alarm types:

TCA_B1

TCA_B2

TCA_B3

Alarm messages appear in the CLI and logs.

In all cases, test the quality of the cables and connections.

TCA_B2

Threshold crossing alarm - B2

Minor

   

TCA_B3

Threshold crossing alarm - B3

Minor

   

BER_SF

Signal Fail condition

Minor

BER_SF and BER_SD alarms result in APS cutovers.

In both cases, test the quality of the cables and connections.

BER_SD

Signal degrade condition

Minor

 

For more information on signal fail (BER_SF) and signal degrade (BER_SD) alarms, refer to the APS chapter in the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. These BER thresholds can be specified by the user.


OC-12 POS Line Card Error Messages

Table 5-3 describes hardware-related error messages that the OC-12 POS line card can report. It also defines the problem causing the error message and suggests a corrective action.

Table 5-3 OC-12 POS Hardware-Related Error Messages 

Error Message
Cause of Error Message
Corrective Action
LC_ERROR: Slot(x/0) 
1oc12pos-1 DataPathControl: 
Hardware Fatal Error

These alarm messages announce unrecoverable OC-12 POS line card hardware failures.

In all cases, follow this procedure:

1. Replace the OC-12 POS line card.

2. Call the Cisco TAC.

LC_ERROR: Slot(x/0) 
1oc12pos-1 FromPRE: Hardware 
Fatal Error
   
LC_ERROR: Slot(x/0) 
1oc12pos-1 ToPRE: Hardware 
Fatal Error
   
LC_ERROR: Slot(x/0) 
1oc12pos-1 Port0: Transmit 
Clock Hardware Fatal Error
   
LC_ERROR: Slot(x/0) 
1oc12pos-1 ToPRE/FromPRE: 
Hardware Download Fatal 
Error
   
LC_ERROR: Slot(x/0) 
1oc12pos-1 FromPRE: Hardware 
Configuration Fatal Error
   
LC_ERROR: Slot(x/0) 
1oc12pos-1 ToPRE: Hardware 
Configuration Fatal Error
   

Table 5-4 describes the software-related error messages that the OC-12 POS line card might generate. It also defines the problem causing the error message and suggests a corrective action.

Table 5-4 OC-12 POS Software-Related Error Messages

Error Message
Cause of Error Message
Corrective Action
LC_ERROR: Slot(x/0) 
1oc12pos-1 Software: 
Received unknown command 
from IOS (%d)

The most likely cause of these messages is an incompatible version of IOS.

In both cases:

1. Type

Router#show version

2. Verify that the IOS version is a valid release for the Cisco 10000 series router.

3. If the version is wrong, get the most recent IOS image for the Cisco 10000 series router.

4. If the version is correct, Call the Cisco TAC.

LC_ERROR: Slot(x/0) 
1oc12pos-1 Software: Queue 
timer expired
   

Channelized T3 Line Card Alarms and Error Messages

This section describes alarm and error messages that can be generated by the channelized T3 line card. Where possible, the source of the error message is explained and a corrective action is suggested below.

Channelized T3 Line Card Alarms

Table 5-5 describes the standard T1 alarms that can be raised on the channelized T3 line card.

Table 5-5 CT3 Line Card Alarms: T1 Alarms

Alarm Type
Cause of Alarm
Corrective Action

AIS

Receive Alarm Indication Signal

Major

A received alarm indication signal (AIS) indicates an alarm raised on a line upstream from the Cisco 10000 series router.

1. Check the status of the adjacent network element to determine if the problem is there. If the problem is not in the adjacent network element, go to Step 2.

2. Ask your T1 service provider to trace the source of the AIS signal.

3. Contact the Cisco TAC.

LOF

Loss of Frame

Major

A loss of frame (LOF) condition typically happens in one of two situations:

1. The configuration settings on the port are not correct for the line.

2. The port configuration is correct but the line is experiencing other errors that result in an LOF alarm.

1. Check to see if the framing format configured on the port matches the framing format on the line.

2. Try the other framing format and see if the alarm clears.

3. Configure a remote loopback on the affected interface, then run an unframed T1 BERT. This will help determine if there are problems on the line.

If you find evidence of a bad line, you can isolate the problem using loopbacks. (Refer to the "Using Loopback Tests" section.)

RAI

Receive Remote Alarm Indication

Major

RAI indicates a problem between the C10000 T3 transmitter and the far end T3 receiver, but it may not be in the segment between the C10000 and the adjacent node.

1. Connect an external loopback cable to the port. If there are no alarms, the problem is not with the Cisco 10000 series router. (Refer to the "Using Loopback Tests" section)

2. Examine adjacent network elements and check for LOS or LOF alarms.


Channelized T3 Line Card Error Messages

Table 5-6 lists the error messages that the channelized T3 line card can generate.

Table 5-6 Channelized T3 Line Card Error Messages 

Error Message
Corrective Action
LC_ERROR:Slot[x/0] Received %s (%d:%d) command, for invalid t3 %d 
from CRE-RP!

In all cases, contact the Cisco TAC for assistance.

LC_ERROR:Slot[x/0] Received %s (%d:%d) command, for invalid t3 %d, 
t1 %d, from CRE-RP!
 
LC_ERROR:Slot[x/0] Received %s (%d:%d) command, for t3 %d, invalid 
t1 %d, from CRE-RP!\n
 
LC_ERROR:Slot[x/0] Received %s (%d:%d) command, for invalid t3 %d, 
t1 %d chan %d, from CRE-RP!
 
LC_ERROR:Slot[x/0] ct3_cmd_dispatcher:Unknown command %d!
 
LC_ERROR:Slot[x/0] c10k_lc_cmd_q timer expired!
 
LC_ERROR:Slot[x/0] ct3 i960 stats queue timeout!
 
LC_ERROR:Slot[x/0] Error getting channel stats array pointer
 
LC_ERROR:Slot[x/0] Error getting message unit array pointer
 
LC_ERROR:Slot[x/0] request_channel_stats challoc failed!!
 
LC_WARNING:Slot[x/0] Lunar Pmon data not latched for collection. 
T3=%d, T1=%d
 

Gigabit Ethernet Line Card Error Messages

Table 5-7 describes the error messages associated with the gigabit Ethernet (GE) line card. Where possible, the source of the error message is explained and a corrective action is suggested.

Table 5-7 Gigabit Ethernet Line Card Error Messages 

Error Message
Possible Cause
Corrective Action
LC_ERROR:Slot[x/0] TX 
fault

GBIC failure. GBIC transmit failure is characterized by a steadily blinking Fail LED on the gigabit Ethernet line card.

1. Replace the GBIC.

LC_ERROR:Slot[x/0] 
Unknown GBIC type

There are several types of GBICs. If the software cannot determine the type, this error is displayed. This does not mean that the GBIC will not function.

1. If this message is accompanied by a GBIC failure, replace the GBIC with a model supported by the Cisco 10000 series router.

2. If the GBIC is Cisco supplied, it may be faulty. Try replacing the GBIC.

3. If none of the suggestions above work, call the Cisco TAC.

See the Cisco 10000 Series Router Hardware Installation Guide for a list of GBIC part numbers.

LC_ERROR:Slot[x/0] 
Unknown GBIC CC type

Occurs if a GBIC (containing an EEPROM) type cannot be determined.

1. Reinsert the GBIC.

2. Insert a new GBIC.

3. Try the GBIC in a different gigabit Ethernet line card.

LC_ERROR:Slot[x/0] Fatal 
interrupt from x

Occurs if there is a problem with the hardware.

Call the Cisco TAC.

%C10KGE-3-GBIC_MISSING:In
terface 
GigabitEthernet1/0/0, 
Gigabit Interface 
Converter (GBIC) missing

Error messages appear in the logs and the CLI.

1. Make sure the GBIC is properly inserted.

2. Replace the GBIC.