Cisco MGX 8850 VISM, Release 1.5 Installation and Configuration
Troubleshooting
Downloads: This chapterpdf (PDF - 171.0KB) | Feedback

Troubleshooting

Table Of Contents

Troubleshooting

Troubleshooting Tools

Card LED Indicators

Displaying Information about the VISM card

The VISM Log

Alarms

Loopback Paths

Symptoms and Possible Causes/Remedies

VISM Card did not become active

Cannot `CC' to VISM card

Card becomes active and then resets again or the card resets occasionally

Cannot download FW to VISM

Echo can be heard on the voice call

VISM card after insertion in a chasis has all leds turned off


Troubleshooting


This chapter provides a description of the available VISM 1.5 troubleshooting tools together with some remedial actions that can be taken under various abnormal situations.

Troubleshooting Tools

Card LED Indicators

The first level of troubleshooting tools is the set of LED indicators located on the VISM card itself.

The VISM card has 3 card status LEDs and 8 line status LEDs, organized as shown in Figure 8-1.

Figure 8-1 ViSM Front Card LEDs

The ACTIVE LED is GREEN, when the card is in the ACTIVE state.

STANDBY LED will be lit Orange, when the card is in the STANDBY or MISMATCH state or when the card's DSPs are getting downloaded as part of card booting up. STANDBY LED is a blinking Orange when the card is in the boot state.

FAIL LED will be lit RED when the card is in the FAIL state or during some stages of the bootup process.

The 8 line LEDs correspond to the eight T1 or E1 ports on the back card and are lit as follows:

GREEN - if the line has been added and there is no alarm on that line.

ORANGE - if the line has been added and there is a YELLOW alarm condition on the line.

RED - if the line has been added and there is a LOS condition (RED alarm condition), LOF or AIS on the line.

Displaying Information about the VISM card

The second level of troubleshooting aids is the range of "display" commands that are available through the Command Line Interface. Some of these commands are performed when logged into the PXM card and some when logged into the VISM card (see next section for details of the VISM display commands and the MGX 8850 Command Reference for details of the PXM display commands). The CLI prompt indicates the current card last logged onto by the user.

For example, executing the dspcd (display card) and version commands provides the following information:

NODENAME.1.3.VISM8.a > dspcd 
ModuleSlotNumber:          17
  FunctionModuleState:       Active
  FunctionModuleType:        VISM-8T1
  FunctionModuleSerialNum:   SAK0331006P
  FunctionModuleHWRev:       0.0
  FunctionModuleFWRev:       1.5.0_11Nov99_2
  FunctionModuleResetReason: ?
  LineModuleType:            LM-RJ48-8T1
  LineModuleState:           Present
  mibVersionNumber:          21
  configChangeTypeBitMap:    CardCnfChng, LineCnfChng
  pcb part no - (800 level): 800-04399-01
  pcb part no - (73 level):  73-03618-01
  Fab Part no - (28 level):  28-02791-01
  PCB Revision:              08

Daughter Card Information:
  Daughter Card Serial Number: SAK0331003P
  pcb part no - (73 level):    73-03722-01
  Fab Part no - (28 level):    28-02905-01
  PCB Revision:                04


sj885044.1.17.VISM8.a > version
***** Cisco Systems. MGX 8850  VISM Card *****
    Firmware Version    = 1.5.0_11Nov99_2
    Backup Boot version = 3.2.13
    Xilinx Firmware version = 10/ 2/1998
    DSPCOM FPGA version     = 10/ 7/1999
    DSPM F irmware Details:
        Major Release   = 3
        Minor Release   = 4
        Build number    = 2
    DSPM ecan Firmware Details:
        Major Release   = 7
        Minor Release   = 3
        Build number    = 9
VxWorks (for R5k PDC) version 5.3.1.
Kernel: WIND version 2.5.
Made on Nov 11 1999, 15:31:28.
Boot line:

The VISM Log

The VISM "dsplog" command can provide useful information when an abnormal situation arises. The format of an entry in the VISM log is:

Date & time of the log

Slot number of the VISM card from which a message is logged

The process on the VISM card that logged that message.

Severity of the message - if it is a fatal error causing card reboot, severity is 1. For all other messages, severity is 6.

A single line description - the log message.

Example:
01/05/1999-02:09:01 03 cmm        VISM-6-9157                 
 VISM got time from PXM

Alarms

Several kinds of alarms can be detected by the VISM-8T1/E1.

Definition of alarm states is given in T1.403 for DS1 and G.704 for E1. The alarms are propagated to the remote end over the ATM network in accordance with ATM specifications.

Table 8-1 VISM Alarms 

Error
Alarm
Type
Down stream
Up Stream
Comments

Link Failure (RX)

LOS

AIS

RAI

RAI returned on TX line

Receive RAI

Yellow

RAI

None

 

Receive LOF

 

AIS

RAI

RAI returned on TX line.

Receive AIS

AIS

AIS

RAI

RAI returned on TX line.


Loopback Paths

The VISM-8T1 and VISM-8E1 card sets provide the capability for creating loopback paths for diagnostic purposes. The available loopback configurations are:

1. Local Line Loopback, (per Line). This loopback is enabled/disabled through the addlnloop and dellnloop commands on the VISM.

2. Remote Line Loopback, (per Line). This loopback is enabled/disabled through the cnfbert command on the PXM

3. OAM Loopback through the CPU towards the network (per VC). This loopback is enabled automatically.

4. Per ds0 loopback at the compression DSP's towards the TDM side. This loopback is enabled using "addendptloop" command, and disabled using the "delendptloop" command.

5. Per VC remote loopback. This loopback is enabled using "addconloop" command, and disabled using the "delconloop" command.

The BERT and loopback functions can be useful in testing the integrity of the T1/E1 lines. Using the cnfbert command on the PXM, the following actions can be taken:

Run BERT on a per line basis on the VISM card.

Put a VISM line on a TDM side loopback.

Put a VISM line on a network side loopback.

Cause the VISM to put a test equipment on the far side into loopback.

The cnfbert command is described in the MGX 8850 Command Reference.

Symptoms and Possible Causes/Remedies

VISM Card did not become active

Four possible causes exist.

1. An E1 card is inserted in a slot where a T1 card WAS present (or viceversa), resulting in configuration mismatch.

2. When at least minimum number of DSPs failed to download - currently the threshold is fixed at 5.

3. When the backcard type doesnot match with the front card type - if the front card is of type T1 (E!) and the backcard type is E1 (T1).

4. When the MIB version in the image on the card doesn't match that of the MIB on the PXM disk.

T1/E1 Mismatch

The T1/E1 mismatch causes can be identified using the PXM dspcds command as follows:

NODENAME.1.7.PXM.a > dspcds 
Slot  CardState    CardType     CardAlarm  Redundancy 
----  -----------  --------     ---------  ----------- 
1.1   Empty                     Clear 
1.2   Empty                     Clear 
1.3   Empty                     Clear 
1.4   Empty                     Clear 
1.5   Mismatch     VISM-8E1     Clear 
1.6   Empty                     Clear 
1.7   Active       PXM1-OC3     Clear 
1.8   Empty                     Clear 
1.9   Empty                     Clear 
1.10  Empty                     Clear 
1.11  Empty                     Clear 
1.12  Empty                     Clear 
1.13  Empty                     Clear 
1.14  Empty                     Clear 
1.15  Empty                     Clear 
1.16  Empty                     Clear 
1.17  Empty                     Clear 
1.18  Empty                     Clear 
1.19  Empty                     Clear 

The PXM cnfsmcnf command can also be used as follows:

NODENAME.1.7.PXM.a > dspsmcnf 
slot      Card        Rate     Channel                         MIB     Feature 
No.      Type       Control    ized       IMA     MULTRKS   Version    Bits 
------   ----------   -------   -------   -------   -------   -------   ------- 
1     ------>  No configuration file exist for this slot  <------ 
2     ------>  No configuration file exist for this slot  <------ 
3     VISM-8T1       Off       Off       Off       Off        20      0x0 
4     ------>  No configuration file exist for this slot  <------ 
5     VISM-8T1       Off       Off       Off       Off        20      0x0 
6     ------>  No configuration file exist for this slot  <------ 
9     ------>  No configuration file exist for this slot  <------ 
10     ------>  No configuration file exist for this slot  <------ 
11     ------>  No configuration file exist for this slot  <------ 
12     ------>  No configuration file exist for this slot  <------ 
13     ------>  No configuration file exist for this slot  <------ 
14     ------>  No configuration file exist for this slot  <------ 
17     ------>  No configuration file exist for this slot  <------ 
18     ------>  No configuration file exist for this slot  <------ 
19     ------>  No configuration file exist for this slot  <------ 
20     ------>  No configuration file exist for this slot  <------ 
21     ------>  No configuration file exist for this slot  <------ 
22     ------>  No configuration file exist for this slot  <------ 
25     ------>  No configuration file exist for this slot  <------ 
26     ------>  No configuration file exist for this slot  <------ 
27     ------>  No configuration file exist for this slot  <------ 
28     ------>  No configuration file exist for this slot  <------ 
29     ------>  No configuration file exist for this slot  <------ 
30     ------>  No configuration file exist for this slot  <------ 

"dsplog" on the PXM shows the following cmm (card mismatch) log entry,  
logged by VISM card on slot 5.  
 
01/01/1999-00:01:47 05 cmm        VISM-6-9025 
VISM going to standby : Config. Mismatch between PXM and VISM 

The VISM "dspcd" commands displays the following information:


NODENAME.1.5.VISM8.s > dspcd 
ModuleSlotNumber:          5 
FunctionModuleState:       Mismatch 
FunctionModuleType:        VISM-8E1 
FunctionModuleSerialNum:   CAB12345678 
FunctionModuleHWRev:       0.13 
FunctionModuleFWRev:       2.2.10g.pm 
FunctionModuleResetReason: WatchDog timeout reset 
LineModuleType:            Missing 
LineModuleState:           Not Present 
mibVersionNumber:          20 
configChangeTypeBitMap:    CardCnfChng, LineCnfChng 
cardIntegratedAlarm:       Clear 
pcb part no - (800 level): 800-03530-01 
pcb part no - (73 level):  73-03021-01 
Fab Part no - (28 level):  28-02492-01 
PCB Revision:              01 
Daughter Card Information: 
Daughter Card Serial Number: CAB12345678 
pcb part no - (73 level):    73-03022-01 
Fab Part no - (28 level):    28-02493-01 
PCB Revision:                01 value = 34 = 0x22 = '"' 

DSP Fail to Load

When atleast minimum number of DSPs failed to download:- "dsplog" on the PXM shows:

01/01/1970-00:02:10 05 tDspmDl    VISM-6-9193 
DSPM task errors     : 6 DSPs failed to download 

If the number of DSPs (6 in the above case) is greater than 5, then, the card will fail to come up ACTIVE. If this condition happens repeatedly, the card has to be replaced.

The DSP condition is shown by the PXM "dspcds" command as shown below:

NODENAME.1.7.PXM.a > dspcds 
Slot  CardState    CardType     CardAlarm  Redundancy 
----  -----------  --------     ---------  ----------- 
1.1   Empty                     Clear 
1.2   Empty                     Clear 
1.3   Empty                     Clear 
1.4   Empty                     Clear 
1.5   Failed       VISM-8E1     Clear 
1.6   Empty                     Clear 
1.7   Active       PXM1-OC3     Clear 
1.8   Empty                     Clear 
1.9   Empty                     Clear 
1.10  Empty                     Clear 
1.11  Empty                     Clear 
1.12  Empty                     Clear 
1.13  Empty                     Clear 
1.14  Empty                     Clear 
1.15  Empty                     Clear 
1.16  Empty                     Clear 
1.17  Empty                     Clear 
1.18  Empty                     Clear 
1.19  Empty                     Clear 

Frontcard / Backcard Mismatch

The PXM "dsplog" command can indicate that the backcard type does not match with the front card type:

NODENAME.1.7.PXM.a > dsplog 
01/01/1970-00:02:24 05 cmm        VISM-6-9025 
VISM going to standby : Config. Mismatch between ASC and VISMV 
01/01/1970-00:02:24 05 cmm        VISM-6-9023 
Mismatch Backcard 
01/01/1970-00:02:24 05 cmm        VISM-6-9023 
Mismatch Backcard 

In this mismatch condition, the PXM "dspcds" command displays the following inforamation.

NODENAME.1.7.PXM.a > dspcds 
Slot  CardState    CardType     CardAlarm  Redundancy 
----  -----------  --------     ---------  ----------- 
1.1   Empty                     Clear 
1.2   Empty                     Clear 
1.3   Empty                     Clear 
1.4   Empty                     Clear 
1.5   Mismatch     VISM-8E1     Clear 
1.6   Empty                     Clear 
1.7   Active       PXM1-OC3     Clear 
1.8   Empty                     Clear 
1.9   Empty                     Clear 
1.10  Empty                     Clear 
1.11  Empty                     Clear 
1.12  Empty                     Clear 
1.13  Empty                     Clear 
1.14  Empty                     Clear 
1.15  Empty                     Clear 
1.16  Empty                     Clear 
1.17  Empty                     Clear 
1.18  Empty                     Clear 
1.19  Empty                     Clear 

The VISM "dspcd" command displays the following information:

NODENAME.1.5.VISM8.s > dspcd 
ModuleSlotNumber:          5 
FunctionModuleState:       Mismatch 
FunctionModuleType:        VISM-8E1 
FunctionModuleSerialNum:   CAB0246014P 
FunctionModuleHWRev:       0.0 
FunctionModuleFWRev:       2.2.10g.pm 
FunctionModuleResetReason: Reset by ASC from Cell Bus 
LineModuleType:            LM-RJ48-8T1 
LineModuleState:           Invalid 
mibVersionNumber:          20 
configChangeTypeBitMap:    CardCnfChng, LineCnfChng 
cardIntegratedAlarm:       Clear 
pcb part no - (800 level): 800-04399-01 
pcb part no - (73 level):  73-03618-01 
Fab Part no - (28 level):  28-02791-01 
PCB Revision:              05 
Daughter Card Information: 
Daughter Card Serial Number: CAB024601FF 
pcb part no - (73 level):    73-03722-01 
Fab Part no - (28 level):    28-02905-01 
PCB Revision:                02 value = 34 = 0x22 = '"' 

Cannot `CC' to VISM card

Use the PXM "dspcds" command to check whether the card is in the ACTIVE/STANDBY state. If the VISM card is not in the ACTIVE/STANDBY state, "cc" to that slot is not possible.

Card becomes active and then resets again or the card resets occasionally

Reasons why the above conditions can occur include:

Bad hardware device on the card.

Daughter card is not attached correctly to the VISM card, as a result, VISM card is not able to come up.

Make sure that the daughter card is making electrical contact to the mother board, and mechanically secure.

Cannot download FW to VISM

For downloading a slot specific image, there must be a VISM card in the slot to which firmware is being downloaded. The card must be in either the ACTIVE or BOOT state. Confirm this is the case and try again.

Make sure that VISM card is seated in the slot, and making the electrical contacts to the backplane.

Echo can be heard on the voice call

Make sure the call has echo cancellation turn on. If the echo delay is longer than the provision tail length, the echo cancellation will not work. Provision the tail length (cnfecantail) to a bigger value.

VISM card after insertion in a chasis has all leds turned off

The VISM card is not inserted completely in the slot. Firmware does not see the Card Insert Bit status as set. Make sure that the VISM card is seated in the slot correctly, with top and bottom half portions of the VISM card making electrical contact with back plane.