Guest

Inverse Multiplexing over ATM (IMA)

Troubleshooting ATM IMA Links on Cisco 2600 and 3600 Routers

Cisco - Troubleshooting ATM IMA Links on Cisco 2600 and 3600 Routers

Document ID: 10452

Updated: Jun 05, 2005

   Print

Introduction

Before you can ping across an inverse multiplexing over ATM (IMA) interface, you need to ensure that both the IMA logical interface and the physical interfaces that make up that IMA logical interface are up and active. In some cases, these interfaces do not show as up/up in the output of the show interface command. There are two key commands to help determine the error condition:

  • show controllers atm [mod/port] for the physical interfaces that compose the IMA group.

  • show controllers atm[mod]/ima[group#] for the IMA logical interfaces.

This document explains the values that you can see in the output of these commands. Find more information in the specification for IMA of the ATM Forum, called AF-PHY-0086.001. Click here to download the IMA and other approved technical specifications.

Prerequisites

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

The show controllers atm Command for Physical Interfaces

The first step in order to troubleshoot IMA link-down problems is to check the physical interfaces. The show controllers atm command displays this information:

  • IMA - The status of the IMA state machine on the interface.

  • Framer - The configuration of physical-layer parameters, such as framing and line codes.

  • Digital signal level 1 (DS-1) MIB DATA - Physical-layer line error statistics, as defined in RFC 1406 (DS-1/E1 management information base [MIB]). The statistics are tracked in 15-minute intervals, so you can see statistics for the current interval and total statistics for the previous few intervals.

  • SAR counter totals across all links and groups - Statistics for layer 2, such as number of cells input and output and system errors like peripheral component interconnect (PCI) bus errors.

    ATM#show controllers atm 1/2
    
       Part of IMA group 1 
       Link 2 IMA Info: 
        group index is 2 
        Tx link id is 2, TX link state is unusableFault 
        Rx link id is 99, Rx link state is unusableFailed 
        Rx link failure status is lodsFailure, 
        1 TX failures, 3 rx failures 
       Link 2 Framer Info: 
        framing is ESF, line code is B8ZS, fdl is ANSI 
        cable-length is short 0-133ft, 
        clock src is line, payload-scrambling is disabled, no loopback 
        line status is 0x1; or no alarm. 
        port is active, link is available 
        0 idle rx, 0 correctable hec rx, 6 uncorrectable hec rx 
        275527047 cells rx, 0 cells TX, 790977 rx fifo overrun.    
       Link (2):DS-1 MIB DATA: 
        Data in current interval (831 seconds elapsed): 
        0 Line Code Violations, 0 Path Code Violations    
        0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins 
        0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs 
        Total Data (last 84 15 minute intervals): 
        130660 Line Code Violations, 38 Path Code Violations,    
        0 Slip Secs, 309 Fr Loss Secs, 11 Line Err Secs, 0 Degraded Mins, 
        1 Errored Secs, 1 Bursty Err Secs, 0 Severely Err Secs, 339 Unavail Secs 
       SAR counter totals across all links and groups: 
        0 cells output, 0 cells stripped 
        0 cells input, 0 cells discarded, 0 AAL5 frames discarded    
        0 pci bus err, 0 dma FIFO full err, 0 rsm parity err 
        0 rsm syn err, 0 rsm/seg q full err, 0 rsm overflow err    
        0 hs q full err, 0 no free buff q err, 0 seg underflow err    
        0 host seg stat q full err  
    

See Link 2 IMA Information for a detailed discussion of this output.

Link 2 IMA Information

In the output produced, the Link 2 IMA Info section contains these lines:

  Link 2 IMA Info: 
  group index is 2 
  TX link id is 2, TX link state is unusableFault 
  Rx link id is 99, Rx link state is unusableFailed 
  Rx link failure status is lodsFailure, 
  1 TX failures, 3 rx failures

Notice what this output means. An IMA interface follows a state machine, in which the interface moves through several states before it becomes active. Special cells called IMA control protocol (ICP) cells carry the state information between the two ends. This table explains the possible states:

State Explanation
Not in group The link is not configured to be an IMA group member.
Unusable The link is configured to be an IMA group member, but is not in use due to fault or inhibition. Link error conditions, such as physical link or IMA errors and vendor-dependent conditions, such as exceeding a certain bit error rate (BER), can lead to an unusable state.
Usable The link is ready to operate, but waits for the other end to be usable or active.
Active The link can pass ATM layer cells from and to the ATM layer.
Blocking (Receive state only.) The link transitions to the Unusable state. The Blocking sub-state allows graceful transition into the Unusable state without loss of ATM cells.

When the near end reaches a Usable receive state, it only moves into an Active receive state if the far end (FE) reports a Usable transmit state. Similarly, when the near end reaches a Usable transmit state, it only moves into an Active transmit state if the far end reports an Active receive state. IMA defines separate states for the transmit and receive directions at each end, and each end can report a different state.

In addition to main states, IMA defines sub-states for the transmit and receive directions. These sub-states provide further details on the link status.

Transmit Sub-states
Not in Group  
Unassigned No information about the link exists.
Deleted The link has been removed from the group.
Unusable  
Test Failed Bad connectivity has been found as a result of a test; such as a test pattern procedure.
Fault Fault has been detected either on the link or in the link protocol.
Inhibited The operation of the link is blocked for some locally defined application or vendor-specific reason. The link can otherwise be used.

Receive Sub-states
Not in Group  
Unassigned No information about the link exists.
Deleted The interface has been removed from an IMA group. This transitional state ensures that the other end is no longer Active and transmits ATM layer cells before the interface moves to the Unassigned substate.
Unusable  
Idle The link establishment cannot be started because of insufficient information.
Failed The receiver has failed because of a continuous defect, such as link defects, LIF and LODS.
Inhibited The link operation is blocked because of an implementation reason. The link can be used for testing.
Fault A fault in the link or in the protocol has been detected. The definition of this error condition can be implementation dependent.

Synchronization allows an ATM interface to determine where a frame starts and ends, and ensures that the bits are interpreted properly. Synchronization is important for ATM physical links and also is important for IMA virtual links. The IMA frame synchronization mechanism is based on the cell delineation mechanism defined in ITU-T I.432. Synchronization on an IMA virtual link can be in one of three states: IMA Hunt, IMA PreSync and IMA Sync. There is a cell by cell review scheme for the IMA Hunt state and a frame-by-frame scheme for the IMA PreSync and IMA Sync states.

In order to determine synchronization status, an ATM interface checks several fields or bytes in ICP cells.

Field Explanation
IMA OAM Label Defines the IMA version: 00000001 for version 1.0 or 00000011 for version 1.1.
LID Defines the cell ID and the link ID.
IMA Frame Number One byte in length, with valid values from zero to 255.
ICP Cell Offset Indicates the position of the ICP cell within an IMA frame.
IMA ID One byte in length, with valid values from zero to 255.

The values in these fields determine whether the cells are valid, invalid, or errored. An ATM interface must receive several valid ICP cells in consecutive order for the link to be synchronized.

Now that we understand synchronization, we can discuss the defects that cause an ATM physical interface to be in the UnusableFailed state. An ATM interface enters the Unusable state when it is configured to be a member of an IMA group, but cannot be used because of a detected fault. These are the three fault conditions:

  • Link Defects - Interface experiences loss of signal, loss of frame, alarm indication signal, or loss of cell delineation.

  • Loss of IMA Frame (LIF) - Interface experiences an out of IMA frame (OIF) or loss of synchronization state for at least two IMA frames. When this fault condition is active, the IMA sublayer replaces all received data cells with filler cells.

  • Loss of Delay Synchronization (LODS) - Interface has detected that the timing delay between this link and other links in the IMA group has exceeded the configured maximum value. This value ranges from 25 to 200 milliseconds on the NM-4T1-IMA. Normally, the IMA protocol adjusts for differences in delay so that all links in a group actively carry network traffic. Otherwise, an ATM interface that exceeds the differential timing delay maximum value is removed from the IMA group.

Link 2 Framer Information

The Link 2 Framer Info section contains these lines:

  Link 2 Framer Info: 
  framing is ESF, line code is B8ZS, fdl is ANSI 
  cable-length is short 0-133ft, 
  clock src is line, payload-scrambling is disabled, no loopback  
  line status is 0x1; or no alarm. 
  Port is active, link is available  
  0 idle rx, 0 correctable hec rx, 6 uncorrectable hec rx  
  275527047 cells rx, 0 cells TX, 790977 rx FIFO overrun.

An IMA module uses ATM at layer 2 and standard T1 or E1 parameters at layer 1. In order to troubleshoot IMA link problems, ensure that several T1 or E1 values match on both the Cisco IMA interface and the attached IMA interface (typically a Telephone Company switch).

  • framing - T1 lines organize the 24 digitally sampled byte streams into a 193-bit frame, which consist of 192 data bits and one framing bit. 12 frames create a superframe (SF or D4), and 24 frames create an extended superframe (ESF).

  • line code - T1 lines support two line codes: bipolar eight zero substitution (B8ZS) and alternate mark inversion (AMI). Line codes define how the digital ones and zeros are represented on the wire.

  • payload-scrambling - Scrambling randomizes the ATM cell payload to avoid non-variable bit patterns and improve the efficiency of the cell delineation algorithms of the ATM. By default, payload scrambling is on for E1 links and off for T1 links.

  • rx FIFO overrun - The number of times the framer hardware was unable to hand receive data to a hardware buffer. This happens when the input rate exceeds the ability of the framer to handle the data. The framer has a limited buffer to hold inbound data streams to assemble as a T1 frame. If this is seen, the next step is to verify clocking on the interface. Verify that the interface is configured for the reverse of what the directly connected device is configured for (that is, clock source internal on local device and clock source line on directly connected device).

    Note: "Directly connected device" refers to any device to which the interface in question is connected, whether it be a telco switch or a back-to-back environment.

DS-1 MIB Data

The next section to look at is the DS-1 MIB Data, a section shown in the output:

  Link (2):DS-1 MIB DATA: 
  Data in current interval (831 seconds elapsed): 
  0 Line Code Violations, 0 Path Code Violations  
  0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins 
  0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err  Secs, 0 Unavail Secs 
  Total Data (last 84 15 minute intervals): 
  130660 Line Code Violations, 38 Path Code Violations,  
  0 Slip Secs, 309 Fr Loss Secs, 11 Line Err Secs, 0 Degraded Mins, 
  1 Errored Secs, 1 Bursty Err Secs, 0 Severely Err Secs, 339 Unavail Secs

RFC 1406 exit.gif defines the error conditions or events and performance parameters that can be tracked for a DS-1 or E1 interface. The RFC specifies that the performance parameters be accumulated in 15-minute intervals and that up to 96 intervals (for a 24 hour period) be kept by an ATM device.

Error Event Description
Line Coding Violation (LCV) Occurrence is of one of these:
  • Bipolar Violation (BPV) - Defined on links that use AMI code as two consecutive pulses of the same polarity. Defined on links that use B8ZS or HDB3 line code as two consecutive pulses of the same polarity that are not part of the zero substitution code.
  • Excessive Zeroes (EXZ) - Defined on links that use AMI code that receives more than 15 contiguous zeroes. Defined on links that use B8ZS code that receives more than seven contiguous zeroes.
Path Coding Violation (PCV) Frame synchronization bit error in the D4 and E1 no-cyclic redundancy check (CRC) formats, or a CRC error in the Extended Super Frame (ESF) and E1 CRC formats.
Controlled Slip (CS) Replication or deletion of payload bits in a DS-1 frame to account for differences in the timing between a synchronous receiving interface (on a router) and the received signal.

Performance Parameters Description
Fr Loss Secs A severely errored framing second with one or more out-of-frame (OOF)* defects or a detected alarm indication signal (AIS) defect.
Line Err Secs Seconds in which one or more line code violations were detected.
Degraded Mins Minutes in which the estimated error rate exceeded a threshold in a one-minute interval. Degraded minutes is calculated from the number of available seconds, minus any severely errored seconds.
Errored Secs Defined on an ESF and E1 CRC link as one or more of these: path code violations, OOF defects, controlled slip events, or AIS defect. Also defined on a D4 and E1 no-CRC link as a bipolar violation.
Bursty Err Secs Seconds with from two to 320 path code violations, no severely errored frame defects and no AIS defects.
Severely Err Secs Severely errored second with one or more OOF defects or a detected AIS defect.
Unavail Secs Seconds in which the interface was not available because the router detected 10 contiguous severely errored seconds or a condition that led to a failure.

Note: OOF defects are defined on DS-1 links that detect two or more framing errors within a three-millisecond (msec) period for ESF signals and 0.75 msec for D4 signals, or two or more errors out of five or fewer consecutive framing bits. OOF defects are defined on E1 links that receive three consecutive frame alignment errors. In order to correct an out of frame defect, the interface searches for the correct framing pattern.

SAR Counter Totals

This is the segmentation and reassembly (SAR) counter totals information produced in the output:

  SAR counter totals across all links and groups: 
  0 cells output, 0 cells stripped 
  0 cells input, 0 cells discarded, 0 AAL5 frames discarded  
  0 pci bus err, 0 DMA FIFO full err, 0 rsm parity err 
  0 rsm syn err, 0 rsm/seg q full err, 0 rsm overflow err  
  0 Hs q full err, 0 no free buff q err, 0 seg underflow err  
  0 host seg stat q full err 

The IMA module for Cisco 2600 and 3600 series uses a single SAR chip for the multiple ATM physical interfaces. The SAR counter shows a single set of statistics for all links and groups.

Statistic Explanation
cells output The number of cells transmitted by the SAR across all interfaces.
cells stripped The number of cells stripped by the field programmable gate array (FPGA) on the network module. Can occur when the IMA layer is not up, but the SAR tries to send out cells.
Cells input The number of cells received by the SAR on all interfaces.
Cells discarded The number of cells discarded by the SAR because of such reasons as an unknown virtual circuit (VC).
AAL5 frames discarded The number of complete ATM adaptation layer 5 (AAL5) frames discarded.
pci bus error The number of times the interface experienced a read or write problem to host input/output (I/O) memory across the system PCI bus.
DMA FIFO full err The number of times the direct memory access (DMA) first-in-first-out (FIFO) was full when it transferred data to and from the host I/O memory.
pci bus err The number of times the PCI bus master encountered a fatal error and halted operation.
DMA FIFO full err The number of times the inbound Direct Memory Access (DMA) burst FIFO was nearly full.
rsm parity err The number of times the system detected a parity error on the reassembly (rsm) side of the ATM physical interface.
rsm syn err The number of times a synchronization error occurred on the reassembly side of the ATM physical interface.
rsm/seg q full err The number of times the ATM interface experienced a reassembly or segmentation queue full condition.
rsm overflow err The number of times the ATM interface experienced a reassembly overflow condition in which a cell was lost because the FIFO memory was full.
Hs q full err The number of times the host status queue was full.
seg underflow err The number of times a scheduled cell cannot be sent due to lack of PCI bandwidth.
host seg stat q full err The number of times the segmentation host status queue was full.

If your router reports RSM SYN ERROR log messages when you run the debug atm error command, check for any physical layer alarms as well as the clocking at both ends of the circuit.

If you experience a problem with your IMA virtual link, and see any non-zero values for any SAR counters other than cell input and output, please open a case with the Cisco Technical Assistance Center.

The show controllers atm Command for IMA Virtual Interfaces

Like physical ATM T1 interfaces, the IMA virtual group interface can appear in one of several states before it becomes active. The group starts in the Not Configured state and then enters the Start-Up state.

The show controllers atm[mod#]/ima[group#] command displays several important values at the bottom of the output. These values are highlighted in bold:

   ATM#show controllers atm1/ima0 
   Interface ATM1/IMA0 is down 
    Hardware is ATM IMA 
   [snip] 
   atm channel number is 2 
   link members are 0x4, active links are 0x0    
   Group status is blockedNe, 1 links configured,    
   Group Info: Configured links bitmap 0x4, Active links bitmap 0x0, 
    TX/Rx IMA_id 0x11/0x63,    
    NE    Group status is startUp, 
    frame length 0x80, Max Diff Delay 477, 
    1 min links, clock mode ctc, symmetry symmetricOperation, trl 2, 
    Group    Failure status is startUpNe. 
    Test pattern procedure is disabled    
   SAR counter totals across all links and groups:    
    0 cells output, 0 cells stripped    
    0 cells input, 0 cells discarded, 0 AAL5 frames discarded 
    0 pci bus err, 0 DMA FIFO full err, 0 rsm parity err 
    0 rsm syn err, 0 rsm/seg q full err, 0 rsm overflow err 
    0 Hs q full err, 0 no free buff q err, 0 seg underflow err 
    0 host seg stat q full err

This table lists the states of the IMA group state machine.

State Description
Not Configured The IMA group does not exist.
Start-up This end (the near end) waits to receive an indication that the far end is in Startup. The IMA group moves to the Start-up-Ack state when it can communicate with the far end and has recorded IMA ID, group symmetry, and other IMA group parameters.
Start-up-Ack When both sides of the link are enabled, they move through this state to the Insufficient-Links state.
Config-Aborted The far-end uses unacceptable configuration parameters, such as an unsupported IMA frame size, an incompatible group symmetry, or an unsupported IMA version.
Insufficient-Links The near end has accepted the far end group parameters, but the far end does not have sufficient links to move into the Operational state.
Blocked The group is blocked, even though sufficient links are active in both directions.
Operational The group is not inhibited and has sufficient links in both directions. The IMA interface can receive ATM layer cells and pass them from the IMA sublayer to the ATM layer.

Addition and Removal of Links to the IMA Group

The IMA specification of the ATM Forum exit.gif defines a Link Addition and Slow Recovery procedure to add links to the IMA group. These links can be new links or links on which physical layer errors have been corrected.

ICP cells advertise the state of all links in the IMA group. Each physical link has one state for receive and one state for transmit. In order to add new links to an IMA group, use the show controllers atm {mod#/port#} command and check these parameters on the physical link:

  • IMA Info - What is the receive (Rx) link state? What is the transmit (TX) link state? Ensure that both states are Active. If not, continue to analyze the output.

  • Link 2 Framer Info - What is the line status? Is it 0x1 or no alarm? Is the port active? Is the link available?

  • DS-1 MIB DATA - Are there any non-zero values for the physical-layer error counters in the current interval?

Once you have determined the status of the physical link, check the current status of the IMA group and its configured links with the show ima interface atm[mod#]/ima[group#] command. This command was introduced in Cisco IOS® Software Release 12.0(5)XK.

   Router#show ima interface atm2/IMA2 
   Interface atm2/IMA2 is up 
    Group index is 2 
    Ne state is operational, failure status is noFailure 
    active links bitmap 0x30    
    IMA Group Current Configuration: 
    TX/Rx configured links bitmap 0x30/0x30 
    TX/Rx minimum required links 1/1 
    Maximum allowed diff delay is 25ms, TX frame length 128 
    NE TX clock mode CTC, configured timing reference link atm2/4 
    Test pattern procedure is disabled    
    IMA Group Current Counters (time elapsed 12 seconds):    
    3 NE Failures, 3 Fe Failures, 4 Unavail Secs 
    IMA Group Total Counters (last 0 15 minute intervals):    
    0 NE Failures, 0 Fe Failures, 0 Unavail Secs 
    IMA link Information: 
    Link    Physical Status NearEnd Rx Status    Test Status 
    ----    --------------- -----------------    ----------- 
    atm2/4              up             active      disabled 
    atm2/5              up             active      disabled 

Related Information

Updated: Jun 05, 2005
Document ID: 10452