Guest

Cisco Catalyst 8500 Series Multiservice Switch Routers

Hardware Troubleshooting for Catalyst 8540/8510 MSRs and LightStream 1010 ATM Switch: Interface Connection Testing and Debugging

Document ID: 22014

Updated: Jun 05, 2005

   Print


Interface Connection Testing and Debugging


Contents


<<<Previous Section      Next Section>>>



Note: For detailed cabling and hardware information for each interface module, refer to the ATM Port Adapter and Interface Module Installation Guide. The default configurations for the various port adapters are described in Configuring ATM Network Interfaces.

Using SNMP to Gather Drop Statistics

The CISCO-ATM-CONN-MIB provides management information base (MIB) objects to gather per-VC statistics related to cell drops and discarded cells. Some of these objects were described in detail in previous sections.

The following are some of the more common objects that are used to troubleshoot VCs:

CiscoAtmVclEntry
ciscoAtmVclRxUpcMode
ciscoAtmVclEpdEnable
ciscoAtmVclUpcViolations
ciscoAtmVclEpdTpdCellDrops
ciscoAtmVclEpdTpdPacketDrops
ciscoAtmVclEpdTpdPacketsIn
ciscoAtmVclLsPerVcQThreshGrp
ciscoAtmVclClp0VcqFullCellDrops
ciscoAtmVclVcqClpThreshCellDrops

Note: This MIB is not available on routers with ATM interfaces.

Using the ping atm interface Command

To check ATM connection reachability and network connectivity, use the ping atm interface atm command in either privileged or user mode. You can use either an IP address or an ATM address prefix as a ping destination. You can also ping a neighbor switch router by selecting the segment loopback option. In privilege extended command mode, you can select other parameters such as repeat count and timeout values.

Command Purpose
ping atm interface atm card/subcard/port vpi vci {atm-prefix prefix | end-loopback
ip-address ip-address | seg-loopback}
Checks the interface connection.

Follow these steps to ping a specific ATM prefix in both normal and extended mode:

Step 1   In normal mode, use the ping atm interface atm command to confirm connectivity through a specific interface to an ATM address prefix.

Switch# ping atm interface atm 1/0/0 0 5 atm-prefix 47.009181000000000000000001
Type escape sequence to abort.
Sending 5, 53-byte OAM Echoes to 47.0091.8100.0000.0000.0000.0001..., timeout is
 5 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
Step 2   Check the Success rate field. It should be 100 percent. If not, check the interface configuration.

Step 3   In extended mode, use the ping atm interface atm command to confirm connectivity through a specific interface to an ATM address prefix and modify the default repeat or timeout.

Switch# ping
Protocol [ip]: atm

Interface [card/subcard/port]: 1/0/0

VPI [0]: 0

VCI [0]: 5

Send OAM-Segment-Loopback ? [no]:
Target IP address:
Target NSAP Prefix: 47.009181000000000000000001

Repeat count [5]:
Timeout in seconds [5]:10

Type escape sequence to abort.
Sending 5, 53-byte OAM Echoes to 47.0091.8100.0000.0000.0000.0001..., timeout is
 10 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms

Step 4   Check the Success rate field. It should be 100 percent. If not, check the interface configuration.

Note:  If you skip both destination IP address and the ATM prefix fields, the extended ping considers its neighbor switch as its destination and uses a segment-Loopback operation, administration, and maintenance (OAM) cell. In an IP address or ATM prefix case, the ping command always uses an end-to-end OAM loopback cell.

If the success rate is less than 100 percent, refer to the Configuring ATM Network Interfaces chapter in the ATM Switch Router Software Configuration Guide and confirm the interface configuration.

Performing OAM Loopback Tests

OAM performs fault management and performance management functions at the ATM management-plane (M-plane) layer.

Note: Current OAM implementation supports only the fault management function, which includes connectivity verification and alarm surveillance.

The ATM switch router fully supports the following ATM OAM cell flows:

  • F4 flows— OAM information flows between network elements used within virtual paths to report an unavailable path or a virtual path (VP) that cannot be guaranteed.
  • F5 flows— OAM information flows between network elements used within virtual connections to report degraded virtual channel (VC) performance such as late arriving cells, lost cells, and cell insertion problems.

You can configure both F4 and F5 flows as either end-to-end or segment-loopback, and they can be used with alarm indication signal (AIS) and remote defect indication (RDI) functions.

Note: Cells can be sent either on demand or periodically to verify link and connection integrity.

In addition to the standard OAM functions, the ATM switch router can also send OAM pings. See the Determining Network Connectivity section in the ATM Switch Router Software Configuration Guide. Using OAM cells containing the ATM node addresses or IP addresses of intermediate switch routers, you can determine the integrity of a chosen connection at any intermediate point along that connection. With this information, you can debug and troubleshoot the network connection.

OAM Operation

OAM software implements ATM Layer F4 and F5 OAM fault management functions. OAM performs standard loopback (end-to-end or segment) and fault detection and notification (AIS and RDI) for each connection. It also maintains a group of timers for the OAM functions. When there is an OAM state change such as loopback failure, OAM software notifies the connection management software. The network operator can enable or disable OAM operation for the following ATM switch router components:
  • Entire ATM switch router
  • Specific ATM interface
  • Each ATM connection
OAM AIS, RDI, and loopback operations are enabled or disabled for the entire switch router using the atm oam command in global configuration mode. Use the atm oam command in interface mode to configure OAM on a specific connection. For more information about configuring OAM operations, refer to the "Configuring Operation, Administration, and Maintenance" chapter in the ATM Switch Router Software Configuration Guide.

Note: The OAM configuration commands are not stored in the nonvolatile random-access memory (NVRAM).

If OAM operation is disabled, outgoing OAM cells are not generated, and all incoming OAM cells are discarded.

To support various OAM operations, the ATM switch router hardware provides OAM cell routing functions on a per-connection basis for each direction and for different OAM cell spans (segment and end-to-end). The hardware OAM cell routing determines the destination of an OAM cell received from the link or the network and then determines whether OAM cells are processed by the ATM switch router software.

The hardware can perform the following functions on OAM cells:

  • Intercept— Intercepted to the route processor queue and processed by the ATM switch router software
  • Relay— Relayed along user cells by hardware without any software processing
  • Discard— Discarded by hardware
An ATM connection consists of a group of network points, which are the edges of each ATM switch router or end system.

Each point can be one of the following:

  • Connection endpoint— The end of a connection where the user ATM cells are terminated
  • Segment endpoint— The end of a connection segment
  • Connecting point— The middle point of a connection segment
The following illustration shows the various loopback operations available:

OAM Loopback Testing

You can use the loopback test to pinpoint faults by looping a signal at various points in the network. Use the loopback test before and after the initiation of service. The following illustration shows how ATM OAM cell loopbacks are performed, first across the interface and then across different segments of the connection.

An ATM switch router generates the OAM cells and forwards them to another network element, which is responsible for returning them to the generating network elements.

Each loopback cell contains the ID of the generating network element and the ID of the network element that is looping the cells back to the originator. Any intermediate site must pass the cells on to the loopback site (the farthest point to which the cells progress) and the generating site (the point to which the cells return).

See ATM Cell Structures for a format description of the OAM loopback cell.

The ATM switch router provides the following three types of loopback tests:

  • Diagnostic
  • Line
  • PIF (physical interface)
Note:   If the loopback test is successful, then data is reaching the I/O module properly. However, a successful test does not verify whether the I/O module correctly encodes the data sent onto the line.

Configuring Loopback Examples

The following examples show how to perform loopback tests on the interfaces shown in the illustration below.

If users connected to the Fast Ethernet Catalyst 5000 switch in the manufacturing building are not able to connect to the other users outside their building (including the DNS server in the administration building), you should try a loopback test. Use the procedures described in this section to test the ATM switch router connections starting in the middle section and proceeding outward. The following illustration shows a loopback test configuration example:
 

Use the atm ping command to confirm the ATM connection between the administration and manufacturing buildings.

Command Purpose
ping atm interface atm card/subcard/port vpi [vci] [atm-prefix prefix] | [end-loopback] | [ip-address ip-address] | [seg-loopback]} Checks the interface connection.

Perform the interface loopback tests in the following order:

Test 1— Segment network-side loopback between ATM switch router AdminFl1Ls1, interface 1/0/0, and ATM switch router ManuFl1Ls1, interface 4/0/1

Test 2— Segment link-side loopback between the DNS server and ATM switch router AdminFl1Ls1, interface 4/0/0

Test 3— End-to-end loopback between the DNS server and the Catalyst 5000 Fast Ethernet switch, ManuFl1CaS1, interface 1/1

Test 1— Segment Network-Side Loopback Process

Follow these steps to ping the ATM virtual channel 2, 130 between the administration and manufacturing buildings, with a segment loopback signal in normal mode:

Step 1   Use the ping atm interface atm card/subcard/port command to confirm the VP connectivity.

AdminFl1Ls1# ping atm interface atm 1/0/0 2 seg-loopback


Type escape sequence to abort.
Sending Seg-Loopback 5, 53-byte OAM Echoes to a neighbor, timeout is 5 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
AdminFl1Ls1#
Step 2   Use the same command to ping the ATM virtual channel 2, 130 between the administration and manufacturing buildings with a segment loopback signal in normal mode:
AdminFl1Ls1# ping atm interface atm 1/0/0 2 130 seg-loopback

Type escape sequence to abort.
Sending Seg-Loopback 5, 53-byte OAM Echoes to a neighbor, timeout is 5 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
AdminFl1Ls1#
Step 3   Check the Success rate field. If the success rate is less that 100 percent, you have a problem on the 622-Mbps connection between the administration and manufacturing buildings.

Step 4   Check the cables and the interface configuration, using the procedures in the Performing Basic Interface Checks section in the ATM Switch Router Software Configuration Guide.

If the success rate is 100 percent, then this segment of the connection is not the problem. Proceed with the next phase of the interface loopback test.

Test 2— Segment Link-Side Loopback Process

Log in to the ATM switch router in the manufacturing building and use the ping atm interface atm command again to confirm the ATM connection between the ATM switch router and the Catalyst 5000 switches in the manufacturing building.

Use the following steps to ping the ATM virtual path 2 between the ATM switch router and the Catalyst 5000 switches in the manufacturing building, with a segment loopback signal in normal mode:

Step 1   Use the ping atm interface atm card/subcard/port command to confirm the VP connectivity.

ManuFl1Ls1# ping atm interface atm 4/0/0 2 seg-loopback

Type escape sequence to abort.
Sending Seg -Loopback 5, 53-byte OAM Echoes to a neighbor, timeout is 5 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
ManuFl1Ls1#
Step 2   Check the Success rate field. If the success rate is less that 100 percent, there is a problem on the OC-3 155-Mbps connection between the ATM switch router and the Catalyst 5000 switch in the manufacturing building.

Step 3   Check the cables and the interface configuration, using the procedures in Performing Basic Interface Checks in the ATM Switch Router Software Configuration Guide.

If the success rate is 100 percent, then this segment of the connection is not the problem. Proceed with the next phase of the interface loopback test.

Test 3— End-to-End Loopback Process

Check the end-to-end connection between the DNS server and the Catalyst 5000 switch in the manufacturing building.

The following is an example of the steps to ping the entire ATM virtual path between the administration and manufacturing buildings, with an end-to-end loopback signal in normal mode:

  Step 1   Use the ping atm interface atm card/subcard/port command to confirm the VP connectivity.

AdminFl1Ls1# ping atm interface atm 4/0/0 2 end-loopback

Type escape sequence to abort.
Sending end-Loopback 5, 53-byte OAM Echoes to a neighbor, timeout is 5 seconds:
!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
Step 2   Check the Success rate field. If the success rate is less that 100 percent, you have a problem on the OC-3 155-Mbps connection between the ATM switch router and the Catalyst 5000 switch in the manufacturing building.

Step 3   Check the cables and the interface configuration using the procedures in Performing Basic Interface Checks in the ATM Switch Router Software Configuration Guide.

If the success rate is 100 percent, then this segment of the connection is not the problem. Continue with the next phase of the interface test.

Using the debug Commands to Troubleshoot an Interface

The debug privileged EXEC commands can provide a wealth of information about the traffic being seen (or not seen) on an interface.

Caution: Exercise care when using debug commands. Many of these commands are processor intensive and can cause serious network problems (such as degraded performance or loss of connectivity) if they are enabled on an already heavily loaded switch router. When you finish using a debug command, remember to disable it with its specific no debug command (or use the no debug all command to turn off all debugging).

For detailed information about using the debug commands, see Debugging a Switch Router in the ATM Switch Router Software Configuration Guide.

To isolate problems and troubleshoot the physical connections of the ATM switch router, use the following debug commands in privileged EXEC mode. Use the no form of these commands to disable debugging.

Command Purpose
debug ports {aal5 [interface atm card/subcard/port] | dcu | ds3e3 | netclock | oc12 | oc3 | t1e1}  Starts debugging at the driver level for a specific port.
debug atm oam-all Starts debugging, using generic OAM cells.
debug atm oam-pkt Starts debugging, using OAM packets.
debug atm errors Starts debugging to display all ATM errors.
no debug all Disables all debugging.

Refer to the Configuring Interfaces chapter in the ATM Switch Router Software Configuration Guide in order to confirm the interface configuration.

Troubleshooting Information for TAC

Please provide the following information when opening a case with Cisco TAC to troubleshoot rejected cells:
  • What is the topology? What other devices feed into this ATM switch router?
  • Which interfaces have the highest traffic loads? Are these the same interfaces that are experiencing the rejected cells?
  • On which MSC chip are the rejected cells occurring? Is there a high amount of traffic flowing through this set of ports? For example, drops on MSC 0 point to a high amount of traffic on interfaces in slots 0 and 1.
  • Output from the following show commands:
    • show hardware
    • show running
    • show switch fabric
    • show atm resource or show controller (depending on the platform)

Related Information


Updated: Jun 05, 2005
Document ID: 22014