Guest

Device Signaling

T1/E1 Loopback Testing and Troubleshooting

T1/E1 Loopback Testing and Troubleshooting

Document ID: 116492

Updated: Oct 09, 2013

Contributed by Baktha Muralidharan and Apor Kurucz, Cisco TAC Engineers.

   Print

Introduction

A common issue in VoIP networks with a digital interface connection to a telecommunications provider (telco) is that the ISDN or channel associated signaling (CAS) circuit does not come up or stay up. Such issues can be complex because:

  1. Faulty components might reside in several places - for example, within the Cisco domain or in a third-party (telco) domain.
  2. Multiple components impact the status of the ISDN Primary Rate Interface (PRI) or T1 CAS circuit. The problem could be mismatched configuration across the telco interface, which leads to clock slips, line/path violations, a damaged cable, a bad card, or telco issues.
  3. The Cisco Technical Assistance Center (TAC) does not directly deal with third party organizations.

How can there be efficient and effective progress on the issue? This document describes an important and useful troubleshooting method, known as loopback testing, and covers various loopback testing techniques.

Notes:

Use the Command Lookup Tool (registered customers only) in order to obtain more information on the commands used in this document.

The Output Interpreter Tool (registered customers only) supports certain show commands. Use the Output Interpreter Tool in order to view an analysis of show command output.

Refer to Important Information on Debug Commands before you use debug commands.

Background

Loopback testing is a very effective way to isolate a failing T1 (or E1). The basic idea behind loopback testing is:

  1. Start at the Voice/WAN Interface Card (VWIC) on the Cisco gateway.
  2. Perform loopback testing. If the testing is successful, it eliminates the VWIC as the problem component.
  3. Move the loopback testing out to the next component, and repeat Steps 1-3. Progress in stages toward the telco point of demarcation (demarc).

The components you should try to eliminate as problematic include the VWIC (card and port) and the cable run (up to the SmartJack).

SmartJack

The SmartJack is often referred to or is involved in TAC calls on T1/E1 issues. A SmartJack is a type of network interface device (NID) that terminates the PRI/T1 from the Cisco gateway and that also provides some diagnostic capabilities.

A very common capability provided by a SmartJack is loopback, where the signal from the telco is transmitted back to the telco.

Telcos consider everything connected to the inside of the SmartJack as the local loop and consider all local loop equipment the responsibility of the customer. This figure illustrates a SmartJack.

116492-trouble-t1e1-01.png

Loopback Test Types

This figure provides a general overview of loopback testing.

116492-trouble-t1e1-02.png

This document describes three types of loopback tests:

  • Soft loopbacks (also known as software loops or soft loops) are commands from test equipment that cause a network interface unit (NIU) or CSU to automatically send traffic back towards the sender.
  • A hard loopback (also known as a hard loop) is a physical loop created by wire. A loopback plug or an RJ-48X connector can create this hard loopback.
  • The telco-assisted loopback is done with the assistance of the telco. You should explore this option only after you rule out the Cisco gateway and the cable run (to the telco demarc) as the source of problems.

See Loopback Tests for T1/56K Lines for detailed descriptions of loopback tests. You can safely ignore the references to CSUs and Data Service Units (DSUs) in this document. In Cisco voice gateways, CSUs and DSUs are integral to the VWICs on Cisco voice-enabled gateways.

Soft Loopback

Note: Soft loopback is intrusive and will impact service.

Soft loopback testing is accomplished with a set of Cisco IOS® software configuration commands on the Cisco gateway. The commands cause the WAN interface card (WIC) driver to automatically send traffic back towards the sending T1/E1 port.

Soft loopback does not require any hardware changes or re-configuration, as shown in this figure.

116492-trouble-t1e1-03.jpg

This procedure describes how to test a soft loopback:

  1. Put the T1 or E1 in local loopback mode.
  2. Configure the channel-group on the controller.
  3. Configure an IP address on the serial interface.
  4. Perform Internet Control Message Protocol (ICMP) pings, and verify that the counts for 'packets input' and 'packets output' increment. See Hard and Soft Loopback Verification for details of this step.

This is an example of the configuration of the channel-group on the controller:

Router#configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

Router(config)#controller t1 0/0/0

Router(config-controller)#no pri-group timeslots 1-24
Router(config-controller)#channel-group 0 timeslots 1-24 speed 64


!--- This automatically creates a single Serial0:0 interface.

Router(config-controller)#loopback local

!--- The loopback local command above is only necessary for software loopbacks.

Router(config-controller)#exitRouter(config)#interface serial 0/0/0:0

Router(config-if)#encapsulation hdlc

!--- Note: All loopback testing is done with hdlc encapsulation.

Hard Loopback

Note: Hard loopback tests are intrusive and will impact service.

In a hard loopback, a special loopback plug is used in order to loop the traffic from the T1 port back into the T1 port. This figure illustrates the setup for hard loopback.

116492-trouble-t1e1-04.jpg

There are two approaches to test a hard loopback:

  1. Test as an ISDN circuit.
  2. Test as an IP interface.

ISDN Circuit

The first approach, test as an ISDN circuit, offers limited scope for testing and verification.

ISDN Layer1 can be tested. If the VWIC is working correctly, the show controller t1 command produces output similar to that shown in this example:

T1 0/0/0 is up.
Applique type is Channelized T1
Cablelength is long 0db
No alarms detected.
alarm-trigger is not set
Soaking time: 3, Clearance time: 10
AIS State:Clear LOS State:Clear LOF State:Clear
Version info Firmware: 20100222, FPGA: 13, spm_count = 0
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
CRC Threshold is 320. Reported from firmware is 320.
Data in current interval (24 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

ISDN Layer 2 can be partially tested. While you can verify that  Set Asynchronous Balanced Mode (SABME) messages flow across the interface, the other, usual Q.921 messages, such as RR, RRf, and RRp, are not seen. Instead, you see this type of output:

004800: *Aug 12 16:17:01.319: ISDN Se0/0/0:23 Q921: L2_EstablishDataLink: 
sending SABME
004801: *Aug 12 16:17:01.319: ISDN Se0/0/0:23 Q921: User TX ->
SABMEp sapi=0 tei=0
004802: *Aug 12 16:17:01.323: ISDN Se0/0/0:23 Q921: User RX <-
BAD FRAME(0x00017F)

004803: *Aug 12 16:17:02.319: ISDN Se0/0/0:23 Q921: User TX ->
SABMEp sapi=0 tei=0

This is expected. For the ISDN interface to work, one side must be configured as a protocol network, and the other side as a protocol user. However, this is not possible since there is only one interface with the loopback. Consequently, you see that the ISDN status oscillates between AWAITING_ESTABLISHMENT and TEI_ASSIGNED.

ISDN Serial0/0/0:23 interface
dsl 0, interface ISDN Switchtype = primary-4ess
Layer 1 Status:
ACTIVE
Layer 2 Status:
TEI = 0, Ces = 1, SAPI = 0, State = AWAITING_ESTABLISHMENT
Layer 3 Status:
0 Active Layer 3 Call(s)
Active dsl 0 CCBs = 0
The Free Channel Mask: 0x807FFFFF
Number of L2 Discards = 0, L2 Session ID = 5

ISDN Layer 3 never comes up.

Another limitation with this approach is that it does not work if the T1 is configured as a T1 CAS.

One advantage of this approach however, is that no configuration change is required on the Cisco IOS software. The only procedure is:

  1. Make or buy a loopback plug.
  2. Plug the loopback into the RJ-45 connector on the port in question on the VWIC.

Use the show controller t1 command in order to verify that the T1 controller comes up, and use the debug isdn q921 command in order to verify the flow of Q.921 messages. ISDN Layer 3 is, of course, not possible.

IP Interface

The other approach, test as an IP interface, is also known as "test as a data T1." This approach lets you conduct ICMP ping tests, which seems better because you can verify that the VWIC (card and port) is good all the way up to Layer 3. Note, however, that the Layer 3 is Open System Interconnection (OSI) Layer 3, not ISDN Layer 3.

Tip: This method is versatile because it works regardless of whether the T1 is being used as an ISDN interface or as an T1 CAS interface.

This procedure describes how to test as an IP interface:

  1. Make or buy a loopback plug.
  2. Plug the loopback into the RJ-45 connector on the port in question on the VWIC.
  3. Configure the channel-group on the controller.
  4. Configure an IP address on the serial interface.
  5. Perform ICMP pings, and verify that the count for 'packets input' and 'packets output' increments. See Hard and Soft Loopback Verification for details of this step.

See the procedure to create a loopback plug for a T1 CSU/DSU in Loopback Tests for T1/56K Lines.

This is an image of a T1/E1 loopback plug:

116492-trouble-t1e1-05.jpg

This is an example of the configuration of the channel-group on the controller:

Router#configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

Router(config)#controller t1 0/0/0
Router(config-controller)#no pri-group timeslots 1-24
Router(config-controller)#channel-group 0 timeslots 1-24 speed 64

!--- This automatically creates a single Serial0/0/0:0 interface.

Router(config-controller)#exit

Router(config)#interface serial 0/0/0:0
Router(config-if)#encapsulation hdlc

!--- Note: All loopback testing is done with hdlc encapsulation.

The show controller command produces results similar to these:

T1 0/0/0 is up.
Applique type is Channelized T1
Cablelength is long 0db
No alarms detected.
alarm-trigger is not set
Soaking time: 3, Clearance time: 10
AIS State:Clear LOS State:Clear LOF State:Clear
Version info Firmware: 20100222, FPGA: 13, spm_count = 0
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
CRC Threshold is 320. Reported from firmware is 320.
Data in current interval (2 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

Hard and Soft Loopback Verification

Loopback testing to verify the interface at ISDN Layer 3 level is not possible because ISDN Layer 2 does not come up on a loopback setup. Therefore, only testing as an IP interface is possible. Once the configuration steps are complete, perform ICMP pings:

Router(config-if)#ip address 172.53.11.1 255.255.0.0
Router(config-if)#ping 172.53.11.1

Check the interface counters in order to verify that the count for 'packets input' and 'packets output' increments. This is a sample output of the show interfaces serial slot/port command:

Router#sho int ser 0/0/0:0
Serial0/0/0:0 is up, line protocol is up
Hardware is GT96K Serial
Internet address is 172.53.11.1/16
MTU 1500 bytes, BW 1536 Kbit/sec, DLY 20000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation HDLC, loopback not set
Keepalive set (10 sec)
Last input 00:00:04, output 00:00:04, output hang never
Last clearing of "show interface" counters 00:47:05
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: weighted fair
Output queue: 0/1000/64/0 (size/max total/threshold/drops)
Conversations 0/1/256 (active/max active/max total)
Reserved Conversations 0/0 (allocated/max allocated)
Available Bandwidth 1152 kilobits/sec
5 minute input rate 1000 bits/sec, 1 packets/sec
5 minute output rate 1000 bits/sec, 1 packets/sec
20 packets input, 2723 bytes, 0 no buffer
Received 4 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
20 packets output, 2723 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
1 carrier transitions

Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags

Note: Perform an extended ping in order to test for possible flapping conditions.

Cable Run Test

Once you determine that the VWIC is working correctly, use this procedure to test and eliminate the cable run (to the telco demarc) as the source of problems:

  1. Remove the loopback plug from the VWIC port.
  2. Connect the cable to the VWIC port.
  3. Disconnect the cable from SmartJack.
  4. Plug the loopback to that end of the cable run.
  5. Perform loopback tests.

If the ICMP pings are successful, the test is succesful, which indicates that the cable is fine. If there are cuts or other damages to the cable run, you see that the T1 controller stays down, caused by the loss of signal (LOS):

Router#show controller t1
T1 0/0/0 is down.
Applique type is Channelized T1
Cablelength is long 0db
Transmitter is sending remote alarm.
Receiver has loss of signal.
alarm-trigger is not set
Soaking time: 3, Clearance time: 10
AIS State:Clear LOS State:Failure LOF State:Failure
Version info Firmware: 20100222, FPGA: 13, spm_count = 0
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
CRC Threshold is 320. Reported from firmware is 320.
Data in current interval (395 seconds elapsed):
25 Line Code Violations, 1 Path Code Violations
0 Slip Secs, 0 Fr Loss Secs, 1 Line Err Secs, 0 Degraded Mins
1 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 34 Unavail Secs
Total Data (last 24 hours)
25 Line Code Violations, 1 Path Code Violations,
14 Slip Secs, 0 Fr Loss Secs, 3 Line Err Secs, 1 Degraded Mins,
15 Errored Secs, 0 Bursty Err Secs, 2 Severely Err Secs, 349 Unavail Secs

Note: Non-zero line and path code violations do not necessarily indicate problems with the cable. When you move the loopback plug from the VWIC port to the end of the cable run, line and path code violations are triggered. After you move the loopback plug, you can clarify this if you first clear the controller counters with the clear controller t1 0/0/0 command, then see if line and path code violations increment.

T1 CAS

Use the procedure described in IP Interface.

E1

There is no difference between loopback testing for a T1 or an E1.

Telco-Assisted Loopback

Note: Telco-assisted loopback tests may impact service.

The telco (also known as the carrier, telephone company, telecommunications provider, or service provider) is the provider of the T1/E1 circuit service.

If you are unable to perform the hard and soft loopback tests or if the hard and soft loopback tests reveal that that the Cisco gateway and the cable run (to the telco demarc) are working correctly, telco-assisted loopback might be an option.

There are two possibilities:

  1. Ask the telco to provide loopbacks towards your premises from the telco switches. Monitor the looped circuit from the router. In this scenario, test the circuit as an ISDN interface.
  2. Contact your telco, and ask them to perform a loopback test to the SmartJack. The telco can test the line from the central office  and does not need to have test equipment at your site. Usually, the telco can activate the loopback remotely and does not need to have personnel at your site. When looped back, your equipment is disconnected from the line.

This is a figure of the two possibilities for telco-assisted loopbacks:

116492-trouble-t1e1-06.png

Related Information

Updated: Oct 09, 2013
Document ID: 116492