Contents

Introduction

Troubleshoot Serial Interfaces on Channelized STM-1/OC-3 SPA

Introduction

This document describes basic understanding and troubleshooting of Channelized Serial Interfaces on SPA-1XCHSTM1/OC3.

Troubleshoot Serial Interfaces on Channelized STM-1/OC-3 SPA

1. Issue "show ip interface brief' on the router to check status of serial interface in question. If interface status is "down/down", it means issues is at layer 1 but if interface status is "up/down" then issue is at Layer 2. If issue is at layer 2 then first step is to ensure that both sides are configured with the same encapsulation.

2. Find out the corresponding Administrative Unit (AU) and Tributary Unit Group (TUG) under the respective Synchronous Optical NETwork (SONET) Controller for the interface in question.

Example:

Assume interface Serial3/0/0.1/1/1/1:1 is down/ down. This notation can be explained as "interface serial [slot/subslot/port].[au-4/tug-3/tug-2/e1]:[channel-group]". So for interface Serial3/0/0.1/1/1/1:1 the parameters become:

Slot = 3

Subslot = 0

Port = 0

Au-4 =1

Tug-3 = 1

Tug-2 =1

E1= 1

Channel-group = 1

Note: This example would be used as a reference for rest of the document.

1. Check status of SONET [slot/subslot/port] controller with these commands:
Router#show running-config | sec controller SONET 3/0/0

controller SONET 3/0/0

framing sdh

clock source line

aug mapping au-4

!

au-4 1 tug-3 1

mode c-12

tug-2 1 e1 1 channel-group 0 timeslots 1-8

tug-2 1 e1 1 channel-group 1 timeslots 13-14

tug-2 1 e1 1 channel-group 2 timeslots 9-12

<Output suppressed>

Router#show controllers sonet 3/0/0 | begin AU-4 1, TUG-3 1, TUG-2 1, E1 1

AU-4 1, TUG-3 1, TUG-2 1, E1 1 (C-12 1/1/1/1) is down

VT Receiver has LP-T_MIS.

timeslots: 1-30

Transmitter is sending LOF Indication.

Receiver is getting AIS.

Framing is crc4, Clock Source is Internal, National bits are 0x1F.

Data in current interval (693 seconds elapsed):

0 Line Code Violations, 0 Path Code Violations

0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins

0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs

262 Unavail Secs, 0 Stuffed Secs

2. Next step is to find if issue is local to the router or on the remote side. Enable a soft loop on AU-4 1, TUG-3 1, TUG-2 1, E1 1 with the use of these commands:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#controller SONET 3/0/0

Router(config-controller)# au-4 1 tug-3 1

Router(config-ctrlr-tug3)#  tug-2 1 e1 1 loopback local

Note: You can enable loopback on complete E1 and not just on specific channels. If loopback is applied on complete E1, all corresponding channels-groups of that E1 will go down.

If interface comes up at both layer 1 and 2, then issue is not local to the router. At this moment, you need to troubleshoot on the remote side. If however, interface does not come up then proceed to Step 5.

Remember to remove the loopback in both cases with the use of these commands:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#controller SONET 3/0/0

Router(config-controller)# au-4 1 tug-3 1

Router(config-ctrlr-tug3)#  no tug-2 1 e1 1 loopback local

3. If interface doesn’t come up then try to do a shut-no-shut on the TUG as shown:

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#controller SONET 3/0/0

Router(config-controller)# au-4 1 tug-3 1

Router(config-ctrlr-tug3)#  tug-2 1 e1 1 shutdown

Router(config-ctrlr-tug3)#  no tug-2 1 e1 1 shutdown

4. If interface still doesn’t come up then try to reconfigure the TUGS as shown:

Router#show running-config | sec controller SONET 3/0/0

controller SONET 3/0/0

framing sdh

clock source line

aug mapping au-4

!

au-4 1 tug-3 1

mode c-12

tug-2 1 e1 1 channel-group 0 timeslots 1-8

tug-2 1 e1 1 channel-group 1 timeslots 13-14

tug-2 1 e1 1 channel-group 2 timeslots 9-12

<Output suppressed>

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#controller SONET 3/0/0

Router(config-controller)# au-4 1 tug-3 1

Router(config-ctrlr-tug3)#  no tug-2 1 e1 1

Router(config-ctrlr-tug3)#  tug-2 1 e1 1 channel-group 0 timeslots 1-8

Router(config-ctrlr-tug3)#  tug-2 1 e1 1 channel-group 1 timeslots 13-14

Router(config-ctrlr-tug3)#  tug-2 1 e1 1 channel-group 2 timeslots 9-12

Note: Ensure to configure all channel-groups for the TUG that you reconfigured.

5. If all the interfaces are down under the controller, then check and give soft loop to the complete SONET controller.

Router#show ip interface brief

Interface              IP-Address          OK? Method Status                Protocol

Serial3/0/0.1/1/1/1:0  192.168.1.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/1:1  192.168.2.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/1:2  192.168.3.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/1:3  192.168.4.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/1:4  192.168.5.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/1:5  192.168.6.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/2:0  192.168.7.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/2:1  192.168.8.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/2:2  192.168.9.1         YES NVRAM  down                  down

Serial3/0/0.1/1/1/2:3  192.168.10.1        YES NVRAM  down                  down

Serial3/0/0.1/1/1/3:0  192.168.11.1        YES NVRAM  down                  down

Serial3/0/0.1/1/1/3:1  192.168.12.1        YES NVRAM  down                  down

Serial3/0/0.1/1/1/3:2  192.168.13.1        YES NVRAM  down                  down

Serial3/0/0.1/1/2/1:0  192.168.14.1        YES NVRAM  down                  down

Serial3/0/0.1/1/2/2:0  192.168.15.1        YES NVRAM  down                  down

Serial3/0/0.1/1/2/3:0  192.168.16.1        YES NVRAM  down                  down

Router(config)#controller sonet 3/0/0

Router(config-controller)#loopback local

If the controller comes up (as shown) then issue is again not with the local card and you need to troubleshoot the remote end.

Router#show ip interface brief

Interface              IP-Address          OK? Method Status                Protocol

Serial3/0/0.1/1/1/1:0  192.168.1.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/1:1  192.168.2.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/1:2  192.168.3.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/1:3  192.168.4.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/1:4  192.168.5.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/1:5  192.168.6.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/2:0  192.168.7.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/2:1  192.168.8.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/2:2  192.168.9.1         YES NVRAM  up                    up

Serial3/0/0.1/1/1/2:3  192.168.10.1        YES NVRAM  up                    up

Serial3/0/0.1/1/1/3:0  192.168.11.1        YES NVRAM  up                    up

Serial3/0/0.1/1/1/3:1  192.168.12.1        YES NVRAM  up                    up

Serial3/0/0.1/1/1/3:2  192.168.13.1        YES NVRAM  up                    up

Serial3/0/0.1/1/2/1:0  192.168.14.1        YES NVRAM  up                    up

Serial3/0/0.1/1/2/2:0  192.168.15.1        YES NVRAM  up                    up

Serial3/0/0.1/1/2/3:0  192.168.16.1        YES NVRAM  up                    up

Remember to remove the loopback configuration before you proceed further:

Router(config)#controller sonet 3/0/0

Router(config-controller)#no loopback local

6. If the interfaces and the controller does not come up then the issue could be with local Shared Port Adapter (SPA) or with SPA Interface Processor (SIP). Soft reset on the SPA or SIP can help resolve the issue.

If issue is with specific SPA in a SIP and all other SPAs work fine (interfaces are UP and passing traffic), then, if you swap the SPAs in the SIP, it can be helpful to determine if problem is with SIP or SPA. If only one SPA is present in the SIP, then SIP reload can be also be done.

Note: When you relaod a SIP or SPA, it can cause network downtime as all the links associated with SIP or SPA would go down.

Command to reload SPA

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#service internal

Router(config)#end

Router#hw-module subslot <number> reload

Command to reload SIP

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#service internal

Router(config)#end

Router#hw-module module <SIP Slot number> reset

For further assistance, open a Service Request with Cisco Technical Assistance Centre (TAC). Provide details of all the tests done as well as the ‘show tech-support’ output from the router.