Cisco ONS 15327 Troubleshooting Guide, Release 7.0 - Chapter 1, General Troubleshooting [Cisco ONS 15300 Series]

Table Of Contents

General Troubleshooting

1.1  Troubleshooting Circuit Paths with Loopbacks

1.2  Troubleshooting Electrical Circuit Paths with Loopbacks

1.2.1  Perform a Facility Loopback on a Source Node XTC Port

Create the Facility Loopback on the Source Node XTC Port

Test the DS-N Facility Loopback

Test the DS-N Cabling

Test the XTC Card

Test the MIC Cabling

Test the MIC Card

1.2.2  Perform a Hairpin on a Source Node XTC Port

Create the Hairpin on the Source Node Port

Test the Hairpin Circuit

Test the Alternate Source XTC Card

Retest the Original Source XTC Card

1.2.3  Perform a Terminal Loopback on a Destination Node XTC Port

Create the Terminal Loopback on a Destination Node XTC Port

Test the Terminal Loopback Circuit on the Destination Node XTC Port

Test the Destination XTC Card

1.2.4  Perform a Hairpin Test on a Destination Node XTC Port

Create the Hairpin Loopback Circuit on the Destination Node XTC Card

Test the Hairpin Circuit

Test the Alternate Destination XTC Card

Retest the Original Destination XTC Card

1.2.5  Perform a Facility Loopback on a Destination XTC Card

Create a Facility Loopback Circuit on a Destination Node XTC Port

Test the Facility Loopback Circuit

Test the DS-N Cabling

Test the XTC Card

Test the MIC Card

1.3  Troubleshooting Optical Circuit Paths with Loopbacks

1.3.1  Perform a Facility Loopback on a Source-Node OC-N Port

Create the Facility Loopback on the Source OC-N Port

Test the Facility Loopback Circuit

Test the OC-N Card

1.3.2  Perform a Cross-Connect Loopback on the Source OC-N Port

Create the Cross-Connect Loopback on the Source OC-N Port

Test the Cross-Connect Loopback Circuit

Test the Standby XTC Card

Retest the Original XTC Card

1.3.3  Perform a Terminal Loopback on a Source-Node OC-N Port

Create the Terminal Loopback on a Source Node OC-N Port

Test the Terminal Loopback Circuit

Test the OC-N Card

1.3.4  Perform a Facility Loopback on an Intermediate-Node OC-N Port

Create the Facility Loopback on an Intermediate-Node OC-N Port

Test the Facility Loopback Circuit

Test the OC-N Card

1.3.5  Perform a Terminal Loopback on an Intermediate-Node OC-N Port

Create the Terminal Loopback on an Intermediate-Node OC-N Port

Test the Terminal Loopback Circuit

Test the OC-N Card

1.3.6  Perform a Facility Loopback on a Destination-Node OC-N Port

Create the Facility Loopback on a Destination Node OC-N Port

Test the Facility Loopback Circuit

Test the OC-N Card

1.3.7  Perform a Terminal Loopback on a Destination-Node OC-N Port

Create the Terminal Loopback on a Destination-Node OC-N Port

Test the Terminal Loopback Circuit

Test the OC-N Card

1.4  Using CTC Diagnostics

1.4.1  Card LED Lamp Tests

Verify Card LED Operation

Verify G-Series Ethernet Card LED Operation

Verify E-Series Ethernet Card LED Operation

1.4.2  Retrieve Diagnostics File Button

Off-Load the Diagnostics File

1.4.3  Data Communications Network (DCN) Tool

1.4.4  Bidirectional Diagnostic Circuit

Create a Bidirectional Diagnostic Circuit

1.5  Restoring the Database and Default Settings

1.5.1  Restore the Node Database

1.6  PC Connectivity Troubleshooting

1.6.1  PC System Minimum Requirements

1.6.2  Sun System Minimum Requirements

1.6.3  Supported Platforms, Browsers, and JREs

1.6.4  Unsupported Platforms and Browsers

1.6.5  Unable to Verify the IP Configuration of Your PC

Verify the IP Configuration of Your PC

1.6.6  Browser Login Does Not Launch Java

Reconfigure the PC Operating System Java Plug-in Control Panel

Reconfigure the Browser

1.6.7  Unable to Verify the NIC Connection on Your PC

1.6.8  Verify PC Connection to the ONS 15327 (Ping)

Ping the ONS 15327

1.6.9  The IP Address of the Node is Unknown

Retrieve Unknown Node IP Address

1.7  CTC Operation Troubleshooting

1.7.1  Unable to Launch CTC Help After Removing Netscape

Set Internet Explorer as the Default Browser for CTC

1.7.2  Unable to Change Node View to Network View

Reset the CTC_HEAP Environment Variable for Windows

Reset the CTC_HEAP Environment Variable for Solaris UNIX Workstation

1.7.3  Browser Stalls When Downloading CTC JAR Files from XTC Card

Disable the VirusScan Download Scan

1.7.4  CTC Does Not Launch

Redirect the Netscape Cache to a Valid Directory

1.7.5  Sluggish CTC Operation or Login Problems

Delete the CTC Cache File Automatically

Delete the CTC Cache File Manually

1.7.6  Node Icon is Gray on CTC Network View

1.7.7  CTC Cannot Launch Due to Applet Security Restrictions

Manually Edit the java.policy File

1.7.8  Java Runtime Environment Incompatible

Launch CTC to Correct the Core Version Build

1.7.9  Different CTC Releases Do Not Recognize Each Other

1.7.10  Username or Password Does Not Match the XTC Card Information

Verify Correct Username and Password

1.7.11  No IP Connectivity Exists Between Nodes

1.7.12  DCC Connection Lost

1.7.13  "Path in Use" Error When Creating a Circuit

Cancel the Circuit Creation and Start Over

1.7.14  Calculate and Design IP Subnets

1.7.15  Ethernet Connections

Verify Ethernet Connections

1.7.16  VLAN Cannot Connect to Network Device from Untag Port

Change VLAN Port Tag and Untagged Settings

1.8  Circuits and Timing

1.8.1  Circuit Transitions to the OOS-PARTIAL Service State

View the State of Circuit Nodes

1.8.2  Circuits Remain in PARTIAL Status

1.8.3  AIS-V on XTC-28-3 Card Unused VT Circuits

Clear AIS-V on XTC-28-3 Card Unused VT Circuits

1.8.4  Circuit Creation Error with VT1.5 Circuit

1.8.5  XTC Card Does Not Report AIS-P From External Equipment

1.8.6  OC-3 and DCC Limitations

1.8.7  ONS 15327 Switches Timing Reference

1.8.8  Holdover Synchronization Alarm

1.8.9  Free-Running Synchronization Mode

1.8.10  Daisy-Chained BITS Not Functioning

1.8.11  Blinking STAT LED after Installing a Card

1.9  Fiber and Cabling

1.9.1  Bit Errors Appear for a Traffic Card

1.9.2  Faulty Fiber-Optic Connections

Verify Fiber-Optic Connections

Remove SFP Connectors

Install SFP Connectors

1.10  Power and LED Tests

1.10.1  Power Supply Problems

Isolate the Cause of Power Supply Problems

1.10.2  Power Consumption for Node and Cards

1.10.3  Lamp Tests for Card LEDs

General Troubleshooting

Note The terms "Unidirectional Path Switched Ring" and "UPSR" may appear in Cisco literature. These terms do not refer to using Cisco ONS 15xxx products in a unidirectional path switched ring configuration. Rather, these terms, as well as "Path Protected Mesh Network" and "PPMN," refer generally to Cisco's path protection feature, which may be used in any topological network configuration. Cisco does not recommend using its path protection feature in any particular topological network configuration.

This chapter provides procedures for troubleshooting the most common problems encountered when operating a Cisco ONS 15327. To troubleshoot specific ONS 15327 alarms, see "Alarm Troubleshooting." If you cannot find what you are looking for contact the Cisco Technical Assistance (TAC). For more information, see the "Obtaining Technical Assistance" section on page xxxvii.

For an update on End-of-Life and End-of-Sale notices, refer to http://cisco.com/en/US/products/hw/optical/ps2001/prod_eol_notices_list.html.

This chapter includes the following sections about network problems:

•Troubleshooting Circuit Paths with Loopbacks—Describes loopbacks and hairpin circuits, which you can use to test circuit paths through the network or logically isolate faults.

Note For network acceptance tests, refer to the Cisco ONS 15327 Procedure Guide.

•Troubleshooting Electrical Circuit Paths with Loopbacks—Explains how to use loopback tests described in "Troubleshooting Circuit Paths with Loopbacks" to isolate trouble on DS-N electrical circuits.

•Troubleshooting Optical Circuit Paths with Loopbacks—Explains how to use loopback tests described in "Troubleshooting Circuit Paths with Loopbacks" to isolate trouble on OC-N optical circuits.

The remaining sections describe symptoms, problems, and solutions that are categorized according to the following topics:

•Using CTC Diagnostics—Provides procedures for testing LED functionality, downloading a machine-readable diagnostic information file to be used by Cisco TAC, and creating a diagnostic Virtual Tributary (VT) circuit.

•Restoring the Database and Default Settings—Provides procedures for restoring software data and restoring the node to the default setup.

•PC Connectivity Troubleshooting—Provides troubleshooting procedures for PC and network connectivity to the ONS 15327.

•CTC Operation Troubleshooting—Provides troubleshooting procedures for Cisco Transport Controller (CTC) login or operation problems.

•Circuits and Timing—Provides troubleshooting procedures for circuit creation and error reporting as well as timing reference errors and alarms.

•Fiber and Cabling—Provides troubleshooting procedures for fiber and cabling connectivity errors.

•Power and LED Tests—Provides information about power issues and LED testing for specific cards.

1.1  Troubleshooting Circuit Paths with Loopbacks

Use loopbacks and hairpins to test newly created circuits before running live traffic or to logically locate the source of a network failure. All ONS 15327 line (traffic) cards, except the E10/100-4, allow loopbacks and hairpins. The G-Series Ethernet card allows terminal and facility loopbacks on the OC-N circuit path, like the ONS 15327 OC-N cards.

Caution On OC-N cards, a facility loopback applies to the entire card and not an individual circuit. Exercise caution when using loopbacks on an OC-N card carrying live traffic.

Note Do not use loopbacks to verify circuit switch times or traffic hits because it could exceed 60 msec. For switch times, a test set should be placed at both ends of the circuits.

Caution A lock out of protection must be executed before putting a two-fiber or four-fiber bidirectional line switched ring (BLSR) span into a facility or terminal loopback state. That is, a span lockout of one side (such as the east side) of a two-fiber BLSR is required before operating a facility loopback on the same (east) side of the ring. A span lockout of one protection side (such as the east protection side) of a four-fiber BLSR is required before operating a facility or terminal loopback on the same (east) side working line of the ring. If you do not execute the lockout prior to creating the loopback, the ring can become stuck in an anomalous state after you release the loopback.

Note In CTC, a terminal loopback is sometimes called a terminal (inward) loopback, and a facility loopback can be called a facility (line) loopback. These terms are used to indicate the direction of the loopback signal--that is, inward toward the port for the terminal loopback, and outward toward the line for the facility loopback.

A facility loopback tests the line interface unit (LIU) of a card, the mechanical interface card (MIC), and related cabling. After applying a facility loopback on a port, use a test set to run traffic over the loopback. A successful facility loopback isolates the LIU, the MIC, or the cabling plant as the potential cause of a network problem. Figure 1-1 shows a facility loopback on an XTC-14 or XTC-28-3 card.

Figure 1-1 Facility Loopback Process on an XTC Card

To test the LIU on an OC-N card, connect an optical test set to the OC-N port and perform a facility loopback or use a loopback or hairpin on a card that is farther along the circuit path. Figure 1-2 shows a facility loopback on an OC-N card.

Caution Before performing a facility loopback on an OC-N card, make sure the card contains at least two data communications channel (DCC) paths to the node where the card is installed. A second DCC provides a nonlooped path to log into the node after the loopback is applied, thus enabling you to remove the facility loopback. Ensuring a second DCC is not necessary if you are directly connected to the ONS 15327 containing the loopback OC-N card.

Figure 1-2 Facility Loopback Process on an OC-N Card

A terminal loopback tests a circuit path as it passes through the XTC card and loops back from the card with the loopback. Figure 1-3 shows a terminal loopback on an OC-N card. The test-set traffic comes in on the MIC DS-N ports and goes through the XTC card to the OC-N card. The terminal loopback on the OC-N card turns the signal around before it reaches the LIU and sends it back through the XTC card to the MIC. This test verifies that the XTC card cross-connect circuit paths are valid, but does not test the LIU on the OC-N card.

Setting a terminal loopback on the G-Series card might not stop the Tx Packets counter or the Rx Packet counters on the CTC card view Performance > Statistics tab from increasing. The counters can increment even though the looped-back port has temporarily disabled the transmit (Tx) laser and is dropping any received packets.

The Tx Packet statistic continues to increment because the statistic is not based on the packets transmitted by the Tx laser but on the Tx signal inside the G-Series card. In normal in-service port operation, the Tx signal being recorded does result in the Tx laser transmitting packets, but in a terminal loopback this signal is being looped back within the G-Series card and does not result in the Tx laser transmitting packets.

The Rx Packet counter might also continue to increment when the G-Series card is in terminal loopback. Receive (Rx) packets from any connected device are dropped and not recorded, but the internally looped back packets follow the G-Series card's normal receive path and register on the Rx Packet counter.

Figure 1-3 shows a terminal loopback on an OC-N card.

Figure 1-3 Terminal Loopback Process on an OC-N Card

Figure 1-4 shows a terminal loopback on a G-Series card. The test-set traffic comes in on the MIC DS-N ports and goes through the XTC card to the G-Series card. The terminal loopback on the G-Series card turns the signal around before it reaches the LIU and sends it back through the XTC card to the MIC. This test verifies that the XTC card cross-connect circuit paths are valid, but does not test the LIU on the G-Series card.

Figure 1-4 Terminal Loopback Process on a G1000-2 Card

A hairpin circuit brings traffic in and out on a DS-N port instead of sending the traffic onto the OC-N. A hairpin loops back only the specific synchronous transport signal (STS) or VT circuit and does not cause an entire OC-N port to loop back, which would drop all traffic on the OC-N port. The hairpin allows you to test a circuit on nodes running live traffic. Figure 1-5 shows the hairpin circuit process on a OC-N card.

Figure 1-5 Hairpin Circuit Process on an OC-N Card

A cross-connect (XC) loopback tests a circuit path as it passes through the cross-connect card and loops back to the port being tested. Testing and verifying circuit integrity often involves taking down the whole line; however, a cross-connect loopback allows you to create a loopback on any embedded channel at supported payloads at the STS-1 granularity and higher. For example, you can loop back a single STS-1, STS-3c, STS-6c, etc., on an optical facility without interrupting the other STS circuits.

You can create a cross-connect loopback on all working or protect optical ports unless the protect port is used in a 1+1 protection group and is in working mode. If a terminal or facility loopback exists on a port, you cannot use the cross-connect loopback. Figure 1-6 shows a cross-connect loopback on an OC-N port.

Figure 1-6 Cross-Connect Loopback Process on an OC-N Port

1.2  Troubleshooting Electrical Circuit Paths with Loopbacks

Facility loopbacks, hairpin circuits, and terminal loopbacks are often used to test a circuit path through the network or to logically isolate a fault. Performing a loopback test at each point along the circuit path systematically isolates possible points of failure.

The example in this section tests an DS-N circuit on a two-node bidirectional line switched ring (BLSR). Using a series of loopbacks and hairpins, the path of the circuit is traced and the possible points of failure are tested and eliminated. A logical progression of five network test procedures applies to this scenario:

1. Facility loopback on the source-node XTC port

2. Hairpin circuit on the source-node XTC card DS-N port

3. Terminal loopback to the destination-node XTC port

4. Hairpin circuit on the destination-node XTC card DS-N port

5. Facility loopback to the destination node XTC port

Note The test sequence for your circuits differs according to the type of circuit and network topology.

Note All loopback tests require on-site personnel.

Note These procedures are performed when power connections to the nodes or sites are assumed to be within necessary specifications. If the network tests do not isolate the problems, troubleshoot outward for power failure.

1.2.1  Perform a Facility Loopback on a Source Node XTC Port

The facility loopback test is performed on the node source port in the network circuit; in this example, the test is routed through the MIC and performed on the XTC port in the source node. Completing a successful facility loopback on this port isolates the cabling, MIC, and XTC card as possible failure points. Figure 1-7 shows an example of a facility loopback on a source node XTC port.

Figure 1-7 Facility Loopback on a Source Node XTC Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Note To create a loopback on a port, the port must be in the Out-of-Service and Management, Maintenance (OOS-MA,MT) service state. The resulting service state is Out-of-Service and Management, Loopback and Maintenance (OOS-MA,LPBK & MT).

Create the Facility Loopback on the Source Node XTC Port

Step 1 Connect an electrical test set to the port you are testing.

Use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the MIC, which interfaces with the XTC card. Both Tx and Rx connect to the same port. Adjust the test set accordingly. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Use CTC to create the facility loopback on the port being tested:

a. In node view, double-click the card where you are performing the loopback.

b. Click the Maintenance > DS1 > Loopback tabs or Maintenance > DS3 > Loopback tabs.

Note If you click a protect XTC card, all provisioning information for this card will be retrieved from the working card when a protection switch is initiated.

c. Choose OOS,MT from the Admin State column for the port being tested. If this is a multiport card, select the appropriate row for the port being tested. This places the port in OOS-MA, MT service state.

d. Choose Facility from the Loopback Type column for the port being tested. If this is a multiport card, select the appropriate row for the port being tested.

e. Click Apply.

f. Click Yes in the confirmation dialog box.

Step 3 Continue with the "Test the DS-N Facility Loopback" procedure.

Test the DS-N Facility Loopback

Step 1 If the test set is not already sending traffic, send test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, clear the loopback:

a. Click the Maintenance > DS1 > Loopback tabs or Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty MIC, faulty XTC card, or faulty cabling from the DS-N port. Continue with the "Test the DS-N Cabling" task.

Test the DS-N Cabling

Step 1 Replace the suspect cabling (the cables from the test set to the MIC ports) with a known-good cable. For instructions to use the test-set equipment, consult the manufacturer.

If a known-good cable is not available, test the suspected bad cable with a test set. Remove the suspected bad cable from the MIC and connect the cable to the Tx and Rx terminals of the test set. Run traffic to determine whether the cable is good or defective.

Step 2 Resend test-set traffic on the loopback circuit with a known-good cable installed. If the test set indicates a good circuit, the problem is probably the defective cable.

Step 3 Replace the defective cable.

Step 4 Clear the loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 5 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the XTC Card" procedure.

Test the XTC Card

Step 1 Complete the "Physically Replace a Card" procedure for the suspected bad card and replace it with a known-good one.

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. See the procedures in the "Protection Switching, Lock Initiation, and Clearing" section. For more information, refer to Chapter 3, "Card Protection," in the Cisco ONS 15327 Reference Manual.

Step 2 Resend test traffic on the loopback circuit with a good card installed. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a good circuit, the problem is probably a defective card.

Step 3 Return the defective card to Cisco through the returned materials authorization (RMA) process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the bad card.

Step 5 Clear the loopback:

a. Click the Maintenance > DS1 > Loopback tabs or Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 6 If the test set indicates a fault circuit, the problem might be faulty electrical cabling from the MIC to the XTC card or a faulty MIC. Continue with the "Test the MIC Cabling" procedure.

Test the MIC Cabling

Step 1 Replace the suspect cabling (the cables from the test set to the MIC or from the MIC to the XTC) with a known-good cable.

If a known-good cable is not available, test the suspected bad cable with a test set. Remove the suspected bad cable and connect it to the Tx and Rx terminals of the test set. Run traffic to determine whether the cable is good or defective. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Resend test traffic on the loopback circuit with a cable that is known to be good installed.

Step 3 If the test set indicates a good circuit, the problem is probably the defective cable. Replace the defective cable.

Step 4 Clear the facility loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 5 If the test set indicates a faulty circuit, the problem might be a faulty MIC. Continue with the "Test the MIC Card" procedure.

Test the MIC Card

Step 1 Complete the "Physically Replace a Card" procedure for the suspected bad card and replace it with a known-good one.

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. See the procedures in the "Protection Switching, Lock Initiation, and Clearing" section. For more information, refer to Chapter 3, "Card Protection," in the Cisco ONS 15327 Reference Manual.

Step 2 Resend test-set traffic on the loopback circuit with a good card installed. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a good circuit, the problem is probably a defective card.

Step 3 Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the bad card.

Step 5 Clear the loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 6 If the test set indicates a faulty circuit, repeat all of the facility loopback procedures. Continue with the "Perform a Hairpin on a Source Node XTC Port" procedure.

1.2.2  Perform a Hairpin on a Source Node XTC Port

The hairpin test is performed on the first XTC card in the network circuit. A hairpin circuit uses the same port for both source and destination. Completing a successful hairpin through this card isolates the possibility that the source XTC card is the cause of the faulty circuit. Figure 1-8 shows an example of a hairpin circuit on a source node XTC port.

Figure 1-8 Hairpin Circuit on a Source Node XTC Port

Note An XTC card is required to operate the ONS 15327 and can be used in a redundant or nonredundant configuration.

Create the Hairpin on the Source Node Port

Step 1 Connect an electrical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer.

•If you just completed the "Perform a Facility Loopback on a Source Node XTC Port" procedure, leave the electrical test set hooked up to the MIC port.

•If you are starting the current procedure without the electrical test set hooked up to the MIC port, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the MIC connectors for the port you are testing. The Tx and Rx terminals connect to the same port.

Step 2 Adjust the test set accordingly.

Step 3 Use CTC to set up the hairpin on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as Hairpin1.

c. Set the circuit Type and Size to the normal preferences.

d. Uncheck the Bidirectional check box and click Next.

e. In the Circuit Source dialog box, select the same Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, use the same Node, card Slot, Port, and Type used for the Circuit Source dialog box and click Finish.

Step 4 Confirm that the newly created circuit appears on the Circuits tab list as a one-way circuit.

Step 5 Continue with the "Test the Hairpin Circuit" procedure.

Test the Hairpin Circuit

Step 1 If the test set is not already sending traffic, send test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the hairpin loopback circuit. Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the hairpin circuit is deleted from the Circuits tab list.

Step 4 If the test set indicates a faulty circuit, there might be a problem with the XTC card.

Step 5 Continue with the "Test the Alternate Source XTC Card" procedure.

Test the Alternate Source XTC Card

Step 1 Perform a reset on the active XTC card:

a. Determine the active XTC card. On both the physical node and the Cisco Transport Controller (CTC) window, the active XTC card has a green ACT LED, and the standby XTC card has an amber SBY LED.

b. Position the cursor over the active cross-connect card.

c. Right-click and choose Reset from the shortcut menu.

d. In the Resetting Card dialog box, click Yes. After 20 to 40 seconds, a "lost node connection, changing to network view" message appears.

e. Click OK. On the network view map, the node where you reset the XTC is gray.

f. After the node icon turns green (within 1 to 2 minutes), double-click it. On the shelf graphic, observe the following:

•The previously standby XTC displays a green ACT LED.

•The previously active XTC LEDs go through the following LED sequence: NP (card not present), Ldg (software is loading), amber SBY LED (XTC is in standby mode).

•The LEDs should complete this sequence within 5 to 10 minutes.

Step 2 Resend test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer. The test-set traffic now travels through the alternate XTC card.

Step 3 If the test set indicates a faulty circuit, assume that the XTC card is not causing the problem. Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the hairpin circuit is deleted from the Circuits tab list.

Step 4 If the test set indicates a good circuit, the problem might be a defective card. To confirm a defective original XTC card, continue with the "Retest the Original Source XTC Card" procedure.

Retest the Original Source XTC Card

Step 1 Perform a side switch of the XTC cards to make the original card the active card.

Step 2 Resend test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a faulty circuit on the original card, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the defective XTC card.

c. Clear the hairpin circuit:

•Click the Circuits tab.

•Choose the hairpin circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

•Confirm that the hairpin circuit is deleted from the Circuits tab list.

d. Continue with Step 5.

Step 4 If the test set indicates a good circuit, the original XTC card might have had a temporary problem that was cleared by the side switch. Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the hairpin circuit is deleted from the Circuits tab list.

Step 5 Continue with the "Perform a Terminal Loopback on a Destination Node XTC Port" procedure.

1.2.3  Perform a Terminal Loopback on a Destination Node XTC Port

The terminal loopback test is performed on the node destination port in the circuit; in this example, the XTC port in the destination node. First, create a bidirectional circuit that starts on the source node DS-N port and terminates on the destination node DS-N port. Then continue with the terminal loopback test. Completing a successful terminal loopback to a destination node XTC port verifies that the circuit is good up to the destination XTC. Figure 1-9 shows an example of a terminal loopback on a destination node XTC port.

Figure 1-9 Terminal Loopback on a Destination Node XTC Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Terminal Loopback on a Destination Node XTC Port

Step 1 Connect an electrical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Hairpin on a Source Node XTC Port" procedure, leave the electrical test set hooked up to the DS-N port in the source node.

b. If you are starting the current procedure without the electrical test set hooked up to the MIC, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the MIC connectors for the port you are testing. The Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as DSNtoDSN.

c. Set circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the DS-N port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note To create a loopback on a port, the port must be in the OOS-MA,MT service state. The resulting service state is OOS-MA,LPBK & MT. It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. Go to the node view of the destination node:

•From the View menu, choose Go To Other Node.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the XTC card in the destination node.

c. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

d. Select OOS,MT from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

f. Click Apply.

g. Click Yes in the confirmation dialog box.

Step 5 Continue with the "Test the Terminal Loopback Circuit on the Destination Node XTC Port" procedure.

Test the Terminal Loopback Circuit on the Destination Node XTC Port

Step 1 If the test set is not already sending traffic, send test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary on the loopback circuit:

a. Clear the terminal loopback:

•Double-click the XTC card in the destination node with the terminal loopback.

•Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) in the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the Destination XTC Card" procedure.

Test the Destination XTC Card

Step 1 Complete the "Physically Replace a Card" procedure for the suspected bad card and replace it with a known-good one.

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. See the procedures in the "Protection Switching, Lock Initiation, and Clearing" section. For more information, refer to Chapter 3, "Card Protection," in the Cisco ONS 15327 Reference Manual.

Step 2 Resend test-set traffic on the loopback circuit with a known-good card. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a good circuit, the problem is probably the defective card.

Step 3 If the card is defective, return it to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the XTC card.

Step 5 Clear the terminal loopback:

a. Double-click the XTC card in the destination node with the terminal loopback.

b. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

c. Select None from the Loopback Type column for the port being tested.

d. Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) in the Admin State column for the port being tested.

e. Click Apply.

f. Click Yes in the confirmation dialog box.

Step 6 Clear the loopback circuit:

a. Click the Circuits tab.

b. Choose the loopback circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

Step 7 Continue with the "Perform a Hairpin Test on a Destination Node XTC Port" procedure.

1.2.4  Perform a Hairpin Test on a Destination Node XTC Port

The hairpin test is performed on the XTC card in the destination node. To perform this test, you must also create a bidirectional circuit from the source MIC to the source OC-N node in the transmit direction. Creating the bidirectional circuit and completing a successful hairpin isolates the possibility that the source and destination OC-N cards, the source and destination XTC cards, or the fiber span is responsible for the faulty circuit. Figure 1-10 shows an example of a hairpin circuit on a destination node XTC card.

Figure 1-10 Hairpin on a Destination Node XTC Card

Create the Hairpin Loopback Circuit on the Destination Node XTC Card

Step 1 Connect an electrical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer.

Use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the DSx panel for the port you are testing. The Tx and Rx terminals connect to the same port. Adjust the test set accordingly.

Step 2 Use CTC to set up the source loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as Hairpin1.

c. Set the circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Step 4 Use CTC to set up the destination hairpin circuit on the port being tested.

Note The destination loopback circuit on a port is a one-way test.

For example, in a typical east-to-west slot configuration, the Slot 1 (east) OC-N card on the source node is one end of the fiber span, and the Slot 2 (west) OC-N card on the destination node is the other end.

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as Hairpin1.

c. Set the Circuit Type and Size to the normal preferences.

d. Uncheck the Bidirectional check box and click Next.

e. In the Circuit Source dialog box, select the same Node, card Slot, Port, and Type where the previous circuit is connected and click Next.

f. In the Circuit Destination dialog box, use the same Node, card Slot, Port, and Type used for the Circuit Source dialog box and click Finish.

Step 5 Confirm that the newly created circuit appears on the Circuits tab list as a one-way circuit.

Step 6 Verify that the circuits connect to the correct slots. For example, verify that source node/Slot 1 OC-N card (east slot) is connected to destination node/Slot 2 (west slot). If two east slots or two west slots are connected, the circuit does not work. Except for the distinct slots, all other circuit information, such as ports, should be identical.

Step 7 Continue with the "Test the Hairpin Circuit" procedure.

Test the Hairpin Circuit

Step 1 If the test set is not already sending traffic, send test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic received by the test set. Look for errors or any other signal information indicated by the test set.

Step 3 If the test set indicates a good circuit, no further testing is necessary. Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the hairpin circuit is deleted from the Circuits tab list.

Step 4 If the test set indicates a faulty circuit, the problem might exist with the destination XTC card. Continue with the "Test the Alternate Destination XTC Card" procedure.

Test the Alternate Destination XTC Card

Step 1 Perform a software reset on the active XTC card.

Caution XTC side switches are service-affecting. Any live traffic on any card in the node endures a hit of up to 50 ms.

Note After the active XTC goes into standby, the original standby slot becomes active. This causes the ACT/STBY LED to become green on the former standby card.

Step 2 Resend test traffic on the loopback circuit. The test traffic routes through the alternate XTC card. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a faulty circuit, assume that the XTC card is not causing the problem and clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the hairpin circuit is deleted from the Circuits tab list.

Step 4 If the test set indicates a good circuit, the problem might be a defective card. To confirm a defective original XTC card, continue with the "Retest the Original Destination XTC Card" procedure.

Retest the Original Destination XTC Card

Step 1 Perform a side switch of the XTC cards to make the original card the active card.

Note After the active XTC goes into standby, the original standby slot becomes active. This causes the ACT/STBY LED to become green on the former standby card.

Step 2 Resend test traffic on the loopback circuit. The test traffic routes through the original XTC card. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a faulty circuit, the problem is probably the defective card.

Step 3 Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the cross-connect card.

Step 5 Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

Step 6 If the test set indicates a good circuit, the XTC card might have had a temporary problem that was cleared by the side switch.

Clear the hairpin circuit:

a. Click the Circuits tab.

b. Choose the hairpin circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

Step 7 Continue with the "Perform a Facility Loopback on a Destination XTC Card" procedure.

1.2.5  Perform a Facility Loopback on a Destination XTC Card

The facility loopback test is performed on the last port in the circuit, in this case the XTC port in the destination node. Completing a successful facility loopback on this port isolates the possibility that the destination node cabling, MIC, or line interface is responsible for a faulty circuit. Figure 1-11 shows an example of a facility loopback on a destination node XTC port.

Figure 1-11 Facility Loopback on a Destination XTC Card

Caution Performing a loopback on an in-service circuit is allowed but is service-affecting.

Note To create a loopback on a port, the port must be in the OOS-MA,MT service state. The resulting service state is OOS-MA,LPBK & MT.

Create a Facility Loopback Circuit on a Destination Node XTC Port

Step 1 Connect an electrical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Hairpin Test on a Destination Node XTC Port" procedure, leave the electrical test set hooked up to the DS-N port in the destination node.

b. If you are starting the current procedure without the electrical test set hooked up to the DS-N port, use appropriate cabling to attach the Tx and Rx terminals of the electrical test set to the DSx panel for the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to create the facility loopback on the port being tested:

a. In node view, double-click the card where you are performing the loopback.

b. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

c. Select Facility from the Loopback Type column for the port being tested. If this is a multiport card, select the row appropriate for the desired port.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Step 3 Continue with the "Test the Facility Loopback Circuit" procedure.

Test the Facility Loopback Circuit

Step 1 If the test set is not already sending traffic, send test-set traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the loopback circuit.

a. Clear the facility loopback:

•Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

The entire DS-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty MIC or faulty electrical cabling from the MIC to the XTC card.

Step 5 Continue with the "Test the DS-N Cabling" procedure.

Test the DS-N Cabling

Step 1 Replace the suspected bad cabling (the cables from the test set to the MIC ports) with a known-good cable.

If a known-good cable is not available, test the suspect cable with a test set. For instructions to use the test-set equipment, consult the manufacturer. Remove the suspected bad cable from the MIC and connect the cable to the Tx and Rx terminals of the test set. Run traffic to determine whether the cable is good or suspect.

Step 2 Resend test traffic on the loopback circuit with a known-good cable installed. If the test set indicates a good circuit, the problem is probably the defective cable.

Step 3 Replace the defective cable.

Step 4 Clear the facility loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

The entire DS-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

Step 5 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the XTC Card" procedure.

Test the XTC Card

Step 1 Complete the "Physically Replace a Card" procedure for the suspected bad XTC card and replace it with a known-good one.

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. See the procedures in the "Protection Switching, Lock Initiation, and Clearing" section. For more information, refer to the Cisco ONS 15327 Procedure Guide.

Step 2 Resend test-set traffic on the loopback circuit with a known-good card installed. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a good circuit, the problem is probably the defective card.

Step 3 Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the bad card.

Step 5 Clear the facility loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS, OOS, or OOS_AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

The entire DS-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

Step 6 If the test set indicates a faulty circuit, the problem might be a faulty MIC. Continue with the "Test the MIC Card" procedure.

Test the MIC Card

Step 1 Complete the "Physically Replace a Card" procedure for the suspected bad card and replace it with a known-good one.

Caution Removing a card that currently carries traffic on one or more ports can cause a traffic hit. To avoid this, perform an external switch if a switch has not already occurred. See the procedures in the "Protection Switching, Lock Initiation, and Clearing" section. For more information, refer to the Cisco ONS 15327 Procedure Guide.

Step 2 Resend test-set traffic on the loopback circuit with a known-good card installed. For instructions to use the test-set equipment, consult the manufacturer. If the test set indicates a good circuit, the problem is probably the defective card.

Step 3 Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Step 4 Complete the "Physically Replace a Card" procedure for the bad card.

Step 5 Clear the facility loopback:

a. Click the Maintenance > DS1 > Loopback tabs or the Maintenance > DS3 > Loopback tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

The entire DS-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

Step 6 If the test set indicates a faulty circuit, contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

1.3  Troubleshooting Optical Circuit Paths with Loopbacks

Facility loopbacks, terminal loopbacks, and cross-connect loopback circuits are often used together to test the circuit path through the network or to logically isolate a fault. Performing a loopback test at each point along the circuit path systematically isolates possible points of failure.

The example in this section tests an OC-N circuit on a three-node BLSR. Using a series of facility loopbacks and terminal loopbacks, the path of the circuit is traced and the possible points of failure are tested and eliminated. A logical progression of seven network test procedures applies to this sample scenario:

1. Facility loopback on the source-node OC-N port

2. Cross-connect loopback on the source-node OC-N port

3. Terminal loopback on the source-node OC-N port

4. Facility loopback on the intermediate-node OC-N port

5. Terminal loopback on the intermediate-node OC-N port

6. Facility loopback on the destination-node OC-N port

7. Terminal loopback on the destination-node OC-N port

Note The test sequence for your circuits differs according to the type of circuit and network topology.

Note All loopback tests require on-site personnel.

Note The G-Series Ethernet card allows terminal and facility loopbacks on the OC-N circuit path, like the ONS 15327 OC-N cards.

1.3.1  Perform a Facility Loopback on a Source-Node OC-N Port

The facility loopback test is performed on the node source port in the network circuit, in this example, the source OC-N port in the source node. Completing a successful facility loopback on this port isolates the OC-N port as a possible failure point. Figure 1-12 shows an example of a facility loopback on a circuit source OC-N port.

Figure 1-12 Facility Loopback on a Circuit Source OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Facility Loopback on the Source OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer.

Use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. The Tx and Rx terminals connect to the same port. Adjust the test set accordingly.

Step 2 Use CTC to create the facility loopback circuit on the port being tested:

a. In node view, double-click the card where you are performing the loopback.

b. Click the Maintenance > Loopback > Port tabs.

c. Choose OOS,MT-MA from the Admin State column for the port being tested. If this is a multiport card, select the appropriate row for the desired port.

d. Choose Facility from the Loopback Type column for the port being tested. If this is a multiport card, select the appropriate row for the desired port.

e. Click Apply.

f. Click Yes in the confirmation dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Step 3 Continue with the "Test the Facility Loopback Circuit" procedure.

Test the Facility Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the facility loopback. Clear the facility loopback:

a. Click the Maintenance > Loopback > Port tabs.

b. Choose None from the Loopback Type column for the port being tested.

c. Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty OC-N card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card installed. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the faulty card. See the "Physically Replace a Card" procedure for details.

c. Clear the facility loopback:

•Click the Maintenance > Loopback > Port tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

Step 4 Continue with the "Perform a Cross-Connect Loopback on the Source OC-N Port" procedure.

1.3.2  Perform a Cross-Connect Loopback on the Source OC-N Port

The cross-connect loopback test occurs on the XTC in a network circuit. Completing a successful cross-connect loopback through the XTC card isolates the possibility that the XTC card is the cause of the faulty circuit. Figure 1-13 shows an example of a cross-connect loopback on a source OC-N port.

Figure 1-13 Cross-Connect Loopback on a Source OC-N Port

Create the Cross-Connect Loopback on the Source OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Facility Loopback on a Source-Node OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to put the circuit being tested out of service:

a. In node view, click the Circuits tab.

b. Click the circuit and then click Edit.

c. In the Edit Circuit dialog box, click the State tab.

d. Choose OOS,MT from the Target Circuit State drop-down list.

e. Click Apply.

f. Click Yes in the confirmation dialog box.

Step 3 Use CTC to set up the cross-connect loopback on the circuit being tested:

a. In node view, double-click the OC-N card to open the card view.

b. Click the Provisioning > SONET STS tabs.

c. Click the check box in the XC Loopback column for the port being tested.

d. Click Apply.

e. Click Yes in the confirmation dialog box.

Step 4 Continue with the "Test the Cross-Connect Loopback Circuit" procedure.

Test the Cross-Connect Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the cross-connect. Clear the cross-connect loopback:

a. In card view, click the Provisioning > SONET STS tabs.

b. Uncheck the check box in the XC Loopback column for the circuit being tested.

c. Click Apply.

d. Click Yes in the confirmation dialog box.

Step 4 If the test set indicates a faulty circuit, there might be a problem with the cross-connect card. Continue with the "Test the Standby XTC Card" procedure.

Test the Standby XTC Card

Step 1 Perform a reset on the active XTC card:

a. Determine which cross-connect card is active. On both the physical node and the CTC window, the active XTC has a green ACT LED, and the standby XTC has an amber SBY LED.

b. Position the cursor over the active cross-connect card.

c. Right-click and choose Reset from the shortcut menu.

d. In the Resetting Card dialog box, click Yes. After 20 to 40 seconds, a "lost node connection, changing to network view" message appears.

e. Click OK. On the network view map, the node where you reset the XTC is gray.

f. After the node icon turns green (within 1 to 2 minutes), double-click it. On the shelf graphic, observe the following:

•The previously standby XTC displays a green ACT LED.

•The previously active XTC LEDs go through the following LED sequence: NP (card not present), Ldg (software is loading), amber SBY LED (XTC is in standby mode).

•The LEDs should complete this sequence within 5 to 10 minutes.

Step 2 Resend test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

The test traffic now travels through the alternate cross-connect card.

Step 3 If the test set indicates a faulty circuit, assume that the cross-connect card is not causing the problem and clear the cross-connect loopback circuit:

a. Click the Circuits tab.

b. Choose the cross-connect loopback circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

e. Confirm that the cross-connect loopback circuit is deleted from the Circuits tab list.

Step 4 If the test set indicates a good circuit, the problem might be a defective cross-connect card. Continue with the "Retest the Original XTC Card" procedure.

Retest the Original XTC Card

Step 1 Do a manual switch of the XTC cards to make the original XTC card the active card:

a. Determine the active cross-connect card. On both the physical node and the CTC window, the active XTC has a green ACT LED, and the standby XTC has an amber SBY LED.

b. Position the cursor over the active cross-connect card.

c. Right-click and choose Reset from the shortcut menu.

d. In the Resetting Card dialog box, click Yes. After 20 to 40 seconds, a "lost node connection, changing to network view" message appears.

e. Click OK. On the network view map, the node where you reset the XTC is gray.

f. After the node icon turns green (within 1 to 2 minutes), double-click it. On the shelf graphic, observe the following:

•The previously standby XTC displays a green ACT LED.

•The previously active XTC LEDs go through the following LED sequence: NP (card not present), Ldg (software is loading), amber SBY LED (XTC is in standby mode).

•The LEDs should complete this sequence within 5 to 10 minutes.

Step 2 Resend test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a faulty circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the faulty XTC card. See "Physically Replace a Card" procedure for details.

c. Clear the cross-connect loopback:

•Click the Circuits tab.

•Choose the cross-connect loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

d. Continue with Step 5.

Step 4 If the test set indicates a good circuit, the XTC card might have had a temporary problem that was cleared by the switch. Clear the cross-connect loopback:

a. Click the Circuits tab.

b. Choose the cross-connect loopback circuit being tested.

c. Click Delete.

d. Click Yes in the Delete Circuits dialog box.

Step 5 Continue with the "Perform a Terminal Loopback on a Source-Node OC-N Port" procedure.

1.3.3  Perform a Terminal Loopback on a Source-Node OC-N Port

The terminal loopback test is performed on the node destination port in the circuit, in this example, the destination OC-N port in the source node. First, create a bidirectional circuit that starts on the node source OC-N port and loops back on the node destination OC-N port. Then continue with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N port verifies that the circuit is good up to the destination OC-N. Figure 1-14 shows an example of a terminal loopback on a source node OC-N port.

Figure 1-14 Terminal Loopback on a Source-Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Terminal Loopback on a Source Node OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Cross-Connect Loopback on the Source OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as OCN1toOCN2.

c. Set circuit Type and Size to the normal preferences.

d. Verify that the Bidirectional check box is checked.

e. Click Next.

f. In the Circuit Source dialog box, fill in the same Node, card Slot, Port, and Type where the test set is connected and click Next.

g. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the OC-N port in the source node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the source node.

b. Click the Maintenance > Loopback > Port tabs.

c. Select OOS,MT from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

d. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

e. Click Apply.

f. Click Yes in the confirmation dialog box.

Step 5 Continue with the "Test the Terminal Loopback Circuit" procedure.

Test the Terminal Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary on the loopback circuit:

a. Clear the terminal loopback:

•Double-click the OC-N card in the source node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the defective OC-N card. See the "Physically Replace a Card" procedure for details.

c. Clear the terminal loopback before testing the next segment of the network circuit path:

•Double-click the OC-N card in the source node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

d. Clear the terminal loopback circuit before testing the next segment of the network circuit path:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 Continue with the "Perform a Facility Loopback on an Intermediate-Node OC-N Port" procedure.

1.3.4  Perform a Facility Loopback on an Intermediate-Node OC-N Port

The facility loopback test is performed on the node initiating port in the network circuit, in this example, the OC-N port in the intermediate node. Completing a successful facility loopback on this port isolates the OC-N port as a possible failure point. Figure 1-15 shows an example of a facility loopback on a intermediate node circuit source OC-N port.

Figure 1-15 Facility Loopback on an Intermediate-Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Facility Loopback on an Intermediate-Node OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Terminal Loopback on a Source-Node OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the facility loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as OCN1toOCN3.

c. Set circuit Type and Size to the normal preferences.

d. Verify that Bidirectional is checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the OC-N port in the intermediate node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note It is normal for a LPBKFACILITY condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the facility loopback on the destination port being tested:

a. Go to the node view of the intermediate node:

•From the View menu, choose Go To Other Node.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the intermediate node.

c. Click the Maintenance > Loopback > Port tabs.

d. Select OOS,MT from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

f. Click Apply.

g. Click Yes in the confirmation dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Step 5 Continue with the "Test the Facility Loopback Circuit" procedure.

Test the Facility Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the facility loopback:

a. Clear the facility loopback:

•Click the Maintenance > Loopback > Port tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the facility loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty OC-N card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card installed. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the faulty card. See the "Physically Replace a Card" procedure for details.

c. Clear the facility loopback:

•Click the Maintenance > Loopback > Port tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

d. Clear the facility loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 Continue with the "Perform a Terminal Loopback on an Intermediate-Node OC-N Port" procedure.

1.3.5  Perform a Terminal Loopback on an Intermediate-Node OC-N Port

The terminal loopback test is performed on the node receiving port in the circuit, in this example, the OC-N port in the intermediate node. First, create a bidirectional circuit that starts on the node source OC-N port and loops back on the node destination OC-N port. Then continue with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N port verifies that the circuit is good up to the destination OC-N port. Figure 1-16 shows an example of a terminal loopback on an intermediate node destination OC-N port.

Figure 1-16 Terminal Loopback on an Intermediate-Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Terminal Loopback on an Intermediate-Node OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Facility Loopback on an Intermediate-Node OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as OCN1toOCN4.

c. Set circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the OC-N port in the intermediate node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. Go to the node view of the intermediate node:

•From the View menu, choose Go To Other Node.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the intermediate node.

c. Click the Maintenance > Loopback > Port tabs.

d. Select OOS,MT-MA from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

f. Click Apply.

g. Click Yes in the confirmation dialog box.

Step 5 Continue with the "Test the Terminal Loopback Circuit" procedure.

Test the Terminal Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary on the loopback circuit:

a. Clear the terminal loopback:

•Double-click the OC-N card in the intermediate node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the defective OC-N card. See the "Physically Replace a Card" procedure for details.

c. Clear the terminal loopback:

•Double-click the OC-N card in the source node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

d. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 Continue with the "Perform a Facility Loopback on a Destination-Node OC-N Port" procedure.

1.3.6  Perform a Facility Loopback on a Destination-Node OC-N Port

The facility loopback test is performed on the node initiating port in the network circuit, in this example, the OC-N port in the destination node. Completing a successful facility loopback on this port isolates the OC-N port as a possible failure point. Figure 1-17 shows an example of a facility loopback on a destination node circuit source OC-N port.

Figure 1-17 Facility Loopback on a Destination Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Facility Loopback on a Destination Node OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Terminal Loopback on an Intermediate-Node OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the facility loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as OCN1toOCN5.

c. Set circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the OC-N port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note It is normal for a LPBKFACILITY condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the facility loopback on the destination port being tested:

a. Go to the node view of the destination node:

•From the View menu, choose Go To Other Node.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the destination node.

c. Click the Maintenance > Loopback > Port tabs.

d. Select OOS,MT from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

f. Click Apply.

g. Click Yes in the confirmation dialog box.

Note It is normal for a LPBKFACILITY condition to appear during loopback setup. The condition clears when you remove the loopback.

Step 5 Continue with the "Test the Facility Loopback Circuit" procedure.

Test the Facility Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the traffic received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary with the facility loopback:

a. Clear the facility loopback:

•Click the Maintenance > Loopback > Port tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the facility loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty OC-N card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card installed. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the faulty card. See the "Physically Replace a Card" procedure for details.

c. Clear the facility loopback:

•Click the Maintenance > Loopback > Port tabs.

•Choose None from the Loopback Type column for the port being tested.

•Choose the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

d. Clear the facility loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 Continue with the "Perform a Terminal Loopback on a Destination-Node OC-N Port" procedure.

1.3.7  Perform a Terminal Loopback on a Destination-Node OC-N Port

The terminal loopback test is performed on the node receiving port in the circuit, in this example, the OC-N port in the destination node. First, create a bidirectional circuit that starts on the node source OC-N port and loops back on the node destination OC-N port. Then continue with the terminal loopback test. Completing a successful terminal loopback to a node destination OC-N port verifies that the circuit is good up to the destination OC-N. Figure 1-18 shows an example of a terminal loopback on an intermediate node destination OC-N port.

Figure 1-18 Terminal Loopback on a Destination-Node OC-N Port

Caution Performing a loopback on an in-service circuit is service-affecting.

Create the Terminal Loopback on a Destination-Node OC-N Port

Step 1 Connect an optical test set to the port you are testing. For instructions to use the test-set equipment, consult the manufacturer. Complete the following steps:

a. If you just completed the "Perform a Facility Loopback on a Destination-Node OC-N Port" procedure, leave the optical test set hooked up to the OC-N port in the source node.

b. If you are starting the current procedure without the optical test set hooked up to the OC-N port, use appropriate cabling to attach the Tx and Rx terminals of the optical test set to the port you are testing. Both Tx and Rx connect to the same port.

c. Adjust the test set accordingly.

Step 2 Use CTC to set up the terminal loopback circuit on the port being tested:

a. Click the Circuits tab and click Create.

b. Give the circuit an easily identifiable name, such as OCN1toOCN6.

c. Set circuit Type and Size to the normal preferences.

d. Leave the Bidirectional check box checked and click Next.

e. In the Circuit Source dialog box, fill in the source Node, card Slot, Port, and Type where the test set is connected and click Next.

f. In the Circuit Destination dialog box, fill in the destination Node, card Slot, Port, and Type (the OC-N port in the destination node) and click Finish.

Step 3 Confirm that the newly created circuit appears on the Circuits tab list as a two-way circuit.

Note It is normal for a LPBKTERMINAL condition to appear during a loopback setup. The condition clears when you remove the loopback.

Step 4 Create the terminal loopback on the destination port being tested:

a. Go to the node view of the destination node:

•From the View menu, choose Go To Other Node.

•Choose the node from the drop-down list in the Select Node dialog box and click OK.

b. In node view, double-click the card that requires the loopback, such as the destination OC-N card in the destination node.

c. Click the Maintenance > Loopback > Port tabs.

d. Select OOS,MT-MA from the Admin State column. If this is a multiport card, select the row appropriate for the desired port.

e. Select Terminal from the Loopback Type column. If this is a multiport card, select the row appropriate for the desired port.

f. Click Apply.

g. Click Yes in the confirmation dialog box.

Step 5 Continue with the "Test the Terminal Loopback Circuit" procedure.

Test the Terminal Loopback Circuit

Step 1 If the test set is not already sending traffic, send test traffic on the loopback circuit. For instructions to use the test-set equipment, consult the manufacturer.

Step 2 Examine the test traffic being received by the test set. Look for errors or any other signal information that the test set is capable of indicating.

Step 3 If the test set indicates a good circuit, no further testing is necessary on the loopback circuit:

a. Clear the terminal loopback:

•Double-click the OC-N card in the intermediate node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

b. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

c. The entire OC-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

Step 4 If the test set indicates a faulty circuit, the problem might be a faulty card. Continue with the "Test the OC-N Card" procedure.

Test the OC-N Card

Step 1 Replace the suspect card with a card known to be good. See the "Physically Replace a Card" procedure for details.

Step 2 Resend test traffic on the loopback circuit with a good card. For instructions to use the test-set equipment, consult the manufacturer.

Step 3 If the test set indicates a good circuit, the problem is probably the defective card:

a. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

b. Replace the defective OC-N card. See the "Physically Replace a Card" procedure for details.

c. Clear the terminal loopback:

•Double-click the OC-N card in the source node with the terminal loopback.

•Click the Maintenance > Loopback > Port tabs.

•Select None from the Loopback Type column for the port being tested.

•Select the appropriate state (IS; OOS,DSBLD; OOS,MT; IS,AINS) from the Admin State column for the port being tested.

•Click Apply.

•Click Yes in the confirmation dialog box.

d. Clear the terminal loopback circuit:

•Click the Circuits tab.

•Choose the loopback circuit being tested.

•Click Delete.

•Click Yes in the Delete Circuits dialog box.

Step 4 The entire OC-N circuit path has now passed its comprehensive series of loopback tests. This circuit qualifies to carry live traffic.

1.4  Using CTC Diagnostics

In Release 7.0, CTC provides the following diagnostic functions:

•Verification of proper card application-specific integrated circuit (ASIC) function

•Verification of standby card operation

•Verification of proper card LED operation

•Diagnostic circuit creation

•Notification of problems detected through alarms

•Provision of a downloaded, machine-readable diagnostic file to be used by Cisco TAC

Some of these functions, such as ASIC verification and standby card operation, are invisibly monitored in background functions. Change or problem notifications are provided in the Alarms and Conditions window. Other diagnostic functions—verifying card LED function, creating BLSR diagnostic circuits, and also downloading diagnostic files for Cisco TAC—are available to the user in the node view Maintenance > Diagnostic tab. The user-operated diagnostic features are described in the following paragraphs.

1.4.1  Card LED Lamp Tests

A card LED lamp test determines whether card-level indication LEDs are operational. This diagnostic test is run as part of the initial ONS 15327 turn-up, during maintenance routines, or any time you question whether an LED is in working order. Users who are maintenance-level or higher can complete the following tasks to verify LED operation.

Verify Card LED Operation

Step 1 In CTC card view, click the Maintenance > Diagnostic tabs.

Step 2 Click Lamp Test.

Step 3 Watch to make sure all the port LEDs illuminate simultaneously for several seconds.

Step 4 Click OK in the Lamp Test Run dialog box.

If an OC-N or XTC LED does not light up, the LED is faulty. Return the defective card to Cisco through the RMA process. Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Verify G-Series Ethernet Card LED Operation

Step 1 Complete the procedure in the "Lamp Tests for Card LEDs" section to verify that card-level LEDs are operational.

Step 2 Use the following list of guidelines to physically test whether the G-Series Ethernet port LEDs are operating correctly. If the LED appears as described when the listed state is occurring for the port, the LED is considered to be functioning correctly. Consult the following guidelines:

•Clear port LED: Should only occur if there is a loss of receive link (such as a disconnected link or unplugged Gigabit Interface Converter [GBIC]). An LOS alarm could be present on the port.

•Amber port LED: Should only occur if a port is disabled but the link is connected, or if the port is enabled and the link is connected, but a transport failure (TPTFAIL) alarm might be present on the port.

•Green port LED: Should occur if the port is enabled and has no errors against it or traffic in it; can also occur if the port is enabled, has no errors, and is running traffic proportionate to the blink rate. No traffic-affecting port alarms should be present.

Step 3 If you are unable to determine the port state, contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

Verify E-Series Ethernet Card LED Operation

Step 1 Complete the procedure in the "Lamp Tests for Card LEDs" section to verify that card-level LEDs are operational.

Step 2 Use the following list of guidelines to physically test whether the two E-Series Ethernet port LEDs are operating correctly. If the LED appears as described when the listed state is occurring for the port, the port is considered operational. Consult the following guidelines:

•Right port LED is clear: The port is not receiving traffic.

•Right port LED is amber: The port is receiving traffic.

•Left port LED is clear: The link is down.

•Left port LED is green: The link is up.

Step 3 If you are unable to determine the port state, contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447.

1.4.2  Retrieve Diagnostics File Button

When you click the Retrieve Diagnostics File button in the Maintenance window, CTC retrieves system data that can be off-loaded by a Maintenance or higher-level user to a local directory and sent to Cisco TAC for troubleshooting purposes. The diagnostics file is in machine language and is not human-readable, but can be used by TAC for problem analysis. Complete the following task to off-load the diagnostics file.

Note In addition to the machine-readable diagnostics file, the ONS 15327 also stores an audit trail of all system events such as user logins, remote logins, configuration, and changes. This audit trail is considered a record-keeping feature rather than a troubleshooting feature. Information about the feature is located in the "Maintain the Node" chapter in the Cisco ONS 15327 Procedure Guide.

Off-Load the Diagnostics File

Step 1 In node view, click the Maintenance > Diagnostic tabs.

Step 2 Click Retrieve Tech Support Log.

Step 3 In the Saving Diagnostic File dialog box, navigate to the directory (local or network) where you want to save the file.

Step 4 Enter a name in the File Name field.

You do not have to give the archive file a particular extension. It is readable in any application that supports text files, such as WordPad, Microsoft Word (imported), etc.

Step 5 Click Save.

The Get Diagnostics status window shows a progress bar indicating the percentage of the file being saved, then shows "Get Diagnostics Complete."

Step 6 Click OK.

1.4.3  Data Communications Network (DCN) Tool

In Software R7.0, CTC contains a data communications network (DCN) tool that assists with network troubleshooting for Open Shortest Path First (OSPF) networks. This tool, located in network view, is shown in Figure 1-19. It executes an internal dump command to retrieve information about all nodes accessible from the entry point.

Figure 1-19 DCN Tools OSPF Dump

The dump, which provides the same information as a dump executed by special networking commands, is available in the network view Maintenance > Diagnostic tab. You can select the access point node in the Select Node drop-down list. To create the dump, click Retrieve. (To clear the dump, click Clear.)

The contents of this file can be saved or printed and furnished to Cisco Technical Support for use in OSPF network support.

1.4.4  Bidirectional Diagnostic Circuit

In Release 7.0, CTC provides a diagnostic bidirectional loopback circuit feature that uses pseudo-random bit sequence (PRBS) error detection to monitor standby path protection, BLSR, 1+1, or unprotected circuit path readiness.

The diagnostic circuit is bidirectional and uses a single VT 1.5 in an STS. The circuit can traverse multiple nodes, but it must be carried by the same STS for the entire path. The circuit originates and ultimately destinates on the same node, but can be looped (by a hairpin circuit) through other nodes. Once the circuit returns to the originating node, the signal result is detected and analyzed for errors by PRBS.

This type of circuit is created much the same way as a normal standby protection channel access (PCA) circuit, but is designated by checking the Diagnostic check box during circuit creation. A normal circuit uses line cards as the endpoints, but if a circuit is configured as a diagnostic, the endpoints are cross-connect cards.

Each card type utilizes the diagnostic feature differently. Standby electrical cards run PRBS tests to ensure signal path integrity. Optical cards do not run PRBS tests, but instead run ASIC tests to test card operability. Cross-connect cards verify the standby paths.

The diagnostic circuit can be configured for an end-to-end or multiple-node path layout, traversing the transmit and receive standby paths as shown in Figure 1-20.

Figure 1-20 CTC Node View Diagnostic Window

In Release 7.0, the maximum diagnostic circuit size is VT1.5, and the maximum quantity of available diagnostic circuits is one per node. (In other words, if you create a diagnostic VT within an STS, the remaining 27 VTs can still be provisioned, and they might also contain diagnostic circuits originating on other nodes.)

As with all bidirectional circuits, a diagnostic circuit can only be created if the same STS is available on each span the circuit traverses. When you use a bidirectional diagnostic that traverses one or more intermediate nodes, create or utilize an existing bidirectional circuit on each intermediate node. At the terminating node, you will need to create a hairpin loopback at the end of the PRBS source span to return the signal.

Note The diagnostic VT circuit does not raise a failure alarm if AIS-P or UNEQ-P is returned to the PRBS detector. In order to see an alarm indicating a failed diagnostic circuit, the circuit must be returned to the PRBS detector with a different payload than the generator sources and without the AIS-P or UNEQ-P conditions.

Create a Bidirectional Diagnostic Circuit

Step 1 Log into the node where you will create the diagnostic circuit. (For login instructions, refer to the "Connect the PC and Log into the GUI" chapter in the Cisco ONS 15327 Procedure Guide.)

Step 2 If you want to assign a name to the circuit source and destination ports before you create the circuit, refer to the task for assigning a name to a port in the "Create Circuits and VT Tunnels" chapter of the Cisco ONS 15327 Procedure Guide. If not, continue with Step 3.

Step 3 From the View menu, choose Go to Network View.

Step 4 Click the Circuits tab, then click Create.

Step 5 In the Circuit Creation dialog box, complete the following fields:

•Circuit Type—Choose VT.

•Number of Circuits—Enter 1 (the maximum diagnostic circuit quantity available per STS).

•Auto-ranged—Uncheck the box. (This option is not applicable to diagnostic circuits.)

Step 6 Click Next.

Step 7 Define the circuit attributes in the Circuit Creation Dialog Box (Figure 1-21) using the following parameters:

•Name—Assign a name to the circuit. The name can be alphanumeric and up to 48 characters, (including spaces). Circuit names should be 44 characters or less if you want the ability to create monitor circuits. If you leave the field blank, CTC assigns a default name to the circuit.

•Size—VT1.5 is the default. You cannot change it.

•Bidirectional—This is the default value. Leave it checked for this circuit.

•State—This option is not available when you check the Diagnostic option.

•Diagnostic—Check this box to create a diagnostic circuit.

•Apply to drop ports—Leave this box unchecked.

•Create cross-connects only (TL1-like)—Not applicable to diagnostic circuits.

•Inter-domain (UCP) SLA—Not applicable to diagnostic circuits.

•Protected Drops—Not applicable to diagnostic circuits.

Figure 1-21 Network View Circuit Creation Dialog Box

Step 8 Click Next.

Step 9 In the Source area of the Circuit Creation pane, complete the following:

a. From the Node drop-down list, choose the node.

b. From the Slot drop-down list, choose PRBS Generator.

c. Click Next.

Step 10 In the Destination area of the Circuit Creation pane, complete the following:

a. From the Node drop-down list, choose the node. The only selectable item in the list is the node chosen as the source node.

b. From the Slot drop-down list, choose the slot where the span originates.

c. From the STS drop-down list, choose the STS.

d. From the VT drop-down list, choose the VT.

e. Click Next.

Step 11 Click Finish.

Step 12 In the Circuits window, verify that the new circuit(s) appear in the circuits list.

Stop. You have completed this procedure.

1.5  Restoring the Database and Default Settings

This section contains troubleshooting procedures for node operation errors that require restoration of software data or the default node setup.

1.5.1  Restore the Node Database

Symptom    One or more nodes are not functioning properly or have incorrect data.
Possible Cause    Incorrect or corrupted node database.
Recommended Action    Complete the procedures in the "Maintain the Node" chapter of the Cisco ONS 15327 Procedure Guide.
1.6  PC Connectivity Troubleshooting

This section contains information about system minimum requirements, supported platforms, browsers, and JREs for Software R7.0, and troubleshooting procedures for PC and network connectivity to the ONS 15327.

1.6.1  PC System Minimum Requirements

Workstations running CTC R7.0 for the ONS products on Windows platforms need to have the following minimum requirements:

•Pentium III or higher processor

•Processor speed of at least 700 MHz

•256 Mb or more of RAM

•50 Mb or more of available hard disk space

•20 GB or larger hard drive

1.6.2  Sun System Minimum Requirements

Workstations running CTC R7.0 for the ONS products on Sun workstations need to have the following minimum requirements:

•UltraSPARC or faster processor

•256 Mb or more of RAM

•50 Mb or more of available hard disk space

1.6.3  Supported Platforms, Browsers, and JREs

Software R7.0 CTC supports the following platforms:

•Windows NT

•Windows 98

•Windows XP

•Windows 2000

•Solaris 8

•Solaris 9

Software R7.0 CTC supports the following browsers and JREs:

•Netscape 7 browser (on Solaris 8 or 9 with Java plug-in 1.4.2 or 5.0)

•PC platforms with Java plug-in 1.4.2 or 5.0

•Internet Explorer 6.0 browser (on PC platforms with Java plug-in 1.4.2 or 5.0)

•Mozilla application suite for browsers

Note You can obtain browsers at the following URLs:
Netscape: http://channels.netscape.com/ns/browsers/default.jsp
Internet Explorer: http://www.microsoft.com
Mozilla: http://mozilla.org

Note JRE 1.4.2 or JRE 5.0 is required to run Software R7.0. JRE 1.4.2 is provided on the software CD.

1.6.4  Unsupported Platforms and Browsers

Software R7.0 does not support the following platforms:

•Windows 95

•Solaris 2.5

•Solaris 2.6

Software R7.0 does not support the following browsers and JREs:

•Netscape 4.73 for Windows

•Netscape 4.76 on Solaris

•Netscape 7 on Solaris 8 or 9 is only supported with JRE 1.4.2 and later.

1.6.5  Unable to Verify the IP Configuration of Your PC

Symptom    When connecting your PC to the ONS 15327, you are unable to successfully ping the IP address of your PC to verify the IP configuration.
Possible Cause    The IP address is typed incorrectly.
Possible Cause    The IP configuration of your PC is not properly set.
Recommended Action    Verify that the IP address used to ping the PC matches the IP address displayed when the Windows IP Configuration information is retrieved from the system. Verify the IP configuration of your PC, see the "Verify the IP Configuration of Your PC" procedure. If this procedure is unsuccessful, contact your network administrator for instructions to correct the IP configuration of your PC.

Verify the IP Configuration of Your PC

Step 1 Open a DOS command window by selecting Start > Run from the Start menu.

Step 2 In the Open field, type command and then click OK. The DOS command window appears.

Step 3 At the prompt in the DOS window, type one of the following appropriate commands:

•For Windows 98, NT, 2000, and XP, type ipconfig and press the Enter key.

The Windows IP configuration information appears, including the IP address, subnet mask, and the default gateway.

Step 4 At the prompt in the DOS window, type ping followed by the IP address shown in the Windows IP configuration information.

Step 5 Press the Enter key to execute the command.

If the DOS window displays multiple (usually four) replies, the IP configuration is working properly.

If you do not receive a reply, your IP configuration might not be properly set. Contact your network administrator for instructions to correct the IP configuration of your PC.

1.6.6  Browser Login Does Not Launch Java

Symptom    The message "Loading Java Applet" does not appear and the JRE does not launch during the initial login.
Possible Cause    The PC operating system and browser are not properly configured.
Recommended Action    Reconfigure the PC operating system java plug-in control panel and the browser settings. See the "Reconfigure the PC Operating System Java Plug-in Control Panel" procedure and the "Reconfigure the Browser" procedure.

Reconfigure the PC Operating System Java Plug-in Control Panel

Step 1 From the Windows start menu, click Settings > Control Panel.

Step 2 If Java Plug-in Control Panel does not appear, the JRE might not be installed on your PC.

a. Run the Cisco ONS 15327 software CD.

b. Open the CD-drive:\Windows\JRE folder.

c. Double-click the j2re-1_4_2-win icon to run the JRE installation wizard.

d. Follow the JRE installation wizard steps.

Step 3 From the Windows start menu, click Settings > Control Panel.

Step 4 In the Java Plug-in Control Panel window, double-click the Java Plug-in 1.4.2 icon.

Step 5 Click the Advanced tab on the Java Plug-in Control Panel.

Step 6 From the Java Run Time Environment menu, select JRE 1.4 in C:\ProgramFiles\JavaSoft\JRE\1.4.2.

Step 7 Click Apply.

Step 8 Close the Java Plug-in Control Panel window.

Reconfigure the Browser

Step 1 From the Start Menu, launch your browser application.

Step 2 If you are using Netscape Navigator:

a. From the Netscape Navigator menu bar, click Edit > Preferences.

b. In the Preferences window, click the Advanced > Proxies categories.

c. In the Proxies window, click the Direct connection to the Internet check box and click OK.

d. From the Netscape Navigator menu bar, click Edit > Preferences.

e. In the Preferences window, click the Advanced > Cache categories.

f. Confirm that the Disk Cache Folder field shows one of the following paths:

•For Windows 98/ME, C:\ProgramFiles\Netscape\Communicator\cache

•For Windows NT/2000/XP, C:\ProgramFiles\Netscape\username\Communicator\cache

g. If the Disk Cache Folder field is not correct, click the Choose Folder button.

h. Navigate to the file listed in step f and click OK.

i. Click OK in the Preferences window and exit the browser.

Step 3 If you are using Internet Explorer:

a. On the Internet Explorer menu bar, click Tools > Internet Options.

b. In the Internet Options window, click the Advanced tab.

c. In the Settings menu, scroll down to Java (Sun) and click the Use Java 2 v1.4.2 for <applet> (requires restart) check box.

d. Click OK in the Internet Options window and exit the browser.

Step 4 Temporarily disable any virus-scanning software on the computer. See the "Browser Stalls When Downloading CTC JAR Files from XTC Card" section.

Step 5 Verify that the computer does not have two network interface cards (NICs) installed. If the computer does have two NICs, remove one.

Step 6 Restart the browser and log into the ONS 15327.

Step 7 After completing browser configuration, enable the virus-scanning software on the computer.

1.6.7  Unable to Verify the NIC Connection on Your PC

Symptom    When connecting your PC to the ONS 15327, you are unable to verify that the NIC connection is working properly because the link LED is not illuminated or flashing.
Possible Cause    The CAT-5 cable is not plugged in properly.
Recommended Action    Confirm that both ends of the cable are properly inserted. If the cable is not fully inserted due to a broken locking clip, the cable should be replaced.
Possible Cause    The CAT-5 cable is damaged.
Recommended Action    Ensure that the cable is in good condition. If in doubt, use a cable known to be good. Often, cabling is damaged due to pulling or bending.
Possible Cause    Incorrect type of CAT-5 cable is being used.
Recommended Action    If connecting an ONS 15327 directly to your laptop/PC or a router, use a straight-through CAT-5 cable. When connecting the ONS 15327 to a hub or a LAN switch, use a crossover CAT-5 cable. For details on the types of CAT-5 cables, see the "Crimp Replacement LAN Cables" section.
Possible Cause    The NIC is improperly inserted or installed.
Recommended Action    If you are using a Personal Computer Memory Card International Association (PCMCIA)-based NIC, remove and re-insert the NIC to make sure the NIC is fully inserted. If the NIC is built into the laptop/PC, verify that the NIC is not faulty.
Possible Cause    The NIC is faulty.
Recommended Action    Confirm that the NIC is working properly. If you have no issues connecting to the network (or any other node), then the NIC should be working correctly. If you have difficulty connecting to the network (or any other node), then the NIC might be faulty and needs to be replaced.
1.6.8  Verify PC Connection to the ONS 15327 (Ping)

Symptom    The TCP/IP connection is established and then lost, and a DISCONNECTED alarm appears on CTC.
Possible Cause    A lost connection between the PC and the ONS 15327.
Recommended Action    Use a standard ping command to verify the TCP/IP connection between the PC and the ONS 15327 XTC card. A ping command works if the PC connects directly to the XTC card or uses a LAN to access the XTC card. See the "Ping the ONS 15327" procedure.

Ping the ONS 15327

Step 1 Open the command prompt:

a. If you are using a Microsoft Windows operating system, from the Start menu choose Run, type command prompt in the Open field of the Run dialog box, and click OK.

b. If you are using a Sun Solaris operating system, from the Common Desktop Environment (CDE) click the Personal Application tab and click Terminal.

Step 2 For both the Sun and Microsoft operating systems, at the prompt type:
ping ONS-15327-IP-address
For example:
ping 192.1.0.2
Step 3 If the workstation has connectivity to the ONS 15327, the ping is successful and displays a reply from the IP address. If the workstation does not have connectivity, a "Request timed out" message appears.

Step 4 If the ping is successful, it demonstrates that an active TCP/IP connection exists. Restart CTC.

Step 5 If the ping is not successful and the workstation connects to the ONS 15327 through a LAN, check that the workstation's IP address is on the same subnet as the ONS node.

Step 6 If the ping is not successful and the workstation connects directly to the ONS 15327, check that the link light on the workstation's NIC is illuminated.

1.6.9  The IP Address of the Node is Unknown

Symptom    The IP address of the node is unknown and you are unable to log in.
Possible Cause    The node is not set to the default IP address.
Recommended Action    Leave one XTC card in the shelf. Connect a PC directly to the remaining XTC card and perform a hardware reset of the XTC card. The XTC card transmits the IP address during the reset to enable you to capture the IP address for login after the XTC has completed reset. See the "Retrieve Unknown Node IP Address" procedure.

Retrieve Unknown Node IP Address

Step 1 Connect your PC directly to the active XTC card Ethernet port on the faceplate.

Step 2 Start the Sniffer application on your PC.

Step 3 Perform a hardware reset by pulling and reseating the active XTC card.

Step 4 After the XTC card completes resetting, it broadcasts its IP address. The Sniffer software on your PC captures the IP address being broadcast.

1.7  CTC Operation Troubleshooting

This section contains troubleshooting procedures for CTC login or operation problems.

1.7.1  Unable to Launch CTC Help After Removing Netscape

Symptom    After removing Netscape and running CTC using Internet Explorer, the user is unable to launch the CTC Help and receives an "MSIE is not the default browser" error message.
Possible Cause    Loss of association between browser and Help files.
Recommended Action    When the CTC software and Netscape are installed, the Help files are associated with Netscape by default. When you remove Netscape, the Help files are not automatically associated with Internet Explorer as the default browser. Set Internet Explorer as the default browser so that CTC will associate the Help files to the correct browser. See the "Set Internet Explorer as the Default Browser for CTC" procedure to associate the CTC Help files to the correct browser.

Set Internet Explorer as the Default Browser for CTC

Step 1 Open the Internet Explorer browser.

Step 2 From the menu bar, click Tools > Internet Options. The Internet Options window appears.

Step 3 In the Internet Options window, click the Programs tab.

Step 4 Click the Internet Explorer should check to see whether it is the default browser check box.

Step 5 Click OK.

Step 6 Exit any and all open and running CTC and Internet Explorer applications.

Step 7 Launch Internet Explorer and open a new CTC session. You should now be able to access the CTC Help.

1.7.2  Unable to Change Node View to Network View

Symptom    When activating a large, multinode BLSR, some of the nodes may appear grayed out, such as when you upgrade from R3.2 to R3.3. Logging into the new CTC, the user is unable to change node view to network view on any and all nodes, from any workstation. This is accompanied by an "Exception occurred during event dispatching: java.lang.OutOfMemoryError" in the java window.
Possible Cause    The large, multinode BLSR requires more memory for the graphical user interface (GUI) environment variables.
Recommended Action    Reset the system or user CTC_HEAP environment variable to increase the memory limits. See the "Reset the CTC_HEAP Environment Variable for Windows" procedure or the "Reset the CTC_HEAP Environment Variable for Solaris UNIX Workstation" procedure to enable the CTC_HEAP variable change.

Note This problem typically affects large networks where additional memory is required to manage large numbers of nodes and circuits.

Reset the CTC_HEAP Environment Variable for Windows

Step 1 Exit any and all open and running CTC and Netscape applications.

Step 2 From the Windows Desktop, right-click on My Computer and choose Properties from the shortcut menu.

Step 3 In the System Properties window, click the Advanced tab.

Step 4 Click Environment Variables to open the Environment Variables window.

Step 5 Click New in the User variables field or the System variables field.

Step 6 Type CTC_HEAP in the Variable Name field.

Step 7 Type 256 in the Variable Value field, and then click OK to create the variable.

Step 8 Click OK in the Environment Variables window to accept the changes.

Step 9 Click OK in the System Properties window to accept the changes.

Step 10 Restart the browser and CTC software.

Reset the CTC_HEAP Environment Variable for Solaris UNIX Workstation

Step 1 From the user shell window, end any CTC applications.

Step 2 End any Netscape applications.

Step 3 In the user shell window, set the environment variable to increase the heap size:
% setenv CTC_HEAP 256 
Step 4 Restart the browser and CTC software in the same user shell window.

1.7.3  Browser Stalls When Downloading CTC JAR Files from XTC Card

Symptom    The browser stalls or hangs when downloading a CTC Java archive (JAR) file from the XTC card.
Possible Cause    McAfee VirusScan software might be interfering with the operation. The problem occurs when the VirusScan Download Scan is enabled on McAfee VirusScan 4.5 or later.
Recommended Action    Disable the VirusScan Download Scan feature. See the "Disable the VirusScan Download Scan" procedure.

Disable the VirusScan Download Scan

Step 1 From the Windows start menu, choose Programs > Network Associates > VirusScan Console.

Step 2 Double-click the VShield icon listed in the VirusScan Console dialog box.

Step 3 Click Configure on the lower part of the Task Properties window.

Step 4 Click the Download Scan icon on the left of the System Scan Properties dialog box.

Step 5 Uncheck the Enable Internet download scanning check box.

Step 6 Click Yes when the warning message appears.

Step 7 Click OK in the System Scan Properties dialog box.

Step 8 Click OK in the Task Properties window.

Step 9 Close the McAfee VirusScan window.

1.7.4  CTC Does Not Launch

Symptom    CTC does not launch; usually an error message appears before the login window appears.
Possible Cause    The Netscape browser cache might point to an invalid directory.
Recommended Action    Redirect the Netscape cache to a valid directory. See the "Redirect the Netscape Cache to a Valid Directory" procedure.

Redirect the Netscape Cache to a Valid Directory

Step 1 Launch Netscape.

Step 2 From the Edit menu, choose Preferences.

Step 3 In the Category column on the left side, expand Advanced and select the Cache tab.

Step 4 Change your disk cache folder to point to the cache file location.

The cache file location is usually C:\ProgramFiles\Netscape\Users\yourname\cache. The yourname segment of the file location is often the same as the user name.

1.7.5  Sluggish CTC Operation or Login Problems

Symptom    You experience sluggish CTC operation or have problems logging into CTC.
Possible Cause    The CTC cache file might be corrupted or might need to be replaced.
Recommended Action    Delete the CTC cache file. This operation forces the ONS 15327 to download a new set of JAR files to your computer hard drive. See the "Delete the CTC Cache File Automatically" procedure or the "Delete the CTC Cache File Manually" procedure.

Delete the CTC Cache File Automatically

Caution All running sessions of CTC must be halted before deleting the CTC cache. Deleting the CTC cache might cause any CTC sessions running on this system to behave in an unexpected manner.

Step 1 Enter an ONS 15327 IP address in the browser URL field. The initial browser window shows a Delete CTC Cache button.

Step 2 Close all open CTC sessions and browser windows. The PC operating system does not allow you to delete files that are in use.

Step 3 Click Delete CTC Cache on the initial browser window to clear the CTC cache. Figure 1-22 shows the Delete CTC Cache window.

Figure 1-22 Deleting the CTC Cache

Delete the CTC Cache File Manually

Caution All running sessions of CTC must be halted before deleting the CTC cache. Deleting the CTC cache might cause any CTC running on this system to behave in an unexpected manner.

Step 1 To delete the JAR files manually, from the Windows Start menu choose Search > For Files or Folders.

Step 2 Enter ctc*.jar or cms*.jar in the "Search for files or folders named" field on the Search Results dialog box and click Search Now.

Step 3 Click the Modified column on the Search Results dialog box to find the JAR files that match the date when you downloaded the files from the XTC.

Step 4 Highlight the files and press the keyboard Delete key.

Step 5 Click Yes in the Confirm dialog box.

1.7.6  Node Icon is Gray on CTC Network View

Symptom    The CTC network view shows one or more node icons as gray in color and without a node name.
Possible Cause    Different CTC releases are not recognizing each other. Usually accompanied by an INCOMPATIBLE-SW alarm.
Recommended Action    Correct the core version build as described in the "Different CTC Releases Do Not Recognize Each Other" section.
Possible Cause    A username/password mismatch. Usually accompanied by a NOT-AUTHENTICATED alarm.
Recommended Action    Correct the username and password as described in the "Username or Password Does Not Match the XTC Card Information" section.
Possible Cause    No IP connectivity between nodes. Usually accompanied by Ethernet-specific alarms.
Recommended Action    Verify the Ethernet connections as described in the "Ethernet Connections" section.
Possible Cause    A lost DCC connection. Usually accompanied by an EOC alarm.
Recommended Action    Clear the EOC alarm and verify the DCC connection as described in the "EOC" section.
1.7.7  CTC Cannot Launch Due to Applet Security Restrictions

Symptom    The error message "Unable to launch CTC due to applet security restrictions" appears after you enter the IP address in the browser window.
Possible Cause    Did not execute the javapolicyinstall.bat file.
Possible Cause    The java.policy file might be incomplete.
Recommended Action    Verify that you have executed the javapolicyinstall.bat file on the ONS 15327 software CD. This file is installed when you run the CTC Setup Wizard. (Refer to the CTC installation information in the Cisco ONS 15327 Procedure Guide for instructions). If you ran the javapolicyinstall.bat file but still receive the error message, you must manually edit the java.policy file on your computer. See the "Manually Edit the java.policy File" procedure.

Manually Edit the java.policy File

Step 1 Search your computer for this file and open it with a text editor (Notepad or Wordpad).

Step 2 Verify that the end of this file has the following lines:
		// Insert this into the system-wide or a per-user java.policy file.
	// DO NOT OVERWRITE THE SYSTEM-WIDE POLICY FILE--ADD THESE LINES!
	grant codeBase "http://*/fs/LAUNCHER.jar" {
permission java.security.AllPermission;
	};
Step 3 If these five lines are not in the file, enter them manually.

Step 4 Save the file and restart Netscape.

CTC should now start correctly.

Step 5 If the error message is still reported, save the java.policy file as ".java.policy." On Windows 98, Windows 2000, and Windows XP PCs, save the file to the C:\Windows folder. On Windows NT 4.0 PCs, save the file to all of the user folders on that PC, for example, C:\Winnt\profiles\joeuser.

1.7.8  Java Runtime Environment Incompatible

Symptom    The CTC application does not run properly.
Possible Cause    The compatible Java 2 JRE is not installed.
Recommended Action    The Java 2 JRE contains the Java virtual machine, runtime class libraries, and Java application launcher that are necessary to run programs written in the Java programming language. The ONS 15327 CTC is a Java application. A Java application, unlike an applet, cannot rely completely on a web browser for installation and runtime services. When you run an application written in the Java programming language, you need the correct JRE installed. The correct JRE for each CTC software release is included on the Cisco ONS 15327 software CD and on the Cisco ONS 15327 documentation CD. See the "Launch CTC to Correct the Core Version Build" procedure. If you are running multiple CTC software releases on a network, the JRE installed on the computer must be compatible with all of the releases that you are running. See Table 1-1.

Note Software R4.0 and later will notify you if an older version JRE is running on your PC or UNIX workstation.

Table 1-1 shows JRE compatibility with ONS 15327 software releases.

Table 1-1 JRE Compatibility

ONS Software Release

JRE 1.2.2 Compatible

JRE 1.3 Compatible

JRE 1.4 Compatible

JRE 1.4.2 Compatible¹

JRE 5.0 Compatible

ONS 15327 R1.0.0

Yes

No

No

No

No

ONS 15327 R1.0.1

Yes

Yes

No

No

No

ONS 15327 R2.2.1 and earlier to R1.0.1

Yes

No

No

No

No

ONS 15327 R2.2.2

Yes

Yes

No

No

No

ONS 15327 R3.0

Yes

Yes

No

No

No

ONS 15327 R3.1

Yes

Yes

No

No

No

ONS 15327 R3.2

Yes

Yes

No

No

No

ONS 15327 R3.3

Yes

Yes

No

No

No

ONS 15327 R3.4

No

Yes

No

No

No

ONS 15327 R4.0

No

Yes

No

No

No

ONS 15327 R4.1

No

Yes

No

No

No

ONS 15327 R4.6

No

Yes

Yes

No

No

ONS 15327 R5.0

No

Yes

Yes

No

No

ONS 15327 R6.0

No

No

No

Yes

No

ONS 15327 R7.0

No

No

No

Yes

Yes

¹JRE 1.4.2 is the recommended version and is provided on the software CD.

Launch CTC to Correct the Core Version Build

Step 1 Exit the current CTC session and completely close the browser.

Step 2 Start the browser.

Step 3 Type the ONS 15327 IP address of the node that reported the alarm. This can be the original IP address you logged in with or an IP address other than the original.

Step 4 Log into CTC. The browser downloads the JAR file from the XTC's version CTC.

Note After Software R2.2.2, the single CMS.jar file evolved into core and element files. Core files are common to both the ONS 15327 and ONS 15454, while the element files are unique to the particular product. For example, ONS 15327 R1.0 uses a Version 2.3 core build and a Version 1.0 element build. To view the CTC Core Version number, from the CTC menu bar click Help > About CTC. This lists the core and element builds discovered on the network.

1.7.9  Different CTC Releases Do Not Recognize Each Other

Symptom    The CTC version on the workstation does not recognize the software on the XTC card.
Possible Cause    The software loaded on the connecting workstation and the software on the XTC card are incompatible.
Recommended Action    This occurs when the XTC software is upgraded but the PC has not yet upgraded the compatible CTC JAR file. It also occurs on login nodes with compatible software that encounter other nodes in the network that have a newer software version. See the "Launch CTC to Correct the Core Version Build" procedure.

Note Remember to always log into the ONS node with the latest CTC core version first. If you initially log into an ONS node running a CTC core version of 2.2 or earlier and then attempt to log into another ONS node in the network running a later CTC core version, the earlier version node does not recognize the new node.

Note This situation is often accompanied by the INCOMPATIBLE-SW condition.

1.7.10  Username or Password Does Not Match the XTC Card Information

Symptom    A mismatch often occurs concurrently with a NOT-AUTHENTICATED condition.
Possible Cause    The username or password entered does not match the information stored in the XTC.
Recommended Action    All ONS nodes must have the same username and password created to display every ONS node in the network. You can also be locked out of certain ONS nodes on a network if your username and password were not created on those specific ONS nodes. For initial login to the ONS 15327, enter the CISCO15 user name in capital letters and click Login and use the password "otbu+1," which is case-sensitive. See the "Verify Correct Username and Password" procedure. If the node has been configured for RADIUS authentication, the username and password are verified against the RADIUS server database rather than the security information in the local node database. For more information about RADIUS security, refer to the "Security" chapter in the Cisco ONS 15327 Reference Manual.

Verify Correct Username and Password

Step 1 Ensure that your keyboard Caps Lock key is not turned on and affecting the case-sensitive entry of the username and password.

Step 2 Contact your system administrator to verify the username and password.

Step 3 Contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447 to have them enter your system and create a new user name and password.

1.7.11  No IP Connectivity Exists Between Nodes

Symptom    The nodes have a gray icon that is usually accompanied by alarms.
Possible Cause    Lost Ethernet connection,
Recommended Action    Usually, this condition is accompanied by Ethernet-specific alarms. Verify the Ethernet connections as described in the "Ethernet Connections" section.
1.7.12  DCC Connection Lost

Symptom    The node is usually accompanied by alarms and the nodes in the network view have a gray icon. This symptom is usually accompanied by an EOC alarm.
Possible Cause    A lost DCC connection.
Recommended Action    Usually, this condition is accompanied by an EOC alarm. Clear the EOC alarm and verify the DCC connection as described in the "EOC" section.
1.7.13  "Path in Use" Error When Creating a Circuit

Symptom    While creating a circuit, you get a "Path in Use" error that prevents you from completing the circuit creation.
Possible Cause    Another user has already selected the same source port to create another circuit.
Recommended Action    CTC does not remove a card or port from the available list until a circuit is completely provisioned. If two users simultaneously select the same source port to create a circuit, the first user to complete circuit provisioning gets use of the port. The other user gets the "Path in Use" error. See the "Cancel the Circuit Creation and Start Over" procedure.

Cancel the Circuit Creation and Start Over

Step 1 Cancel the circuit creation:

a. Click Cancel.

b. Click Back until you return to the initial circuit creation window.

Step 2 Check the list of available ports. The previously selected port no longer appears in the available list because it is now part of a provisioned circuit.

Step 3 Select a different available port and begin the circuit creation process.

1.7.14  Calculate and Design IP Subnets

Symptom    You cannot calculate or design IP subnets on the ONS 15327.
Possible Cause    The IP capabilities of the ONS 15327 require specific calculations to properly design IP subnets.
Recommended Action    Cisco provides a free online tool to calculate and design IP subnets. Go to http://www.cisco.com/techtools/ip_addr.html. For information about ONS 15327 IP capability, refer to the Cisco ONS 15327 Reference Manual.
1.7.15  Ethernet Connections

Symptom    Ethernet connections appear to be broken or are not working properly.
Possible Cause    Improperly seated connections.
Possible Cause    Incorrect connections.
Recommended Action    You can fix most connectivity problems in an Ethernet network by following a few guidelines. See Figure 1-23 when consulting the steps in the "Verify Ethernet Connections" procedure.
Figure 1-23 Ethernet Connectivity Reference

Verify Ethernet Connections

Step 1 Verify that the alarm filter is turned off.

Step 2 Check for SONET alarms on the STS-N that carries the VLAN #1 Ethernet circuit. Clear any alarms by looking them up in "Alarm Troubleshooting."

Step 3 Check for Ethernet-specific alarms. Clear any raised alarms by looking up that alarm in "Alarm Troubleshooting."

Step 4 Verify that the ACT LED on the Ethernet card is green.

Step 5 Verify that Ports 1 and 3 on ONS Node #1 and Ports 1 and 2 on ONS Node #2 have green link-integrity LEDs illuminated.

Step 6 If no green link-integrity LED is illuminated for any of these ports:

a. Verify physical connectivity between the ONS 15327s and the attached device.

b. Verify that the ports are enabled on the Ethernet cards.

c. Verify that you are using the proper Ethernet cable and that it is wired correctly, or replace the cable with an Ethernet cable known to be good.

d. Check the status LED on the Ethernet card faceplate to ensure that the card booted up properly. This LED should be steady green. If necessary, remove and reinsert the card and allow it to reboot.

e. It is possible that the Ethernet port is functioning properly but the link LED itself is broken. Complete the "Lamp Tests for Card LEDs" procedure.

Step 7 Verify connectivity between device A and device C by pinging between these locally attached devices (complete the "Verify PC Connection to the ONS 15327 (Ping)" procedure). If the ping is unsuccessful:

a. Verify that Device A and Device C are on the same IP subnet.

b. In node view, double-click the Ethernet card. The card view appears.

c. Click the Provisioning > VLAN tabs to verify that both Port 1 and Port 3 on the card are assigned to the same VLAN.

d. If a port is not assigned to the correct VLAN, click that port column in the VLAN row and set the port to Tagged or Untag. Click Apply.

Step 8 Repeat Step 7 for devices B and D.

Step 9 Verify that the Ethernet circuit that carries VLAN #1 is provisioned and that ONS Node #1 and ONS Node #2 ports also use VLAN #1.

1.7.16  VLAN Cannot Connect to Network Device from Untag Port

Symptom    Networks that have a VLAN with one ONS 15327 Ethernet card port set to Tagged and one ONS 15327 Ethernet card set to Untag might have difficulty implementing Address Resolution Protocol (ARP) for a network device attached to the Untag port (Figure 1-24). They might also see a higher than normal runt packets count at the network device attached to the Untag port. This symptom/limitation also exists when ports within the same card or ports within the same chassis are put on the same VLAN, with a mix of tagged and untagged.
Possible Cause    The tagged ONS 15327 adds the IEEE 802.1Q tag and the untagged ONS 15327 removes the Q-tag without replacing the bytes. The NIC of the network device categorizes the packet as a runt and drops the packet.
Possible Cause    Dropped packets can also occur when ARP attempts to match the IP address of the network device attached to the Untag port with the physical MAC address required by the network access layer.
Recommended Action    See the "Change VLAN Port Tag and Untagged Settings" procedure. The solution is to set both ports in the VLAN to Tagged to stop the stripping of the four bytes from the data packet and prevents the NIC card in the network access device from recognizing the packet as a runt and dropping it. Network devices with IEEE 802.1Q-compliant NIC cards can accept the tagged packets. Network devices with non-IEEE 802.1Q compliant NIC cards still drop these tagged packets. The solution might require upgrading network devices with non-IEEE 802.1Q compliant NIC cards to IEEE 802.1Q-compliant NIC cards. You can also set both ports in the VLAN to Untag, but you lose IEEE 802.1Q compliance.
Figure 1-24 VLAN with Ethernet Ports at Tagged and Untag

Change VLAN Port Tag and Untagged Settings

Step 1 In node view, double-click the Ethernet card involved in the problem VLAN. The card view appears.

Step 2 Click the Provisioning > VLAN tabs (Figure 1-25).

Figure 1-25 Configuring VLAN Membership for Individual Ethernet Ports

Step 3 If the port is set to Tagged, continue to look at other cards and their ports in the VLAN until you find the port that is set to Untag.

Step 4 At the VLAN port set to Untag, click the port and choose Tagged.

Note The attached external devices must recognize IEEE 802.1Q VLANs.

Step 5 After each port is in the appropriate VLAN, click Apply.

1.8  Circuits and Timing

This section provides solutions to circuit creation and reporting errors, as well as common timing reference errors and alarms.

1.8.1  Circuit Transitions to the OOS-PARTIAL Service State

Symptom    An automatic or manual transition of a circuit from one state to another state results in the OOS-PARTIAL status, where at least one of the connections in the circuit is in OOS state and at least one other connection in the circuit is in the IS-NR, OOS-MA-MT, or OOS-AU,AINS service state.
Possible Cause    During a manual transition, CTC cannot communicate with one of the nodes or one of the nodes is on a version of software that does not support the new state model.
Recommended Action    Repeat the manual transition operation. If the partial state persists, determine which node in the circuit is not changing to the desired state. Refer to the "View the State of Circuit Nodes" procedure. Log onto the circuit node that did not change to the desired state and determine the version of software. If the software on the node is Software R4.1 or earlier, upgrade the software. Refer to the Cisco ONS 15327 Software Upgrade Guide for software upgrade procedures.

Note If the node software cannot be upgraded to Software R4.5 or later, the partial state condition can be avoided by only using the circuit states supported in the earlier software release.

Possible Cause    During an automatic transition, some path-level defects and/or alarms were detected on the circuit.
Possible Cause    One end of the circuit is not properly terminated.
Recommended Action    Determine which node in the circuit is not changing to the desired state. Refer to the "View the State of Circuit Nodes" procedure. Log into the circuit node that did not change to the desired state and examine the circuit for path-level defects, improper circuit termination, or alarms. Refer to the Cisco ONS 15327 Procedure Guide for procedures to clear alarms and change circuit configuration settings. Resolve and clear the defects and/or alarms on the circuit node and verify that the circuit transitions to the desired state.

View the State of Circuit Nodes

Step 1 Click the Circuits tab.

Step 2 From the Circuits tab list, select the circuit with the *_PARTIAL state condition.

Step 3 Click Edit. The Edit Circuit window appears.

Step 4 In the Edit Circuit window, click the State tab.

The State tab window lists the Node, CRS End A, CRS End B, and CRS State for each of the nodes in the circuit.

1.8.2  Circuits Remain in PARTIAL Status

Symptom    Circuits remain in the PARTIAL status.
Possible Cause    The MAC address changed.
Recommended Action    Repair the circuits. See the "Repair Circuits" procedure.
1.8.2.1  Repair Circuits

Step 1 In node view, click the Circuits tab. Note that all circuits listed are PARTIAL.

Step 2 In node view, choose Repair Circuits from the Tools drop-down list. The Circuit Repair dialog box appears.

Step 3 Read the instructions in the Circuit Repair dialog box. If all the steps in the dialog box have been completed, click Next. Ensure that you have the old and new MAC addresses.

Step 4 The Node MAC Addresses dialog box appears:

a. From the Node drop-down list, choose the name of the node.

b. In the Old MAC Address field, enter the old MAC address.

c. Click Next.

Step 5 The Repair Circuits dialog box appears. Read the information in the dialog box and click Finish.

Note The CTC session freezes until all circuits are repaired. Circuit repair can take up to five minutes or more depending on the number of circuits provisioned.

When the circuit repair is complete, the Circuits Repaired dialog box appears.

Step 6 Click OK.

Step 7 In the node view of the new node, click the Circuits tab. Note that all circuits listed are DISCOVERED. If all circuits listed do not have a DISCOVERED status, contact Cisco TAC at http://www.cisco.com/en/US/support/index.html or 1-800-553-2447 to open an RMA request.

1.8.3  AIS-V on XTC-28-3 Card Unused VT Circuits

Symptom    An incomplete circuit path causes an alarm indications signal (AIS).
Possible Cause    The port on the reporting node is in-service but a node upstream on the circuit does not have an OC-N port in service.
Recommended Action    An AIS-V indicates that an upstream failure occurred at the virtual tributary (VT) layer. AIS-V alarms also occur on XTC-28-3 card VT circuits that are not carrying traffic and on stranded bandwidth. Perform the "Clear AIS-V on XTC-28-3 Card Unused VT Circuits" procedure.

Clear AIS-V on XTC-28-3 Card Unused VT Circuits

Step 1 Determine the affected port.

Step 2 Record the node ID, slot number, port number, and VT number.

Step 3 Create a unidirectional VT circuit from the affected port back to itself, such as Source node/Slot 2/Port 2/VT 13 cross connected to Source node/Slot 2/Port 2/VT 13.

Step 4 Uncheck the bidirectional check box in the circuit creation window.

Step 5 Give the unidirectional VT circuit an easily recognizable name, such as "delete me."

Step 6 Display the XTC-28-3 card in CTC card view. Click the Maintenance > DS1 tabs.

Step 7 Locate the VT that is reporting the alarm (for example, DS3 #2, DS1 #13).

Step 8 From the Loopback Type list, choose Facility and click Apply.

Step 9 Click Circuits.

Step 10 Find the one-way circuit you created in Step 3. Select the circuit and click Delete.

Step 11 Click Yes in the Delete Confirmation dialog box.

Step 12 In node view, double-click the XTC-28-3 card. The card view appears.

Step 13 Click the Maintenance > DS1 tabs.

Step 14 Locate the VT in Facility Loopback list.

Step 15 From the Loopback Type list, choose None and then click Apply.

Step 16 Click the Alarm tab and verify that the AIS-V alarms have cleared.

Step 17 Repeat this procedure for all the AIS-V alarms on the XTC-28-3 cards.

1.8.4  Circuit Creation Error with VT1.5 Circuit

Symptom    You might receive an "Error while finishing circuit creation. Unable to provision circuit. Unable to create connection object at node-name" message when trying to create a VT1.5 circuit in CTC.
Possible Cause    You might have run out of bandwidth on the VT cross-connect matrix at the ONS 15327 node indicated in the error message.
Recommended Action    The matrix has a maximum capacity of 336 bidirectional VT1.5 cross-connects. Certain configurations exhaust VT capacity with less than 336 bidirectional VT1.5s in a BLSR or less than 224 bidirectional VT1.5s in a path protection or 1+1 protection group. Refer to the Cisco ONS 15327 Reference Manual for more information.
1.8.5  XTC Card Does Not Report AIS-P From External Equipment

Symptom    An XTC card does not report STS AIS-P from the external equipment/line side.
Possible Cause    The card is functioning as designed.
Recommended Action    This card terminates the port signal at the backplane so STS AIS-P is not reported from the external equipment/line side. XTC cards have DS-3 header monitoring functionality, which allows you to view performance monitoring (PM) on the DS-3 path. Nevertheless, you cannot view AIS-P on the STS path. For more information on the PM capabilities of the XTC cards, refer to the Cisco ONS 15327 Procedure Guide.
1.8.6  OC-3 and DCC Limitations

Symptom    Limitations to OC-3 and DCC usage.
Possible Cause    OC-3 and DCC have limitations for the ONS 15327.
Recommended Action    For an explanation of OC-3 and DCC limitations, refer to the "DCC Tunnels" section in the Cisco ONS 15327 Procedure Guide.
1.8.7  ONS 15327 Switches Timing Reference

Symptom    Timing references switch when one or more problems occur.
Possible Cause    The optical or building integrated timing supply (BITS) input is receiving loss of signal (LOS), loss of frame (LOF), or AIS alarms from its timing source.
Possible Cause    The optical or BITS input is not functioning.
Possible Cause    Synchronization status message (SSM) is set to Do Not Use for Synchronization (DUS).
Possible Cause    SSM indicates a Stratum 3 or lower clock quality.
Possible Cause    The input frequency is off by more than 15 ppm.
Possible Cause    The input clock wanders and has more than three slips in 30 seconds.
Possible Cause    A bad timing reference existed for at least two minutes.
The ONS 15327 internal clock operates at a Stratum 3 level of accuracy. This gives the ONS 15327 a free-running synchronization accuracy of +/- 4.6 ppm and a holdover stability of less than 255 slips in the first 24 hours or 3.7 x 10^-7 per day, including temperature. ONS 15327 free-running synchronization relies on the Stratum 3 internal clock. Use a higher quality Stratum 1 or Stratum 2 timing source. This results in fewer timing slips than a lower quality Stratum 3 timing source.

1.8.8  Holdover Synchronization Alarm

Symptom    The clock is running at a different frequency than normal and the HLDOVRSYNC alarm appears.
Possible Cause    The last reference input has failed.
Recommended Action    The clock is running at the frequency of the last valid reference input. This alarm is raised when the last reference input fails. See the "HLDOVRSYNC" section for a detailed description of this alarm.

Note The ONS 15327 supports holdover timing per Telcordia GR-4436 when provisioned for external (BITS) timing.

1.8.9  Free-Running Synchronization Mode

Symptom    The clock is running at a different frequency than normal and the free-running synchronization FRNGSYNC condition appears.
Possible Cause    No reliable reference input is available.
Recommended Action    The clock is using the internal oscillator as its only frequency reference. This occurs when no reliable, prior timing reference is available. See the "FRNGSYNC" section for a detailed description of this alarm.
1.8.10  Daisy-Chained BITS Not Functioning

Symptom    You are unable to daisy-chain the BITS.
Possible Cause    Daisy-chaining BITS is not supported on the ONS 15327.
Recommended Action    Daisy-chaining BITS causes additional wander buildup in the network and is therefore not supported. Instead, use a timing signal generator to create multiple copies of the BITS clock and separately link them to each ONS 15327.
1.8.11  Blinking STAT LED after Installing a Card

Symptom    After installing a card, the STAT LED blinks continuously for more than 60 seconds.
Possible Cause    The card cannot boot because it failed the Power On Shelf Test (POST) diagnostics.
Recommended Action    The blinking STAT LED indicates that POST diagnostics are being performed. If the LED continues to blink more than 60 seconds, the card has failed the POST diagnostics test and has failed to boot. If the card has truly failed, an EQPT-BOOT alarm is raised against the slot number with an "Equipment Fails To Boot" description. Check the alarm tab for this alarm to appear for the slot where the card is installed. To attempt recovery, remove and reinstall the card and observe the card boot process. If the card fails to boot, replace the card.
1.9  Fiber and Cabling

This section explains problems typically caused by cabling connectivity errors. It also includes instructions for crimping CAT-5 cable and lists the optical fiber connectivity levels.

1.9.1  Bit Errors Appear for a Traffic Card

Symptom    A traffic card has multiple bit errors.
Possible Cause    Faulty cabling or low optical-line levels.
Recommended Action    Bit errors on line (traffic) cards usually originate from cabling problems or low optical-line levels. The errors can be caused by synchronization problems, especially if pointer justification (PJ) errors are reported. Moving cards into different error-free slots isolates the cause. Use a test set whenever possible because the cause of the errors could be external cabling, fiber, or external equipment connecting to the ONS 15327. (For instructions to use the test-set equipment, consult the manufacturer.) Troubleshoot cabling problems using the "Troubleshooting Circuit Paths with Loopbacks" section. Troubleshoot low optical levels using procedures in the "Faulty Fiber-Optic Connections" section.
1.9.2  Faulty Fiber-Optic Connections

Symptom    A line card has multiple SONET alarms and/or signal errors.
Possible Cause    Faulty fiber-optic connections.
Recommended Action    Faulty fiber-optic connections can be the source of SONET alarms and signal errors. See the "Verify Fiber-Optic Connections" procedure.
Possible Cause    Faulty CAT-5 cables.
Recommended Action    Faulty CAT-5 cables can be the source of SONET alarms and signal errors. See the "Crimp Replacement LAN Cables" procedure.
Possible Cause    Faulty GBICs.
Recommended Action    Faulty GBICs can be the source of SONET alarms and signal errors. See the "Replace Faulty SFP Connectors" procedure.
Warning Invisible laser radiation may be emitted from disconnected fibers or connectors. Do not stare into beams or view directly with optical instruments. Statement 272

Warning Class 1 laser product. Statement 1008

Verify Fiber-Optic Connections

Step 1 Ensure that a single-mode fiber connects to the ONS 15327 card.

SM or SM Fiber should be printed on the fiber span cable. ONS 15327 cards do not use multimode fiber.

Step 2 Ensure that the connector keys on the SC fiber connector are properly aligned and locked.

Step 3 Check that the single-mode fiber power level is within the specified range:

a. Remove the Rx end of the suspect fiber.

b. Connect the receive end of the suspect fiber to a fiber-optic power meter, such as a GN Nettest LP-5000.

c. Determine the power level of fiber with the fiber-optic power meter.

d. Verify that the power meter is set to the appropriate wavelength for the optical card being tested (either 1310 nm or 1550 nm depending on the specific card).

e. Verify that the power level falls within the range specified for the card; see the "Optical Card Transmit and Receive Levels" section.

Step 4 If the power level falls below the specified range:

a. Clean or replace the fiber patchcords. Clean the fiber according to site practice or, if none exists, follow the procedure in the Cisco ONS 15327 Procedure Guide. If possible, do this for the OC-N card you are working on and the far-end card.

b. Clean the optical connectors on the card. Clean the connectors according to site practice or, if none exists, follow the procedure in the Cisco ONS 15327 Procedure Guide. If possible, do this for the card you are working on and the far-end card.

c. Ensure that the far-end transmitting card is not an ONS intermediate-range (IR) card when an ONS long-range (LR) card is appropriate.

IR cards transmit a lower output power than LR cards.

d. Replace the far-end transmitting card to eliminate the possibility of a degrading transmitter on this card.

e. If the power level still falls below the specified range with the replacement fibers and replacement card, check for one of these three factors that attenuate the power level and affect link loss (LL):

•Excessive fiber distance—Single-mode fiber attenuates at approximately 0.5 dB/km.

•Excessive number or fiber connectors—Connectors take approximately 0.5 dB each.

•Excessive number of fiber splices—Splices take approximately 0.5 dB each.

Note These are typical attenuation values. Refer to the specific product documentation for the actual values or use an optical time domain reflectometer (OTDR) to establish precise link loss and budget requirements.

Step 5 If no power level shows on the fiber, the fiber is bad or the transmitter on the optical card failed:

a. Check that the Tx and Rx fibers are not reversed. LOS and EOC alarms normally accompany reversed Tx and Rx fibers. Switching reversed Tx and Rx fibers clears the alarms and restores the signal.

b. Clean or replace the fiber patchcords. Clean the fiber according to site practice or, if none exists, follow the procedure in the Cisco ONS 15327 Procedure Guide. If possible, do this for the card you are working on and the far-end card.

c. Retest the fiber power level.

d. If the replacement fiber still shows no power, replace the optical card.

Step 6 If the power level on the fiber is above the range specified for the card, ensure that an ONS LR card is not being used when an ONS IR card is appropriate.

LR cards transmit a higher output power than IR cards. When used with short runs of fiber, an LR transmitter is too powerful for the receiver on the receiving card.

Receiver overloads occur when maximum receiver power is exceeded.

Tip To prevent overloading the receiver, use an attenuator on the fiber between the card transmitter and the receiver. Place the attenuator on the receive transmitter of the cards. Refer to the attenuator documentation for specific instructions.

Tip Most fiber has text printed on only one of the two fiber strands. Use this to identify which fiber is connected to Tx and which fiber is connected to Rx.

1.9.2.1  Crimp Replacement LAN Cables

You can crimp your own LAN cables for use with the ONS 15327. Use #22 or #24 AWG shielded wire with RJ-45 connectors, and a crimping tool.

Use a crossover cable when connecting an ONS 15327 to a hub, LAN modem, or switch, and use a LAN cable when connecting an ONS 15327 to a router or workstation.

Figure 1-26 shows the layout of an RJ-45 connector.

Figure 1-26 RJ-45 Pin Numbers

Figure 1-27 shows the layout of a LAN cable.

Figure 1-27 LAN Cable Layout

Table 1-2 shows LAN cable pinouts.

Table 1-2 LAN Cable Pinout

Pin

Color

Pair

Name

Pin

1

White/orange

2

Transmit Data +

1

2

Orange

2

Transmit Data -

2

3

White/green

3

Receive Data +

3

4

Blue

1

—

4

5

White/blue

1

—

5

6

Green

3

Receive Data -

6

7

White/brown

4

—

7

8

Brown

4

—

8

Figure 1-28 shows the layout of a cross-over cable.

Figure 1-28 Cross-Over Cable Layout

Table 1-3 shows crossover cable pinouts.

Table 1-3 Crossover Cable Pinout

Pin

Color

Pair

Name

Pin

1

White/orange

2

Transmit Data +

3

2

Orange

2

Transmit Data -

6

3

White/green

3

Receive Data +

1

4

Blue

1

—

4

5

White/blue

1

—

5

6

Green

3

Receive Data -

2

7

White/brown

4

—

7

8

Brown

4

—

8

Note Odd-numbered pins always connect to a white wire with a colored stripe.

1.9.2.2  Replace Faulty SFP Connectors

Small Form-factor Pluggable (SFP) connectors are hot-swappable and can be installed or removed while the card or shelf assembly is powered and running.

Warning Class 1 laser product. Statement 1008

Warning Invisible laser radiation may be emitted from disconnected fibers or connectors. Do not stare into beams or view directly with optical instruments. Statement 272

SFPs are input/output devices that plug into a Gigabit Ethernet card to link the port with the fiber-optic network. The type of SFP determines the maximum distance that the Ethernet traffic can travel from the card to the next network device. For a description of SFPs and their capabilities, see Table 1-4 and refer to the Cisco ONS 15327 Reference Manual.

Note SFPs must be matched on either end by type: SX to SX, LX to LX.

Table 1-4 Available SFP Connectors

SFP

Associated Cards

Application

Fiber

Product Number

1000BaseSX

G1000-2

Short reach

Multimode fiber up to 550 m long

15327-SFP-LC-SX=

1000BaseLX

G1000-2

Long reach

Single-mode fiber up to 5 km long

15327-SFP-LC-LX=

Remove SFP Connectors

Step 1 Disconnect the network fiber cable from the SFP LC duplex connector.

Warning Invisible laser radiation may be emitted from disconnected fibers or connectors. Do not stare into beams or view directly with optical instruments. Statement 272

Step 2 Release the SFP from the slot by simultaneously squeezing the two plastic tabs on each side.

Step 3 Slide the SFP out of the Gigabit Ethernet module slot. A flap closes over the SFP slot to protect the connector on the Gigabit Ethernet card.

Install SFP Connectors

Step 1 Remove the SFP from its protective packaging.

Step 2 Check the label to verify that the SFP is the correct type (SX or LX) for your network.

Step 3 Verify that you are installing compatible SFPs; for example, SX to SX, LX to LX.

Step 4 Grip the sides of the SFP with your thumb and forefinger and insert the SFP into the slot on the G-Series card.

Note SFPs are keyed to prevent incorrect installation.

Step 5 Slide the SFP through the flap that covers the opening until you hear a click. The click indicates the SFP is locked into the slot.

Step 6 When you are ready to attach the network fiber-optic cable, remove the protective plug from the SFP and save the plug for future use.

1.9.2.3  Optical Card Transmit and Receive Levels

Each G-Series and OC-N card has a transmit and receive connector on its faceplate. Table 1-5 describes the SFPs and their capabilities.

Table 1-5 Optical Card Transmit and Receive Levels

Optical Card

Receive

Transmit

G1000-2

-8 to -28 dBm

-8 to -15 dBm

OC3 IR 4 1310

-8 to -28 dBm

-8 to -15 dBm

OC12 IR 1310

-8 to -28 dBm

-8 to -15 dBm

OC12 LR 1550

-8 to -28 dBm

+2 to -3 dBm

OC48-1-IR

0 to -18 dBm

0 to -5 dBm

OC48 LR 1550

-8 to -28 dBm

+3 to -2 dBm

1.10  Power and LED Tests

This section provides symptoms and solutions for power supply, power consumption, and LED indicator problems.

1.10.1  Power Supply Problems

Symptom    Loss of power or low voltage, resulting in a loss of traffic and causing the LCD clock to reset to the default date and time.
Possible Cause    Loss of power or low voltage.
Possible Cause    Improperly connected power supply.
Recommended Action    The ONS 15327 requires a constant source of DC power to properly function. Input power is -48 VDC. Power requirements range from -42 VDC to -57 VDC. A newly installed ONS 15327 that is not properly connected to its power supply does not operate. Power problems can be confined to a specific ONS 15327 or can affect several pieces of equipment on the site. See the "Isolate the Cause of Power Supply Problems" procedure.

Note A loss of power or low voltage can result in a loss of traffic and causes the LCD clock on the ONS 15327 to default to January 1, 1970, 00:04:15. To reset the clock, in node view click the Provisioning > General tabs and change the Date and Time fields.

Warning Only trained and qualified personnel should be allowed to install, replace, or service this equipment. Statement 1030

Warning During this procedure, wear grounding wrist straps to avoid ESD damage to the card. Do not directly touch the backplane with your hand or any metal tool, or you could shock yourself. Statement 94

Caution Operations that interrupt power supply or short the power connections to the ONS 15327 are service-affecting.

Isolate the Cause of Power Supply Problems

Step 1 If a single ONS 15327 show signs of fluctuating power or power loss:

a. Verify that the -48 VDC #8 power terminals are properly connected to a fuse panel.

b. Verify that the power cable is #12 or #14 AWG and in good condition.

c. Verify that the power cable connections are properly crimped. Stranded #12 or #14 AWG does not always crimp properly with Staycon-type connectors.

d. Verify that 20 A fuses are used in the fuse panel.

e. Verify that the fuses are not blown.

f. Verify that the DC power source has enough capacity to carry the power load.

g. If the DC power source is battery-based:

•Verify that the output power is high enough. Power requirements range from -42 VDC to -57 VDC.

•Check the age of the batteries. Battery performance decreases with age.

•Check for opens and shorts in batteries, which might affect power output.

•If brown-outs occur, the power load and fuses might be too high for the battery plant.

Step 2 If multiple pieces of site equipment show signs of fluctuating power or power loss:

a. Check the uninterruptible power supply (UPS) or rectifiers that supply the equipment. Refer to the UPS manufacturer's documentation for specific instructions.

b. Check for excessive power drains caused by other equipment, such as generators.

c. Check for excessive power demand on backup power systems or batteries when alternate power sources are used.

1.10.2  Power Consumption for Node and Cards

Symptom    You are unable to power up a node or the cards in a node.
Possible Cause    Improper power supply.
Recommended Action    Refer to power information in the Cisco ONS 15327 Procedure Guide.
1.10.3  Lamp Tests for Card LEDs

The LED lamp test determines whether card-level LEDs are operational. For optical and electrical cards, this test also causes port-level LEDs to illuminate. For all other data cards, only card-level LEDs light. For these cards, port-level LEDs can be compared to the given guidelines to determine whether they are working correctly.

Symptom    Optical (OC-N) or electrical (DS-N) card LEDs do not light, or you are unsure whether the LEDs are working properly.
Possible Cause    Faulty optical and electrical port LED.
Recommended Action    A lamp test verifies that all the port LEDs work. Run this diagnostic test as part of the initial ONS 15327 turn-up, a periodic maintenance routine, or any time you question whether an LED is in working order. Complete the "Verify Card LED Operation" procedure.

Note Optical and electrical port LEDs light during the lamp test. Other data card types only illuminate card-level LEDs during the test.

Symptom    G-Series Ethernet card LEDs do not light, or you are unsure if LEDs are working properly.
Possible Cause    Faulty LED.
Recommended Action    Complete the "Verify G-Series Ethernet Card LED Operation" procedure.

Note G-Series Ethernet card-level LEDs illuminate during a lamp test, but the port-level LEDs do not.

Symptom    E-Series Ethernet card LED does not light, or you are unsure if LEDs are working properly.
Possible Cause    Faulty LED.
Recommended Action    Complete the "Verify E-Series Ethernet Card LED Operation" procedure.

Note E-Series card-level LEDs illuminate during a lamp test, but the port-level LEDs do not.