Guest

Cisco ONS 15454 Series Multiservice Provisioning Platforms

ONS 15454 Timing Issues

Document ID: 15234

Updated: Jan 31, 2006

   Print

Introduction

There are several common issues that arise when you configure timing on the Cisco ONS 15454. This document explains these issues and provides an example of a timing best practice that you can use in a four-node ONS 15454 network. This document covers these areas:

Use this document with the Setting Up ONS 15454 Timing section of the Cisco ONS 15454 User Documentation. The lab setup used is based on the network in the user documentation. However, you can also use this document as a standalone configuration and troubleshooting guide.

Note: You must set the Synchronous Optical Network (SONET) timing parameters for each ONS 15454. Timing can be set to either the external, line, or mixed node. In most ONS 15454 networks, one node is set to external and the other nodes are set to line. The external node accepts its timing from a Building Integrated Timing Supply (BITS) source wired to the BITS backplane pins. The BITS source gets its timing from a primary reference source (PRS), such as a Stratum 1 (ST1) clock or global positioning satellite (GPS) signal. The line nodes accept their timing from optical carrier cards. Up to three timing references can be identified for protection. These are typically two BITS level or line-level sources and an internal reference. The internal reference is the Stratum 3 (ST3) clock provided on every ONS 15454 Timing Communication and Control (TCC) card.

Table 1 shows the clock accuracy and why it is an ST3. The timing source must be within the timing tolerance for a minimum of 24 hours when the ONS 15454 goes into holdover and is timing from its own internal clock.

Table 1 – Clock Accuracy

Stratum Accuracy Adjustment Range Pull-In-Range Stability Time To First Frame Slip
1 1 x 10-11 NA NA 72 days
2 1.6 x 10-8 Must be able to synchronize to clock with accuracy of +/-1.6 x 10-8 1 x 10-10/day 14 days
3E 4.6 x 10-6 Must be able to synchronize to clock with accuracy of +/-4.6 x 10-6 1 x 10-8/day 17 hours
3 4.6 x 10-6 Must be able to synchronize to clock with accuracy of +/-4.6 x 10-6 3.7 x 10-7/day 23 minutes
SONET Minimum Clock 20 x 10-6 Must be able to synchronize to clock with accuracy of +/-20 x 10-6 Not specified Not specified
4E 32 x 10-6 Must be able to synchronize to clock with accuracy of +/-32 x 10-6 Same as Accuracy Not specified
4 32 x 10-6 Must be able to synchronize to clock with accuracy of +/-32 x 10-6 Same as Accuracy Not specified

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Timing Concepts

A common mistake is to think that the timing reference is controlled from an ONS 15454 node to its adjacent node. Each node independently accepts its own timing reference from one of these sources:

  • The BITS Input pins on the ONS 15454 backplane

  • An optical carrier card installed in the ONS 15454

  • The internal ST3 clock on the TCC/TCC+/TCC2 card

The ONS 15454 must be set for external timing if the timing comes from the BITS Input pins. The ONS 15454 must set for line timing if the timing comes from an optical carrier card. The default timing source is the internal ST3 clock. The ONS 15454 can be set for either external, line, mixed, or internal timing. The ONS 15454 times all of its interfaces by the timing source that it accepts.

Note: DS-1s delivered over traffic links are not suitable BITS sources. The primary reason for this is that SONET compensation for off-frequency DS-1s results in jitter because controlled slips are not performed.

Mixed Mode Timing

The reference list options include two BITS sources and the internal clock in external timing mode. The reference list options include all optical ports and the internal clock in line timing mode. Both external and line timing sources can be included in the synchronization reference list with mixed mode timing. Be careful when you use mixed mode timing because it can result in inadvertent timing loops. The Cisco Transport Controller window shows how mixed mode timing is provisioned.

Note: The reference list includes both BITS and optical ports as timing sources.

Internal (Free-Running) Synchronization

The ONS 15454 has an internal clock in the TCC/TCC+/TCC2 that is used to track a higher quality reference. The clock provides holdover timing or a free-running clock source in the event of node isolation. The internal clock is a certified ST3 clock with enhanced capabilities that match the Stratum 3E specifications for:

  • Free-run accuracy.

  • Holdover frequency drift.

  • Wander tolerance.

  • Wander generation.

  • Pull-in and hold-in.

  • Reference locking/settling time.

  • Phase transient (tolerance and generation).

A typical example of line timing is when ports on each node of the ring are configured to be primary and secondary timing references for other nodes in the ring. The primary timing reference is then accepted in one direction around the ring, and the secondary timing reference is accepted in the opposite direction.

This is the best practice recommendation when line timing:

Configure primary timing clockwise around the ring, and secondary timing counterclockwise around the ring. Figure 1 shows the timing topology:

Figure 1 – Timing Topology Diagram

15454-timingissues-1.gif

Node 3-1 accepts the external ST1 reference on its BITS 1 pins in this timing topology diagram. Node 3-1 is the master from which the other nodes that use line timing accept their primary and secondary timing reference in a clockwise or counterclockwise direction.

However, another equally valid timing topology is to have each of these four nodes timed from separate ST1 primary timing references on their backplane BITS pins.

Note: Daisy-chaining of BITS sources is not supported. This means that one BITS source is not used to time multiple nodes from the same wire-wrap pins. Each timing line card from timing sources typically provides pin outputs to one set of timing inputs. Use a protect card or another timing line card for the other BITS input for redundancy and timing protection.

The ONS 15454 sends back a "Don't Use for Synchronization (DUS)" SSM message on that optical card in the opposite direction if it performs line timing and accepts its primary timing reference from one of its optical cards. Figure 2 represents this scenario (this is not automatic; it must be selected on the line card). This is the preferred method of timing with the use of the DUS, as checked on the line card.

Figure 2 – Timing With the Use of the DUS

15454-timingissues-2.gif

The do not use (DUS) Source Specific Multicast. (SSM) message displays because the same timing reference circles from one ONS 15454 to the other ONS 15454 if the ONS 15454 accepted its primary timing reference from the same interface from which its opposite node accepted the primary timing reference. This results in a timing loop. Node 3-1 does not receive a DUS SSM message because it externally times from its backplane BITS pins, as opposed to line timing.

Node 3-1 must show a DUS SSM message on slot 5, port 1, because that interface provides line timing for adjacent Node 4-1. However, it does not report the DUS SSM message because slot 5 is not one of the primary, secondary, or third timing references on Node 3-1 (Node 3-1 timing references are BITS 1, BITS 2, and internal). Similarly, the DUS SSM messages disappear if you remove the secondary timing references (slot 5) from Nodes 4-1 and 4-2.

The DUS SSM messages are logged in the active alarms window within the Cisco Transport Controller. The logged DUS SSM messages allow you to verify the timing topology. You can use them to check the direction from which each ONS 15454 accepts its timing.

The clockwise primary timing reference and counter-clockwise secondary timing reference timing topology appears logical and simple to comprehend. However, take Node 4-2 to check what happens if you turn one of the three-line timed nodes primary timing reference around to be accepted from the opposite direction. Instruct the node that its primary timing reference must now be accepted from Node 3-2, as opposed to its current Node 4-1, as Figure 3 shows:

Figure 3 – Primary Timing Reference Accepted From Node 3-2

15454-timingissues-3.gif

Each of the three-line timed nodes are able to accept a primary and secondary timing reference in Figure 3. This timing topology is not as easy to understand. IThere also seems to be a timing loop between Node 4-1 and Node 4-2. Check whether there is be a timing loop if Node 4-1 lost its primary timing reference and had to use its secondary timing reference. This is where the importance of the DUS SSM message comes in, as Figure 4 shows:

Figure 4 – The Importance of the DUS SSM Message

15454-timingissues-4.gif

A DUS SSM message is sent to tell Node 4-2 not to use this interface for a timing reference instead of a timing loop due to Node 4-1 that uses its secondary timing reference. Node 4-2 is forced to accept the default timing reference of its internal ST3E clock if it loses its primary timing reference, as opposed to its secondary timing reference from Node 4-1, as Figure 5 shows:

Figure 5 – Default Timing Reference of the Internal ST3E Clock

15454-timingissues-5.gif

The question now arises about why this complex timing topology must be used when a more easily understandable clockwise primary timing reference and anti-clockwise secondary timing reference topology can be used. To answer this question, expand this network into a larger topology, as Figure 6 shows:

Figure 6 – When the Network is Expanded into a Larger Topology

15454-timingissues-6.gif

Each node accepts its primary timing reference in a clockwise direction and its secondary timing reference in an anti-clockwise direction in this eight-node bidirectional line switch ring (BLSR) optical carrier (OC) (BLSR OC 48) ring.

The problem with this timing topology is that by the time the primary timing reference is accepted by Node 8, it has been regenerated six times. Timing problems such as slips can occur in large networks when the primary timing reference has to be line timed round the entire ring.

A solution is to ensure that the primary timing reference is accepted in both directions around the ring. This means that the primary timing reference only has to travel halfway around the ring, as Figure 7 shows:

Figure 7 – The Primary Timing Reference Travels Halfway Around the Ring

15454-timingissues-7.gif

The primary timing reference only needs to be accepted halfway around the ring in Figure 7. You can also see that if any of the links between the nodes are broken, they can still accept a secondary timing reference.

A complete discussion on timing is beyond the scope of this document. However, this section provides a basic explanation of the concepts behind timing on the ONS 15454.

Synchronization Modes

The ONS 15454 operates in one of these synchronization modes based on the network condition:

  • Normal Mode: The system clock is synchronized to a reference source. The output frequency of the clock is the same as the input reference frequency over the long term. The SYNC LED on the TCC/TCC+/TCC2 and XC/XCVT/XC10G card indicates Normal Mode.

  • Fast Start Mode: Fast Start is used for fast 'pull-in' of a reference clock and is active when the internal reference frequency is offset from the external reference clock. The secondary reference source is selected if the frequency is offset by more than 2 ppm (parts per million) every 30 seconds (called the "wander threshold"). The node reverts back to the primary reference source when it is within the specified threshold (for example, +/- 15 ppm). The internal clock in Fast Start mode during the switching process. Fast Start is sometimes referred to as the "Acquire State".

  • Holdover Mode: The ONS 15454 goes into holdover when the last available reference is lost and the node is synchronized to that reference for more than 140 seconds. The internal clock is held at the last known value of the phase lock loop (PLL) parameters when the node is still synchronized to the reference clock during this period. The ONS 15454 switches to free-run mode if the holdover frequency value is corrupted.

  • Free-Run Mode: The ONS 15454 is considered to be in rree-run mode when it operates on its own internal clock. Free-run accuracy for the ONS 15454 and most SONET nodes is ST3. The minimum accuracy for any SONET node must be better than SONET Minimum Clock (SMC), which is +/- 20 ppm.

Use the BITS Backplane Pins for External Timing References

The ONS 15454 backplane supports two BITS clock pin fields. The first four BITS pins (rows 3 and 4) support output and input from the first external timing device. The last four BITS pins (rows 1 and 2) perform the identical functions for the second external timing device. See Table 2 for the pin assignments for the BITS timing pin fields.

Table 2 – Pin Assignments for the BITS Timing Pin Fields

External Device Contact Tip and Ring Function
First External Device A3 (BITS 1 Out) Primary ring (-) Output to external device
B3 (BITS 1 Out) Primary tip (+) Output to external device
A4 (BITS 1 In) Secondary ring (-) Input from external device
B4 (BITS 1 In) Secondary tip (+) Input from external device
Second External Device A1 (BITS 2 Out) Primary ring (-) Output to external device
B1 (BITS 2 Out) Primary tip (+) Output to external device
A2 (BITS 2 In) Secondary ring (-) Input from external device
B2 (BITS 2 In) Secondary tip (+) Input from external device

Figure 8 – BITS In and Out

15454-timingissues-8.gif

Figure 9 – The 15454 Backplane

15454-timingissues-9.gif

Understand the Timing Alarms

The ONS 15454 uses an active alarm window that displays various alarms to provide a summary of the current status. All timing conditions appear in blue to indicate that they must be treated as non-critical event notification messages or conditions although they appear as alarms in older operating software.

New timing event notification alarms appear in blue, and the old timing event notification alarms expire and turn to white when a timing event notification (such as a change in the timing topology) displays. They are then removed when the alarm display window is refreshed.

This sections shows a summary of the timing event notification type codes.

Figure 10 – The BITS-1 Type Code

15454-timingissues-10.gif

The BITS 1 type code indicates that the BITS 1 interface on the ONS 15454 generates the timing event notification.

Figure 11 – The BITS-2 Type Code

15454-timingissues-11.gif

The BITS 2 type code indicates that the BITS 2 interface on the ONS 15454 generates the timing event notification.

Figure 12 – The SYNC-NE Type Code

15454-timingissues-12.gif

The SYNC-NE type code indicates that the synchronization on the TCC card generates the timing event notification for the ONS 15454.

Figure 13 – The FAC-6-X-Y Type Code

15454-timingissues-13.gif

The FAC-6-X-Y type code indicates that the facility on slot X, port Y generates the timing event notification for the ONS 15454.

Figure 14 – The SYNC-BITS 1 Type Code

15454-timingissues-14.gif

The SYNC-BITS 1 type code indicates that the synchronization on the TCC card generates the timing event notification for the BITS 1 interface.

Figure 15 – The SYNC-BITS 2 Type Code

15454-timingissues-15.gif

The SYNC-BITS 2 type code indicates that the synchronization on the TCC card generates the timing event notification for the BITS 2 interface.

Best Practice Timing Topology Lab Setup

This lab setup demonstrates a typical timing setup for the ONS 15454. This setup is based on a lab setup that consists of four ONS 15454 nodes in a BLSR OC 48 ring. This lab setup shows:

  • How one node accepts a BITS 1 external timing reference

  • How the node that acts as the master uses line timing for the other nodes in the ring to accept their clockwise primary timing reference and synchronize themselves from it

  • That the ring is deliberately broken

This lab setup indicates how the nodes accept the anti-clockwise secondary timing reference to recover and resynchronize their timing. The ring is then repaired and the nodes resynchronize their timing back to accept the clockwise primary timing reference.

See Figure 16 for the network topology used in the lab setup:

Figure 16 – Best Practice Timing Topology Lab Setup

15454-timingissues-16.gif

The same timing topology displays through the Cisco Transport Controller network view in Figure 17. All nodes that are line timing are synchronized to accept their clockwise primary timing reference.

Figure 17 – The CTC Network View

15454-timingissues-17.gif

Externally Timing the First Node

The first node to be configured for timing is Node 3-1. Use the Cisco Transport Controller interface to navigate to the timing window through the Provisioning > Timing tabs. The ONS 15454 can accept its timing either from the line (one of the optical cards) or from an external BITS 1 source. Specify external for the timing mode. Node 3-1 is instructed to use the BITS backplane pins to accept its primary timing source when you specify external.

The timing configuration example in Figure 18 shows that the NE Reference REF 1 field is set to BITS 1. This instructs Node 3-1 to use the BITS 1 IN backplane pins to accept its primary timing reference. The BITS 1 STATE field is placed In Service (IS) to enable the BITS 1 pins.

Node 3-1 uses its BITS 1 IN backplane pins as its primary timing reference source when Node 3-1 initializes. Node 3-1 accepts its secondary timing reference from the internal ST3 clock that runs on the TCC card if it cannot use the BITS 1 backplane pins. It accepts its third timing reference (the internal ST3 clock) if this action fails.

Node 3-1 switches to accept its synchronization from it if Node 3-1 initializes to accept the secondary or third timing source but at a later time its primary timing source becomes available. This is because the revertive option is selected in the configuration window. A reversion time of five minutes is set, which is the time that Node 3-1 waits for its primary timing reference to become available to switch to accept it.

All of the interface cards in Node 3-1 are timed by an ST1 clock if Node 3-1 uses the BITS 1 pins as its primary timing reference source. Otherwise, Node 3-1 uses its secondary or third timing reference, and all of the interface cards are timed by an ST3 clock.

Refer to the Setting Up ONS 15454 Timing section of the ONS 15454 User Documentation for a complete description of the options available from the timing configuration window.

Figure 18 – Configuration Example Where the First Node is Externally Timed

15454-timingissues-18.gif

Alarms for Externally Timing the First Node

Three alarms are generated when you configure Node 3-1 to accept an external BITS 1 timing reference, as Figure 19 shows. Go to the alarms window through the Alarms tab to view these alarms through the Cisco Transport Controller interface. The alarms indicate that Node 3-1:

  • Detected an ST1 Traceable PRS

  • Successfully switched to the ST1 Traceable PRS

  • Is incoming on the BITS 1 backplane pins

Note: The severity of the alarms is all Not Reported (NR) or Not Alarmed (NA). This indicates that the alarms are informational only.

Figure 19 – Three Alarms Generated When the First Node is Externally Timed

15454-timingissues-19.gif

Line Timing the Second Node

The next node configured for timing is Node 3-2. Use the Cisco Transport Controller interface to navigate to the timing window through the Provisioning > Timing tabs. Specify line for the timing mode. Node 3-2 is instructed to look at the optical card in slot 6 to accept its primary timing reference and slot 5 to accept its secondary timing reference when you specify line.

The NE Reference REF 1 field has been set to slot 6, port 1 in the timing configuration window. This is where you instruct Node 3-2 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 6 to find its primary timing source.

The NE Reference REF 2 field has been set to slot 5, port 1. This is where you instruct Node 3-2 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 5 to find its secondary timing source.

Node 3-2 uses the OC 48 card in slot 6 to accept its primary timing reference when Node 3-2 initializes. Node 3-2 accepts its secondary timing reference from the OC 48 card in slot 5 if it cannot use this OC 48 card. Node 3-2 accepts its third timing reference from the internal ST3 clock that runs on the TCC card if both primary and secondary timing sources cannot be accepted.

If Node 3-2 accepts its secondary or third timing source to initialize, but at a later time the primary timing reference becomes available, Node 3-2 switches to accept it. This is because the revertive option is selected in the configuration window. A reversion time of five minutes is set, which is the time that Node 3-2 waits from its primary timing reference to become available to switch to accept it.

All interface cards in Node 3-2 are timed by the OC 48 card in slot 6 if Node 3-2 accepts its primary timing reference. All interface cards are timed by the OC 48 card in slot 5 if Node 3-2 accepts its secondary timing reference. Otherwise, all interface cards are timed by the internal ST3 clock.

Refer to the Setting Up ONS 15454 Timing section of the ONS 15454 User Documentation for a complete description of the options available from the timing configuration window.

Figure 20 – Configuration Example When Second Node is Line Timed

15454-timingissues-20.gif

Alarms for Line Timing the Second Node

Rour alarms are generated when you configure Node 3-2 for line timing, as Figure 21 shows. Go to the alarms window through the Alarms tab to view these alarms from the Cisco Transport Controller interface. From the alarms, it can be inferred that:

  • Node 3-2 successfully switched to a ST1 Traceable PRS.

  • ST1 Traceable PRS available on slot 6, port 1.

  • Node 3-2 has detected a ST1 Traceable PRS.

  • ST1 Traceable PRS is available on slot 5, port 1.

Note: The severity of the alarms is all NR or NA. This indicates that the alarms are informational only.

Figure 21 – Alarms Generated When the Second Node is Line Timed

15454-timingissues-21.gif

Line Timing the Third Node

The next node configured for timing is Node 4-1. Use the Cisco Transport Contoller interface to navigate to the timing window through the Provisioning > Timing tabs. Specify line for the timing mode. Node 4-1 is instructed to look at the Optical Card in slot 6 to accept its primary timing reference, and slot 5 to accept its secondary timing reference when you specify line.

The NE Reference REF 1 field is set to slot 6, port 1 in the timing configuration window. This is where you instruct Node 4-1 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 6 to find its primary timing source.

The NE Reference REF 2 field is set to slot 5, port 1. This is where you instruct Node 4-1 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 5 to find its secondary timing source.

Node 4-1 uses the OC 48 card in slot 6 to accept its primary timing reference when Node 4-1 initializes. Node 4-1 accepts its secondary timing reference from the OC 48 card in slot 5 if it cannot use this OC 48 card. Node 4-1 accepts its third timing reference from the internal Stratum 3 clock running on the TCC card if both primary and secondary timing sources cannot be accepted.

If Node 4-1 initializes accepting its secondary or third timing source, but at a later time the primary timing reference becomes available, Node 4-1 switches to accept it. This is because the revertive option is selected in the configuration window. A reversion time of five minutes is set, which is the time that Node 4-1 waits for its primary timing reference to become available to switch to accept it.

All interface cards in Node 4-1 are timed by the OC 48 card in slot 6 if Node 4-1 accepts its primary timing reference. All interface cards are timed by OC 48 card in slot 5 if Node 4-1 accepts its secondary timing reference. Otherwise, all interface cards are timed by the internal ST3 clock.

Refer to the Setting Up ONS 15454 Timing section of the ONS 15454 User Documentation for a complete description of the options available from the timing configuration window.

Figure 22 – Configuration Example When Third Node is Line Timed

15454-timingissues-22.gif

Alarms for Line Timing the Third Node

The same alarms are reported for Node 4-1 as for Node 3-2 with the exception of the DUS SSM message. This alarm is important because it allows you to recognize the timing topology within your network. If an ONS 15454 is line timing and uses a particular incoming circuit on an optical card as its primary timing reference, it will send a DUS SSM message back down that interface in order to prevent timing loops.

Note: This might not happen. The DUS SSM message is only sent when you have checked that feature under the Provisioning tab of the line card. You must do this in order to send the DUS SSM message.

See the Timing Topology Changes When the Ring Is Broken section of this document for more information.

Figure 23 – Alarms Generated When the Third Node is Line Timed

15454-timingissues-23.gif

Line Timing and Providing a BITS OUT Timing Reference on the Fourth Node

The last node configured is Node 4-2. Use the Cisco Transport Controller interface to navigate to the timing window through the Provisioning > Timing tabs. Specify line for the timing mode. Node 4-2 is instructed to look at the optical card in slot 6 to accept its primary timing reference, and the optical card in slot 5 to accept its secondary timing reference when you specify line.

The BITS OUT pins and the ONS 15454 itself have separate fields where you specify the timing references that you want to use. These fields are explained here:

  • The BITS 1 OUT, REF 1 field is set to slot 6, port 1. This instructs Node 4-2 to accept the 125 microsecond SONET packets on the OC 48 optical card in slot 6 as its primary timing reference for the BITS 1 OUT pins on the backplane.

  • The BITS 1 OUT, REF 2 field is set to slot 5, port 1. Again, this instructs Node 4-2 to accept the 125 microsecond SONET packets on the OC 48 optical card in slot 5 as its secondary timing reference for the BITS 1 OUT pins on the backplane.

  • The BITS 1 STATE field is placed IS to enable the BITS 1 pins.

  • The NE Reference REF 1 field is set to slot 6, port 1. This is where you instruct Node 4-1 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 6 to find its primary timing source.

  • The NE Reference REF 2 field is set to slot 5, port 1. This is where you instruct Node 4-1 to look at the 125 microsecond SONET packets on the OC 48 optical card in slot 5 to find its secondary timing source.

Node 4-2 uses the OC 48 card in slot 6 to accept its primary timing reference when Node 4-2 initializes. Node 4-2 accepts its secondary timing reference from the OC 48 card in slot 5 if it cannot use this OC 48 card. Node 4-2 accepts its third timing reference from the internal ST3 clock that runs on the TCC card if both primary and secondary timing sources cannot be accepted.

If Node 4-2 initializes accepting its secondary or third timing source, but at a later time the primary timing reference becomes available, Node 4-2 switches to accept it. This is because the revertive option is selected in the configuration window. A reversion time of five minutes is set, which is the time that Node 4-2 waits for its primary timing reference to become available to switch to accept it.

All interface cards in Node 4-2 are timed by the OC 48 card in slot 6 if Node 4-2 accepts its primary timing reference. All interface cards are timed by OC-48 card in slot 5 if Node 4-2 accepts its secondary timing reference. Otherwise, all interface cards are timed by the internal ST3 clock.

Refer to the Setting Up ONS 15454 Timing section of the ONS 15454 User Documentation for a complete description of the options available from the timing configuration window.

Figure 24 – Example for the Fourth Node When it is Line Timed and Provided a BITS Out Reference

15454-timingissues-24.gif

Alarms for Line Timing and Providing a BITS OUT Timing Reference On the Fourth Node

A DUS SSM message displays again as the next hop Node 3-2 is line timing and uses interface slot 5, port 1 as a primary timing reference for Node 4-2. An ONS 15454 sends a DUS SSM message back down that interface in order to prevent timing loops if an ONS 15454 uses a particular optical card as a timing reference. See the Timing Topology Changes When the Ring Is Broken section of this document for more information.

A oss of signal (LOS) alarm for the BITS 1 backplane pins also dispays. This is because there is no equipment physically wire wrapped to those pins although the BITS 1 backplane pins have been put into service. There is no incoming signal on the BITS 1 IN backplane pins.

Figure 25 – Alarms Generated for the Fourth Node

15454-timingissues-25.gif

The four node ONS 15454 lab setup is now complete. There are four nodes configured in an OC 48 BLSR ring topology. Node 3-1 acts as the master and supplies the ST1 timing reference through its incoming BITS 1 IN backplane pins.

The other three nodes in the ring are each line timing from Node 3-1. Node 4-2 also supplies an ST1 timing reference through its BITS 1 OUT backplane pins.

This is a simple timing topology with the primary timing reference accepted clockwise around the ring, and the secondary timing reference accepted anti-clockwise around the ring.

Timing Topology Changes when the Ring Is Broken

The ring is stable with the PRS accepted clockwise around the ring in the lab setup, as Figure 26 shows:

Figure 26 – PRS Accepted Clockwise Around the Ring

15454-timingissues-26.gif

The ring is now deliberately broken. Disconnect the OC 48 link between Node 4-1 and Node 4-2 to do this. The next section uses alarm window to explain how the ring recovers.

Figure 27 shows what the resynchronized timing topology of the ring looks like after the link between Node 4-1 and Node 4-2 is broken.

Figure 27 – Topology When Link Between Node 4-1 and Node 4-2 is Broken

15454-timingissues-27.gif

Node 3-1 still accepts the ST1 primary timing reference through the BIT 1 pins on its backplane. This is because Node 3-1 externally times, and does not line time. Node 3-1 is unaffected by the break in the ring.

Node 4-1 is upstream from the fiber break, and can still accept the clockwise primary timing reference.

Node 4-2 is downstream from the fiber break and has been forced to switch to accept the anti-clockwise secondary timing reference.

Node 3-2 is also downstream from the fiber break and has also been forced to accept the anti-clockwise secondary timing reference.

Use the Alarm Screens to Explain Timing Topology Changes

You must look at the network level Cisco Transport Controller view of the changed timing topology before you attempt to understand the timing changes on the individual nodes after the ring is broken.

Figure 28 – Modified Timing Topology

15454-timingissues-28.gif

Now look at the individual nodes in turn.

Timing Topology Changes for the First Node

Each ONS 15454 has three timing sources, primary, secondary, and third. Node 3-1 is configured for external timing and accepts its timing references from these:

  • Primary —The BITS 1 pins on the ONS 15454 backplane

  • Secondary —The internal ST3 clock on the TCC card

  • Third —The internal ST3 clock on the TCC card

Node 3-1 is unaffected by the break in the ring as its primary timing reference source is connected directly onto its BITS 1 IN backplane pins with this configuration. Node 3-1 remains unchanged, as Figure 29 shows:

Figure 29 – Alarm Screen Indicating that Node 3-1 is Unchanged

15454-timingissues-29.gif

Timing Topology Changes for the Second Node

Node 3-2 is configured for line timing, and accepts its timing references from these:

  • Primary —The slot 6, port 1 Optical Carrier OC 48 line card.

  • Secondary —The slot 5, port 1 Optical Carrier OC 48 line card.

  • Third —The internal ST3 clock on the TCC card.

Node 3-2 is affected by a break in the ring with this configuration. This is because it accepts its timing from the primary timing reference source coming clockwise around the ring across a break that is introduced into the ring.

Node 3-2 detects a loss of its primary timing source and switches to its secondary timing source.

Figure 30 – Node 3-2 Detects a Loss of its Primary Timing Source

15454-timingissues-30.gif

Timing Topology Changes for the Third Node

Node 4-1 is configured for line timing and accepts its timing references from these:

  • Primary —The slot 6, port 1 Optical Carrier OC 48 line card.

  • Secondary —The slot 5, port 1 Optical Carrier OC 48 line card.

  • Third —The internal ST3 clock on the TCC card.

Node 4-1 is affected by the break in the ring with this configuration. This is because it accepts its timing from the primary timing reference that comes clockwise around the ring from Node 3-2 before the break that is introduced into the ring. However, alarms are reported for the break in the ring.

Figure 31 – Alarms Reported for the Break in the Ring

15454-timingissues-31.gif

Timing Topology Changes for the Fourth Node

Node 4-2 is configured for line timing and accepts its timing references from these:

  • Primary— The slot 6, port 1 OC 48 line card.

  • Secondary— The slot 5, port 1 OC 48 line card.

  • Third— The internal ST3 clock on the TCC card.

Node 4-2 is affected by the break in the ring and switches to its secondary timing source with this configuration. This is because it accepts its timing from the primary timing reference that comes clockwise around the ring from Node 3-2 across the break that is introduced into the ring. Alarms are also reported for the break in the ring.

Figure 32 – Node 4-2 Affected by the Break in the Ring

15454-timingissues-32.gif

Timing Topology Recovery (Reversion)

Each node has the revertive option selected in the Cisco Transport Controller timing configuration window in the lab setup. The node is instructed that if it loses its primary timing reference and has to switch, it must accept either the secondary or third timing reference when you select this option. It can switch back to accept it if it later recovers its primary timing reference.

Each node also had its reversion timer set to five minutes. The reversion timer specifies how long after a node regains its primary timing reference it waits before it switches back to accept it.

The fiber break in the lab setup is now repaired. The nodes recognize that the break is repaired but do not change their timing topologies until after the reversion timers have expired. The reversion timers expire after five minutes, and the timing topology reverts to its original state with each node accepting the ST1 primary timing reference that comes clockwise round the ring from the BITS 1 pins on Node 3-2.

Figure 33 shows the Cisco Transport Controller network view of the timing topology three minutes after the fiber break has been repaired. The nodes have detected that the fiber break has been repaired but still have two minutes to wait before their reversion timers expire.

Figure 33 – Cisco Transport Controller Network View of the Timing Topology 3 Minutes After the Fiber Break is Repaired

15454-timingissues-33.gif

These messages are sorted by node. All the minor (MN), major (MJ), and critical (CR) alarms caused by the fiber break between Node 4-1 and Node 4-2 are now white. This indicates that Node 4-1 and Node 4-2 have detected that the fiber break has been repaired.

The DUS SSM message on Node 4-1 is also white. This is because Node 4-2 accepts its secondary timing reference from Node 3-2 and sends DUS back to Node 3-2. Node 4-2 does not switch back to accept it until the reversion timer has expired although Node 4-2 now has a valid primary timing reference incoming from Node 4-1 over the repaired fiber link.

Normally, an ONS 15454 only sends back DUS on the interface on which it accepts its timing.

Figure 34 shows the window just after the five-minute reversion timer has expired.

Figure 34 – The 5-Minute Reversion Timer has Expired

15454-timingissues-34.gif

These messages are sorted by node. These are the messages for each node in turn:

  • Node 3-1— Remains unchanged because it accepts its primary timing reference from its BITS 1 pins it is unaffected by the timing topology changes.

  • Node 3-2 — Lost its primary timing reference source when the fiber break occurs. This is because it is downstream from the clockwise primary timing reference from Node 3-1. It has to switch to accept its secondary timing reference that comes anti-clockwise from Node 3-1. Node 4-2 also has to change to its secondary timing reference because it is also downstream of the fiber break. Node 4-2 accepts its secondary timing reference provided anti-clockwise from Node 3-2.

    The first alarm that is white for Node 3-2 is DUS. This is because Node 4-2 has switched to use its clockwise primary timing reference and no longer uses the anti-clockwise secondary timing reference from Node 3-2. Normally, an ONS 15454 only sends back DUS on the interface from which it accepts its timing.

    The second alarm that is white for Node 3-2 is Switch To Secondary (SWTOSEC). This is because Node 3-2 has now detected and switched back to use its primary timing reference.

  • Node 4-1— The only timing alarm that is white for Node 4-1 is PRS for FAC 5-1 (Facility). This is because Node 4-2 now uses the primary timing reference that comes clockwise around the ring that Node 4-1 sends. Because it accepts this timing reference it sends a DUS is back. Hence, Node 4-1 can no longer use this interface as a timing reference. Normally, an ONS 15454 only sends back DUS on the interface from which it accepts its timing.

  • Node 4-2— The first two timing alarms (SWTOSEC and PRS) that are white are issued when it switches to accept its secondary timing source from Node 3-2. These alarms are now white because Node 4-2 has now switched back to accept its primary timing reference.

    The third timing alarm (SWTOSEC) that is white is from the BITS 1 interface on Node 4-2, to state that it has switched to its secondary timing reference. This message is now white because the BITS 1 interface on Node 4-2 has now also switched back to its primary timing source.

    The last two timing alarms (SYNCPRI) that are white come from Node 4-2 itself and the BITS 1 interface. This indicates that they both lost their primary timing reference. These messages are now white because the primary timing reference has now been restored.

    Figure 35 shows the final active alarm window after all the alarms are cleared.

Figure 35 – The Final Active Alarm Window

15454-timingissues-35.gif

The timing topology has reverted to its original configuration, wherein each node accepts the primary timing reference clockwise around the ring.

Figure 36 – Each Node Accepts the Primary Timing Reference Clockwise Around the Ring

15454-timingissues-36.gif

Timing Alarms/Conditions and Troubleshooting Timing (software level dependent)

This section describes timing alarms and conditions. It also provides tips and procedures to troubleshoot or solve them.

FRNGSYNC

Free-run synchronization (FRNGSYNC) is a major service-affecting error.

The reporting ONS 15454 is in free-run synchronization mode. External timing sources are disabled and the node uses its internal clock, or the ONS 15454 has lost its designated BITS timing source.

Complete these steps to clear the FRNGSYNC:

  1. Disregard this alarm if the ONS 15454 is configured to operate from its own internal clock.

  2. Verify that the BITS timing source is valid if the ONS 15454 is configured to operate off an external timing source. Common problems with a BITS timing source include reversed wiring and bad timing cards.

FSTSYNC

Fast Start Synchronization (FSTSYNC) is a minor, non-service-affecting alarm.

FSTSYNC mode means the ONS 15454 chooses a new timing reference. The previous timing reference has failed. This informational alarm disappears after approximately 30 seconds.

HLDOVERSYNC

Holdover Synchronization (HLDOVERSYNC) is a major service-affecting alarm.

Loss of the primary or secondary timing reference raises the HLDOVERSYNC alarm. Timing reference loss occurs when line coding on the timing input is different than the configuration on the ONS 15454. It also usually occurs during the selection of a new node reference clock. This alarm indicates that the ONS 15454 has gone into holdover and uses the ONS 15454 internal reference clock, which is a ST3-level timing device. The alarm clears when primary or secondary timing is re-established.

Complete these steps to clear the HLDOVERSYNC:

  1. Check for additional alarms that relate to timing.

  2. Re-establish a primary and secondary timing source according to local site practice.

LOF (TCC+)

Loss of Frame (LOF) (TCC+) is a major service-affecting alarm.

A port on the TCC+ BITS input detects a LOF on the incoming BITS timing reference signal. LOF indicates that the receiving ONS 15454 has lost frame delineation in the incoming data.

Note: The procedure assumes that the BITS timing reference signal functions properly. It also assumes the alarm does not appear during node turn-up.

Complete these steps to clear the LOF on the TCC+:

  1. Verify that the line framing and line coding match between the BITS input and the TCC+.

  2. Note the slot and port that reports the alarm in Cisco Transport Controller.

  3. Find the coding and framing formats of the external BITS timing source. This is in the user documentation for the external BITS timing source or on the timing source itself.

  4. Click the Provisioning > Timing tabs to display the General Timing window.

  5. Verify that Coding matches the coding of the BITS timing source (either B8ZS or AMI).

  6. Click Coding to reveal a menu if the coding does not match. Choose the appropriate coding. Refer to these sections for more information:

    • Page 36 of the Cisco ONS 15454 Troubleshooting and Reference Guide

    • Page 78 of 12576-01 June 2001 Alarm Troubleshooting for PalmOS

  7. Verify that Framing matches the framing of the BITS timing source (either ESF or SF [D4]).

  8. Click Framing to reveal the menu if the framing does not match. Choose the appropriate framing.

    Note: The B8ZS coding field is normally paired with ESF in the Framing field on the timing subtab, and the AMI coding field is normally paired with SF (D4) in the Framing field.

  9. Replace the TCC+ card if the alarm does not clear when the line framing and line coding match between the BITS input and the TCC+.

    Note: You do not need to make any changes to the database when you replace a card with an identical type of card.

STU

Synchronization Traceability Unknown (STU) is not alarmed.

The STU alarm occurs when the reporting node is timed to a reference that does not support synchronous status messaging (SSM). SSM is a SONET protocol that communicates information about the quality of the timing source. SSM messages are carried on the S1 byte of the SONET line layer. SSM enables SONET devices to automatically choose the highest quality timing reference and to avoid timing loops. The ONS 15454 supports SSM. This alarm indicates that the reporting node has SSM enabled but the timing source does not support SSM, or the reporting node does not have SSM enabled but the timing source supports SSM.

Complete these steps to clear the STU:

  1. Select the Provisioning > Timing tabs.

  2. Clear the selection if Sync Messaging is checked. Check the box if Sync Messaging is not selected.

  3. Click Apply.

SWTOPRI

Switched to Primary (SWTOPRI ) is not alarmed.

The ONS 15454 has switched to the primary timing source (reference 1). The ONS 15454 uses three ranked timing references. The timing references are typically two BITS-level or line-level sources and an internal reference.

Note: This is a condition and not an alarm. It is for information only and does not require you to troubleshoot.

SWTOSEC

Switched to Secondary (SWTOSEC) is not alarmed. Refer to these sections for more information:

  • Page 56 of the Cisco ONS 15454 Troubleshooting and Reference Guide

  • Page 78 of 12576-01 June 2001 Alarm Troubleshooting for PalmOS

The ONS 15454 has switched to the secondary timing source (reference 2). The ONS 15454 uses three ranked timing references. The timing references are typically two BITS-level or line-level sources and an internal reference.

Look up and troubleshoot alarms related to failures of the primary source, such as the SYNCPRI alarm to clear the SWTOSEC.

SWTOTHIRD

Switched to Third (SWTOTHIRD) is not alarmed.

The ONS 15454 has switched to the third timing source (reference 3). The ONS 15454 uses three ranked timing references. The timing references are typically two BITS-level or line-level sources and an internal reference.

Look up and troubleshoot alarms related to failures of the primary and secondary reference source, such as the SYNCPRI and SYNCSEC alarms to clear the SWTOTHIRD.

SYNCPRI

Loss of Timing on Primary Reference (SYNCPRI) is a minor, non-service-affecting alarm.

A SYNCPRI alarm occurs when the ONS 15454 loses the primary timing source (reference 1). The ONS 15454 uses three ranking timing references. The timing references are typically two BITS-level or line-level sources and an internal reference. The ONS 15454 should switch to its secondary timing source (reference 2) if SYNCPRI occurs. This switch also triggers the SWTOSEC alarm.

Complete these steps to clear the SYNCPRI on the TCC+ Card:

  1. Select the Provisioning > Timing tabs from the card view for the reporting TCC+ card.

  2. Check the current configuration for the REF-1 of the NE Reference.

  3. Follow the procedure in the "LOS (OC-N)" section on page 41 if the primary reference is a BITS input.

  4. Check the primary reference clock if the primary reference clock is an incoming port on the ONS 15454.

SYNCSEC

SYNCSEC is a minor, non-service-affecting alarm.

Refer to these sections for more information:

  • Page 57 of the Cisco ONS 15454 Troubleshooting and Reference Guide

  • Alarm Troubleshooting for Palm OS 78-12576-01 June 2001

A Loss of Timing on Secondary Reference (SYNCSEC) alarm occurs when the ONS 15454 loses the secondary timing source (reference 2). The ONS 15454 uses three ranked timing references. The timing references are typically two BITS-level or line-level sources and an internal reference. If SYNCSEC occurs, the ONS 15454 must switch to the third timing source (reference 3) to obtain valid timing for the ONS 15454. This switch also triggers the SWTOTHIRD alarm.

Complete these steps to clear the SYNCSEC on the TCC+ Card:

  1. Select the Provisioning > Timing tabs from the card view for the reporting TCC+ card.

  2. Check the current configuration of the REF-2 for the NE Reference.

  3. Follow the procedure in the "LOS (OC-N)" section on page 41 if the secondary reference is a BITS input.

  4. Check the secondary timing source if the secondary timing source is an incoming port on the ONS 15454.

SYNCTHIRD

SYNCTHIRD is a minor, non-service-affecting alarm.

A Loss of Timing on Third Reference (SYNCTHIRD) alarm occurs when the ONS 15454 loses the third timing source (reference 3). The ONS 15454 uses three ranking timing references. The timing references are typically two BITS-level or line-level sources and an internal reference. If SYNCTHIRD occurs and the ONS 15454 uses an internal reference for source three, then the TCC+ card might have failed. The ONS 15454 often reports either FRNGSYNC or HLDOVERSYNC after SYNCTHIRD.

Complete these steps to clear the SYNCTHIRD on the TCC+ Card:

  1. Select the Provisioning > Timing tabs from the card view for the reporting TCC+ card.

  2. Check the current configuration of the REF-3 for the NE Reference.

  3. Follow the procedure in the "LOS (OC-N)" section on page 41 if the third timing source is a BITS input.

  4. Check the timing source if the third timing source is an incoming port on the ONS 15454.

  5. Perform a software reset on the TCC+ card if the third timing source uses the internal ONS 15454 timing:

    1. Display the Cisco Transport Controller node view.

    2. Position the cursor over the slot that reports the alarm.

    3. Right-click and select RESET CARD.

  6. Physically reset the TCC+ card if this action fails to clear the alarm.

  7. Replace the TCC+ card if the reset fails to clear the alarm.

Refer to this source for more information:

  • Chapter Two of Cisco ONS 15454 Troubleshooting Guide - Release 4.1.x and Release 4.5 (Alarm Troubleshooting)

Note: You do not need to make any changes to the database when you replace a card with an identical type of card.

Timing Wallchart

Use this PDF wallchart for more information on timing.

Related Information

Updated: Jan 31, 2006
Document ID: 15234