Cisco BPX IGX IPX WAN Software

WAN Switching Network Synchronization Fundamentals

Document ID: 10755

Updated: Apr 17, 2009



When you design the synchronization plan for a network of Cisco WAN switches, the fundamental design goals include:

  1. Synchronize the greatest number of network elements to the smallest number of independent clock sources. Ideally, all network elements are synchronized to a single clock source.

  2. Use clock sources of the highest quality (in terms of stability and long-term accuracy). This implies that available clock sources are used with this priority:

    • Clock sources (typically stratum 2) provided by a national or international carrier.

    • The internal (stratum 3) clock source provided by a BPX node.

    • The internal (stratum 4) clock source provided by an IGX node, a PBX, or other customer premises equipment (CPE).

  3. In order to ensure resiliency of the synchronization, plan in the face of possible failures of clock sources, network elements, or network trunks.



There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.


Refer to Cisco Technical Tips Conventions for more information on document conventions.

Cisco WAN Synchronization Methodology, Guidelines, and Definitions

It is the responsibility of the network administrator to identify and define the available network clock sources during the creation of the network synchronization architecture

  1. Network versus Node Clock Sources

    It is critical to note that the network administrator does not explicitly identify which clock source is to be used by each node in an IGX or BPX network. Instead, the network, as a part of switch software, automatically selects the best available clock source and path for each node.

    For MGX nodes, the network can do this as well if the Network Clock Distribution Protocol (NCDP) features are enabled. Otherwise, the administrator is required to manually select the clock sources per node.

    Available clock sources can be any number of these items:

    • Any trunk between nodes that is explicitly clocked (typically by the carrier that provides the facility). The definition of one or both endpoints of such a "clocked" trunk as clock source(s) allows the network to synchronize to the high-quality clock that the carrier provides.

      Note: This includes subrate trunks with X.21, V.35, or RS-449 interfaces on NTM cards in IGX nodes.

    • Any line that connects a piece of customer-premise equipment to the network. This includes any T1, E1, T3, E3, OC3, and OC12 line (provided the line provides a clock, which may not be the case in all networks). Note that this does not include any T1 or E1 Frame relay ports, or any RS232, V.35, X.21, or RS449 data or Frame Relay ports in the network.

    • Any external clock source that is connected to the External Clock input port.

    This table details the required clock signal formats:

    Switch Type Acceptable Clock Signal
    IGX The RS-422A square wave at 1.544 MHz or 2.048 MHz.
    BPX The T1 or E1 bipolar AMI pulse stream. For example, E1 IAW Table 6 of ITU specification G.703

    Note: Table 10 of ITU specification G.703 is not acceptable.

    MGX with PXM-UI The T1 bipolar AMI stream or E1 IAW G.703, Tables 6 and 10 of ITU specification G.703
    MGX with PXM-UI-S3 The T1 bipolar AMI stream or E1 IAW G.703, Table 6 of ITU specification G.703 only—no square wave signals (Table 10).

    In BPX/IGX networks, each clock source is defined as a primary, secondary, or tertiary clock source. The designation of a clock source as primary, secondary, or tertiary is strictly at the discretion of the network administrator. The best available clock sources are typically defined as primary with other clock sources defined as secondary or tertiary. In MGX nodes, the selections are Primary or Secondary. Under NCDP, the stratum level of a clock can be specified and the protocol selects from the available clock sources considering the stratum level.

  2. Automatic Node Clock Selection Algorithm – BPX/IGX Networks

    Once the available clock sources are defined for the network, the system software of the network automatically determines the specific clock source to be used by each node in the network.

    In order to ensure resiliency, the node clock source selection algorithm is re-executed as a result of these items:

    • The addition or deletion of a network clock source by the network administrator.

    • The failure of any defined network clock source.

    • The failure or repair of any node in the network.

    • The failure, repair, or clock configuration of any trunk in the network.

    The algorithm used for node clock selection is very straightforward. Each node uses the nearest, highest-priority (primary, secondary, tertiary, or internal) clock source available to it.

    Therefore, if there is only one primary clock source defined in the network, then all nodes synchronize to it, if possible. If there is more than one primary clock source defined in the network, then each node synchronizes to the nearest (measured by hop count) primary source. (See item 6 for a discussion of the implications when you have nodes synchronize to multiple clock sources.)

    If there are no primary clock sources defined (or all are failed), then each node synchronizes to the nearest secondary clock source.

    If there are no primary or secondary clock sources defined (or all are failed), then each node synchronizes to the nearest tertiary clock source.

    If there are no primary, secondary, or tertiary clock sources defined (or all are failed), then each node synchronizes to the internal (stratum 3) clock source of the BPX node with the highest internal node number.

    If there are no primary, secondary, or tertiary clock sources defined (or all are failed), and there is no BPX node available, then each node synchronizes to the internal (stratum 4) clock source of the IGX node with the highest internal node number.

  3. Pass Synchronization on a Trunk: Yes or No? What Does it Mean?

    In the algorithm in item 2, and for NCDP, a node must be able to indirectly synchronize to a remote clock source. This is accomplished with the identification of a clock path between the remote clock source and the node. Each element (node or trunk) in the path is synchronized to the previous element "upstream" in the path. Thus, a node is frequency-locked to the upstream trunk, which is then frequency-locked to the upstream node, which is then frequency-locked to the next upstream trunk, and so on. This continues until the defined clock source is reached.

    The key to the success of such a scheme is the ability of a node to frequency-lock to its neighbor over the trunk that joins them. This requires that the trunk between the nodes be "unclocked," or able to pass synchronization between nodes.

    A trunk can have a setting of "Pass Sync : Yes" or "Pass Sync: No". Use the cnftrk command to change the parameter. Configure the trunk to not pass synchronization:

    • If the trunk is clocked by the carrier.

      Unfortunately, there is no way for the nodes to determine automatically whether a particular trunk is clocked or not. Similarly, there is no test procedure that can be performed by the network administrator to determine whether the trunk is clocked or not. This information must be provided by the service provider.

    • If, for any reason, the network administrator wishes to prevent the trunk from being included in the clock path between any nodes in the network.

      This is sometimes done for trunks that are prone to frequent outages.

      Note: Subrate trunks by definition cannot pass clocks and are therefore blocked from being configured as Pass Sync. Virtual trunks are physically unable to pass clocking information but are not restricted from being configured as "Pass Sync: Yes." Ensure that you do not configure the network to pass clocking information across virtual trunks.

    A trunk configured as "Pass Sync: Yes" cannot be configured as a network clock source.

    A trunk configured as "Pass Sync: No" is not used in the clock path for any node.

    Note: An IGX node cannot be included anywhere in the clock path of a BPX node. The reason for this is that the clock recovery circuitry and internal oscillator of the IGX is stratum 4 whereas the internal oscillator on the BPX is stratum 3.

  4. How Can I Tell if a Trunk is Clocked or Unclocked?

    The simple answer is that only the service provider who provides the trunk can determine this. The reason is that a particular trunk can be either clocked or unclocked based on what equipment the trunk traverses inside the infrastructure of the service provider.

    Some reasonable rules of thumb are:

    1. A cable is unclocked.

    2. A fractional T1 trunk is usually clocked because it goes through the Digital Access and Crossconnect System (DACS) of a carrier somewhere.

    3. A full T1 is usually not clocked unless it is provided by Sprint. However, some short-haul trunks provided by other carriers may be clocked.

    4. A T3 trunk is rarely clocked because broadband framing structures are specifically designed to support a large number of DS3 data streams. Each is clocked independently with the performance of dynamic bit stuffing.

  5. Loop Clock on a Trunk or a Line: Yes or No? What Does it Mean?

    In the configuration command for each trunk and each line (the cnftrk and cnfln commands, respectively), there is a parameter that allows the network administrator to specify "Loop Clock: Yes" or "Loop Clock: No." This parameter specifies the source of the transmit clock (used to send bits from the node out onto the trunk or line).

    If "Loop Clock: No" (the default) is chosen, then the transmit clock on the trunk or line is derived from the master clock of the node. (This is not necessarily the internal oscillator of the node. If the node is frequency-locked to a remote clock source or the internal oscillator on a remote node, then the master clock of the node is not its internal oscillator.)

    If "Loop Clock: Yes" is chosen, then the transmit clock on the trunk or line is frequency-locked to the receive clock (derived from the incoming bit stream) on the trunk or line. This is commonly done on:

    • A time-division multiplexing (TDM)-based line (such as one that connects to a PBX) when the device at the other end of the line cannot be synchronized to the node. This allows the device to transmit and receive bits at its own frequency (which can be different than the frequency of the node). This prevents the data loss associated with uncontrolled frame slips. In such a case, the line and the attached CPE have no problem using a frequency that is independent of the master clock of the node.

    • A trunk that is clocked by the carrier and the carrier's clock is not used as the clock source for the node. This configuration prevents uncontrolled frame slips (and the corresponding data loss) in the carrier's facilities.

  6. Is it OK to Have Multiple Clock Sources in Use in the Network?

    In some cases, it is unavoidable for some nodes and trunks in a network to synchronize to one clock source and to synchronize other nodes and trunks in the network to another clock source. This is especially common in international networks or in networks in which trunks are obtained from a variety of service providers. Such a network is said to be synchronized in a plesiochronous fashion.

    If two pieces of equipment that are synchronized to different clock sources are joined by an unclocked trunk, input buffers on the interfaces at each node periodically overflow (at one end) or underflow (at the other end). This overflow or underflow condition is commonly known as a frame slip because an overflow condition usually causes one (or more) frame of data to be discarded.

    In a TDM-based network, almost every frame slip causes data to be lost, since there is likely to be data contained in at least one timeslot of every frame.

    On a trunk in a FastPacket or ATM network, many idle packets or cells transmit every second. All IGX and BPX trunk cards discard idle cells from the network before they are buffered and processed. This prevents the error condition of input buffer overflow from occurring.

    Because of the fundamental characteristic of cell-based networking, a network with a plesiochronous synchronization plan can usually operate free of error.

  7. What Errors Do Clock Problems Cause?

    Clocking problems typically cause frame slips on circuit-line interfaces, especially circuit lines to TDM devices such as a PBX. Frame slips can occur on either or both ends of the line. Both the PBX and the switch can record frame slips. In order to help resolve frame slips, configure the external equipment to receive clock from the network. If the external equipment cannot accept network clock, configure the circuit line interface for loop clock. If the configuration of one end of the circuit line for loop clock does not eliminate frame slips, evaluate the clocking architecture of the network and external equipment.

    Clocking problems typically cause Packet Errors, HEC errors, PLCP errors, or frame-sync errors. The error depends upon the type of trunk interface used. Errors result from a difference in frequency between adjacent nodes or Telco clocking of trunks. Clock errors on trunks typically occur on one end. This is because the BPX or IGX trunk-card suppresses the input buffer overflow by deleting idle cells. The errors indicate underflow slips. Configure the end of the trunk that does not experience errors for loop clock to minimize errors.

    A trunk clocked by the carrier can exhibit errors on both ends. Configure either or both ends of the trunk for loop clock to minimize the error condition.

Related Information

Updated: Apr 17, 2009
Document ID: 10755