Cisco Catalyst 8500 Series Multiservice Switch Routers

Document ID: 9205

Contents

Introduction
Prerequisites
      Requirements
      Components Used
      Conventions
When Do You Need to Worry About Network Clocking?
Network Clock System Architecture
Network Clock Redundancy in the 8540-MSR
Handle Route Processor Faults
Handle NetClkMod Faults
Network Clock Distribution Protocol (NCDP)
Circuit Emulation Clocking
Choose CES Options
Inverse Multiplexing for ATM (IMA) Clocking
Design Recommendations
Clock Sources
Separate Clocking and Data Topologies
Clocking Network Topology
Clocking Loop and Loop Free Topology
Leased Line Services
Revertive/Non-revertive Clocking
Clocking Method
NCDP
Configuration Example
8510MSR-A Configuration
8510MSR-B Configuration
Debug Commands
Related Information

Introduction

Network clocking is the means by which a clock signal is generated or derived and distributed through a network and its individual nodes for the purpose of ensuring synchronized network operation.

Network clocking is an important design consideration in many ATM networks, yet often remains poorly understood. A solid network clocking design is essential to the successful deployment of any ATM network that carries voice or video traffic. The purpose of this document is to describe the architecture, issues, tradeoffs, and options for the design of network clocking for ATM networks in general, as well as those that use the Lightstream LS1010, Catalyst 8510-MSR, and the Catalyst 8540-MSR platforms.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

When Do You Need to Worry About Network Clocking?

You need to seriously consider network clocking in any network that carries voice, video, or any other bit synchronous traffic. Circuit Emulation Service (CES) and VToA require an end-to-end timing relationship and therefore must have a well-designed network clock distribution tree.

Networks that carry purely data traffic that does not require an end-to-end timing relationship, for example, TCP/IP, need not worry as much about clock quality. It is always beneficial, however, to understand the clock sources and distribution topology in any ATM network.

Network Clock System Architecture

A basic comprehension of the clocking capabilities with the different platform and clock hardware options is helpful in order to determine what hardware to select for a particular application. This figure shows the clocking architecture for the LS1010 with FC-PFQ and the 8510-MSR.

There are actually two areas in the system where you can make clocking selections. The first is the system or global clock. The two choices available for the system clock are the local clock on the ASP or any of the clocks derived from network interfaces, see this figure). There are two exceptions as noted in Table 2 in order to use network interfaces to derive clock. These are the 25M ATM port adapter module (PAM) and the 2 port DS3 PAM. The 2 port DS3 PAM is EOL.

The other place where you make a clock selection is on the individual PAM. Each PAM has several clock sources available to it:

• Local free running oscillator, low quality oscillator on PAM

• Network clocking, derived from ASP source

• Loop timed, derived from line

In general, you should never use the local free running oscillator on the PAM as a clock source. In most cases, the clock source are derived from the main network clock. The exceptions are interfaces used in order to take external clocking and interfaces in the clock distribution tree path.

Network Clock Redundancy in the 8540-MSR

You can configure the 8540-MSR as a redundant system with back-up processors and switch fabrics. When configured with redundant route processors with NetClkMods, there are two possible failure scenarios that may occur:

• failure of the entire primary route processor

• failure of the NetClkMod on the primary route processor

Handle Route Processor Faults

State information for the primary NetClkMod is constantly provided to the secondary NetClkMod that resides on a single chassis. Sufficient state information is transferred in order to allow the secondary to take over with minimal disruption. The state information transferred contains the contents of the network clocking internal database. Whenever the database is updated, the changes to the database are mirrored to the secondary.

When the primary route processor experiences a fault and it is necessary to begin to control the network clocking from the secondary route processor, the secondary is notified that it is now the primary. It already has the clocking state and configuration.

Handle NetClkMod Faults

When a node is equipped with two NetClkMods and the primary NetClkMod experiences a fault, the secondary NetClkMod takes over for the failed unit. The switchover is handled transparently by the NetClkMod hardware. The NetClk software is informed of the switchover.

If both NetClkMods fail, the switch hardware switches over to the local oscillator. This is a fairly serious failure and the network should not be allowed to operate in this degraded condition. This condition should be remedied as soon as possible.

Network Clock Distribution Protocol (NCDP)

NCDP is a Cisco value-added software feature that can greatly simplify the configuration and successful deployment of large ATM networks with synchronous clocking requirements. It operates on the same principle as spanning tree to automatically create a loop-free network clock distribution tree with minimized clock path lengths.

The benefit of NCDP is that optimal clock distribution trees are created and maintained in the presence of link failures. NCDP is also very easy to configure. Once the NCDP clock sources are defined, typically a small number for the entire network, all you need to do is enable NCDP on the switches that remain.

When enabled, NCDP communicates over VPI/VCI 0/34 with neighbors across NNI interfaces.

Refer to Initially Configuring the ATM Switch Router for NCDP configuration and design recommendations.

Circuit Emulation Clocking

Circuit Emulation Clocking (CES) is the predominant application that requires you to consider network clocking. CES has several clocking modes. The choice of the best mode depends on the application and circumstances.

Refer to Configuring Circuit Emulation Services for more information on how to configure CES. Refer to An Introduction to Circuit Emulation Services for more background information and detail on CES.

Choose CES Options

You should use synchronous mode clocking and structured service in most cases, and avoid adaptive clocking wherever possible. You should only use SRTS in special cases. See this figure).

Inverse Multiplexing for ATM (IMA) Clocking

The IMA specification outlines two different transmit clock modes:

• Common Transmit Clock (CTC) mode: Uses the same clock source for all links.

• Independent Transmit Clock (ITC) mode: Allows the use of separate clock sources by different links. In most circumstances the use of the CTC mode is advised since it derives clock from one source and is more efficient.

The ATM Switch Processor (ASP) has a default behavior in which the internal oscillator is used in order to clock the T1s in an IMA group. The global configuration command network-clock-select can be used in order to choose an external clock source. Otherwise, IMA interfaces could end up down due to a clocking mismatch. In addition, the show ima interface atmX/Y/ima# interface can possibly display a link in the IMA group as the common reference, but it is possible that it is not the source since the oscillator on the ASP is selected by default.

If you change the mode to Independent Clock Mode (ITC), you can choose clock source looped-timed on each individual link and derive the TX clock from the RX clock.

Do this in order to change to ITC mode:

d12-3-ls1010-26#configure
1w2d: %SYS-5-CONFIG_I: Configured from console by consolet
Enter configuration commands, one per line.  End with CNTL/Z.
d12-3-ls1010-26(config)#interface atm 4/0/ima1
d12-3-ls1010-26(config-if)#ima clock-mode independent
d12-3-ls1010-26(config-if)#

Design Recommendations

There are several recommendations that help in the construction of reliable, high-quality network clock designs.

Clock Sources

Any given design deploys certain hardware based on the overall requirements. These lists all sources that can be available in the order of preference.

1. 8540-MSR with NetClkMod using BITS (Stratum-2 or higher external clock)

2. 8540-MSR with NetClkMod (Internal Stratum-3 Clock)

3. 8510-MSR or LS1010 with ASP-FCPFQ

4. 8540-MSR without NetClkMod or LS1010 with FC-PCQ

Separate Clocking and Data Topologies

There is no requirement that the network clock distribution tree have any relationship to the data forwarding paths. The clock distribution network should be considered an overlay network. Separating clocking and data paths can lead to increased network reliability. In this way, the loss of a data link will not necessarily cause the network clock distribution tree to reconverge.

Clocking Network Topology

The network clock distribution tree should be designed to be as wide and shallow as possible. This minimizes the length of the clock distribution paths.

Avoid clocking loops at all times. Clock loops are created when two interfaces on the ends of a wire attempt to derive clocking from each other. Although network timing can appear to work, the long-term stability of the clocking with clocking loops is susceptible to wander. Phase shifts caused by actions as simple as the change of the length of a cable can cause clocking instabilities.

Clocking Loop and Loop Free Topology

In general it advisable to avoid long, linear network clock topologies. Jitter and wander accumulate as a function of the length of the clocking path. If the natural topology of the network is linear, consider to place the clock source in the center of the network rather that at one of the ends in order to minimize the depth of the clock distribution tree.

Ring topologies typically suffer from the same issues that plague long linear networks. Ring topologies are also susceptible to unintended clocking loops when the primary clock source is lost. Ring topologies tend to maximize clock distribution convergence times.

Leased Line Services

When designing networks use leased line services, it is important to understand how the carrier treats the clock and takes that into consideration. If the carrier also provides clock, then the network devices on both ends should derive clock from the carrier. If the carrier is tunneling the clock through, for example, SONET service, the connection between the two devices connected to the carrier should pass clock between them.

Revertive/Non-revertive Clocking

The choice of revertive vs. non-revertive clocking depends on the network design requirements. Revertive clocking will switch back to the "best" clock source after a failure. Non-revertive clocking will remain on the backup source until manually switched back or until the backup source fails. The design decision is whether it is more important to be using the "best" clock source at all times or better to minimize network disruptions caused by clock distribution reconvergence. For most applications non-revertive reference switching is recommended.

Clocking Method

You should use synchronous clocking for applications that require end-to-end timing and where all network clocking can be traced back to a common reference source. Synchronous clocking is required with structured service.

You should only use SRTS when a network has interfaces that are clocked by sources that are not traceable to a common reference source. SRTS is only suitable for unstructured service.

You should avoid adaptive clocking in almost all cases.

NCDP

NCDP automatically builds optimized clock distribution networks. Whenever possible, you should use this feature.

Configuration Example

This is an example of an NCDP configuration for a network where the primary reference source is defined as LS1010-A, interface cbr1/1/0 with a Stratum-3 clock source.

8510MSR-A Configuration

8510MSR-A#write terminal
Building configuration... 

Current configuration:
!
version 12.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service udp-small-servers
service tcp-small-servers
!
hostname 8510MSR-A!
boot system flash slot0:cat8510m-wp-mz.120-3c.W5.9.bin
enable password ********
!
ip subnet-zero
ip host-routing
no ip domain-lookup
ncdp source 1 CBR1/1/0 3
ncdp source 2 System
ncdp revertive
ncdp

!

8510MSR-B Configuration

8510MSR-B#write terminal
Building configuration... 
Current configuration:
!
version 12.0
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname 8510MSR-B
!
boot system flash slot0:cat8510m-wp-mz.120-3c.W5.9.bin
enable password ********
!
ip subnet-zero
ip host-routing
no ip domain-lookup
ncdp revertive
ncdp

!

These are the results of the show network-clocks command on both switches :

8510MSR-A#show network-clocks
clock configuration performed by Ncdp
Priority 1 clock source: CBR1/1/0(up)
Priority 2 clock source: No clock
Priority 3 clock source: No clock
Priority 4 clock source: No clock
Priority 5 clock source: System clock
Current clock source:CBR1/1/0, priority:1

8510MSR-A#

8510MSR-B#show network-clocks
clock configuration performed by Ncdp
Priority 1 clock source: ATM0/0/0(up)
Priority 2 clock source: No clock
Priority 3 clock source: No clock
Priority 4 clock source: No clock
Priority 5 clock source: System clock
Current clock source:ATM0/0/0, priority:1

8510MSR-B#

When you use an LS1010 PAM in an 8540, you cannot configure two ports on the same PAM to be network clock sources. The system rejects the command and prints an error message, which you can view if console logging is enabled. However, when you use native 8540 PAMs or SuperPAMs, such as the 4-port OC-12 and the 16-port OC-3, you can configure two ports on the same PAM to be network clock sources. These PAMs have the necessary clock hardware to support primary and secondary sources.

Debug Commands

The relevant debug commands are:

• debug ports netclock

• debug NCDP <errors|events|packets>

Related Information

• Configure Network Clocking
• Configure Network Clocking with NCDP
• Configuring CES
• Technical Support & Documentation - Cisco Systems