CEoP/SAToP on Cisco Routing Platforms

Introduction

This document provides an overview of Circuit Emulation over Packet/Structure-agnostic TDM over Packet (CEoP/SAToP) on Cisco platforms and common time-division multiplexing (TDM) clock distribution methods. The context of the presented use-cases will be CEoP in Mobile Wireless backhaul deployments, but this document does not serve as an exhaustive overview of Mobile Wireless devices and their roles. Also, SAToP can certainly be used outside of Mobile Wireless backhaul — it can be used to transport any TDM circuit over an Internet Protocol/Multiprotocol Label Switching (IP/MPLS) core. Finally, this document assumes a basic understanding of Label Distribution Protocol (LDP) and MPLS forwarding. Refer to the end of this document for links to additional resources.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Description

CEoP or SAToP defines a means to provide TDM transport across a packet or label-switched network. SAToP is the standardized name for unstructured transport, while CEoP is often used to refer to Cisco devices capable of SAToP and/or CES structured payload. Instead of leasing or maintaining numerous physical circuits between geographically diverse locations to provide TDM transport, CEoP allows TDM endpoints to connect across an IP/MPLS core. Traditional TDM transport means that dedicated circuits would be physically carried between endpoints through copper and/or optical circuit switching devices. This diagram shows a typical topology:

In this example of Mobile Wireless backhaul, physical circuits are required from the far-end remote all the way back to the Central Office (CO) or Mobile Switching Center (MSC) that houses the aggregating device. Especially if the wireless carrier does not have their own facilities between the remote and central office, leased circuits can be expensive and even circuits owned by the carrier can be expensive to maintain.

SAToP provides an alternative to maintaining physical circuits between TDM endpoints, as long as there is IP/MPLS connectivity available at the TDM endpoint locations.

Note that the endpoints still connect over TDM circuits, but the circuits physically terminate at each local router that is capable of SAToP. The router then transports those TDM frames across the MPLS core via Circuit Emulation (CEM) pseudowires (PWs) to the remote SAToP endpoint so that the TDM endpoints can communicate as if they were directly connected by physical circuits. The migration to this sort of solution compared to classic TDM transport might make sense when an IP/MPLS core is readily available, and in preparation for the TDM endpoints to eventually migrate to native Ethernet connections.

How it Works

The method by which TDM endpoints communicate across a CEM circuit is summarized in five steps. These five steps are outlined in text and in the diagram:

1. Raw TDM frames are generated by the TDM endpoint and transmitted towards the controller on the CEM router.

2. The CEM router receives the raw TDM frame, adds on SAToP encapsulation, adds on the MPLS shim header, and then transmits the frame towards the MPLS core.

3. The MPLS core label-switches the frame based on the LSP that was set up in the PW establishment between the two CEM endpoints.

4. The receiving CEM endpoint receives the frame and associates it with the appropriate cem-group based on the received label. The frame arrives at the cem-group dejitter buffer, and waits to play out to the TDM controller at clock rate.

5. The CEM router serializes the frame from the dejitter buffer towards the TDM endpoint.

The same process is followed bidirectionally. The dejitter buffer mentioned in step four is important. CEM frames must be transmitted/received on the TDM controllers at clock rate, with no exception, in order to emulate a physical TDM circuit end-to-end. Since a circuit is emulated through CEoP/SAToP, obviously the CEM frames are susceptible to delay across the IP/MPLS core. The dejitter buffer is CeoP's means to avoid the consequences of variable delay. Frames are held in the buffer, which is sized in units of milliseconds, to ensure that frames are available to transmit to the TDM controller.

If the dejitter buffer is set to 5ms, then 5ms worth of CEM frames are held in the buffer and transmit out the TDM controller at clock rate. Note that because packets are held in the buffer for the configured amount of time, they experience transmission delay equal to the dejitter buffer size unidirectionally. (Packets arrive at the dejitter buffer on each receiving CEM router.) This means that the total unidirectional delay for a CEM frame is equal to (dejitter buffer size + aggregate network delay).

If the dejitter buffer is empty and does not have a CEM frame to transmit to the TDM controller, a dejitter buffer underrun is accumulated (enter the show cem circuit detail command to check). The TDM endpoint will likely receive errors and/or an alarm, dependent upon the duration that the dejitter buffer is empty. When there is competing traffic along the critical path of the CEM frames, strict QoS for the CEoP traffic is required to prevent variable delay from starving the dejitter buffer. While the dejitter buffer is empty, the CEM idle-pattern plays out to the TDM controller, and this defaults to 0xFF/AIS. The dejitter buffer size is a configurable value, and can be increased to accommodate potential network delay.

TDM Clock Distribution

Just as with traditional physical TDM circuits, TDM clock synchronization is just as important in circuit emulation deployments. The TDM endpoints and router TDM controllers must still sync to common clock sources. While there are many different combinations to distribute a clock between CEM endpoints, here are some common examples:

In-band PW/Adaptive Clocking

In-band PW, or adaptive clocking, is used by remote CEM routers to sync to a single clock source at the Mobile Switching Center (MSC) or Central Office (CO). In this example, the Base Station Controller (BSC) acts as the master clock source, and the aggregation CEM router (7600 or ASR1k) references that clock source with network-clock-select and/or clock source line. The remote CEM router — in this case, a MWR2941 — configures recovered-clock adaptive (cem-group) and network-clock-select 1 PACKET-TIMING. This allows the MWR2941 to derive the clock from the configured transit CEM stream, and then it provides that clock on the TDM controller that faces the Base Transceiver Station (BTS) with clock source internal. This diagram depicts the scenario:

BITS Clocking

Instead of an endpoint like a BSC as the clock source distributed across the CEM path, CEM routers can connect to a common BITS clocking reference for synchronization. In the diagram, both CEM routers are connected to a common upstream BITS clock source (such as a common upstream GPS clock), and then they drive their TDM controllers' clocks based on that. Each router needs a BITS T1/E1 connected from the dedicated BITS controllers on the routers to the clock source. Both routers are configured with network-clock-select 1 BITS and clock source internal to distribute that clock source to the connected TDM endpoints:

Synchronous Ethernet Clocking

Synchronous Ethernet (SyncE), defined by ITU-T G.8262/Y.1362, allows a capable network device to derive a clock sync source from an Ethernet port. Synchronization status messages are sent from clock sources to receivers. Within context of CEM deployments, CEM routers may derive TDM clock sync through SyncE from connected Metro Ethernet devices — perhaps even the same devices that provide the IP/MPLS core transport between aggregation and remote CEM endpoints. Much like with BITS, SyncE is selected with network-clock-select 1 SYNCE # and can act as the master clock to the TDM endpoints with clock source internal configured under the T1/E1 controller for the corresponding CEM group:

Out-of-band PW Clocking (virtual-cem)

Another method to distribute a centralized clock source to remote CEM routers is to use a Virtual-CEM interface in out-of-band PW mode. Unlike in-band PW/adaptive clocking, out-of-band PW clocking establishes a separate, dedicated PW just for clock distribution between the master clock router and the slave clock router. In order to accomplish this, recovered-clock is configured in master mode, generally on the aggregation router that distributes its clock source. The recovered-clock slave is configured on the remote CEM router that will receive the clock. If these commands are configured in both routers, it would spawn a Virtual-CEM interface in the configuration — this interface is specifically to configure the out-of-band clocking PWs between master and slave routers. In the diagram, the aggregating 7600 router uses SyncE as the primary clock source (with network-clock-select SYNCE), which distributes that clock to the local BSC with clock source internal, and also distributes the clock to the remote CEM router through the out-of-band Virtual-CEM PW.

PTP Clocking (Timing over Packet)

IEEE 1588v2 / PTP is a means to distribute clock information across an IP network. There is no PW between master and slave CEM routers when PTP is used — only reliable IP connectivity is required between the devices to distribute clock information in the payload of IP packets. While PTP can also be used to distribute time-of-day information much like NTP, within context of CEoP PTP is used for frequency synchronization. In the diagram, the aggregating 7600 is configured with network-clock-select T1 #/#/# to pull in timing from a connected circuit on the BSC, and then it is configured as a PTP master. The far-end CEM router then has the 7600's IP address configured as a PTP source on the receiving Ethernet interface, so it acts as the slave to derive timing when it uses network-clock-select 1 PACKET-TIMING. Essentially, the 7600 pulls in a clock reference from the BSC circuit, and then distributes that clock over PTP to the remote CEM router.

Clocking Summary

The TDM clock distribution methods outlined above are simple examples to demonstrate the various options available for CEoP deployments. Note that combinations can be mixed together, and as long as the TDM endpoints are synchronized to a single common clock source, there should not be any problems regardless of how that clock is distributed. For thorough documentation of the configuration of these features, refer to the resources section at the end of this document.

Commands

These commands are useful to gather data:

• show network-clocks — shows the status of the platform network-clock

• show controller [T1|E1] — shows the status of the TDM controller facing endpoints

• show xconnect all — shows a summary of all pseudowire status

• show cem circuit — shows a summary of all CEM status

• show cem circuit detail — shows detailed information/statistics for all CEM groups

• show cem circuit interface CEM#/# — shows detailed info for CEM#/#

• show mpls l2transport vc [vcid] detail — shows detailed information regarding PW status

• show platform hardware rtm stat — on MWR2941 with ToP module, shows the timing module statistics

Related Information

• Cisco 7600 Series Router Software Configuration Guide Cisco IOS Release 15.0S
• Cisco MWR 2941-DC Mobile Wireless Edge Router Software Configuration Guide
• Cisco 7600 Series Router SIP, SSC, and SPA Software Configuration Guide
• Cisco ASR 1000 Series Aggregation Services Routers SIP and SPA Software Configuration Guide
• Cisco ASR 901 Series Aggregation Services Router Software Configuration Guide
• Cisco ASR 903 Router Chassis Software Configuration Guide, IOS XE Release 3.7
• Technical Support & Documentation - Cisco Systems