Network Modernization:
A TDM to IP Solution

INTRODUCTION

Operators are struggling to manage their aging SONET/SDH infrastructure, much of which has exceeded its intended lifespan and become a serious operational strain. For years, oper-ators had few architectural options for network modernization. Fortunately, circuit emulation (CEM) has emerged as a means of circuit-to-packet migration, though many operators may not yet be aware of its scalability or its benefits.

This white paper discusses the modernization challenge and presents CEM over MPLS-based networks as the most promising solution for operators and their customers. Underscoring the point that CEM is ready today, we provide a case study of Verizon's circuit-to-packet modernization using CEM with MPLS, which is being deployed today. Verizon's use case has wide applicability to many other operators.

TODAY'S NETWORK CHALLENGES

SONET/SDH equipment has had a long and productive lifespan in telecom networks, but the technology and supporting equipment is well past its prime and in need of replacement. Below, we list the current and primary issues with the installed infrastructure:

• Equipment has reached end-of-life (EOL): For many operators, the SONET/SDH installed base is meeting or exceeding the 10- to 15-year lifespan for which it was intended. Replacement parts are almost, if not already, obsolete and increasingly difficult to find.
• Rising operational costs: Operators face increasing operational costs running legacy networks due to several factors:
  • Higher faults and increased service degradation are increasingly common now in EOL infrastructure, meaning more maintenance for technicians.
  • Manual provisioning and lack of automated processes make maintenance activities longer, increasing costs.
  • Older technologies have large footprints and high-power consumption compared to today's packet-optical technologies, leading to wasted space and excessive and unnecessary power and cooling costs.
  • Lastly, maintaining separate networks for TDM and packet adds to operational and organizational costs.
• Security flaws: The lack of available support (cited above) also includes a lack of security patches for equipment. Given the prominence of security in today's digital world, coupled with the fact that security is one of the key attributes of a private line service, this lack of security is a particularly high concern.
• Lack of vendor support: Vendors have reduced or stopped supporting legacy equipment because the market has dropped 95 percent from its peak. Some legacy suppliers are actually no longer in business.
• Lack of qualified personnel: As equipment has aged, fewer qualified technicians are available to support it. This leads to higher operational costs and increased risk of service outages.

As shown in Figure 1, annual global SONET/SDH equipment spending dropped from a high of $6 billion down to $1 billion by 2015. But this statistic doesn't account for the massive installed base that remains in operation today. Operators have invested $30 billion on SONET/SDH networks since 2008 alone. Nearly all of this equipment remains in operation today, at least for now.

Figure 1: Global SONET/SDH Equipment Spending, 2007-2016

Source: Heavy Reading, 2017

Operators around the globe face a big challenge: How do they move away from the aging infrastructure in their own networks without losing customers? How do they move from a fixed grid to a more flexible grid? How do they move from 10G to coherent?

In the absence of a good strategy, most operators have simply delayed the decision. How-ever, the "TDM problem" has now become so severe that doing nothing is no longer an op-tion. Verizon, profiled in this paper, is an example of an innovative operator that is leading the market in transformation with a new high-capacity and scalable packet-optical network.

CIRCUIT EMULATION OVERVIEW

Fortunately, a viable means of network modernization has emerged in the form of circuit emulation (CEM). CEM is designed to closely match the key service characteristics of a circuit switching technology over a packet network. Critical circuit characteristics include operations, administration and management (OA&M) functions; predictable and deterministic transmis-sion; and sub-50 ms resiliency in failures, achievable using packet network protection tech-nology such as MPLS FRR (fast re-route).

CEM itself is not new. The first IETF requests for comment (RFCs) for CEM over packet net-works appeared in 2005, and CEM has evolved and matured since then. Today, there are three major IETF-defined CEM standards addressing different data rates, packet-switched network types (Ethernet, MPLS, IP) and physical media (i.e., copper and fiber).

• Structure-Agnostic TDM over Packet (SAToP): SAToP defines pseudowire en-capsulation for T1/E1 and T3/E3 bitstreams over packet-switched networks. See RFC 4553.
• Structure-Aware TDM Circuit Emulation Service over Packet Switched Net-work (CESoPSN): CESoPSN defines pseudowire encapsulation of lower bit-rate nxDS-0 streams over packet-switched networks. See RFC 5086.
• SONET/SDH Circuit Emulation over Packet (CEP): CEP defines pseudowire en-capsulation for emulating SONET/SDH circuits over packet-switched networks. Origi-nally, definitions were limited to 622 Mbit/s circuit rates (OC-12/STM-4), but today encapsulation is standardized up to 10 Gbit/s (OC-192/STM-64). See RFC 4842.

While CEM has been around for years, scalability and density were required to enable the transformation described in this white paper. Additionally, implementations of CEP defined in RFC 4842 were missing, meaning that providers could emulate T1/E1 and T3/E3 rates (low data rates), but not at the STS or VT path level required for SONET/SDH services. These advances, combined with MPLS pseudowire transport, have made CEM ready for mass acceptance and adoption.

BENEFITS OF CIRCUIT EMULATION FOR TRANSFORMATION

CEM provides a "bookended" architecture for TDM migration, in which conventional SONET/ SDH equipment transmits and receives TDM traffic at the two endpoints, but the entire net-work between those endpoints is replaced. Figure 2 illustrates a TDM-to-packet migration scenario using CEM.

Figure 2: TDM-to-Packet Migration With CEM

Source: Cisco and Heavy Reading, 2018

In the figure, a CEM-capable Layer 2/3 switch/router receives a private line transmission at the endpoint, encapsulates the data using CEM, and transmits the data as packets through an Ethernet, MPLS or IP network. Another CEM-capable Layer 2/3 switch/router at the receiving endpoint removes the CEM encapsulation and hands off the private line data as TDM traffic to the receiving-end multiplexer.

CEM provides several key benefits that make it appealing for operators that need to retire their legacy SONET/SDH and digital cross-connect equipment. Below, we provide an over-view of the main operator benefits:

• Measured migration for operators and customers: CEM allows operators to swap their aging equipment without forcing service changes on their customers (i.e., it is a maintenance window change). As described above, customer endpoints remain untouched over an end-to-end packet-switched network. Thus, a customer subscribing to an OC-3 service remains an OC-3 customer even after CEM is put in place. With SLAs, handoffs and pricing terms remaining unchanged, there is no opening for a new request for proposal (RFP).
• Immediate operational benefit: Migration with CEM does not require a full net-work overhaul; operators can realize operational benefits immediately even as up-grades occur incrementally. Targeted sites that will yield maximum benefit ahead of time will produce the best results. Power and maintenance costs go down, while central office floor space is opened up for new revenue opportunities, such as data center or colocation services.
• Path to automated network operations: Solutions based on CEM programmable network elements are much better architectures to enable automation. Easier software upgrades and proactive identification of problems before they impact services are immediate benefits. In addition, SONET/SDH equipment lacks streaming telemetry and model-based programmability supported by modern operating systems.
• Maintain 50 ms resiliency: The TDM endpoints of the CEM service can be protected using common protection schemes such as APS/MSP and UPSR/SNCP. The underly-ing MPLS packet transport uses well-established RSVP traffic engineering protocol, signaling for a backup path to be pre-programmed and bandwidth pre-allocated, thus guaranteeing protection.
• Retaining SONET-like OA&M: CEM provides protection, operations, administration and management (OA&M). Not only do OA&M functions closely match, but the "look and feel" is similar, making it relatively easy for TDM-trained technicians to adapt.
• Scalability and operational simplicity with MPLS: CEM circuits are represented by pseudowires mapped over traffic-engineered tunnels to provide protection and the ability to engineer the path across the packet-switched network. These pseudowires and tunnels are not tied to a circuit rate such as SONET/SDH STS or VT paths. Thus, low-order to high-order grooming, one of the major operational challenges, is unnec-essary with CEM. Any node in the network can set up a CEM service of any rate to any node. The traffic engineering control plane finds the optimum path satisfying the defined constraints, and the end-to-end path is programmed on all hops with-out special operator attention being needed. To address a common concern from the past, signaling extensions defined in RFC 7551 have been adopted to create a bi-directional path and provide co-routing of the forward and reverse direction, just like TDM networks.

VERIZON CASE STUDY

Since the 2000s, Verizon has been a leading innovator in the advancement and adoption of packet-optical technologies. Its network architectures have been closely followed by all the major operators around the world. This is also the case with CEM adoption, in which Verizon is pioneering the way.

Verizon's network has hundreds of thousands of circuits, ranging from DS-0 rates to OC-192. This includes 74 different platforms and more than 150,000 TDM-based elements in its central offices. Operating one of the world's largest networks, Verizon had been struggling to determine exactly how to modernize its network. Figure 3 illustrates Verizon's legacy SONET-based transport network.

Figure 3: Verizon's Legacy Transport Network

Source: Verizon, 2018

Verizon began to address the problem when it launched its metro network RFP in 2014. CEM was high on the solution list for that RFP, but a lack of OC-192 support created a problem. Emulating some of the TDM network while maintaining TDM infrastructure for its 10 Gbit/s circuits meant that two separate networks would still be required.

Further, Verizon's network has a lot of channelized OC-192 interfaces, so 10 Gbit/s was a critical piece of the TDM migration strategy. "The standardization of OC-192 rate CEM, com-bined with the availability of products supporting OC-192, removed the 10 Gbit/s barrier and was key to making CEM a viable option," says Verizon's Director of Optical Transport Network Architecture, Design and Planning, Glenn Wellbrock.

Additionally, two other CEM advances were critical for the operator:

• Integrated bit error testing capabilities: meaning a PRBS bit error test can be run on any CEM circuit configured, without the need for an external "test head." This feature eased service activation and troubleshooting.
• Support for M13 multiplexers: the CEM solution understands not only SONET/SDH multiplexing but also PDH, and can seamlessly interoperate between the two. In the past, complex "transmux" configurations in SONET/SDH ADMs were needed.

Using CEM, Verizon has built a graceful migration architecture that retains the customer's endpoint as long as required, but eliminates the intermediate digital cross-connects between those two endpoints. As illustrated in Figure 2, once TDM traffic is encapsulated at the CEM packet switch, it becomes part of the packet network. Verizon selected Cisco's NCS 4200 system as one of its CEM packet switches. The packet network is Verizon's MPLS core.

SONET & Digital Cross-Connect Removal

Each time Verizon rolls a TDM customer to the packet network, intermediate SONET multi-plexers and digital cross-connect ports are idled throughout the network. When a multi-plexer or digital cross-connect no longer carries customer traffic, it can be removed from the network with no customer impact – freeing up the associated space, power and maintenance activities.

According to Wellbrock, digital cross-connect and SONET removal yields the biggest immedi-ate benefits. Digital cross-connects are among the oldest systems in the network (sometimes 20 years old or more), and this older technology occupies massive central office footprints that can immediately be put to better use. For example, digital cross-connect replacement at a single site can remove many rows of equipment that can be substituted with just a few racks, with space savings of up to 90 percent.

Maintaining SLAs

Verizon's CEM over MPLS architecture allows the operator to maintain the same SLAs as the legacy circuit-switched network that is being replaced. When Verizon converts a customer to the packet network, it makes no changes to the contract and no changes to customer pricing. This yields two significant benefits:

• Charging the same fee but using the lower-cost packet network – which immediately boosts service profitability.
• Customers don't put out a new service RFP, so Verizon doesn't risk losing a customer.

Migration Plan

Verizon is "seeding" the network with MPLS packet switches in advance of moving circuits. Specifically, Verizon is using the Cisco NCS 4200 today in edge offices as well as core cen-tral offices. The NCS 2000 and NCS 4000 platforms are also deployed.

Verizon is beginning the rollovers with a targeted plan that identifies high-impact sites through the use of visualization tools and a set of machine learning algorithms. The operator recognizes that the circuit-to-CEM rollover process will take several years.

Figure 4 shows Verizon's next-generation transport network using CEM and an IP/MPLS core.

Figure 4: Verizon's Next-Generation iEN Transport Network

Source: Verizon, 2018

CONCLUSION

The aging infrastructure problem has reached such a critical point that operators can no longer ignore it. Today they face a big challenge: how to migrate away from legacy multi-plexers and digital cross-connects without losing customers that require the private line functionality. CEM has matured to provide the most efficient solution to the problem, ena-bling a measured migration for operators and their customers while retaining reliability (including 50 ms resiliency) and SONET-like OA&M. With CEM, operators can reap the oper-ational benefits of a modern packet-switched network while retaining existing private line service revenue.

A leading innovator in packet-optical transport, Verizon is a first mover in TDM-to-IP/MPLS migration with CEM and has already begun seeding its metro network with CEM equipment, with a multi-year plan to shut down the legacy network. Significantly, the TDM-to-IP approach championed by Verizon has wide applicability to many operators around the world struggling with similar problems. Verizon is committed to raising the visibility of CEM and sharing its learnings with other operators along the way.

Topic: NCS 2000 | NCS 4200 | NCS 4000 | Optical | MPLS | FlexLSP | EPNM | NSO | WAE