A converged transport architecture that benefits from closer integration of the IP and dense wavelength-division multiplexing (DWDM) layers offers an increasingly promising direction for the evolution to a Next-Generation Internet that scales cost-effectively. It allows service providers to better address the huge increase in low-revenue traffic. This paper describes the significant benefits arising from the Cisco® nLight Control Plane Protocol - focusing on the technology innovations that enable this evolution, summarizes the potential phases of multi-layer control plane adoption, and highlights the related benefits in network operations and optimization, particularly in optical restoration.
The Need for a Converged Core Network Transport Architecture
Today, network operators face higher total cost of ownership (TCO) to maintain a core transport infrastructure that can meet traffic needs in terms of both capacity growth and complexity. The dominant source of current and future traffic growth originates from IP applications, predominantly video-based, with often diverse control plane, policy, and quality-of-service (QoS) requirements ranging from best-effort to high-priority content-sensitive routing. IP traffic continues to grow exponentially, typically at 30-percent compound annual growth rate (CAGR), and is expected to exceed 110 exabytes per month in 20161. The operators' TCO challenge is exacerbated further by the increasing amount of hard-to-monetize over-the-top (OTT) content, best-effort peer-to-peer traffic, and settlement-free peering of content originating from massively scalable data centers (MSDCs). As a result, new investment in core transport networks is being challenged by the requirement of a return on investment (ROI) above the cost of capital.
This business challenge has been motivating the evolution to new core network architectures that scale with lower TCO, using the available technology and innovations to best advantage. The universal adoption of the IP Next-Generation Network (IP NGN) as the most cost-effective and scalable network architecture has been a significant step in this direction2. IP NGN has offered a converged, access-agnostic aggregation and transport of Internet traffic, taking full advantage of IP/Multiprotocol Label Switching (IP/MPLS) routers interconnected over point-to-point optical links. The IP NGN routers have been using the significant and extensive advancements in electronics typically characterized by "Moore's Law to double the system speed, density, and most importantly cost-performance metrics every couple of months.
At the same time, innovations in DWDM system design and photonic technologies, including reconfigurable optical add/drop multiplexing (ROADM), coherent DWDM transmission that uses electrical signal processing, and a maturing Wavelength Switched Optical Network (WSON) dynamic control plane, have enabled fiber communication systems that cost-effectively scale to terabits per second and thousands of kilometers with a simplified operational model3. Another trend that helped simplify operations and reduce network costs is the elimination of additional layers in the network. The integration of high-speed optical interfaces in routers, with 100G IP over DWDM (IPoDWDM) as the most recent example, has significantly improved the transport cost and performance metrics. Furthermore, with the introduction of optical-transport-network (OTN) framing in the routers, there is no longer a need for an intermediate dedicated time-division multiplexing (TDM) layer for just the operations, administration, maintenance, and provisioning (OAMP) functions. For this reason, the direct interconnection of core routers through the DWDM layer has increasingly become the architecture of choice in the converged network transport evolution, allowing operators to cope best with the ever-shrinking revenue per bit as they scale their networks4.
As an important part of the IP + Optical architecture, a new multi-layer control plane architecture has been defined, built around extensions to Generalized MPLS (GMPLS) standards and a new set of advanced functions - the Cisco nLight Control Plane Protocol enables the exchange of information between the IP and DWDM layers, allowing for improved network operations and maximization of benefits for a converged transport network. The Cisco nLight Control Plane Protocol is based on a careful balance of providing sufficient information for both IP and DWDM layers to allow them to gain from understanding each other, without burdening the network with excessive information that will encumber and affect its independence and scalability. This scenario is in contrast with the more well-known GMPLS models: the "peer model", which required too much information exchange, leading to complexity; and the "overlay model", which enabled too little exchange, resulting in inefficiency.
Instead, the Cisco nLight Control Plane Protocol is based on a client-server model. To this end, the DWDM "server" actively shares all the necessary information about the optical paths that link the IP "client" layer through an overlay User-Network Interface (UNI) model with the IP layer (Figure 1a). In addition, the IP layer communicates its requirements for optical pathing with the DWDM layer, providing the optical layer with more visibility into the needs of the IP layer. Cisco nLigh technology is an example of introducing software-defined networking (SDN) programmability into the network through the Cisco Open Networking Environment (ONE).
In the most general case, there could be more than two layers in the network, with more than one client-server (UNI) interaction, as in the example depicted in Figure 1b; however, many of the functions discussed in the following section become more complex with more layers, reducing the value of the overall solution. It is important that the participating layers remain independent and reasonably decoupled; as such, for example, the IP/MPLS layer may continue to run multilevel Intermediate System-to-Intermediate System (IS-IS) instances, while the DWDM ROADM layer may be running Open Shortest Path First (OSPF). This decoupling makes the nLight technology particularly powerful for allowing the two layers to more easily scale independently while maintaining the chosen organizational segmentations and taking advantage of their operational expertise. It also fosters multivendor solutions.
Figure 1. The Basic Overlay UNI Model (left, 1a) and the Most General Case, of More Than Two Layers in the Network, with More Than One Tier of Client-Server (UNI) Interactions (right, 1b)
The rest of this paper summarizes the main features, use cases, and benefits arising from the Cisco nLight Control Plane, and especially the significant TCO savings arising from IP + Optical optimal restoration.
The Cisco nLight Control Plane Protocol Value Proposition
The Cisco nLight Control Plane Protocol aims to eliminate multi-layer complexities and inefficiencies through optimal information sharing between the IP and DWDM layers. Cisco nLight technology also forms the foundation for enabling a networkwide multi-layer optimization based on path computation element (PCE)5,6.
The main economic benefits of Cisco nLight technology arise from its ability to maintain the service-level agreement (SLA) of the IP/MPLS services at reduced overall network cost. Conversely, other features allow improving the SLA at equal total network cost. A good example is the significant improvements in network availability that can be achieved by using DWDM shared-risk link group (SRLG) information to guide IP/MPLS routing decisions; for example, to avoid or minimize common SRLGs for the main (working) path and the loop free alternative or traffic engineered fast reroute (LFA/TE FRR) back-up path. Operational benefits of the Cisco nLight Control Plane Protocol also stem from greater flexibility in handling dynamic cloud traffic such as demand uncertainty or variations over time, or optimization based on cost or latency, or improved handling of catastrophic failures.
A typical Cisco nLight Control Plane Protocol use case would be when the IP/MPLS client requests the DWDM layer for a connection (implementing a new IP layer link), the client layer communicates the needs for certain SRLGs to be excluded or included. The DWDM server layer responds based upon the proper visibility of the transport network environment. The Cisco nLight Control Plane Protocol can request to follow the path of a specific existing DWDM connection (by referring to its Circuit-ID) or alternatively request a disjoint path from a specific Circuit-ID. It may also request wavelength routing based on an optimization for shortest latency, or an upper bound on latency, or it may request wavelength routing based on some other optimization metric, such as lowest number of optical regenerations or total optical cost. This information awareness could eliminate multi-layer inefficiencies, support improved network planning, and automate operation. What used to take months to manually plan and coordinate can now be done in minutes. More specifically, the nLight Control Plane Protocol can automate network optimization during:
• Normal operation, by sharing optical layer information such as SRLGs or latency with the IP layer
• Connection setup, by improving the IP/MPLS layer availability through avoidance of certain SRLGs, or improving the IP/MPLS layer QoS by selecting the lowest-latency path
• Traffic or network changes, by improving the use of existing resources in the face of traffic fluctuations, through DWDM or IP/MPLS path reoptimization (as shown in Figure 2), or by rerouting traffic appropriately during a maintenance window
• Restoration, by requesting the optical layer to restore a connection upon physical layer failure, and subsequent network reoptimization when the network recovers from this failure
Figure 2. An Example of Network Re-optimization by Using Existing Total Resources for Different Traffic Demands
From an operational perspective, the initial adoption of the nLight Control Plane in current networks may be easier for the "normal operation" use cases, because they do not require operational adjustments to support automated connection setup or rerouting. In this case, the DWDM layer merely shares with the IP layer information about DWDM properties of existing IP links, such as SRLG or path latency, thereby influencing more intelligent routing decisions. The information sharing between the DWDM and IP layers also requires smaller extensions to the existing GMPLS UNI between the IP and DWDM layers. The adoption of dynamic DWDM switching based on IP requirements may prove a bit more challenging because of traditional operational limitations.7 To overcome the operational limitations a set of DWDM setup procedures, which rely on less-mature impairment-aware extensions, may be necessary. Eventually, network optimization using the nLight Control Plane during traffic and network changes, particularly during failure(s), holds the most significant promise for TCO savings. The next section evaluates the TCO savings from optical restoration using the nLight Control Plane.
Implementing a Multilayer Control Plane
The Cisco nLight Control Plane Protocol solves the complexity, scalability, and operational inefficiency problems of today's separated approaches. The Cisco nLight Control Plane is a model that addresses the shortcomings of the peer and overlay models while enabling advanced features that capitalize on the tighter coupling between packet and transport layers in the network.
TCO Savings from Optical Restoration with Cisco nLight Technology
Using the Cisco nLight Control Plane Protocol for restoring IP links from physical layer failures could offer the most significant TCO benefits when:
• The IP/MPLS layer is designed assuming node failures are a low risk event.
• The SLAs of some best-effort traffic allows for a few seconds of recovery time under failure.
In such a case, if a link failure occurs, the high-priority classes of service can be adequately protected through the current robust IP/MPLS protection mechanisms, such as Fast Reroute (FRR), using the spare (unused), pre-provisioned protection capacity in the IP layer. On the other hand, best-effort traffic without strict failure-related SLA guarantees remains only partially protected in the IP layer until the DWDM layer restores the failed links over alternative DWDM paths, as depicted in Figure 3. At this point, peak traffic will be supported again for all traffic classes. This approach works well as long as this restoration process does not exceed certain time scales, typically no more than several tens of seconds, after which the lack of peak performance for best-effort traffic may be deemed inappropriate for the SLA of such traffic.
Figure 3. Normal operation ( shown in the diagram on the left) and DWDM layer restoration of a failed IP link over alternative DWDM paths (shown in the diagram on the right)
It is important to note here that the benefits from basic optical restoration have been known and analyzed by academia and industry. To summarize, the TCO savings arise from reusing the same expensive interfaces that are used for the working path for the restored path (essentially redirected over an alternate path). In practice, however, DWDM layer restoration has not been sufficiently robust until the new-generation DWDM systems that have recently started to become available. These new DWDM systems benefit from the significant innovation in "contentionless", "directionless", and "colorless" for multi-degree Cisco nLight ROADM designs that enable non-blocking wavelength switching. Moreover, the time to provision a new wavelength has been reduced, making DWDM layer provisioning sufficient for restoration. (The analog control in DWDM systems still prevents the DWDM layer from being fast enough for protection.) The emergence of this new generation of DWDM systems has also motivated the development of many advanced DWDM control plane (WSON) implementations8,9. These DWDM control plane implementations, although proprietary, contribute to the restoration functions discussed here. Figure 4 shows the different phases of such nLight technology restoration and the corresponding time scales.
Another important differentiation of the Cisco nLight restoration approach compared to other known approaches is that the router promotes restoration - the restoration is not autonomously performed by the optical layer. The collaboration between the IP and optical layers is critical to the optimality of the scheme, because the router knows what constraints can be relaxed during restoration and which ones should still be maintained. For example, the router may decide that the restoration path could use a more relaxed latency constraint but still within a certain new bound that guarantees end-to-end latency for some services. Pure optical layer restoration does not have the information to make such decisions.
Figure 4. Cisco nLight Control Plane Protocol - Phases of Link Restoration and the Corresponding Time Scales (Source: "GMPLS-based IP/Optical Integration - Benefits and Case Studies", O. Gerstel, et al., MPLS and Ethernet World Congress 2012, Paris, France, February 10, 2012)
The higher the percentage of best-effort traffic, the higher are the TCO savings because minimal additional protection bandwidth must be pre-provisioned for such traffic in the IP/MPLS layer. For example, in an IP/MPLS network with equal amounts of high-priority and best-effort traffic, without the link-restoration ability of Cisco nLight technology the maximum link usage is limited to 50 percent, whereas with restoration the network could operate close to 100-percent usage, because best-effort traffic requires minimal spare IP capacity. (Of course, traffic distribution and CAGR must also be accounted for here, reducing the network usage in general.) Figure 5 summarizes the results of an actual network model, calculating the capital expenditures (CapEx) required in the case of an nLight-enabled optical restoration versus a router-based protection for link failure (left) or link and node failure (right). More than 30-percent CapEx savings have been identified in the case of link failures for this example. Moreover, the nLight link restoration can also enable recovery from multiple link failures in IP + Optical transport networks.
Figure 5. Results of an Actual Network Model That Estimates the Savings of Cisco nLight-Enabled Optical Restoration Versus a Router-Based Protection for Link Failure, or Link and Node Failure (Source: "GMPLS-based IP/Optical Integration - Benefits and Case Studies", O. Gerstel, et al., MPLS and Ethernet World Congress 2012, Paris, France, February 10, 2012)
From an operational perspective, the nLight restoration scheme activates after the IP layer has quickly protected (for example, with FRR) some or all of the traffic (depending on the use case), relying on QoS to ensure best use of the potentially lower Layer 3 capacity until Layer 1 fully recovers the Layer 3 links. The restoration is also based on re-signaling of the failed connections by the router, and can thus benefit from the rich MPLS-Traffic Engineering (MPLS-TE) feature set. Multiple restoration path computation options could be explored, including trying a preplanned path first, followed by calculating a dynamic path. When the link failure is repaired, the router re-signals the request to trigger Layer 1 reoptimization, again exploiting the richness of MPLS-TE features to allow for multiple recovery options, such as reverting back to the original path or selecting the best current path. During reoptimization, traffic could be temporarily moved hitlessly away from the Layer 3 links that will be disrupted by the rerouting of their Layer 0 path (which is another capability absent from a pure optical restoration approach). Throughout, this process uses the same router interfaces and transponders that have been used to implement the original Layer 3 links, thus requiring minimal incremental cost.
The Path Forward: An Open Solution for Transport Network Optimization
The Cisco nLight Control Plane is an open architecture based on the Cisco Open Networking Environment (ONE). The protocol has been designed around extensions to existing standards, including GMPLS and other related protocols, to enable the sharing of the proper amount of information between layers. The aforementioned decoupled design enables this architecture to work over many different DWDM systems. To facilitate such integration, Cisco has started the effort to standardize all the necessary nLight Control Plane Protocol extensions10,11. At the same time, use of "alien" DWDM channels over different, third-party DWDM systems12,13 is also gaining momentum because of the robustness of coherent modulation. In light of these developments, we envision an open IP + Optical architecture whose benefits manifest over any deployed DWDM system that offers the required optical switching flexibility of colorless, directionless, and contentionless nLight ROADMs and supports the required control plane extensions.
Moreover, network optimization based on a centralized path computation element14,15 and implemented using a distributed open, multivendor, multi-layer control plane offers an additional advancement of the converged Layer 1 and Layer 3 transport16. In this vision, PCE network optimization continuously maximizes the IP/MPLS and optical transport, preventing the network from getting gradually less optimal - as is common in today's networks. In today's networks, optimization is typically performed using offline modeling tools, because a networkwide traffic matrix cannot be easily collected in real time, and manual input of future traffic assumption is typically required17. The benefits from continuously optimized network operation would generally increase as the traffic becomes more dynamic and the network more adaptable. More generally, such a novel architecture becomes a step to the vision of a future fully automated transport infrastructure optimized based on software-defined network operation18.
7 "Optical Network Management and Control", R. Doverspike and J. Yates, Proceedings of the IEEE, vol. 100, no. 5, May 2012.
8 draft-ietf-ccamp-wson-impairments-10.txt (available at http://www.rfc-editor.org/rfc/rfc6566.txt)
9 "Traffic independent heuristics for regenerator sites election for providing any-to-any optical connectivity", C. Saradhi, et al., Proc. Conf. Opt. Fiber Commun. Conf., Los Angeles, Calif., Mar. 2010.
10 "iOverlay Framework", C. Filsfils, Keynote 2 MPLS & Ethernet World Congress 2012, Paris, France, February 8, 2012
12 "Demonstration and Evaluation of IP-over-DWDM Networking as "Alien-Wavelength" over Existing Carrier DWDM Infrastructure", D. Ventori, et. al. IEEE/OSA Conference on Optical Fiber Communications (OFC) `08, paper NME3, San Diego, CA, February 2008.
17 A. Clauberg, "Revolutionizing Carrier Service Delivery using a Software Defined native IP Network", Open Networking Summit 2012 Program, Santa Clara, CA, Wednesday, April 18, 2012 http://opennetsummit.org/
18 D. Ward, "Service Provider Approach to SDN", Open Networking Summit 2012 Program, Santa Clara, CA, Wednesday, April 18, 2012 http://opennetsummit.org/