Published: Aug 2018
Traffic growth, particularly for greater use of video, is demanding more capacity on the expensive primary circuits to remote Cisco offices. Trend reports for Cisco’s Internet circuits indicate that our cloud traffic has grown 76 percent from 2016 to 2018, a major factor in this capacity demand.
We needed to address the challenge of balancing network consumption versus the sporadic bandwidth demand and quality of service (QoS) requirements of applications. This need led the Cisco Networking IT team to seek a solution that would offload some of the less QoS-sensitive traffic to the lower-cost backup circuit in each remote office.
With Cisco’s acquisition of Viptela in 2017, we gained the Cisco SD-WAN solution, which gives us the ability to set and manage policies to load-balance applications between the primary and secondary circuits as well as enable application-aware routing. We expect our implementation of Cisco SD-WAN will reduce our service provider costs by 30 percent over time. It will also simplify our management of the WAN infrastructure, policies, and traffic for remote-office network connections.
We connect Cisco’s remote offices to one of 15 global hubs on our corporate network. In many of these offices, we route network traffic through two circuits—one serves as the primary circuit and one as a secondary, backup circuit. The primary circuit is usually a point-to-point, Layer 2 VPN (L2VPN) connection from a global multiprotocol label switching (MPLS) provider. In most cases, this design provides high-quality and cost-effective network connectivity because we can take advantage of high-capacity circuits. The design also allows us to manage the quality of voice and video services because we can prioritize bandwidth allocation of latency-sensitive traffic on the primary circuits.
The circuit design is different for our larger offices, where we deploy dual symmetrical circuits that are load balanced based on Layer 3 forwarding across equal-cost multi-path (ECMP) links. This load balancing isn’t based on application service level agreement (SLA) or performance, but instead on round-robin placement of flows across both links.
In smaller offices, the secondary backup circuit is either a smaller L2VPN circuit from the MPLS provider or a direct VPN connection over a business-class Internet service to optimize WAN costs. When the primary circuit fails, the secondary circuits will carry only critical data services, not voice or video. However when the primary circuit is active, the backup circuit is not used, meaning that expensive, available bandwidth is wasted.
Today the Cisco SD-WAN solution requires our existing Cisco 4451 Integrated Services Router (ISR) in the remote office to connect to the corporate WAN via a Cisco vEdge 1000 router, which delivers a comprehensive WAN gateway. This two-box design is typically deployed in cases where there is lack of feature parity between the Cisco devices and the Cisco vEdge 1000.
When the Viptela feature set is integrated into Cisco IOS®-XE software in late 2018 or early 2019, we expect to deploy this software on our Cisco ISR 4451 fleet and on Cisco ISRv virtual routers, which are implemented on the Cisco 5000 Enterprise Network Computer System (Cisco ENCS). This software will allow us to implement a single-platform access solution for our remote sites because we will then remove the Cisco vEdge 1000 routers.
In the Cisco IT role as Cisco’s “Customer Zero,” we are initially deploying Cisco SD-WAN as a production pilot that will connect 10 midsized remote offices across three network hubs. This project will help us refine the two-box design and gather data on the best way to optimize WAN costs. Additionally, it will allow Cisco IT to provide valuable feedback to the product development team about deploying and operating SD-WAN on a global, enterprise scale.
The project data collected will be evaluated against two objectives. First is to identify how much bandwidth savings might be possible on the main circuit for the site by load balancing appropriate traffic to the direct Internet access (DIA) circuit. Second is to identify the potential for sending even latency-sensitive voice and video traffic over the DIA circuit during periods of heavy demand on the primary connection. Over time, we will review latency and jitter data for these sites to identify peak traffic periods and adjust the load balancing as appropriate.
To identify which traffic should always be sent over the DIA circuit, we reviewed data on bandwidth usage by application. Some of our cloud applications, such as Webex and Webex Teams, are prime candidates because they are secure and contain higher bandwidth video traffic. These applications have been approved by Cisco InfoSec and implemented by IT, and we determined they will deliver acceptable performance on the DIA circuit.
Cisco SD-WAN will also help us reduce network delay associated with data backhaul, which will improve the user experience for cloud-based software as a service (SaaS) applications such as Cisco Webex and Office 365. To offload that traffic, we will use split tunneling at the remote office DIA circuit. This design will allow us to take advantage of reduced latency delivered by regional content delivery network (CDN) providers for SaaS applications.
Based on our initial testing, we expect that greater use of the DIA circuit in remote offices will reduce our global, aggregate bandwidth usage by approximately 25 percent. The bandwidth reduction alone won’t directly produce cost savings due to the way circuits are priced. However, we expect to continue optimizing and shrinking the expensive primary circuits by sending more traffic to the more economical DIA circuits, which we expect will add up to 30 percent savings in service provider costs over time.
We will deploy the early Cisco vEdge solution in more sites before transitioning to the full SD-WAN IOS software in the remote office routers. We will also expand the test deployment of hardware-based Cisco vEdge routers to 25 remote Cisco offices by late 2018, allowing us to better understand and provide feedback to the product engineering teams about the operation of these routers at enterprise scale.
Our long-term plan is to deploy this SD-WAN solution to all of the smaller Cisco offices worldwide, upgrading the Cisco ISR 4451 routers with the Cisco IOS-XE software to operate as a single, converged access solution. We further expect to reduce costs by transitioning our smaller offices to Cisco 5000 Series ENCS servers running in router mode using Cisco ISRv software and transitioning all remote-office backup circuits to DIA connections.