Global Crossing had been looking for a solution that could bring the predictability and simplicity of capacity and failure planning for legacy transport to its IP networks. In Cisco MATE software, the company found its answer.
The Challenge at Hand
The deployment of IP networks on a global scale has led to some fundamental conundrums as to how carriers should configure their networks and deploy extra capacity to protect traffic during planned outages or failures within the network. On the one hand, spontaneous, unplanned over-provisioning, the prevalent methodology for IP networks today, is not precise or predictable enough to provide sufficient quality guarantees and has proven to be too expensive. On the other, detailed routing and traffic management practices borne out of leased line and telephone networks are cumbersome in the IP context and require considerable operational infrastructure.
Global Crossing has forged a third path by developing a set of methodologies around scientifically advanced software solutions provided by Cisco. The process uses mathematically sophisticated procedures to understand network behavior and fill in the gaps of detailed data and control not conveniently available in IP networks (Figure 1). The result provides service quality guarantees with minimized expenditures and operational simplicity.
Optimal capacity planning is something that occupies the minds of network managers as they grapple with how to provide adequate protection to help ensure continuity of services in the event of an outage. In the legacy transmission world, this is relatively easy: you know what point-to-point routes are the most heavily utilized, and you provide back-up capacity accordingly.
Not so in the world of IP. Yes, you can provide disaster recovery by constructing the network in a series of interconnecting rings or by laying dual routes on major subsea cable systems. But in the event of a break, traffic is automatically switched to alternative routes in a way that is complex to track; each packet finds its way through the network along the path of least resistance, to be reassembled at its destination in a logical flow of voice, data, or video.
At the center of the issue is the packetized nature of IP traffic. In traditional leased line networks, traffic flows can be accurately monitored on each individual circuit to prevent congestion at pinch points in the network. As a result, guaranteeing levels of network availability and service to customers is relatively easy.
In the IP world, however, streams of data and voice are broken down into packets and transmitted along diverse routes, making tracking and capacity planning much more complex. Little understanding has been achieved of the methodology required to master capacity planning in a dynamic IP network environment. Although the total amount of traffic flowing over a network may be quantified with a reasonable amount of accuracy, gauging traffic flows and peak demand along specific routes is difficult. Simple guidelines, such as ensuring that all network links are less than 50 percent utilized in normal operating conditions in the hope that this provides sufficient bandwidth on all routes in the event of a network failure are, at best, inefficient and, at worst, ineffective.
Figure 1. Global Crossing's IP Backbone Designed to Withstand Multiple Simultaneous Failures within Network
The current state of the telecommunications industry means that capital expense budgets have to be spread thin. In the past, deploying huge amounts of bandwidth may have solved the problem of congestion, but it was not an inexpensive solution. Today, need for precision in network planning is even more acute. With the growth in traffic volumes rapidly outpacing the growth in revenues that can be derived by delivering it, network operators have to ensure every infrastructure investment achieves the maximum return on investment. Deploying additional bandwidth “just in case” is not an option.
When Global Crossing started to work with Cisco in 2004, traffic was climbing at over 200 percent per year. It has rarely slowed down since as the world’s demand for IP transport continues to climb unabated (Figure 2). At that time the company had recently been listed as a tier 1 ISP with over 60 Gbps external bandwidth at 149 private peering interconnections with over 40 ISPs, and nine public peering points throughout the world. Voice over IP traffic was also growing rapidly and contributing heavily to the need for greater visibility and control of the IP infrastructure investment requirements.
Figure 2. Traffic on Global Crossing's Network
In IP networks, traffic routing is controlled by setting the metrics and preferences of the internal (IGP) and external (BGP) routing protocols. Finding the right configuration to prevent congestion can be very difficult and, for a human, it is not possible to take all possible failure scenarios into account as well. The challenge has been to automate this process, in effect, elevating the control of IP networks from an art to a science.
Bryan Thompson, IP capacity planning manager at Global Crossing, picks up the story: “Two of the most crucial aspects of our business are network reliability and network performance. Global Crossing operates a global network that spans 87,000 route miles across four continents, containing over 700 cities in more than 70 countries. It's my job to ensure there is enough capacity in our network to maintain the highest quality of service standards for our customers at all times. We aim to deliver better than 99.999% availability. It's a matter of supplying adequate capacity to the right places at the right times, and at the lowest possible cost.”
Until Cisco brought its MATE software to market, no non-intrusive methods existed to isolate city-pair flows or optimize routing for IP networks. Cisco’s software includes features that make it easy to get a snapshot of the network, isolate failure congestion points, design for multiservice resilience, and control IP routing in a scalable way.
This tool maps the network, pinpointing the location of all switching equipment and links. Traffic flows between all points on the network are measured at five-minute intervals, and the overview of the network is refreshed on a daily basis. “What we are looking for is the demand levels on all routes,” says Director of IP Network Architecture and Planning at Global Crossing, Thomas Telkamp. “We can now simulate how the network deals with traffic flows and predict the actual paths, the big difference being that what formerly took over 100 minutes, now takes less than four minutes.”
Global Crossing has avoided forcing unwieldy monolithic management and control systems onto functional groups with diverging needs, and instead relies on operationally viable control mechanisms. Simple-to-use systems such as MATE software have been adopted based on individual needs.
Cisco’s product is used across departments: network planning uses the capacity planning features to minimize capital expenditures, while the engineering group in charge of VoIP uses the backup path planning features to ensure resilience by calculating primary and backup paths that do not share facilities. Simulation features are used across groups to minimize the impact of maintenance operations.
The ramifications of either increasing capacity or adding additional links between two cities can be modeled to see if pinch-points have been eradicated. If the solution is seen to work, then capital dollars can be directed precisely to areas of the network where it is needed most and will have the biggest impact. The system works in a cyclical fashion: the network feeds information into the tool, which then formulates suggestions for network optimization that are then tested and deployed. Network resiliency can be planned well in advance by precisely identifying sections of the global network where backup is needed.
The Cisco MATE software sits at the center of the planning process: Recommendations on network architecture and Multiprotocol Label Switching (MPLS) configuration are tested before being implemented into the network (Figure 3). In turn, information such as configuration files, router links, and statistics on traffic flows are fed back into the MATE software.
The Cisco MATE software’s ability to simulate and solve routing problems over IP and MPLS networks and perform multiservice capacity planning has enabled the network planning and product development groups to work in tandem in such a way that the impact of a new service on capacity requirements can be forecast six months or even a year into the future. This capability has afforded Global Crossing the opportunity to manage capital outlay with a degree of precision never before thought possible.
“Instead of throwing bandwidth at the problem, our process can direct limited capital dollars to points on the network where it is going to have maximum benefit,” says Telkamp. “The days when over-provisioning bandwidth was the only answer to optimal network performance is in the past. We can safely say that we have just the right amount of capacity on the network. It’s financially reassuring that we can now work out precisely how much money we have saved by judicious network expansion.”
The telecommunications industry is at a critical juncture today. Service providers must re-examine policies and processes to determine how new IP-based services can be delivered in a reliable and profitable manner. Using Cisco’s MATE software, Global Crossing has accomplished this and has achieved the multiple goals of reducing the cost per megabit and having a scalable network architecture with predictable network performance.