Cisco Systems, Inc. recently demonstrated the value of advanced modeling and simulation software when it helped one of its large European service provider customers replace its core Multiprotocol Label Switching (MPLS) network routing platform with a higher-performance one.
The Challenge at Hand
Cisco Advanced Services was engaged to help mobile operator Vodafone Germany migrate from its existing Cisco 12000 Series GSR backbone to a network core based on the newer Cisco CRS-1 routers. The CRS-1 multi-terabit forwarding capacity was quickly becoming necessary for Vodafone Germany as unprecedented volumes of multimedia fixed and mobile traffic were traversing their core network links.
The assignment for Cisco was to design the new network core, physically build it, and create the overall migration strategy, including the order in which nearly 300 PE routers would migrate to the new platform. At stake was the possibility of service degradation for the company’s network users. This degradation could take place while edge devices that were still connected to the old core needed to communicate with devices that had already migrated to the new core over interconnections that had been put in place for the duration of the migration. Twelve points of presence (POPs), each with two redundant core GSRs, were to be upgraded to CRSs (Figure 1).
Figure 1. The New Network Was to be Built Alongside the Old
Because so many edge devices had to be moved and because they were geographically dispersed across Germany, making an overnight switch was not an option. “The goal was to migrate in such a way that the multiple 10-Gbps interconnection links between the old core network and the new one didn’t overrun capacity,” says Oliver Boehmer, solutions architect and Cisco lead for the migration project. Getting the migration wrong would cause the interconnections to become congested, drop packets, and affect service for large numbers of Vodafone’s customers.
To build a plan, Boehmer needed to understand the traffic matrix: the complete listing of the end-to-end traffic flows on the network that would have to be carried. That listing would require some form of measurement on the existing GSR-based network before planning could begin.
Once an acceptable plan was developed, the physical migration would then become a joint effort between Cisco and its customer, Vodafone. Cisco had to develop a plan that could be handled by the small joint team of Cisco and Vodafone staff over a period of weeks.
To build the plan his team required, Boehmer relied on the Cisco MATE software.
“The first ‘killer feature’ of the Cisco MATE software was the ability to derive a traffic matrix that we could use to understand the traffic flows the new network would have to handle,” says Boehmer. “Only MATE offers the ability to do this from just link utilization alone; other tools require more complex measurements.”
By gathering measured link-load data via interface statistics from the existing network and using network tomography to calculate the network path that data would take, the MATE tool produced a traffic matrix of all the flows in the network. This traffic matrix could then be used as the blueprint for the expected traffic during the simulation of the new network topologies while the migration project advanced.
Once the traffic matrix was finalized, Boehmer’s next step was to create a topology model of the new network. Since the CRS-1 based network did not exist yet, using the discovery features available in the MATE software was not an option. The solution was going to be either using the GUI to create each network object, configure it correctly, and add it to the network model, or write a script to do the same thing programmatically. Boehmer chose the scripting option for two key reasons. First, it allowed him to quickly create the new network model in the tool by parsing information such as port numbers and IP addresses from a spreadsheet supplied by Vodafone. Second, because it allowed him to quickly adapt the topology to test the ordering of each migration step. This testing worked by running a simulation of the traffic flows created by each link being brought up to connect a PE to the new core or being taken down to remove it from the old one (Figure 2).
“For every move, we calculated the load of the new network,” he says. “The ability of the Cisco MATE software to be scripted is amazing,” he adds. “It’s very customizable. Its power is its versatility to integrate [the traffic demand] into the workflow in machine-readable form,” which relieved Boehmer from having to perform a lengthy series of manual tasks to create the files and to retrieve the simulation results.
To migrate the 300 edge routers, Boehmer calculated 20 to 30 moves and was able to see how the load would be forwarded through the network and whether traffic would remain in balance. Vodafone’s goal was to keep the interconnect link loads under 70 percent utilization.
Figure 2. Boehmer Simulated Connecting Each Edge Node to the New Network and Disconnecting It from the Old
Cisco MATE simulations helped Boehmer discover that “certain nodes couldn’t move at the same time” without causing link utilization overruns or performance/quality-of-service (QoS) problems. “We tried about 10 different migration orders of equipment before finding the one that met all of the utilization and performance requirements during the transition,” Boehmer says. “Without the functionality of the Cisco MATE Design software, it would have been practically impossible to perform this level of modeling economically”.
It is common for networking equipment vendors to handle complete core network upgrades by building a parallel network and then switching the network over from old to new. However, customers always had to be willing to accept that network migrations could be risky and might affect service delivery during implementation. By using the Cisco MATE software, Cisco was able to provide Vodafone with the confidence that the migration would be trouble-free.
“We did not suffer any downtime or packet drop during the migration process,” says Boehmer. “We could reassure our customer, before the migration project began, that we understood and had managed the risks involved.”
A typical approach is “to use common sense and accept a lot of risk, which would have involved much greater operational effort on the part of the customer [Vodafone],” Boehmer says. Without having done the simulations and understanding what was going to happen to the traffic, Vodafone would have had operations staff watching traffic levels on all links in the network 24 hours a day during the migration. Whenever traffic levels rose significantly on a particular link, staff would have had to have taken action to reroute flows to avoid the danger of congestion affecting services.
With the Cisco MATE solution, however, Boehmer had modeled the network and understood the impact of link utilization for each step of the migration. After the first couple of moves had confirmed that everything was working as expected, the team could monitor traffic flows with high confidence that service would be maintained.
“The planning for this migration project took two to three weeks,” says Boehmer. “Without the MATE software, it might have taken …well, we wouldn’t have done it,” he says, “We would have reverted instead to common sense and greater risk. With this unique approach, we were able to offer our customer more reassurance that the project would be completed without incident.”
The tool was “extremely reliable and delivered what was expected,” he adds. “The customer didn’t have to expend resources on weekends and was highly satisfied, which is what we always strive for.”