Cisco on Cisco
IP Contact Center Case Study: How Cisco IT Migrated High-Volume Linksys Contact Center to IP Network
Cisco Unified Contact Center solution receives and routes 40,000 calls per day while providing greater flexibility and availability.
When Cisco Systems® acquired Irvine, California-based Linksys Group, Inc. in mid-2003, Cisco® IT acquired an unprecedented challenge: integrating a contact center whose volume surpassed that of the other 60 Cisco contact centers combined.
“At the time of acquisition, the Linksys contact center served more than 16,000 calls per day, which made upgrading our contact center technology critical to our ability to scale the business,” says Steve Gordon, vice president of Linksys Services. “We needed a scalable call center solution that also would drive operational efficiencies over the long term.”
At the time of the acquisition, the Linksys call center operation, located at the company’s Irvine headquarters, routed 99 percent of incoming calls to three outsourcers in India and the Philippines, with 90 Linksys agents on staff to resolve escalated calls. The company relied on a traditional private branch exchange (PBX) switch that had reached capacity and would need a major hardware and software upgrade to meet growing call volume. “With 18,000 calls a day in 2004 and continuing growth, we needed the new call center to have the capacity to handle 40,000 or more calls a day,” says Mark Dilday, Cisco IT project manager. “To put that in perspective, the combination of 60 Cisco contact centers worldwide serves fewer than 18,000 callers a day.”
Another factor driving the upgrade was a need to preserve business continuity. In mid-2004, California was experiencing widespread electrical outages, and Linksys needed a call center solution that would not fail—even in the event of a power outage. The timing was critical: August 1, 2004 marked the beginning of the academic year in California, and Linksys would field an additional 10,000 calls daily, bringing the total to 30,000, a volume far beyond the capacity of the existing PBX switch. “The challenge for Cisco IT was to deliver a more flexible, robust, and available call center solution that preserved and enhanced the existing solution, and to deploy it in time to meet the increased demand that the August 1 date would generate,” Dilday says.
In 2000, Cisco had migrated all of its contact centers around the globe to a Cisco IP Contact Center (IPCC) environment to gain the benefits of centralized management and support from the Cisco Global Support team. Cisco IT and the Linksys team decided to deploy Cisco IPCC as well. “When you have an existing support infrastructure, it cuts costs and improves operational efficiencies to use it,” Dilday says.
Vice President of Linksys Services
In planning the transition, the first decision Cisco IT had to make was whether to perform pre-routing or post-routing. In high-volume contact center environments, the preferred option is often pre-routing, in which the carrier hosts a menu of options from which the caller selects, and routes across the carrier network to the appropriate endpoint. The carrier sends the caller’s selection to the Cisco Intelligent Contact Manager (ICM), which consults a script before returning call-routing instructions to the carrier. The call itself never touches the Cisco WAN unless that is its destination. The Cisco Technical Assistance Center (TAC), for example, uses a pre-routing architecture. “By keeping the customer call in the service provider’s network, pre-routing reduces the amount of hardware and number of circuits needed at the contact center locations,” says Franklin King, engineer with the Cisco Contact Center Application Technology (CCAT) team. “The resulting cost savings can be considerably greater than the carrier’s per-call charge for pre-routing.”
Figure 1. Linksys Contact Center Architecture
Pre-routing was not an option for Linksys, however, because the company had a contractual agreement with a carrier that did not offer the service. Therefore, Cisco IT would need to build an infrastructure that could first receive and then route 40,000-plus calls a day. The one percent of calls destined for the Irvine staff would travel over the Cisco WAN, while Cisco (rather than the service provider) would route the remaining 99 percent to the appropriate outsourcer, based on callers’ menu selections.
Next, the Cisco IT team had to decide where to physically deploy the Cisco IPCC solution. Bill Lowers, a Cisco IT engineer, had designed all of the other Cisco IPCC deployments. “Our first inclination was to locate the contact center in Irvine, but the existing Linksys data center facility wasn’t big enough to accommodate the circuits and switches required for a post-routing architecture,” he says. “And because 99 percent of calls would be redirected to outsourcers anyway, there was no real gain to locating the infrastructure in Irvine compared to any other location.”
Ultimately, Cisco IT decided to deploy the Cisco IPCC in Research Triangle Park (RTP), North Carolina. “RTP already provided redundant data centers, each with its own circuits, and multiple routes to the local exchange carriers,” says Lowers. “We’d be able to deploy a redundant configuration to ensure the contact center could continue operating even if only one data center became available.” Another factor in RTP’s favor was that Linksys’ carrier had better regional presence in that area than it did in San Jose, California (Cisco’s location), which would make it easier to provision the necessary circuits.
To size the system, Lowers relied on the Voice Technology Group (VTG) design guide (http://www.cisco.com/en/US/netsol/ns340/ns394/ns165/ns268/networking_solutions_design_guidances_list.html ). Based on anticipated volume, he deployed the following (Figure 1).
- Three primary and three backup Cisco CallManager servers, and one publisher/Trivial File Transfer Protocol (TFTP) server
- Four Cisco IP Interactive Voice Response (IVR) servers
- Two Cisco ICM Peripheral Gateway servers
- Two Cisco Catalyst® 6513 switches with Cisco 6608 T1 gateways
“The post-routing deployment requires a lot of hardware,” says Lowers. “With a post-routing architecture, all calls terminate at the cluster in RTP, and then turn right around and go back out again. If the carrier supported pre-routing, then the vast majority of calls would have been routed by the carrier rather than Cisco, and the company could have handled the volume with two Cisco CallManager servers and one or two Cisco IP IVR servers.”
For redundancy, the servers were split between the two RTP data centers. “Ordinarily, Cisco achieves redundancy by rerouting calls to other locations with existing servers,” says Lowers. “But the Linksys call center is centralized in one location, so we had to add enough servers within the cluster to ensure the call center could continue to function if one RTP data center went down.”
Cisco ordered and provisioned six DS-3 circuits: three inbound to RTP and three outbound to the Philippine outsourcer in Manila and the two Indian outsourcers’ points of presence in California and New York. The inbound circuits are expected to handle 40,000 calls each. “We might not ultimately need the capacity of a full DS-3, but it’s less expensive than 15 T1s and faster to provision,” King notes.
While his teammates planned and began deploying the infrastructure, King conducted a full discovery process, working with Linksys contact center managers and the Linksys PBX support engineer to methodically review scripts and call flows from start to finish. It was an enormous task. The main menu contained seven or eight options, some of which had four levels of customer choices branching from them. “When a contact center has been in existence for several years, the logic behind how the contact center has been organized is not always obvious,” he says. “The most important part of a discovery session is understanding why it has been structured this way.”
King diagrammed the options so he could rebuild them in a completely new application, and recorded the verbiage for the IVR menus in .wav files. Then, after recording the existing logic, King recreated it in the Cisco ICM environment. The process took approximately 350 hours. “What it had taken Linksys six years to develop, I had to recreate in a few months,” he says. “In total, we cut over more than 62,800 numbers from their previous Linksys system to IPCC, each with its own Cisco ICM scripts, to provide the call routing logic. We also eliminated over 85 IP IVR scripts, which define menuing and queueing.” The routing logic included French and Spanish menu options for presales and product support.
Because the immediate goal was to replicate the existing contact center capabilities on Cisco IPCC, King did not immediately suggest additional enhancements he felt the Cisco IPCC could enable. “The objective was to transition the Linksys contact center from the PBX as quickly as possible, to prevent a loss of business continuity resulting from high volume or environmental conditions,” he explains. “We wanted to do it quickly, with a minimum of error, and to avoid any change to the customer experience. Enhancements could come later.”
Cisco IT migrated the technical support section of the contact center operations in July 2004, and presales and customer service in August 2004. Agents were trained on the use of Cisco IP phones and the Computer Telephony Integration Object Server (CTIOS) agent desktop, and managers were trained in generating reports using Cisco WebView software. “Reports are essential for contact center management, and WebView makes them easy to generate on demand,” says Dilday.
At the time this case study was written (March 2005), the Cisco IPCC solution for Linksys had routed 2.2 million calls.
With its existing call center, Linksys instructed its carrier to automatically redirect a specified percentage of incoming phone calls to outsourcers. Adjusting the percentage required call center managers to send a request to the carrier, which might take as long as 48 hours to fulfill. In the interim, caller wait times could increase, thus decreasing customer satisfaction. “With Cisco IPCC, contact center managers can adjust the percentage themselves, in two minutes rather than two days,” says Dilday. “They also can adjust the number of calls to be sent to each outsourcer—if, for instance, one outsourcer loses its voice circuit from the U.S. to India—on a minute-by-minute basis without any involvement by the carrier.”
That flexibility applies to call flow as well. “Traditionally, contact center managers don’t have access to the PBX,” says King. “Now, with the Web-based Cisco ICM interface, contact center managers can make changes from their laptops, wherever they are. We’re taking the business support model to a different level.”
When Linksys introduces a sales or marketing promotion, King can activate one of several 800 numbers for this purpose in just hours, compared to the days such an activation would have required before. The numbers point to a predetermined dialed number identification service (DNIS), and King simply changes the DNIS and notifies the carrier. “This capability of Cisco IPCC is especially valuable for consumer businesses, like Linksys, that often need to react quickly to market opportunities with special offers,” he says.
“We have redundancy in the data centers, servers, and telecommunications company facilities,” says Dilday. “The infrastructure is much more stable than the PBX switch Linksys had been using.”
If problems do arise, they are easier to resolve now that the Linksys contact center is part of the Cisco Global Contact Center operations. In the past, the Linksys contact center reported trouble to its staff PBX engineer, who was on call 365 days a year. When the contact center became part of the Cisco operation, Cisco IT implemented new, repeatable processes for reporting a P1 outage or a standard “break-fix” trouble ticket, for example.
Moves, adds, and changes are simpler as well. With Cisco IP phones, contact center agents can move to different desks in a few seconds, simply by disconnecting their phones and reconnecting them in the new location, rather than submitting telephone move requests to change the wiring in a closet.
During any acquisition, IT should be prepared to be both flexible and accommodating, and contact centers are no exception, according to Dilday. “Going into the project, we had our own ideas about the ideal contact center implementation, which in our minds included pre-routing,” he says. “In this case, the customer had other needs that dictated post-routing. Cisco IPCC gave us the flexibility to meet either requirement.” Dilday advises remembering that in a post-routing scenario, more hardware is required, which means IT needs to plan adequate support resources for server security, patch installation, and operating system upgrades.
Another lesson learned involves the primary importance of cost for a consumer-oriented contact center, a new model for Cisco, whose contact center model is based on service quality. “This was our first endeavor supporting a consumer contact center, where priorities are different,” says King. “For example, for Cisco internal contact centers, 35 percent operating expense is acceptable. For this consumer-oriented contact center, we had to hold operating costs to 10 percent. For example, while it makes business sense to incur a $15 call center cost for a $1 million (Cisco) networking device, it doesn’t for a $50 (Linksys) home router.” Cost drove some of the Cisco IT team’s design decisions, such as ending responsibility for a call when it was delivered to the Indian outsourcers’ domestic points of presence rather than their locations in India. “The quality between the domestic point of presence and India was the outsourcer’s responsibility, and the trade-off was made because of the cost savings it delivered,” says King.
Dilday and King both noted that the Cisco Vendor Management Office proved extremely helpful in the migration process. The Cisco IT Vendor Management Organization was formed to provide a consistent and clear process to engage with vendors, and to identify and partner with critical strategic vendors to both companies’ mutual benefit.
“The Vendor Management staff has cultivated relationships with carriers that help them provision circuits efficiently and at the lowest cost,” says King. “It’s mandatory for Cisco IT to get the Vendor Management Office involved early in the process in order to optimize costs.”
The entire team agrees on this lesson learned: When Cisco acquires a contact center, there is no such thing as providing too much training and education about the capabilities of Cisco IPCC. Explaining the relative advantages of pre-routing and post-routing is especially valuable. “Some contact centers, like Linksys, use post-routing exclusively, while others, like Cisco TAC, rely on pre-routing,” says Lowers. “One solution isn’t always clearly the better solution, so giving the customer all the information they need to make the best decision is crucial.”
Enable dynamic routing at the outsourcer locations, using Cisco ICM peripheral gateways, which “speak the language of the legacy PBX at the other end,” according to Lowers. “Instead of simply passing a call to the contact center, we’ll have the insight to know which resource is best equipped to receive the call so we can make more intelligent routing decisions.”
Adopt pre-routing, after working with Linksys’ carrier to add this capability.
Enhance computer telephony integration. “Linksys is very enthusiastic about its new contact center solution, and one reason is that it truly is a contact center, not just a call center,” says Dilday. “The new contact center technology provides the ability to access a database to identify callers by their phone numbers and send a screen pop of that customer’s call history to the contact center agents’ screens. That’s a big advantage for customer service, and allows us to respond to each caller appropriately.”
Extend the Linksys contact center to Latin America and Asia Pacific, where the company is expanding. King has already built a small contact center with 25 agents in a Cisco sales office in China, which works off an existing Cisco IPCC Express server in Sydney, Australia.
Evolve to become a Customer Interaction Network, which will allow customer contacts to be conducted over e-mail and fax as well as phones. “Contact centers are not just in the business of dealing exclusively with phone calls anymore,” says King. “Instead, their role is to deliver different media sources to the agent at the end. Cisco IPCC offers a multitude of paths for growth.”