Cisco on Cisco
Collaboration Case Study: How Cisco Deploys and Supports Its Global Telepresence Service
Cisco IT turned to the Cisco Services group to plan, design, and implement a smooth, globally replicable telepresence deployment and support model.
Let us know what you think.
Take a quick survey
In December 2007, Cisco IT was seeking ways to help the company significantly reduce costs as economic downturn loomed. Travel costs were identified as an area where savings could be quickly achieved. However, it was still important to maintain personal connection and face-to-face interaction whenever possible, and management teams wanted to be able to visit with customers more frequently than time and travel budgets allowed.
Cisco IT decided that it could achieve both goals using Cisco® TelePresence, an ideal solution for supporting face-to-face collaboration. Cisco IT planned to deploy systems in several global locations. However, deploying telepresence systems to connect employees worldwide was a daunting task. There were numerous, global stakeholders in the project. Deployment locations also had to satisfy several characteristics to be chosen. Besides Cisco world headquarters in San Jose, California, the team looked at other locations with high volumes of travel. However, high traffic volumes alone were not enough to qualify a location for Cisco TelePresence. The location also had to have the network capacity to support a Cisco TelePresence system. For example, a location required network connections of at least DS-3 (45 Mbps) and a site network with sufficient capacity to support at least one Cisco TelePresence System 3000.
Physical deployment worldwide would require assistance from local Cisco IT teams and potentially from outside partners. Once the initial systems were deployed, Cisco IT expected rapid growth, and the implementation model had to be replicable globally. Finally, Cisco IT wanted to be able to monitor and support each Cisco TelePresence room as efficiently as possible.
Cisco IT turned to Cisco Services for assistance. The first step in working with global stakeholders was to partner with Cisco’s Communications and Collaboration group and with Cisco IT’s Network, Compute, and Storage group to begin the planning, design, and implementation process.
Cisco Services began the engagement with the planning and design phase. During this phase, the team performed an architectural evaluation and network path assessment to qualify the network and physical meeting locations and verify that they could support the Cisco TelePresence solution. During the assessment, the team emulated video traffic on the network to gauge its ability to handle telepresence sessions. The assessment revealed that some site networks would require upgrading from DS-3 to OC-3 (155 Mbps) to support the planned Cisco TelePresence deployment, and that much of the North American WAN would need to be upgraded from OC-12 (622 Mbps) to at least OC-48 (2.5 Gbps).
Senior Vice President and Chief Information Officer, Cisco
"Taking an architectural approach is essential to a successful implementation," says Rebecca Jacoby, senior vice president and chief information officer at Cisco. "Cisco Services' extensive experience enabled us to identify potential issues in advance and accelerate deployment as a result."
In the implementation phase, the project team developed an implementation-ready detailed design for the overall Cisco TelePresence solution and a comprehensive network implementation plan for each element of the solution, including audiovisual and environmental standards. In addition, the Cisco quality of service (QoS) architecture was redesigned, based on recommendations that had been made to customers with similar needs. Cisco IT arranged for installation to be performed by Cisco Authorized Technology Provider partners, firms that already possessed relevant credentials in video and telepresence technology.
The first four systems were deployed in San Jose, Bombay, Hong Kong, and New York. After deployment, the Cisco Services team performed a system acceptance testing process, and finally, delivered customized training materials and hands-on education to make sure that system administrators, support staff, and end users all could make full use of the Cisco TelePresence technology.
Deployment was then expanded in the United States, Europe, and Asia. During the first year of deployment, demand for the use of newly installed Cisco TelePresence systems quickly outstripped availability. And because network upgrades were required, it took approximately a year to have circuits upgraded. During that time, Cisco Services worked out a process of prioritizing calls so that telepresence sessions with customers received top priority. Once the circuits were upgraded, the prioritization was no longer required.
- Reduce travel costs while retaining important personal face-to-face interaction with customers.
- Deploy TelePresence at many locations worldwide.
- Implement a globally replicable model that Cisco IT could easily support.
- Cisco IT partnered with Cisco Services to plan, design, and implement a replicable process for deploying and supporting Cisco TelePresence worldwide.
- TelePresence has saved Cisco US$566 million in travel costs since December 2007.
- Local Cisco IT teams can now install TelePresence systems themselves.
- Cisco Services integral to the successful deployment. Cisco IT continues to use the same packaged services and processes as Cisco’s large enterprise customers.
- Three-tier support model ensures excellent user experience.
- Cisco's TelePresence deployment continues, with an expected 1200 rooms deployed by July 2011.
Cisco Services began supporting fewer than 100 Cisco TelePresence System 3000 and 1000 units, as well as a few Cisco TelePresence System 500 units. The support model included:
- Cisco TelePresence Remote Management Services: These services include a team of experts who monitor the network at each site from the Cisco global network operations center (NOC) using advanced proactive monitoring tools with Expert Access Service monitoring. Remote Management Services supports Cisco TelePresence Systems, Cisco TelePresence Recording Server, Cisco Unified Communications Manager, and Cisco TelePresence Multipoint Switch systems. Its engineers function as an extension of the local Cisco IT team and help anticipate, identify, and resolve issues quickly. They also provide bug identification and troubleshooting, post-upgrade testing, and performance trending. Tier 1 and 2 services are provided 24 hours a day, and users can also call for assistance when needed.
- Cisco TelePresence Essential Operate Service: This service provides global 24-hour-a-day access to highly skilled engineers, creating a comprehensive support environment that addresses the voice, video, software, and hardware elements of the Cisco TelePresence solution.
Since its initial deployment, the use of Cisco TelePresence at the company has expanded rapidly. Between 2008 and 2009, 600 units were deployed. As of 2010, the number of systems deployed has grown to 922 units in production and under support.
At first, Cisco Services was deeply involved in the early implementations with Cisco Authorized Technology Provider partners. Now a standard checklist accounts for all elements of bandwidth, location, and room readiness, and allows local Cisco IT teams to install systems themselves. Cisco Services becomes involved in deployments and architectures that require new technologies, multiple service providers, or large venues.
Today, the support model has added proactive monitoring. Each Cisco TelePresence room is in contact with Cisco Remote Management Services, which interprets alerts and responds accordingly. At the same time, any user in any TelePresence meeting can push the “LiveDesk” button on the Cisco IP phone and immediately receive assistance.
Cross-technology collaboration extends proactive monitoring and troubleshooting capabilities beyond Cisco’s walls. If a carrier network issue arises, the carrier providing the circuit to the room proactively notifies the Cisco TelePresence team that a circuit issue could potentially affect the gateway, which in turn could affect a TelePresence room. If a problem is identified in the internal network, it is referred immediately to a Cisco IT team.
Approximately 160 executives and strategic customer briefing centers receive video-enabled support. Pushing a button on the Cisco IP Phone summons an engineer to appear on the Cisco TelePresence screen to answer questions and quickly resolve any issues. This unique application uses the Cisco TelePresence system itself to support users.
Approximately 50 percent of Cisco TelePresence customers, including Cisco IT, use Cisco Remote Management Services for their installations. Remote management capabilities allow customers to use Cisco expertise and experience to help ensure an excellent experience while maintaining control over their systems.
Within Cisco, the availability of TelePresence has become integral to many facets of the company’s business. TelePresence is now used for:
- Supply chain management discussions: for faster decision making
- Sales and product training: to present multiple experts to customers from all over the globe
- Technical troubleshooting: to bring customer and Cisco technical resources together to solve issues
- Interviewing candidates: to review a wider pool of applicants in a shorter amount of time
- Press and analyst briefings: to connect with influencers around the globe in one day from one location
- Product demonstrations: to scale scarce expertise around the globe
Since December 2007, there have been 645,868 TelePresence meetings scheduled. Cisco has saved US$566 million in travel costs, with an average of four participants per meeting and an average cost per trip of $1000. Participants in 138,993 meetings have been able to avoid travel, and Cisco CEO John Chambers now spends one-third of his time using Cisco TelePresence, instead of traveling for face-to-face meetings.
Hundreds of special events are conducted each year with Cisco TelePresence. In addition to permanent TelePresence rooms, the company employs temporary systems for large events and mobile units for rear-screen projection in large conference centers.
Cisco’s experience has validated Cisco Services’ approach to planning, designing, and implementing successful TelePresence deployments. The most valuable lessons learned during the past two years include:
- Best practices: Using Cisco Services’ architecture and Plan, Design, and Implementation Service helps ensure smooth, successful deployments. Cisco IT uses the same packaged services and processes as Cisco’s large enterprise customers.
- Three-tier support model: A three-tier support model helps ensure that Cisco TelePresence users always have an outstanding experience. Tier 1 support provides help for all users who call the Remote Management TelePresence Service Desk for assistance. The service desk routes the call to the appropriate group within Cisco for resolution if the service desk cannot resolve the issue. Tier 2 calls include new issues, which might indicate a bug. These calls are routed to the Cisco Emerging Technology IT Operations group for assistance. If the issue requires escalation, a Tier 3 call is sent to the appropriate business unit or to the Technical Assistance Center (TAC) for return materials authorization (RMA) or technology replacement. Regardless of outcome, the Tier 1 group follows the case through to closure.
Cisco continues to deploy TelePresence systems at a more leisurely pace, with the largest amount of growth occurring in the United States. The company had approximately 1050 rooms deployed at the end of October 2010 and expects to have more than 1200 rooms deployed by July 2011.