Cisco on Cisco
Collaboration Case Study: How Cisco Transformed Global Sales Event into a Virtual Experience
More than 200 content sessions, alternative reality games, and social interaction in a rich virtual environment motivate and inspire 19,000-plus global attendees.
When more than 19,000 employees gathered for the four-day Cisco annual global sales meeting in September 2009, not one of them had to board a plane, pack a suitcase, or even leave their office. Instead, they attended the first-ever, entirely virtual Global Sales Experience from 89 countries and 600 Cisco conference rooms, spanning 24 time zones.
"We did not know of any global events that gathered this many people in a virtual, interactive way," says Angie Smith, manager, Global Sales Meeting. "Cisco has conducted several virtual sessions over the last 12 months, but none had more than 4000 attendees. So 19,000 was a big step up, to say the least."
The large-scale virtual event was enabled by multiple technologies, including Cisco's own collaboration, security, and business video, among other products. Content for the event was rich and detailed, with more than 200 sessions delivered live or on prerecorded videos on demand (VoD) using Cisco TelePresence, WebEx Event Center, or Cisco TV. Many sessions included social media interaction through chat rooms, blogs, forums, wikis, interactive polling, and instant messaging. The event also featured a solutions showcase, live executive chat sessions, and sales recognition activities.
With WebEx Event Center, session hosts could mix and match prerecorded and live sessions. The host would start a session, launch a video that would play within the WebEx session on attendees' desktops, and then initiate live Q&A with the attendees. "This was a very effective option for hosts who had to repeat their session several times a day," says Chuck Churchill, IS director, Unified Communications, Cisco. "It kept the content correct, and allowed the hosts to conserve their energy and focus for the live Q&A."
"Providing high-quality audio and video and dynamic, varied content to 19,000 attendees puts a huge strain on the network," says Geert Braeckman, network engineer at Cisco. "Because IT was engaged at the planning stage, well in advance of the actual event, we had very good collaboration with key people orchestrating the event, and were prepared to address the challenges posed by such a big audience."
With a 20-week lead time, Cisco IT faced unique technical challenges. Chief among them:
- Plan capacity and ensure 24x7 connectivity in all the locations that employees could attend the event (home, office, VPN, wired and wireless). While employees were encouraged to attend from a Cisco office, IT allowed them to attend from anywhere in the world over a secure connection (VPN). In some field offices where employees congregated, Cisco IT had to ensure that they did not overwhelm the office wireless network. On the other end of the spectrum were attendees working directly from home. Cisco IT could ensure that the VPN infrastructure would support these users, but last-mile issues were outside IT's control.
- Ensure sufficient bandwidth for live and on-demand videos
- Host and load balance content with minimal impact on the network
- Distribute high-quality video over a single network using both external and internal video sources
- Integrate various platforms onto one unified event interface
- Perform load tests on the various contributing platforms, and set up and test 350 global Cisco conference rooms for onsite viewers
- Provide support and proper escalation measures to accommodate thousands of attendees in a virtual environment
"One of our biggest challenges was fitting the enormous amount of content onto the existing infrastructure," says Rene Gonzalez, technical marketing engineer at Cisco. "There was not enough time to add hardware on the Cisco network, so we had to assess where potential weaknesses might arise in the content delivery system, low-bandwidth offices, and the VPN network."
A big concern for the Global Sales Experience planning team was keeping the attention of a very busy and socially driven sales audience for four days in a virtual environment.
"The move to all virtual was significant because of the sheer size of the event," says Christine Castle, operations director, Worldwide Sales Incentive and Engagement Programs. "This is the one time of the year when we have the opportunity to physically, and now virtually, motivate and inspire the sales team, so it was critical to keep our sales team engaged throughout the entire event."
The planning team conducted market research to determine which interactive components would best encourage field members to fully participate in the online experience. "What we learned was that we needed to make sure that the virtual experience was easy to access and use, fun, and made up of rich content," says Castle.
Cisco chose InXpo to provide the virtual environment for the event, which included web, Flash, audio, video, games, and incorporated WebEx and Cisco TV sessions (see Figure 1). Cisco IPS 4200 Series Sensors helped identify and stop unwanted traffic from causing potential harm to the virtual environment. Load balancing with Cisco CSS 11501 Content Switches allowed service enhancements and upgrades to be made within the environment with little or no interruption to the attendees' experience.
An alternative reality game called the "The Threshold" was immensely popular among the sales teams, with more than 13,000 of the attendees actively participating. Described as part "24" and part "James Bond," the virtual game had an engaging interface, stimulating plot, and hidden clues scattered throughout. Participating teams had to collaborate to solve the clues and win the game prizes.
Figure 1. Global Sales Experience Virtual Environment
As mentioned, the global sales event included more than 200 live or prerecorded sessions. Live sessions were broadcast using Cisco TelePresence, Cisco TV, and WebEx Event Center. Recorded content from live sessions and VoDs were delivered over Cisco TV.
To address the technical challenges related to hosting and supporting the global sales event, Cisco IT relied heavily on network readiness assessment, bandwidth optimization techniques such as pre-positioning and caching content, and bitrate management based on content size.
Employees were encouraged to attend the sales event from Cisco offices over wired connections, and could also attend from anywhere in the world over a secure VPN connection. "We did this so that people who typically work from home or live far from a Cisco office could avoid long commutes in their cars, which decreased impact on the environment and helped maintain their work-life balance," says Braeckman.
The option for employees to attend from any location, however, made capacity planning difficult. "We made best-guess projections based on the attendees' country and city location," adds Braeckman. "Once we had visibility into these numbers, our focus was on efficiently delivering the high-bandwidth audio, video, and web content. We wanted to avoid having individual users request content from the source directly."
Bandwidth optimization was critical in delivering all the event content with minimal impact on the network and the end user's experience. Confidentiality of the VoDs and prerecorded sessions presented IT with additional considerations. Cisco IT hosted as much of the sensitive, high-bandwidth content as it could, unencrypted, on the internal content delivery network, and used pre-positioning and transparent caching features of Cisco Application Content Networking System (ACNS) software.
By pre-positioning content close to the users likely to request it, the content delivery network significantly reduces bandwidth usage on the WAN. Using ACNS software, Cisco IT pushed high-bandwidth content to remote branch offices during off-peak hours, such as night time, over low-speed links. After the content was pre-positioned at a nearby Cisco Delivery Content Engine in the branch office, it could be accessed by attendees using a higher bandwidth connection.
Transparent caching helped Cisco IT efficiently distribute HTTP content. With this feature, a user requests the content from the content source. The request is transparently redirected to a branch-office Cisco Delivery Content Engine, which acquires the content from the source. A copy of the content is stored on the content engine, which forwards the content to the user. Subsequent requests from users at the same branch office are handled locally.
Live sessions were broadcast to attendees at Cisco offices via multicast (one to many). VoDs and live broadcasts to remote locations were delivered via unicast to attendees using VPN and wireless connections.
The embed video playback feature in WebEx also helped to optimize bandwidth and protect the confidentiality of session content. This WebEx feature allows users, during a live session, to download the video before it is played, reducing the risk of buffering problems. The session presenter can check the number of users who have downloaded the video and decide when to start playing the video on end stations.
For guidance on the maximum bitrate to use for the content, Cisco IT compared available capacity at the various event-related locations with the expected load based on the geographical spread and bandwidth optimization techniques. Cisco IT assessed the following capacity:
- Network capacity near the source
- International WAN bandwidth capacity
- Regional office WAN bandwidth capacity
- VPN Hardware encryption capacity
- Internet/ISP bandwidth
"We wanted to reduce the bitrate as much as possible to avoid possible network congestion that might cause videos to break up. However we did not want to reduce the video quality too much as that would diminish the user experience," says Roel Bernaerts, network engineer at Cisco. "The challenge was finding a good balance between video quality and user experience."
Cisco IT found that a bitrate around 300 to 350 Kbps struck a good balance for those who attended the event from Cisco offices over wired connections (Figure 2). Attendees connecting over VPN connections, however, could experience lowered video quality, and Cisco IT set user expectations accordingly.
"While we advised employees to connect at Cisco offices, we also wanted to inform users connecting from home that they might experience issues with the video. No major issues were reported by end users, but it was important for us to set their expectations up front," says Bernaerts.
Figure 2. Maximum Bitrate
Cisco IT tested the virtual environment using applications such as load runner, and set up sessions with multiple users to test the capacity of the video loads onto the Cisco network. Because development time was short for some of the applications, not all of them were tested as Cisco IT would have liked, says Gonzalez. "In the future, we need to build in enough room for proper testing and required action steps."
Virtual teams in three Cisco global contact centers provided technical monitoring and user support 24x7 during the event. If an issue arose during a session, rapid assessment of the global infrastructure was required. With global monitoring of all components, support representatives can determine whether a problem is isolated or systemic and pinpoint issues within each layer of the supporting infrastructure.
Transforming the annual global sales meeting into a virtual event yielded Cisco several business benefits, and even greater participation from the sales team than expected:
- 88 hours of consecutive content
- More than 90 percent in cost savings for the event overall
- 211 million air miles saved in travel
- 334,000 hours saved in travel time
- 84,400 metric tons of CO² saved in commute and transportation costs
- More than 13,000 attendees actively participated in the alternative reality game, The Threshold
- 9627 attendees played mini games, with 55,119 views
- More than 9000 attendees participated in group chats (the Chat Zone)
- A total of 17,551 attendees used V-Cards
The large scale and complexity of Cisco's first virtual global sales event yielded lessons that will be applied in preparing and supporting future virtual meetings:
- Test, test, test. Allocate sufficient time for testing the virtual environment with users. The Global Sales Experience team had only 20 weeks to execute the event, and more time should be allocated for user testing in the virtual environment.
- Include social media tools for communicating to the salesforce; do not rely only on email.
- Ensure that all key event, business, and IT stakeholders are involved in the discovery planning phase before determining the event requirements.
- Provide a mechanism for users to enter data regarding their location, and request that they update/verify the data at least once before the event. The support teams can use this data at checkpoint meetings throughout the event to ensure that there is adequate capacity at each location, verify whether the site has been tested, and communicate attendance guidelines or technical issues to participants, as needed.
- Allow for plenty of interaction among the attendees in the virtual environment, e.g., video chat and demonstrations.
- Perform global monitoring of all infrastructure components, and ensure that all the applicable support team reps are on the same call.
- Assign an IT support representative to all locations where sessions are viewed in a conference room.
- Select virtual environment providers with data centers that are sufficiently monitored for attacks, optimal bandwidth utilization, and load capacity.
- Employees mostly wanted to be together for part if not all of the virtual event. Consider this user preference when booking, prepping, and estimating attendance within conference rooms.
The goals for future annual global sales meetings will remain the same: to educate, share, interact, motivate, and inspire the sales teams. Delivering the event virtually opens up new, exciting, and challenging ways for Cisco to meet these goals.
"We are already scoping out virtual scenarios for the fiscal-year 2011 global sales event," says Smith. "We will be working on a model that combines virtual event best practices and live components that truly motivate, inspire, and recognize the salesforce."