Achieving Reliability and Service Quality Using the Cisco Unified Communications Management Suite
To address the challenges of proactively monitoring the very large Cisco Unified Communications global network and to achieve reliability and service quality goals, Cisco IT turned to the Cisco Unified Communications Management Suite.
The Cisco® internal Unified Communications network is a very large global deployment, having more than 85,000 IP phones spread over six continents.
Cisco IT faced many challenges in managing the large unified communications environment, including monitoring the large, multi-site, multi-cluster network and helping ensure that reliability and service quality goals were met. To keep pace with a very aggressive deployment schedule, they had to go in a few weeks from an initial scenario of not having any Cisco network management applications to one in which they had extensive proactive management and service quality monitoring capabilities.
Being in new and uncharted territory in the initial stages, the Cisco IT team first had the challenge of defining methodologies and metrics for measuring and monitoring the key components of service quality:
• Availability (Can I pick up the phone and get dial tone? Does my call connect? Can I leave and retrieve voicemail?)
• Performance (How is the voice quality? Did I get disconnected mid-call? How does the voicemail sound? How fast is the voicemail prompt?)
The team also had the additional challenge of figuring out approaches, techniques, and tests to:
• Detect configuration issues
• Reduce recovery time in the event of a service failure
Network management design and technical architecture work had to be done and then implemented to help ensure that stringent goals related to application responsiveness and service availability were met.
The Cisco IT team had a critical need to gain visibility into the global unified communications network status, voice quality, and trends, and to accurately measure and manage end user experience service levels. Furthermore, they were also in need of troubleshooting capabilities to aid in rapid problem resolution.
With their reliability and service quality goals in mind, the Cisco IT team defined specific requirements, methods, and procedures, and chose Cisco Unified Communications Management Suite products to effectively address their needs for service quality metrics measurement and diagnostics.
The team implemented a network management architecture that has Cisco Unified Operations Manager running in each of the six Cisco IT data centers, distributed worldwide. (Figure 1) These Cisco Unified Operations Manager instances provide a real-time view of the network, perform end-to-end and synthetic phone tests to measure service availability and performance, and provide a rich set of diagnostics capabilities for troubleshooting. Every minute of every day 175 test calls are initiated across all global theaters of Cisco IT's Unified Communications deployment, and provide crucial information for detecting availability, performance, and service degradation issues. Test failures provide an email notification for operator action. Additional tests are performed as needed using Cisco Unified Operations Manager for troubleshooting purposes, including phone registration, Message Waiting Indicator (MWI) testing, conference connection, and emergency calling tests.
The Cisco Unified Service Monitor product (another component of the Cisco Unified Communications Management Suite) is used for real-time voice quality monitoring. Forty Cisco Unified Service Monitor 1040 sensors spread across the Cisco Unified Communications deployment collect Mean Opinion Score (MOS) information, which is used to provide detailed voice quality reports. Voice quality degradation alerting thresholds are configured based on specific selection criteria, allowing more focused alerting to facilitate troubleshooting of voice quality issues. The 1040 sensors are also used for detailed data collection on specific network spans, application services, and trouble spots; for service quality monitoring; and for detecting and troubleshooting service quality issues.
Using the Cisco Unified Communications Management Suite products, Cisco IT measures and meets its stringent reliability and service availability goals.
With the Cisco Unified Communications Management Suite products, Cisco IT is able to:
• Proactively identify voice design and implementation problems, which include configuration issues such as:
– Incorrect device setup
– Incorrect gateway configurations leading to low MOS
– Dial plan misconfiguration
Cisco Unified Operations Manager provides extensive diagnostics and dial-plan test capabilities that test and validate configurations for different partitions and locations, and provide a quick assessment on whether dial-plan configurations are being correctly applied on the phones.
• Reduce recovery time. Cisco Unified Service Monitor service quality thresholding and reporting features, and Cisco Unified Operations Manager fault correlation and extensive diagnostics capabilities, have saved many hours when troubleshooting, resulting in rapid root cause identification and recovery
• Monitor and improve unified communications system reliability, availability, and end user experience on a continual basis
– Gain visibility into the global unified communications network status, voice quality, and trends
– Use Cisco Unified Operations Manager end-to-end call tests and Cisco Unified Service Monitor call quality data collection to accurately measure and manage service quality levels
– Use Cisco Unified Operations Manager diagnostics and troubleshooting capabilities to aid in rapid problem resolution
The Cisco IT environment features the latest FCS releases of Cisco products on an ongoing basis, in order to leverage the most advanced capabilities as soon as they are developed. Cisco Unified Communications Management Suite product release schedules align with and support the latest Cisco Unified Communications system releases. Cisco IT will be upgrading its existing installations of the Cisco Unified Operations Manager and Cisco Unified Service Monitor management products as it upgrades the underlying Cisco Unified Communications system infrastructure. The Cisco IT team also plans to install and utilize another component of the Cisco Unified Communications Management Suite, the Cisco Unified Service Statistics Manager. This product will provide historical unified communications performance reporting on call volumes and resource utilization, and will identify trends in order to upgrade network resources before they run out of capacity. Cisco IT is also investigating the usage of Cisco Unified Provisioning Manager 2.0 in the future and leveraging its programmatic interfaces for integration with Cisco global IT systems for unified communications provisioning and service activation.