Operational Imperative of Automated Active Testing and Monitoring

Executive summary

To stay competitive, telecom operators strive to deliver network services more quickly and with assured quality. Using fully automated and programmable network solutions is therefore a primary necessity.

To increase agility and the rate of delivery, network environments are becoming increasingly software-defined and programmable, and, as a consequence, the rate at which these dynamic environments change is soaring, driven by on-demand requests from end users and automation loops that trigger device reconfigurations. In addition, networks are also subject to other unpredictable events caused by the varying nature of end-user traffic.

Traditional test and assurance solutions built for static environments are not suited for this new, dynamic landscape. Without support for programmability, these traditional solutions prevent agility and innovation.

Cisco® Orchestrated Assurance powered by Netrounds is a concept combining Cisco Network Services Orchestrator (NSO) as the network service orchestrator with Netrounds as the programmable active test and assurance system. Orchestrated Assurance uses Netrounds’ software-based Test Agents, which are suited for both physical and virtualized networks.

Orchestrated Assurance improves your network operations organization and related work processes in several fundamental ways:

• It fully automates activation testing and ongoing active monitoring. This ensures that your services are delivered right the first time and that services continue to function over their lifetimes.
  • By delivering services right the first time, you reduce trouble tickets caused by undiscovered issues.
  • Through automation, you cut overall ticket handling times and reduce escalations by solving issues earlier in first and second line support.
  • With remote escalations capabilities, you reduce field technician costs and efforts. You dispatch field engineers to fix problems, not to find them. Seems more succinct.
• It provides the orchestrator/OSS with direct feedback about service performance as seen from a customer perspective instead of just giving a general view of network and service health.
• It runs on your compute infrastructure and existing bare-metal servers, which lowers the total cost of ownership.

These changes will give a huge boost to your overall operations efficiency and to your ability to control customer experience in an agile and dynamic environment.

This white paper elaborates on the motivation and drivers to start implementing Orchestrated Assurance today.

Introduction

The purpose of this white paper is to describe how Orchestrated Assurance can benefit your organization and assist you in maintaining a competitive advantage in today’s dynamic network environments. It first describes what end users care about and how this relates to service quality assurance though testing and assuring services. The paper then proceeds to explain how you can tackle and solve operational efficiency and other network
quality problems by implementing Orchestrated Assurance from Netrounds and Cisco.

In an accompanying white paper, you will find a thorough walk-through of the principles and fundamentals of how Orchestrated Assurance works. For technical details, refer to this document.

What end users care about

As network operators or communications service providers, your customers expect their services to be delivered instantly after they order them, and they expect those services to work immediately at the time of delivery. In addition, customers expect that problems will be solved immediately if they occur during the lifetime of the service. (See Figure 1.)

The value of Orchestrated Assurance using active testing

The main benefits for you as an operator or service provider, are decreased costs related to your network operations and improved customer satisfaction. (See Figure 3.)

Figure 3. Benefits of implementing Orchestrated Assurance using Netrounds active testing

Depending on your specific network and operational environment, you can expect savings in the order of $1M to $6M over a five-year period. The main contributors to cost savings are:

• Reduced trouble tickets caused by undiscovered issues
• Overall ticket handling times cut and reduced escalation by solving issues earlier in first and second line support
• Reduced field efforts: dispatch to fix problems, not to find them

Refer to a thorough ROI calculation in a separate accompanying document.

Where Orchestrated Assurance fits in

Orchestrated Assurance revolves around two simple, yet very important, themes:

• At time of service delivery or after any changes, validate that the service works across all layers and domains.
• Make sure that the service continues to function throughout its entire lifetime using active monitoring, with notifications should any SLA thresholds be violated or any quality issues occur.

Cisco NSO automates network-wide configuration of network devices and VNFs across domains and network layers. NSO is independent of the type of vendor or services being provisioned. Figure 4 illustrates how Cisco NSO and Netrounds work together to achieve Orchestrated Assurance.

Figure 4. How Cisco NSO and Netrounds work together

To make sure that the service has been provisioned successfully, Cisco NSO requests Netrounds to perform comprehensive and automated tests as part of the service activation process.

After successful verification of service provisioning, Netrounds also provides continuous service quality insight from the end users’ viewpoint through ongoing, active monitoring from the same Netrounds Test Agents already deployed in your network.

This provides Cisco NSO and other network management systems with real-time quality metrics to help discover issues earlier and resolve those issues more quickly.

Orchestrated Assurance positively impacts your network as it adds active testing capabilities in the fulfillment and assurance phases of your service delivery processes. This is depicted as orange building blocks in Figure 5. Note that active testing is complementary to the existing components.

Figure 5. Active testing in the service Orchestration loop

Contrasting today’s traditional assurance with Orchestrated Assurance

Active testing in the assurance landscape

There is some confusion in the industry regarding the term assurance and what it really means. Most network operators already have a wide range of legacy systems in place for fault and performance management, as illustrated by the gray elements in Figure 6. However, as the orange region in Figure 6 suggests, it is clear that active testing is required to answer two critical end-user questions:

• Does the service work at time of provisioning?
• Will it continue to work during its lifetime?

Figure 6. How Netrounds active testing (orange) complements traditional assurance components

Looking at the legacy assurance systems that might exist in your network today:

• Fault Management (FM) is the most fundamental and predominant type of assurance and is based on observing indicators that are available in every network node. These indicators help detect problems or faults with the hardware or software components of the network devices and interconnecting links and interfaces. Examples of fault management solutions include HP OpenView, IBM Netcool, CA Spectrum, and Moogsoft.
• Service Impact is an add-on to fault management and uses a mapping between provisioned services and a network device inventory. If a device reports a problem, service impact systems make a table lookup to estimate which services might be affected. Note that service impact normally does not take into account failover and redundancy protocols that often prevent, the service from being affected. In addition, because service impact is based on faults on the devices specifically, it is completely blind to issues such as QoS misconfigurations, underperforming links, and poor video conference quality.
• Performance Management (PM) is predominantly based around passive monitoring. In this context, passive implies that devices listen to network traffic as it flows through them and store statistics as counters which are available for later polling.
  Devices also normally store counters of observed internal resources such as CPU load, memory usage, and interface queue utilization. These counters are collected by central components of the performance management system to build historical views and to help understand if there is overutilization somewhere in the network and if a capacity increase is imminently necessary. Examples of performance management solutions are CA Performance Management, ZenOSS, SevOne, Infovista, Teoco, and MycomOSI.
• Passive Probing analyzes the content of existing traffic to better understand the mix of various traffic protocols in the data plane. Passive probing is also used to intercept management traffic, especially from mobile networks, to understand events such as failed handovers and dropped calls. Vendors of these solutions include NetScout, Polystar, Tektronix, and Astellia.

The Impossible Mapping Machine

Existing assurance solutions often attempt to estimate how end users are experiencing their end-to-end services by combining and correlating device-oriented and passive infrastructure counters with other alarm sources, syslogs, traps, and inventories. This cumbersome and unreliable correlation activity, and the use of passive, non-real-time data sources that are not naturally related, results in an inaccurate picture of end-to-end service quality as experienced by your customers. This is illustrated in Figure 7.

Figure 7. The Impossible Mapping Machine

Comparing Orchestrated Assurance with traditional approaches

Table 1 sums up how testing and assurance are typically done today with legacy assurance systems and how they are accomplished when you have implemented Orchestrated Assurance.

Table 1. Testing and assurance today and in an Orchestrated Assurance scenario

The most fundamental improvements that Orchestrated Assurance brings to your organization and to your processes are these:

• It enables pervasive automation of testing and monitoring.
• It provides direct feedback to Cisco NSO and other management systems about how the service works from a customer perspective, instead of just showing a general view of network and service health.
• It reduces your capital expenditures because it runs on your compute infrastructure and existing bare-metal servers (standard x86 hardware), rather than expensive purpose-built hardware.

Combined, these changes give a huge boost to your overall operations efficiency and to your ability to control customer experience in an agile and dynamic environment.

Conclusion

The concept of Orchestrated Assurance is key to achieving full network automation. It addresses two main end user requirements for network operators:

• Deliver services right the first time: validate that new services are provisioned correctly and that future changes keep the service intact.
• Find and fix problems quickly: detect quality issues from end users’ point of view.

Table 2 describes the three main benefits you achieve with Cisco Orchestrated Assurance powered by Netrounds.

Table 2. Primary benefits of Cisco Orchestrated Assurance powered by Netrounds