Getting started

This section explains these topics:

Important changes in behavior

The following table describes important changes in behavior in this release.

Topic

Description

Discontinuation of Amazon Web Services (AWS) external storage for Service Health historical data.

Service Health historically allowed configuration of external storage using an AWS cloud account to store monitoring data beyond the internal storage capacity. This external storage acted as a cloud-based archive while the internal storage functioned as a local cache.

Service Health no longer supports storing historical data on AWS. With this change, historical data retention is now limited to the 50 GB internal storage capacity.

To prevent access to older historical data, regularly observe the health of the monitored services.

Before you begin

We recommend that you familiarize yourself with the following concepts and complete any planning and information-gathering steps:

  • From the Crosswork Network Controller home page, the VPN service health dashlet delivers an at-a-glance summary of VPN service statuses. This enables enhanced visibility and filtering for monitoring status, such as errors, in Service Health.

    The VPN Services page presents detailed health and monitoring data for all VPN services, enhanced by customizable columns and powerful filters that facilitate focused troubleshooting.

    The Service Health Dashboard complements these views by aggregating service metrics and highlighting SLA breaches, enabling efficient oversight of network performance and service quality. For additional information on viewing and filtering monitored VPN services, see Analyze Service Health.

  • Crosswork Network Controller monitors services at two levels: Basic and Advanced.

    • Basic Monitoring: This type of monitoring offers the option of monitoring a higher number of services and provides limited subservice metrics, resulting in lower resource consumption. Additionally, the graphic map renderings are smaller compared to more detailed monitoring.

    • Advanced Monitoring: This monitoring approach is supported for a fewer number of services, as it monitors a larger number of component subservices and consumes more compute resources. Additionally, advanced monitoring results in an increased number of subservice metrics and larger graphic map renderings.

      To view only Basic or Advanced services from the Crosswork Network Controller home page, click the highlighted number within the dashlet. The VPN Services page appears with the filtered service information.

    For more information, see Service Health scale information.

  • Crosswork Network Controller's Service Health supports single virtual machine (VM) deployment and monitors devices at two levels - Basic and Advanced. These monitoring level details also apply to Service Health single VM deployment.

    For more information, see Service Health single VM scale information.

  • For L2VPN services, Crosswork Network Controller monitors the overall health based on the subservices, while for L3VPN services, the monitoring occurs at the node level.

  • Crosswork Network Controller has implemented a rate-limiting process to manage service monitoring requests efficiently. This means that there may be a delay in publishing service monitoring requests if the number of requests raised per minute exceeds a specific threshold. The thresholds are:

    • L2VPN services

      • 50 Basic Monitoring requests per minute per service

      • 5 Advanced Monitoring requests per minute per service

    • L3VPN services:

      • 500 Basic Monitoring requests per minute per vpn-node

      • 100 Advanced Monitoring requests per minute per vpn-node

    The rate-limiting process also extends to the monitoring data. For example, during a restore process, when all Data Gateways send data to the Tracker component, the rate at which the Tracker processes this data and forwards it to Assurance Graph Manager is regulated. This may lead to a delayed reporting of Events of Significance (EOS) following the restore.

    An event is triggered with a severity level of warning and a corresponding description to notify you of the delay. The event is cleared once Crosswork Network Controller resumes normal publishing of monitoring requests.

  • Crosswork Network Controller can store up to 50 GB of monitoring data. When storage usage reaches 70 percent of this capacity, it raises an alarm to alert you of potential storage depletion. If more storage is needed, you can configure external storage in the cloud using an Amazon Web Services (AWS) account. See Configure the additional external storage.

  • Crosswork Network Controller uses a set of rules, expressed in low-code format and saved in packages called heuristics packages to monitor the health of the services.

    • A Heuristic Package contains what to monitor, how to compute the monitored metrics, and symptoms associated with service health degradation. The overall health of the service is determined by applying the rules from the Heuristic Package.

    • The default Heuristic Packages provided with Crosswork Network Controller are referred to as system packages and cannot be altered. Crosswork Network Controller uses these system packages' predefined rules to deploy various testing probes, including Y.1731, TWAMP, SR-PM, and Provider Assurance Connectivity (previously known as Accedian Skylight), to evaluate service health and determine whether the service complies with the Service Level Agreement (SLA) (applicable only to Provider Assurance Connectivity probes).

      If the default system packages do not fully meet your needs, you have the flexibility to customize them to better suit your specific requirements. Export an existing package, modify it, and import it to create a custom Heuristic Package. See Heuristic Packages.

  • Monitoring type filtering for VPN services is available when using the Service Health interface, allowing you to display VPN services based on their monitoring type: Basic or Advanced. This filtering capability enables rapid identification of the Heuristic Package applied to each service and supports efficient monitoring management.

    For example, to quickly review all VPN services configured with Advanced monitoring for compliance checks, apply the Advanced filter to immediately generate a focused list without manually reviewing individual service configurations.

  • Extended CLI support using Crosswork Network Controller's system device packages allows for more comprehensive service monitoring capabilities. These packages are capable of deriving exact sensor paths for metric health calculation, and can be installed as a bundle. To add or extend CLI-based KPI collections, you will need support from Cisco Professional Services. Engage with your Cisco account team for more details regarding this.

Getting started

Service Health is available as part of the Crosswork Network Controller Advantage Package (refer to the Get Started chapter in the Crosswork Network Controller 7.2 Installation guide).

Summary

You need a functional Crosswork Network Controller environment with devices onboard and services provisioned before you can start monitoring services. This workflow includes links to documents and processes needed to accomplish those tasks, which are beyond the scope of this document.

Workflow

To set up and start monitoring services, complete Steps 1 through 6. Steps 7 to 9 are optional and cover advanced use cases.

  1. Install Crosswork Network Controller Advantage package.
  2. Do the basic reachability checks from the Crosswork Network Controller UI.
  3. Create and provision the required Layer 2 VPN (L2VPN) and Layer 3 VPN (L3VPN) services.
  4. Determine if you would like to configure additional external storage.

    Note

    You can configure external storage at any time.
    • If you anticipate monitoring health of many services, Cisco recommends configuring external storage after you install Service Health and before you begin monitoring the services. See Workflow: Managing stored data.
  5. Enable health monitoring for the provisioned services.
  6. Establish your operational processes for responding to degraded services.
  7. (Optional) Use SR-PM to probe and monitor links and TE policies in the network.
  8. (Optional) Use Provider Connectivity Assurance to probe Service Health.
  9. (Optional) Customize and import Heuristic Packages.
    • Service Health offers a default set of Heuristic Packages for monitoring. If these packages do not fully meet your needs, you have the option to customize these packages to align with your specific requirements.. See Workflow: Customizing Heuristic Packages.

Monitor service health using these workflows

Use this section to perform procedures in different scenarios and functionalities described in the Getting started section.

Workflow: Managing stored data

Crosswork Network Controller provides 50 gigabytes (GB) of storage for monitoring data. If storage reaches its limit, the system deletes the least recently used monitoring data first.

Summary

When the storage exceeds 70% capacity, Crosswork Network Controller generates an alarm prompting you to configure external storage in order to save older monitoring data. The actions detailed in the section describe how to monitor storage usage, reduce the amount of data being stored and how to add additional external storage.

Workflow

To manage stored date, use this workflow:

  1. Reduce the number of services being monitored by stopping the monitoring for a few services. Review the monitoring data already stored on your system. Delete any data you do not need to free up storage space.
  2. Switch services that are using Advanced Monitoring to Basic Monitoring to monitor the services in lesser detail.
  3. If you still need additional storage, configure additional external storage on Amazon Web Services (AWS) Cloud.

Workflow: Analyzing the cause of service degradation

To analyze the cause of service degradation is an operational workflow and it is iterative.

Summary

Explore the impacted services and subservices' health, and examine the root cause of the service degradation using any of these methods.

Workflow

  1. View monitored services and identify degraded services.
  2. Identify cause of the service degradation.
  3. Confirm if the reported degradation is a valid issue. If the issue is not valid, you may need to adjust the monitoring level—Basic Monitoring or Advanced Monitoring—to ensure accurate reporting of a service's health. Alternatively, modify the system Heuristic Package to create a custom Heuristic Package, which helps resolve false positive reporting of a service's health. If the reported issue is valid, proceed to the next step.
  4. Analyze if the service degradation is on account of an issue with device health.

Workflow: Monitoring Service Health using Cisco Provider Connectivity Assurance (formerly Accedian Skylight)

Crosswork Network Controller can use external probing from Cisco Provider Connectivity Assurance (formerly Accedian Skylight) to measure performance metrics of the L3VPN services.


Note

Monitoring L3VPN services using Provider Connectivity Assurance is supported only with Advanced monitoring and requires a Provider Connectivity Assurance Essentials license. See Provider Connectivity Assurance Licensing Tiers for more information.

Summary

The performance metrics of the L3VPN services are compared with the contracted service-level agreement (SLA, defined in the Heuristic Package) with the results accessible on the UI for further analysis.

Workflow

To add Provider Connectivity Assurance as a provider in Crosswork Network Controller, follow steps 1 and 2 in the workflow. Follow the remaining steps iteratively for operational purposes.

  1. Install the Provider Connectivity Assurance Solution.
  2. Add Provider Connectivity Assurance as a provider in Crosswork Network Controller.
  3. Set up probe sessions for the L3VPN service.
  4. View the metrics in the Crosswork Network Controller UI.
  5. Analyze the cause of the service degradation.
  6. Confirm whether the reported degradation is valid. If it is not valid, modify the system Heuristic Package to create a custom Heuristic Package for a customized report on service health.

Workflow: Customizing Heuristic Packages

Crosswork Network Controller uses Heuristic Packages as the core logic to monitor and report the health of services. Heuristic Packages define a list of rules, configuration profiles, supported subservices and associated metrics for every service type. Heuristic Packages provided by the system are read-only and cannot be modified.

Summary

If you find that the Heuristic Packages provided by the system do not meet your monitoring requirements, in terms of monitoring metrics or monitoring thresholds, you can create a customized Heuristic Package that caters to your specific monitoring requirements using the procedures in this workflow.

Customizing Heuristic Packages is not included in the standard Day 2 support responsibilities. For assistance, please reach out to the Cisco account team or contact Cisco Professional Services.

Workflow

To customize Heuristic Packages, follow the workflow:

  1. Analyze your network services to identify monitoring requirements. Check the system Heuristic Packages for rules, subservices, and metrics to ensure that the system packages do not already include the necessary metrics, services, or thresholds. Determine the package that most closely matches the conditions you wish to identify in your network.
  2. Export the package or packages that include the functions you want to use.
  3. Using the supplied packages as your template, build a new package that gathers the data you need to make determinations about the health of the service you want to monitor. In the simplest use case, you may simply need to edit the threshold points based on the service level agreements (SLAs) used in your network. In more complicated use cases, you might need to build a Heuristic Package from scratch.
  4. Import the customized Heuristic Package in Crosswork Network Controller.
  5. Apply the custom package to each service that requires it.
  6. Verify that the custom package is providing the monitoring data that you need to meet your requirements.

Service Health audit logging

Crosswork Network Controller provides enhanced audit logging capabilities for Service Health operations. The system includes the Source IP address for specific changes, allowing you to track the originating IP address of users making modifications using either the UI or API. This provides a comprehensive audit trail.

This increased visibility is critical for maintaining accountability and strengthening operational security across various Service Health configurations.

Service Health audit logs record the Source IP for these actions:

  • Importing custom Heuristic Packages: identifies the source of new or updated custom packages.

  • Enabling or disabling service monitoring: records who initiated changes to the monitoring status of services.

  • Changing monitoring levels: tracks modifications to the granularity or type of service monitoring.

This logging capability provides a clear record of administrative actions and serves purely for auditing purposes without impacting system operation.

Viewing source IP in Service Health audit logs

The inclusion of source IP in Service Health audit logs does not require explicit configuration. You can access and review these enhanced audit logs through the Crosswork Network Controller UI.

Before you begin

To view source IP in Service Health audit logs:

Procedure

Step 1

From the main menu, choose Administration > Audit Logs.

Step 2

Filter or search for entries related to Service Health, Heuristic Packages, or service monitoring actions.

Step 3

The audit log entries for these actions now include the source IP address. This information helps identify the originating IP of the user who performed the action.

Service Health Monitoring scale information

You can monitor a maximum of 52,000 services in total. This means you may monitor either 52,000 services using only Basic Monitoring, or a combination of Basic and Advanced Monitoring up to 52,000 services total, with no more than 2,000 using Advanced Monitoring.

Table 1. Monitoring support

Type of monitoring

Supports

Basic Monitoring

52,000 services

Advanced Monitoring

2,000 services

Note

For large Layer 3 (L3) VPN deployments, we support either Basic or Advanced monitoring for up to three large VPNs, with a maximum of 4,000 VPN nodes and up to 20,000 endpoints per deployment.


Note

If you enable large VPN monitoring services while L2 and L3 services are still being discovered, health reporting may be delayed by up to nine hours because of heavy system load and concurrent processing. For best practices, workflow steps, and examples of VPN health reporting that typically completes in one hour, see the Enabling large VPN services in Service Health article.

Service Health Monitoring single-VM scale information

You can monitor a maximum of 2,200 services using Basic Monitoring and Advanced Monitoring, with 200 of those services using Advanced Monitoring. In addition, one L3VPN (more than 200 nodes) service and 200 probe sessions for end-to-end monitoring are available.

For more information on Service Health Monitoring single virtual machine (VM) support, see the Crosswork Network Controller 7.2 Administration guide.

Table 2. Single-VM scale information

Type of monitoring

Supports

Basic Monitoring

2,000 services

Advanced Monitoring

200 services

L3VPN (up to 200 nodes)

1 service

Probe sessions for end-to-end monitoring

200 sessions