This document provides information about Cisco Crosswork Change Automation and Health Insights, including compatibility information, known issues and limitations, and updates for the 3.2.2 release.

Overview

Cisco Crosswork Change Automation and Health Insights retrieves real-time information from the network, analyzes the data, and uses APIs to apply network changes. The Cisco Crosswork Network Automation platform brings together streaming telemetry and model-driven application programming interfaces (APIs) to redefine service provider network operations.

Cisco Crosswork Change Automation and Health Insights enables service providers to quickly deploy intent-driven, closed-loop operations. The platform provides a ready-to-use solution supporting the following use cases:

  • Monitor Key Performance Indicators (KPIs) and notify of any anomalies.

  • Intergration with other Crosswork products such as the Cisco Crosswork Situation Manager.

  • Prepare network changes triggered by changes in KPIs and roll out these changes.

  • Automate change-impact and remediation.

To support the various use cases within the Cisco Crosswork Network Automation platform, consistent telemetry is required.

Change History

The following table provides details of updates since the initial release of Cisco Crosswork Change Automation and Health Insights 3.2.2.

Table 1. Updates to Cisco Crosswork Change Automation and Health Insights 3.2.2

Update Type and Version

Date

Description

Patch: Cisco Crosswork Change Automation and Health Insights 3.2.2.1

For patch installation and activation instructions, see Patch Activation Workflow.

2020-09-02

The patch release provides the following enhancements to the Traffic Steering Playbook:

  1. Traffic steering with Segment routing is split into 3 separate playbooks.

    • Dynamic Latency Path

    • Optimized Path

    • Strict Static Tunnel

  2. Supports review of policy paths prior to configuration.

  3. Supports option to configure and delete service policy on multiple interfaces.

  4. Supports a new Playbook called Enable/Disable traffic collector.

  5. Support for NSO sync-from play in all traffic steering playbooks and in the new traffic collector playbook.

Functionality added in Cisco Crosswork Change Automation and Health Insights 3.2.2

This section lists the new features/functionalities delivered in Cisco Crosswork Change Automation and Health Insights 3.2.2.

Install and Configuration

  • Upgrade support from release version 3.2 to version 3.2.2.

  • Installation ID modified to be persistent and immutable during VM upgrade.

  • Single interface support for CAHI and CDG installation.

Inventory Management

  • Added audit log mapping from Cisco Crosswork Change Automation and Health Insights user to the Cisco NSO user in all applications and platform infrastructure actions.

  • Added audit log traceability and commit labeling for Cisco NSO configuration commits.

  • Source Interface and loopback address added for telemetry configuration.

Change Automation

  • Added traffic steering playbook that provides ability to create segment routing traffic engineering (SR-TE) tunnel to steer the traffic based on DSCP or extended ACL policies.

Release Dates

Cisco releases updated builds on the Cisco Support & Download site when needed.

Table 2. Version 3.2.x Release Dates
Version Build Filename Date

3.2.2

18

cw-na-cahi-3.2.2-18-release-200628.ova

2020-07-06

Compatibility Information

The following table lists hardware and software versions that have been tested and are known to be compatible with Cisco Crosswork Change Automation and Health Insights. For complete installation requirements, see the Cisco Crosswork Change Automation and Health Insights Installation Guide.


Note

This release supports new deployments and migration from older versions of Cisco Crosswork Change Automation and Health Insights.


Hardware/Software Supported Version

Hardware platform

Cisco Unified Computing System (UCS) B or C series

Note 

The hardware requirements are unchanged from version 3.2. For more information, refer the Cisco Crosswork Change Automation and Health Insights Installation Guide.

Hypervisor and vCenter

  • VMware vCenter Server 6.7 Update 3g or later (ESXi 6.7 Update 1 installed on hosts)

  • VMware vCenter Server 6.5 Update 2d or later (ESXi 6.5 Update 2 installed on hosts)

Cisco Network Services Orchestrator(Cisco NSO)

  • Version 5.2.0.3 Function Pack 1.0.0

  • IOS XR Network Element Driver (NED) version 7.18.3, 7.21

  • IOS XE NED version 6.36

Software platform

  • Cisco IOS XR 6.4.1 or later (recommended 6.5.x or later)

  • Cisco IOS XE 16.10 or later (limited functionality)

  • Cisco NX-OS (N7K - 8.4(1), N9K - 7.0(3) I7(2))

Cisco Crosswork Data Gateway

Version 1.1.3

Browsers

  • Google Chrome—70 or later

  • Mozilla Firefox—60 or later

Installation/Upgrade Paths

The following table lists the valid path for installing/upgrading to Cisco Crosswork Change Automation and Health Insights 3.2.2 from previous versions.

Table 3.

Current Cisco Crosswork Change Automation and Health Insights version

Install the following to upgrade to Cisco Crosswork Change Automation and Health Insights 3.2.2

Cisco Crosswork Change Automation and Health Insights 3.1

Cisco Crosswork Change Automation and Health Insights 3.1 > 3.2 > 3.2.2

Cisco Crosswork Change Automation and Health Insights 3.2

Cisco Crosswork Change Automation and Health Insights 3.2 > 3.2.2

See the relevant installation guide for installation prerequisites and procedures for Cisco Crosswork Change Automation and Health Insights versions.


Note

The upgrade operation requires both the old and the new VM to coexist for a period of time while configuration and other data is moved from one machine to the other. While this process temporarily requires more data center resources, it minimizes downtime.


Usage Guidelines and Important Notes

This section provides guidelines and important notes to consider when using Cisco Crosswork Change Automation and Health Insights.

Installation and Configuration

For configuration and installation procedures, see the Cisco Crosswork Change Automation and Health Insights Installation Guide.

  • It is recommended to deploy Cisco Crosswork Change Automation and Health Insights on a highly available cluster (vSphere HA) with shared storage.

  • Use VMware vCenter and vSphere Web Client (flash mode) for OVA deployment.

  • Managed devices, VM host and the VMs should use the same NTP source to avoid time synchronization issues.

  • Confirm that the DNS and NTP servers are properly configured.

  • If the UI does not appear within 30 minutes after installation, log into the VM and check the installation logs for configuration errors. If necessary, collect logs to give to Cisco TAC (see the "Perform Administrative Tasks" chapter of the Cisco Crosswork Change Automation and Health Insights User Guide).

  • Use Terminal Access-Control System Plus (TACACS+), Lightweight Directory Access Protocol (LDAP) or Role-Based Access Control (RBAC) for auditing purposes.

  • During configuration, note the Cisco Crosswork Change Automation and Health Insights UI and CLI user names and passwords. Due to added security, the only way to recover the administrator password is to re-install the software.

  • In situations where Cisco Crosswork Change Automation and Health Insights is expected to work with SR-PCE (SR-PCE is used for L3 topology discovery) we recommend the use of dual SR-PCEs.

Health Insights

For information on how to use Health Insights, see the "Monitor Network Health and KPIs" chapter in the Cisco Crosswork Change Automation and Health Insights User Guide.

  • Group the KPIs within a KPI Profile for monitoring relevant device metrics. For best results, limit to adding 10 KPIs per KPI Profile. Maximum number of KPIs that can be supported within a KPI profile is 50.

  • For optimal performance, enable a KPI Profile in batches of no more than 50 devices.

  • Link the playbook with KPI before enabling the corresponding KPI Profile on a device.

  • For best system performance, use the default KPI cadence.

Change Automation

For information on how to use Change Automation, see the "Automate Network Changes" chapter in the Cisco Crosswork Change Automation and Health Insights User Guide.

  • Use the scheduling feature to perform planned maintenance operations.

  • Use the Install and Uninstall packages to upgrade or downgrade software patches.

  • When using dynamic tags to run a playbook on a set of devices the playbook will be executed on groups of 20 devices at a time untill the playbook has been run on all of the tagged devices.

Collection Services

  • Use tags to select a group of devices when enabling collection jobs for better operational efficiency.

Monitor and Troubleshoot

For administrative and monitoring tasks see the "Perform Adminstrative Tasks" chapter in the Cisco Crosswork Change Automation and Health Insights Installation Guide.

  • Contact the Cisco Customer Experience team to when troubleshooting Cisco Crosswork Change Automation and Health Insights, before restarting services (via UI or CLI).

  • Monitor and view application state, health, performance, and summary details from the Crosswork Manager window.

  • Use custom certificates signed by your local CA to avoid browser warning messages. To manage certificates, go to the Certificate Management application.

Known Issues and Limitations

The following are known issues, limitations, and workarounds in Cisco Crosswork Change Automation and Health Insights.

Upgrade

  • Dual stack deployment modes are not supported in version 3.2.2. Only single stack deployment modes (IPv4 only or IPv6 only) are supported.

  • Without adequate resources (storage, memory and disk), the upgrade operation cannot load the services successfully, and will fail in the first boot.

  • After upgrade to version 3.2.2, all historical alerts will be available only for an hour. Future alerts will be available once devices are mapped to Cisco Crosswork Data Gateway for the enabled KPIs.

  • Post upgrade, and after enabling NSO, the devices need to be mapped to Cisco Crosswork Data Gateway for auto sync as the software type information is not migrated to version 3.2.2.

Inventory Management

  • Sometimes, NETCONF reachability times out for IOS XE devices. To recover, try increasing the NETCONF reachability timer to a higher timeout value (for example, 120 seconds).

  • While retrieving device inventory via API from Cisco Crosswork Change Automation and Health Insights, use page size of 200.

  • The maximum number of tags that can be created by a Cisco Crosswork Change Automation and Health Insights user is 100.

  • While importing large number of devices via a CSV file, value for the TE Router ID field should be populated.

Change Automation

  • Sometimes, under certain load conditions, the execution of a Playbook times out. If it continues to fail for a specific device, try changing the time out for the job, or changing the device state to DOWN and UP again.

  • While scheduling playbooks across a dynamic group tag, the corresponding job set screen for the job in the Job History page will not display the relevant devices, until the job is scheduled for execution.

  • Running a Playbook on multiple devices at the same time with different collection_type parameter values may result in failure. Re-executing the failed Playbook will resolve the issue.

  • Under extreme load conditions, Change Automation may fail to cleanup the collection jobs that were created. These stale collections jobs can be deleted using API.

Health Insights

  • Setting the alert flag as ON for an enabled KPI Profile is not displayed on the corresponding Health Insights job details page as the update operation is an internal system transaction. If the job completes successfully, the alerts triggered can be viewed on the alert dashboard.

  • After a KPI Profile is enabled, editing cadence & threshold parameter for any of the associated KPIs can be achieved using one of the three procedures detailed below:

    • Create a custom KPI with the required cadence & threshold parameters and associate with the existing KPI Profile.

    • Create a new KPI Profile with the relevant KPIs associated. Update the cadence & alert parameters before enabling this new KPI Profile.

    • Disable the KPI Profile & perform the modifications on relevant associated KPIs and re-enable the KPI Profile.

  • For custom KPIs:

    • While creating custom KPI, user is restricted to selecting leaf sensors only upto a certain hierarchy (gather path).

    • All leaf sensor paths are reserved for that KPI use only.

    • Multiple KPIs cannot use the same sensor path. Therefore, custom KPIs cannot share a sensor path with each other or with an existing default KPI that is active on a device.

  • Any Health Insights job stuck in the processing state and does not complete within the stipulated time out of 60 minutes, will fail.

GUI

  • Importing large number of devices with incorrect CSV values using a Firefox browser may render the window unusable. If this happens, login to Cisco Crosswork Change Automation and Health Insights in a new tab or window, and onboard devices with correct CSV values.

  • Filtering is case sensitive for the Manage KPIs and Enable-Disable KPI Profiles pages.

  • While importing devices or providers via UI using a CSV file, user should wait for the operation to complete. Clicking the Import button while the operation is in progress will lead to duplicate entries for each device or provider.

  • While adding a destination from the Data Gateway Global Settings page, user should wait for the operation to complete. Clicking the Save button while the operation is in progress will add a duplicate destination. Duplicate entry can be deleted via API.

Alerting

  • Alerting service can become unresponsive during stress testing. Alerts related to Change Automation or Health Insights may not be generated during this time. If this happens, the Cisco Crosswork Change Automation and Health Insights infrastructure will recover the alerting by automatically restarting the service.

Collection Services

  • With large number of collection jobs, periodic updates from the Cisco Crosswork Data Gateway for collection metrics data may overwhelm the Magellan pod and lead to restarting the service.

Network Visualization

  • L2 links are discovered on topology utilizing either Cisco Discovery Protocol (CDP) or Link Layer Discovery Protocol (LLDP).

  • Point-to-point CDP or LLDP is supported in topology.

  • PCE is required for L3 link topology mapping.

  • Enable traps on routers to receive L2 link down and up status changes quickly. Otherwise, it may take one SNMP poll cadence (default is 5 minutes) to see the L2 link status change.

  • For visualizing L3 links in topology, devices should be onboarded to Cisco Crosswork Change Automation and Health Insights with the TE Router ID field populated.

Admin

  • The JSON Web Token is only valid up to 8 hours.

Smart Licensing

In rare cases, after the successful registration, the License Authorization Status in the Smart Licensing page is not changed and will continue to display as being in EVALUATION mode. As a consequence, the evaluation timer will be started and incorrect messages will be displayed to the user. As a workaround, please de-register and register the product again.

Patch Activation Workflow

The unit of a patch is a TAR file consisting of the patch metadata, a list of docker images, checksum and signature. The metadata contains platform and product details, patch version, type of patch and other creation details. Signature is a security requirement in order to safeguard the patch; the signature is verified by the patch framework. It also helps to perform error correction mechanisms and detect if the patch is corrupted or not.

Follow the workflow below to add and activate the patch:

  1. Download the patch from the Cisco Software Download page and save it on a host that is accessible to the Crosswork VM.

  2. Validate the patch

    API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/validate

    This validates the Crosswork patch file for accuracy and compatibility to the product version.

  3. Add the patch to the system

    API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/add

    After the patch is validated, it must be added to the corresponding registry in the system. The Add operation prepares the system for the patch to be activated. This is an asynchronous operation and may take around 15 minutes. Once the Add operation is initiated, a corresponding job ID is received and the operation is performed in the background.

  4. Check status

    • Check the current status of the patch framework, such as if the Add operation is successful or ongoing, or if the Activate operation has been triggered.

      API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/status

      This will provide the CWPatchID which will be required in order to activate the patch.

    • Check the status of a specific job based on the job ID.

      API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/jobstatus

  5. Activate the patch

    API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/activate

    After being added to the system, the patch must be activated before any other patch can be added. Activate, like Add, is an asynchronous operation that generates a job ID and is performed in the background. Activation takes the backup of the current state and updates the configuration. If the patch fails, the auto-rollback functionality rolls back to the previous version and the status is updated with the failure details.

    To activate the patch, you must provide the CWPatchId as payload to this API. This CWPatchId can be obtained by invoking the patch status API, as described in the previous step.

    For example, "CWPatchId": "2f718c8d-85e0-4f87-a9f9-5a6bb96b316a"

  6. Get Summary

    API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/summary

    Provides the overall summary of the patch framework, including the different patch types and patch versions.

    The summary can also be accessed from the GUI. Go to Admin > Crosswork Manager. You can also verify the health of the system from here.


    Note

    After a reboot of the Cisco Crosswork Change Automation and Health Insights VM, the information in both API and GUI reverts to the original release version that was installed, and does not show the patch details. The patch itself remains intact and its functionality is not affected.

    To verify that the patch is intact after rebooting the VM: In the GUI, go to Admin > Crosswork Manager, locate the application on which the patch was applied, and check the UP time. The UP time should reflect the time the patch was applied as opposed to the time the system was installed.


Removing a Patch

API: https://<ip or host>:<port number>/crosswork/platform/v1/patch/remove

A patch can be removed after it has been added to the system prior to activation, or after it has been activated.


Note

When the 3.2.2.1 patch is removed, the new playbooks added as part of the patch will not be removed. They will be visible in the GUI, but cannot be executed.


Cisco Bug Search Tool

You can use the Cisco Bug Search Tool to search for a specific bug or to search for all bugs in a release.

  1. Go to the Cisco Bug Search Tool.

  2. Enter your registered Cisco.com username and password, and click Log In.

    The Bug Search page opens.


    Note

    If you do not have a Cisco.com username and password, you can register here.


  3. Use any of these options to search for bugs, and then press Enter (Return) to initiate the search:

    — To search for a specific bug, enter the bug ID in the Search For field.

    — To search for bugs based on specific criteria, enter search criteria, such as a problem description, a feature, or a product name, in the Search For field.

    — To search for bugs based on products, enter or choose the product from the Product list. For example, enter Cisco Crossswork Change Automation or Cisco Crosswork Health Insights.

    — To search for bugs based on releases, in the Releases list choose whether to search for bugs affecting a specific release, bugs that were fixed in a specific release, or both. Then enter one or more release numbers in the Releases field.

  4. When the search results are displayed, use the filter tools to narrow the results. You can filter the bugs by status, severity, and so on.


Tip

To export the results to a spreadsheet, click Export Results to Excel.


Accessibility Features

All product documents are accessible except for images, graphics and some charts. If you would like to receive the product documentation in audio format, braille, or large print, contact the Cisco Accessiblity Team on the Web or send email to accessibility@cisco.com .

Obtain Documentation and Submit a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation.

To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What’s New in Cisco Product Documentation RSS feed. The RSS feeds are a free service.