Cisco Crosswork Hierarchical Controller 11.0 Release Notes

 

Cisco Crosswork Hierarchical Controller version 11.0 includes new functions and enhancements, as well as bug fixes.

New software features

Cisco CDG Adapter Integrated into Cisco CNC Adapter

The Cisco CDG adapter functionality is integrated into the Cisco CNC adapter for improved scalability and performance. The Cisco CNC adapter discovers all the devices, eliminating the need to manually attach devices to the Cisco CDG adapter, and eliminating the dedicated collection job created per device.

Note: During the upgrade, if the Cisco CNC adapter is configured with the same destination name, old CDG adapter collection jobs are automatically removed from the Cisco CNC controller. If a different destination name is used, the old collection jobs must be manually deleted from the Cisco CNC controller.

Important: Ensure that you disable all the adapters, especially the CDG adapters, before upgrading. If you do not, the default DYNAMIC_APP_PORT=65001 will not be available after upgrade for the Cisco CNC adapters, and this will require additional configuration to use a different port.

Note: To install another Cisco CNC adapter or adapter instance, specify a different GRPC_LISTEN_PORT. The range of ports depends on the CDG controller. Each adapter instance should also have a unique DYNAMIC_APP_GUID.

The Cisco CNC adapter also leverages the Cisco CNC tagging mechanism which attaches a gNMI tag to all the devices (in Cisco CNC that support the gNMI protocol), enabling the Cisco CNC adapter to create a single collection job for all devices tagged accordingly.

A custom tag (Crosswork Tag) may also be added to the devices in Cisco CNC (in bulk or individually) and used to create the Cisco CNC adapter collection job. The collection of statistics will then only be for the devices tagged accordingly. Each Cisco CNC adapter may collect statistics from devices with one type of tag.

The statistics are available to view in the Performance application and in 3D Explorer, and in the Link Assurance application.

Performance

 

3D Explorer – Performance

The CDG Configuration is available on the Cisco CNC adapter Adapters > General tab.

Refer to the Cisco CNC adapter documentation for more details on the configuration options.

Device Manager – CDG Configuration

The counters selected in the collection parameters, appear as sensors in the Cisco CNC collection job.

Cisco CNC – Collection Job

Cisco IOS-XR Adapter Deprecated

The Cisco IOS-XR adapter is deprecated from version 11, and the Cisco CNC adapter is used instead. The Cisco CNC adapter is compatible with all Cisco CNC variants: Essential, Advantage and Premier.

Cisco CNC Adapter Scalability Uplift and Upgrade

The Cisco CNC adapter uses new APIs to improve scalability. These new APIs support bulk operations as well as parallelization when collecting information on termination points, significantly improving performance.

Cisco CNC Essential 7.1 (topology and inventory) is supported for the RON Starter use case. By default, this includes discovery of L2 links and LAG information. PCE may be added as a provider to Cisco CNC, in which case it will start exposing L3 links, then discoverable by Cisco Crosswork Hierarchical Controller adapters.

For a list of the APIs used by the CNC adapter, refer to the Cisco CNC adapter documentation.

Cisco CNC Maps LxVPN Service Endpoints to Underlay SR Policies/RSVP TE Tunnels

From Cisco CNC version 7.1, you can find the SR policies, SRV6, or RSVP TE underlay tunnels mapped to L2 VPN, L3 VPN, and E-Line services in the Crosswork Hierarchical Controller model.

The notifications are bi-directional, notifications from the services are used to update the underlying SR Policies and RSVP TE tunnels, and notifications from the SR Policies and RSVP TE tunnels are used to update the services.

Note: The TE mapping is performed using NSO or the TE manager in CNC 7.1. Crosswork Hierarchical Controller discovers and visualizes the mapping.

The following APIs are used for L3 VPN and L2 VPN service health and discovered underlay transport data:

●     GET https://{{cat.cw.host}}:{{cat.cw.port}}/crosswork/nbi/cat-inventory/v1/restconf/data/ietf-l3vpn-ntw:l3vpn-ntw/vpn-services/vpn-service?content=nonconfig&offset=10&limit=50

●     GET https://{{cat.cw.host}}:{{cat.cw.port}}/crosswork/nbi/cat-inventory/v1/restconf/data/ietf-l2vpn-ntw:l2vpn-ntw/vpn-services/vpn-service?content=nonconfig&offset=10&limit=50

 

{

  "ietf-l3vpn-ntw:vpn-service": [

    {

      "vpn-id": "l3vpn-90",

      "status": {

        "oper-status": {

          "status": "ietf-vpn-common:op-up",

          "last-change": "2024-06-03T05:26:27.826Z"

        }

      },

      "underlay-transport": {

        "cisco-l3vpn-ntw:discovered-underlay-transport": {

          "sr-policy-ref": [

            {

              "headend": "TB4-PE-A",

              "color": 90,

              "endpoint": "100.100.100.6"

            },

            {

              "headend": "TB4-PE-C",

              "color": 100,

              "endpoint": "100.100.100.5"

            }

          ]

        }

      }

    },

    {

      "vpn-id": "l3vpn-200",

      "status": {

        "oper-status": {

          "timestamp": "2021-06-07T01:04:57.807Z",

          "status": "ietf-vpn-common-oper-status-aa-ext:operational-state-monitor-failed"

        }

      },

      "underlay-transport": {

        "cisco-l3vpn-ntw:discovered-underlay-transport": {

          "te-tunnel-ref": [

            {

              "source": "TB4-PE-B",

              "tunnel-id": 200,

              "destination": "100.100.100.7"

            }

          ]

        }

      }

    }

  ]

}

Cisco CNC – L3VPN Underlay Transport API

This feature enables you to view the relationship between the VPN service links and SR Policies and RSVP TE underlay tunnels.

You can drill-down from the Network Inventory to explore this in 3D Explorer.

Network Inventory – Services - L3VPN

In 3D Explorer, you can view the L3 VPN and Links.

 

3D Explorer – L3VPN and Links

When selecting a VPN service link, the Path shows the lower layers, including the SR Policies and RSVP TE underlay tunnels.

Cisco CNC – L3 VPN Link Path

It is also possible to drill down from an SR Policy or RSVP TE tunnel to view the links and services that use the SR Policy or RSVP TE tunnel.

 

Network Inventory – Connections – SR Policy

View the Links and Services that use the SR Policy in the Used By tab.

  

Cisco CNC – SR Policy Info and Used By (Links and Services)

Similarly, you can view the E-Line link path.

Cisco CNC – E-Line Link Path

The associated SR Policies and RSVP TE underlay tunnels can be viewed in the Service Assurance application

Service Assurance – Underlay Paths

For a list of the APIs used by the CNC adapter, refer to the Cisco CNC adapter documentation.

Cisco CNC Adapter Notifications

From version 11, Crosswork Hierarchical Controller with Cisco CNC 7.1 has notifications support for inventory, topology and services (including service health).

The Cisco CNC adapter now fully supports notifications for Cisco CNC. This includes notifications for adding, deleting, or changing inventory, L2 and IGP topology, RSVP-TE tunnels, SR policies, and L2 and L3 VPN services.

Notification processing performance may vary based on overall network scale and specific use cases. In large-scale environments, notification throughput and responsiveness may degrade

For Cisco CNC 6.0, only services notifications are supported.

For example, adding or deleting an E-Line service in Crosswork Network Service Orchestrator (NSO), or updating a service, for example, changing the srlb (segment-routing global-block) value on the IGP device.

The notifications appear in the log when set to DEBUG level.

For notification configuration, refer to the Cisco CNC adapter documentation.

Cisco CNC – Create E-Line Service Notification in Log

Cisco CNC – New E-Line Service in SHQL

Cisco CNC – Create RSVP-TE Tunnel Notification in Log

Cisco CNC – New RSVP-TE Tunnel in SHQL

 

Cisco CNC – Updated IGP srlb in SHQL

Cisco CNC Adapter Pushes SRLG Labels on IGP Links

The SRLG Manager app detects and manages SRLGs on IP links (IGP). These SRLGs are provisioned end-to-end, supported by the Cisco CNC adapter version 7.1, NSO version 6.4.2.1, TSDN CFP 7.1.0 Drop 4, and PCE and XR Devices version 25.2.1.

Insert actions are generated in the SRLG Manager. These actions are sent to the CNC adapter, which in turn, sends these operation requests to NSO. NSO publishes the operations to the links in Cisco Crosswork Network Controller. The SRLGs then appear in the Cisco Crosswork Hierarchical Controller model. The related events can be tracked in the SRLG Manager and the Device Manager.

The IGP SRLGs settings used to detect risks are:

●     Shared physical equipment: One inventory item (node, shelf or card) is used by multiple IGP links.

●     Active physical links: One OTS link has multiple IGP links running on top of it.

●     Passive fiber SLRGs: These are transferred from the fiber violations. This checks whether there are IGP links running on top of the discovered OMS or OTS links. An IGP layer risk is then created.

If there are multiple risks with the same affected links, these are merged into one SRLG. This is visible in the Delta Report tab, with a column added for Affected Links.

SRLG Manager – Delta Report – Affected Links

Crosswork Network Controller – SRLGs

SHQL – SRLGs

The SRLG deletion flow works in a similar manner and the SRLGs are removed from the links in Cisco Crosswork Network Controller and the Cisco Crosswork Hierarchical Controller model.

Cisco CNC Adapter DLM API

The Cisco CNC adapter now supports the Cisco Crosswork Network Controller (CNC) crosswork/inventory/v1/nodes/query API for collecting device information via DLM. This includes:

●     connectivity_info

●     fqdn_domain_name

●     fqdn_host_name

●     ipv6_router_id

●     profile

●     te_router_id

●     vendor

The Cisco CNC Adapter also uses the existing restconf/data/v1/cisco-resource-physical API to model the Inventory data and merges the data from both APIs to create a full picture of the device. 

SHQL – Router JSON

Note: The DLM API does not support notifications. This means that the fields acquired from this API are acquired every 24/48 hours and are not reflected as part of notifications flow in Cisco Crosswork Hierarchical Controller. 

For a list of the APIs used by the CNC adapter, refer to the Cisco CNC adapter documentation for details.

Cisco CNC Adapter IGP OSPF Support

The Cisco CNC adapter now supports OSPF version 2 as an IGP protocol (in addition to ISIS). This includes support for:

1.     MPLS-based segment-routing topology info.

2.     Unnumbered interfaces.

There is no configuration required at the Cisco CNC adapter-level. IPv6 is not supported in this release.

Cisco CNC VPN Service Health Discovery API

The Cisco CNC adapter improves scalability by using new CNC 7.1 APIs for L2/L3 VPN service health (this also supports paging):

●     crosswork/nbi/cat-inventory/v1/restconf/data/ietf-l3vpn-ntw:l3vpn-ntw/vpn-services/vpn-service=<service>/status/oper-status

●     crosswork/nbi/cat-inventory/v1/restconf/data/ietf-l2vpn-ntw:l2vpn-ntw/vpn-services/vpn-service=<service>/status/oper-status

For a list of the APIs used by the CNC adapter, refer to the Cisco CNC adapter documentation for details.

L3 VPN Service – ServiceHealth

Cisco CNC QDD-OLS Pluggable

Crosswork Hierarchical Controller introduces a new optical QDD-OLS pluggable: a bi-directional optical amplifier to complement an optical device-free connectivity topology. Using this pluggable, a Multiplexer (MUX) can be used to send multiple ZR signals over a single fiber, preserving the full potential range of the connection.

The QDD-OLS pluggable can be inserted into the same router as the ZR pluggable, or on separate routers. The modeling stays the same but the visual representation in Link Assurance changes.

Network Inventory – MC Ports

Link Manager allows for the creation of an NMC cross link between a router OCH port and a QDD-QLS MC port. Whereas an OCH port can be used in one link only, the QDD-QLS MC port can be used in multiple links.

Note: After creating the QDD-OLS cross link in the Link Manager app, a full discovery cycle of the corresponding Cisco CNC adapter is required to incorporate it into the Cisco Crosswork Hierarchical Controller network topology.

SHQL Query – OCH Links between QDD Devices

Cisco CNC QDD-OLS Pluggable Discovery

The Cisco QDD-OLS pluggable can be discovered in Cisco CNC. This creates the Cisco QSFP-DD Pluggable Optical Line System inventory item and the related MC, OMS, and OTS ports.

●     QDD-OLS Line port is modeled by two optical transport ports: OMS, OTS

●     QDD-OLS COM port is modeled by an internal port of type MC.

Inventory – Cisco QSFP-DD Pluggable Optical Line System (QddOlsInventoryItem)

Port – McPort, OMS Port, OTS Port

OTS and MC Ports are listed in the Inventory app.

Network Inventory – Ports – MC and OTS 

  

3D Explorer – Ports – OTS  and MC

3D Explorer – Ports – OMS

Cisco CNC QDD-OLS Pluggable Link Assurance

If the ZR and the QDD-OLS pluggables are located on the same router, a winding NMC crosslink is added between the OCH and NMC ports, with the related access identifier. Lines were added to visually divide PHY and ETH, as well as OCH and NMC.  Clicking on the link provides more information.

Link Assurance – QDD OLS – Same Router

The Access Identifier is visible in 3D Explorer.

3D Explorer – QDD OLS

If the ZR and the QDD-OLS pluggables are located on different routers, the display is similar to the regular RON use case.

Link Assurance – QDD OLS – Different Routers

Cisco CNC SR Policy Links Performance Metrics

The Cisco CNC adapter now supports the collection of LSP and SR Policy counters (interfaces and links were previously supported), delay and jitter performance metrics for interfaces, links, SR policies, and tunnels, as well as discard packets count for items supported by the adapter.

These can be viewed in the Performance app in the Traffic Utilization and Performance (OAM) tabs. The relevant options must be enabled in the adapter configuration.

Device Manager  – Adapter Configuration

Cisco ONC Uplift

The Cisco ONC adapter now supports Cisco CONC version 25.1.1.

Cisco EPNM Adapter Service Manager OCH-NS Provisioning

Provisioning of OCH-NC for non-RON use cases is now supported, specifically for Cisco EPNM.

Cisco EPNM Adapter Uplift

The Cisco EPNM adapter now supports Cisco EPNM version 8.1, while maintaining backward compatibility with previously supported versions (7.0.x, 7.1.x, and 8.0).

Cisco NSO Adapter Uplift

The Cisco NSO adapter now supports Crosswork NSO version 6.4.2 and supports embedded NSO for the RON Starter Use Case.

The Cisco NSO adapter is used for RON provisioning for the RON Starter use case.

This works in conjunction with the updated Cisco CNC adapter for RON inventory and topology.

To access the Crosswork NSO Manager:

●     https://<server>:8443/webui-one/

Coherent Optics Pluggable Modules Support

Crosswork Hierarchical Controller now supports the following Coherent Optics Pluggable Modules:

·       QSFP28 100G ZR Coherent Optics Module

·       400G QSFP-DD Ultra Long-Haul Coherent Optics Module

SHQL – ULH Pluggable Ports – OCH Ports, ZRP Media Ports, ZRP Channel Ports, Ethernet Ports

3D Explorer –OCH Port

Coherent Optics Pluggable Modules Configuration via Appselcode

You can now provision Coherent Optics Pluggable Modules using Appselcodes in Service Manager.

The Appselcode is sent as part of the provisioning request from Service Manager to the Cisco CNC adapter, which sends it to NSO to provision the link. This link is then reflected in NSO and Cisco CNC. Once provisioned, the link can be deleted from Service Manager (not edited).

Service Manager – IP Link Creation – Appsel

This Router Configuration Only means that the links are created between the pluggables using the settings in the Appselcode selected (the Appselcode options available depend on the bandwidth selected and port capability). The Appselcode specifies the following:

●     Appselcode

●     Bandwidth

●     Description

●     Mediacode

●     Hostcode

●     Mode, for example, muxponder, transponder

●     Client Rate

●     FEC

●     Modulation

●     Baud Rate

●     Dac Rate

●     Transmit Power

Service Manager – IP Link Creation – Appsel Selector – Detailed View

This feature utilizes an APPSEL codes database file that is uploaded in the Services Settings page of Service Manager.

The link is reflected in NSO and Cisco CNC.

IP Link in NSO

 

IP Link in CNC

The IP Link Service and L3 Physical Link appear in the model.

IP Link Service

L3 Physical Link

The Path is viewable in 3D Explorer.

3D Explorer - Path

Service Provisioning using Route/Connectivity Optimization Goals (SLA/SLS)

This feature automates the deployment of the underlay for L2 and L3 VPN services. The transport resources required to meet the SLA/SLS  (Service Level Agreement and Service Level Specifications) configuration specified for the service are deployed automatically, without needing to pre-configure and associate these to the service manually. This saves time and reduces the risk of potential errors.  

The underlay-transport templates are configured and deployed using CNC 7.1 TE Manager. The TE Manager defines the criteria that matches the SLA/SLS and VPN service configuration by associating the policy templates to the criteria.  Whenever a VPN service is deployed, the TE Manager backend matches the criteria, creates the underlay (policies), and updates the VPN service to use these policies. 

This enables you to provision with TE policies for:

·       SR ODN

·       SR Policy

·       RSVP-TE

The SLA/SLS underlay-transport additions to the provisioning payload:

SLA/SLS Payload – underlay-transport

CNC – VPN Services – SR Policy

For more information, refer to the Cisco Crosswork Network Controller 7.1 Solution Workflow Guide.

Unnumbered Interface Discovery

Crosswork Hierarchical Controller now supports unnumbered interface logical topology discovery, including logical ports, logical links, IGP links (ISIS and OSPF), RSVP tunnels, and services.

Unnumbered interfaces are used for point-to-point connections. This type of interface borrows an IP address from an existing interface (for example, loopback which is always up and has an address).

This feature uses the .adjacencies field in the logical ports and the .unnumberedInterfaceId port field, which is populated with the index received from the raw data.

SHQL – Unnumbered Logical Port

The logical link has the extra.creation_reasons set to Adjacency.

SHQL – Unnumbered Logical Link

Model Settings User-Driven Events

Crosswork Hierarchical Controller includes events for user operations that affect the model, such as adding, deleting, or editing tags, sites, regions, and hyperlinks, as well as importing and exporting files.

When querying from SHQL, the event type is model-settings-srv. The subType options are ADD, UPDATE, DELETE, IMPORT, and EXPORT. Bulk operations appear as a single event.

SHQL Query - Model Settings - Events

Brownfield Services Modification 

A brownfield service can be promoted in Service Manager or using the POST /service-intent​/promote API. This promotes a brownfield service intent (that is, a service intent created outside of Crosswork Hierarchical Controller and detected by Crosswork Hierarchical Controller) to a service intent managed by Crosswork Hierarchical Controller (and available for editing and deletion).

The Is Managed by HCO field is updated accordingly.

Service Manager – Brownfield Service

 

Service Manager – Events

Service Manager – Greenfield Service

Further Support for IPv6/SRv6

This is phased implementation, and this release supports discovery for all types of SIDs as part of IGP ports discovery, with the full hop-by-hop path of the SR Policies. To keep the hop-by-hop paths updated in the model, Cisco Crosswork Hierarchical Controller calculates the paths for all the segments, where no SIDs are listed, based on the IGP path. The path is kept up to date with every change in topology (link up/down/added/removed).

The following functionality was added:

CNC Adapter

●     Acquires SRv6 link and node information topology data.

●     Enables SRv6 policy extraction.

●     Extends LxVPN mapping to SRv6 tunnel.

Cisco Crosswork Hierarchical Controller

●     Enables SRv6 policies with an SRv6 link layer in the Cisco Crosswork Hierarchical Controller model and tunnel path creation.

●     Layer Relations app supports IPv6.

●     Service Manager LxVPN configuration supports IPv6.

 

SR Policy Link - IPv6

SR Policy and SR Segment – Path

Enable Read-Only Access to SHQL

Users with read-only permissions can now query data using the SHQL application without needing higher-level permissions. Read-only users may also use saved queries (but not save new queries) and export data from the results table.

Model Selector in Performance App Filters out IP/IGP links with no LSPs or SR Segments

In the Advanced tab, when selecting a layer, non-relevant links are filtered out. For example, when selecting LSP Links, only IP/IGP links of related LSPs appear. The relevant links include IP/IGP links with related SR segments.

Note: This does not apply to the 3D Explorer tab.

Performance – Advanced

This is equivalent to the following SHQL query.

SHQL – Query

Audit Events for User-Driven Device Manager Operations

Audit events are recorded for the following user operations (both from the app UI and the APIs):

●     Add adapter

●     Add/remove device

●     Assign/de-assign device to adapter

●     Change adapter configuration

●     Set credentials

SHQL – Event

Ciena MCP Upgrade and Downgrade Packet E-Line Service Bandwidth

You can upgrade/downgrade the bandwidth (CIR, EIR, CBS and EBS) for brownfield and greenfield Packet E-Line services using the Service Manager app or using the Service Manager NBI API.

This requires Ciena MCP – MPLS Controller version 8.2.

Note: To modify any supported brownfield service via Crosswork Hierarchical Controller, it must be promoted using the Service Manager app or using the Service Manager NBI API
(POST /service-intent​/promote). Once a brownfield service is promoted and managed through Crosswork Hierarchical Controller, it should not be modified/deleted outside of Crosswork Hierarchical Controller, as this may result in “out of sync” data and affect the visibility of a newly created service using the same endpoints.

For more information, see the Cisco Crosswork Hierarchical Controller Provisioning Guide and Cisco Crosswork Hierarchical Controller NBI and SHQL Reference Guide.

Ciena MCP Provision and Update Q-in-Q Services

You can now provision Q-in-Q services using the Service Manager app or using the Service Manager NBI API.

You can upgrade/downgrade the bandwidth (CIR, EIR, CBS and EBS) for brownfield and greenfield Q-in-Q services with the following types of ports:

·       ENNI-UNI

·       ENNI-Untagged

You cannot modify the bandwidth for Q-in-Q services between ENNI-ENNI ports.

Note: To modify any supported brownfield service via Crosswork Hierarchical Controller, it must be promoted using the Service Manager app or using the Service Manager NBI API
(POST /service-intent​/promote). Once a brownfield service is promoted and managed through Crosswork Hierarchical Controller, it should not be modified/deleted outside of Crosswork Hierarchical Controller, as this may result in “out of sync” data and affect the visibility of a newly created service using the same endpoints.

This requires Ciena MCP – MPLS Controller version 8.2.

For more information, see the Cisco Crosswork Hierarchical Controller Provisioning Guide and Cisco Crosswork Hierarchical Controller NBI and SHQL Reference Guide.

KVM Hosts

Cisco Crosswork Hierarchical Controller is released with a QCOW2 file. The QCOW2 file is a disk image used with QEMU and HEL 9.4 KVM. on Linux systems. It functions as a virtual hard drive for a VM, and it contains a basic operating system and the Cisco Crosswork Hierarchical Controller installation files.

This can be deployed on KVM hosts and supports Standalone (SA) or supercluster deployment models.

For more information, see the Cisco Crosswork Hierarchical Controller Installation Guide.

Note: Clean install only for v11.

Force Supercluster to Standalone

You can force a cluster without the supercluster quorum to act as a standalone cluster. For more information, see the Cisco Crosswork Hierarchical Controller Admin Guide.

 

Resolved issues

High Availability

●     CSCwn53432. Embedded NSO HA is disabled after switchover in a supercluster configuration. Workaround: Re-enable the HA config on Embedded NSO on the standby cluster:

sedo shell <zone-a/zone-b>/nso-manager-srv

request high-availability enable

request high-availability be-secondary-to node <cluster_id>

Discovery

●     CSCwn09136. For RON assurance on EPNM, cannot see optical stats on NMC ports of an IP link.

3D Explorer

●     CSCwm66756. The History tab has no record for IGP layer link changes.

●     CSCwj78397. When user searchers in the 3D Explorer, there is no way to close the sidebar by clicking on the map. Sidebar will remain open.

Failure Impact

●     CSCwf10902. The filter on the “Type” column in the Failure Impact test result does not work properly. It does not filter the table by the resource type.

Device Manager

●     CSCwk09122. The API to get the device status using the REST APIs for Device Manager doesn't work properly and throws an error.

Service Manager

●     CSCwj93481. Service Manager REST APIs can be used only with Admin user credentials.

Cisco Crosswork Network Controller Adapter

●     CSCwj08306. Polling of network info from Cisco Crosswork Network Controller may face some slowness due to issues in pagination of response.

●     CSCwj38618. Service Assurance: Service Health parameter is not discovered from Cisco Crosswork Network Controller for LxVPN services.

Cisco ONC Adapter

●     CSCwk07350. Some Add/Drop ports are discovered and modeled by Crosswork Hierarchical Controller but are not displayed in the Network Inventory ports table.

Other

●     CSCwf42365. Make sure that the size of the imported GeoJSON file with sites info does not exceed 20Mb. For a larger file, it is recommended to split the file into multiple files.

●     CSCwd09835. REST APIs exposed by Link Manager application can be used only by the admin user.

●     CSCwe71587. When restarting an application using the sedo command (‘sedo system restart’), it is recommended to disable and then enable all apps, so that the restarted app will be launched immediately. Use ‘sedo apps disable all’; wait 10 seconds, then run ‘sedo apps enable all’.

Open issues

Dashboard

●     CSCwp35299. Dashboard icons do not appear in the same order after restore from backup on KVM setup.

Installation

●     In Cisco Crosswork Hierarchical Controller, adding an adapter uses the ‘sedo service install <adapter-service-pack-file>’ command. At times it may be required to run more instances per adapter.  In such a case it is required to manually input the DYNAMIC_APP_GUID and make sure it is different than the default. In Cisco Crosswork Hierarchical Controller, there is no validation of the param used, hence there is a potential for the param used to be an illegal param which could lead to adapter not loading properly until removed and re-added correctly. For details on how to manually validate the param, see the Cisco Crosswork Hierarchical Controller Administration Guide.

Geo Redundancy (Supercluster)

●     CSCwo91729. Automatic failover on supercluster and previous active cluster is still alive.

●     CSCwp20197. Cisco Crosswork Hierarchical Controller crashes after a power off/on of the entire supercluster.
Workaround: Restore DB replication by cleaning up the DB on the standby node to trigger resync:

sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n nxf-system scale --replicas 0 statefulset/postgres

sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n nxf-system get pv,pvc | grep postgres | awk '{print $1}' | xargs sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n nxf-system delete

sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n nxf-system delete configmap -l app=postgres -l cluster-name=postgres

remote=$(kubectl -n nxf-system get cm nxf-supercluster-config -o jsonpath='{.data.config\.json}' | yq '.peers[] | select(.role == "worker") | .clusterName')

kubectl -n nxf-system patch statefulset postgres -p '{"spec":{"template":{"spec":{"containers":[{"name":"postgres","env":[{"name":"STANDBY_CLUSTER","value":"postgres-0.patroni.nxf-system.svc.'${remote}'.local"}]}]}}}}'

sudo kubectl --kubeconfig /etc/kubernetes/admin.conf -n nxf-system scale --replicas 1 statefulset/postgres

●     CSCwq39062. Crosswork Hierarchical Control 11.0 Web UI may not be reachable due to bgp service failure. Workaround: Restart the bgp service by disabling and enabling the service:
sedo service disable bgp
sedo service bgp enable bgp

●     CSCwq39012. Automatic switchover may occur if NTP becomes unreachable and node clocks drift out of sync. To avoid this issue, ensure that NTP is configured and reachable.

Scale Issues

●     CSCwk49932. Using the Export to File option in the Network Inventory app may take time when it comes to large tables. The generation of the file may take 20 minutes and more without a progress bar to let user know of the status.

●     CSCwq58571. In large-scale environments, supercluster switchover may be triggered by memory pressure and raft failure. In the event of a cluster switchover, consider disabling non‑essential applications.

Logging

●     CSCwk49782. Changing the logging level of Brain not taking effect.

Discovery

●     CSCwn35964. WSON Circuit name not updated in case of modification name in Cisco Crosswork Hierarchical Controller. Workaround: Full polling.

●     SDN-3244. For a device with multiple shelves, the device attributes displayed are those of the first shelf. Attributes of other shelves are not displayed.

●     CSCwm91843. restconf-notification-srv. A number format exception is thrown when providing invalid values.

●     CSCwn20985. RON assurance. For IP link creation, an IP address (this may be a dummy IP address) must be provided.

Layer Relations

●     FRB-57. Currently, only links on the main path of a selected prime object are displayed. The related objects used in the protection path of the prime objects are not displayed (for example, when showing all L3 links over OMS, the displayed L3 links are only those over the main path of the OMS).

SRLG Manager

●     CSCwn32684. Violations are not displayed in the Delta Report tab after a provisioning failure.

●     CSCwn40822. After switchover the Delta Report under Risks tab hangs.

●     CSCwn36417. SRLG provisioning fails after the resource pool is full. Workaround: enlarge the pool size or free existing slots.

3D Explorer

●     CSCwp65554. Delay in the visualization of links and sites on the map after upgrading from Cisco Crosswork Hierarchical Controller version 10.1 to  version 11.

●     FLD-617. An OCH link between two ZR pluggables is displayed in metro view but its wavelength number is not displayed as a label on the link. Such label appears for other OCH links between transponders.

●     FLD-603. Filter map by tags does not work properly when the network model contains fiber paths.

●     SDN-4684. The satellite view option in 3D map only works when the client machine has an internet connection. The satellite view button is still enabled even when no internet connection is detected.

●     SDN-4221. Service ports that appear under Ports in the sidebar for a selected service may show inconsistent association with a link. A link can sometimes be the service or the PW.

●     CSCwd65311. The ZR channel and media ports are not displayed in the Ports tab for selected router in the sidebar.

Device Manager

●     CSCwk85913. Cached data prevents Device Manager from displaying adapters after tab refresh or user login.

Link Assurance

●     CSCwp35560. No performance data for OMS/ZRM link.

Performance

●     CSCwo89020. Delay values appear as 0 in Performance (OAM) tab.

Link Manager

●     Application currently does not support adding router-to-router links.

●     CSCwe64457. If the last cross-link in the table is deleted, then it is wrongly added to the table although it was removed by the user.

Network Inventory

●     The Reachability column for devices is displayed in Device Manager or in 3D explorer when selecting the device. This is due to an improper and misleading report on reachability per device when managed by SDN controllers.

●     CSCwf33767. Under the network inventory app for connections and path in the Explorer app, you can't see the wavelength column being populated for OCH.

Service Manager

●     CSCwp63738. IP Link creation fails for 400G ULH and 100G QSFP using NCS1010.

●     CSCwp67467. IPv6 addresses are not supported for RON IP link creation.

●     CSCwp64150. An SVO link creation fails with a different Appsel code.

●     Creation of SDH line service is part of the release content, however it was not tested properly with an Optical Controller. Hence its quality and proper functioning cannot be guaranteed.

●     CSCwj19933. NSO Manager. The Transport Mode sent in VPWS service request is not pushed to Cisco Crosswork Network Controller as Cisco Crosswork Network Controller does not handle this parameter.

Backend

●     CSCwo49068. L3VPN service deletion inconsistency in Cisco Crosswork Hierarchical Controller and other applications.

●     CSCwp23875. Delay in cross-mapper stitching an IP link, accompanied by a warning that the machine was found but not the port.

●     CSCwp25934. Delay in removing an IP link from the Cisco Crosswork Hierarchical Controller model.

Topology

●     CSCwp67417. IGP links are not discovered after the notification cycle but are discovered after the polling cycle.

NSO

●     CSCwp17269. Cisco Crosswork Network Controller NSO FP does not support L2VPN and L3VPN with IETF deviations.

●     CSCwp91807. NSO backup restore functionality not working.

Cisco Crosswork Network Controller Adapter

●     CSCwp65792. Issue with Termination API response from Cisco CNC for QDD-OLS pluggable.

●     CSCwp18326. When an NSO provider is deleted from Cisco CNC, the deletion notification is not received by Cisco Crosswork Network Controller. As a result, services associated with the deleted NSO provider are not removed.

●     CSCwo87463. Notification not triggered after updating metric on device - OSPF topology.

●     CSCwp27475. CNC adapter provides wrong IPv6 address.

●     CSCwp33575. Incorrect/missing subnet masks values in Network Inventory.

●     CSCwp90083. Error on Cisco Crosswork Network Controller adapter "Error parsing notification: 'l3-termination-point-attributes' L3 link creation”.

●     CSCwj08637. Different APIs used for integration have different pagination size defined. Polling of network info from Cisco Crosswork Network Controller may face some slowness due to issues in pagination of response.

●     CSCwj40068. On some occasions, L3VPN services discovered from Cisco Crosswork Network Controller based on notifications and frequent polling, may have some of the service endpoints missing. The full list of service endpoints for all services is synced once in 24 hours.

●     CSCwk89105. If there are more than one L3VPN wrongly configured on the same interface, the discovery of L3VPN services in Cisco Crosswork Hierarchical Controller may be impacted and no services will be discovered.

●     CSCwq58600. If the adapter full sync collection takes longer than 30 minutes, L2 link delete notifications and device status change notifications are only processed by the adapter during the first 30 minutes. Workaround: Contact Cisco TAC.

●     CSCwq49483. Full sync and CDG collection started though "No" for full collection was selected.

Cisco ONC Adapter

●     CSCwp66028. Crosslink validation not working for regression and new RON pluggable scenario.

●     CSCwk07350. Some Add/Drop ports are discovered and modeled by Crosswork Hierarchical Controller but are not displayed in the Network Inventory ports table.

Cisco EPNM Adapter

●     CSCwp65755. L2VPN services not detected in Crosswork Hierarchical Controller for Cisco EPNM.

●     CSCwp51069. Optical devices(that is, TCC devices) are down in EPNM but show as Reachable in Crosswork Hierarchical Controller.

●     CSCwp65873. When performance testing OCH-NC links, the frequency shows as N/A in Service Manager.

●     CSCwk73777. The description for a Cisco EPNM OCHCC WSON collection is not updated in the SHQL data.

●     CSCwn35964. The WSON circuit name is not updated when the name is modified in Crosswork Hierarchical Controller.

Known issues

●     CSCwj24829. NSO Manager. LxVPN services provisioned to Cisco Crosswork Network Controller get the route target values automatically from Cisco Crosswork Network Controller, the values included in service intent are ignored.

●     SDN-3440. When querying for an inventory item, the children references are missing. Need to use the “downward” command as transformation to object/s children.

●     FLD-214. System or user-driven events can be viewed using the SHQL command ‘event’ in SHQL app. The application is currently limited and cannot display more than a few thousand events in a single view. Hence it is recommended to filter the view by event type, sub type, or object guid.

●     FLD-382. The sidebar window in the 3D explorer shows a visual view of aggregated links (LAG) and IP logical links. This view is disabled by default. To enable it, please contact your Cisco support team.

●     SDN-3867. The View option in SHQL does not allow setting a column name with spaces.

●     CSCwc80510. The new filter in the Network Inventory application allows for filtering the inventory resources by a site or device. The Model Selector allows for selecting other resource type as filters. This should be avoided. Only sites and devices can be used as filters.

●     CSCwd96670. It is recommended to use sedo commands to enable or disable an adapter. Doing it from the Device Manager application would work but the wrong status may be shown, and the container will still be running although the adapter will be paused.

●     Services Manager. Note that the Packet E-Line wizard works for this service in an optical network, under MPLS-TP tunnel. The menu to create Packet E-Line as T-LDP PW over an IP network is supported in the link referring to the NSO page.

Scalability

Crosswork Hierarchical Controller certified scaling.

Important: The scale figures presented above were observed during integration with Cisco Hierarchical Controller (HCO) version 11.0 and Crosswork Network Controller (CNC) version 7.1.0. These results were based on use cases using the Crosswork Hierarchical Controller Network Inventory and SHQL applications. Functionality of Crosswork Hierarchical Controller applications may be limited in large-scale environments. Support for these applications or additional controllers is provided on a best-effort basis and may vary depending on deployment size and use case. ​

Supported software packages

ESXi Requirements

Cisco Crosswork Hierarchical Controller is released with a VMWare OVA file distribution. OVA is a disk image deployed using vCenter on any ESXi host. This OVA packages together several components including a file descriptor (OVF) and virtual disk files containing a basic operating system and the Cisco Crosswork Hierarchical Controller installation files.

OVA can be deployed using vCenter on ESXi hosts supporting Standalone (SA) or supercluster deployment models.

VMware requirements

●     Hypervisor and vCenter supported:

◦       VMware vCenter Server 8.0 update 3 and up  

◦       ESXi 7.0 update 3 and up

●     Cisco Crosswork VM (Hybrid node) must be hosted on hardware with Hyper Threading disabled.

Note: The system was tested with VMware versions as described above. The system is expected to function as expected with other VMware sub-versions (within the specified major version only) as well. If you are using a sub-version other than those specified above and you encounter any issues, contact your Cisco support representative.

KVM Requirements

Cisco Crosswork Hierarchical Controller is released with a QCOW2 file.

The QCOW2 file is a disk image used with QEMU and KVM on Linux systems. It functions as a virtual hard drive for a VM, and it contains a basic operating system and the Cisco Crosswork Hierarchical Controller installation files.

This can be deployed on KVM hosts and supports Standalone (SA) or supercluster deployment models.

●     RHEL 9.4 with VM management utilities:

◦       virt-manager version 4.1.0

◦       virsh version 10.0.0

●     QCOW2 file for VM install. This is the disk of the VM and must have a unique name.

Adapters

Crosswork Hierarchical Controller11.0 comes with a list of network adapters that are updated to work with this version:

●     Cisco Crosswork Network Controller (CNC)

●     Cisco EPN Manager (EPNM)

●     Cisco Optical Network Controller (ONC)

●     Cisco Network Services Orchestrator (NSO)

For details on specific adapter capabilities, refer to the Cisco CNC adapter documentation.

Adapters are also released independently of the Crosswork Hierarchical Controller version.

Note: Not all adapters are generally available (GA). Some are available for specific customers but not as GA, and hence, need BU involvement before use.

Note: Third-party adapters have their own documentation and are not part of the Crosswork Hierarchical Controller 11.0 release documentation.

Upgrade

Crosswork Hierarchical Controller 10 can be upgraded to version 10.1. Version 10.1 can be upgraded to version 11.

Web Browser

Chrome version 75 or later is recommended.

Client Machine

The PC or MAC used for the web client with Google Chrome must be equipped with GPU. This is mandatory to run the 3D visualization map in Crosswork Hierarchical Controller.

Build Numbers

<> 

Supported hardware

Primary or Standalone Nodes - Non-Scale

This spec is for primary or standalone instances of Crosswork Hierarchical Controller. This applies to deployments with up to 1000 devices.

Note: The hardware specification for scale deployments will vary. Please contact the Cisco Team.

Witness Node - Non-Scale/Scale

This spec is for the witness (or arbitrator) instance of Crosswork Hierarchical Controller.

Primary or Standalone Nodes - Scale

This spec is for primary or standalone instances of Crosswork Hierarchical Controller.

Note: Consult with Cisco on scale deployments.

This spec is for the witness (or arbitrator) instance of Crosswork Hierarchical Controller.

Related resources

In this release, all Cisco Crosswork Hierarchical Controller documents are relevant and can be used.

This includes: