Cisco Crosswork Hierarchical Controller 12.0 Release Notes

 

Cisco Crosswork Hierarchical Controller version 12.0 includes new functions and enhancements, as well as bug fixes.

New software features

Remove Devices Automatically when Device Deleted from Controller

In previous Cisco Hierarchical Controller versions, deletion of devices from the Device Manager was a manual process. Devices are now removed automatically when they are removed by the controllers:

●     Optical devices (Cisco EPNM and Cisco ONC) are automatically removed from Device Manager when deleted from the Controller (in previous versions they were removed from the model but not from Device Manager).

●     IP devices are automatically removed from the model and Device Manager when deleted from the Controller.

When the device is deleted from the controller, the related topology and inventory database entries are updated and the associated services, links, ports, and so on, are deleted in Cisco Hierarchical Controller.

Note: P2P Services of the removed devices will be deleted. Multipoint services will not be deleted from Cisco Hierarchical Controller.

Note: To delete devices from the controller, switch auto onboarding=off on the controller.

This feature applies to  the Cisco CNC, Cisco EPNM, and Cisco ONC adapters. There is a difference in the mechanism used:

·       For Cisco EPNM and Cisco ONC adapters, these device updates are made during discovery cycles and via notifications.

·       For Cisco CNC, these device updates are only made during discovery cycles and not via notifications.

Cisco Network Controller – Delete Device(s)

The devices are automatically removed from Device Manager.

Device Manager – Devices

To verify the deletion, query the inventory in SHQL Query

SHQL Query – Inventory

To view more information about the deletion, check the DELETE events.

 

SHQL Query – Deletion Events

Switching controllers (by updating the host in the adapters settings) is also supported, and the devices will automatically be updated in Device Manager.

When devices with a Fully Qualified Domain Name (FQDN) are added in the controller, the device’s FQDN appears in Device Manager. This avoids any possibility of duplication.

All statistics data is preserved when a device is deleted.

Device Manager – FQDN Device Name

Refer to the Cisco adapter documentation for more details on the configuration options, for example, for Cisco CNC, how to configure the full data fetch interval.

Bulk Delete Devices

You can delete multiple devices in Device Manager.

To delete devices:

1.     In the applications bar in Crosswork Hierarchical Controller, select Services > Device Manager.

2.     Select the required adapter.

3.     Select the Managed Devices tab.

4.     Select the devices to delete.

Device Manager – Managed Devices

5.     Click Delete Device(s). A confirmation message appears.

6.     Click Confirm to delete the devices, unassign them from all adapters and delete the devices from the model.

AI Assistant for SHQL

Cisco Crosswork Hierarchical Controller leverages SHQL to enable exploration and extraction of insights from Cisco Crosswork Hierarchical Controller's complex data model, providing deep visibility across multiple layers of network infrastructure. For optimal results, however, this requires knowledge of both SHQL syntax and the Cisco Crosswork Hierarchical Controller data model, which are highly specialized skills.

The SHQL Query AI Assistant, is an LLM-powered chat agent, integrated into the SHQL Query application. The AI Assistant enables you to conduct a natural language conversation and assists you in generating suggested SHQL queries, as well as helping you better navigate the Cisco Crosswork Hierarchical Controller data model and gathering custom insights.

In-scope use cases:

·       SHQL, Cisco Crosswork Hierarchical Controller data model, and networking technology-related questions and answers, with SHQL prompts if applicable.

Out-of-scope use cases:

·       SHQL command execution

·       Cisco Crosswork Hierarchical Controller data analysis

·       Product documentation

·       Multi-step troubleshooting

·       Queries concerning performance metric data

·       Any other topic

For details on how to install the AI assistant, see the Cisco Crosswork Hierarchical Controller Installation Guide.

What can you do?

AI Assistant

I'm a SHQL (Sedona Hierarchical Query Language) expert designed to help you with queries for Cisco's Crosswork Hierarchical Controller (HCO) product. Here's what I can do for you:

1. Generate valid SHQL syntax based on your requirements

2. Help troubleshoot and optimize your existing SHQL queries

3. Explain network concepts related to the hierarchical data model

4. Provide guidance on querying different network layers (from Fibre to Service layers)

5. Assist with complex queries involving multiple table transformations

6. Help with time-based historical queries

7. Explain how to traverse network hierarchies using functions like upward() and downward()

8. Demonstrate how to filter and group network data effectively

9. Show how to join different data tables using SHQL operators

You can ask me to create queries for specific network scenarios, help understand the relationships between different network layers, or explain how to extract particular information from your network data.

Would you like me to demonstrate a specific SHQL query example or explain a particular aspect of the language?

1.     In SHQL Query, click AI Assistant.

2.     Click Next.

3.     Click Next.

4.     Click Launch AI Assistant. Your past threads appear on the left.

5.     Click on an existing thread or ask the AI Assistant a question to start a New thread.

6.     Review the answer and if you want to run the recommended query, click RUN.

7.     You can also ask the AI Assistant for general Cisco Crosswork Hierarchical Controller information.

 

Cisco CNC Adapter - Direct Collection of Counters and Performance Metrics

Counters and Performance Metrics (PMs) are no longer collected from Cisco CNC using collection jobs. Traffic counters are now collected via DPM, and PMs are collected via NAPM.

●     Traffic Counters (via DPM): Interface, Tunnel, SR Policy, SRV6, OTU Optic, ZR/ZR+, and QDD.

●     Performance Metrics (via NAPM): Interface SR, SR Policy SR and Tunnel SR.

 

For Cisco CNC 7.2, the collector cadence is driven by user config. For Cisco CNC 7.1 or earlier, counters are collected every 15 minutes.

Refer to the Cisco adapter documentation for more details on the configuration options.

1.     In Device Manager, select the Cisco CNC adapter.

2.     In the General tab, ensure that Enable Traffic Counters is selected. This option is automatically configured for collection.

Counters and PM Collection Parameters

3.     Ensure that a HCO Public IP Address port 65001 for GRPC messages is configured and reachable by the Cisco CNC Controller.

4.     The Crosswork Hierarchical Controller adapter is automatically added as a data destination on the Cisco CNC Controller. This can be viewed in Administration > Data Destinations.

Cisco CNC Controller – Data destinations

5.     The Crosswork Hierarchical Controller data subscriptions are automatically created on the Cisco CNC Controller. These can be viewed in Administration > Users and Roles > Data subscription.

Cisco CNC Controller – Data subscription

6.     The Statistics Status appears in the Device tab.

Device Manager – Adapters

7.     Drill down to view the events.

Device Manager – Events

8.     The statistics are used by the Performance App.

Performance – Traffic Utilization

Performance – Optical Power

Performance – ZR

Cisco CNC Adapter Uplift

The Cisco CNC adapter now supports Cisco CNC version 7.2, while maintaining backward compatibility with previously supported versions (6.0.x, 7.0.x, and 7.1.x).

Cisco EPNM Adapter Uplift

The Cisco EPNM adapter now supports Cisco EPNM version 8.1.2, while maintaining backward compatibility with previously supported versions (7.1.x, 8.0, 8.1.0, and 8.1.1).

Cisco ONC Uplift

The Cisco ONC adapter now supports Cisco CONC version 25.1.2. This is the only supported version.

Cisco NSO Adapter Uplift

The Cisco NSO adapter now supports Crosswork NSO version 6.4.7 and preserves functionality from previous releases.

Scale Warning

For all Crosswork Hierarchical Controller applications that are sensitive to scale issues, a message appears when the application is launched and there are more than 5000 devices. This warning does not appear for 3D Explorer, Device Manager, Network Inventory, or SHQL Query.

Scale Environment Warning

Endpoints Added to P2P Services Tables in Network Inventory

For Crosswork Hierarchical Controller point-to-point services, the following endpoint columns have been added to the Services table in Network Inventory:

●     Endpoint 1 Device

●     Endpoint 1 Port

●     Endpoint 2 Device

●     Endpoint 2 Port

 

Network Inventory – Services

Wavelength Added to OCH Table in Network Inventory

A global configuration allows you to show either frequency or wavelength in the Crosswork Hierarchical Controller Network Inventory and Service Assurance apps (this is set to Frequency by default). The Service Manager wizard input field appears as Central Frequency, with the corresponding Central Wavelength displayed. For information on how to modify this setting, contact Cisco Customer Support.

Note: In the model, Crosswork Hierarchical Controller ports that already have a LambdaNm field remain unchanged.

 

Service Manager – OCH-NC Creation

System Events

You can now view geo-redundancy and health system events. For more details, see the Cisco Crosswork Hierarchical Controller Administration Guide.

System Info – System Events

Resolved issues

Geo Redundancy (Supercluster)

●     CSCwo91729/CSCwq03250. Automatic failover on supercluster and previous active cluster is still alive.

●     CSCwq39062. Crosswork Hierarchical Control 11.0 Web UI may not be reachable due to bgp service failure.

SRLG Manager

●     CSCwn40822. After switchover the Delta Report under Risks tab hangs.

Service Manager

●     CSCwp63738. IP Link creation fails for 400G ULH and 100G QSFP using NCS1010.

●     CSCwp64150. An SVO link creation fails with a different Appsel code.

Backend

●     CSCwo49068. L3VPN service deletion inconsistency in Cisco Crosswork Hierarchical Controller and other applications.

Topology

●     CSCwp67417/CSCwo87463. IGP links are not discovered after the notification cycle but are discovered after the polling cycle.

NSO

●     CSCwp91807. NSO backup restore functionality not working.

Cisco Crosswork Network Controller Adapter

●     CSCwo87463. Notification not triggered after updating metric on device - OSPF topology.

●     CSCwp27475. CNC adapter provides wrong IPv6 address.

●     CSCwp90083/CSCwo87463. Error on Cisco Crosswork Network Controller adapter "Error parsing notification: 'l3-termination-point-attributes' L3 link creation”.

●     CSCwq58600. If the adapter full sync collection takes longer than 30 minutes, L2 link delete notifications and device status change notifications are only processed by the adapter during the first 30 minutes.
Workaround: Contact Cisco TAC.

●     CSCwq49483/CSCwq15628. Full sync and CDG collection started though "No" for full collection was selected.

Cisco EPNM Adapter

●     CSCwp51069. Optical devices(that is, TCC devices) are down in EPNM but show as Reachable in Crosswork Hierarchical Controller.

Scale Issues

●     CSCwk49932. Using the Export to File option in the Network Inventory app may take time when it comes to large tables. The generation of the file may take 20 minutes and more without a progress bar to let user know of the status.

●     CSCwq58571. In large-scale environments, supercluster switchover may be triggered by memory pressure and raft failure. In the event of a cluster switchover, consider disabling non‑essential applications.

Open issues

SHQL AI

●     CSCwt59236. Problem communicating with AI: Deprecated claude-3-7-sonnet-20250219 model.
Workaround:
Use the default Open AI model/LLM.

- Or -

Change the default for Anthropic using the Crosswork Hierarchical Controller CLI commands:

sedo shell shql-ai-assistant

sed -i 's/model="claude-3-7-sonnet-20250219"/model="claude-sonnet-4-6"/'hco_dialogue/config/data/prompt_configs.toml

exit

Dashboard

●     CSCwp35299. Dashboard icons do not appear in the same order after restore from backup on KVM setup.

Geo Redundancy (Supercluster)

●     CSCwq39012. Automatic switchover may occur if NTP becomes unreachable and node clocks drift out of sync. To avoid this issue, ensure that NTP is configured and reachable.

Scale Issues

●     CSCwq58571. In large-scale environments, supercluster switchover may be triggered by memory pressure and raft failure. In the event of a cluster switchover, consider disabling non‑essential applications.

Logging

●     CSCwk49782. Changing the logging level of Brain not taking effect.

System Events and Audit Logs

●     CSCwq9520/CSCwt12740. On the Audit Log and System Events pages, the navigation buttons may stop working as expected. Refresh.

●     CSCwq95284. Lengthy Audit Log or System Event messages may not be fully visible.

Discovery

●     CSCwn35964. WSON Circuit name not updated in case of modification name in Cisco Crosswork Hierarchical Controller.
Workaround: Full polling

●     SDN-3244. For a device with multiple shelves, the device attributes displayed are those of the first shelf. Attributes of other shelves are not displayed.

●     CSCwm91843. restconf-notification-srv. A number format exception is thrown when providing invalid values.

●     CSCwn20985. RON assurance. For IP link creation, an IP address (this may be a dummy IP address) must be provided.

●     CSCwt30274. Persisting tunnels may take an extended period of time when SR Policy discovery is enabled.

Layer Relations

●     FRB-57. Currently, only links on the main path of a selected prime object are displayed. The related objects used in the protection path of the prime objects are not displayed (for example, when showing all L3 links over OMS, the displayed L3 links are only those over the main path of the OMS).

SRLG Manager

●     CSCwn32684. Violations are not displayed in the Delta Report tab after a provisioning failure.

●     CSCwn36417. SRLG provisioning fails after the resource pool is full.
Workaround: Enlarge the pool size or free existing slots.

Link Assurance

●     CSCwp35560. No performance data for OMS/ZRM link.

3D Explorer

●     FLD-617. An OCH link between two ZR pluggables is displayed in metro view but its wavelength number is not displayed as a label on the link. Such label appears for other OCH links between transponders.

●     FLD-603. Filter map by tags does not work properly when the network model contains fiber paths.

●     SDN-4684. The satellite view option in 3D map only works when the client machine has an internet connection. The satellite view button is still enabled even when no internet connection is detected.

●     SDN-4221. Service ports that appear under Ports in the sidebar for a selected service may show inconsistent association with a link. A link can sometimes be the service or the PW.

●     CSCwd65311. The ZR channel and media ports are not displayed in the Ports tab for selected router in the sidebar.

Device Manager

●     CSCwk85913. Cached data prevents Device Manager from displaying adapters after tab refresh or user login.

Performance

●     CSCwo89020. Delay values appear as 0 in Performance (OAM) tab.

Link Manager

●     Application currently does not support adding router-to-router links.

●     CSCwe64457. If the last cross-link in the table is deleted, then it is wrongly added to the table although it was removed by the user.

Network Inventory

●     The Reachability column for devices is displayed in Device Manager or in 3D explorer when selecting the device. This is due to an improper and misleading report on reachability per device when managed by SDN controllers.

●     CSCwf33767. Under the network inventory app for connections and path in the Explorer app, you can't see the wavelength column being populated for OCH.

Service Manager

●     CSCwj19933. NSO Manager. The Transport Mode sent in VPWS service request is not pushed to Cisco Crosswork Network Controller as Cisco Crosswork Network Controller does not handle this parameter.

●     Creation of SDH line service is part of the release content, however it was not tested properly with an Optical Controller. Hence its quality and proper functioning cannot be guaranteed.

●     CSCwp06673. When creating a circuit E-Line service, the Configuration State appears as In Progress while the Operations > Lifecycle State shows as Done.

●     SCwt37005. In Service Manager, when adding an IP Link to an existing brownfield bundle, it may create two or three duplicate IP Links.

Backend

●     CSCwp23875. Delay in cross-mapper stitching an IP link, accompanied by a warning that the machine was found but not the port.

●     CSCwp25934. Delay in removing an IP link from the Cisco Crosswork Hierarchical Controller model.

NSO

●     CSCwp17269. Cisco Crosswork Network Controller NSO FP does not support L2VPN and L3VPN with IETF deviations.

●     CSCwr97359. The Device Manager always displays the Start New Cycle dialog (Yes/No) after saving the adapter configuration, even if no cycle is currently running. If a cycle is running, selecting Yes restarts the cycle and No continues the current cycle. If a cycle is not running (i.e., listening to notifications), selecting either option has no effect.

●     CSCwr99746. For NSO, in Device Manager, the "IP Link create timeout" (Provisioning parameter) and "Timeout" (Server Configuration parameter) may be negative.

Cisco Crosswork Network Controller Adapter

●     CSCwp65792. Issue with Termination API response from Cisco CNC for QDD-OLS pluggable.

●     CSCwp33575. Incorrect/missing subnet masks values in Network Inventory.

●     CSCwj40068. On some occasions, L3VPN services discovered from Cisco Crosswork Network Controller based on notifications and frequent polling, may have some of the service endpoints missing. The full list of service endpoints for all services is synced once in 24 hours.

●     CSCwk89105. If there are more than one L3VPN wrongly configured on the same interface, the discovery of L3VPN services in Cisco Crosswork Hierarchical Controller may be impacted and no services will be discovered.

●     CSCws16959.  When disabling an adapter service using "sedo service disable", the adapter status is not updated in Device Manager. Workaround: Rather disable the adapter in the Device Manager UI.

●     CSCwr51167. QDD-OLS: MC ports of devices are not visible in Cisco Crosswork Network Controller with Cisco CNC 7.2.

●     CSCws52272. For the Cisco CNC adapter, empty IP and port details may be allowed.

●     CSCwt53724. SR policy mappings to/from VPN services are not rediscovered/shown after the policy was re-added via Notifications.
Workaround:  The issue resolves after a full discovery cycle.

Cisco ONC Adapter

●     CSCwk07350. Some Add/Drop ports are discovered and modeled by Crosswork Hierarchical Controller but are not displayed in the Network Inventory ports table.

●     CSCwp66028. Crosslink validation not working for regression and new RON pluggable scenario.

Cisco EPNM Adapter

●     CSCwp65755. L2VPN services not detected in Crosswork Hierarchical Controller for Cisco EPNM.

●     CSCwp65873. When performance testing OCH-NC links, the frequency shows as N/A in Service Manager.

●     CSCwk73777. The description for a Cisco EPNM OCHCC WSON collection is not updated in the SHQL data.

●     CSCwn35964. The WSON circuit name is not updated when the name is modified in Crosswork Hierarchical Controller.

Known issues

●     CSCwj24829. NSO Manager. LxVPN services provisioned to Cisco Crosswork Network Controller get the route target values automatically from Cisco Crosswork Network Controller, the values included in service intent are ignored.

●     SDN-3440. When querying for an inventory item, the children references are missing. Need to use the “downward” command as transformation to object/s children.

●     FLD-214. System or user-driven events can be viewed using the SHQL command ‘event’ in SHQL app. The application is currently limited and cannot display more than a few thousand events in a single view. Hence it is recommended to filter the view by event type, sub type, or object guid.

●     FLD-382. The sidebar window in the 3D explorer shows a visual view of aggregated links (LAG) and IP logical links. This view is disabled by default. To enable it, please contact your Cisco support team.

●     CSCwd96670. It is recommended to use sedo commands to enable or disable an adapter. Doing it from the Device Manager application would work but the wrong status may be shown, and the container will still be running although the adapter will be paused.

●     Services Manager. Note that the Packet E-Line wizard works for this service in an optical network, under MPLS-TP tunnel. The menu to create Packet E-Line as T-LDP PW over an IP network is supported in the link referring to the NSO page.

●     In ISIS topologies, the ISIS System ID must be unique across all IGP domains.

●     Network Inventory tables are based on SHQL queries. Only columns included in the main query support sorting and filtering. Columns that require extra processing or additional SHQL queries can’t be sorted or filtered.

●     CSCwr83945. L2 VPN services with names containing “~~~” may appear. These are automatically created as part of the multi-domain stitching process and are used internally by the HCO product. They do not represent actual L2 VPN services and can be safely ignored.

●     CSCwt49838. When all devices are deleted from the Cisco CNC Controller, IGP, OSPF, LSP, and SR policy links are only deleted for domains with ID 0 or 1.

Workaround: For all domains other than 0 or 1, leave at least one device per domain in Cisco CNC Controller.

 

Scalability

Crosswork Hierarchical Controller certified scaling.

 

Important: The scale numbers above were certified during integration with Cisco Crosswork Network Controller and validated through the Crosswork Hierarchical Controller Network Inventory and SHQL applications. Support for other controllers or Crosswork Hierarchical Controller applications will be provided on a best-effort basis.

Supported software packages

ESXi Requirements

Cisco Crosswork Hierarchical Controller is released with a VMWare OVA file distribution. OVA is a disk image deployed using vCenter on any ESXi host. This OVA packages together several components including a file descriptor (OVF) and virtual disk files containing a basic operating system and the Cisco Crosswork Hierarchical Controller installation files.

OVA can be deployed using vCenter on ESXi hosts supporting Standalone (SA) or supercluster deployment models.

VMware requirements

●     Hypervisor and vCenter supported:

◦       VMware vCenter Server 8.0 update 3 and up  

◦       ESXi 7.0 update 3 and up

●     Cisco Crosswork VM (Hybrid node) must be hosted on hardware with Hyper Threading disabled.

Note: The system was tested with VMware versions as described above. The system is expected to function as expected with other VMware sub-versions (within the specified major version only) as well. If you are using a sub-version other than those specified above and you encounter any issues, contact your Cisco support representative.

KVM Requirements

Cisco Crosswork Hierarchical Controller is released with a QCOW2 file.

The QCOW2 file is a disk image used with QEMU and KVM on Linux systems. It functions as a virtual hard drive for a VM, and it contains a basic operating system and the Cisco Crosswork Hierarchical Controller installation files.

This can be deployed on KVM hosts and supports Standalone (SA) or supercluster deployment models.

●     RHEL 9.4 with VM management utilities:

◦       virt-manager version 4.1.0

◦       virsh version 10.0.0

●     QCOW2 file for VM install. This is the disk of the VM and must have a unique name.

Adapters

Crosswork Hierarchical Controller comes with a list of network adapters that are updated to work with this version:

●     Cisco Crosswork Network Controller (CNC)

●     Cisco EPN Manager (EPNM)

●     Cisco Optical Network Controller (ONC)

●     Cisco Network Services Orchestrator (NSO)

For details on specific adapter capabilities, refer to the Cisco CNC adapter documentation.

Adapters are also released independently of the Crosswork Hierarchical Controller version.

Note: Not all adapters are generally available (GA). Some are available for specific customers but not as GA, and hence, need BU involvement before use.

Note: Third-party adapters have their own documentation and are not part of the Crosswork Hierarchical Controller release documentation.

Upgrade

Crosswork Hierarchical Controller 11 can be upgraded directly to version 12. Version 11.0.1 can also be upgraded to version 12.

Web Browser

Chrome version 145 or later is recommended.

Client Machine

The PC or MAC used for the web client with Google Chrome must be equipped with GPU. This is mandatory to run the 3D visualization map in Crosswork Hierarchical Controller.

Build Numbers

<> 

Supported hardware

Primary or Standalone Nodes - Non-Scale

This spec is for primary or standalone instances of Crosswork Hierarchical Controller. This applies to deployments with up to 1000 devices.

Note: The hardware specification for scale deployments will vary. Please contact the Cisco Team.

Witness Node - Non-Scale/Scale

This spec is for the witness (or arbitrator) instance of Crosswork Hierarchical Controller.

Primary or Standalone Nodes - Scale

This spec is for primary or standalone instances of Crosswork Hierarchical Controller.

Note: Consult with Cisco on scale deployments.

Geo-Redundancy Latency

Related resources

In this release, all Cisco Crosswork Hierarchical Controller documents are relevant and can be used.

This includes: