Cisco Crosswork AI Getting Started, Release 2.0 - For Crosswork Network Controller

What is Crosswork AI?

Cisco Crosswork AI is a network intelligence platform designed to enhance network reliability and operational efficiency through AI-driven insights. Its multi-agentic AI framework distributes tasks across specialized AI agents to solve complex network problems efficiently and proactively. Crosswork AI supports both on-demand and scheduled analysis, enabling continuous monitoring, early detection, and agent-driven troubleshooting of network issues.

Crosswork AI is vendor and device agnostic, applying consistent detection and troubleshooting processes across diverse network devices and operating systems. It automates routine network audits, anomaly detection, risk-pattern identification, and root-cause investigation, helping reduce manual effort and human error. Crosswork AI integrates with Cisco Crosswork Network Controller (CNC) and Network Services Orchestrator (NSO), providing intelligent automation and analytics.

Crosswork AI also provides a Knowledge Graph for exploring network entities, relationships, topology, and dependencies, and observability views for reviewing agent activity, LLM usage, tool calls, and token consumption.

Supported capabilities

Deployment profiles

Crosswork AI supports the following deployment profiles. Select the profile based on the expected scale of the deployment, including the number of devices, links, and interfaces managed by the controller.

The data disk size is set to 1000 GB by default during installation. You can change the data disk size during deployment. The minimum supported data disk size is 500 GB, but use the recommended data disk size for the selected deployment profile.

Agent concurrency on the Large profile

Crosswork AI agents share CPU and memory resources. As a result, the number of concurrent runs available to one agent can depend on which other agents are running.

For the Large deployment profile, the baseline concurrent capacity is:

• Toxic Factor Detection: 1 concurrent run

• Configuration Drift Detection: Up to 3 concurrent runs

• Deep Network Troubleshooting: Up to 5 concurrent sessions per DNT runtime instance

One DNT runtime instance supports up to five concurrent DNT sessions. If the first DNT runtime reaches this limit, Crosswork AI can create another DNT runtime instance only when sufficient CPU and memory are available in the shared resource pool.

---

These capacity values are not independent guarantees that can always be used simultaneously. On the Large profile, additional DNT capacity is available only when Toxic Factor Detection and Configuration Drift Detection are not consuming the shared resources required for another DNT runtime.

---

Deployment prerequisites

Before you deploy Crosswork AI, ensure that:

Pre-installation checklist

Before you deploy Crosswork AI, gather the values required during deployment, system bringup, and setup for easy reference during the process.

Deploying Crosswork AI in VMware ESXi

This section explains how to deploy Crosswork AI on VMware ESXi using the Crosswork AI OVA image and the VMware vCenter Deploy OVF Template wizard.

Deploying Crosswork AI in Linux KVM

You can deploy Crosswork AI in Linux KVM by using a QCOW2 image. The VM connects to the management and data networks through Linux bridges configured on the KVM host.

Configure Linux bridge networking

For Linux KVM deployments, configure Linux bridge networking on the KVM host. With bridge networking, the Crosswork AI VM connects to the same Layer 2 network as the physical network connected to the KVM host.

A Linux bridge acts as a virtual switch. The physical NIC is attached to the bridge, and the VM interface connects to the bridge instead of directly to the physical NIC.

Replace the variables in the following commands with values for your environment.

To create the bridge connection:

nmcli connection add type bridge ifname <bridge-name> con-name <bridge-name>

To configure a static IP address on the bridge:

nmcli connection modify <bridge-name> ipv4.method manual \
  ipv4.addresses "<bridge-ip>/<prefix>" \
  ipv4.gateway "<gateway-ip>" \
  ipv4.dns "<dns-ip>" \
  ipv6.method ignore

To add the physical NIC as a bridge slave:

nmcli connection add type bridge-slave ifname <physical-nic> con-name <bridge-port-name> master <bridge-name>

To bring up the bridge and bridge slave connections:

nmcli connection up <bridge-name>
nmcli connection up <bridge-port-name>

---

Before you add the physical NIC as a bridge slave, remove or disable any existing IP configuration on the physical NIC. After the bridge is configured, the host uses the bridge as its Layer 3 interface.

---

If the management and data networks use separate physical networks, configure separate Linux bridges for each network.

Prerequisites

Configure an external proxy

Crosswork AI agents do not connect directly to LLM providers. They use the Crosswork AI LLM proxy service as the central interface to the configured LLM provider. The LLM proxy service then connects to the external LLM provider.

External proxy and LLM proxy

If the VM has direct internet access to the configured LLM provider, skip this section.

To configure external proxy settings:

After you save the settings, Crosswork AI restarts the LLM proxy microservice. Wait for the restart to complete before you configure LLM provider models.

Download cwaictl

You can configure LLM provider models by using either the cwaictl CLI tool or the LLM REST APIs. To use cwaictl, download the tool from the Crosswork AI server and run it directly on your system.

You can also use cwaictl to run Crosswork AI setup and agent management commands, such as managing Configuration Drift Detection logical groups, training and inference tasks, and retrieving task results.

Set up LLM proxy

The LLM proxy service allows Crosswork AI agents to access configured LLM provider models. You can set up LLM proxy by using either cwaictl or the LLM REST APIs. Use the method that matches your deployment workflow.

export CWAI_HOST="https://<crosswork-ai-host>"
AUTH_COOKIE=$(curl -vk -H 'Content-Type: application/json' \
  -X POST "$CWAI_HOST/login" \
  -d '{"userName": "<username>", "userPasswd": "<password>", "domain": "DefaultAuth"}' \
  | jq .jwttoken)

-H "Cookie: AuthCookie=${AUTH_COOKIE}"

PUT /api/v1/llm/model

cwaictl llm model set \
  --model "openai/*" \
  --model-ref "openai/*" \
  --api-key "<openai-api-key>" -k

cwaictl llm model set \
  --model "gemini/*" \
  --model-ref "gemini/*" \
  --api-key "<gemini-api-key>" -k

cwaictl llm model set \
  --model anthropic/claude-sonnet-4-20250514 \
  --model-ref claude-sonnet-4 \
  --api-key "<anthropic-api-key>" -k

cwaictl llm model set \
  --model anthropic/claude-3-5-haiku-20241022 \
  --model-ref claude-haiku-3-5 \
  --api-key "<anthropic-api-key>" \
  -k

cwaictl llm model set \
  --model ollama/llama3 \
  --custom \
  --model-ref Ollama/llama3 \
  --api-base "" -k

cwaictl llm model set \
  --model "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" \
  --model-ref "nemotron-3-super-120b" \
  --api-base "https://integrate.api.nvidia.com/v1/" \
  --api-key "<nvidia-nim-api-key>" -k

cwaictl llm model list -k

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/models"

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/models?provider=<provider-name>"

cwaictl llm model get --name "<model-ref>" -k

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/model?name=<model-ref>"

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/model?id=<model-id>"

cwaictl llm default set \
  --models '["<model-ref>"]' -k

cwaictl llm default get -k

curl -k -X PUT "$CWAI_HOST/api/v1/llm/default" \
  -H "Content-Type: application/json" \
  -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  -d '{
    "models": ["<model-ref>"]
  }'

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/default"

cwaictl llm association set --type application --with Routing \
  --models '["<model-ref>"]' -k

cwaictl llm association set --type agent --with dnt \
  --models '["<model-ref>"]' -k

curl -k -X PUT "$CWAI_HOST/api/v1/llm/association" \
  -H "Content-Type: application/json" \
  -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  -d '{
    "association_type": "application",
    "association_with": "Routing",
    "models": ["<model-ref>"]
  }'

curl -k -X PUT "$CWAI_HOST/api/v1/llm/association" \
  -H "Content-Type: application/json" \
  -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  -d '{
    "association_type": "agent",
    "association_with": "dnt",
    "models": ["<model-ref>"]
  }'

List associations

Using cwaictl

To list all LLM model associations:

cwaictl llm association list -k

Using REST API

To list all LLM model associations:

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/associations"

To search for agent associations:

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/associations?search=agent"

To search for associations for the Routing application:

curl -k -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  "$CWAI_HOST/api/v1/llm/associations?search=Routing"

---

The search parameter performs a case-insensitive substring match against the association type, entity name, and full name. It is not an exact field filter.

---

cwaictl llm connectivity-check \
  --app <application-or-agent-name> \
  --model <model-ref> -k

cwaictl llm connectivity-check --model <model-ref> -k

curl -k -X POST "$CWAI_HOST/api/v1/llm/connectivity-check" \
  -H "Content-Type: application/json" \
  -H "Cookie: AuthCookie=${AUTH_COOKIE}" \
  -d '{"app": "<application-or-agent-name>", "model": "<model-ref>"}'

{
  "result": "connected!"
}

Set up SSO for cross-launch from Crosswork Network Controller

Single Sign-On (SSO) allows only authorized users, authenticated through your organization’s identity provider, to access Crosswork AI. In this setup Crosswork Network Controller acts as the SAML Identity Provider (IDP) and Crosswork AI acts as the SAML Service Provider (SP), trusting Crosswork Network Controller to authenticate users.

curl -k https://<cnc-host>:<port>/crosswork/sso/idp/metadata -o idp-metadata.xml

Synchronize roles

For SSO to work correctly, Crosswork Network Controller and Crosswork AI must use matching role names.

Create the same role names in both systems.

---

Use lowercase role names without spaces or special characters. Avoid role names that include uppercase letters, spaces, or special characters.

---

Examples of acceptable role names:

• cwai

• aioperator

Set up mTLS for BGP data

Set up mutual TLS between Crosswork AI and Crosswork Network Controller so BGP data can be sent to the Crosswork Network Controller Data Retrieval Adapter (DRA).

Start Data Retrieval Adapters instances

Data Retrieval Adapters (DRAs) allow Crosswork AI to retrieve and normalize data from external systems, such as Crosswork Network Controller and Network Services Orchestrator (NSO).

Create the DRAs required for the agents that you plan to use. If you plan to use all three agents, you must define DRAs for both CNC and NSO.

Content-Type: application/json
Accept: application/json
Cookie: AuthCookie=<jwttoken>

Verify Crosswork Network Controller integration

After you start the required DRA instances, verify that the required objects are created in Crosswork Network Controller.

Crosswork AI dashboard

Figure 9. Crosswork AI dashboard

The Crosswork AI dashboard provides a summary of the available agents, their status, and quick access to the key workflows. You can:

Agent Catalog

Use Agent Catalog to manage and organize AI agents in Crosswork AI. Each agent card shows the agent name, status, source, tags, and description.

Figure 10. Agent Catalog

Agent Catalog includes built-in agents and custom agents.

View and manage agents

To manage an agent:

1. In Crosswork AI, choose Agent Catalog.

2. Switch between list view and card view as needed.

3. Search for an agent by name or use filters to narrow the list by status, source, or tags.

4. Click the more options menu (…​) on the agent card or row.

5. Select the action you want to perform.

   Available actions depend on the agent type, status, and permissions assigned to the user.

   • Agent details: View additional information about the agent, such as status, source, version, date added, tags, user roles, and description.

   • Update: Replace the uploaded agent bundle or update agent metadata. This option applies to custom agents.

   • Activate or Deactivate: Change whether the agent is available for new runs.

   • Delete agent: Remove an inactive custom agent.

---

A deactivated agent cannot be used to start new runs. Agent runs that are already in progress continue until they complete.

---

Administration

Use Administration to manage system-level settings and operational controls for Crosswork AI. Administration includes areas for system details, software updates, backup and restore operations, certificates, users and roles, AAA, audit logs, and system settings.

Refer to Crosswork AI Administration for details.

Knowledge Graph Explorer

Use Knowledge Graph to explore network entities and the relationships between them, understand topology and dependencies across the network, and inspect the context available to Crosswork AI agents. You can browse entity types, review attributes and relationships, inspect available instances, and query graph data using GraphQL.

Refer to Crosswork AI Knowledge Graph Explorer for details.

Observability Service

Use the observability service to gain visibility into agentic flows across agents, services, and LLM-related activity through logs, metrics, and traces. OpenTelemetry-based observability helps you follow request flows, understand agent execution behavior, review LLM and tool usage, and forward observability data to external OTEL endpoints.

Refer to Crosswork AI Observability for details.

AI Assistant

AI Assistant provides a conversational interface for working with Crosswork AI agents. You can ask questions, start agent workflows, and review previous agent tasks.

When you start a workflow from an agent tile, AI Assistant opens in a side panel. It provides suggested prompts or presents options for running agents and listing scheduled tasks for on-demand or scheduled execution.

Figure 11. AI Assistant

Ask a question or start an agent workflow

Work with threads

Change the AI Assistant view

Review AI Assistant notifications

Account preferences

To manage account preferences, click in the header. From the user menu, you can:

Collect tech support logs

Showtech is the primary tool for collecting logs and system information for troubleshooting. If you encounter a problem that you cannot resolve, or if Cisco Support requests diagnostic information, generate and download a showtech archive from the UI.

To collect tech support logs:

Bootstrap process fails

Bootstrap can fail because of insufficient VM resources, incorrect network settings, DNS or NTP reachability issues, proxy configuration problems, or TFTP server reachability issues. The bootstrap UI might not show the exact failure reason. Use acs health from the VM console to view detailed health checks and identify the failing component.

To troubleshoot bootstrap failure:

Cross-launch from Crosswork Network Controller does not open Crosswork AI

If the Crosswork AI launch icon is not available or the cross-launch fails, verify that:

Sign-in fails

Sign-in can fail if your assigned role is not configured in Crosswork AI. Contact your administrator to add the role, then try again.

The role name in Crosswork AI must match the role provided by the identity provider.

BGP session down events are not included in Toxic Factor Identification

If BGP session down events are not included in Toxic Factor Identification analysis, verify that: