Cisco Crosswork Network Controller 7.2.x Administration Guide

PDF

Cisco Crosswork Network Controller 7.2.x Administration Guide

External data destinations

Updated: August 14, 2026

Want to summarize with AI?

Log in

Explains what external data destinations are, their attributes, and how they are managed and used within Crosswork Network Controller.

An external data destination is a configurable endpoint that allows you to:

  • receive data from Crosswork Data Gateway collection jobs and applications

  • support integration with platforms like Kafka or external gRPC, and

  • allow management through Crosswork Network Controller interface.

Characteristics of external data destinations

Each data destination has a unique identifier (UUID), which is automatically generated by Crosswork Network Controller when a destination is created.

  • When creating collection jobs via the Crosswork Network Controller UI, select the destination from a drop-down list of configured destinations.

  • When creating a collection job via the API, use the UUID of the destination the collector should send data to.

The Data destinations page allows users to:

  • add new data destinations

  • update settings for existing data destinations, and

  • delete data destinations.

The Data destinations page displays all approved data destinations that collection jobs can use to deposit data.

To view details of a data destination in the Data destinations page, click Info icon next to the name of the data destination you want to view.

Figure 1. Data destinations

Add or edit a data destination

Add a new data destination or edit an existing one to ensure proper data collection and secure communication for your Crosswork Data Gateway environment.

Use these steps to add or edit a data destination:

Procedure

1.

Access the data destination configuration:

  1. Go to Administration > Data Destinations.
2.

Add or edit a destination:

Note

Updating a data destination causes Data Gateway using it to reestablish a session with that data destination. Data collection will be paused and resumes once the session is reestablished.

  1. To add a new destination, click Add New Destination and fill in the required fields.

  2. To edit an existing destination, click the Edit icon icon.
3.

Enter the required destination details. For information about the fields, see Parameters for configuring data destinations.

For telemetry-based collection, it is recommended to use the destination settings of Batch size as 16,384 bytes and Linger as 500 ms, for optimal results.
4.

If you selected Data Gateway or Any as the data source and the server is set to Kafka, you can configure custom values for individual collectors when needed. To override the global properties for a Kafka destination, use the settings in the Destination – Per Collector Properties pane.

  1. Select a Collector.

  2. Enter the values as:

    • Custom buffer memory

    • Custom batch size

      Note
      The Custom batch size cannot exceed the value of the Custom buffer memory at run time. In case, you do not provide a value in the Custom buffer memory field, the Custom batch size will be validated against the value in the Buffer memory field.

    • Custom linger

    • Custom request timeout

    Figure 2. Add destination
    Add Destination Window

  3. Click + Add another to repeat this step and add custom settings for another collector.

Note
Properties entered here for individual collectors take precedence over the global settings entered in Step 3. If you do not enter values in any field here, the values for the same will be taken from the Global properties entered in Step 3.
5.

Select the protocol and host details in the Connection details sections. The supported protocols are IPv4, IPv6, dual stack, and FQDN. For information about the accepted range, see Parameters for configuring data destinations.

Note

The FQDN addresses are supported only for the Kafka destinations.
6.

Complete the Connection details fields as described in the following table. The fields displayed vary with the connectivity type you chose. The values you enter must match the values configured on the external Kafka or gRPC server. For information about the connection details, see Parameters for configuring data destinations.

Note

You can modify the port numbers only for user-defined destinations and not for system-created destinations.

If the IP and port (or FQDN and port) connectivity details match an existing destination, you'll be prompted with a confirmation message for creating a duplicate destination.
7.

(Optional) Enable security configurations.

  1. If the data source is set to Data Gateway, the Enable secure communication check box is displayed. To connect securely to a Kafka or gRPC-based data destination, select this check box. Then select the type of authentication process from the available options.

    • Mutual-Auth: Authenticates external server and the Crosswork Data Gateway collector after the CA certificate, and Intermediate certificate or Key is uploaded to the Crosswork UI. Mutual-Auth is the default authentication process.

      Note

      Crosswork supports mutual authentication only for destinations with the data source set to Application or Any.

    • Server-Auth: Authenticates external server and the Crosswork Data Gateway collector after the CA certificate is uploaded to the Crosswork UI.

  2. If the data source is set to Any or Application, the Enable secure communication with mutual auth check box is displayed. Select this check box to enable the security feature.
8.

Click Save.

What to do next

  1. This step applies if you have selected the data source as Data Gateway or Any.

    Ensure that the required Kafka topics exist:

    • The topics must exist in the external Kafka at the time of data dispatch; otherwise, Crosswork logs may display an exception:

      destinationContext: topicmdt4
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

  2. If you have enabled secure communication when adding the destination, go to the Certificate Management page in the Crosswork UI (Administration > Certificate Management) and add the relevant certificate for the newly added data destination. This step is mandatory to establish a secure communication to the device. See Manage Certificates for more information.

    Note

    When the data source is set to Data Gateway or Any, a missing or incomplete certificate causes the destination to enter an error state. The associated collection job is marked as Degraded. For details about certificate requirements and management, see your platform’s certificate management documentation.

Requirement to prepare external servers for data destination

To use an external Kafka server as a data destination in Crosswork Data Gateway, ensure these requirements are met:

  • Determine the data source for your destination as Data Gateway or application (Element Management Function, Service Health, and so on). If you are unsure, you can select Any. The form shows or hides specific fields depending on the selected data source. For example, encoding types and security details. Be prepared to provide the fields that apply to your chosen source.

  • Configure the external Kafka server with these properties:

    • num.io.threads = 8

    • num.network.threads = 3

    • message.max.bytes = 30000000

    Refer to the official Kafka documentation for details on these property configurations.

  • Confirm that the external Kafka server is reachable and that port connectivity is properly established.

  • If security is enabled, provide certificates in PEM-encoded format and use PKCS#8 format for key files.

  • For client authentication, ensure the required certificate, key files, and password (if necessary) are available.

  • Use the same IP protocol (IPv4 or IPv6) on the external destination as specified during the Crosswork Network Controller deployment.

Best practice for adding external data destinations

When configuring an external data destination like a Kafka server, consider these behaviors and practices:

  • If you reinstall an existing Kafka destination using the same IP address, restart collectors for the changes to take effect.

  • Secure the communication channel between the management system and the destination; enabling security may impact performance.

    Note

    Enabling security may impact performance.

  • If the external destination requires TLS, prepare these configurations in advance:

    • Public certificate for server authentication

    • Client certificate and key files for mutual authentication

    • If the client key is password-encrypted, configure the password during data destination provisioning.

  • Verify port connectivity for the external destination; if unreachable, data collection fails.

  • Configure custom values for a Kafka destination in the destination properties; this is not supported for gRPC destinations.

  • Mandatory global properties specified in the Destination Details panel apply to all Kafka destinations, but collector-level custom values override them for that collector.

  • Match the IP version (IPv4 or IPv6) between the external destination and the deployment settings.

  • Changes to hostname-to-IP mappings only take effect after the DNS TTL expires; to apply changes immediately, reboot the VM.

Parameters for configuring data destinations

These tables list and describe the parameters required for adding or editing the data destinations.
Table 1. Parameters and their descriptions
Parameters Description Available in

gRPC

 Available in

Kafka

Destination name

Enter a descriptive name (up to 128 characters). Valid characters include letters, numbers, hyphens (-), underscores (_), and periods (.). Avoid all other special characters.

If you have many data destinations, choose an informative name to allow for easier identification later.

Yes

Yes

Data source

Identifies which Crosswork component or application will use the external Kafka or gRPC destination to send data. This field determines the available configuration options, validation rules, and security features for the destination.

Select one of the data sources

  • Data Gateway: destination exclusively used by Crosswork Data Gateway for telemetry and network data collection.

  • Application: destination exclusively used by Crosswork applications for data such as alarms, inventory notifications, performance monitoring. The application could be Element Management Function or Crosswork Optimization Engine.

  • Any: destination can be shared by both Data Gateway and Applications.

Note

  • If you do not choose the data source, it defaults to Data Gateway.

  • If you set the data source to Any, you cannot change it later. To select a different data source, delete the destination and create a new one.

  • When you change the data source from Data Gateway to Any during editing a destination, Crosswork automatically switches the authentication type to mutual authentication and displays a warning message.

  • Crosswork Data Gateway does not monitor the availability of Kafka destinations configured with the dispatch source as Application (Dispatch Source="application"). If a destination becomes unreachable, applications such as Service Health fail to detect the issue or notify users, which can result in silent data loss.

  • When upgrading from Crosswork 7.1 or earlier, all destinations default to Data Gateway.

  • Destinations that have Application as the data source are removed after the upgrade.

Yes

Yes

Server type

Select the server type as Kafka or gRPC of your data destination.

Yes

Yes

Encoding type

Note

This field appears only when the data source is set to Data Gateway or Any.

Choose the compression method as either Json or Gpbkv.

Yes

Yes

Compression method

Choose the desired compression type.

Yes

Supported compression types are snappy, gzip, and deflate.

Yes

Supported compression types are snappy, gzip, zstd, and none.

Note

zstd compression type is supported only for Kafka 2.0 or higher.

Dispatch type

This parameter is available when the Server Type field is set to gRPC.

Select stream or unary as the dispatch method. By default, unary is used. Crosswork Data Gateway sends the collected data using either data streams or unary transmission.

Yes

No

Maximum message size

Enter the maximum message size in bytes.

  • Default value: 100000000 bytes/100 MB

  • Min: 1000000 bytes/1 MB

  • Max: 100000000 bytes/100 MB

No

Yes

Buffer memory

Enter the buffer memory required, in bytes.

  • Default value: 52428800 bytes

  • Min: 52428800 bytes

  • Max: 314572800 bytes

No

Yes

Batch size

Enter the required batch size in bytes.

  • Default value: 1048576 bytes/1.048576 MB

  • Min: 16384 bytes/16.38 KB

  • Max: 314572800 bytes/6.4 MB

No

Yes

Linger time

Enter the required linger time in milliseconds.

  • Default value: 2000 ms

  • Min: 0 ms

  • Max: 5000 ms

No

Yes

Request timeout

Enter the duration that the request waits for a response. When the configured duration is reached, the request expires.

  • Default value: 30 ms

  • Min: 30 ms

  • Max: 60 ms

No

Yes
Table 2. Connection details
Connectivity Type Fields

Available in gRPC

Available in Kafka

IPv4

Enter the required IPv4 address, subnet mask, and port. You can add multiple IPv4 addresses by clicking +Add another.

IPv4 subnet mask ranges from 1 to 32 and port range from 1024 to 65535.

Yes

Yes

IPv6

Enter the required IPv6 address/Subnet mask, and Port. You can add multiple IPv6 addresses by clicking +Add another.

IPv6 subnet mask ranges from 1 to 128.

Yes

Yes

Dual stack

Enter the IPv4 address/Subnet mask, IPv6 address/Subnet mask, and Port. You can add multiple addresses by clicking + Add another.

IPv4 subnet mask ranges from 1 to 32 and port range from 1024 to 65535.

IPv6 subnet mask ranges from 1 to 128.

Note

The Dual Stack option is available only when your system supports this configuration.

Yes

Yes

FQDN

Enter the required Host name, Domain name, and Port.

The supported port range is from 1024 to 65535.

You can add multiple FQDN addresses by clicking + Add another.

Yes

Yes

Delete a data destination

Remove data destinations that are no longer required for data gateway configuration.

Delete a data destination to remove outdated or unused endpoints from your Data Gateway settings. Default destinations, such as Crosswork_Kafka, cannot be deleted.

Procedure

1.

Go to Administration > Data destinations.

2.

Select the data destinations you want to remove.
3.

Delete the selected destinations.

When prompted, confirm the deletion.

The selected data destinations are removed from your configuration and the corresponding data subscriptions are also deleted.

What to do next

Review the configuration to confirm that no necessary data destinations have been deleted by mistake.

Subscription APIs

After configuring data destinations, data subscriptions must be created to define what data gets sent where.

Data subscription types include data such as alarms, inventory changes and performance metrics.

API details

API endpoint: POST /crosswork/notification/v2/subscription


*destinationName*: Name of an existing data destination
*destinationType*: Type of destination ('Kafka' or 'gRPC')
*subscriptionDataType*: Type of data subscription
  - 	Possible types for Kafka: Inventory_Changes, Alarm, System_Audit, Device_Performance_Monitoring, Network_Performance_Monitoring, Service_Health_Monitoring
  - 	Possible types for gRPC: Device_Performance_Monitoring, Network_Performance_Monitoring
*subscriptionData*: policy_instance=performance monitoring policy (Example: Device health or Interface health)
*topicName*: Kafka or gRPC topic name
*filter*: Optional filter criteria applicable only for Inventory_Changes data type (set to 'null' if not required)

A success response is returned when the request is completed.

Subscription validation criteria

Successful subscription for Kafka or gRPC destination types is validated using a unique combination of the following four parameters:

  • destinationTypee

  • subscriptionDataType

  • subscriptionData

  • topicName

Examples:

  • For Kafka:

    • destinationType: "Kafka"

    • subscriptionDataType: "Service_Health_Monitoring"

    • subscriptionData: "PCA_Probes"

      topicName: "sh.tracker.topic.PCA_probes"

  • For gRPC:

    • destinationType: "gRPC"

    • subscriptionDataType: "Network_Performance_Monitoring"

    • subscriptionData: "SR_PM_Policy"

    • topicName: "pmdata-test-grpc-NPM"

A subscription is considered successful only if this parameter combination, along with a unique topic name, is validated.

Sample: Kafka subscription request for alarms

This sample creates a Kafka data subscription for alarm monitoring: 
POST /crosswork/notification/v2/subscription
{
  "destinationName": "kafka-alarm-destination",
  "destinationType": "Kafka",
  "subscriptionDataType": "Alarm",
  "subscriptionData": null,
  "topicName": "topic_name",
  "filter": null
}

Sample: Kafka subscription request for performance monitoring

This sample creates a Kafka data subscription for performance monitoring.

Note

subscriptionData is applicable only for device performance monitoring subscriptions in Kafka.

POST /crosswork/notification/v2/subscription
{
  "destinationName": "kafka-alarm-destination",
  "destinationType": "Kafka",
  "subscriptionDataType": "Device_Performance_Monitoring",
   "subscriptionData": "policy_instance=device_health",
   "topicName": "pm-topic",
   "filter": null
}

Sample: gRPC subscription request for device performance monitoring

This sample creates a gRPC data subscription for device performance monitoring: 
POST /crosswork/notification/v2/subscription
{
  "destinationName": "grpc-device-perf-destination",
  "destinationType": "gRPC",
  "subscriptionDataType": "Device_Performance_Monitoring",
  "subscriptionData": "policy_instance=device_health",
  "topicName": "device-perf-context-001",
  "filter": null
}

Sample: gRPC subscription request for network performance monitoring

This sample creates a gRPC data subscription for network performance monitoring: 
POST /crosswork/notification/v2/subscription
{
"destinationName": "grpc-secure",
"destinationType": "gRPC",
"filter": null,
"subscriptionData": "SR_PM_Policy",
"subscriptionDataType": "Network_Performance_Monitoring",
"topicName": "pmdata-test-grpc-NPM"
}

Sample: Service health PCA_probes and Y1731_probes payloads

This sample creates a Service_Health_Monitoring subscription for PCA_probes and Y1731_probes:

PCA_probes
{
"destinationName": "kafka-fqdn",
"destinationType": "KAFKA",
"subscriptionDataType": "Service_Health_Monitoring",
"subscriptionData": "PCA_Probes",
"topicName": "sh.tracker.topic.PCA_probes",
"filter": null
}


Y1731_probes
{
"destinationName": "kafka-test",
"destinationType": "KAFKA",
"subscriptionDataType": "Service_Health_Monitoring",
"subscriptionData": "Y1731_Probes",
"topicName": "sh.tracker.topic.Y1731",
"filter": null
}

Refer to Crosswork Network Controller APIs for more details about adding an external Kafka or gRPC subscription.

Manage data subscriptions

Use the Data subscriptions option to view or delete active Kafka or gRPC subscriptions.

Procedure

1.

From the main menu, choose Administration > Users and Roles.

2.

Click Data subscriptions.
3.

Filter subscriptions by selecting the destination type and data type from the available options.
4.

To delete a subscription, choose the subscription you want to remove and click the Delete icon.