Cisco Crosswork Network Controller 7.2.x Administration Guide

PDF

Cisco Crosswork Network Controller 7.2.x Administration Guide

Data destinations in Embedded Collectors

Updated: August 14, 2026

Want to summarize with AI?

Log in

Data destinations in Embedded Collectors are the endpoints within Crosswork infrastructure where network telemetry and performance data is sent for processing, storage, and analysis. They require proper configuration to ensure efficient and reliable data flow.

A data destination in the context of Embedded Collectors is a component in the Crosswork infrastructure that

  • serves as the endpoint where collected network telemetry and performance data from Embedded Collectors is sent for processing, storage, and analysis,

  • in a single VM deployment, is typically the local Crosswork data processing engine co-located within the same VM along with Embedded Collectors, forwarding data through internal buses or message brokers to processing services, databases, and analytics engines, and

  • equires configuration with accurate IP addresses/hostnames, port numbers, authentication credentials, and protocol specifications (such as gRPC, TCP, UDP), while respecting system resource constraints and network connectivity rules to ensure optimized performance and reliable data flow.

Licensing requirements for external collection jobs

To set up collection jobs that send data to the external destinations, you need the CNC Collection RTM - External Application End-Point license. We recommend installing the license before configuring Crosswork to use an external destination. If you don't install the license first, you can still use the feature for 90 days under the trial license before it gets disabled.

When the License Authorization Status is "Out of Compliance", Crosswork Network Controller continues to allow users to create new external collection jobs and to view or delete existing jobs. This state occurs if registration with Cisco Smart Software Manager is not completed after the evaluation period or if the device limit for external collection jobs is exceeded.

View the license status

Check whether your system is properly registered and authorized to use licensed features.

Use this task when you need to confirm that your system has an active license, is in compliance, and is enabled for reserved features. This ensures ongoing access and compliance with Cisco Smart Software Manager requirements.

Procedure

1.

Go to Administration > Smart Licenses.

The Smart licenses tab under the Application management page is displayed.
2.

Ensure that the status is as:

  • Registration Status: Registered. Indicates you have registered with Cisco Smart Software Manager (CSSM) and are authorized to use the reserved licensed features.

  • License Authorization Status: Authorized (In Compliance). Indicates you have not exceeded the device count in the external collection jobs.

  • Under Smart Licensing Usage, the entry "CNC Collection RTM - External Application End-Point" should show a status of In Compliance.

Managing data destinations

Explain how to manage external data destinations in Cisco Crosswork, including creating, modifying, and selecting data destinations for telemetry collection jobs.

Cisco Crosswork enables the creation of external data destinations, such as Kafka or external gRPC, which are utilized by the collection jobs to deposit the telemetry data.

To manage the data destinations, you can navigate to Administration > Data Destinations. From there, you have the options to

  • add or modify a data destination

  • delete any unused destinations, and

  • view all the configured destinations.

Figure 1. Data destination

UUIDs for data destinations

The UUID is the unique identifier for the data destination. Cisco Crosswork automatically generates this ID when you create an external data destination.

When you create collection jobs using the Cisco Crosswork UI, you select the data destination from a drop-down list of configured destinations. When using the API, you need to know the UUID of the destination where the collector sends its data.

Add or edit a data destination

Add a new data destination or modify an existing one, enabling Embedded Collectors to send collected data to the desired destination.

Use this procedure to direct collected data to a new location or to update the parameters for an existing data destination, such as Kafka or gRPC endpoints.

Before you begin

Review the prerequisites and ensure you have all required information, such as destination details and authentication requirements.

Procedure

1.

Go to Administration > Data Destinations.

2.

On the Data destinations page:

  1. To add a new destination, click + Add another. Repeat this step for each additional collector. The Data destinations page opens.

  2. To edit an existing destination, select it and click Edit icon. The Edit destination page opens showing the current parameters. Update them as needed.

    Note

    When you update a data destination, the collector using it establishes a new session with that destination. Data collection pauses and resumes once the session is restored.
3.

Enter or update the required values for your external destination. If you are unsure about a value, use the default settings. For parameter information, see Parameters for adding and editing data destinations.
4.

If you selected Data Gateway or Any as the data source and the server is set to Kafka, you can configure custom values for individual collectors when needed. To override the global properties for a Kafka destination, use the settings in the Destination – Per Collector Properties pane:

  1. Select Collector.

  2. Enter values for all required fields.

    • Custom buffer memory

    • Custom batch size

      Note
      The Custom batch size cannot exceed the value of the Custom buffer memory at run time. If you do not enter a value for Custom buffer memory, validation occurs against the global Buffer Memory value.

    • Custom linger time

    • Custom request timeout

    Figure 2. Add destination
    Add Destination Window

  3. Click + Add Another to repeat for additional collectors.

Note
Properties set here override the global settings for the selected collectors. If you leave a field blank, the global properties are used.
5.

Select the protocol and host details in the Connection details sections. The supported protocols are IPv4, IPv6, and FQDN. For connection parameters, see Parameters for adding and editing data destinations.

Note

FQDN is supported only for Kafka destinations.

6.

Complete the fields in Connection Details according to your connectivity type. Ensure the values match those configured on the external Kafka or gRPC server.

Note

You can change port numbers only for user-defined destinations. The ports of system-created destinations cannot be modified.
7.

(Optional) Enable security configurations.

  1. If the data source is set to Data Gateway, the Enable secure communication check box is displayed. To connect securely to a Kafka or gRPC-based data destination, select this check box. Then select the type of authentication process from the available options.

    • Mutual-Auth: Authenticates external server and the Crosswork Data Gateway collector after the CA certificate, and Intermediate certificate or Key is uploaded to the Crosswork UI. Mutual-Auth is the default authentication process.

    • Server-Auth: Authenticates external server and the Crosswork Data Gateway collector after the CA certificate is uploaded to the Crosswork UI.

  2. If the data source is set to Any or Application, the Enable secure communication with mutual auth check box is displayed. Select this check box to enable the security feature.
8.

For Kafka or gRPC-based destinations, select an authentication option:

  • Mutual-authentication: Authenticates both server and Embedded Collectors after uploading CA certificate and Intermediate certificate or Key. (Default)

  • Server-authentication: Authenticates both external server and Embedded Collectors using the CA certificate only.

Note

Authentication options are available only when Enable secure communication is enabled.
9.

Click Save.

The new or updated data destination is saved. The Embedded Collectors send data using the new configuration.

What to do next

  1. This step applies if you have selected the data source as Data Gateway or Any.

    Create the required Kafka topics:

    • Configure the Kafka destination with the reachability-topic before initiating a new collection job. This is required for health monitoring of the destination.

    • The topics must exist in the external Kafka at the time of data dispatch; otherwise, Crosswork logs may display an exception:

      destinationContext: topicmdt4
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

  2. If you enabled secure communication, go to the Certificate Management page of your Crosswork UI and add the relevant certificate for the data destination. This step is required to establish secure communication.

    When the data source is set to Data Gateway or Any, a missing or incomplete certificate causes the destination to enter an error state. The associated collection job is marked as Degraded. For details about certificate requirements and management, see your platform’s certificate management documentation.

Requirements to add data destination

To use an external Kafka server as a data destination in Embedded Collectors, ensure these requirements are met:

  • Determine the data source for your destination as Data Gateway or application (Element Management Function, Service Health, and so on). If you are unsure, you can select Any. The form shows or hides specific fields depending on the selected data source. For example, encoding types and security details. Be prepared to provide the fields that apply to your chosen source.

  • Configure Kafka server properties: Set these properties on your external Kafka server:

    • num.io.threads = 8

    • num.network.threads = 3

    • message.max.bytes= 30000000

    Refer to Kafka documentation for detailed descriptions and usage of these properties.

  • Configure data destinations

    • You can configure multiple data destinations as needed.

    • If you reinstall an existing external Kafka destination using the same IP address, restart the collectors for changes to take effect.

  • Validate connectivity and custom properties

    • Verify port connectivity for each data destination. If the port is unreachable, the collection will fail.

    • Embedded Collectors support custom values in Kafka destination properties. This customization feature is not supported for gRPC destinations.

    • Global properties entered in the Destination Details pane are mandatory and applied to all Kafka destinations by default unless overridden at the individual collector level.

  • Secure communication (Optional)

    You can secure communication between Crosswork and the Kafka data destination (either Crosswork Kafka or external Kafka). Note that encryption can impact performance.

    If TLS is required:

    • Keep the public certificate ready for TLS verification.

    • For client authentication, keep the client certificate and key files available.

    • If the key file is password-protected, configure the password as part of destination provisioning.

    • Embedded Collectors currently support IP-based certificates only.

      Ensure that:

      • Certificates are PEM-encoded.

      • Key files are in PKCS#8 format when generated using a Certificate Authority.

Parameters for adding and editing data destinations

These tables list and describe the parameters required for adding or editing the data destinations.

These parameters are grouped in two categories:

  • General configuration parameters: Core options that affect the identity and encoding of the data destination.

  • Connection configurations: Addressing and connectivity options, such as IP addresses or hostnames and ports.

Table 1. General configuration parameters for data destinations
Field Description Available in

gRPC

 Available in

Kafka

Destination name

Enter a descriptive name (up to 128 characters). Valid characters include letters, numbers, hyphens (-), underscores (_), and periods (.). Avoid all other special characters.

Yes

Yes

Server type

Select the gRPC or Kafka for your data destination from the drop-down.

Yes

Yes

Data source

Identifies which Crosswork component or application will use the external Kafka or gRPC destination to send data. This field determines the available configuration options, validation rules, and security features for the destination.

Select the data source as

  • Data Gateway: destination exclusively used by Crosswork Data Gateway for telemetry and network data collection.

  • Application: destination exclusively used by Crosswork applications such as application events, alerts, and notifications.

  • Any: destination can be shared by both Data Gateway and Applications.

Note

  • If you do not choose the data source, it defaults to Data Gateway.

  • If you set the data source to Any, you cannot change it later. To select a different data source, delete the destination and create a new one.

  • When you change the data source from Data Gateway to Any during editing a destination, Crosswork automatically switches the authentication type to mutual authentication and displays a warning message.

  • Embedded collectors does not monitor the availability of Kafka destinations configured with the dispatch source as Applications (Dispatch Source="application"). If a destination becomes unreachable, applications such as Service Health fail to detect the issue or notify users, which can result in silent data loss.

  • When upgrading from Crosswork 7.1 or earlier, all destinations default to Data Gateway.

  • Destinations that have Application as the data source are removed after the upgrade.

Yes

Yes

Encoding type

Note

This field appears only when the data source is set to Data Gateway or Any.

Choose encoding as Json or Gpbkv.

Yes

Yes

Compression method

Choose the compression method. Default is snappy.

  • Kafka supports snappy, gzip, zstd, none (zstd: Kafka ≥2.0).

  • gRPC supports snappy, gzip, deflate.

Yes

Yes

Maximum message size

Enter the max message size in bytes.

  • Default Value: 100,000,000 (100 MB).

  • Min: 1,000,000 (1 MB)

  • Max: 100,000,000 (100 MB).

No

Yes

Buffer memory

Enter required buffer memory in bytes.

  • Default Value: 52428800 bytes/52.4288 MB

  • Min: 52428800 bytes/52.4288 MB

  • Max: 314572800 bytes/314.5728 MB

No

Yes

Batch size

Enter required batch size in bytes.

  • Default Value: 1048576 bytes/1.048576 MB

  • Min: 16384 bytes/16.38 KB

  • Max: 314572800 bytes/314572.8 KB

No

Yes

Linger time

Enter linger time in ms.

  • Default Value: 2000 ms

  • Min: 0 ms

  • Max: 5000 ms

No

Yes

Request timeout

Enter request timeout duration in seconds.

  • Default Value: 30 seconds

  • Min: 30 seconds

  • Max: 60 seconds

No

Yes
Note

  • For fields with limits, abide by min/max/default values to prevent configuration errors.

  • Some parameters are only available for Kafka destinations, not for gRPC (e.g., buffer memory, batch size).

  • Use informative names for easier management if you have many data destinations.

Table 2. Connection configuration parameters for data destinations
Field Description

Available in gRPC

Available in Kafka

IPv4

Enter IPv4 Address/Subnet Mask and Port. Add multiple IPv4 addresses via + Add another

  • Subnet mask: 1–32

  • Port: 1024–65535

Yes

Yes

IPv6

Enter IPv6 Address/Subnet Mask and Port. Add multiple IPv6 addresses via + Add another.

  • Subnet mask: 1–128

  • Port: 1024–65535

IPv6 subnet mask ranges from 1 to 128 and ports range from 1024 to 65535.

Yes

Yes

FQDN

Enter Host Name, Domain Name, and Port (1024–65535). Add multiple via + Add another.

Yes

Yes
Note

Additional details:

  • For all connection fields, you can add multiple entries using the + Add another option.

  • Always verify firewall settings to allow configured ports.

  • Supported port ranges apply consistently across IPv4, IPv6, and FQDN settings.

View the data destination details

Review the configuration and attributes of a selected data destination within the Data Gateway.

Use this task when you need to audit, verify, or analyze the data destination endpoints configured for your organization. Accurate review ensures that data flows to intended endpoints and assists in diagnosing configuration or routing issues.

Procedure

1.

Go to Administration > Data Destinations.

2.

Click the icon next to the data destination whose details you want to review.

Destination details appear with associated configuration information.
Figure 3. View destination details

The details for the selected data destination are displayed, allowing you to verify attributes and configuration.

What to do next

After viewing, confirm that the data destination matches the desired configuration. If you find any discrepancies, adjust the settings. If there are no discrepancies, continue regular monitoring.

Delete a data destination

Delete one or more data destinations from the system.

Data destinations store information collected by the data collector. You may need to delete a destination if it is no longer needed or to maintain system hygiene.

Before you begin

  • A data destination can only be deleted if it is not associated with any collection job. Deleting a destination also removes all associated data subscriptions.

  • Default destinations, such as Crosswork_Kafka, cannot be deleted.

  • Check the Collection Jobs view to determine if any collection jobs are using the data destination.

Procedure

1.

Got to Administration > Data Destinations.
2.

Select one or more data destinations to delete from the displayed list.
3.

Click Delete icon.
4.

In the Delete Data Destination(s) pop-up, click Delete to confirm.

The selected data destinations are deleted from the system.

What to do next

Verify the destination has been removed.