Getting Your DCNM Network Sites Ready for Nexus Dashboard Insights

 

Introduction

Nexus Dashboard Insights (NDI) is a unified monitoring and analytics tool that reduces mean time to detect (MTTD) and mean time to resolve (MTTR) network issues by providing

●     comprehensive visibility into the infrastructure,

●     reliable issue analysis, and,

●     definitive suggestions for addressing day2 operation challenges in complex data center networks.

Nexus Dashboard Insights processes and analyzes telemetry data continuously streamed from all the devices in the infrastructure to provide network operators with real-time monitoring and analytics of the network besides being able to proactively provide lifecycle management suggestions and foresight into infrastructure change management.

Note:       The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

This paper details the best deployment configurations and settings to implement Nexus Dashboard Insights for your DCNM managed or monitored NX-OS network sites. To enable streaming telemetry data from all the devices on the sites, specific configurations, you must configure DCNM and devices. We use a typical two-tier (spine and leaf) network fabric as an example in this white paper to illustrate the necessary configurations, either on the DCNM controller or the switches themselves to prepare the site for Nexus Dashboard Insights. As a network site can be fully managed or monitored by DCNM, the white paper also explains the configuration for both DCNM managed mode and the monitored mode.

Software and Hardware Product Versions

The example in this white paper has the following product software versions:

●     DCNM version 11.5.2

●     Nexus Dashboard version 6.0.1.23

●     Nexus Dashboard Insights version 2.1(1d)

For more information about supported software versions and compatibilities of related products, refer to the Cisco Nexus Dashboard and Services Compatibility Matrix at the following link:

https://www.cisco.com/c/dam/en/us/td/docs/dcn/tools/dcn-apps/index.html

For support of the required features, we’re using all cloud scale switches.

Pre-Requisite Configuration for DCNM Sites

Cisco DCNM enables controlling, automating, monitoring, and visualization and troubleshooting of networks built of the Nexus Data Center switches. To configure DCNM, refer to the configuration guide:

https://www.cisco.com/c/en/us/td/docs/dcn/dcnm/1151/configuration/lanfabric/cisco-dcnm-lanfabric-configuration-guide-1151.html

The following requirements enable to prepare DCNM and its managed or monitored network sites ready for onboarding on Nexus Dashboard and use with Nexus Dashboard Insights:

●     Configure DCNM in-band Network

Nexus Dashboard onboarding and telemetry data transfer use the DCNM in-band data network. IP reachability between the data network of the Nexus Dashboard and the DCNM in-band IP addresses is required.

●     Configure routable loopback interfaces on switches for Nexus Dashboard data network connectivity

Nexus Dashboard data network needs IP reachability to the network site and the DCNM. A routable loopback interface is used on the network site switches for this connectivity and to source telemetry data. We could use a dedicated loopback interface or an existing loopback interface on the switches. Configuring/usage of a loopback interface for both monitored and managed modes are discussed in further sections.

The Round-trip time (RTT) between the Nexus Dashboard cluster and the network site must not exceed 50m for Nexus Dashboard Insights to run for the network site properly.

●     Configure DCNM in-band Network

Nexus Dashboard onboarding and telemetry data transfer uses the DCNM in-band data network. IP reachability between the data network of the Nexus Dashboard and the DCNM in-band IP addresses is required.

●     Enable and Configure NTP (Network Time Protocol) in the network

NTP is an essential clock service that syncs all the elements of the network site. NTP service is not only required for setting up the DCNM cluster and managing network site switches but also for software telemetry to work. It maintains consistency and coherence of logs between the switches and the DCNM controller. NTP configurations and verifications for monitored and managed modes are discussed in the further sections.

●     Enable and Configure PTP (Precision Time protocol) in the network

Nexus Dashboard Insights requires a microsecond-level accurate PTP clock for the network site in order to perform flow analytics across the site and calculate the end-to-end network latency of the flows.

For a DCNM managed or monitored NX-OS network site, the PTP grandmaster needs to be an external device that can provide at least a microsecond-level clock.

The PTP configurations and verifications are explained in further sections for both DCNM managed and monitored network sites.

The following sections discuss in detail about the configuration and verification for each of the prerequisite requirements.

Configure DCNM In-band Network

Nexus Dashboard onboarding and telemetry data transfer use the DCNM in-band data network. You must establish IP reachability from the data network of the Nexus Dashboard to the DCNM in-band IP addresses. This is a part of DCNM installation and initial setup; however, this section provides instructions to configure DCNM in-band network.

DCNM uses eth2 interface for in-band connectivity. For Nexus Dashboard and Nexus Dashboard Insights, you must configure eth2 interface and must be reachable from the Nexus Dashboard data network for a single standalone instance or in-band eth2 VIP interfaces for DCNM in HA mode (Active/Standby) installation. The rest of this section discusses the procedure of configuring and verifying the DCNM in-band network.

Procedure

Step 1. Configure the IP address on ‘eth2’ interface.

[sysadmin@DCNM]$ ifconfig eth2 192.168.1.201 netmask 255.255.255.0

Step 2. Verify that the IP address and netmask configuration.

[sysadmin@DCNM]$  ifconfig eth2

eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 192.168.1.201  netmask 255.255.255.0  broadcast 192.168.1.255

        inet6 fe80::250:56ff:feaf:a4b0  prefixlen 64  scopeid 0x20<link>

        ether 00:50:56:af:a4:b0  txqueuelen 1000  (Ethernet)

        RX packets 1406372  bytes 316914681 (302.2 MiB)

        RX errors 0  dropped 10  overruns 0  frame 0

        TX packets 1488439  bytes 669987921 (638.9 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Step 3. Verify the reachability to the Nexus Dashboard cluster :

1.    On the DCNM, check the route to the Nexus Dashboard data network. Make sure that there is a valid route for the Nexus Dashboard data network via the DCNM eth2 interface. In the flowing example, the IP address 192.168.1.101 is one of the Nexus Dashboard node data network IP address.

[sysadmin@DCNM]$ ip route get 192.168.1.101

192.168.1.101 dev eth2 src 192.168.1.201

2.    On the DCNM, ping one of the Nexus Dashboard data network IP address. As long as there is no firewall or other network devices blocking the ICMP traffic, the ping should succeed.

[sysadmin@DCNM]$ ping 192.168.1.101 -> verifying reachability to ND

PING 192.168.1.101 (192.168.1.101) 56(84) bytes of data.

64 bytes from 192.168.1.101: icmp_seq=1 ttl=64 time=1.16 ms

64 bytes from 192.168.1.101: icmp_seq=2 ttl=64 time=2.34 ms

64 bytes from 192.168.1.101: icmp_seq=3 ttl=64 time=0.682 ms

64 bytes from 192.168.1.101: icmp_seq=4 ttl=64 time=0.130 ms

64 bytes from 192.168.1.101: icmp_seq=5 ttl=64 time=0.288 ms

^C

--- 192.168.1.101 ping statistics ---

5 packets transmitted, 5 received, 0% packet loss, time 4003ms

rtt min/avg/max/mdev = 0.130/0.922/2.346/0.796 ms

Prepare DCNM-Managed Network Sites for Nexus Dashboard Insights

DCNM supports managed mode which requires discovering nodes, adding them to a fabric and assigning fabric roles. In managed mode, DCNM manages the switch configuration. It deploys the config-profiles as configured in the fabric building process and monitors the network site.

This section provides information to prepare a DCNM-managed network site for Nexus Dashboard and Nexus Dashboard Insights. The task flow is as listed below:

●     Create fabric

●     Discover switches

●     Create/Use routable loopback interfaces on switches

●     Configure NTP

●     Configure PTP

Create a Network Fabric

To create a controlled SDN fabric that can be managed and monitored by the DCNM, perform the following steps:

Procedure:

Step 1. Click Control > Create Fabric.

DCNM supports multiple fabric types of fabrics (for example: LAN classic, VxLAN fabrics and so on). Select the template of interest. For this paper, we use a VxLAN fabric as the example. In the Add Fabric page enter Fabric Name and choose ‘Easy_Fabric_11_1’ template to add a managed VxLAN fabric.

For this paper, we use a VxLAN fabric as the example. In the Add Fabric page enter Fabric Name and choose ‘Easy_Fabric_11_1’ template to add a managed VxLAN fabric.

Step 2. Fill in the details in different fields populated for the chosen template. Ex: Easy_Fabric_11_1. Click Save to submit the configuration.

Refer to ‘Easy Provisioning of VxLAN BGP EVPN Fabrics’ in the DCNM configuration guide https://www.cisco.com/c/en/us/td/docs/dcn/dcnm/1151/configuration/lanfabric/cisco-dcnm-lanfabric-configuration-guide-1151.html.

Discover Switches in the Fabric

DCNM can use a single seed or multiple IPs in the fabric and dynamically discover the switches for a set number of hops defined in ‘Max Hops’. DCNM can also list all the switch IPs in the fabric with a hop count ‘0’. You can select the switches to add to the fabric.

Procedure:

Step 1. Click on Add switches and provide a seed IP (management IP) of any switch in the fabric to be discovered. Add all the switches DCNM discovers pertaining to the fabric.

Step 2. After the switches are added to the fabric, assign roles to the switches by right-clicking on any of the switches and choosing the intended role, such as Spine, Leaf, Border Gateway and etc.

Step 3. Click Save & Deploy

Configure a routable loopback interface on the switches

For a managed easy fabric, DCNM deploys an underlay routing loopback and a VTEP loopback on the switches. To deploy Nexus Dashboard Insights for the DCNM site, you can use either of the loopback if it is reachable to the data network of the Nexus Dashboard cluster.

Procedure:

Step 1. Navigate to the fabric settings by choosing the managed fabric that you deployed.

Step 2. Go to the Protocols tab and enable loopback IDs and Save

Step 3. Save and deploy the fabric to get loopback IDs created.

Step 4. Log in to the switch to check if the loopback mentioned above is created on the switches and can ping the Nexus Dashboard cluster.

 

leaf11# show run interface loopback 0

 

interface loopback0 >>> Check if the loopback 0 is created

  description Routing loopback interface

  ip address 10.2.0.1/32 >>> IP address dynamically allocated

  ip router ospf UNDERLAY area 0.0.0.0

  ip pim sparse-mode

 

leaf11# show interface loopback 0

loopback0 is up >>> Check status of the loopback 0

admin state is up,

  Hardware: Loopback

  Description: Routing loopback interface

  Internet Address is 10.2.0.1/32

  MTU 1500 bytes, BW 8000000 Kbit , DLY 5000 usec

  reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation LOOPBACK, medium is broadcast

  Auto-mdix is turned off

    346284 packets input 30982434 bytes

    0 multicast frames 0 compressed

    0 input errors 0 frame 0 overrun 0 fifo

    0 packets output 0 bytes 0 underruns

    0 output errors 0 collisions 0 fifo

    0 out_carrier_errors

 

leaf11# ping 192.168.1.201 source-interface loopback 0 >> checking reachability to the Nexus Dashboard

PING 192.168.1.201 (192.168.1.201): 56 data bytes

64 bytes from 192.168.1.201: icmp_seq=0 ttl=62 time=0.738 ms

64 bytes from 192.168.1.201: icmp_seq=1 ttl=62 time=0.45 ms

64 bytes from 192.168.1.201: icmp_seq=2 ttl=62 time=0.483 ms

64 bytes from 192.168.1.201: icmp_seq=3 ttl=62 time=0.429 ms

64 bytes from 192.168.1.201: icmp_seq=4 ttl=62 time=0.367 ms

 

--- 192.168.1.201 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss

round-trip min/avg/max = 0.367/0.493/0.738 ms

 

NTP (Network Time Protocol) Configuration

For a network site managed by the DCNM, enable and configure NTP on the DCNM. The DCNM pushes the NTP configs to all the switches.

Procedure:

Step 1. From the DCNM homepage Click on Control -> Fabric Builder to go to the fabric builder page.

Step 2. Click on the fabric to setup NTP. The following example selects the fabric named “SJ_DCNM”.

Step 3. Click on Fabric Settings.

Step 4. Navigate to the ‘Manageability’ tab to enter NTP server IP and VRF details and click Save.

Step 5. Click on Save & Deploy to ensure that switches are configured with the required NTP server settings as configured in the DCNM. Click on Deploy Config in the popup.

The switches turn green to indicate In Sync or Success.

PTP (Precision Time Protocol) Configuration

When PTP is enabled, it becomes the default clock even if NTP is enabled on the switches. PTP requires a source loopback used for exchanging PTP packets and a PTP domain ID that defines the boundaries of the PTP messages.

DCNM offers easy site setup for enabling PTP.

Procedure:

Step 1. From the DCNM homepage Click on Control -> Fabric Builder to go to the fabric builder page.

Step 2. Click on the fabric to setup PTP. The example shown below selects the fabric named “SJ_DCNM”.

Step 3. Click on Fabric Settings

Step 4. Navigate to the ‘Advanced’ tab and select ‘Enable Precision Time Protocol (PTP)’. Provide the ‘PTP Source Loopback Id’, the ‘PTP Domain Id’ and click Save. This enables PTP globally and on core-facing interfaces.

Step 5. Click on Save & Deploy to ensure that switches are configured with the required PTP settings as configured in the DCNM. Click on Deploy Config in the popup page

 

Step 6. After the switches turn green (In Sync/Success). Log in to the switches to confirm the configuration and clock settings.

Prepare DCNM Monitored Network Sites for Nexus Dashboard Insights

DCNM supports monitored mode which requires discovering nodes and adding them to a fabric. In monitored mode, DCNM does not manage the switch configuration and typically only helps monitor the fabric. This mode can work complementary to any configuration tools and methods used as it is agnostic of the switch configurations. For a DCNM monitored network site, users need to deploy and verify the needed switch configuration for Nexus Dashboard and Nexus Dashboard Insights by themselves.

This section provides information to prepare the DCNM monitored fabric for Nexus Dashboard and Nexus Dashboard Insights. The task flow is as follows:

●     Create fabric

●     Discover switches

●     Create/Use routable loopback interfaces on the switches

●     Configure NTP

●     Configure PTP

Create Fabric

This section shows you how to create a DCNM monitored fabric.

Procedure:

Step 1. Click Control > Create Fabric.

Step 2. In the Add Fabric page enter Fabric Name and choose ‘External_Fabric_11_1’ template to add a monitored VxLAN fabric.

Step 3. Fill in the BGP AS# used in the site.

Step 4.   Check the box next to “Fabric Monitor Mode”.

Step 5.   Click Save.

Note:       As the fabric is only monitored by the DCNM, the switches in this fabric are configured by the user. DCNM does not deploy any configurations to the switches in this mode.

Discover Switches in the Fabric

DCNM can use a single seed or multiple IPs in the fabric and dynamically discover the switches for a set number of hops defined in ‘Max Hops’. DCNM can also list of all switch IPs in the fabric with a hop count ‘0’. It allows selection of switches to be added to the fabric.

Procedure:

Step 1. Click on Add switches and provide a seed IP (management IP) of any switch in the fabric to be discovered. Add all the switches DCNM discovers pertaining to the fabric.

Step 2. After the switches are added to the fabric, assign roles to the switches, such as Spine, Leaf, Border Gateway and etc.

Step 3. Click Save & Deploy

Configure a Routable Loopback Interface on Switches

Each switch in the network site needs a routable loopback interface to source the telemetry data to the Nexus Dashboard Insights. Any existing loopback on the switches with the required IP connectivity to the Nexus Dashboard data network could be used or users can create a new loopback for the purpose.

For a DCNM monitored fabric, users need to configure and manage such a loopback interface on the switches by themselves. Below shows the procedure.

Procedure:

Step 1. Configure a loopback interface on the switches

leaf21(config)# interface loopback 0

leaf21(config-if)# description Routing loopback interface

leaf21(config-if)# ip address 20.2.0.1/32

leaf21(config-if)# ip router ospf underlay area 0.0.0.0

 

leaf21# show run interface loopback 0 >> check for interface loopback0

  description Routing loopback interface

  ip address 20.2.0.1/32

  ip router ospf underlay area 0.0.0.0

 

leaf21# show interface loopback 0

loopback0 is up

admin state is up,

  Hardware: Loopback

  Description: Routing loopback interface

  Internet Address is 20.2.0.1/32

  MTU 1500 bytes, BW 8000000 Kbit , DLY 5000 usec

  reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation LOOPBACK, medium is broadcast

  Auto-mdix is turned off

    1031148 packets input 96462171 bytes

    0 multicast frames 0 compressed

    0 input errors 0 frame 0 overrun 0 fifo

    0 packets output 0 bytes 0 underruns

    0 output errors 0 collisions 0 fifo

    0 out_carrier_errors

Step 2. Check if the loopback created can reach the Nexus Dashboard Data Network by pinging the Nexus Dashboard Data Network IP address from the loopback interface. In the example below, the IP address 192.168.1.201 is one of the Nexus Dashboard Data Network IP address. If there is no firewalls or other network devices blocking the ICMP traffic, the ping should succeed.

leaf21# ping 192.168.1.201 source-interface loopback 0 

 

PING 192.168.1.201 (192.168.1.201): 56 data bytes

64 bytes from 192.168.1.201: icmp_seq=0 ttl=62 time=0.56 ms

64 bytes from 192.168.1.201: icmp_seq=1 ttl=62 time=0.431 ms

64 bytes from 192.168.1.201: icmp_seq=2 ttl=62 time=0.38 ms

64 bytes from 192.168.1.201: icmp_seq=3 ttl=62 time=0.449 ms

64 bytes from 192.168.1.201: icmp_seq=4 ttl=62 time=0.379 ms

 

--- 192.168.1.201 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss

round-trip min/avg/max = 0.379/0.439/0.56 ms

Note:       For an effective verification, make sure that the ping is sourced from the Loopback interface.

NTP (Network Time Protocol) Configuration

You must enable NTP and configure the NTP server on each switch in the network site monitored by the DCNM. Ensure that all the switches have the consistent NTP configuration and in synchronization with the same NTP server.

Procedure:

In DCNM monitored mode, all the switches in the network site must be configured with an NTP server.

spine12# config terminal

Enter configuration commands, one per line. End with CNTL/Z.

spine12(config)# ntp server 72.163.32.44 use-vrf management

PTP (Precision Time Protocol) Configuration

To provide flow analytics functions, Nexus Dashboard Insights s microsecProtocol(PTP).

For a DCNM managed or monitored network site, it is required to have an external PTP grandmaster that provides a clock source with at least microsecond accuracy.

When PTP is enabled, it becomes the default clock even if NTP is enabled on the switches. PTP requires a source loopback used for exchanging PTP packets and a PTP domain ID that defines the boundaries of the PTP messages.

For a DCNM monitored network site, you must configure  PTP on each network switch by themselves since the DCNM does not manage the switch configuration. This section discusses how to configure PTP on an NX-OS switch. 

Procedure:

Step 1. Enable feature PTP:

leaf21# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

leaf21(config)#  feature ptp  -> Enabling feature ptp

Step 2. Configure PTP domain ID and PTP source interface (a routable loopback)

leaf21(config)# ptp domain 1  -> PTP domain ID

leaf21(config)# ptp source 10.2.0.1-> PTP source IP

Step 3. Configure PTP under core facing interfaces and ttag under Host facing interfaces:

interface Ethernet1/1  -> Core facing interface

  ptp

 

interface Ethernet1/33 -> host facing interface

  ttag

  ttag-strip

NTP and PTP verifications

With either managed or monitored mode fabrics, verifications on the switch remain the same.

NTP verifications

Log in to the switches to confirm the configuration and clock settings. Verify below commands for NTP setup on the switch as the clock time source.

 

Leaf21(config)# show running configuration ntp

 

!Command: show running-config ntp

!No configuration change since last restart

!Time: Thu Aug 26 21:54:40 2021

 

version 9.3(7) Bios:version 05.40

ntp server 72.163.32.44 use-vrf management -> Verify the configuration

 

Leaf11(config)# show clock

21:53:34.997 UTC Thu Aug 26 2021

Time source is NTP -> Verify that NTP is the time source

 

Leaf11(config)# show ntp peers

--------------------------------------------------

  Peer IP Address               Serv/Peer         

--------------------------------------------------

  72.163.32.44           Server (configured) -> Verify that the server is configured

PTP Verifications

After enabling PTP using DCNM or CLI configurations, use the following commands to verify PTP on the switch as the clock time source.

leaf11# show run ptp

feature ptp. -> Verify that PTP is enabled and configured on the interfaces

ptp source 10.2.0.1

ptp domain 0

interface Ethernet1/1

  ptp

interface Ethernet1/33

  ttag

  ttag-strip   

 

leaf11# show clock

01:56:04.353 UTC Fri Aug 27 2021

Time source is PTP -> Verify that PTP is the time source

 

leaf11# show ptp clock foreign-masters record

 

P1=Priority1, P2=Priority2, C=Class, A=Accuracy,

OSLV=Offset-Scaled-Log-Variance, SR=Steps-Removed

GM=Is grandmaster

 

---------   -----------------------  ---  ----  ----  ---  -----  --------

Interface         Clock-ID           P1   P2    C     A    OSLV   SR  

---------   -----------------------  ---  ----  ----  ---  -----  --------

 

Eth1/1   2c:4f:52:ff:fe:56:61:1f    255  255   248   254  65535  1   

 

-> Verify it can reach the grand master on its ptp configured interfaces

 

leaf11# show ptp clock

PTP Device Type : boundary-clock 

PTP Device Encapsulation : NA

PTP Source IP Address : 10.2.0.1 -> Verify that source loopback IP is as configured

Clock Identity : d4:78:9b:ff:fe:19:87:c3

Clock Domain: 0

Slave Clock Operation : Two-step

Master Clock Operation : Two-step

Slave-Only Clock Mode : Disabled

Number of PTP ports: 3

Priority1 : 255

Priority2 : 255

Clock Quality:

        Class : 248

        Accuracy : 254

        Offset (log variance) : 65535

Offset From Master : 12

Mean Path Delay : 168

Steps removed : 2

Correction range : 100000

MPD range : 1000000000

Local clock time : Fri Aug 27 01:56:08 2021

PTP Clock state               : Locked

 

leaf11# show ptp parent

 

PTP PARENT PROPERTIES

 

Parent Clock:

Parent Clock Identity: 2c:4f:52:ff:fe:56:61:1f

Parent Port Number: 4

Observed Parent Offset (log variance): N/A

Observed Parent Clock Phase Change Rate: N/A

 

Parent IP: 10.2.0.4

Grandmaster Clock:

Grandmaster Clock Identity: 00:ee:ab:ff:fe:3a:16:e7 -> Get the GM clock ID

Grandmaster Clock Quality:

        Class: 248

        Accuracy: 254

        Offset (log variance): 65535

        Priority1: 255

        Priority2: 255

 

spine12# show ptp clock foreign-masters record

 

P1=Priority1, P2=Priority2, C=Class, A=Accuracy,

OSLV=Offset-Scaled-Log-Variance, SR=Steps-Removed

GM=Is grandmaster

 

---------   -----------------------  ---  ----  ----  ---  -----  --------

Interface         Clock-ID           P1   P2    C     A    OSLV   SR  

---------   -----------------------  ---  ----  ----  ---  -----  --------

 

Eth1/4    00:ee:ab:ff:fe:3a:16:e7    255  255   248   254  65535  0    GM    

 

-> Check the GM clock ID and confirm the right GM registration on clients

Onboard a DCNM Network Site to Nexus Dashboard

Before adding a DCNM network site to Nexus Dashboard, ensure the following:

●     Both primary and standby DCNM servers must return HA status: OK

●     Nexus dashboard cluster has configured fabric connectivity and has cluster health: OK

Procedure to add a DCNM site to the Nexus Dashboard:

Step 1. From the Nexus Dashboard home page click on Admin Console > Sites > Add Site.

Step 2. Select ‘Site Type’ as DCNM. In the Hostname/IP address field, provide the DCNM in-band IP address configured on the ‘eth2’ interface of the DCNM server. Provide ‘Username’ and ‘Password’ of the DCNM server. Click ‘Select Sites’ to choose the sites to onboard onto the Nexus Dashboard by clicking ‘Save’.

Chosen sites now are onboarded onto the Nexus Dashboard.

Enable Nexus Dashboard Insights for a DCNM Network Site

Below is the task checklist to enable Nexus Dashboard Insights service for a DCNM network site:

●     Configure the required persistent IP addresses on the Nexus Dashboard

●     Install and Enable the Nexus Dashboard Insights service on the Nexus Dashboard (only required if the Insights service is not installed or enabled).

●     Enable the Nexus Dashboard Insights service for the site

Configure persistent IP addresses in Nexus Dashboard

Nexus Dashboard Insights requires 6 mandatory persistent IP addresses for software and hardware telemetry services for a DCNM managed or monitored network site. The 6 persistent IP addresses are assigned to:

●     3 UTR(Software Telemetry) application services, and,

●     3 flow collector (Hardware Telemetry) application services.

These IP addresses are mandatory for Nexus Dashboard Insights operations. A 7th persistent IP address is required if IPv6 is used for the assurance data collection and transport between the network site and the Nexus Dashboard. These persistent IP addresses allows you to retain the same IP addresses for the Nexus Dashboard Insights services even if the internal microservice pods are relocated to different Nexus Dashboard nodes.

These persistent IP addresses are in the Nexus Dashboard Data Network subnet. They must be configured on the Nexus Dashboard prior to enabling the Nexus Dashboard Insights app.

Procedure:

Step 1. On the Nexus Dashboard home page click ‘Admin Console’

Step 2. Navigate to Infrastructure > Cluster Configuration.

Step 3. Edit ‘External Service Pools’ and add six IP addresses from the data network subnet and click Save.

After the persistent IP addresses are added to the Nexus Dashboard, the Nexus Dashboard Insights App can be installed and enabled on the Nexus Dashboard.

Note:       If Nexus Dashboard Insights is used for a DCNM managed/monitored network site, it’s required that the Nexus Dashboard nodes are Layer-2 adjacent to one another in the Data Network. In another word, the Nexus Dashboard nodes need to be in the same IP subnet for their Data Network.  This is needed to allow the persistent IP addresses to move among the Nexus Dashboard nodes.

Install and Enable Nexus Dashboard Insights App

Procedure:

Step 1. On navigating to Admin Console -> Services -> App Store tab, you can Install the Nexus Dashboard Insights App.

Step 2. If the Nexus Dashboard cannot reach the Cisco App Store directly due to network connectivity constrains, users can perform an “offline” installation of the Nexus Dashboard Insights app. First,  download the application from the Cisco DC App Center at https://dcappcenter.cisco.com/ to your local computer or a web server. Next, install the app on the Nexus Dashboard by following the steps in below:

1.    On the Nexus Dashboard UI, navigate to Admin Console -> Services -> Installed Services.

2.    Click on the “Action” button, then choose “Upload Service”.

3.    On the pop-up “Upload Service” screen, choose “Local” for location if the app image file is on your local computer, then choose the file from your local drive. Alternatively, you can choose “Remote” if the app file is on a webserver, then provide the URL for the file.

4.    Click on the “Upload” button to start the upload and installation.

 

Step 3. Once Nexus Dashboard Insights is installed, click ‘Enable’ to get a popup that lets you chose the deployment profile. Select the appropriate profile by reading the description and complying to the Resource Requirement provided for each of the Deployment Profile. Click ‘Enable’ again to apply certificates and start services required for Nexus Dashboard Insights.

 

Refer to the detailed guide for Nexus Dashboard Insights Installation App on the Nexus Dashboard: https://www.cisco.com/c/dam/en/us/td/docs/dcn/ni/5x/user-guide/dcnm/cisco-ni-user-guide-release-511-dcnm.pdf

Enable Nexus Dashboard Insights Service for a DCNM Site

This task is performed on the Nexus Dashboard Insights UI.

Procedure:

Step 1.     Create a site group and add the DCNM site to the group.

Step 2. If this is the first site group on the Nexus Dashboard Insights, the following screen shows up on the UI to guide the user to add the site group. Click on “Configure Site Group” on the screen to start the process.

Alternative, go to the gearbox on the top-right corner, select “Site Groups” -> “Manage”, then choose “Add New Site Group”.

Step 3. Click on ‘Configure Site Groups’ and review the prerequisites for Insights. Click ‘Let’s Get Started > Configure > Add New Site Group’ to provide a ‘Name’ for the Site group. Select ‘Add Site(s)’ and click ‘+Add Member > Select Member’ to select the sites from Nexus Dashboard. Choose the site to be added and click ‘Select’.

Step 4. Once the site is added in the Site Group, click ‘Enable ‘ and ‘configure’ and fill in the fabric type and credentials. Provide the Loopback interface information that is configured in the previous section. This Loopback interface IP address need to be reachable to the Nexus Dashboard Data Network.

Step 5. Click ‘Save’ on the pop-up and change the Status to ‘Enable’.

Step 6. Click on the menu (…) option Configure Site Group to view the sites enabled with Nexus Dashboard Insights services.

 

Enabling Bug Scan, Best Practices and Assurance Analysis

Procedure:

Step 1. After the site-group is configured, to enable Bug Scan feature, click on Bug Scan > (Edit) > Enabled and provide parameters for the Start and End time along with frequency of scan.

Step 2. Click Save.

Step 3. Click Run Now to trigger an instant bug scan.

Step 4. To enable Best Practices checks, click on Best Practices > (Edit) > Enabled and provide parameters for Start and End time along with frequency of best practice checks.

Step 5. Click Save.

Step 6. Click Run Now to trigger an instant best practice check in the requested site.

Step 7. To enable Assurance Analysis, click on Assurance Analysis > (Edit) > Enabled. Provide parameters for Start and End time along with frequency of best practice checks and timeout parameter for how long to run the job.

Step 8.   Click Save.

Step 9.   Click Run Now to trigger an instant Assurance Analysis in the requested site.

Enabling Software and Flow Telemetry

Nexus Dashboard Insights provides intuitive and simple software and flow telemetry enabling with DCNM network sites.

Enable Software Telemetry

Procedure:

Step 1. When configuring the site-group, after the site is enabled and Configuration Status is OK. Nexus Dashboard Insights generates the required configuration per switch to enable software telemetry.

Step 2.   In DCNM-managed mode, the generated software telemetry configurations are deployed on the switches. However, in DCNM monitored mode, you must configure the switches with the generated configurations.

Step 3.   To fetch generated configurations to use for DCNM monitored mode network sites, Click on the visual statuses under Node Status.

Step 4. • Click on any switch/node and check for SOFTWARE TELMETRY CONFIGURATION STATUS under Status.

Step 5.   To fetch the auto-generated configurations, click on View Expected Configuration -> Software Telemetry tab. To copy the configuration to the clipboard click on Copy Configuration.

Step 6.   Repeat the process for each of the switches in the DCNM monitored mode network site.

Enable Flow Telemetry

Procedure:

Step 1. In the Configure Site Group page, click Flows > (Edit) of the desired site. In the pop-up page,, choose the desired flow collection mode.

Step 2. Under the Flow Telemetry Rules, create a rule name and then populate the 5-tuple information

Step 3.   Click Save.

Nexus Dashboard Insights auto-generates the configurations required on the switches in the network sites for enabling the created rules.

Step 4.   If the network sites are in DCNM-managed mode, the auto-generated flow telemetry configurations are auto deployed on the switches by DCNM. If the sites are  in DCNM monitored mode, the required configuration must be deployed to the switches by the users. 

Step 5.   To fetch the auto-generated configuration, in the Configure Site Group page, click General > Site under Collection Status > Node Status > Switch/Node > View Expected Configuration

Step 6.   On the pop-up Configuration window, select Flow Telemetry tab. Configuration with ACLs per rules created in the previous step are auto-generated along with collector information, bucket IDs, ports, and other parameters such as filters and frequency of information collection.

It copies the configuration to the default clipboard. Use it to configure the switch in the DCNM monitored mode network site.

Step 7.   Repeat the process for all the switches in the DCNM monitored mode network site.

Software and Flow Telemetry configuration verifications

Verifications on the Nexus Dashboard Insights

●     In the DCNM-managed mode, the auto-generated configurations by Nexus Dashboard Insights to enable software and flow telemetry is pushed to all switches in the site. The status reflects as success when you click on the status for each switch.

●     In the DCNM monitored mode, although the configurations are auto-generated by Nexus Dashboard Insights to enable software and flow telemetry the configuration on the switches is manual. The status in this case is monitoring.

Verifications on the switches in the network site

Whether the switches are configured manually in DCNM monitored mode or configurations are auto deployed in DCNM-managed mode, switch verification methods remain consistent for the two modes.

Software Telemetry Configuration Verifications:

Run ‘show run telemetry’ on each of the switches to match the configuration to Nexus Dashboard Insights generated configuration.

Verify that the correct receiver IP and port are configured on the switch and is connected.

spine21# show telemetry transport

Session Id      IP Address      Port       Encoding     Transport  Status   

--------------------------------------------------------------------------------

0               192.168.1.71    57500      GPB-compact  gRPC       Connected

0               192.168.1.71    57500      GPB          gRPC       Connected

--------------------------------------------------------------------------------

 

Retry buffer Size:             10485760           

Event Retry Messages (Bytes):  0                  

Timer Retry Messages (Bytes):  0                  

Total Retries sent:            0                  

Total Retries Dropped:         0                  

 

Hardware Telemetry Configuration Verifications:

Verify that collectors, port and ACL information are configured.

 

Conclusion

There is a set of prerequisite configuration for a DCNM managed or monitored network site before it can run Nexus Dashboard Insights properly. This white paper discusses the details on such-required configuration. You can use this as a reference to get your DCNM managed or monitored network sites ready for Nexus Dashboard Insights. It also provides the step-by-step guidance on how to add a network site to Nexus Dashboard Insights and how to enable Nexus Dashboard Insights services, such as bug scan, software telemetry, and flow telemetry functions.

References

Nexus Dashboard 2.1.x User Guide

https://www.cisco.com/c/dam/en/us/td/docs/dcn/nd/2x/user-guide/cisco-nexus-dashboard-user-guide-211.pdf

Nexus Dashboard Insights 6.0.x User Guide

https://www.cisco.com/c/dam/en/us/td/docs/dcn/ndi/6x/user-guide/dcnm/cisco-ndi-user-guide-release-601-dcnm.pdf

 

Getting your Cisco ACI Fabrics Ready for Nexus Dashboard Insights

https://www.cisco.com/c/en/us/td/docs/dcn/whitepapers/getting-aci-ready-for-ndi.html