Editing External Fabric Settings

General Parameters

The General Parameters tab is displayed by default. The fields in this tab are described in the following table.

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Advanced

The fields in the Advanced tab are described in the following table.

Field	Description
Power Supply Mode	Choose the appropriate power supply mode.
Enable MPLS Handoff	Check this check box to enable the MPLS Handoff feature. For more information, see MPLS SR and LDP Handoff.
Underlay MPLS Loopback Id	Specifies the underlay MPLS loopback ID. The default value is 101.
Enable AAA IP Authorization	Enables AAA IP authorization, after IP authorization is enabled on the AAA server.
Enable NDFC as Trap Host	Check this check box to enable the Nexus Dashboard Fabric Controller as a trap host.
Enable CDP for Bootstrapped Switch	Check the check box to enable CDP for the bootstrapped switch.
Enable NX-API	Specifies enabling of NX-API on HTTPS. This check box is unchecked by default.
NX-API HTTPS Port NUmber	Specifies the NX-API HTTPS port number.
Enable HTTP NX-API	Specifies enabling of NX-API on HTTP. This check box is unchecked by default. Enable this check box and the Enable NX-API check box to use HTTP. If you uncheck this check box, the applications that use NX-API and supported by Cisco Nexus Dashboard Fabric Controller, such as Endpoint Locator (EPL), Layer 4-Layer 7 services (L4-L7 services), VXLAN OAM, and so on, start using the HTTPS instead of HTTP. If you check the Enable NX-API check box and the Enable NX-API on HTTP check box, applications use HTTP.
NX-API HTTP Port NUmber	Specifies the NX-API HTTP port number.
Inband Mgmt	For External and Classic LAN fabrics, this knob enables Nexus Dashboard Fabric Controller to import and manage switches with inband connectivity (reachable over switch loopback, or a routed interface, or SVI interfaces), in addition to management of switches with out-of-band connectivity (reachable over the switch mgmt0 interface). The only requirement is that for inband-managed switches, there should be IP reachability from Nexus Dashboard Fabric Controller to the switches over the Nexus Dashboard data interface, also known as an inband interface. For this purpose, static routes may be needed on the Nexus Dashboard Fabric Controller, that in turn can be configured from Admin > System Settings > Routes. After enabling inband management, during discovery, provide the IPs of all the switches to be imported using inband management and set maximum hops to 0. Nexus Dashboard Fabric Controller has a precheck that validates that the inband-managed switch IPs are reachable over the Nexus Dashboard data interface. After completing the precheck, Nexus Dashboard Fabric Controller discovers and learns about the interface on that switch that has the specified discovery IP in addition to the VRF that the interface belongs to. As part of the process of switch import/discovery, this information is captured in the baseline intent that is populated on the Nexus Dashboard Fabric Controller. For more information, see the section "Inband Management in External Fabrics and LAN Classic Fabrics" in Configuring Inband Management, Inband POAP Management, and Secure POAP. Bootstrap or POAP is only supported for switches that are reachable over out-of-band connectivity, that is, over switch mgmt0. The various POAP services on the Nexus Dashboard Fabric Controller are typically bound to the eth1 or out-of-band interface. In scenarios, where Nexus Dashboard Fabric Controller eth0/eth1 interfaces reside in the same IP subnet, the POAP services are bound to both interfaces.
Enable Precision Time Protocol (PTP)	Enables PTP across a fabric. When you check this check box, PTP is enabled globally and on core-facing interfaces. You can also edit the PTP Source Loopback Id and PTP Domain Id fields. For more information, see the section Precision Time Protocol for External Fabrics.
PTP Source Loopback Id	Specifies the loopback interface ID loopback that is used as the source IP address for all Precision Time Protocol (PTP) packets. The valid values range from 0-1023. The PTP loopback ID cannot be the same as RP, Phantom RP, NVE, or the MPLS loopback ID. Otherwise, an error is generated. The PTP loopback ID can be the same as the BGP loopback or user-defined loopback that is created from Nexus Dashboard Fabric Controller. If the PTP loopback ID is not found during a Save & Deploy, the following error is generated: Loopback interface to use for the PTP source IP is not found. Please create a PTP loopback interface on all the devices to enable the PTP feature.
PTP Domain Id	Specifies the PTP domain ID on a single network. The valid values range from 0-127.
Enable Real Time Interface Statistics Collection	Valid for NX-OS only. Check the box to enable the collection of real time interface statistics.
Interface Statistics Load Interval	Enter the time, in seconds, for the interface statistics load interval (Min:5, Max:300).
CoPP Profile	Choose the appropriate control plane policing (CoPP) profile for the fabric. These profiles are available. dense lenient moderate strict manual The manual option is chosen by default. In general, a fabric-wide CoPP policy is applied to Nexus switches. If manual option is chosen, a customized CoPP profile policy must be defined separately.
Fabric Freeform	You can apply configurations globally across all the devices that are discovered in the external fabric using this freeform field. The devices in the fabric should belong to the same device type and the fabric should not be in monitor mode. The different device types are: NX-OS IOS-XE IOS-XR Others Depending on the device type, enter the configurations accordingly. If some of the devices in the fabric do not support these global configurations, they go out-of-sync or fail during the deployment. Hence, ensure that the configurations you apply are supported on all the devices in the fabric or remove the devices that do not support these configurations.
AAA Freeform Config	You can apply AAA configurations globally across all devices that are discovered in the external fabric using this freeform field.

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Resources

The fields in the Resources tab are described in the following table.

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Configuration Backup

The fields in the Configuration Backup tab are described in the following table.

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Bootstrap

You can configure automatic Cisco Plug n Play (PnP) IP assignment of Cisco Catalyst 9000 Series switches in an External Connectivity Network or a Custom Network fabric by configuring the following.

The fields in the Bootstrap tab are described in the following table.

Field	Description
Enable Bootstrap (For NX-OS and IOS-XE(Cat9K) Switches Only	Check this check box to enable the bootstrap feature for NX-OS and IOS-XE Cisco Catalyst 9000 Series switches. After you enable bootstrap, you can enable the DHCP server for automatic IP address assignment.
Enable Inband POAP	Check this check box to enable inband POAP. You must enable Inband Mgmt on the Advanced tab to enable this option.
Enable Local DHCP Server	Enable the Local DHCP Server check box and enter details for the remaining mandatory fields. From Cisco NDFC Release 12.1.1e, you can choose inband POAP or out-of-band POAP for external fabrics.
Enable Plug n Play for Cat9K	Check this check box to enable PnP for automatic IP assignment for Cisco Catalyst 9000 Series switches. Cisco PnP is supported for Cisco Catalyst 9000 Series switches only.
DHCP Version	Choose DHCPv4 or DHCPv6 from this drop-down list. When you choose DHCPv4, Switch Mgmt IPv6 Subnet Prefix field is disabled. If you choose DHCPv6, the Switch Mgmt IP Subnet Prefix is disabled. Cisco Nexus Dashboard Fabric Controller IPv6 POAP is not supported with Cisco Nexus 7000 Series switches. Cisco Nexus 9000 and 3000 Series switches support IPv6 POAP only when switches are either Layer 2 adjacent (eth1 or out-of-band subnet must be a /64) or they are Layer 3 adjacent residing in some IPv6 /64 subnet. Subnet prefixes other than /64 are not supported. NDFC supports Cisco PnP for IPv4 only. There is no support for IPv6. If you do not check this check box, Nexus Dashboard Fabric Controller uses the remote or external DHCP server for automatic IP address assignment.
Domain name	Specify the domain name for the DHCP server PnP block.
DHCP Scope Start Address and DHCP Scope End Address	Specifies the first and last IP addresses of the IP address range to be used for the switch out-of-band POAP.
Switch Mgmt Default Gateway	Specifies the default gateway for the management VRF on the switch.
Switch Mgmt IP Subnet Prefix	Specifies the prefix for the mgmt0 interface on the switch. The prefix range is 8-30.
Switch Mgmt IPv6 Subnet Prefix	Specifies the IPv6 prefix for the Mgmt0 interface on the switch. The prefix should be from 112 through 126. This field is editable if you enable IPv6 for DHCP.
Enable AAA Config	Check this check box to include AAA configs from Advanced tab during device bootup.
Bootstrap Freeform Config (Optional)	Enter other commands as needed. For example, if you are using AAA or remote authentication-related configurations, add these configurations in this field to save the intent. After the devices boot up, they contain the intent that is defined in the Bootstrap Freeform Config field. Copy-paste the running-config to a freeform config field with correct indentation, as seen in the running configuration on the NX-OS switches. The freeform config must match the running config. For more information, see the section "Enable freeform configurations on fabric switches" in Working with Inventory in Your Nexus Dashboard LAN or IPFM Fabrics.
DHCPv4 Multi Subnet Scope	Specifies the field to enter one subnet scope per line. This field is editable after you check the Enable Local DHCP Server check box. The format of the scope should be defined as: DHCP Scope Start Address, DHCP Scope End Address, Switch Management Default Gateway, Switch Management Subnet Prefix Example: 10.6.0.2, 10.6.0.9, 10.6.0.1, 24

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Flow Monitor

The fields in the Flow Monitor tab are described in the following table.

Field	Description
Enable NetFlow	Check this check box to enable NetFlow on VTEPs for this fabric. By default, NetFlow is disabled. On Enable, NetFlow configuration will be applied to all VTEPS that support NetFlow. When NetFlow is enabled on the fabric, you can choose not to have NetFlow on a particular switch by having a dummy no_netflow PTI. If NetFlow is not enabled at the fabric level, an error message is generated when you enable NetFlow at the interface, network, or VRF level. For information about NetFlow support for Cisco NDFC, see the "Configuring Netflow support" section in Creating LAN and ACI Fabrics and Fabric Groups.
NetFlow Exporter	Click Actions > Add to add one or more NetFlow exporters. This exporter is the receiver of the NetFlow data. The fields on this area are: Exporter Name - Specifies the name of the exporter. IP - Specifies the IP address of the exporter. VRF - Specifies the VRF over which the exporter is routed. Source Interface - Enter the source interface name. UDP Port - Specifies the UDP port over which the NetFlow data is exported. Click Save to configure the exporter. Click Cancel to discard. You can also choose an existing Netflow exporter and choose Actions > Edit or Actions > Delete to perform the relevant actions.
NetFlow Record	Click Actions > Add to add one or more NetFlow records. The fields on this area are: Record Name - Specifies the name of the record. Record Template - Specifies the template for the record. Enter one of the record templates names. You can create custom NetFlow record templates. Custom record templates that are saved in the template library are available for use here. netflow_ipv4_record - to use the IPv4 record template. netflow_l2_record - to use the Layer 2 record template. Is Layer 2 Record - Check this check box if the record is for a Layer 2 NetFlow. Click Save to configure the report. Click Cancel to discard. You can also choose an existing record and choose Actions > Edit or Actions > Delete to perform the relevant actions.
NetFlow Monitor	Click Actions > Add to add one or more NetFlow monitors. The fields on this area are: Monitor Name - Specifies the name of the monitor. Record Name - Specifies the name of the record for the monitor. Exporter1 Name - Specifies the name of the exporter for the NetFlow monitor. Exporter2 Name - (optional) Specifies the name of the secondary exporter for the NetFlow monitor. The record name and exporters referred to in each NetFlow monitor must be defined in the Netflow Record and Netflow Exporter areas. Click Save to configure the monitor. Click Cancel to discard. You can also choose an existing Netflow monitor and choose Actions > Edit or Actions > Delete to perform the relevant actions.
NetFlow Sampler	Click Actions > Add to add one or more NetFlow samplers. The fields on this area are: Sampler Name - Specifies the name of the Netflow sampler. Number of Samples - Specifies the number of Netflow samples. Number of Packets in Each Sampling - Specifies the number of packets in each Netflow sampling. The Netflow Sampler is applicable to Cisco Nexus 7000 Series switches only. Click Save to configure the Netflow sampler. Click Cancel to discard. You can also choose an existing Netflow sampler and choose Actions > Edit or Actions > Delete to perform the relevant actions.

What’s next: Complete the configurations in another tab if necessary, or click Save when you have completed the necessary configurations.

Edit configuration settings

The Configuration tab includes these settings.

Edit NAS settings

Nexus Dashboard allows you to export captured flow records to a remote NAS device using the Network File System (NFS) protocol. Nexus Dashboard defines the directory structure on NAS where the flow records are exported.

You can choose between the two export modes.

Nexus Dashboard needs both read and write permissions on the NAS to perform the export successfully. If Nexus Dashboard cannot write to the NAS, it will generate an alert to notify you of the issue.

Disable Telemetry

You can uncheck the Telemetry check box on your fabric’s Edit Fabric Settings > General > page to disable the telemetry feature for your fabric. Disabling telemetry puts the telemetry feature in a transition phase and eventually the telemetry feature is disabled.

In certain situations, the disable telemetry workflow can fail, and you may see the Force disable telemetry option on your fabric’s Edit Fabric Settings page.

If you disable the telemetry option using the instructions provided in [Perform force disable telemetry] on your fabric, Nexus Dashboard acknowledges the user intent to disable telemetry feature for your fabric, ignoring any failures.

The Nexus Dashboard Force disable telemetry allows you to perform a force disable action for the telemetry configuration on your fabric. This action is recommended when the telemetry disable workflow has failed and you need to disable the telemetry feature on your fabric.

Using the Force disable telemetry feature may leave switches in your fabric with stale telemetry configurations. You must manually clean up these stale configurations on the switches before re-enabling telemetry on your fabric.

Perform force disable telemetry on your fabric

Follow these steps to perform a force disable telemetry on your fabric.

NAS

You can export flow records captured by Nexus Dashboard on a remote Network Attached Storage (NAS) with NFS.

Nexus Dashboard defines the directory structure on NAS where the flow records are exported.

You can export the flow records in Base or Full mode. In Base mode, only 5-tuple data for the flow record is exported. In Full mode the entire data for the flow record is exported.

Nexus Dashboard requires read and write permission to NAS in order to export the flow record. A system issue is raised if Nexus Dashboard fails to write to NAS.

Guidelines and limitations for network attached storage

Add network attached storage to export flow records

The workflow to add Network Attached Storage (NAS) to export flow records includes the following steps:

Add the onboarded NAS to Nexus Dashboard

Follow these steps to add the onboarded NAS to Nexus Dashboard.

1. Navigate to the Fabrics page:

   Manage > Fabrics

2. Choose the fabric with the telemetry feature enabled.

3. Choose Actions > Edit Fabric Settings.

4. Click Telemetry.

5. Click the NAS tab in the Telemetry window.

6. Make the necessary configurations in the General settings area.

   1. Enter the name in the Name field.

   2. In the NAS server field, choose the NAS server added to Nexus Dashboard from the drop-down list.

7. In the Collection settings area, choose the flow from the Flows drop-down list.

   • In Base mode, only 5-tuple data for the flow record is exported.

   • In Full mode, the entire data for the flow record is exported.

8. Click Save.

The traffic from the flows displayed in the Flows page is exported as a JSON file to the external NAS in the following directory hierarchy.

Navigate to Analyze > Flows to view the flows that will be exported.

Each flow record is written as a line delimited JSON.

JSON output file format for a flow record in base mode

{"fabricName":"myapic","terminalTs":1688537547433,"originTs":1688537530376,"srcIp":"2000:201:1:1::1","dstIp":"2000:201:1:1::3","srcPort":1231,"dstPort":1232,"ingressVrf":"vrf1","egressVrf":"vrf1","ingressTenant":"FSV1","egressTenant":"FSV1","protocol":"UDP"}

{"fabricName":"myapic","terminalTs":1688537547378,"originTs":1688537530377,"srcIp":"201.1.1.127","dstIp":"201.1.1.1","srcPort":0,"dstPort":0,"ingressVrf":"vrf1","egressVrf":"","ingressTenant":"FSV2","egressTenant":"","protocol":"ANY-HOST"}

JSON output file format for a flow record in full mode

{"fabricName":"myapic","terminalTs":1688538023562,"originTs":1688538010527,"srcIp":"201.1.1.121","dstIp":"201.1.1.127","srcPort":0,"dstPort":0,"ingressVrf":"vrf1","egressVrf":"vrf1","ingressTenant":"FSV2","egressTenant":"FSV2","protocol":"ANY-HOST","srcEpg":"ext-epg","dstEpg":"ext-epg1","latencyMax":0,"ingressVif":"eth1/15","ingressVni":0,"latency":0,"ingressNodes":"Leaf1-2","ingressVlan":0,"ingressByteCount":104681600,"ingressPktCount":817825,"ingressBurst":0,"ingressBurstMax":34768,"egressNodes":"Leaf1-2","egressVif":"po4", "egressVni":0,"egressVlan":0,"egressByteCount":104681600,"egressPktCount":817825,"egressBurst":0,"egressBurstMax":34768,"dropPktCount":0,"dropByteCount":0,"dropCode":"","dropScore":0,"moveScore":0,"latencyScore":0,"burstScore":0,"anomalyScore":0,"hashCollision":false,"dropNodes":"[]","nodeNames":"[\"Leaf1-2\"]","nodeIngressVifs":"[\"Leaf1-2,eth1/15\"]","nodeEgressVifs":"[\"Leaf1-2,po4\"]“ ,"srcMoveCount":0,"dstMoveCount":0,"moveCount":0,"prexmit":0,"rtoOutside":false,"events":"[[\\\"1688538010527,Leaf1-2,0,3,1,no,no,eth1/15,,po4,po4,,,,,0,64,0,,,,,,,,\\\"]]"}

Flow collection

Understanding flow telemetry

Flow telemetry allows users to see the path taken by different flows in detail. It also allows you to identify the EPG and VRF instance of the source and destination. You can see the switches in the flow with the help of flow table exports from the nodes. The flow path is generated by stitching together all the exports in order of the flow.

You can configure the Flow Telemetry rule for the following interface types:

In a Cisco ACI fabric, if you want to configure routed sub-interfaces from the UI, select L3 Out.

In an NX-OS fabric, physical or port channel flow rules are supported only on routed interfaces.

Flow telemetry monitors the flow for each fabric separately, as there is no stitching across the fabrics in a fabric group. Therefore, flow telemetry is for individual flows. For example, if there are two fabrics (fabric A and fabric B) within a fabric group, and traffic is flowing between the two fabrics, they will be displayed as two separate flows. One flow will originate from Fabric A and display where the flow exits. And the other flow from Fabric B will display where it enters and where it exits.

Configure flows

Configure flow collection modes

Follow these steps to configure flow collection modes.

1. Navigate to Admin > System Settings > Flow collection.

2. In the Flow collection mode area, choose Flow telemetry.

---

Enabling Flow Telemetry automatically activates Flow Telemetry Events. Whenever a compatible event takes place, an anomaly will be generated, and the What’s the impact? section in the Anomaly page will display the associated flows. You can manually configure a Flow Telemetry rule to acquire comprehensive end-to-end information about the troublesome flow.

---

Configure flow collection rules in an NX-OS fabric

Follow these steps to configure flow collection rules in an NX-OS fabric.

1. Navigate to the Telemetry window for your fabric.

   1. Navigate to the main Fabrics page:

      Manage > Fabrics

   2. In the table showing all of the Nexus Dashboard fabrics that you have already created, locate the LAN or IPFM fabric where you want to configure telemetry settings.

   3. Single-click on that fabric.

      The Overview page for that fabric appears.

   4. Click Actions > Edit Fabric Settings.

      The Edit fabric_name Settings window appears.

   5. Verify that the Telemetry option is enabled in the Enabled features area.

      The Telemetry tab doesn’t become available unless the Telemetry option is enabled in the Enabled features area.

   6. Click the Telemetry tab to access the telemetry settings for this fabric.

2. Click the Flow collection tab in the Telemetry window.

3. In the Mode area, click Flow telemetry.

4. In the Flow collections rules area, determine what sort of flow collection rule that you want to add.

   • VRF

   • Physical interface

   • Port channel

VRF

To add a VRF rule:

1. Click the VRF tab.

   A table with already-configured VRF flow collection rules is displayed.

   For any VRF flow collection rule in this table, click the ellipsis (…​), then click Edit rule to edit that rule or Delete rule to delete it.

2. Add a new rule by clicking Create flow collection rule.

   1. In the General area, complete the following:

      1. Enter the name of the rule in the Rule Name field.

      2. The VRF field is disabled. The flow rule applies to all the VRF instances.

      3. In the Flow Properties area, select the protocol for which you intend to monitor the flow traffic.

      4. Enter the source and destination IP addresses. Enter the source and destination port.

      5. Click Save.

Physical interface

To add a physical interface rule:

1. Click the Physical interface tab.

   A table with already-configured physical interface flow collection rules is displayed.

   For any physical interface flow collection rule in this table, click the ellipsis (…​), then click Edit rule to edit that rule or Delete rule to delete it.

2. Add a new rule by clicking Create flow collection rule.

   1. In the General area, complete the following:

      1. Enter the name of the rule in the Rule Name field.

      2. Check the Enabled check box to enable the status. If you enable the status, the rule will take effect. Otherwise, the rule will be removed from the switches.

      3. In the Flow Properties area, select the protocol for which you intend to monitor the flow traffic.

      4. Enter the source and destination IP addresses. Enter the source and destination port.

      5. In the Interface List area, click Select a Node. Use the search box to select a node.

      6. From the drop-down list, select an interface. You can add more than one row (node+interface combination) by clicking Add Interfaces. However, within the rule, a node can appear only once. Configuration is rejected if more than one node is added.

      7. Click Save.

Port channel

To add a port channel rule:

1. Click the Port channel tab.

   A table with already-configured port channel flow collection rules is displayed.

   For any port channel flow collection rule in this table, click the ellipsis (…​), then click Edit rule to edit that rule or Delete rule to delete it.

2. Add a new rule by clicking Create flow collection rule.

   1. In the General area, enter the name of the rule in the Rule Name field.

      1. Select the Enabled check box to enable the status. If you enable the status, the rule will take effect. Otherwise, the rule will be removed from the switches.

      2. In the Flow Properties area, select the protocol for which you intend to monitor the flow traffic.

      3. Enter the source and destination IP addresses. Enter the source and destination port.

      4. From the drop-down list, select an interface. You can add more than one row (node+interface combination) by clicking Add Interfaces. However, within the rule, a node can appear only once. Configuration is rejected if more than one node is added.

      5. Click Save.

3. Click Done.

Understanding Netflow

Netflow is an industry standard where Cisco routers monitor and collect network traffic on an interface. Netflow version 9 is supported.

Netflow enables the network administrator to determine information such as source, destination, class of service, and causes of congestion. Netflow is configured on the interface to monitor every packet on the interface and provide telemetry data. You cannot filter on Netflow.

Netflow in Nexus series switches is based on intercepting the packet processing pipeline to capture summary information of network traffic.

The components of a flow monitoring setup are as follows:

In an NX-OS fabric, port channel support is available if you monitor only the host-facing interfaces.

Understanding sFlow

sFlow is an industry standard technology traffic in data networks containing switches and routers. Nexus Dashboard supports sFlow version 5 on Cisco Nexus 3000 series switches.

sFlow provides the visibility to enable performance optimization, an accounting and billing for usage, and defense against security threats.

The following interfaces are supported for sFlow:

Configure external streaming settings

Follow these steps to configure external streaming settings.

Guidelines and limitations

Email

The email scheduler feature in Nexus Dashboard automates the distribution of summarized data collected from Nexus Dashboard. It allows customization of selection of email recipients, choice of email format, scheduling frequency settings, and configuring the types of alerts and reports.

To configure email at the system settings level, see [Add email configuration].

Follow these steps to configure an email scheduler.

To add a new email configuration, click Add email in the Email page.

You will receive an email about the scheduled job on the provided Start Date and at the time provided in the Collection frequency in days field. The subsequent emails follow after Collect Every frequency expires. If the provided time is in the past, Nexus Dashboard will send you an email immediately and trigger the next email after the duration from the provided start time expires.

Message bus

Syslog

Nexus Dashboard supports the export of anomalies in syslog format. You can use the syslog configuration feature to develop network monitoring and analytics applications on top of Nexus Dashboard, integrate with the syslog server to get alerts, and build customized dashboards and visualizations.

After you choose the fabric where you want to configure the syslog exporter and set up the syslog export configuration, Nexus Dashboard establishes a connection with the syslog server and sends data to the syslog server.

Nexus Dashboard exports anomaly records to the syslog server. With syslog support, you can export anomalies to your third-party tools even if you do not use Kafka.

External Fabrics

You can add switches to an external fabric.

External Fabric Depiction in an MSD Fabric Topology

The MSD topology page displays MSD member fabrics and external fabrics together. The external fabric External65000 is displayed as part of the MSD topology.

---

When you deploy networks or VRFs for the VXLAN fabric, the deployment page (MSD topology view) shows the VXLAN and external fabrics that are connected to each other.

---

Adding Switches to the External Fabric

Switches in each fabric are unique, and hence, each switch can only be added to one fabric. To add switches to the external fabric, perform he following steps:

Switch Settings for External Fabrics

External Fabric Switch Settings vary from the VXLAN fabric switch settings. Double-click on the switch to view the Switch Overview screen to edit/modify options.

The options are:

Set Role - By default, no role is assigned to an external fabric switch. You can assign desired role to the fabric. Assign the Core Router role for a Multi-Site Inter-Fabric Connection (IFC) and the Edge Router role for a VRF Lite IFC between the external fabric and VXLAN fabric border devices.

Changing of switch role is allowed only before executing Deploy Config. vPC Pairing - Select a switch for vPC and then select its peer.

Change Modes - Allows you to modify the mode of switch from Active to Operational.

Manage Interfaces - Deploy configurations on the switch interfaces.

Straight-through FEX, Active/Active FEX, and breakout of interfaces are not supported for external fabric switch interfaces.

View/edit Policies - Add, update, and delete policies on the switch. The policies you add to a switch are template instances of the templates available in the template library. After creating policies, deploy them on the switch using the Deploy option available in the View/edit Policies screen.

History - View per switch deployment history.

Recalculate Config - View the pending configuration and the side-by-side comparison of the running and expected configuration.

Deploy Config - Deploy per switch configurations.

Discovery - You can use this option to update the credentials of the switch, reload the switch, rediscover the switch, and remove the switch from the fabric.

Click Deploy from the Actions drop-down list. The template and interface configurations form the configuration provisioning on the switches.

When you click Deploy, the Deploy Configuration screen comes up.

Click Config at the bottom part of the screen to initiate pending configuration onto the switch. The Deploy Progress screen displays the progress and the status of configuration deployment.

Click Close after the deployment is complete.

If a switch in an external fabric does not accept default credentials, you should perform one of the following actions:
* Remove the switch in the external fabric from inventory, and then rediscover.
* LAN discovery uses both SNMP and SSH, so both passwords need to be the same. You need to change the SSH password to match the SNMP password on the switch. If SNMP authentication fails, discovery is stopped with authentication error. If SNMP authentication passes but SSH authentication fails, Nexus Dashboard Fabric Controller discovery continues, but the switch status shows a warning for the SSH error.

Discovering New Switches

To discover new switches, perform the following steps:

Adding Non-Nexus Devices to External Fabrics

From Cisco Nexus Dashboard Fabric Controller Release 12.0.1a, you can add Cisco IOS-XR devices to external fabrics in managed mode as well. You can manage the following Cisco IOS-XR devices in external fabrics:

From Cisco Nexus Dashboard Fabric Controller Release 12.1.1e, you can also add Cisco 8000 Series Routers to external fabrics both in managed mode and monitored mode.

You can discover non-Nexus devices in an external fabric and perform the configuration compliance of these devices as well. For more information, see the Configuration Compliance in External Fabrics section.

Refer the Cisco Compatibility Matrix to see the non-Nexus devices supported by Cisco Nexus Dashboard Fabric Controller.

Only Cisco Nexus switches support SNMP discovery by default. Hence, configure all the non-Nexus devices before adding it to the external fabric. Configuring the non-Nexus devices includes configuring SNMP views, groups, and users. See the Configuring Non-Nexus Devices for Discovery section for more information.

Cisco CSR 1000v is discovered using SSH. Cisco CSR 1000v does not need SNMP support because it can be installed in clouds where SNMP is blocked for security reasons. See the Connecting Cisco Data Center and a Public Cloud chapter to see a use case to add Cisco CSR 1000v to an external fabric.

However, Cisco Nexus Dashboard Fabric Controller can only access the basic device information like system name, serial number, model, version, interfaces, up time, and so on. Cisco Nexus Dashboard Fabric Controller does not discover non-Nexus devices if the hosts are part of CDP or LLDP.

The settings that are not applicable for non-Nexus devices appear blank, even if you get many options when you right-click a non-Nexus device in the fabric topology window. You cannot add or edit interfaces for ASR 9000 Series Routers and Arista switches.

You can add IOS-XE devices like Cisco Catalyst 9000 Series switches and Cisco ASR 1000 Series Routers as well to external fabrics.

Managing Cisco IOS-XR Devices using NDFC

In general, workload requires communication with services outside of the data center domain in a data center fabric. This includes users accessing an application and services from the internet and WAN. VXLAN EVPN fabrics with border devices are considered as a handoff for north-south connectivity. These border devices are in peer with IOS-XR routers, which are the backbone routers for WAN and internet connectivity.

In DCNM Release 11.5(x), users with an admin role can control VXLAN EVPN fabrics with capabilities such as monitoring, automation, and compliance. You can only monitor the IOS-XR routers in monitored mode. Therefore, there is a requirement for a single fabric controller to manage, and automate configurations between these devices to balance and check configuration compliance for communicating between different services.

From NDFC Release 12.0.1a, users with an admin role can manage IOS-XR routers that is limited to automation and compliance checking. New templates and policies are introduced to automate and manage eBGP VRF Lite handoff between border switches and IOS-XR routers. NDFC allows you to check configuration compliance for IOS-XR devices similar to Cisco Nexus switches in external fabrics.

---

For all non-Nexus devices, only the message-digest algorithm (MD5) protocol is supported for Simple Network Management Protocol version 3 (SNMPv3) authentication.

---

---

Beginning with NDFC 12.2.1, you do not need to configure the Simple Network Management Protocol (SNMP) for IOS-XR discovery of switches. NDFC uses Secure Shell (SSH) for IOS-XR device discovery.

---

Configuring IOS-XR as an Edge Router

To extend VRF Lite from a Cisco Nexus 9000 fabric with border devices for IOS-XR as the edge router, see the VRF Lite Between Cisco Nexus 9000 Based Border and Non-Nexus Device section for more information.

For more information, see the video at Managing and Configuring ASR 9000 using NDFC.

Configuring Non-Nexus Devices for Discovery

Before discovering any non-Nexus device in Cisco Nexus Dashboard Fabric Controller, configure it on the switch console.

Configuring IOS-XE Devices for Discovery

---

In case of failure or issues configuring devices contact Cisco Technical Assistance Center (TAC). Before you discover the Cisco IOS-XE devices in Nexus Dashboard Fabric Controller, perform the following steps:

---

1. Run the following SSH commands on the switch console.

   switch (config)# hostname <hostname>
   switch (config)# ip domain name <domain_name>
   switch (config)# crypto key generate rsa
   switch (config)# ip ssh time-out 90
   switch (config)# ip ssh version 2
   switch (config)# line vty 1 4
   switch (config-line)# transport input ssh
   switch (config)# username admin privilege secret <password>
   switch (config)# aaa new-model
   switch (config)# aaa authentication login default local
   switch (config)# aaa authorization exec default local none

2. Before you run SNMP command on the switch, ensure that the IP addresses, username and SNMP related configurations are defined on the switch. Run the following SNMP command on the switch console.

   aaa new-model
   aaa session-id common
    ip domain name cisco
   username admin privilege 15 secret 0 xxxxx
   snmp-server group group1 v3 auth read view1 write view1
   snmp-server view view1 mib-2 included
   snmp-server view view1 cisco included
   snmp-server user admin group1 v3 auth md5 xxxxx priv des xxxxx
   line vty 0 4
   privilege level 15
   transport input all
   line vty 5 15
   privilege level 15
   transport input all
   line vty 16 31
   transport input ssh

Configuring Arista Devices for Discovery

Enable Privilege Exec mode using the following command:

switch> enable
switch#

switch# show running confiruation | grep aaa        /* to view the authorization*/
aaa authorization exec default local

Run the following commands in the switch console to configure Arista devices:

switch# configure terminal
switch (config)# username ndfc privilege 15 role network-admin secret passwd
snmp-server view _view_name_ SNMPv2 included
snmp-server view _view_name_ SNMPv3 included
snmp-server view _view_name_ default included
snmp-server view _view_name_ entity included
snmp-server view _view_name_ if included
snmp-server view _view_name_ iso included
snmp-server view _view_name_ lldp included
snmp-server view _view_name_ system included
snmp-server view sys-view default included
snmp-server view sys-view ifmib included
snmp-server view sys-view system included
snmp-server community private ro
snmp-server community public ro
snmp-server group _group_name_ v3 auth read _view_name_
snmp-server user username _group_name_ v3 auth md5 _password_ priv aes _password_

---

SNMP password should be same as the password for username.

---

You can verify the configuration by running the show run command, and view the SNMP view output by running the show snmp view command.

Show Run Command

switch (config)# snmp-server engineID local f5717f444ca824448b00
snmp-server view _view_name_ SNMPv2 included
snmp-server view _view_name_ SNMPv3 included
snmp-server view _view_name_ default included
snmp-server view _view_name_ entity included
snmp-server view _view_name_ if included
snmp-server view _view_name_ iso included
snmp-server view _view_name_ lldp included
snmp-server view _view_name_ system included
snmp-server view sys-view default included
snmp-server view sys-view ifmib included
snmp-server view sys-view system included
snmp-server community private ro
snmp-server community public ro
snmp-server group _group_name_ v3 auth read _view_name_
snmp-server user _user_name__group_name_ v3 localized f5717f444ca824448b00 auth md5 be2eca3fc858b62b2128a963a2b49373 priv aes be2eca3fc858b62b2128a963a2b49373
!
spanning-tree mode mstp
!
service unsupported-transceiver labs f5047577
!
aaa authorization exec default local
!
no aaa root
!
username admin role network-admin secret sha512 $6$5ZKs/7.k2UxrWDg0$FOkdVQsBTnOquW/9AYx36YUBSPNLFdeuPIse9XgyHSdEOYXtPyT/0sMUYYdkMffuIjgn/d9rx/Do71XSbygSn/
username cvpadmin role network-admin secret sha512 $6$fLGFj/PUcuJT436i$Sj5G5c4y9cYjI/BZswjjmZW0J4npGrGqIyG3ZFk/ULza47Kz.d31q13jXA7iHM677gwqQbFSH2/3oQEaHRq08.
username ndfc privilege 15 role network-admin secret sha512 $6$M48PNrCdg2EITEdG$iiB880nvFQQlrWoZwOMzdt5EfkuCIraNqtEMRS0TJUhNKCQnJN.VDLFsLAmP7kQBo.C3ct4/.n.2eRlcP6hij/

Show SNMP View Command

configure terminal# show snmp view
view_name SNMPv2 - included
view_name SNMPv3 - included
view_name default - included
view_name entity - included
view_name if - included
view_name iso - included
view_name lldp - included
view_name system - included
sys-view default - included
sys-view ifmib - included
sys-view system - included
leaf3-7050sx#show snmp user

User name : _user_name_
Security model : v3
Engine ID : f5717f444ca824448b00
Authentication : MD5
Privacy : AES-128
Group : _group_name_

Configuring and Verifying Cisco IOS-XR Devices for Discovery

To configure IOS-XR devices, run the following commands on the switch console:

switch# configure terminal
switch (config)# snmp-server viewview_namecisco included
snmp-server view _view_name_ mib-2 included
snmp-server group _group_name_ v3 auth read _view_name_ write _view_name_
snmp-server user _user_name__group_name_ v3 auth md5 password priv des56 password SystemOwner

Below shown example of configuring IOS-XR device on a switch.

RP/0/RSP0/CPU0:ios(config)#snmp-server view view_name cisco included
RP/0/RSP0/CPU0:ios(config)#snmp-server view view_namemib-2 included
RP/0/RSP0/CPU0:ios(config)#snmp-server group group_name v3 auth read view_name write view_name
RP/0/RSP0/CPU0:ios(config)#snmp-server user user_name_group_name_ v3 auth md5 password priv des56 passwordSystemOwner
RP/0/RSP0/CPU0:ios(config)#commit Day MMM DD HH:MM:SS Timezone

To verify IOS-XR devices, run the following command:

RP/0/RSP0/CPU0:ios(config)#
RP/0/RSP0/CPU0:ios(config)#show run snmp-server Day MMM DD HH:MM:SS Timezone snmp-server user user_name group1 v3 auth md5 encrypted 10400B0F3A4640585851 priv des56 encrypted 000A11103B0A59555B74 SystemOwner
snmp-server view _view_name_cisco included
snmp-server view _view_name_mib-2 included
snmp-server group group_name v3 auth read view_name write view_namev3 auth read _view_name_ write _view_name_

Managing Non-Nexus Devices to External Fabrics

From Nexus Dashboard Fabric Controller 12.0.1a, IOS-XR is supported in managed mode.

---

Configuration compliance is enabled for IOS-XE and IOS-XR switches, similar to the way the Nexus switches are handled in External Fabric. For more information, see Configuration Compliance in External Fabrics.
Nexus Dashboard Fabric Controller sends commit at the end of deployment for IOS-XR devices.
Nexus Dashboard Fabric Controller provides a few templates for IOS-XR devices. Use the ios_xr_Ext_VRF_Lite_Jython.template for IOS-XR switch to be an edge router to establish eBGP peering with border. This will create config for vrf, eBGP peering for the vrf and the sub-interface. Similarly, ios_xe_Ext_VRF_Lite_Jython can be used for IOS-XE switch to be an edge router to establish eBGP peering with border.

---

Creating a vPC Setup

You can create a vPC setup for a pair of switches in an external fabric. Ensure that the switches are of the same role and connected to each other.

Undeploying a vPC Setup

Precision Time Protocol for External Fabrics

In the Fabric settings for the External Fabric template, select the Enable Precision Time Protocol (PTP) check box to enable PTP across a fabric. When you select this check box, PTP is enabled globally and on core-facing interfaces. Additionally, the PTP Loopback Id and PTP Domain Id fields are editable.

The PTP feature is supported with Cisco Nexus 9000 Series cloud-scale switches, with NX-OS version 7.0(3)I7(1) or later. Warnings are displayed if there are non-cloud scale devices in the fabric, and PTP is not enabled. Examples of the cloud-scale devices are Cisco Nexus 93180YC-EX, Cisco Nexus 93180YC-FX, Cisco Nexus 93240YC-FX2, and Cisco Nexus 93360YC-FX2 switches. For more information, refer to https://www.cisco.com/c/en/us/products/switches/nexus-9000-series-switches/index.html.

PTP global configuration is supported with Cisco Nexus 3000 Series switches; however, PTP and TTAG configurations are not supported.

For more information, see the Configuring PTP chapter in Cisco Nexus 9000 Series NX-OS System Management Configuration Guide and Cisco Nexus Insights for Cisco User Guide.

For External fabric deployments, you have to enable PTP globally, and also enable PTP on core-facing interfaces. The interfaces could be configured to the external PTP server like a VM or Linux-based machine. Therefore, the interface should be edited to have a connection with the grandmaster clock. For PTP and TTAG configurations to be operational on External fabrics, you must sync up of Switch Configs to Nexus Dashboard Fabric Controller using the host_port_resync policy. For more information, see the section "Out-of-Band Switch Interface Configurations" in Add Interfaces for LAN Operational Mode.

It is recommended that the grandmaster clock should be configured outside of Data Center VXLAN EVPN and it is IP reachable. The interfaces toward the grandmaster clock need to be enabled with PTP via the interface freeform config.

All core-facing interfaces are auto-enabled with the PTP configuration after you click Deploy Config. This action ensures that all devices are PTP synced to the grandmaster clock. Additionally, for any interfaces that are not core-facing, such as interfaces on the border devices and leafs that are connected to hosts, firewalls, service-nodes, or other routers, the TTAG related CLI must be added. The TTAG is added for all traffic entering the VXLAN EVPN fabric and the TTAG must be stripped when traffic is exiting this fabric.

Here is the sample PTP configuration:

feature ptp

ptp source 100.100.100.10 -> IP address of the loopback interface (loopback0)
that is already created, or user-created loopback interface in the fabric settings

ptp domain 1 -> PTP domain ID specified in fabric settings

interface Ethernet1/59 -> Core facing interface
  ptp

interface Ethernet1/50 -> Host facing interface
  ttag
  ttag-strip

The following guidelines are applicable for PTP:

First Published: 2025-01-31
Last Modified: 2025-01-31