This procedure shows how to create a new VXLAN BGP EVPN fabric.

Note that this procedure contains descriptions for the IPv4 underlay. For information about IPv6 underlay, see IPv6 Underlay Support for Easy Fabric.

1. Choose Control > Fabric Builder.

   The Fabric Builder screen appears. When you log in for the first time, the Fabrics section has no entries. After you create a fabric, it is displayed on the Fabric Builder screen, wherein a rectangular box represents each fabric.

   A standalone or member fabric contains Switch_Fabric (in the Type field), the AS number (in the ASN field), and mode of replication (in the Replication Mode field).

2. Click Create Fabric. The Add Fabric screen appears.

   The fields are explained:

   Fabric Name - Enter the name of the fabric.

   Fabric Template - From the drop-down menu, choose the Easy_Fabric_11_1 fabric template. The fabric settings for creating a standalone fabric comes up.

   

   The tabs and their fields in the screen are explained in the subsequent points. The overlay and underlay network parameters are included in these tabs.

   Note	If you are creating a standalone fabric as a potential member fabric of an MSD fabric (used for provisioning overlay networks for fabrics that are connected through EVPN Multi-Site technology), then browse through the Multi-Site Domain for VXLAN BGP EVPN Fabrics topic before member fabric creation.

3. The General tab is displayed by default. The fields in this tab are:

   BGP ASN: Enter the BGP AS number the fabric is associated with.

   Enable IPv6 Underlay: Enable the IPv6 underlay feature. For information, see IPv6 Underlay Support for Easy Fabric.

   Enable IPv6 Link-Local Address: Enables the IPv6 Link-Local address.

   Fabric Interface Numbering : Specifies whether you want to use point-to-point (p2p) or unnumbered networks. For information about how to change this field value for an existing fabric, see Changing Fabric Interface Numbering.

   Underlay Subnet IP Mask - Specifies the subnet mask for the fabric interface IP addresses.

   Link-State Routing Protocol : The IGP used in the fabric, OSPF, or IS-IS.

   Route-Reflectors – The number of spine switches that are used as route reflectors for transporting BGP traffic. Choose 2 or 4 from the drop down box. The default value is 2.

   To deploy spine devices as RRs, DCNM sorts the spine devices based on their serial numbers, and designates two or four spine devices as RRs. If you add more spine devices, existing RR configuration will not change.

   Increasing the count - You can increase the route reflectors from two to four at any point in time. Configurations are automatically generated on the other 2 spine devices designated as RRs.

   Decreasing the count

   When you reduce four route reflectors to two, you must remove the unneeded route reflector devices from the fabric. Follow these steps to reduce the count from 4 to 2.

   1. Change the value in the drop-down box to 2.

   2. Identify the spine switches designated as route reflectors.

      An instance of the rr_state policy is applied on the spine switch if it is a route reflector. To find out if the policy is applied on the switch, right-click the switch, and choose View/edit policies. In the View/Edit Policies screen, search rr_state in the Template field. It is displayed on the screen.

   3. Delete the unneeded spine devices from the fabric (right-click the spine switch icon and choose Discovery > Remove from fabric).

      If you delete existing RR devices, the next available spine switch is selected as the replacement RR.

   4. Click Save and Deploy at the top right part of the fabric topology screen.

   Anycast Gateway MAC : Specifies the anycast gateway MAC address.

   NX-OS Software Image Version : Select an image from the list.

   If you upload Cisco NX-OS software images through the image upload option, the uploaded images are listed in this field. If you select an image, the system checks if the switch has the selected version. If not, an error message is displayed. You can resolve the error by clicking on Resolve. The image management screen comes up and you can proceed with the ISSU option. Alternatively, you can delete the release number and save it later.

   If you specify an image in this field, all switches in the fabric should run that image. If some devices do not run the image, a warning is prompted to perform an In-Service Software Upgrade (ISSU) to the specified image. Till all devices run the specified image, the deployment process will be incomplete.

   If you want to deploy more than one type of software image on the fabric switches, don’t specify any image. If an image is specified, delete it

4. Click the Replication tab. Most of the fields are auto generated. You can update the fields if needed.

   

   Replication Mode : The mode of replication that is used in the fabric, Ingress Replication, or Multicast.

   When you choose Ingress replication, the multicast replication fields get disabled.

   You can change the fabric setting from one mode to the other, if no overlay profile exists for the fabric.

   Multicast Group Subnet : IP address prefix used for multicast communication. An unique IP address is allocated from this group for each overlay network.

   In the DCNM 11.0(1) release, the replication mode change is not allowed if a policy template instance is created for the current mode. For example, if a multicast related policy is created and deployed, you cannot change the mode to Ingress.

   Enable Tenant Routed Multicast (TRM) – Select the checkbox to enable Tenant Routed Multicast (TRM) as the fabric overlay multicast protocol.

   Default MDT Address for TRM VRFs: The multicast address for Tenant Routed Multicast traffic is populated. By default, this address is from the IP prefix specified in the Multicast Group Subnet field. When you update either field, ensure that the TRM address is chosen from the IP prefix specified in Multicast Group Subnet.

   Rendezvous-Points - Enter the number of spine switches acting as rendezvous points.

   RP mode – Choose from the two supported multicast modes of replication, ASM (for Any-Source Multicast [ASM]) or BiDir (for Bidirectional PIM [BIDIR-PIM]).

   When you choose ASM, the BiDir related fields are not enabled. When you choose BiDir, the BiDir related fields are enabled.

   Note	BIDIR-PIM is supported on Cisco's Cloud Scale Family platforms 9300-EX and 9300-FX/FX2, and software release 9.2(1) onwards.

   When you create a new VRF for the fabric overlay, this address is populated in the Underlay Multicast Address field, in the Advanced tab.

   Underlay RP Loopback ID – The loopback ID used for the rendezvous point (RP), for multicast protocol peering purposes in the fabric underlay.

   The next two fields are enabled if you choose BIDIR-PIM as the multicast mode of replication.

   Underlay Primary RP Loopback ID – The primary loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   Underlay Backup RP Loopback ID – The secondary loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   Underlay Second Backup RP Loopback Id and Underlay Third Backup RP Loopback Id: Used for the second and third fallback Bidir-PIM Phantom RP.

5. Click the vPC tab. Most of the fields are auto generated. You can update the fields if needed.

   

   vPC Peer Link VLAN – VLAN used for the vPC peer link SVI.

   vPC Peer Keep Alive option – Choose the management or loopback option. If you want to use IP addresses assigned to the management port and the management VRF, choose management. If you use IP addresses assigned to loopback interfaces (and a non-management VRF), choose loopback.

   If you use IPv6 addresses, you must use loopback IDs.

   vPC Auto Recovery Time - Specifies the vPC auto recovery time-out period in seconds.

   vPC Delay Restore Time - Specifies the vPC delay restore period in seconds.

   vPC Peer Link Port Channel ID - Specifies the Port Channel ID for a vPC Peer Link. By default, the value in this field is 500.

   vPC IPv6 ND Synchronize – Enables IPv6 Neighbor Discovery synchronization between vPC switches. The check box is enabled by default. Clear the check box to disable the function.

   vPC advertise-pip - Select the check box to enable the Advertise PIP feature.

   Enable the same vPC Domain Id for all vPC Pairs: Enable the same vPC Domain ID for all vPC pairs. When you select this field, the vPC Domain Id field is editable.

   vPC Domain Id - Specifies the vPC domain ID to be used on all vPC pairs.

6. Click the Protocols tab. Most of the fields are auto generated. You can update the fields if needed.

   

   Underlay Routing Loopback Id - The loopback interface ID is populated as 0 since loopback0 is usually used for fabric underlay IGP peering purposes.

   Underlay VTEP Loopback Id - The loopback interface ID is populated as 1 since loopback1 is usually used for the VTEP peering purposes.

   Link-State Routing Protocol Tag - The tag defining the type of network.

   OSPF Area ID – The OSPF area ID, if OSPF is used as the IGP within the fabric.

   Note	The OSPF or IS-IS authentication fields are enabled based on your selection in the Link-State Routing Protocol field in the General tab.

   Enable OSPF Authentication – Select the check box to enable OSPF authentication. Deselect the check box to disable it. If you enable this field, the OSPF Authentication Key ID and OSPF Authentication Key fields get enabled.

   OSPF Authentication Key ID - The Key ID is populated.

   OSPF Authentication Key - The OSPF authentication key must be the 3DES key from the switch.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the encrypted key and enter it in this field. Refer the Retrieving the Authentication Key section for details.

   IS-IS Level - Select the IS-IS level from this drop-down list.

   Enable IS-IS Authentication - Select the check box to enable IS-IS authentication. Deselect the check box to disable it. If you enable this field, the IS-IS authentication fields are enabled.

   IS-IS Authentication Keychain Name - Enter the Keychain name, such as CiscoisisAuth.

   IS-IS Authentication Key ID - The Key ID is populated.

   IS-IS Authentication Key - Enter the Cisco Type 7 encrypted key.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the encrypted key and enter it in this field. Refer the Retrieving the Authentication Key section for details.

   Enable BGP Authentication - Select the check box to enable BGP authentication. Deselect the check box to disable it. If you enable this field, the BGP Authentication Key Encryption Type and BGP Authentication Key fields are enabled.

   Note	If you enable BGP authentication using this field, leave the iBGP Peer-Template Config field blank to avoid duplicate configuration.

   BGP Authentication Key Encryption Type – Choose the 3 for 3DES encryption type, or 7 for Cisco encryption type.

   BGP Authentication Key - Enter the encrypted key based on the encryption type.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the encrypted key and enter it in the BGP Authentication Key field. Refer the Retrieving the Authentication Key section for details.

   Enable BFD: Select the check box to enable feature bfd on all switches in the fabric. This feature is valid only on IPv4 underlay and the scope is within a fabric.

   From Cisco DCNM Release 11.3(1), BFD within a fabric is supported natively. The BFD feature is disabled by default in the Fabric Settings. If enabled, BFD is enabled for the underlay protocols with the default settings. Any custom required BFD configurations must be deployed via the per switch freeform or per interface freeform policies.

   The following config is pushed after you select the Enable BFD check box:

   feature bfd

   For information about BFD feature compatibility, refer your respective platform documentation and for information about the supported software images, see Compatibility Matrix for Cisco DCNM.

   Enable BFD for iBGP: Select the check box to enable BFD for the iBGP neighbor. This option is disabled by default.

   Enable BFD for OSPF: Select the check box to enable BFD for the OSPF underlay instance. This option is disabled by default, and it is grayed out if the link state protocol is ISIS.

   Enable BFD for ISIS: Select the check box to enable BFD for the ISIS underlay instance. This option is disabled by default, and it is grayed out if the link state protocol is OSPF.

   Enable BFD for PIM: Select the check box to enable BFD for PIM. This option is disabled by default, and it is be grayed out if the replication mode is Ingress.

   Here are the examples of the BFD global policies:

   
   router ospf <ospf tag>
      bfd
   
   router isis <isis tag>
     address-family ipv4 unicast
       bfd
   
   ip pim bfd
   
   router bgp <bgp asn>
     neighbor <neighbor ip>
       bfd
   

   Enable BFD Authentication: Select the check box to enable BFD authentication. If you enable this field, the BFD Authentication Key ID and BFD Authentication Key fields are editable.

   Note

   BFD Authentication is not supported when the Fabric Interface Numbering field under the General tab is set to unnumbered. The BFD authentication fields will be grayed out automatically. After you upgrade from DCNM Release 11.2(1) with BFD enabled to DCNM Release 11.3(1), the following configs are pushed to the switch: no ip redirects no ipv6 redirects

   BFD Authentication Key ID: Specifies the BFD authentication key ID for the interface authentication. The default value is 100.

   BFD Authentication Key: Specifies the BFD authentication key.

   For information about how to retrieve the BFD authentication parameters, see Retrieving the Encrypted BFD Authentication Key.

   iBGP Peer-Template Config – Add iBGP peer template configurations on the leaf switches to establish an iBGP session between the leaf switch and route reflector.

   If you use BGP templates, add the authentication configuration within the template and clear the Enable BGP Authentication check box to avoid duplicate configuration.

   In the sample configuration, the 3DES password is displayed after password 3.

   router bgp 65000
       password 3 sd8478fswerdfw3434fsw4f4w34sdsd8478fswerdfw3434fsw4f4w
   

7. Click the Advanced tab. Most of the fields are auto generated. You can update the fields if needed.

   

   VRF Template and VRF Extension Template: Specifies the VRF template for creating VRFs, and the VRF extension template for enabling VRF extension to other fabrics.

   Network Template and Network Extension Template: Specifies the network template for creating networks, and the network extension template for extending networks to other fabrics.

   Site ID - The ID for this fabric if you are moving this fabric within an MSD. The site ID is mandatory for a member fabric to be a part of an MSD. Each member fabric of an MSD has a unique site ID for identification.

   Intra Fabric Interface MTU - Specifies the MTU for the intra fabric interface. This value should be an even number.

   Layer 2 Host Interface MTU - Specifies the MTU for the layer 2 host interface. This value should be an even number.

   Power Supply Mode - Choose the appropriate power supply mode.

   CoPP Profile - Choose the appropriate Control Plane Policing (CoPP) profile policy for the fabric. By default, the strict option is populated.

   VTEP HoldDown Time - Specifies the NVE source interface hold down time.

   Brownfield Overlay Network Name Format: Enter the format to be used to build the overlay network name during a brownfield import or migration. The network name should not contain any white spaces or special characters except underscore (_) and hyphen (-). The network name must not be changed once the brownfield migration has been initiated. See the Creating Networks for the Standalone Fabric section for the naming convention of the network name. The syntax is [<string> | $$VLAN_ID$$] $$VNI$$ [<string>| $$VLAN_ID$$] and the default value is Auto_Net_VNI$$VNI$$_VLAN$$VLAN_ID$$. When you create networks, the name is generated according to the syntax you specify. The following table describes the variables in the syntax.

   Variables	Description
   $$VNI$$	Specifies the network VNI ID found in the switch configuration. This is a mandatory keyword required to create unique network names.
   $$VLAN_ID$$	Specifies the VLAN ID associated with the network. VLAN ID is specific to switches, hence DCNM will pick the VLAN ID from one of the switches, where the network is found, randomly and use it in the name. We recommend not to use this unless the VLAN ID is consistent across the fabric for the VNI.
   <string>	This variable is optional and you can enter any number of alphanumeric characters that meet the network name guidelines.

   Example overlay network name: Site_VNI12345_VLAN1234

   Note	Ignore this field for greenfield deployments. The Brownfield Overlay Network Name Format applies for the following brownfield imports: CLI-based overlays Configuration profile-based overlay where the configuration profiles were created in Cisco DCNM Release 10.4(2).

   Enable VXLAN OAM - Enables the VXLAM OAM function for existing switches.

   This is enabled by default. Clear the check box to disable VXLAN OAM function.

   If you want to enable the VXLAN OAM function on specific switches and disable on other switches in the fabric, you can use freeform configurations to enable OAM and disable OAM in the fabric settings.

   Note	The VXLAN OAM feature in Cisco DCNM is only supported on a single fabric or site.

   Enable Tenant DHCP – Select the checkbox to enable the tenant DHCP support.

   Note	Ensure that Enable Tenant DHCP is enabled before enabling DHCP related parameters in the overlay profiles.

   Enable NX-API - Specifies enabling of NX-API.

   Enable NX-API on HTTP - Specifies enabling of NX-API on HTTP.

   Enable Policy-Based Routing (PBR) - Select this check box to enable routing of packets based on the specified policy.

   Enable Strict Config Compliance - Enable the Strict Config Compliance feature by selecting this check box. By default, this feature is disabled.

   Enable AAA IP Authorization - Enables AAA IP authorization, when IP Authorization is enabled in the AAA Server

   Enable DCNM as Trap Host - Select this check box to enable DCNM as a trap host.

   Greenfield Cleanup Option – Enable or disable the switch cleanup option for greenfield switches.

   Enable Precision Time Protocol (PTP): Enables PTP across a fabric. When you select this check box, PTP is enabled globally and on core-facing interfaces. Additionally, the PTP Source Loopback Id and PTP Domain Id fields are editable. For more information, see Precision Time Protocol for Easy Fabric.

   PTP Source Loopback Id: Specifies the loopback interface ID Loopback that is used as the Source IP Address for all PTP packets. The valid values range from 0 to 1023. The PTP loopback ID cannot be the same as RP, Phantom RP, NVE, or MPLS loopback ID. Otherwise, an error will be generated. The PTP loopback ID can be the same as BGP loopback or user-defined loopback which is created from DCNM.

   If the PTP loopback ID is not found during Save & Deploy, the following error is generated:

   Loopback interface to use for PTP source IP is not found. Please create PTP loopback interface on all the devices to enable PTP feature.

   PTP Domain Id: Specifies the PTP domain ID on a single network. The valid values range from 0 to 127.

   Enable MPLS Handoff: Select the check box to enable the MPLS Handoff feature. For more information, see Border Provisioning Use Case in VXLAN BGP EVPN Fabrics - MPLS SR and LDP Handoff.

   Note: For the brownfield import, you need to select the Enable MPLS Handoff feature. Most of the IFC configuration will be captured in switch_freeform.

   Underlay MPLS Loopback Id: Specifies the underlay MPLS loopback ID. The default value is 101.

   Enable Default Queuing Policies: Check this check box to apply QoS policies on all the switches in this fabric. To remove the QoS policies that you applied on all the switches, uncheck this check box, update all the configurations to remove the references to the policies, and save and deploy. From Cisco DCNM Release 11.3(1), pre-defined QoS configurations are included that can be used for various Cisco Nexus 9000 Series Switches. When you check this check box, the appropriate QoS configurations are pushed to the switches in the fabric. The system queuing is updated when configurations are deployed to the switches. You can perform the interface marking with defined queuing policies, if required, by adding the required configuration to the per interface freeform block.

   Review the actual queuing policies by opening the policy file in the template editor. From Cisco DCNM Web UI, choose Control > Template Library. Search for the queuing policies by the policy file name, for example, queuing_policy_default_8q_cloudscale. Choose the file and click the Modify/View template icon to edit the policy.

   See the Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide for platform specific details.

   N9K Cloud Scale Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus 9200 Series Switches and the Cisco Nexus 9000 Series Switches that ends with EX, FX, and FX2 in the fabric. The valid values are queuing_policy_default_4q_cloudscale and queuing_policy_default_8q_cloudscale. Use the queuing_policy_default_4q_cloudscale policy for FEXes. You can change from the queuing_policy_default_4q_cloudscale policy to the queuing_policy_default_8q_cloudscale policy only when FEXes are offline.

   N9K R-Series Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus switches that ends with R in the fabric. The valid value is queuing_policy_default_r_series.

   Other N9K Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all other switches in the fabric other than the switches mentioned in the above two options. The valid value is queuing_policy_default_other.

   Leaf Freeform Config - Add CLIs that should be added to switches that have the Leaf, Border, and Border Gateway roles.

   Spine Freeform Config - Add CLIs that should be added to switches with a Spine, Border Spine, and Border Gateway Spine roles.

   Freeform CLIs - Fabric level freeform CLIs can be added while creating or editing a fabric. They are applicable to switches across the fabric. You must add the configurations as displayed in the running configuration, without indentation. Switch level freeform configurations such as VLAN, SVI, and interface configurations should only be added on the switch. Refer the Freeform Configurations on Fabric Switches topic for a detailed explanation and examples.

   Intra-fabric Links Additional Config - Add CLIs that should be added to the intra-fabric links.

8. Click the Resources tab.

   

   Manual Underlay IP Address Allocation – Do not select this check box if you are transitioning your VXLAN fabric management to DCNM.

   • By default, DCNM allocates the underlay IP address resources (for loopbacks, fabric interfaces, etc) dynamically from the defined pools. If you select the check box, the allocation scheme switches to static, and some of the dynamic IP address range fields are disabled.

   • For static allocation, the underlay IP address resources must be populated into the Resource Manager (RM) using REST APIs.

     Refer the Cisco DCNM REST API Reference Guide, Release 11.2(1) for more details. The REST APIs must be invoked after the switches are added to the fabric, and before you use the Save & Deploy option.

   • The Underlay RP Loopback IP Range field stays enabled if BIDIR-PIM function is chosen for multicast replication.

   • Changing from static to dynamic allocation keeps the current IP resource usage intact. Only future IP address allocation requests are taken from dynamic pools.

   Underlay Routing Loopback IP Range - Specifies loopback IP addresses for the protocol peering.

   Underlay VTEP Loopback IP Range - Specifies loopback IP addresses for VTEPs.

   Underlay RP Loopback IP Range - Specifies the anycast or phantom RP IP address range.

   Underlay Subnet IP Range - IP addresses for underlay P2P routing traffic between interfaces.

   Underlay MPLS Loopback IP Range: Specifies the underlay MPLS loopback IP address range.

   Note that the IP range should be a unique range, that is, it should not overlap with IP ranges of the other fabrics.

   Layer 2 VXLAN VNI Range and Layer 3 VXLAN VNI Range - Specifies the VXLAN VNI IDs for the fabric.

   Network VLAN Range and VRF VLAN Range - VLAN ranges for the Layer 3 VRF and overlay network.

   Subinterface Dot1q Range - Specifies the subinterface range when L3 sub interfaces are used.

   VRF Lite Deployment - Specify the VRF Lite method for extending inter fabric connections.

   If you select Manual, the VRF Lite subnet details are required so that the resource manager can reserve the address space.

   If you select Back2BackOnly, ToExternalOnly, or Both, then the VRF Lite subnet fields are enabled.

   VRF Lite Subnet IP Range and VRF Lite Subnet Mask – These fields are populated with the DCI subnet details. Update the fields as needed.

   The values shown in your screen are automatically generated. If you want to update the IP address ranges, VXLAN Layer 2/Layer 3 network ID ranges or the VRF/Network VLAN ranges, ensure the following:

   Note

   When you update a range of values, ensure that it does not overlap with other ranges. You should only update one range of values at a time. If you want to update more than one range of values, do it in separate instances. For example, if you want to update L2 and L3 ranges, you should do the following. Update the L2 range and click Save. Click the Edit Fabric option again, update the L3 range and click Save.

   Service Network VLAN Range - Specifies a VLAN range in the Service Network VLAN Range field. This is a per switch overlay service network VLAN range. The minimum allowed value is 2 and the maximum allowed value is 3967.

   Route Map Sequence Number Range - Specifies the route map sequence number range. The minimum allowed value is 1 and the maximum allowed value is 65534.

9. Click the Manageability tab.

   

   The fields in this tab are:

   DNS Server IPs - Specifies the comma separated list of IP addresses (v4/v6) of the DNS servers.

   DNS Server VRFs - Specifies one VRF for all DNS servers or a comma separated list of VRFs, one per DNS server.

   NTP Server IPs - Specifies comma separated list of IP addresses (v4/v6) of the NTP server.

   NTP Server VRFs - Specifies one VRF for all NTP servers or a comma separated list of VRFs, one per NTP server.

   Syslog Server IPs – Specifies the comma separated list of IP addresses (v4/v6) IP address of the syslog servers, if used.

   Syslog Server Severity – Specifies the comma separated list of syslog severity values, one per syslog server. The minimum value is 0 and the maximum value is 7. To specify a higher severity, enter a higher number.

   Syslog Server VRFs – Specifies one VRF for all syslog servers or a comma separated list of VRFs, one per syslog server.

   AAA Freeform Config – Specifies the AAA freeform configs.

   If AAA configs are specified in the fabric settings, switch_freeform PTI with source as UNDERLAY_AAA and description as AAA Configurations will be created.

10. Click the Bootstrap tab.

    

    Enable Bootstrap - Select this check box to enable the bootstrap feature.

    After you enable bootstrap, you can enable the DHCP server for automatic IP address assignment using one of the following methods:

    • External DHCP Server: Enter information about the external DHCP server in the Switch Mgmt Default Gateway and Switch Mgmt IP Subnet Prefix fields.

    • Local DHCP Server: Enable the Local DHCP Server checkbox and enter details for the remaining mandatory fields.

    Enable Local DHCP Server - Select this check box to initiate enabling of automatic IP address assignment through the local DHCP server. When you select this check box, the DHCP Scope Start Address and DHCP Scope End Address fields become editable.

    If you do not select this check box, DCNM uses the remote or external DHCP server for automatic IP address assignment.

    DHCP Version – Select DHCPv4 or DHCPv6 from this drop-down list. When you select DHCPv4, the Switch Mgmt IPv6 Subnet Prefix field is disabled. If you select DHCPv6, the Switch Mgmt IP Subnet Prefix is disabled.

    Note

    Cisco DCNM IPv6 POAP is not supported with Cisco Nexus 7000 Series Switches. Cisco Nexus 9000 and 3000 Series Switches support IPv6 POAP only when switches are either L2 adjacent (eth1 or out-of-band subnet must be a /64) or they are L3 adjacent residing in some IPv6 /64 subnet. Subnet prefixes other than /64 are not supported.

    DHCP Scope Start Address and DHCP Scope End Address - Specifies the first and last IP addresses of the IP address range to be used for the switch out of band POAP.

    Switch Mgmt Default Gateway - Specifies the default gateway for the management VRF on the switch.

    Switch Mgmt IP Subnet Prefix - Specifies the prefix for the Mgmt0 interface on the switch. The prefix should be between 8 and 30.

    DHCP scope and management default gateway IP address specification - If you specify the management default gateway IP address 10.0.1.0 and subnet mask 24, ensure that the DHCP scope is within the specified subnet, between 10.0.1.1 and 10.0.1.254.

    Switch Mgmt IPv6 Subnet Prefix - Specifies the IPv6 prefix for the Mgmt0 interface on the switch. The prefix should be between 112 and 126. This field is editable if you enable IPv6 for DHCP.

    Enable AAA Config – Select this check box to include AAA configs from the Manageability tab during device bootup.

    Bootstrap Freeform Config - (Optional) Enter additional commands as needed. For example, if you are using AAA or remote authentication related configurations, you need to add these configurations in this field to save the intent. After the devices boot up, they contain the intent defined in the Bootstrap Freeform Config field.

    Copy-paste the running-config to a freeform config field with correct indentation, as seen in the running configuration on the NX-OS switches. The freeform config must match the running config. For more information, see Resolving Freeform Config Errors in Switches.

    DHCPv4/DHCPv6 Multi Subnet Scope - Specifies the field to enter one subnet scope per line. This field is editable after you check the Enable Local DHCP Server check box.

    The format of the scope should be defined as:

    DHCP Scope Start Address, DHCP Scope End Address, Switch Management Default Gateway, Switch Management Subnet Prefix

    For example: 10.6.0.2, 10.6.0.9, 10.6.0.1, 24

11. Click the Configuration Backup tab. The fields on this tab are:

    

    Hourly Fabric Backup: Select the check box to enable an hourly backup of fabric configurations and the intent.

    You can enable an hourly backup for fresh fabric configurations and the intent as well. If there is a configuration push in the previous hour, DCNM takes a backup.

    Intent refers to configurations that are saved in DCNM but yet to be provisioned on the switches.

    Scheduled Fabric Backup: Check the check box to enable a daily backup. This backup tracks changes in running configurations on the fabric devices that are not tracked by configuration compliance.

    Scheduled Time: Specify the scheduled backup time in a 24-hour format. This field is enabled if you check the Scheduled Fabric Backup check box.

    Select both the check boxes to enable both back up processes.

    The backup process is initiated after you click Save.

    The backup configuration files are stored in the following path in DCNM: /usr/local/cisco/dcm/dcnm/data/archive

    The number of archived files that can be retained is set in the # Number of archived files per device to be retained: field in the Server Properties window.

    Note

    Hourly and scheduled backup processes happen only during the next periodic configuration compliance activity, and there can be a delay of up to an hour. To trigger an immediate backup, do the following: Choose Control > Fabric Builder. The Fabric Builder screen comes up. Click within the specific fabric box. The fabric topology screen comes up. From the Actions pane at the left part of the screen, click Re-Sync Fabric.

    You can also initiate the fabric backup in the fabric topology window. Click Backup Now in the Actions pane.

12. Click Save after filling and updating relevant information. A note appears briefly at the bottom right part of the screen, indicating that the fabric is created. When a fabric is created, the fabric page comes up. The fabric name appears at the top left part of the screen.

    (At the same time, the newly created fabric instance appears on the Fabric Builder screen. To go to the Fabric Builder screen, click the left arrow (←) button above the Actions pane [to the left of the screen]).

    The Actions pane allows you to perform various functions. One of them is the Add switches option to add switches to the fabric. After you create a fabric, you should add fabric devices. The options are explained:

    • Tabular View - By default, the switches are displayed in the topology view. Use this option to view switches in the tabular view.

    • Refresh topology - Allows you to refresh the topology.

    • Save Layout – Saves a custom view of the topology. You can create a specific view in the topology and save it for ease of use.

    • Delete saved layout – Deletes the custom view of the topology

    • Topology views - You can choose between Hierarchical, Random and Custom saved layout display options.

      • Hierarchical - Provides an architectural view of your topology. Various Switch Roles can be defined that draws the nodes on how you configure your CLOS topology.

      • Random - Nodes are placed randomly on the window. DCNM tries to make a guess and intelligently place nodes that belong together in close proximity.

      • Custom saved layout - You can drag nodes around to your liking. Once you have the positions as how you like, you can click Save Layout to remember the positions. Next time you come to the topology, DCNM will draw the nodes based on your last saved layout positions.

    • Restore Fabric – Allows you to restore the fabric to a prior DCNM configuration state (one month back, two months back, and so on). For more information, see Restore Fabric section.

    • Backup Now: You can initiate a fabric backup manually by clicking Backup Now. Enter a name for the tag and click OK. Regardless of the settings you choose under the Configuration Backup tab in the Fabric Settings dialog box, you can initiate a backup using this option.

    • Resync Fabric - Use this option to resynchronize DCNM state when there is a large scale out-of-band change, or if configuration changes do not register in the DCNM properly. The resync operation does a full CC run for the fabric switches and recollects “show run” and “show run all” commands from the switches. When you initiate the re-sync process, a progress message is displayed on the window. During the re-sync, the running configuration is taken from the switches. Then, the OUT-OF-SYNC/IN-SYNC status for the switch is recalculated based on the intent or expected configuration defined in DCNM versus the current running configuration that was taken from the switches.

    • Add Switches – Allows you to add switch instances to the fabric.

    • Fabric Settings – Allows you to view or edit fabric settings.

    • Cloud icon - Click the Cloud icon to display (or not display) an Undiscovered cloud.

      When you click the icon, the Undiscovered cloud and its links to the selected fabric topology are not displayed.

      Click the Cloud icon again to display the Undiscovered cloud.

SCOPE - You can toggle between fabrics by using the SCOPE drop-down box at the top right. The current fabric is highlighted. An MSD and its member fabrics are distinctly displayed, wherein the member fabrics are indented, under the MSD fabric.

From Cisco DCNM Release 11.3(1), you can create a Easy fabric with IPv6 only underlay. The IPv6 underlay is supported only for the Easy_Fabric_11_1 template. In the IPv6 underlay fabric, intra-fabric links, routing loopback, vPC peer link SVI, and NVE loopback interface for VTEP are configured with IPv6 addresses. EVPN BGP neighbor peering is also established using IPv6 addressing.

The following guidelines are applicable for IPv6 underlay:

• IPv6 underlay is supported for the Cisco Nexus 9000 Series switches with Cisco NX-OS Release 9.3(1) or higher.

• VXLANv6 is only supported Cisco Nexus 9332C, Cisco Nexus C9364C, and Cisco Nexus modules that end with EX, FX, FX2, FX3, or FXP.

• In VXLANv6, the platforms supported on spine are all Nexus 9000 Series and Nexus 3000 Series platforms.

• The overlay routing protocol supported for the IPv6 fabric is BGP EVPN.

• vPC with physical multichassis EtherChannel trunk (MCT) feature is supported for the IPv6 underlay network in DCNM. The vPC peer keep-alive can be loopback or management with IPv4 or IPv6 address.

• Brownfield migration is supported for the fabrics with IPv6 underlay networks.

• DHCPv6 is supported for the IPv6 underlay network.

• The following features are not supported for VXLAN IPv6 underlay:

  • Multicast underlay

  • Tenant Routed Multicast (TRM)

  • ISIS, OSPF, and BGP authentication

  • VXLAN Multi-site

  • Dual stack underlay

  • vPC with Virtual MCT

  • DCI SR-MPLS or MPLS-LDP handoff

  • BFD

  • Super Spine switch roles

  • NGOAM

This document explains Brownfield deployments, wherein you transition your VXLAN BGP EVPN fabric management to DCNM. The transition involves migrating existing networks configurations to DCNM.

Typically, your fabric is created and managed through manual CLI configuration or custom automation scripts. Now, you want to start managing the fabric through DCNM. After the migration, the fabric underlay and overlay networks will be managed by DCNM.

The migration procedure only supports VXLAN BGP EVPN networks that use the best practices mentioned in the Prerequisites section.

Support of simplified CLIs for VXLAN EVPN fabrics is not supported in either Greenfield or brownfield deployments.

Note	The Brownfield deployment section is applicable for the Easy_Fabric_11_1 template.

Prerequisites

• DCNM-supported NX-OS software versions. For details, refer Cisco DCNM Release Notes, Release 11.3(1).

• Underlay routing protocol is OSPF or IS-IS.

• The supported underlay is based on the DCNM 10.2(1) POAP template's best practices for the VXLAN fabric (dcnm_ip_vxlan_fabric_templates.10.2.1.ST.1.zip) available on Cisco.com.

• The following fabric-wide loopback interface IDs must not overlap:

  • Routing loopback interface for IGP/BGP.

  • VTEP loopback ID

  • Underlay rendezvous point loopback ID if ASM is used for multicast replication.

• BGP configuration uses the ‘router-id’, which is the IP address of the routing loopback interface.

• If the iBGP peer template is configured, then it must be configured on the leaf switches and route reflectors. The template name that needs to be used between leaf and route reflector should be identical.

• The BGP route reflector and multicast rendezvous point (if applicable) functions are implemented on spine switches. Leaf switches do not support the functions.

• Install DCNM 11.2(1) release software. Refer the Installation Guide for more details. Log in to DCNM and set the default LAN Credentials when prompted.

• Familiarity with the DCNM 11.2(1) fabric management and monitoring features before initiating the migration process.

• Familiarity with VXLAN BGP EVPN fabric concepts and functioning of the fabric from the DCNM perspective.

• Fabric switch nodes are operationally stable and functional and all fabric links are up.

• vPC switches and the peer links are up before the migration. Ensure that no configuration updates are in progress or changes pending.

• Create an inventory list of the switches in the fabric with their IP addresses and credentials. DCNM uses this information to connect to the switches.

• Shut down any other controller software you are using presently so that no further configuration changes are made to the VXLAN fabric. Alternatively, disconnect the network interfaces from the controller software (if any) so that no changes are allowed on the switches.

• The switch overlay configurations must have the mandatory configurations defined in the shipping DCNM Universal Overlay profiles. Additional network or VRF overlay related configurations found on the switches are preserved in the freeform configuration associated with the network or VRF DCNM entries.

• All the overlay network and VRF profile parameters such as VLAN name and route map name should be consistent across all devices in the fabric for the brownfield migration to be successful.

Guidelines and Limitations

• Fabric interfaces can be numbered or unnumbered.

• Various other interface types are supported.

• The following features are unsupported.

  • eBGP underlay

  • Layer 3 port channel

  • Configuration profiles present in the brownfield configurations (the expectation is that the overlays should be configured through regular CLIs).

  • vPC Fabric Peering

• Take a backup of the switch configurations and save them before the migration.

• No configuration changes (unless instructed to do so in this document) must be made to the switches until the migration is completed. Else, significant network issues can occur.

• Migration to Cisco DCNM is only supported for Cisco Nexus 9000 switches.

• Multi-line banner configuration on the switch is preserved in the switch_freeform configuration, along with other configurations captured in the switch_freeform configuration, if any.

• From DCNM Release 11.2(1), the Border Spine and Border Gateway Spine roles are supported for the brownfield migration.

• Fabrics with IS-IS Level-1 and Level-2 are supported for the Brownfield migration.

• Switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images support the Brownfield migration. For information about feature compatibility, refer your respective platform documentation and for information about the supported software images, see Compatibility Matrix for Cisco DCNM.

  Note the following guidelines and limitations:

  • The VLAN name for the network or VRF is not captured in the overlay profile if at least one of the non-spine switches have the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images. The VLAN name is captured in the freeform config associated with the overlay network or VRF. The VLAN name can be changed by updating the freeform config. For more information, see Modifying VLAN Names in a Switch with Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) Images.

  • Config compliance difference for TCAM CLIs on Cisco Nexus 9300 Series switches and Cisco Nexus 9500 Series switches with X9500 line cards. For more information, see Resolving Config Compliance Error on Switches with Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) Images.

  • The overlay profile refresh feature is unsupported for the brownfield migration of switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images.

• Cisco Nexus 9500 Series Switches are supported as VTEPs with border spine, BGW spine, or leaf roles for Cisco NX-OS Release 7.0.3.I7(3) or later.

• During the brownfield migration in the Cisco DCNM Release 11.1(1), the overlay configuration profiles are deployed to switches and all the overlay related configurations are captured in the respective network or VRF freeform configs. Post migration, switches have both the original configuration CLIs and the config-profiles.

  From Cisco DCNM Release 11.2(1), during the brownfield migration, the overlay config-profiles are deployed to the switches, and the original configuration CLIs are removed. Post migration, the switches only have the configuration profiles and any extra configuration that is not part of the configuration profile if the switches in the brownfield migration have the following Cisco NX-OS images:

  • Cisco NX-OS Release 7.0(3)I7(6) or newer

  • Cisco NX-OS Release 9.2(3) or newer

  If the switches do not meet these requirements, the brownfield migration behavior is the same as described for the Cisco DCNM Release 11.1(1).

Procedure

Transitioning VXLAN fabric management to DCNM involves these steps.

1. Creating a new VXLAN BGP EVPN fabric in DCNM – This step creates a VXLAN fabric outline.

2. Initiating VXLAN fabric management transition to DCNM – This step adds switch instances to DCNM and initiates the transition.

Creating a New VXLAN BGP EVPN Fabric

First, guidelines for updating the settings are noted. Then each VXLAN fabric settings tab is explained:

• Some values (BGP AS Number, OSPF, etc) are considered as reference points to your existing fabric, and the values you enter must match the existing fabric values.

• For some fields (such as IP address range, VXLAN ID range), the values that are auto-populated or entered in the settings are only used for future allocation. The existing fabric values are honored during migration.

• Some fields relate to new functions that may not exist in your existing fabric (such as advertise-pip). Enable or disable it as per your need.

• At a later point in time, after the fabric transition is complete, you can update settings if needed.

1. Choose Control > Fabric Builder.

   The Fabric Builder screen appears. When you log in for the first time, the Fabrics section has no entries. After you create a fabric, it is displayed on the Fabric Builder screen, wherein a rectangular box represents each fabric.

   A standalone or member fabric contains Switch_Fabric (in the Type field), the AS number (in the ASN field), and mode of replication (in the Replication Mode field).

2. Click Create Fabric. The Add Fabric screen appears. The fields are explained:

   Fabric Name - Enter the name of the fabric.

   Fabric Template - From the drop-down menu, choose the Easy_Fabric_11_1 fabric template. The fabric settings for creating a standalone fabric comes up.

   The tabs and their fields in the screen are explained in the subsequent points. The overlay and underlay network parameters are included in these tabs.

   Note	If you are creating a standalone fabric as a potential member fabric of an MSD fabric (used for provisioning overlay networks for fabrics that are connected through EVPN Multi-Site technology), then browse through the Multi-Site Domain for VXLAN BGP EVPN Fabrics topic before member fabric creation.

3. The General tab is displayed by default. The fields in this tab are:

   

   BGP ASN: Enter the BGP AS number the fabric is associated with.

   For information about IPv6 underlay, see IPv6 Underlay Support for Easy Fabric.

   Fabric Interface Numbering: Specify whether you are using a point-to-point (p2p) or unnumbered network in your existing setup.

   Underlay Subnet IP Mask - Specify the subnet mask you are using for the fabric underlay IP address subnets in your existing setup.

   Link-State Routing Protocol: The IGP used in the existing fabric, OSPF, or IS-IS.

   Route-Reflectors – The Route Reflector count is only applicable post-migration. The existing route reflector configuration is honored when importing into the DCNM setup.

   The number of spine switches that are used as route reflectors for transporting BGP traffic. Choose 2 or 4 from the drop-down box. The default value is 2.

   To deploy spine devices as route reflectors, DCNM sorts the spine devices based on their serial numbers, and designates two or four spine devices as route reflectors. If you add more spine devices, existing route reflector configuration will not change.

   Increasing the count - You can increase the route reflectors from two to four at any point in time. Configurations are automatically generated on the other 2 spine devices designated as route reflectors.

   Decreasing the count

   When you reduce four route reflectors to two, you must remove the unneeded route reflector devices from the fabric. Follow these steps to reduce the count from 4 to 2.

   1. Change the value in the drop-down box to 2.

   2. Identify the spine switches designated as route reflectors.

      An instance of the rr_state policy is applied on the spine switch if it is a route reflector. To find out if the policy is applied on the switch, right-click the switch, and choose View/edit policies. In the View/Edit Policies screen, search rr_state in the Template field. It is displayed on the screen.

   3. Delete the unneeded spine devices from the fabric (right-click the spine switch icon and choose Discovery > Remove from fabric).

      If you delete existing route reflector devices, the next available spine switch is selected as the replacement route reflector.

   4. Click Save and Deploy at the top right part of the fabric topology screen.

   Anycast Gateway MAC: Enter the Anycast gateway MAC address of the existing fabric.

   NX-OS Software Image Version: Leave this field blank. You can update this post-transition, as desired.

4. Click the Replication tab. Most of the fields are auto generated.

   

   Replication Mode: The mode of replication that is used in the existing fabric, Ingress Replication, or Multicast.

   When you choose Ingress replication, the multicast replication fields get disabled.

   Multicast Group Subnet - The IP address prefix for multicast communication is used for post-migration allocation. The IP address prefix used in your existing fabric is honored during the transition.

   A unique IP address is allocated from this group for each overlay network.

   Enable Tenant Routed Multicast – Select the check box to enable Tenant Routed Multicast (TRM) as the fabric overlay multicast protocol.

   If you enable TRM, the Multicast address for TRM must be entered. All the TRM specific tenant configuration is captured in the switch freeform policy linked to the tenant network and VRF profile.

   Note that the TRM feature is unsupported on switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images.

   Default MDT address for TRM VRFs – Enter the default multicast distribution tree (MDT) IPv4 address for TRM VRFs.

   Rendezvous-Points - Enter the number of spine switches acting as rendezvous points.

   RP mode – Select asm (Any-Source Multicast) or bidir (Bidirectional PIM) mode.

   When you choose ASM, the BiDir related fields are not enabled.

   The asm RP mode supports up to 4 RPs.

   The bidir mode supports up to 2 RPs. An error message is displayed if the BIDIR configuration indicates that more than 2 RPs are used.

   After brownfield migration, only 2 RPs are supported in the migrated fabric. An error message is displayed when you click Save & Deploy after changing the RP count to 4.

   If an RP is down or deleted from the fabric, this RP cannot be replaced by another spine as Easy Fabric does not remember the configuration of a removed switch. Easy Fabric uses a specific scheme to generate RP configuration for Bidir. Therefore, the generated Bidir configuration will not work with the brownfield imported configuration. After brownfield migration, if you change the RP count or add new spine or leaf switches, you should manually configure the PIM-Bidir feature. If a manual configuration is required, a warning message is displayed after you click Save & Deploy. For more information, see Manually Adding PIM-BIDIR Configuration for Leaf or Spine Post Brownfield Migration.

   You can also modify a brownfield imported bidir configuration to use the configuration generated by Fabric Builder. For more information, see Changing a Brownfield Imported BIDIR Configuration.

   Underlay RP Loopback ID – The loopback ID has to match your existing setup's loopback ID. This is the loopback ID used for the rendezvous point (RP), for multicast protocol peering purposes in the fabric underlay.

   The next two fields are enabled if you choose BIDIR-PIM as the multicast mode of replication.

   Underlay Primary RP Loopback ID – The primary loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   Underlay Backup RP Loopback ID – The secondary loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   The next two fields are enabled if Rendezvous-Points is set to 4. However, the fabric can have only 2 RPs for the brownfield migration.

   Underlay Second Backup RP Loopback Id – The second fallback loopback ID for Phantom RP, for multicast protocol peering purposes in the fabric underlay.

   Underlay Third Backup RP Loopback Id – The third fallback loopback ID for Phantom RP, for multicast protocol peering purposes in the fabric underlay.

5. Click the vPC tab. Most of the fields are auto generated.

   

   vPC Peer Link VLAN - Enter the VLAN ID used for the vPC peer link SVI in the existing fabric.

   vPC Peer Keep Alive option – Choose the management or loopback option, as used in the existing fabric. If you want to use IP addresses assigned to the management port and the management VRF, choose management. If you use IP addresses assigned to loopback interfaces (and a non-management VRF), choose loopback.

   If you only use IPv6 addresses on the management interface, you must use the loopback option.

   During the transition, the switch configuration is not checked for the following fields in the vPC tab. The switch configurations will get updated if they are different.

   vPC Auto Recovery Time - Specify the vPC auto recovery time-out period in seconds, as needed.

   vPC Delay Restore Time - Specify the vPC delay restore period in seconds, as needed.

   vPC Peer Link Port Channel ID - Specifies the Port Channel ID for a vPC Peer Link. By default, the value in this field is 500. Change the value based on your existing settings.

   vPC IPv6 ND Synchronize – Enables IPv6 Neighbor Discovery synchronization between vPC switches. The check box is enabled by default. Clear the check box to disable the function as needed.

   vPC advertise-pip - Select the check box to enable the Advertise PIP feature.

   Note that the Advertise PIP feature is unsupported on switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images.

   Enable the same vPC Domain Id for all vPC Pairs: Enable the same vPC Domain ID for all vPC pairs. When you select this field, the vPC Domain Id field is editable.

   vPC Domain Id - Specifies the vPC domain ID to be used on all vPC pairs.

6. Click the Protocols tab. Most of the fields are auto generated. You can update the fields if needed.

   

   Underlay Routing Loopback Id - The loopback interface ID is populated as 0 since loopback0 is usually used for fabric underlay IGP peering purposes. This must match the existing configuration on the switches. This must be the same across all the switches.

   Underlay VTEP Loopback Id - The loopback interface ID is populated as 1 since loopback1 is usually used for the VTEP peering purposes. This must match the existing configuration on the switches. This must be the same across all the switches where VTEPs are present.

   Link-State Routing Protocol Tag - Enter the existing fabric’s routing protocol tag in this field to define the type of network.

   OSPF Area ID – The OSPF area ID of the existing fabric, if OSPF is used as the IGP within the fabric.

   Note	The OSPF or IS-IS authentication fields are enabled based on your selection in the Link-State Routing Protocol field in the General tab.

   Enable OSPF Authentication – Select the check box to enable the OSPF authentication. Deselect the check box to disable it. If you enable this field, the OSPF Authentication Key ID and OSPF Authentication Key fields are enabled.

   OSPF Authentication Key ID – Enter the OSPF authentication key ID.

   OSPF Authentication Key - The OSPF authentication key must be the 3DES key from the switch.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the OSPF authentication details.

   You can obtain the OSPF authentication details by using the show run ospf command on your switch.

   
   # show run ospf | grep message-digest-key
   ip ospf message-digest-key 127 md5 3 c7c83ec78f38f32f3d477519630faf7b
   
   

   In this example, the OSPF authentication key ID is 127 and the authentication key is c7c83ec78f38f32f3d477519630faf7b.

   For information about how to configure a new key and retrieve it, see Retrieving the 3DES Encrypted OSPF Authentication Key.

   IS-IS Level - Select the IS-IS level from this drop-down list.

   Enable IS-IS Authentication - Select the check box to enable IS-IS authentication. Deselect the check box to disable it. If you enable this field, the IS-IS authentication fields are enabled.

   IS-IS Authentication Keychain Name - Enter the keychain name.

   IS-IS Authentication Key ID - Enter the IS-IS authentication key ID.

   IS-IS Authentication Key - Enter the Cisco Type 7 encrypted key.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the IS-IS authentication details.

   You can obtain the IS-IS authentication details by using the show run | section “key chain” command on your switch.

   
   # show run | section “key chain”
   key chain CiscoIsisAuth
     key 127
         key-string 7 075e731f
   
   

   In this example, the keychain name is CiscoIsisAuth, the key ID is 127, and the type 7 authentication key is 075e731f.

   Enable BGP Authentication - Select the check box to enable BGP authentication. Deselect the check box to disable it. If you enable this field, the BGP Authentication Key Encryption Type and BGP Authentication Key fields are enabled.

   BGP Authentication Key Encryption Type – Choose the 3 for 3DES encryption type, and 7 for Cisco encryption type.

   BGP Authentication Key - Enter the encrypted key based on the encryption type.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the BGP authentication details.

   You can obtain the BGP authentication details by using the show run bgp command on your switch.

   
   # show run bgp
   neighbor 10.2.0.2 
   remote-as 65000 
   password 3 sd8478fswerdfw3434fsw4f4w34sdsd8478fswerdfw3434fsw4f4w3
   
   

   In this example, the BGP authentication key is displayed after the encryption type 3.

   Enable BFD feature – Select the check box to enable the BFD feature.

   The BFD feature is disabled by default.

   Make sure that the BFD feature setting matches with the switch configuration. If the switch configuration contains feature bfd but the BFD feature is not enabled in the fabric settings, config compliance generates diff to remove the BFD feature after brownfield migration. That is, no feature bfd is generated after migration.

   From Cisco DCNM Release 11.3(1), BFD within a fabric is supported natively. The BFD feature is disabled by default in the Fabric Settings. If enabled, BFD is enabled for the underlay protocols with the default settings. Any custom required BFD configurations must be deployed via the per switch freeform or per interface freeform policies.

   The following config is pushed after you select the Enable BFD check box:

   feature bfd

   For information about BFD feature compatibility, refer your respective platform documentation and for information about the supported software images, see Compatibility Matrix for Cisco DCNM.

   Enable BFD for iBGP: Select the check box to enable BFD for the iBGP neighbor. This option is disabled by default.

   Enable BFD for OSPF: Select the check box to enable BFD for the OSPF underlay instance. This option is disabled by default, and it is grayed out if the link state protocol is ISIS.

   Enable BFD for ISIS: Select the check box to enable BFD for the ISIS underlay instance. This option is disabled by default, and it is grayed out if the link state protocol is OSPF.

   Enable BFD for PIM: Select the check box to enable BFD for PIM. This option is disabled by default, and it is be grayed out if the replication mode is Ingress.

   Here are the examples of the BFD global policies:

   
   router ospf <ospf tag>
      bfd
   
   router isis <isis tag>
     address-family ipv4 unicast
       bfd
   
   ip pim bfd
   
   router bgp <bgp asn>
     neighbor <neighbor ip>
       bfd
   

   Enable BFD Authentication: Select the check box to enable BFD authentication. If you enable this field, the BFD Authentication Key ID and BFD Authentication Key fields are editable.

   Note

   BFD Authentication is not supported when the Fabric Interface Numbering field under the General tab is set to unnumbered. The BFD authentication fields will be grayed out automatically. After you upgrade from DCNM Release 11.2(1) with BFD enabled to DCNM Release 11.3(1), the following configs are pushed to the switch: no ip redirects no ipv6 redirects

   BFD Authentication Key ID: Specifies the BFD authentication key ID for the interface authentication. The default value is 100.

   BFD Authentication Key: Specifies the BFD authentication key.

   For information about how to retrieve the BFD authentication parameters, see Retrieving the Encrypted BFD Authentication Key.

   iBGP Peer-Template Config – Add iBGP peer template configurations on the leaf switches and route reflectors to establish an iBGP session between the leaf switch and route reflector. Set this field based on switch configuration. If this field is blank, it implies that the iBGP peer template is not used. If the iBGP peer template is used, enter the peer template definition as defined on the switch. The peer template name on devices configured with BGP should be the same as defined here.

   Note	If you use the iBGP peer template, include the BGP authentication configuration in this template config field. Additionally, uncheck the Enable BGP Authentication check box to avoid duplicating the BGP configuration.

7. Click the Advanced tab. Most of the fields are auto generated.

   

   VRF Template and VRF Extension Template: Specifies the VRF template for creating VRFs, and the VRF extension template for enabling VRF extension to other fabrics.

   Network Template and Network Extension Template: Specifies the network template for creating networks, and the network extension template for extending networks to other fabrics.

   You must not change the templates when migrating. Only the Universal templates are supported for overlay migration.

   Site ID - The ID for this fabric if you are moving this fabric within an MSD. You can update this field post-migration.

   Intra Fabric Interface MTU - Specifies the MTU for the intra fabric interface. This value should be an even number.

   Layer 2 Host Interface MTU - Specifies the MTU for the layer 2 host interface. This value should be an even number.

   Power Supply Mode - Choose the appropriate power supply mode.

   CoPP Profile - Choose the Control Plane Policing (CoPP) profile policy used in the existing fabric. By default, the strict option is populated.

   VTEP HoldDown Time - Specifies the NVE source interface hold down time.

   Brownfield Overlay Network Name Format: Enter the format to be used to build the overlay network name during a brownfield import or migration. The network name should not contain any white spaces or special characters except underscore (_) and hyphen (-). The network name must not be changed once the brownfield migration has been initiated. See the Creating Networks for the Standalone Fabric section for the naming convention of the network name. The syntax is [<string> | $$VLAN_ID$$] $$VNI$$ [<string>| $$VLAN_ID$$] and the default value is Auto_Net_VNI$$VNI$$_VLAN$$VLAN_ID$$. When you create networks, the name is generated according to the syntax you specify. The following table describes the variables in the syntax.

   Variables	Description
   $$VNI$$	Specifies the network VNI ID found in the switch configuration. This is a mandatory keyword required to create unique network names.
   $$VLAN_ID$$	Specifies the VLAN ID associated with the network. VLAN ID is specific to switches, hence DCNM will pick the VLAN ID from one of the switches, where the network is found, randomly and use it in the name. We recommend not to use this unless the VLAN ID is consistent across the fabric for the VNI.
   <string>	This variable is optional and you can enter any number of alphanumeric characters that meet the network name guidelines.

   Example overlay network name: Site_VNI12345_VLAN1234

   Note	Ignore this field for greenfield deployments. The Brownfield Overlay Network Name Format applies for the following brownfield imports: CLI-based overlays Configuration profile-based overlay where the configuration profiles were created in Cisco DCNM Release 10.4(2).

   Enable VXLAN OAM - Enables the VXLAM OAM function for existing switches.

   This is enabled by default. Clear the check box to disable VXLAN OAM function.

   If you want to enable the VXLAN OAM function on specific switches and disable on other switches in the fabric, you can use freeform configurations to enable OAM and disable OAM in the fabric settings.

   Note	The VXLAN OAM feature in Cisco DCNM is only supported on a single fabric or site.

   Note that the NGOAM feature is unsupported on switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images.

   Enable Tenant DHCP – Select the check box to enable the tenant DHCP support.

   Note	Ensure that Enable Tenant DHCP is enabled before enabling DHCP related parameters in the overlay profiles.

   Enable NX-API - Specifies enabling of NX-API.

   Enable NX-API on HTTP - Specifies enabling of NX-API on HTTP.

   Enable Policy-Based Routing (PBR) - Select this check box to enable routing of packets based on the specified policy.

   Enable Strict Config Compliance - Enable the Strict Config Compliance feature by selecting this check box. By default, this feature is disabled.

   Enable AAA IP Authorization - Enables AAA IP authorization, when IP Authorization is enabled in the AAA Server

   Greenfield Cleanup Option – Enable or disable the switch cleanup option for Greenfield switches. This is applicable post-migration when new switches are added.

   Enable Precision Time Protocol (PTP): Enables PTP across a fabric. When you select this check box, PTP is enabled globally and on core-facing interfaces. Additionally, the PTP Source Loopback Id and PTP Domain Id fields are editable. For more information, see Precision Time Protocol for Easy Fabric.

   PTP Source Loopback Id: Specifies the loopback interface ID Loopback that is used as the Source IP Address for all PTP packets. The valid values range from 0 to 1023. The PTP loopback ID cannot be the same as RP, Phantom RP, NVE, or MPLS loopback ID. Otherwise, an error will be generated. The PTP loopback ID can be the same as BGP loopback or user-defined loopback which is created from DCNM.

   If the PTP loopback ID is not found during Save & Deploy, the following error is generated:

   Loopback interface to use for PTP source IP is not found. Please create PTP loopback interface on all the devices to enable PTP feature.

   PTP Domain Id: Specifies the PTP domain ID on a single network. The valid values range from 0 to 127.

   Enable MPLS Handoff: Select the check box to enable the MPLS Handoff feature. For more information, see Border Provisioning Use Case in VXLAN BGP EVPN Fabrics - MPLS SR and LDP Handoff.

   Note: For the brownfield import, you need to select the Enable MPLS Handoff feature. Most of the IFC configuration will be captured in switch_freeform.

   Underlay MPLS Loopback Id: Specifies the underlay MPLS loopback ID. The default value is 101.

   Enable Default Queuing Policies: Check this check box to apply QoS policies on all the switches in this fabric. To remove the QoS policies that you applied on all the switches, uncheck this check box, update all the configurations to remove the references to the policies, and save and deploy. From Cisco DCNM Release 11.3(1), pre-defined QoS configurations are included that can be used for various Cisco Nexus 9000 Series Switches. When you check this check box, the appropriate QoS configurations are pushed to the switches in the fabric. The system queuing is updated when configurations are deployed to the switches. You can perform the interface marking with defined queuing policies, if required, by adding the required configuration to the per interface freeform block.

   Review the actual queuing policies by opening the policy file in the template editor. From Cisco DCNM Web UI, choose Control > Template Library. Search for the queuing policies by the policy file name, for example, queuing_policy_default_8q_cloudscale. Choose the file and click the Modify/View template icon to edit the policy.

   See the Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide for platform specific details.

   N9K Cloud Scale Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus 9200 Series Switches and the Cisco Nexus 9000 Series Switches that ends with EX, FX, and FX2 in the fabric. The valid values are queuing_policy_default_4q_cloudscale and queuing_policy_default_8q_cloudscale. Use the queuing_policy_default_4q_cloudscale policy for FEXes. You can change from the queuing_policy_default_4q_cloudscale policy to the queuing_policy_default_8q_cloudscale policy only when FEXes are offline.

   N9K R-Series Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus switches that ends with R in the fabric. The valid value is queuing_policy_default_r_series.

   Other N9K Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all other switches in the fabric other than the switches mentioned in the above two options. The valid value is queuing_policy_default_other.

   Leaf Freeform Config and Spine Freeform Config - You can enter these fields after fabric transitioning is complete, as needed.

   Intra-fabric Links Additional Config - You can enter this field after fabric transitioning is complete, as needed.

8. Click the Resources tab.

   

   Manual Underlay IP Address Allocation – Do not select this check box if you are transitioning your VXLAN fabric management to DCNM.

   Review the ranges and ensure they are consistent with the existing fabric. The migration will honor the existing resources as found on the fabric. The range settings apply to post migration allocation.

   Underlay Routing Loopback IP Range - Specifies loopback IP addresses for the protocol peering.

   Underlay VTEP Loopback IP Range - Specifies loopback IP addresses for VTEPs.

   Underlay RP Loopback IP Range - Specifies the anycast or phantom RP IP address range.

   Underlay Subnet IP Range - IP addresses for underlay P2P routing traffic between interfaces.

   Layer 2 VXLAN VNI Range and Layer 3 VXLAN VNI Range - Specifies the VXLAN VNI IDs for the fabric.

   Network VLAN Range and VRF VLAN Range - VLAN ranges for the Layer 3 VRF and overlay network.

   Subinterface Dot1q Range - Specifies the subinterface range when L3 sub interfaces are used.

   VRF Lite Deployment - Specify the VRF Lite method for extending inter fabric connections.

   If you select Manual, the VRF Lite subnet details are required so that the resource manager can reserve the address space.

   If you select Back2BackOnly, ToExternalOnly, or Both, then the VRF Lite subnet fields are enabled.

   VRF Lite Subnet IP Range and VRF Lite Subnet Mask – These fields are populated with the DCI subnet details. Update the fields as needed.

   The values shown in your screen are automatically generated. If you want to update the IP address ranges, VXLAN Layer 2/Layer 3 network ID ranges or the VRF/Network VLAN ranges, ensure the following:

   Note

   When you update a range of values, ensure that it does not overlap with other ranges. You should only update one range of values at a time. If you want to update more than one range of values, do it in separate instances. For example, if you want to update L2 and L3 ranges, you should do the following. Update the L2 range and click Save. Click the Edit Fabric option again, update the L3 range and click Save.

   Service Network VLAN Range - Specifies a VLAN range in the Service Network VLAN Range field. This is a per switch overlay service network VLAN range. The minimum allowed value is 2 and the maximum allowed value is 3967.

   Route Map Sequence Number Range - Specifies the route map sequence number range. The minimum allowed value is 1 and the maximum allowed value is 65534.

   The remaining tabs do not require updates. However, their purpose is mentioned.

9. Click the Manageability tab - Enter the DNS, NTP, AAA, or syslog servers’ IP address, VRF, and other applicable information matching the switch configuration. If there are more than two servers for these features, add the configurations of the additional servers to the Leaf Freeform Config and Spine Freeform Config fields in the Advanced tab.

   Note	If AAA configs are not specified in the fabric settings, switch_freeform PTI with source as UNDERLAY_AAA and description as DCNM Extra AAA Configurations will be created.

10. Click the Bootstrap tab. Update the fields in this tab post transition, when new switches are added to the fabric.

11. Click the Configuration Backup tab. Leave the fields in this tab blank. You can update post transition.

12. Click Save after filling and updating relevant information. A note appears briefly at the bottom right part of the screen, indicating that the fabric is created. When a fabric is created, the fabric page comes up. The fabric name appears at the top left part of the screen.

    

    The Actions panel at the left part of the screen allows you to perform various functions. One of them is the Add switches option to add switches to the fabric. After you create a fabric, you should add fabric devices. The process is explained next:

Adding Switch Instances and Transitioning VXLAN Fabric Management to DCNM

1. In the fabric topology screen, click Add switches. The Inventory Management screen comes up. The Discover Existing Switches tab is displayed by default.

   

   The POAP tab is only used for adding new switches to the fabric. Use the tab only after migrating your existing fabric to DCNM.

2. Enter the IP address (Seed IP), administrator username and password (Username and Password fields) of the switch, and set the Max Hops count for the switch. Ensure that all fabric switches can be added to DCNM at once.

   Important - Ensure that the Preserve Config field remains set to yes. Selecting 'no' can cause significant configuration loss and fabric disruption.

   

3. Click Start discovery, at the bottom part of the screen. The switch with the specified IP address and switches up to two hops away (depending on the setting of Max Hops) from it are populated in the Scan Details section.

   

4. Select the check box next to the concerned switches and click Import into fabric.

   It is a best practice to discover multiple switches at once. The switches must be properly cabled and connected to the DCNM server and the switch status must be manageable.

   

   The switch discovery process is initiated. The Progress column displays progress for all the selected switches. It displays done for each switch on completion.

   Note

   You must not close the screen (and try to import switches again) till all selected switches are imported or an error message comes up. If an error message comes up, close the screen. The fabric topology screen comes up. The error messages are displayed at the top right part of the screen. Resolve the errors and initiate the import process again by clicking on Add Switches in the Actions panel.

   

   After DCNM discovers all the switches, and the Progress column displays done for all switches, close the screen. The fabric topology screen comes up again. The switch is in Migration Mode now and the Migration mode label is displayed on the switch icons.

   At this point, you must not try to add Greenfield or new switches. Support is not available for adding new switches during the migration process. It might lead to undesirable consequences for your network. However, you can add a new switch after the migration process is complete.

   

   Note

   The switch discovery process might fail for a few switches, and the Discovery Error message displayed. However, such switches are still displayed in the fabric topology. You must remove such switches from the fabric (Right-click the switch icon and click Discovery > Remove from fabric), and import them again. You must not proceed to the next step till all switches in the existing fabric are discovered in DCNM.

5. Each switch’s role and vPC pairing must be set during the fabric migration process.

   Right-click the switch icon and use the Set role option (Leaf, Border, etc) to update switch role.

   

   If you choose the Hierarchical layout for display (in the Actions panel), the topology automatically gets aligned as per role assignment, with the leaf switches at the bottom, the spine switches connected on top of them, and the border switches at the top.

   Note	The supported roles for switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images are Border Leaf, Border Spine, Leaf, and Spine

   vPC Pairing - The vPC pairing must be done for switches where the Layer 3 vPC peer-keep alive is used. The vPC configuration is automatically picked up from the switches when the vPC peer keep alive is established through the management option. This pairing reflects in the GUI only after the migration is complete.

   1. Right-click the switch icon and click vPC Pairing to set a vPC switch pair.

      

      The Select vPC peer screen comes up. It lists potential vPC peer switches.

   2. Select the appropriate switch and click OK. The fabric topology comes up again. The vPC pair is formed now.

      

   Note	Check if you have added all switches from the current fabric. If you have missed adding switches, add them now. Once you are certain that you have imported all existing switches, move to the next step, the Save and Deploy option.

6. Use the Save and Deploy option (at the top right part of the screen) to sync configurations between the switch and DCNM.

   The Saving Fabric Configuration message comes up immediately. This indicates that overlay and underlay network migration, and switch and port channel settings migration to DCNM is initiated.

   If there are configuration mismatches, error messages are displayed. Update changes in the fabric settings or the switch configuration as needed, and click Save and Deploy again.

   After the migration of underlay and overlay networks, the Configuration Deployment screen comes up.

   

   The Preview Config column is updated with entries denoting a specific number of lines.

   

   We strongly recommend that you preview the configuration before proceeding to deploy it on the switches. Click the Preview Config column entry. The Config Preview screen comes up. It lists the pending configurations on the switch.

   The Side-by-side Comparison tab displays the running configuration and expected configuration side-by-side.

   

   Close the preview screen.

7. Click Deploy Config at the bottom part of the screen to initiate pending configuration onto the switch. The Status column displays FAILED or SUCCESS state. For a FAILED status, investigate the reason for failure to address the issue.

   The progress bar shows 100% for each switch. After correct provisioning and successful configuration compliance, close the screen. In the fabric topology screen that comes up, all imported switch instances are displayed in green color, indicating successful configuration. Also, the Migration Mode label is not displayed on any switch icon.

   

Post-transitioning of VXLAN fabric management to DCNM - This completes the transitioning process of VXLAN fabric management to DCNM. Now, you can add new switches and provision overlay networks for your fabric. For details, refer the respective section in the Fabrics topic in the configuration guide.

Fabric Options

• Tabular View - By default, the switches are displayed in the topology view. Use this option to view switches in the tabular view.

• Refresh topology - Allows you to refresh the topology.

• Save Layout – Saves a custom view of the topology. You can create a specific view in the topology and save it for ease of use.

• Delete saved layout – Deletes the custom view of the topology

• Topology views - You can choose between Hierarchical, Random and Custom saved layout display options.

  • Hierarchical - Provides an architectural view of your topology. Various Switch Roles can be defined that draws the nodes on how you configure your CLOS topology.

  • Random - Nodes are placed randomly on the screen. DCNM tries to make a guess and intelligently place nodes that belong together in close proximity.

  • Custom saved layout - You can drag nodes around to your liking. Once you have the positions as how you like, you can click Save Layout to remember the positions. Next time you come to the topology, DCNM will draw the nodes based on your last saved layout positions.

• Restore Fabric – Allows you to restore the fabric to a prior DCNM configuration state (one month back, two months back, and so on). For more information, see Restore Fabric section.

• Backup Now: You can initiate a fabric backup manually by clicking Backup Now. Enter a name for the tag and click OK. Regardless of the settings you choose under the Configuration Backup tab in the Fabric Settings dialog box, you can initiate a backup using this option.

• Resync Fabric - Use this option to resynchronize DCNM state when there is a large scale out-of-band change, or if configuration changes do not register in the DCNM properly. The resync operation does a full CC run for the fabric switches and recollects “show run” and “show run all” commands from the switches. When you initiate the re-sync process, a progress message is displayed on the screen. During the re-sync, the running configuration is taken from the switches. The OUT-OF-SYNC/IN-SYNC status for the switches is recalculated based on the intent defined in DCNM.

• Add Switches – Allows you to add switch instances to the fabric.

• Fabric Settings – Allows you to view or edit fabric settings.

1. Choose Control > Fabric Builder.

   The Fabric Builder screen appears. When you log in for the first time, the Fabrics section has no entries. After you create a fabric, it is displayed on the Fabric Builder screen, wherein a rectangular box represents each fabric.

   A standalone or member fabric contains Switch_Fabric (in the Type field), the AS number (in the ASN field), and mode of replication (in the Replication Mode field).

   The technology is for a fabric with eBGP Routed Fabric or eBGP VXLAN EVPN Fabric. The mode of replication is only applicable for the eBGP VXLAN EVPN fabric, and not eBGP Routed fabric.

2. Click Create Fabric. The Add Fabric screen appears.

   The fields are explained:

   Fabric Name - Enter the name of the fabric.

   Fabric Template - From the drop-down menu, choose the Easy_Fabric_eBGP fabric template. The fabric settings for creating a standalone routed fabric comes up.

   
3. The General tab is displayed by default. The fields in this tab are:

   BGP ASN for Spines: Enter the BGP AS number of the fabric’s spine switches.

   BGP AS Mode: Choose Multi-AS or Dual-AS.

   In a Multi-AS fabric, the spine switches have a unique BGP AS number and each leaf switch has a unique AS number. If two leaf switches form a vPC switch pair, then they have the same AS number.

   In a Dual-AS fabric, the spine switches have a unique BGP AS number and the leaf switches have a unique AS number.

   The fabric is identified by the spine switch AS number.

   Underlay Subnet IP Mask - Specifies the subnet mask for the fabric interface IP addresses.

   Manual Underlay IP Address Allocation – Select this check box to disable Dynamic Underlay IP Address Allocations.

   Underlay Routing Loopback IP Range: Specifies loopback IP addresses for the protocol peering.

   Underlay Subnet IP Range: IP addresses for underlay P2P routing traffic between interfaces.

   Subinterface Dot1q Range: Specifies the subinterface range when L3 sub interfaces are used.

   NX-OS Software Image Version: Select an image from the drop-down list.

   If you upload Cisco NX-OS software images through the image upload option, the uploaded images are listed in this field. If you select an image, the system checks if the switch has the selected version. If not, an error message is displayed. You can resolve the error by clicking on Resolve. The image management screen comes up and you can proceed with the ISSU option. Alternatively, you can delete the release number and save it later.

   If you specify an image in this field, all switches in the fabric should run that image. If some devices do not run the image, a warning is prompted to perform an In-Service Software Upgrade (ISSU) to the specified image. Till all devices run the specified image, the deployment process will be incomplete.

   If you want to deploy more than one type of software image on the fabric switches, don’t specify any image. If an image is specified, delete it.

4. Click EVPN. Most of the fields in this tab are auto-populated. The fields are:

   

   Enable EVPN VXLAN Overlay: Enables the VXLAN overlay provisioning for the fabric.

   You can convert a routed fabric to a VXLAN enabled fabric by selecting this option. When the fabric is VXLAN enabled, you can create and deploy overlay networks or VRFs. The procedure for creating and deploying networks or VRFs is the same as in Easy_Fabric_11_1.

   Note	The rest of the fields in the EVPN tab section are only applicable if you enable the EVPN VXLAN Overlay.

   Routed Fabric: You must disable the Enable EVPN VXLAN Overlay field for Routed fabric (an IP fabric with no VXLAN encapsulation) creation.

   Whether you create an eBGP Routed or eBGP VXLAN fabric, the fabric uses eBGP as the control plane to build intra-fabric connectivity. Links between spine and leaf switches are autoconfigured with point-to-point (p2p) numbered IP addresses with eBGP peering built on top.

   If a network or a VRF is created in a fabric, you cannot switch between VXLAN EVPN mode and Routed Fabric mode by selecting the Enable EVPN VXLAN Overlay check box. You need to delete these networks or VRFs to change the fabric setting.

   First Hop Redundancy Protocol: Specifies the FHRP protocol. Choose either hsrp or vrrp.

   Note	After a network has been created, you cannot change this fabric setting. You should delete all networks, and then change the FHRP setting.

   Anycast Gateway MAC: Anycast gateway MAC address for the leaf switches.

   Enable VXLAN OAM: Enables the VXLAM OAM function for existing switches. This is enabled by default. Clear the check box to disable VXLAN OAM function.

   If you want to enable the VXLAN OAM function on specific switches and disable on other switches in the fabric, you can use freeform configurations to enable OAM and disable OAM in the fabric settings.

   Note	The VXLAN OAM feature in Cisco DCNM is only supported on a single fabric or site.

   Enable Tenant DHCP: Enables tenant DHCP support.

   vPC advertise-pip: Check the check box to enable the Advertise PIP feature.

   Replication Mode : The mode of replication that is used in the fabric, Ingress Replication, or Multicast.

   Multicast Group Subnet: IP address prefix used for multicast communication. A unique IP address is allocated from this group for each overlay network.

   Enable Tenant Routed Multicast: Check the check box to enable Tenant Routed Multicast (TRM) as the fabric overlay multicast protocol.

   Default MDT Address for TRM VRFs: The multicast address for Tenant Routed Multicast traffic is populated. By default, this address is from the IP prefix specified in the Multicast Group Subnet field. When you update either field, ensure that the TRM address is chosen from the IP prefix specified in Multicast Group Subnet.

   Rendezvous-Points: Enter the number of spine switches acting as rendezvous points.

   RP mode: Choose from the two supported multicast modes of replication, ASM (for Any-Source Multicast [ASM]) or BiDir (for Bidirectional PIM [BIDIR-PIM]). When you choose ASM, the BiDir related fields are not enabled. When you choose BiDir, the BiDir related fields are enabled.

   Note	BIDIR-PIM is supported on Cisco's Cloud Scale Family platforms 9300-EX and 9300-FX/FX2, and software release 9.2(1) onwards.

   Underlay RP Loopback ID: The loopback ID used for the rendezvous point (RP), for multicast protocol peering purposes in the fabric underlay. The default is 254.

   The following fields are enabled if you choose bidir. Depending on the RP count, either 2 or 4 phantom RP loopback ID fields are enabled.

   • Underlay Primary RP Loopback ID: The primary loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   • Underlay Backup RP Loopback ID: The secondary (or backup) loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   The following Loopback ID options are applicable only when the RP count is 4.

   • Underlay Second Backup RP Loopback ID: The second backup loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   • Underlay Third Backup RP Loopback ID: The third backup loopback ID used for the phantom RP, for multicast protocol peering purposes in the fabric underlay.

   VRF Template and VRF Extension Template: Specifies the VRF template for creating VRFs, and the VRF extension template for enabling VRF extension to other fabrics.

   Network Template and Network Extension Template: Specifies the network template for creating networks, and the network extension template for extending networks to other fabrics.

   Underlay VTEP Loopback IP Range: Specifies the loopback IP address range for VTEPs.

   Underlay RP Loopback IP Range: Specifies the anycast or phantom RP IP address range.

   Layer 2 VXLAN VNI Range and Layer 3 VXLAN VNI Range: Specifies the VXLAN VNI IDs for the fabric.

   Network VLAN Range and VRF VLAN Range: VLAN ranges for the Layer 3 VRF and overlay network.

   VRF Lite Deployment: Specifies the VRF Lite method for extending inter fabric connections. Only the 'Manual' option is supported.

5. Click vPC. The fields in the tab are:

   

   vPC Peer Link VLAN: VLAN used for the vPC peer link SVI.

   vPC Peer Keep Alive option: Choose the management or loopback option. If you want to use IP addresses assigned to the management port and the management VRF, choose management. If you use IP addresses assigned to loopback interfaces (and a non-management VRF), choose loopback. If you use IPv6 addresses, you must use loopback IDs.

   vPC Auto Recovery Time: Specifies the vPC auto recovery time-out period in seconds.

   vPC Delay Restore Time: Specifies the vPC delay restore period in seconds.

   vPC Peer Link Port Channel Number - Specifies the Port Channel ID for a vPC Peer Link. By default, the value in this field is 500.

   vPC IPv6 ND Synchronize: Enables IPv6 Neighbour Discovery synchronization between vPC switches. The check box is enabled by default. Clear the check box to disable the function.

   Fabric wide vPC Domain Id: Enables the usage of same vPC Domain Id on all vPC pairs in the fabric. When you select this field, the vPC Domain Id field is editable.

   vPC Domain Id - Specifies the vPC domain ID to be used on all vPC pairs.

6. Click the Protocols tab. The fields in the tab are:

   

   Routing Loopback Id - The loopback interface ID is populated as 0 by default. It is used as the BGP router ID.

   VTEP Loopback Id - The loopback interface ID is populated as 1 since loopback1 is usually used for the VTEP peering purposes.

   Enable BGP Authentication: Select the check box to enable BGP authentication. Deselect the check box to disable it. If you enable this field, the BGP Authentication Key Encryption Type and BGP Authentication Key fields are enabled.

   BGP Authentication Key Encryption Type: Choose the 3 for 3DES encryption type, or 7 for Cisco encryption type.

   BGP Authentication Key: Enter the encrypted key based on the encryption type.

   Note	Plain text passwords are not supported. Login to the switch, retrieve the encrypted key and enter it in the BGP Authentication Key field. Refer the Retrieving the Authentication Key section for details.

   Enable BFD: Select the check box to enable feature bfd on all switches in the fabric. This feature is valid only on IPv4 underlay and the scope is within a fabric.

   From Cisco DCNM Release 11.3(1), BFD within a fabric is supported natively. The BFD feature is disabled by default in the Fabric Settings. If enabled, BFD is enabled for the underlay protocols with the default settings. Any custom required BFD configurations must be deployed via the per switch freeform or per interface freeform policies.

   The following config is pushed after you select the Enable BFD check box:

   feature bfd

   For information about BFD feature compatibility, refer your respective platform documentation and for information about the supported software images, see Compatibility Matrix for Cisco DCNM.

   Enable BFD for BGP: Select the check box to enable BFD for the BGP neighbor. This option is disabled by default.

   Enable BFD Authentication: Select the check box to enable BFD authentication. If you enable this field, the BFD Authentication Key ID and BFD Authentication Key fields are editable.

   BFD Authentication Key ID: Specifies the BFD authentication key ID for the interface authentication.

   BFD Authentication Key: Specifies the BFD authentication key.

   For information about how to retrieve the BFD authentication parameters, see Retrieving the Encrypted BFD Authentication Key.

7. Click the Advanced tab. The fields in the tab are:

   

   Enable Policy-Based Routing (PBR) - Select this check box to enable routing of packets based on the specified policy.

   Intra Fabric Interface MTU - Specifies the MTU for the intra fabric interface. This value should be an even number.

   Layer 2 Host Interface MTU - Specifies the MTU for the layer 2 host interface. This value should be an even number.

   Power Supply Mode: Choose the appropriate power supply mode.

   CoPP Profile: Choose the appropriate Control Plane Policing (CoPP) profile policy for the fabric. By default, the strict option is populated.

   VTEP HoldDown Time - Specifies the NVE source interface hold down time.

   VRF Lite Subnet IP Range and VRF Lite Subnet Mask – These fields are populated with the DCI subnet details. Update the fields as needed.

   Enable NX-API - Specifies enabling of NX-API.

   Enable NX-API on HTTP - Specifies enabling of NX-API on HTTP.

   Enable Strict Config Compliance - Enable the Strict Config Compliance feature by selecting this check box. By default, this feature is disabled.

   Enable AAA IP Authorization - Enables AAA IP authorization, when IP Authorization is enabled in the AAA Server

   Enable DCNM as Trap Host - Select this check box to enable DCNM as a trap host.

   Greenfield Cleanup Option: Enable or disable the switch cleanup option for greenfield switches.

   Enable Default Queuing Policies: Check this check box to apply QoS policies on all the switches in this fabric. To remove the QoS policies that you applied on all the switches, uncheck this check box, update all the configurations to remove the references to the policies, and save and deploy. From Cisco DCNM Release 11.3(1), pre-defined QoS configurations are included that can be used for various Cisco Nexus 9000 Series Switches. When you check this check box, the appropriate QoS configurations are pushed to the switches in the fabric. The system queuing is updated when configurations are deployed to the switches. You can perform the interface marking with defined queuing policies, if required, by adding the required configuration to the per interface freeform block.

   Review the actual queuing policies by opening the policy file in the template editor. From Cisco DCNM Web UI, choose Control > Template Library. Search for the queuing policies by the policy file name, for example, queuing_policy_default_8q_cloudscale. Choose the file and click the Modify/View template icon to edit the policy.

   See the Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide for platform specific details.

   N9K Cloud Scale Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus 9200 Series Switches and the Cisco Nexus 9000 Series Switches that ends with EX, FX, and FX2 in the fabric. The valid values are queuing_policy_default_4q_cloudscale and queuing_policy_default_8q_cloudscale. Use the queuing_policy_default_4q_cloudscale policy for FEXes. You can change from the queuing_policy_default_4q_cloudscale policy to the queuing_policy_default_8q_cloudscale policy only when FEXes are offline.

   N9K R-Series Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all Cisco Nexus switches that ends with R in the fabric. The valid value is queuing_policy_default_r_series.

   Other N9K Platform Queuing Policy: Choose the queuing policy from the drop-down list to be applied to all other switches in the fabric other than the switches mentioned in the above two options. The valid value is queuing_policy_default_other.

   Leaf Freeform Config: Add CLIs that should be added to switches that have the Leaf, Border, and Border Gateway roles.

   Spine Freeform Config - Add CLIs that should be added to switches with a Spine, Border Spine, and Border Gateway Spine roles.

   Intra-fabric Links Additional Config - Add CLIs that should be added to the intra-fabric links.

8. Click the Manageability tab.

   

   The fields in this tab are:

   DNS Server IPs - Specifies the comma separated list of IP addresses (v4/v6) of the DNS servers.

   DNS Server VRFs - Specifies one VRF for all DNS servers or a comma separated list of VRFs, one per DNS server.

   NTP Server IPs - Specifies comma separated list of IP addresses (v4/v6) of the NTP server.

   NTP Server VRFs - Specifies one VRF for all NTP servers or a comma separated list of VRFs, one per NTP server.

   Syslog Server IPs – Specifies the comma separated list of IP addresses (v4/v6) IP address of the syslog servers, if used.

   Syslog Server Severity – Specifies the comma separated list of syslog severity values, one per syslog server. The minimum value is 0 and the maximum value is 7. To specify a higher severity, enter a higher number.

   Syslog Server VRFs – Specifies one VRF for all syslog servers or a comma separated list of VRFs, one per syslog server.

   AAA Freeform Config – Specifies the AAA freeform configs.

   If AAA configs are specified in the fabric settings, switch_freeform PTI with source as UNDERLAY_AAA and description as “AAA Configurations” will be created.

9. Click the Bootstrap tab.

   

   Enable Bootstrap - Select this check box to enable the bootstrap feature.

   After you enable bootstrap, you can enable the DHCP server for automatic IP address assignment using one of the following methods:

   • External DHCP Server: Enter information about the external DHCP server in the Switch Mgmt Default Gateway and Switch Mgmt IP Subnet Prefix fields.

   • Local DHCP Server: Enable the Local DHCP Server checkbox and enter details for the remaining mandatory fields.

   Enable Local DHCP Server - Select this check box to initiate enabling of automatic IP address assignment through the local DHCP server. When you select this check box, the DHCP Scope Start Address and DHCP Scope End Address fields become editable.

   If you do not select this check box, DCNM uses the remote or external DHCP server for automatic IP address assignment.

   DHCP Version – Select DHCPv4 or DHCPv6 from this drop-down list. When you select DHCPv4, the Switch Mgmt IPv6 Subnet Prefix field is disabled. If you select DHCPv6, the Switch Mgmt IP Subnet Prefix is disabled.

   Note

   Cisco DCNM IPv6 POAP is not supported with Cisco Nexus 7000 Series Switches. Cisco Nexus 9000 and 3000 Series Switches support IPv6 POAP only when switches are either L2 adjacent (eth1 or out-of-band subnet must be a /64) or they are L3 adjacent residing in some IPv6 /64 subnet. Subnet prefixes other than /64 are not supported.

   DHCP Scope Start Address and DHCP Scope End Address - Specifies the first and last IP addresses of the IP address range to be used for the switch out of band POAP.

   Switch Mgmt Default Gateway - Specifies the default gateway for the management VRF on the switch.

   Switch Mgmt IP Subnet Prefix - Specifies the prefix for the Mgmt0 interface on the switch. The prefix should be between 8 and 30.

   DHCP scope and management default gateway IP address specification - If you specify the management default gateway IP address 10.0.1.0 and subnet mask 24, ensure that the DHCP scope is within the specified subnet, between 10.0.1.1 and 10.0.1.254.

   Switch Mgmt IPv6 Subnet Prefix - Specifies the IPv6 prefix for the Mgmt0 interface on the switch. The prefix should be between 112 and 126. This field is editable if you enable IPv6 for DHCP.

   Enable AAA Config – Select this check box to include AAA configs from the Manageability tab during device bootup.

   Bootstrap Freeform Config - (Optional) Enter additional commands as needed. For example, if you are using AAA or remote authentication related configurations, you need to add these configurations in this field to save the intent. After the devices boot up, they contain the intent defined in the Bootstrap Freeform Config field.

   Copy-paste the running-config to a freeform config field with correct indentation, as seen in the running configuration on the NX-OS switches. The freeform config must match the running config. For more information, see Resolving Freeform Config Errors in Switches.

   DHCPv4/DHCPv6 Multi Subnet Scope - Specifies the field to enter one subnet scope per line. This field is editable after you check the Enable Local DHCP Server check box.

   The format of the scope should be defined as:

   DHCP Scope Start Address, DHCP Scope End Address, Switch Management Default Gateway, Switch Management Subnet Prefix

   For example: 10.6.0.2, 10.6.0.9, 10.6.0.1, 24

10. Click the Configuration Backup tab. The fields on this tab are:

    

    Hourly Fabric Backup: Select the check box to enable an hourly backup of fabric configurations and the intent.

    You can enable an hourly backup for fresh fabric configurations and the intent as well. If there is a configuration push in the previous hour, DCNM takes a backup.

    Intent refers to configurations that are saved in DCNM but yet to be provisioned on the switches.

    Scheduled Fabric Backup: Check the check box to enable a daily backup. This backup tracks changes in running configurations on the fabric devices that are not tracked by configuration compliance.

    Scheduled Time: Specify the scheduled backup time in a 24-hour format. This field is enabled if you check the Scheduled Fabric Backup check box.

    Select both the check boxes to enable both back up processes.

    The backup process is initiated after you click Save.

    Note

    Hourly and scheduled backup processes happen only during the next periodic configuration compliance activity, and there can be a delay of up to an hour. To trigger an immediate backup, do the following: Choose Control > Fabric Builder. The Fabric Builder screen comes up. Click within the specific fabric box. The fabric topology screen comes up. From the Actions panel at the left part of the screen, click Re-Sync Fabric.

    You can also initiate the fabric backup in the fabric topology window. Click Backup Now in the Actions pane.

11. Click Save after filling and updating relevant information. A note appears briefly at the bottom right part of the screen, indicating that the fabric is created. When a fabric is created, the fabric page comes up. The fabric name appears at the top left part of the screen.

    (At the same time, the newly created fabric instance appears on the Fabric Builder screen. To go to the Fabric Builder screen, click the left arrow (←) button above the Actions panel [to the left of the screen]).

    The Actions panel at the left part of the screen allows you to perform various functions. One of them is the Add switches option to add switches to the fabric. After you create a fabric, you should add fabric devices. The options are explained:

    • Tabular View - By default, the switches are displayed in the topology view. Use this option to view switches in the tabular view.

    • Refresh topology - Allows you to refresh the topology.

    • Save Layout – Saves a custom view of the topology. You can create a specific view in the topology and save it for ease of use.

    • Delete saved layout – Deletes the custom view of the topology

    • Topology views - You can choose between Hierarchical, Random and Custom saved layout display options.

      • Hierarchical - Provides an architectural view of your topology. Various Switch Roles can be defined that draws the nodes on how you configure your CLOS topology.

      • Random - Nodes are placed randomly on the screen. DCNM tries to make a guess and intelligently place nodes that belong together in close proximity.

      • Custom saved layout - You can drag nodes around to your liking. Once you have the positions as how you like, you can click Save Layout to remember the positions. Next time you come to the topology, DCNM will draw the nodes based on your last saved layout positions.

    • Restore Fabric – Allows you to restore the fabric to a prior DCNM configuration state (one month back, two months back, and so on). For more information, see Restore Fabric section.

    • Backup Now: You can initiate a fabric backup manually by clicking Backup Now. Enter a name for the tag and click OK. Regardless of the settings you choose under the Configuration Backup tab in the Fabric Settings dialog box, you can initiate a backup using this option.

    • Resync Fabric - Use this option to resynchronize DCNM state when there is a large scale out-of-band change, or if configuration changes do not register in the DCNM properly. The resync operation does a full CC run for the fabric switches and recollects “show run” and “show run all” commands from the switches. When you initiate the re-sync process, a progress message is displayed on the screen. During the re-sync, the running configuration is taken from the switches. Then, the OUT-OF-SYNC/IN-SYNC status for the switch is recalculated based on the intent or expected configuration defined in DCNM versus the current running configuration that was taken from the switches.

    • Add Switches – Allows you to add switch instances to the fabric.

    • Fabric Settings – Allows you to view or edit fabric settings.

    • Cloud icon - Click the Cloud icon to display (or not display) an Undiscovered cloud.

      When you click the icon, the Undiscovered cloud and its links to the selected fabric topology are not displayed.

      Click the Cloud icon again to display the Undiscovered cloud.

SCOPE - You can toggle between fabrics by using the SCOPE drop-down box at the top right part of the screen. The current fabric is highlighted. An MSD and its member fabrics are distinctly displayed, wherein the member fabrics are indented, under the MSD fabric.

In DCNM 11.1(1) release, you can add switches to the external fabric. Generic pointers:

• An external fabric is a monitor-only or managed mode fabric.

• You can import, remove, and delete switches for an external fabric.

• For Inter-Fabric Connection (IFC) cases, you can choose Cisco 9000, 7000 and 5600 Series switches as destination switches in the external fabric.

• You can use non-existing switches as destination switches.

• The template that supports an external fabric is External_Fabric.

• If an external fabric is an MSD fabric member, then the MSD topology screen displays the external fabric with its devices, along with the member fabrics and their devices.

  When viewed from an external fabric topology screen, any connections to non-DCNM managed switches are represented by a cloud icon labelled as Undiscovered.

• You can set up a Multi-Site or a VRF-lite IFC by manually configuring the links for the border devices in the VXLAN fabric or by using an automatic Deploy Border Gateway Method or VRF Lite IFC Deploy Method. If you are configuring the links manually for the border devices, we recommend using the Core Router role to set up a multisite eBGP underlay from a Border Gateway device to a Core Router and the Edge Router role to set up a VRF-lite Inter-Fabric Connection (IFC) from a Border device to an Edge device.

• You can discover the following non-Nexus devices in an external fabric:

  • IOS-XE family devices: Cisco CSR 1000v, Cisco IOS XE Gibraltar 16.10.x

  • IOS-XR family devices: ASR 9000 Series Routers, IOS XR Release 6.5.2 and Cisco NCS 5500 Series Routers, IOS XR Release 6.5.3

  • Arista 4.2 (Any model)

• Configure all the non-Nexus devices, except Cisco CSR 1000v, before adding them to the external fabric.

• You can connect a Cisco data center to a public cloud using Cisco CSR 1000v. See the Connecting Cisco Data Center and a Public Cloud chapter for a use case.

External Fabric Depiction in an MSD Fabric Topology

The MSD topology screen displays MSD member fabrics and external fabrics together. The external fabric External65000 is displayed as part of the MSD topology.

Note	When you deploy networks or VRFs for the VXLAN fabric, the deployment page (MSD topology view) shows the VXLAN and external fabrics that are connected to each other.

A Multi-Site Domain (MSD) is a multifabric container that is created to manage multiple member fabrics. An MSD is a single point of control for definition of overlay networks and VRFs that are shared across member fabrics. When you move fabrics (that are designated to be part of the multifabric overlay network domain) under the MSD as member fabrics, the member fabrics share the networks and VRFs created at the MSD-level. This way, you can consistently provision network and VRFs for different fabrics, at one go. It significantly reduces the time and complexity involving multiple fabric provisionings.

Since server networks and VRFs are shared across the member fabrics (as one stretched network), the new networks and VRFs provisioning function is provided at the MSD fabric level. Any new network and VRF creation is only allowed for the MSD. All member fabrics inherit any new network and VRF created for the MSD.

In DCNM 11.1(1) release, in addition to member fabrics, the topology view for the MSD fabric is introduced. This view displays all member fabrics, and how they are connected to each other, in one view.

Also, a deployment view is introduced for the MSD fabric. You can deploy overlay networks (and VRFs) on member fabrics from a single topology deployment screen, instead of visiting each member fabric deployment screen separately and deploying.

Note	The MSD feature is unsupported on the switches with the Cisco NX-OS Release 7.0(3)I4(8b) and 7.0(4)I4(x) images. The VXLAN OAM feature in Cisco DCNM is only supported on a single fabric or site.

A few fabric-specific terms:

• Standalone fabric: A fabric that is not part of an MSD is referred as a standalone fabric from the MSD perspective. Before the MSD concept, all fabrics were considered standalone, though two or more such fabrics can be connected with each other.

• Member fabrics: Fabrics that are part of an MSD are called member fabrics or members. Create a standalone fabric (of the type Easy_Fabric) first and then move it within an MSD as a member fabric.

When a standalone fabric is added to the MSD, the following actions take place:

• The standalone fabric's relevant attributes and the network and VRF definitions are checked against that of the MSD. If there is a conflict, then the standalone fabric addition to the MSD fails. If there are no conflicts, then the standalone fabric becomes a member fabric for the MSD. If there is a conflict, the exact conflicts are logged in the pending errors log for the MSD fabric. You can remedy the conflicts and then attempt to add the standalone fabric to the MSD again.

• All the VRFs and networks definitions from the standalone fabric that do not have presence in the MSD are copied over to the MSD and in turn inherited to each of its other existing member fabrics.

• The VRFs (and their definitions) from the MSD (such as the MSD's VRF, and L2 and L3 VNI parameters that do not have presence in the standalone fabric) are inherited into the standalone fabric that just became a member.

MSD and Member Fabric Process Flow

An MSD has multiple sites (and hence, multiple member fabrics under an MSD). VRFs and networks are created for the MSD and get inherited by the member fabrics. For example, VRF-50000 (and L3 network with ID 50000), and L2 networks with IDs 30000 and 30001 are created for the MSD, in one go.

A high-level flow chart of the MSD and member fabric creation and MSD-to-member fabric inheritance process:

The sample flow explained the inheritance from the MSD to one member. An MSD has multiple sites (and hence, multiple member fabrics under an MSD). A sample flow from an MSD to multiple members:

In this example, VRF-50000 (and L3 network with ID 50000), and L2 networks with IDs 30000 and 30001 are created in one go. Networks and VRFs are deployed on the member fabric switches, one after another, as depicted in the image.

In DCNM 11.1(1), you can provision overlay networks through a single MSD deployment screen.

Note	If you move a standalone fabric with existing networks and VRFs to an MSD, DCNM does appropriate validation. This is explained in detail in an upcoming section.

Upcoming sections in the document explain the following:

• Creation of an MSD fabric.

• Creation of a standalone fabric (as a potential member) and its movement under the MSD as a member.

• Creation of networks and VRFs in the MSD and their inheritance to the member fabrics.

• Deployment of networks and VRFs from the MSD and member fabric topology views.

• Other scenarios for fabric movement:

  • Standalone fabric with existing networks and VRFs to an MSD fabric.

  • Member fabric from one MSD to another.

Creating and Deploying Networks and VRFs in an MSD Fabric

In standalone fabrics, networks and VRFs are created for each fabric. In an MSD fabric, networks and VRFs should be created at the MSD fabric level. The networks and VRFs are inherited by all the member networks. You cannot create or delete networks and VRFs for member fabrics. However, you can edit them.

For example, consider an MSD fabric with two member fabrics. If you create three networks in the MSD fabric, then all three networks will automatically be available for deployment in both the member fabrics.

Though member fabrics inherit the MSD fabric's networks and VRFs, you have to deploy the networks and VRFs distinctly, for each fabric.

In DCNM 11.1(1) release, a deployment view is introduced for the MSD, in addition to the per-fabric deployment view. In this view, you can view and provision overlay networks for all member fabrics within the MSD, at once. However, you still have to apply and save network and VRF configurations distinctly, for each fabric.

Note

Networks and VRFs are the common identifiers (represented across member fabrics) that servers (or end hosts) are grouped under so that traffic can be sent between the end hosts based on the network and VRF IDs, whether they reside in the same or different fabrics. Since they have common representation across member fabrics, networks and VRFs can be provisioned at one go. As the switches in different fabrics are physically and logically distinct, you have to deploy the same networks and VRFs separately for each fabric.

For example, if you create networks 30000 and 30001 for an MSD that contains two member fabrics, the networks are automatically created for the member fabrics and are available for deployment.

In DCNM 11.1(1) release, you can deploy 30000 and 30001 on the border devices of all member fabrics through a single (MSD fabric) deployment screen. Prior to this, you had to access the first member fabric deployment screen, deploy 30000 and 300001 on the fabric's border devices, and then access the second member fabric deployment screen and deploy again.

Networks and VRFs are created in the MSD and deployed in the member fabrics. The steps are explained below:

1. Create networks and VRFs in the MSD fabric.

2. Deploy the networks and VRFs in the member fabric devices, one fabric at a time.

Editing Networks in the Member Fabric

An MSD can contain multiple fabrics. These fabrics forward BUM traffic via Multicast or Ingress replication. Even if all the fabrics use multicast for BUM traffic, the multicast groups within these fabrics need not be the same.

When you create a network in MSD, it is inherited by all the member fabrics. However, the multicast group address is a fabric instance variable. To edit the multicast group address, you need to navigate to the member fabric and edit the network. For more information about the Multicast Group Address field, see Creating Networks for the Standalone Fabric.

1. Select the network and click the Edit option at the top left part of the window. The Edit Network window comes up.

2. Update the multicast group address in one of the following ways:

   • Under Network Profile, click the Generate Multicast IP button to generate a new multicast group address for the selected network, and click Save.

   • Click the Advanced tab in the Network Profile section, update the multicast group address, and click Save.

   

Note	The Generate Multicast IP option is only available for member fabric networks and not MSD networks.

Deployment and Undeployment of Networks and VRFs in Member Fabrics

Before you begin, ensure that you have created networks at the MSD fabric level since the member fabric inherits networks and VRFs created for the MSD fabric.

Note	The deployment (and undeployment) of networks and VRFs in member fabrics are the same as explained for standalone fabrics. Refer the standalone fabric documentation (Networks Deployment and VRFs Deployment sections in the Networks and VRFs Creation and Deployment in a Standalone Fabric topic).

You can right-click the switch to view various options:

• Set Role - Assign a role to the switch. You can assign any one of the following roles to a switch:

  • Spine

  • Leaf (Default role)

  • Border

  • Border Spine

  • Border Gateway

  • Border Gateway Spine

Note	Changing of the switch role is allowed only before executing Save & Deploy.

Starting from DCNM 11.1(1), you can change switch roles if there are no overlays on the switches. The updated configuration is then generated after you click Save and Deploy. The following switch role changes are allowed:

• Leaf to Border

• Border to Leaf

• Leaf to Border Gateway

• Border Gateway to Leaf

• Border to Border Gateway

• Border Gateway to Border

• Spine to Border Spine

• Border Spine to Spine

• Spine to Border Gateway Spine

• Border Gateway Spine to Spine

• Border Spine to Border Gateway Spine

• Border Gateway Spine to Border Spine

Changing of switch role from any Leaf role to any Spine role and from any Spine role to any Leaf role is not allowed.

Switch[<serial-number>]: Role change from <switch-role> to <switch-role> is not permitted.

You can then change the switch role to the role that was set earlier, or set a new role that adheres to the rules mentioned above, and configure the fabric.

If you have not created any policy template instances before clicking Save and Deploy, and there are no overlays, you can change the role of a switch to any other required role.

Switches role should be the same for VPC pairing. peer1 <serial-number>: [<switch-role>], peer2 <serial-number>: [<switch-role>]

To prevent this scenario, change the switch roles of both the switches in the vPC pair to the same role.

In the fabric topology screen, click Tabular view option in the Actions panel, at the left part of the screen. The Switches | Links screen comes up.

The Switches tab is for managing switch operations and the Links tab is for adding and updating fabric links. Each row represents a switch in the fabric, and displays switch details, including its serial number.

The buttons at the top of the table are explained, from left to right direction. Some options are also available when you right-click the switch icon. However, the Switches tab enables you to provision configurations on multiple switches (for example, adding and deploying policies) simultaneously.

• Add switches to the fabric. This option is also available in the topology page (Add switches option in Actions panel).

• Initiate the switch discovery process by DCNM afresh.

• Update device credentials such as authentication protocol, username and password.

• Reload the switch.

• Remove the switch from the fabric.

• View/Edit Policies: Add, update and delete a policy. The policies are template instances of templates in the template library. After creating a policy, you should deploy it on the switches using the Deploy option available in the View/edit Policies screen. You can select more than one policy and view them.

  Note	If you select multiple switches and deploy a policy instance, then it will be deployed on all the selected switches.

• Manage Interfaces: Deploy configurations on the switch interfaces.

• History: View deployment history on the selected switch.

• Deploy: Deploy switch configurations. From Cisco DCNM Release 11.3(1), you can deploy configurations for multiple devices using the Deploy button.

  Note	In an MSD fabric, you can deploy configurations only on the Border Gateway, Border Gateway Spine, Border Gateway Super-Spine, or External Fabric switches.

You can add links between border switches of different fabrics (inter-fabric links) or between switches in the same fabric (intra-fabric links). You can only create an inter-fabric connection (IFC) for a switch that is managed by DCNM.

There are scenarios where you might want to define links between switches before connecting them physically. The links could be inter-fabric or intra-fabric links. Doing so, you can express and represent your intent to add links. The links with intent are displayed in a different colour till they are actually converted to functional links. Once you physically connect the links, they are displayed as connected.

Management links might show up in the fabric topology as red colored links. To remove such links, right-click the link and click Delete Link.

From Cisco DCNM Release 11.1(1), the Border Spine and Border Gateway Spine roles are added to switch roles for border switches.

You can create links between existing and pre-provisioned devices as well by selecting the pre-provisioned device as the destination device.

• Check the Auto Deploy Flag check box in the Link Management dialog box. Checking this check box enables VRF lite deployment on both ends of the link for managed devices.

• When you extend the VRF lite in a back-to-back scenario, the VRF should already be present in the peer fabric and the VRF name should be the same. An error message appears if the VRF is not present in the peer fabric and if you try to extend the VRF lite.

• When you extend the VRF lite between an easy fabric and an external fabric, the VRF name can be the same as that of the source fabric, default, or another VRF name. However, the child PTIs for the subinterface and the VRF creation or peering on the external fabric has the source. Hence, you cannot edit or delete the policies from the View/Edit policies window.

• Besides the IPv6 address, enter the IP mask, IPv4 address, and the neighbor IP address as well to deploy VRF from topdown using symmetric VRF lite.

• Deploy configurations in both the fabrics.

• You can edit or delete IFCs in the Link tab in the VXLAN fabric. The extra consideration for auto configured IFCs is that, in order to prevent the regeneration of IFC on next save and deploy, change the mode back to manual mode, or save the configuration only on the relevant devices.

• In a back-to-back scenario, if you delete the VRF lite IFC on one of the fabrics, the VRF lite is deleted from the peer fabric as well.

• When you want to delete a VRF lite between an easy fabric and an external fabric, delete the extension in the easy fabric using the top-down approach. The extension is automatically deleted from the external fabric.

• Deploy the configurations in both the fabrics.

See the Border Provisioning Use Case in VXLAN BGP EVPN Fabrics - VRF Lite chapter for a use case on VRF Lite.

From Cisco DCNM 11.3(1), support for the Top-of-Rack (ToR) switches is added in DCNM. You can add the Layer 2 ToR switches in an external fabric, and they can be connected to the Leaf switches in the Easy Fabric. Typically, the Leaf and ToR devices are connected with back-to-back vPC connection. For more information, see Supported Topologies for ToR Switches.