Cisco Nexus 1000V VXLAN Configuration Guide, Release 4.2(1)SV2(2.1a)
Configuring VXLANs
Downloads: This chapterpdf (PDF - 1.37MB) The complete bookPDF (PDF - 2.75MB) | The complete bookePub (ePub - 316.0KB) | Feedback

Configuring VXLANs

Configuring VXLANs

This chapter contains the following sections:

Information About VXLANs

Prerequisites for VXLANs

VXLANs have the following prerequisites:

  • The Cisco Nexus 1000V uplink port profiles and all interconnecting switches and routers between the ESX hosts must have their supported maximum transmission unit (MTU) set to at least 50 bytes larger than the MTU of the Virtual Machines (VMs). For example, the VMs default to using a 1500 byte MTU (same as the uplinks and physical devices), so you must set them to at least 1550 bytes. If this configuration is not possible, you should lower all VM vNICs MTU to 50 bytes smaller than what the physical network supports, such as 1450 bytes. For more information, see the Cisco Nexus 1000V Port Profile Configuration Guide.
  • If the Cisco Nexus 1000V is using a port channel for its uplinks, you should set the load distribution algorithm to a 5-tuple hash (IP/Layer 4/Layer 4 ports). Use the same setting for any port channels on the physical switches. For more information, see the Cisco Nexus 1000V Interface Configuration Guide.
  • VXLAN uses MAC in IP (UDP) with a destination port of 8472. You must allow this port through any intermediate firewall.
  • If you are using the VXLAN multicast mode, you must configure an IGMP querier in the VXLAN transport VLANs.

Guidelines and Limitations for VXLANs

VXLAN has the following configuration guidelines and limitations:

  • VXLANs in unicast-only mode are supported only between VEMs that are managed by a single VSM. A VXLAN in unicast-only mode cannot be shared across two different distributed virtual switches.
  • When a VXLAN is configured in the unicast-only mode with MAC distribution enabled, the VXLAN gateway does not register any MAC addresses that it learns on the VLAN side. If these MAC addresses have not been learned yet, the traffic to these MAC addresses is delivered by replicating of unknown unicast packets to the VXLAN gateway. This is the only scenario where unknown unicast packets are replicated in the MAC distribution mode.
  • Microsoft Network Load Balancing (NLB) servers in unicast mode require unknown unicast packets to be delivered to all the server ports, because the shared MAC address of the NLB servers is never discovered. This solution will break the unknown unicast semantics of unicast-only mode with MAC distribution. We recommend that you use either multicast mode or unicast-only mode without MAC distribution.
  • You cannot enable the MAC distribution mode and the multi-MAC capability feature together. You must use either the MAC distribution or the muti-MAC capability feature.
  • The Cisco Nexus 1000V switch in ESXi 5.5 supports VXLAN offload NICs. The Cisco Nexus 1000V switch is designed to assume that either all or none of the physical NICs (PNICs) in a port channel support the VXLAN offload capability.

VXLAN has the following configuration guidelines and limitations for changing the VXLAN configuration:

  • Use the segment mode unicast-only command to change the global configuration mode from multicast to unicast. This command affects all bridge domains with no overrides.
  • You can use multicast or unicast mode if you override the global configuration for the bridge domain by entering the segment mode unicast-only or no segment mode unicast-only commands.
  • You can enable the segment distribution MAC command only after entering the segment mode unicast-only command.
  • You can disable the segment distribution MAC address configuration globally by entering the no segment distribution mac command.
  • You cannot use the no segment mode unicast-only command if you already entered the segment distribution MAC command.
  • You must configure a multicast IP address that is required for a VXLAN that is in the multicast mode.
  • If you remove the multicast IP address while VXLAN is in the multicast mode, the ports that use that VXLAN go to the inactive state.

Note


Ports become inactive if you change the mode from unicast to multicast if a multicast IP address is not configured or a segment ID is removed.


Default Settings for VXLANs

The following table lists the default settings for VXLAN parameters.

Table 1 Default VXLAN Parameters

Parameter

Default

Feature Segmentation

Disabled

Configuring VXLANs

Initial Enabling of VXLANs

To enable a VXLAN, you must perform the following two procedures when you first configure a VXLAN.

Enabling VXLANs

Before You Begin

Enter the show system vem feature level command to confirm that the feature level is 4.2(1)SV1(5.1) or a later release. If the feature level is not 4.2(1)SV1(5.1) or a later release, see the Cisco Nexus 1000V Installation and Upgrade Guide.

Procedure
     Command or ActionPurpose
    Step 1switch# configure terminal 

    Enters global configuration mode.

     
    Step 2switch(config)# feature segmentation 

    Enables the VXLAN.

     
    Step 3switch(config)# show feature | grep segmentation   (Optional)

    Displays whether the VXLAN is enabled.

     
    Step 4 switch (config)# [no] segment-mode unicast-only  

    Configures the global configuration mode for all VXLAN bridge domains. If the configuration mode is not entered, the global mode is unicast-only mode without MAC distribution.

     
    Step 5 switch (config)# [no] segment distribution mac  

    Enables or disables MAC distribution globally. All bridge domains with default MAC distribution mode will inherit this configuration and enable or disable MAC distribution.

     
    Step 6switch(config)# copy running-config startup-config  (Optional)

    Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

     

    This example shows how to enable the segmentation feature:

    switch# configure terminal 
    switch(config)# feature segmentation 
    switch(config)# show feature | grep segmentation 
    network-segmentation 1 disabled 
    segmentation         1 enabled 
    switch(config)# copy running-config startup-config 

    Configuring vmknics for VXLAN Encapsulation

    Before You Begin
    • Identify a VLAN to be used for transporting VXLAN-encapsulated traffic.
    • Ensure that it is configured on the uplink port profile for all VEMs on which the VXLAN can be configured.
    Procedure
       Command or ActionPurpose
      Step 1switch# configure terminal 

      Enters global configuration mode.

       
      Step 2switch(config)# port-profile type veth profilename  

      Enters port profile configuration mode for the named port profile. If the port profile does not already exist, it is created using the following characteristics:

      • profilename—The port profile name can be up to 80 characters and must be unique for each port profile on the Cisco Nexus 1000V.
      Note   

      If a port profile is configured as an Ethernet type, it cannot be used to configure VMware virtual ports.

       
      Step 3switch(config-port-prof)# vmware port-group name  

      Designates the port profile as a VMware port group.

      The port profile is mapped to a VMware port group of the same name unless you specify a name here. When you connect the VSM to vCenter Server, the port group is distributed to the virtual switch on vCenter Server.

       
      Step 4switch(config-port-prof)# switchport mode access 

      Designates the interfaces as switch access ports (the default).

       
      Step 5switch(config-port-prof)# switchport access vlan id  

      Assigns a VLAN ID to this port profile.

      Note   

      A VLAN ID must be created and should be in the active state.

       
      Step 6switch(config-port-prof)# capability vxlan  

      Assigns the VXLAN capability to the port profile to ensure that the interfaces that inherit this port profile are used as sources for VXLAN-encapsulated traffic.

       
      Step 7switch(config-port-prof)# no shutdown  

      Administratively enables all ports in the profile.

       
      Step 8switch(config-port-prof)# state enabled  

      Sets the operational state of a port profile.

       
      Step 9switch(config-port-prof)# show port-profile name profilename  

      Displays the port profile configuration.

       
      Step 10switch(config-port-prof)# copy running-config startup-config  (Optional)

      Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

       

      This example shows how to configure a vmknic for VXLAN encapsulation:

      switch# configure terminal
      switch(config)# port-profile type veth vmknic-pp
      switch(config-port-prof)# vmware port-group 
      switch(config-port-prof)# switchport mode access
      switch(config-port-prof)# switchport access vlan 100 
      switch(config-port-prof)# capability vxlan
      switch(config-port-prof)# no shutdown
      switch(config-port-prof)# state enabled
      switch(config-port-prof)# show port-profile name vmknic-pp
      port-profile vmknic-pp
      type: Vethernet
      description:
      status: enabled
      max-ports: 32
      min-ports: 1
      inherit:
      config attributes:
      switchport mode access
      switchport access vlan 100
      capability vxlan
      no shutdown
      evaluated config attributes:
      switchport mode access
      switchport access vlan 100
      capability vxlan
      no shutdown
      assigned interfaces:
      port-group: vmknic-pp
      system vlans: none
      capability l3control: no
      capability iscsi-multipath: no
      capability vxlan: yes
      capability l3-vservice: no
      port-profile role: none
      port-binding: static
      
      switch(config-port-prof)# 
      switch(config-port-prof)# copy running-config startup-config
      What to Do Next

      The vSphere administrator must create a new vmknic on each ESX/ESXi host and assign the previously created port profile to this vmknic. IP address and netmask should be assigned to the vmknic. This IP address will be used for VXLAN packet encapsulation. Use the show module vteps to view the interfaces on the VSM.

      Creating a Bridge Domain

      You are limited to creating a maximum of 2048 VXLAN bridge domains.

      Procedure
         Command or ActionPurpose
        Step 1switch# configure terminal 

        Enters global configuration mode.

         
        Step 2switch(config)# bridge-domain name-string 

        Creates a VXLAN and associates an identifying name to it.

         
        Step 3switch(config-bd)# segment id [number]  

        Specifies the VXLAN segment ID. Only one bridge domain can use a particular segment ID value.

        Valid values are from 4096 to 16000000. (1 to 4095 are reserved for VLANs.)

         
        Step 4switch(config-bd)# group ipaddr   (Optional)

        Associates the multicast group for broadcasts and floods.

        Note   

        Reserved multicast addresses are not allowed.

         
        Step 5switch(config-bd)# show bridge-domain name-string   (Optional)

        Displays bridge domain information.

         
        Step 6switch(config-bd)# copy running-config startup-config  (Optional)

        Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

         

        This example shows how to create a VXLAN:

        switch# configure terminal
        switch(config)# bridge-domain tenant-red
        switch(config-bd)# segment id 4096
        switch(config-bd)# group 239.1.1.1
        switch(config-bd)# show bridge-domain tenant-red 
        Bridge-domain tenant-red (0 ports in all)
        Segment ID: NULL
        Mode: Unicast-only (default)
        MAC Distribution: Disable (default)
        Group IP: 239.1.1.1
        State: UP Mac learning: Enabled
        switch(config-bd)#
        switch(config-bd)# copy running-config startup-config

        Configuring the Bridge Domain Mode

        You can configure a bridge domain in the bridge-domain mode or global mode.

        Procedure
           Command or ActionPurpose
          Step 1 switch# configure terminal 

          Enters global configuration mode.

           
          Step 2switch# bridge-domainbd-name 

          Creates a bridge domain.

           
          Step 3switch (config-bd)# [no] segment mode unicast-only | default segment mode 

          Configures the segment mode as unicast only.

          The mode can be configured globally or for a specific bridge domain. When configured under a specific bridge domain, the mode is treated as an override to the global configuration for that specific bridge domain. Any change in the global configuration affects all the bridge domains that do not have overrides. The mode configuration on a specific bridge domain overwrites the global bridge domain. The overrides configured on the bridge domain can be removed by using the default segment mode.

          Note   

          Use the no segment mode unicast-only command to override the configuration under a bridge domain. If you have unicast enabled globally, the bridge domain can use the multicast mode. To override, use the default segment mode command.

          Note   

          This command cannot be performed globally or under a bridge domain if the segment distribution MAC feature is configured.

           
          Step 4switch (config-bd)# [no] segment distribution mac | default segment distribution mac 

          Enables MAC distribution for the bridge domain.

          Note   

          To configure an override under a bridge domain, you must enter the segment mode unicast-only command as an override first.

           

          This example shows how to configure a bridge domain:


          Note


          The ports are inactive if a segment ID is not configured for a bridge domain and if a multicast IP address is not configured when global configuration or a bridge domain override has the no segment mode unicast-only configuration.


           config terminal
           bridge-domain domain-660
               segment mode unicast-only 
               segment distribution mac
               

          Creating a Port Profile Configured to Use a VXLAN

          Alternatively, you can associate ports with a bridge domain by modifying the configuration of an existing virtual Ethernet port profile to use VXLANs instead of VLANs. To do so, enter the switchport access bridge-domain name command on a profile with switchport mode access configured.

          Procedure
             Command or ActionPurpose
            Step 1switch# configure terminal 

            Enters global configuration mode.

             
            Step 2switch(config)# port-profile [type {vethernet}] name 

            Enters port profile configuration mode for the named port profile. If the port profile does not already exist, it is created using the following characteristics:

            • name—The port profile name can be up to 80 characters and must be unique for each port profile on the Cisco Nexus 1000V.
            • type—The port profile type is virtual Ethernet. Once configured, the type cannot be changed. The default is the virtual Ethernet type. Defining a port profile type as Ethernet allows the port profile to be used for physical (Ethernet) ports. In vCenter Server, the corresponding port group can be selected and assigned to physical ports (PNICs).
              Note   

              If a port profile is configured as an Ethernet type, it cannot be used to configure VMware virtual ports.

             
            Step 3switch(config-port-prof)# vmware port-group [pg_name]  

            Designates the port profile as a VMware port group.

            The port profile is mapped to a VMware port group of the same name unless you specify a name here. When you connect the VSM to vCenter Server, the port group is distributed to the virtual switch on vCenter Server.

             
            Step 4switch(config-port-prof)# switchport mode access  

            Designates that the interfaces are to be used as trunking ports.

            A trunk port transmits untagged packets for the native VLAN and transmits encapsulated, tagged packets for all other VLANs.

             
            Step 5switch(config-port-prof)# switchport access bridge-domain <bridge-domain name>  

            Assigns a VXLAN bridge domain to this port profile.

            You must configure the bridge domain with its segment ID for the port to be active. You should configure a multicast IP address if you prefer multicast mode. The multicast mode is displayed in the running configuration as no segment mode unicast-only.

             
            Step 6switch(config-port-prof)# no shutdown  

            Administratively enables all ports in the profile.

             
            Step 7switch(config-port-prof)# state enabled 

            Sets the operational state of a port profile.

             
            Step 8switch(config-port-prof)# show port-profile [brief | expand-interface | usage] [name profile-name]  (Optional)

            Displays the configuration for verification.

             
            Step 9switch(config-port-prof)# show running-config bridge-domain   (Optional)

            Displays the segmentation configuration.

             
            Step 10switch(config-port-prof)# copy running-config startup-config  (Optional)

            Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

             

            This example shows how to create a port profile configured to use a VXLAN:

            switch# configure terminal
            switch(config)# port-profile tenant-profile 
            switch(config-port-prof)# vmware port-group 
            switch(config-port-prof)# switchport mode access
            switch(config-port-prof)# switchport access bridge-domain tenant-red
            switch(config-port-prof)# no shutdown 
            switch(config-port-prof)# state enabled 
            switch(config-port-prof)# show port-profile name tenant-profile
            port-profile tenant-profile
            type: Vethernet
            description:
            status: enabled
            max-ports: 32
            min-ports: 1
            inherit:
            config attributes:
            switchport mode access
            switchport access bridge-domain tenant-red
            no shutdown
            evaluated config attributes:
            switchport mode access
            switchport access bridge-domain tenant-red
            no shutdown
            assigned interfaces:
            port-group: tenant-profile
            system vlans: none
            capability l3control: no
            capability iscsi-multipath: no
            capability vxlan: no
            capability l3-vservice: no
            port-profile role: none
            port-binding: static
            
            switch(config-port-prof)# 
            switch(config-port-prof)# show running-config bridge-domain
            switch(config-port-prof)# copy running-config startup-config

            Removing Ports from a VXLAN

            By performing this procedure, you move the ports to the default VLAN.

            Procedure
               Command or ActionPurpose
              Step 1switch# configure terminal 

              Enters global configuration mode.

               
              Step 2switch(config)# port-profile [type {vethernet}] name 

              Enters port profile configuration mode for the named port profile. If the port profile does not already exist, it is created using the following characteristics:

              • name—The port profile name can be up to 80 characters and must be unique for each port profile on the Cisco Nexus 1000V.
              • type—The port profile type is vEthernet. Once configured, the type cannot be changed. The default is the vEthernet type. Defining a port profile type as Ethernet allows the port profile to be used for physical (Ethernet) ports. In vCenter Server, the corresponding port group can be selected and assigned to physical ports (PNICs).
                Note   

                If a port profile is configured as an Ethernet type, it cannot be used to configure VMware virtual ports.

               
              Step 3switch(config-port-prof)# no switchport access bridge-domain  

              Removes the VXLAN bridge domain from this port profile.

               
              Step 4switch(config-port-prof)# show port-profile usage  (Optional)

              Displays a list of interfaces that inherited a port profile.

               
              Step 5switch(config-port-prof)# show bridge-domain   (Optional)

              Displays all bridge domains.

               
              Step 6switch(config-port-prof)# copy running-config startup-config  (Optional)

              Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

               

              This example shows how to remove ports from a VXLAN:

              switch# configure terminal
              switch(config)# port-profile tenant-profile 
              switch(config-port-prof)# no switchport access bridge-domain tenant-red 
              switch(config-port-prof)# show port-profile usage
              switch(config-port-prof)# show bridge-domain 
              switch(config-port-prof)# copy running-config startup-config

              Deleting a VXLAN

              When you delete an existing bridge domain with ports on it, all the ports are moved to a down state and traffic stops flowing.

              Procedure
                 Command or ActionPurpose
                Step 1switch# configure terminal 

                Enters global configuration mode.

                 
                Step 2switch(config)# no bridge-domain group-red 

                Deletes a VXLAN.

                 
                Step 3switch(config-bd)# show bridge-domain  (Optional)

                Displays all bridge domains.

                 
                Step 4switch(config-bd)# copy running-config startup-config  (Optional)

                Copies the running configuration to the startup configuration.

                 

                This example shows how to delete a VXLAN:

                switch# configure terminal
                switch(config)# no bridge-domain group-red
                switch(config)# show bridge-domain
                switch(config)# copy running-config startup-config

                Disabling Segmentation

                Procedure
                   Command or ActionPurpose
                  Step 1switch# configure terminal 

                  Enters global configuration mode.

                   
                  Step 2switch(config)# show bridge-domain 

                  Displays all bridge domains.

                  Note   

                  You must identify all bridge domains with nonzero port counts.

                   
                  Step 3switch(config)# show running port-profile  (Optional)

                  Displays the running configuration for all port profiles.

                  Note   

                  You must use this command to identify which port profiles have bridge domains identified in Step 2 configured.

                   
                  Step 4switch(config)# port-profile name 

                  Names the port profile and enters port profile configuration mode. If the port profile does not already exist, it is created using the following characteristics:

                  name—The port profile name can be up to 80 characters and must be unique for each port profile on the Cisco Nexus 1000V.

                  Note   

                  If a port profile is configured as an Ethernet type, it cannot be used to configure VMware virtual ports.

                   
                  Step 5switch(config-port-prof)# no switchport access bridge-domain name-string  

                  Removes the VXLAN bridge domain from this port profile and moves the ports to VLAN1.

                   
                  Step 6switch(config-port-prof)# show port-profile usage  (Optional)

                  Displays a list of interfaces that inherited a port profile.

                   
                  Step 7switch(config-port-prof)# show bridge-domain   (Optional)

                  Displays all bridge domains.

                   
                  Step 8switch(config-port-prof)# no feature segmentation  

                  Removes the segmentation feature.

                   
                  Step 9switch(config-port-prof)# show feature | grep segmentation   (Optional)

                  Displays if the segmentation feature is running or not running.

                   
                  Step 10switch(config-port-prof)# copy running-config startup-config  (Optional)

                  Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

                   

                  This example shows how to disable segmentation:

                  switch# configure terminal
                  switch(config)# show bridge-domain
                  
                  Global Configuration:
                  Mode: Unicast-only
                  MAC Distribution: Disable
                  
                  Bridge-domain tenant-red (4 ports in all)
                  Segment ID: 4096 (Manual/Active)
                  Mode: Unicast-only
                  MAC Distribution: Disable
                  Group IP: NULL
                  State: UP Mac learning: Enabled
                  Veth1, Veth2, Veth4, Veth11
                  
                  switch(config)# show running-config port-profile
                  port-profile default max-ports 32
                  port-profile default port-binding static
                  port-profile type ethernet Unused_Or_Quarantine_Uplink
                  vmware port-group
                  shutdown
                  description Port-group created for Nexus1000V internal usage. Do not use.
                  state enabled
                  port-profile type vethernet Unused_Or_Quarantine_Veth
                  vmware port-group
                  shutdown
                  description Port-group created for Nexus1000V internal usage. Do not use.
                  state enabled
                  port-profile type vethernet tenant-profile
                  vmware port-group
                  switchport mode access
                  switchport access bridge-domain tenant-red
                  no shutdown
                  state enabled
                  
                  switch(config)#
                  switch(config-port-prof)# show port-profile usage
                  
                  port-profile Unused_Or_Quarantine_Uplink
                  
                  port-profile Unused_Or_Quarantine_Veth
                  
                  port-profile tenant-profile
                  Vethernet1
                  Vethernet2
                  Vethernet4
                  Vethernet11
                  
                  switch(config-port-prof)# show bridge-domain
                  
                  Global Configuration:
                  Mode: Unicast-only
                  MAC Distribution: Disable
                  
                  Bridge-domain tenant-red (0 ports in all)
                  Segment ID: 4096 (Manual/Active)
                  Mode: Unicast-only
                  MAC Distribution: Disable
                  Group IP: NULL
                  State: UP Mac learning: Enabled
                  
                  switch(config-port-prof)#
                  switch(config-port-prof)# no feature segmentation
                  switch(config-port-prof)# 2013 May 23 05:34:42 switch-cy %SEG_BD-2-SEG_BD_DISABLED: Feature Segmentation disabled
                  
                  switch(config-port-prof)# show feature | grep seg_bd
                  - NR - 1 - seg_bd
                  
                  

                  Verifying the VXLAN Configuration

                  To display the VXLAN configuration information, perform one of the following tasks:

                  Command

                  Purpose

                  show feature | grep segmentation

                  Displays if the segmentation feature is running.

                  show bridge-domain

                  Displays all bridge domains with the mode.

                  show bridge-domain vteps

                  Displays the bridge domain-to-VTEP mappings that are maintained by the VSM and are pushed to all VEMs.

                  show bridge-domain mac bd-name

                  Displays all the MAC addresses that are learned by the VSMs on VXLANs that are configured with the MAC distribution feature.

                  show run bridge-domain

                  Displays the running bridge domain.

                  show bridge-domain bd-name

                  Displays the specified bridge domain.

                  show bridge-domain bd-name vteps

                  Displays the specific bridge domain-to-VTEP mappings that are maintained by the VSM and are pushed to all VEMs.

                  show interface brief

                  Displays a short version of the interface configuration.

                  show interface switchport

                  Displays information about switchport interfaces.

                  show module vteps

                  Displays the IP addresses available on each module that can be used for VXLAN Tunnel Endpoints.

                  Feature History for VXLAN

                  Feature Name

                  Releases

                  Feature Information

                  Enhanced VXLAN

                  4.2(1)SV2(2.1)

                  Added the enhanced VXLAN commands.

                  VXLAN

                  4.2(1)SV1(5.1)

                  Introduced the Virtual Extensible Local Area Network (VXLAN) feature.