Configuring VXLANs

This chapter contains the following sections:

Prerequisites for VXLANs

VXLANs have the following prerequisites:

  • You must create a VTEP on the host by defining it in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface during the Openstack deployment. For more information, see the Cisco Nexus 1000V for KVM Software Installation Guide.

  • If you plan to configure multiple VTEPs in virtual port channel host mode (vPC-HM) for load balancing in the same subnet, you need to set the vteps_in_the_same_subnet parameter to true in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface before installing the Cisco Nexus 1000V for KVM. For more information, see the Cisco Nexus 1000V for KVM Software Installation Guide.

  • The Cisco Nexus 1000V uplink port profiles and all interconnecting switches and routers between the KVM hosts must have their supported maximum transmission unit (MTU) set to at least 50 bytes larger than the MTU of the Virtual Machines (VMs). For example, the VMs default to using a 1500 byte MTU (same as the uplinks and physical devices), so you must set them to at least 1550 bytes. If this configuration is not possible, you should lower all VM vNICs MTU to 50 bytes smaller than what the physical network supports, such as 1450 bytes. For more information, see the Cisco Nexus 1000V Port Profile Configuration Guide.

  • If the Cisco Nexus 1000V is using a port channel for its uplinks, you should set the load distribution algorithm to a 5-tuple hash (IP/Layer 4/Layer 4 ports). Use the same setting for any port channels on the physical switches. For more information, see the Cisco Nexus 1000V Interface Configuration Guide.

  • By default, VXLAN uses MAC in IP (UDP) with a destination port of 8472. However, you can change this setting to the IANA assigned value of 4789 or any value between 1024 through 65535. Whichever port you use, you must allow it through any intermediate firewall.

  • If you are using the VXLAN multicast mode, you must configure an IGMP querier in the VXLAN transport VLANs.

Guidelines and Limitations for VXLANs

VXLAN has the following configuration guidelines and limitations:

  • You must configure and make all changes to VXLANs in OpenStack.

    You must consistently use OpenStack for all VM network, subnet, and port configurations. If you create VM networks, subnets, and ports directly on the VSM, the configuration is lost when the OpenStack synchronization occurs.

  • When encapsulated traffic is destined to a VEM that is connected to a different subnet, the VEM does not use the Linux host routing table. Instead, it can use either Proxy Address Resolution Protocol (ARP) or a default gateway.

    • To use Proxy ARP, you must configure the upstream router for Proxy ARP. With ARP configured, if the remote VTEP is in the same subnet as the VXLAN Gateway, the VEM uses ARP to obtain the IP address of the remote VTEP. If the remote VTEP is in a different subnet than the VXLAN Gateway, the VEM uses ARP to obtain the IP address of the VXLAN Gateway.

    • To use a default gateway, you must configure the VTEP with the transport ip address external command to specify the netmask and gateway IP address for the VTEP to use. For example, from the interface command mode, enter transport ip address external netmask 255.255.255.0 gateway 1.2.3.4.

  • If you configure load-balancing with a VPC-HM where multiple VTEPS exist in the same subnet on the KVM platform, you might experience a Linux kernel issue where ARP responses from the Linux kernel for the VTEPs might have the wrong MAC address. This situation could adversely affect the flow of VXLAN traffic.

  • VXLANs in unicast-only mode are supported only between VTEPs that are managed by a single VSM. A VXLAN in unicast-only mode cannot be shared across two different distributed virtual switches.

Default Settings for VXLANs

The following table lists the default settings for VXLAN parameters.

Table 1 Default VXLAN Parameters

Parameter

Default

Feature Segmentation

Enabled

Configuring VXLANs

Steps to Configure VXLANs

You can configure a VXLAN using the OpenStack CLI or Horizon dashboard.


Note

You must consistently use OpenStack for all VM network, subnet, and port configurations. If you create VM networks, subnets, and ports directly on the VSM, the configuration is lost when the OpenStack synchronization occurs.

Before You Begin
Procedure
     Command or ActionPurpose
    Step 1 Enable the Segmentation feature in the VSM using the CLI.
     

    See Enabling the VXLAN Segmentation Feature.

     
    Step 2 Ensure that the VTEP on the host has been defined in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface during the OpenStack deployment.
     

    See Cisco Nexus 1000V for KVM Software Installation Guide.

     
    Step 3 If you plan to configure multiple VTEPs in virtual port channel host mode (vPC-HM) for load balancing in the same subnet, you need to set the vteps_in_the_same_subnet parameter to true in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface before installing the Cisco Nexus 1000V for KVM.
          		(Optional)

    See the Cisco Nexus 1000V for KVM Software Installation Guide.

     
    Step 4Using the VSM CLI, configure a vEthernet port profile with VXLAN capability.    
    Step 5Using the OpenStack CLI, create a multicast or unicast VXLAN network profile.

    Example:neutron cisco-network-profile-create name vxlan --subtype multicast --segment_range segment-range --multicast_ip_range ip-range

    Example:neutron cisco-network-profile-create name vxlan --subtype unicast --segment_range segment-range  

    For more information or to perform these steps using the OpenStack Horizon Dashboard, see the Cisco Nexus 1000V for KVM Virtual Network Configuration Guide.

     
    Step 6Using the OpenStack CLI, create a network and associate it with a Cisco Nexus 1000V switch network profile.

    Example:neutron net-create name --n1kv:profile_id profileId  

    For more information or to perform these steps using the OpenStack Horizon Dashboard, see the Cisco Nexus 1000V for KVM Virtual Network Configuration Guide.

     
    Step 7Using the VSM CLI, verify that the VXLAN has been configured on the VSM.

    Example:show bridge-domain brief       		This command lists all bridge domains and their corresponding status and ports.  
    Step 8Save the configuration on the VSM.

    Example:copy running-config startup-config       		—  

    Enabling the VXLAN Segmentation Feature

    If you have installed the Cisco Nexus 1000V for KVM on a VM, the segmentation feature is enabled by default. However, if you have installed the Cisco Nexus 1000V for KVM on a Cloud Services Platform, you must enable the segmentation feature.

    Procedure
       Command or ActionPurpose
      Step 1switch# configure terminal  

      Enters global configuration mode.

       
      Step 2switch(config)# show feature | grep segmentation   (Optional)

      Displays whether the VXLAN feature is enabled.

       
      Step 3switch(config)# feature segmentation 

      Enables the VXLAN segmentation feature.

       
      Step 4switch(config)# show feature | grep segmentation   (Optional)

      Displays whether the VXLAN feature is enabled.

       
      Step 5switch(config)# copy running-config startup-config   (Optional)

      Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

       

      This example shows how to enable the VXLAN segmentation feature:

      switch# configure terminal 
switch(config)# show feature | grep segmentation
network-segmentation 1 disabled 
segmentation         1 disabled 
switch(config)# feature segmentation 
switch(config)# show feature | grep segmentation 
network-segmentation 1 disabled 
segmentation         1 enabled 
switch(config)# copy running-config startup-config

      Configuring a VTEP Profile for VXLAN Encapsulation

      Before You Begin

      • Identify a VLAN to be used for transporting VXLAN-encapsulated traffic.

      • Ensure that the VLAN is configured on the uplink port profile for all VEMs on which the VXLAN can be configured.

      • Create the VTEP on the host by defining it in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface during the OpenStack deployment. For details, see the Cisco Nexus 1000V for KVM Software Installation Guide.

      Procedure
         Command or ActionPurpose
        Step 1switch# configure terminal  

        Enters global configuration mode.

         
        Step 2switch(config)# port-profile type veth profilename  

        Enters port profile configuration mode for the named port profile. If the port profile does not already exist, it is created using the following characteristics:

        • profilename—The port profile name can be up to 80 characters and must be unique for each port profile on the Cisco Nexus 1000V.

        Note   

        If a port profile is configured as an Ethernet type, it cannot be used to configure VTEPs.

         
        Step 3switch(config-port-prof)# switchport mode access  

        Designates the interfaces as switch access ports (the default).

         
        Step 4switch(config-port-prof)# switchport access vlan id  

        Assigns a VLAN ID to this port profile.

        Note   

        A VLAN ID must be created and should be in the active state.

         
        Step 5switch(config-port-prof)# capability vxlan  

        Assigns the VXLAN capability to the port profile to ensure that the interfaces that inherit this port profile are used as sources for VXLAN-encapsulated traffic.

         
        Step 6switch(config-port-prof)# transport ip address external netmask netmask [gateway gw-ip]   (Optional)

        Configures the VTEP with the netmask and gateway IP address to use to reach a VEM that is connected to a different subnet. Alternatively, you can configure the default router for Proxy ARP. For more information, see Guidelines and Limitations for VXLANs.

        Note   

        You must create the VTEP on the host by defining it in the Red Hat Enterprise Linux OpenStack Platform Installer graphical user interface during the OpenStack deployment. For details, see the Cisco Nexus 1000V for KVM Software Installation Guide.

         
        Step 7switch(config-port-prof)# no shutdown  

        Administratively enables all ports in the profile.

         
        Step 8switch(config-port-prof)# state enabled  

        Sets the operational state of a port profile.

         
        Step 9switch(config-port-prof)# publish port-profile  

        Pushes the port profile to the OpenStack controller.

         
        Step 10switch(config-port-prof)# show port-profile name profilename  

        Displays the port profile configuration.

         
        Step 11switch(config-port-prof)# copy running-config startup-config   (Optional)

        Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

         

        This example shows how to configure an interface for VXLAN encapsulation: 

        switch# configure terminal
switch(config)# port-profile type veth vxlan-pp
switch(config-port-prof)# switchport mode access
switch(config-port-prof)# switchport access vlan 100 
switch(config-port-prof)# capability vxlan
switch(config-port-prof)# no shutdown
switch(config-port-prof)# state enabled
switch(config-port-prof)# publish port-profile
switch(config-port-prof)# show port-profile name vxlan-pp
port-profile vxlan-pp
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
switchport mode access
switchport access vlan 100
capability vxlan
no shutdown
evaluated config attributes:
switchport mode access
switchport access vlan 100
capability vxlan
no shutdown
assigned interfaces:
port-group: vmknic-pp
system vlans: none
capability l3control: no
capability iscsi-multipath: no
capability vxlan: yes
capability l3-vservice: no
port-profile role: none
port-binding: static

switch(config-port-prof)# 
switch(config-port-prof)# copy running-config startup-config

        Changing the UDP Port for VXLAN Encapsulation

        You can change the default UDP port number to another port number.

        Procedure
           Command or ActionPurpose
          Step 1switch# configure terminal  

          Enters global configuration mode.

           
          Step 2switch(config)# vxlan udp port port-number 

          Changes the UDP port to the specified port number. The default UDP port number is 8472. Valid port numbers are in the range 1024 to 65535.

           
          Step 3switch(config)# show running-config | inc "vxlan udp"  Displays the VXLAN UDP port number. 
          Step 4switch(config)# copy running-config startup-config   (Optional)

          Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

           
          This example shows how to change the UDP port to 4789: 
          switch# configure terminal
switch(config)# vxlan udp port 4789
switch(config)# show running-config | inc "vxlan udp"
vxlan udp port 5656
switch(config)# copy running-config startup-config

          Disabling the VXLAN Segmentation Feature

          If you have enabled the segmentation feature on a Cloud Services Platform, you can disable it. If you have installed the Cisco Nexus 1000V for KVM on a VM, the feature is enabled by default and cannot be disabled.

          Procedure
             Command or ActionPurpose
            Step 1switch# configure terminal  

            Enters global configuration mode.

             
            Step 2switch(config)# show bridge-domain 

            Displays all bridge domains.

            Note   

            You must identify all bridge domains with nonzero port counts.

             
            Step 3switch(config)# show running port-profile  (Optional)

            Displays the running configuration for all port profiles.

            Note   

            You must use this command to identify which port profiles have bridge domains identified in Step 2 configured.

             
            Step 4switch(config)# port-profile name 

            Enters configuration mode for the specified port profile.

             
            Step 5switch(config-port-prof)# no switchport access bridge-domain name-string  

            Removes the VXLAN bridge domain from the port profile and moves the ports to VLAN1.

             
            Step 6switch(config-port-prof)# show port-profile usage  (Optional)

            Displays a list of interfaces that inherited a port profile.

             
            Step 7switch(config-port-prof)# show bridge-domain   (Optional)

            Displays all bridge domains.

             
            Step 8switch(config-port-prof)# no feature segmentation  

            Removes the segmentation feature.

             
            Step 9switch(config-port-prof)# show feature | grep segmentation   (Optional)

            Displays if the segmentation feature is running or not running.

             
            Step 10switch(config-port-prof)# copy running-config startup-config   (Optional)

            Saves the change persistently through reboots and restarts by copying the running configuration to the startup configuration.

             

            This example shows how to disable segmentation:

            switch# configure terminal
switch(config)# show bridge-domain

Global Configuration:
Mode: Unicast-only
MAC Distribution: Disable

Bridge-domain tenant-red (4 ports in all)
Segment ID: 4096 (Manual/Active)
Mode: Unicast-only
MAC Distribution: Disable
Group IP: NULL
State: UP Mac learning: Enabled
Veth1, Veth2, Veth4, Veth11

switch(config)# show running-config port-profile
port-profile default max-ports 32
port-profile default port-binding static
port-profile type ethernet Unused_Or_Quarantine_Uplink
vmware port-group
shutdown
description Port-group created for Nexus1000V internal usage. Do not use.
state enabled
port-profile type vethernet Unused_Or_Quarantine_Veth
vmware port-group
shutdown
description Port-group created for Nexus1000V internal usage. Do not use.
state enabled
port-profile type vethernet tenant-profile
vmware port-group
switchport mode access
switchport access bridge-domain tenant-red
no shutdown
state enabled

switch(config)#
switch(config-port-prof)# show port-profile usage

port-profile Unused_Or_Quarantine_Uplink

port-profile Unused_Or_Quarantine_Veth

port-profile tenant-profile
Vethernet1
Vethernet2
Vethernet4
Vethernet11

switch(config-port-prof)# show bridge-domain

Global Configuration:
Mode: Unicast-only
MAC Distribution: Disable

Bridge-domain tenant-red (0 ports in all)
Segment ID: 4096 (Manual/Active)
Mode: Unicast-only
MAC Distribution: Disable
Group IP: NULL
State: UP Mac learning: Enabled

switch(config-port-prof)#
switch(config-port-prof)# no feature segmentation
switch(config-port-prof)# 2013 May 23 05:34:42 switch-cy %SEG_BD-2-SEG_BD_DISABLED: Feature Segmentation disabled

switch(config-port-prof)# show feature | grep seg_bd
- NR - 1 - seg_bd

            Verifying the VXLAN Configuration

            You create a bridge domain on the VSM when you create a VXLAN network on the OpenStack controller. For more information, see the Cisco Nexus 1000V for KVM Virtual Network Configuration Guide.

            To display the VXLAN configuration information, perform one of the following tasks:

            Command

            Purpose

            show feature | grep segmentation

            Displays if the segmentation feature is running.

            show bridge-domain

            Displays all bridge domains with the mode.

            show bridge-domain brief

            Lists all bridge domains and their corresponding status and ports.

            show bridge-domain vteps

            Displays the bridge domain-to-VTEP mappings that are maintained by the VSM and are pushed to all VEMs.

            show run bridge-domain

            Displays the running bridge domain.

            show bridge-domain bd-name

            Displays the specified bridge domain.

            show bridge-domain bd-name vteps

            Displays the specific bridge domain-to-VTEP mappings that are maintained by the VSM and are pushed to all VEMs.

            show interface brief

            Displays a short version of the interface configuration.

            show interface switchport

            Displays information about switchport interfaces.

            show module vteps

            Displays the IP addresses available on each module that can be used for VXLAN Tunnel Endpoints.

            Feature History for VXLAN

            Feature Name

            Releases

            Feature Information

            VXLAN

            Release 5.2(1)SK1(2.1)

            Introduced the Virtual Extensible Local Area Network (VXLAN) feature, including the enhanced VXLAN commands.