VXLAN has the following guidelines and limitations:

Switch or port restrictions

• FEX ports do not support IGMP snooping on VXLAN VLANs.

• The VXLAN UDP port number is used for VXLAN encapsulation. For Cisco Nexus NX-OS, the UDP port number is 4789. It complies with IETF standards and is not configurable.

• Cisco Nexus 9300 Series switches with 100G uplinks only support VXLAN switching/bridging.

  Cisco Nexus 9300-FX, and Cisco Nexus 9300-FX2 platform switches do not have this restriction.

  Note	For VXLAN routing support, a 40G uplink module is required.

• Beginning with Cisco NX-OS Release 10.2(3)F, VXLAN can coexist with the GRE tunnel feature or the MPLS (static or segment-routing) feature.

• Native VLANs are supported as transit traffic over a VXLAN fabric on Cisco Nexus 9300-FX/FX2/FX3/GX/GX2/H2R/H1 Series switches and 9800 Series switches.

• A FEX HIF (FEX host interface port) is supported for a VLAN that is extended with VXLAN.

• Bind NVE to a loopback address that is separate from other loopback addresses that are required by Layer 3 protocols. A best practice is to use a dedicated loopback address for VXLAN. This best practice should be applied not only for the vPC VXLAN deployment, but for all VXLAN deployments.

• Tenant VRF (VRF with VNI on it) cannot be used on an SVI that has no VNI binding into it (underlay infra VRF).

• For traceroute through a VXLAN fabric when using L3VNI, the following scenario is the expected behavior:

  If L3VNI is associated with a VRF and an SVI, the associated SVI does not have an L3 address that is configured but instead has the "ip forward" configuration command. Due to this interface setup it cannot respond back to the traceroute with its own SVI address. Instead, when a traceroute involving the L3VNI is run through the fabric, the IP address reported will be the lowest IP address of an SVI that belongs to the corresponding tenant VRF.

• In an ingress replication vPC setup, Layer 3 connectivity is needed between vPC peer devices.

VXLAN configuration restrictions

• show commands with the internal keyword are not supported.

• The lacp vpc-convergence command can be configured in VXLAN and non-VXLAN environments that have vPC port channels to hosts that support LACP.

• For scale environments, the VLAN IDs related to the VRF and Layer-3 VNI (L3VNI) must be reserved with the system vlan nve-overlay id command.

• The load-share keyword has been added to the Configuring a Route Policy procedure for the PBR over VXLAN feature.

  For information regarding the load-share keyword usage for PBR with VXLAN, see the Guidelines and Limitations for Policy-Based Routing section of the Cisco Nexus 9000 Series NX_OS Unicast Routing Configuration Guide, Release 9.x.

• The lacp vpc-convergence command is added for better convergence of Layer 2 EVPN VXLAN:

  interface port-channel10
    switchport
    switchport mode trunk
    switchport trunk allowed vlan 1001-1200
    spanning-tree port type edge trunk
    spanning-tree bpdufilter enable
    lacp vpc-convergence  
    vpc 10
   
  interface Ethernet1/34  <- The port-channel member-port is configured with LACP-active mode (for example, no changes are done at the member-port level.)
    switchport
    switchport mode trunk
    switchport trunk allowed vlan 1001-1200
    channel-group 10 mode active
    no shutdown

• The VXLAN network identifier (VNID) 16777215 is reserved and should not be configured explicitly.

• To refresh the frozen duplicate host during fabric forwarding, use only fabric forwarding dup-host-recovery-timer command and do not use fabric forwarding dup-host-unfreeze-timer command, as it is deprecated.

ISSU restrictions

• VXLAN supports In-Service Software Upgrades (ISSUs). However, VXLAN ISSU is not supported for Cisco Nexus 9300-GX platform switches.

• To remove configurations from an NVE interface, we recommend manually removing each configuration rather than using the default interface nve command.

• Rollback is not supported on VXLAN VLANs that are configured with the port VLAN mapping feature.

Feature support and restrictions

• ACL

  • ACL Options for VXLAN Traffic on Cisco Nexus 9348GC-FXP Switches.

    ACL Direction	ACL Type	VTEP Type	Port Type	Flow Direction	Traffic Type	Supported
    Ingress	PACL	Ingress VTEP	L2 port	Access to Network [GROUP:encap direction]	Native L2 traffic [GROUP:inner]	YES
    	VACL	Ingress VTEP	VLAN	Access to Network [GROUP:encap direction]	Native L2 traffic [GROUP:inner]	YES
    Ingress	RACL	Ingress VTEP	Tenant L3 SVI	Access to Network [GROUP:encap direction]	Native L3 traffic [GROUP:inner]	YES
    Egress	RACL	Ingress VTEP	Uplink L3/L3-PO/SVI	Access to Network [GROUP:encap direction]	VXLAN encap [GROUP:outer]	NO
    Ingress	RACL	Egress VTEP	Uplink L3/L3-PO/SVI	Network to Access [GROUP:decap direction]	VXLAN encap [GROUP:outer]	NO
    Egress	PACL	Egress VTEP	L2 port	Network to Access [GROUP:decap direction]	Native L2 traffic [GROUP:inner]	NO
    	VACL	Egress VTEP	VLAN	Network to Access [GROUP:decap direction]	Native L2 traffic [GROUP:inner]	NO
    Egress	RACL	Egress VTEP	Tenant L3 SVI	Network to Access [GROUP:decap direction]	Post-decap L3 traffic [GROUP:inner]	YES

  • Support added for MultiAuth Change of Authorization (CoA). For more information, see the Cisco Nexus 9000 Series NX-OS Security Configuration Guide, Release 9.3(x).

• Multicast

  • Beginning with Cisco NX-OS Release 10.3(1)F, the Non-blocking Multicast (NBM) feature and VXLAN can co-exist on the same box but in two different VRFs.

    Note	Make sure that the NBM is not enabled on the default VRF where underlay runs.

  • NLB in the unicast, multicast, and IGMP multicast modes is not supported on Cisco Nexus 9000 switch VXLAN VTEPs. The work-around is to move the NLB cluster behind the intermediary device (which supports NLB in the respective mode) and inject the cluster IP address as an external prefix into the VXLAN fabric.

  • On Cisco Nexus 9500 Series switches, if feature nv overlay is enabled, ensure that the interface NVE is configured and in UP state. Otherwise, multicast traffic may be silently dropped in Fabric Modules when it needs to be forwarded out of sub-interfaces.

  • If multiple VTEPs use the same multicast group address for underlay multicast but have different VNIs, the VTEPs should have at least one VNI in common. Doing so ensures that NVE peer discovery occurs and underlay multicast traffic is forwarded correctly. For example, leafs L1 and L4 could have VNI 10 and leafs L2 and L3 could have VNI 20, and both VNIs could share the same group address. When leaf L1 sends traffic to leaf L4, the traffic could pass through leaf L2 or L3. Because NVE peer L1 is not learned on leaf L2 or L3, the traffic is dropped. Therefore, VTEPs that share a group address need to have at least one VNI in common so that peer learning occurs and traffic is not dropped. This requirement applies to VXLAN bud-node topologies.

• PIM BiDir

  • PIM BiDir for VXLAN underlay with and without vPC is supported.

    The following features are not supported when PIM BiDir for VXLAN underlay is configured:

    • Flood and Learn VXLAN

    • Tenant Routed Multicast (TRM)

    • VXLAN EVPN Multi-Site

    • VXLAN EVPN Multihoming

    • vPC attached VTEPs

    For redundant RPs, use Phantom RP.

    For transitioning from PIM ASM to PIM BiDir or from PIM BiDir to PIM ASM underlay, we recommend that you use the following example procedure:

    no ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8
    clear ip mroute *
    clear ip mroute date-created *
    clear ip pim route *
    clear ip igmp groups *
    clear ip igmp snooping groups * vlan all

    Wait for all tables to clean up.

    ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8 bidir

  • When entering the no feature pim command, NVE ownership on the route is not removed so the route stays and traffic continues to flow. Aging is done by PIM. PIM does not age out entries having a VXLAN encap flag.

• ARP suppression

  • Beginning with Cisco NX-OS Release 9.3(3), ARP suppression is supported for Cisco Nexus 9300-GX platform switches.

  • Beginning with Cisco NX-OS Release 9.3(5), ARP suppression is supported with reflective relay for Cisco Nexus 9300-FX/FX2/FXP, and 9300-GX platform switches. For information on reflective relay, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide.

  • ARP suppression is supported for a VNI only if the VTEP hosts the First-Hop Gateway (Distributed Anycast Gateway) for this VNI. The VTEP and SVI for this VLAN must be properly configured for the Distributed Anycast Gateway operation (for example, global anycast gateway MAC address configured and anycast gateway with the virtual IP address on the SVI).

  • ARP suppression is a per-L2VNI fabric-wide setting in the VXLAN fabric. Enable or disable this feature consistently across all VTEPs in the fabric. Inconsistent ARP suppression configuration across VTEPs is not supported.

• FCoE/NPV

  Fibre Channel over Ethernet (FCoE) N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-FX switches.

  Fibre Channel N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-FX switches. VXLAN can exist only on the Ethernet front-panel ports and not on the FC front-panel ports.

• Subinterfaces

  • Beginning with Cisco NX-OS Release 9.3(5), the subinterfaces on VXLAN uplinks has the ability to carry non-VXLAN L3 IP traffic for Cisco Nexus 9300-FX/FX2/FXP, and 9300-GX platform switches and Cisco Nexus 9500 platform switches with Nexus X97160YC-EX, 9700-FX line cards. This feature is supported for VXLAN flood and learn and VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI.

  • Beginning with Cisco NX-OS Release 10.5(2)F, the subinterfaces on VXLAN uplinks has the ability to carry non-VXLAN L3 IP traffic for Cisco Nexus 9500 platform switches with N9K-X9736C-FX3. This feature is supported for VXLAN flood and learn and VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI.

  • Beginning with Cisco NX-OS Release 10.1(1), VXLAN-encapsulated traffic over Parent Interface that Carries Subinterfaces is supported on Cisco Nexus 9300-FX3 platform switches.

  • Beginning with Cisco NX-OS Release 9.3(5), VTEPs support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured. This feature is supported for VXLAN flood and learn, VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI. As shown in the following configuration example, VXLAN traffic is forwarded on the parent interface (eth1/1) in the default VRF, and L3 IP (non-VXLAN) traffic is forwarded on subinterfaces (eth1/1.10) in the tenant VRF.

    interface ethernet 1/1
     description VXLAN carrying interface
     no switchport
     ip address 10.1.1.1/30
    
    interface ethernet 1/1.10
     description NO VXLAN
     no switchport
     vrf member Tenant10
     encapsulation dot1q 10
      ip address 10.10.1.1/30

Restrictions of Cisco Nexus 9504 and 9508 switches with -R line cards

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN Layer 2 Gateway is supported on the 9636C-RX line card. VXLAN and MPLS cannot be enabled on the Cisco Nexus 9508 switch at the same time.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, if VXLAN is enabled, the Layer 2 Gateway cannot be enabled when there is any line card other than the 9636C-RX.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, PIM/ASM is supported in the underlay ports. PIM/Bidir is not supported. For more information, see the Cisco Nexus 9000 Series NX_OS Multicast Routing Configuration Guide, Release 9.3(x).

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, IPv6 hosts routing in the overlay is supported.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, ARP suppression is supported.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN with ingress replication is not supported.

• VXLAN does not support coexistence with MVR and MPLS for Cisco Nexus 9504 and 9508 with -R line cards.

• For Cisco Nexus 9504 and 9508 switches with -R line cards, the L3VNI's VLAN must be added on the vPC peer-link trunk's allowed VLAN list.

Not supported features

• VXLAN is not supported on the N9K-C92348GC-FX3 switches.

• MDP is not supported for VXLAN configurations.

• Consistency checkers are not supported for VXLAN tables.

• VTEP connected to FEX host interface ports is not supported.

• Resilient hashing (port-channel load-balancing resiliency) and VXLAN configurations are not compatible with VTEPs using ALE uplink ports.

  Note	Resilient hashing is disabled by default.

• Routing protocol adjacencies using Anycast Gateway SVIs is not supported.

• Beginning with Cisco NX-OS Release 10.4(3)F, DHCP snooping (Dynamic Host Configuration Protocol snooping) is not supported on VXLAN VLANs.

• RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs support is not available for de-capsulated packets in the network to access direction on the inner payload.

  As a best practice, use PACLs/VACLs for the access to the network direction.

• The QoS buffer-boost feature is not applicable for VXLAN traffic.

• The following limitations apply to releases prior to Cisco NX-OS Release 9.3(5):

  • VTEPs do not support VXLAN-encapsulated traffic over subinterfaces, regardless of VRF participation or IEEE 802.1Q encapsulation.

  • VTEPs do not support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured, regardless of VRF participation.

  • Mixing subinterfaces for VXLAN and non-VXLAN VLANs is not supported.

• Point-to-multipoint Layer 3 and SVI uplinks are not supported.

• SVI and subinterfaces as uplinks are not supported.

Supported Features of CloudScale switches

Guidelines and Limitations for VXLAN Support on Cisco Nexus 9364E-SG2 Series Switches

• Beginning with Cisco NX-OS Release 10.5(3)F, the following VXLAN features are supported on Cisco Nexus 9364E-SG2-Q and 9364E-SG2-O switches.

• For the VXLAN scale limit information, see the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Release 10.5(3)F.

• The following table lists the supported VXLAN features as a leaf :

  Features	Limitations
  VXLAN BGP EVPN	—
  VXLAN L2VNI and L3VNI	—
  Distributed Anycast Gateway	This feature is supported with fabric forwarding anycast-mode command. This configuration is supported with the following combination of configuration On vxlan-vlan only With SVI configured/up on such vxlan vlan With global Fabric Anycast Mac config in system
  ARP Suppression	—
  DHCP Relay	—
  IPv4 and IPv6 unicast overlay traffic	—
  IPv4 unicast underlay	—
  BUM traffic	Supported only with IPv4 Ingress Replication
  VXLAN Uplinks	Only Ethernet and Port-channel routed interfaces are supported as uplinks. SVI or L3-subinterfaces as uplinks are not supported.
  VXLAN Counters	VNI based total packet and byte counters are supported peer counters or per-peer-per-vni counters are not supported.
  VXLAN NGOAM	VTEP and Host reachability are supported
  Unconditional DLB (Dynamic Load Balancing) for VXLAN underlay. For more information, see Dynamic Load Balancing on Silicon One switches.	—

• Cisco Nexus 9364E-SG2-Q and 9364E-SG2-O switches support anycast BGW functions with VRF or L3VNI.

  Use the redistribute or route-map commands on leaf and BGW to distribute the host information.

• Beginning with Cisco NX-OS Release 10.6(1)F, Cisco Nexus 9364E-SG2-Q and 9364E-SG2-O switches support anycast BGW functions with both VLAN (L2VNI) and VRF (L3VNI) together.

  Note	Support for VRF (L3VNI) was introduced in Cisco NX-OS Release 10.5(3)F.

• VLAN based L3VNI configuration is not supported. Only vni vni-id l3 command is supported.

• The following table lists the supported VXLAN features as anycast BGW:

  Features	Limitations
  VXLAN BGP EVPN	—
  L3VNI and L2VNI	—
  Fabric advertise PIP	—
  IPv4 and IPv6 unicast overlay traffic	—
  IPv4 unicast underlay	—
  VXLAN Uplinks	Only Ethernet and Port-channel routed interfaces are supported as uplinks. SVI or L3-subinterfaces as uplinks are not supported
  VXLAN Counters	VNI based total packet or byte counters are supported peer counters or per-peer-per-vni counters are not supported.
  VXLAN NGOAM	VTEP and Host reachability are supported
  Unconditional DLB (Dynamic Load Balancing) for VXLAN underlay. For more details, see Dynamic Load Balancing on Silicon One switches.	—
  L3 QoS policies	—

• Cisco Nexus 9364E-SG2-Q and 9364E-SG2-O switches do not support the following features:

  • VXLAN Flood and Learn

  • IPv4/IPv6 multicast overlay

  • vPC based multihoming

  • VXLAN IPv6 underlay

  • Multicast underlay (PIMv4 SM or PIMv6 ASM, PIM-BiDir, BUD node)

  • IGMP or MLD snooping

  • TRMv4/v6, and TRM data MDT

  • ND suppression

  • Virtual Network Functions (VNF) Multipath

  • VXLAN Policy-Based Routing

  • VXLAN QoS policy

  • DSVNI and route leak

  • VXLAN Access Features

    • Private VLAN (PVLAN)

    • 802.1x

    • Multitag

    • Cross Connect

    • Port security

    • Port VLAN translation

    • QinVNI, Selective QinVNI

    • L2PT

  • VXLAN ESI multihoming

  • Firewall clustering with VXLAN

  • Southbound loop detection

  • Micro-segmentation with Security Groups (SGs) and Security Group ACLs (SGACLs)

  • VXLAN Traffic Engineering

To configure VXLAN features, see Configuring VXLAN BGP EVPN.

Guidelines and limitations for VXLAN support on Cisco Nexus 9336C-SE1 switches

Beginning with Cisco NX-OS Release 10.6(1)F, the following VXLAN features are supported on Cisco Nexus 9336C-SE1 switches.

For the VXLAN scale limit information, see the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Release 10.6(1)F.

The VXLAN static tunnels feature has the following guidelines and limitations:

• The Cisco Nexus 9300-FX/FX2/FX3, 9300-GX and 9300-FX3platform switches support VXLAN static tunnels.

• Beginning with Cisco NX-OS Release 10.1(1), VXLAN Static Tunnels are supported on Cisco Nexus 9300-FX3 platform switches.

• Beginning with Cisco NX-OS Release 10.2(3)F, the VXLAN Static Tunnels are supported on Cisco Nexus 9300-GX2 platform switches.

• Beginning with Cisco NX-OS Release 10.4(1)F, the VXLAN Static Tunnels are supported on Cisco Nexus 9332D-H2R switches.

• Beginning with Cisco NX-OS Release 10.4(2)F, the VXLAN Static Tunnels are supported on Cisco Nexus 93400LD-H1 switches.

• Beginning with Cisco NX-OS Release 10.4(3)F, the VXLAN Static Tunnels are supported on Cisco Nexus 9364C-H1 switches.

• The following guidelines apply to software VTEPs:

  • The software VTEP must be configured as needed to determine how to forward traffic from the VNI.

  • The software VTEP must be compliant with RFC 7348.

• The underlay can be OSPFv2, BGP, IS-IS, or IPv4.

• The overlay can be IPv4 only.

• Additional VXLAN features (such as TRM, Multi-Site, OAM, Cross Connect, and VXLAN QoS), IGMP snooping, MPLS handoff, static MPLS, SR, and SRv6 are not supported.

• Pings across the overlay from local tenant VRF loopback to a host behind the software VTEP is not supported.

• Static tunnels do not support ECMP configuration.

• Static tunnels cannot be configured in the same fabric as traditional flood and learn or BGP EVPN fabrics.

• Local hosts are not supported for VNI-enabled VLANs. Therefore, you cannot have a host in the same VLAN where you configured the VNI.

• Fabric forwarding is supported with static tunnels. When fabric forwarding is enabled, be aware that it affects how SVIs and MAC addresses are used. Consider the following example configuration.

  feature fabric forwarding
  fabric forwarding anycast-gateway-mac 0000.0a0a.0a0a
  
  interface Vlan802
  no shutdown
  vrf member vrfvxlan5201
  ip address 103.33.1.1/16
  fabric forwarding mode anycast-gateway

  When fabric forwarding is enabled:

  • all SVIs where fabric forwarding mode anycast-gateway is configured (for example, Vlan802) are used.

  • the MAC address configured with fabric forwarding anycast-gateway-mac anycast-mac-address (0000.0a0a.0a0a) is used.

The following configuration examples shows VXLAN static tunnel configurations through the supported methods.

NX-OS CLI

vlan 2001
vlan 2001
  vn-segment 20001
 
interface Vlan2001
  no shutdown
  vrf member cust1
  ip forward

vrf context cust1
  vni 20001


feature ofm

tunnel-profile test
  encapsulation vxlan
  source-interface loopback1
  route vrf cust1 101.1.1.2/32 7.7.7.1 next-hop-vrf default vni 20001 dest-vtep-mac f80f.6f43.036c