VXLAN has the following guidelines and limitations:

Switch or port restrictions

• FEX ports do not support IGMP snooping on VXLAN VLANs.

• The VXLAN UDP port number is used for VXLAN encapsulation. For Cisco Nexus NX-OS, the UDP port number is 4789. It complies with IETF standards and is not configurable.

• Cisco Nexus 9300 Series switches with 100G uplinks only support VXLAN switching/bridging.

  Cisco Nexus 9200, Cisco Nexus 9300-EX, and Cisco Nexus 9300-FX, and Cisco Nexus 9300-FX2 platform switches do not have this restriction.

  Note	For VXLAN routing support, a 40G uplink module is required.

• When SVI is enabled on a VTEP (flood and learn, or EVPN), make sure that ARP-ETHER TCAM is carved using the hardware access-list tcam region arp-ether 256 command. This requirement does not apply to Cisco Nexus 9200, 9300-EX, 9300-FX/FX2/FX3, and 9300-GX/GX2 platform switches and Cisco 9500 Series switches with 9700-EX/FX/GX line cards.

• Beginning with Cisco NX-OS Release 10.2(3)F, VXLAN can coexist with the GRE tunnel feature or the MPLS (static or segment-routing) feature.

• Native VLANs are supported as transit traffic over a VXLAN fabric on Cisco Nexus 9300-EX/FX/FX2/FX3/GX/GX2 Series switches.

• A FEX HIF (FEX host interface port) is supported for a VLAN that is extended with VXLAN.

• Bind NVE to a loopback address that is separate from other loopback addresses that are required by Layer 3 protocols. A best practice is to use a dedicated loopback address for VXLAN. This best practice should be applied not only for the vPC VXLAN deployment, but for all VXLAN deployments.

• Tenant VRF (VRF with VNI on it) cannot be used on an SVI that has no VNI binding into it (underlay infra VRF).

• For traceroute through a VXLAN fabric when using L3VNI, the following scenario is the expected behavior:

  If L3VNI is associated with a VRF and an SVI, the associated SVI does not have an L3 address that is configured but instead has the "ip forward" configuration command. Due to this interface setup it cannot respond back to the traceroute with its own SVI address. Instead, when a traceroute involving the L3VNI is run through the fabric, the IP address reported will be the lowest IP address of an SVI that belongs to the corresponding tenant VRF.

• In an ingress replication vPC setup, Layer 3 connectivity is needed between vPC peer devices.

VXLAN configuration restrictions

• show commands with the internal keyword are not supported.

• The lacp vpc-convergence command can be configured in VXLAN and non-VXLAN environments that have vPC port channels to hosts that support LACP.

• For scale environments, the VLAN IDs related to the VRF and Layer-3 VNI (L3VNI) must be reserved with the system vlan nve-overlay id command.

• The load-share keyword has been added to the Configuring a Route Policy procedure for the PBR over VXLAN feature.

  For information regarding the load-share keyword usage for PBR with VXLAN, see the Guidelines and Limitations for Policy-Based Routing section of the Cisco Nexus 9000 Series NX_OS Unicast Routing Configuration Guide, Release 9.x.

• The lacp vpc-convergence command is added for better convergence of Layer 2 EVPN VXLAN:

  interface port-channel10
    switchport
    switchport mode trunk
    switchport trunk allowed vlan 1001-1200
    spanning-tree port type edge trunk
    spanning-tree bpdufilter enable
    lacp vpc-convergence  
    vpc 10
   
  interface Ethernet1/34  <- The port-channel member-port is configured with LACP-active mode (for example, no changes are done at the member-port level.)
    switchport
    switchport mode trunk
    switchport trunk allowed vlan 1001-1200
    channel-group 10 mode active
    no shutdown

• The system nve ipmc command is not applicable to the Cisco Nexus 9200 and 9300-EX platform switches and Cisco Nexus 9500 platform switches with 9700-EX line cards.

• The VXLAN network identifier (VNID) 16777215 is reserved and should not be configured explicitly.

• To refresh the frozen duplicate host during fabric forwarding, use only fabric forwarding dup-host-recovery-timer command and do not use fabric forwarding dup-host-unfreeze-timer command, as it is deprecated.

ISSU restrictions

• VXLAN supports In-Service Software Upgrades (ISSUs). However, VXLAN ISSU is not supported for Cisco Nexus 9300-GX platform switches.

• To remove configurations from an NVE interface, we recommend manually removing each configuration rather than using the default interface nve command.

• Rollback is not supported on VXLAN VLANs that are configured with the port VLAN mapping feature.

Feature support and restrictions

• ACL

  • ACL Options for VXLAN Traffic on Cisco Nexus 92300YC, 92160YC-X, 93120TX, 9332PQ, and 9348GC-FXP Switches.

    ACL Direction	ACL Type	VTEP Type	Port Type	Flow Direction	Traffic Type	Supported
    Ingress	PACL	Ingress VTEP	L2 port	Access to Network [GROUP:encap direction]	Native L2 traffic [GROUP:inner]	YES
    	VACL	Ingress VTEP	VLAN	Access to Network [GROUP:encap direction]	Native L2 traffic [GROUP:inner]	YES
    Ingress	RACL	Ingress VTEP	Tenant L3 SVI	Access to Network [GROUP:encap direction]	Native L3 traffic [GROUP:inner]	YES
    Egress	RACL	Ingress VTEP	Uplink L3/L3-PO/SVI	Access to Network [GROUP:encap direction]	VXLAN encap [GROUP:outer]	NO
    Ingress	RACL	Egress VTEP	Uplink L3/L3-PO/SVI	Network to Access [GROUP:decap direction]	VXLAN encap [GROUP:outer]	NO
    Egress	PACL	Egress VTEP	L2 port	Network to Access [GROUP:decap direction]	Native L2 traffic [GROUP:inner]	NO
    	VACL	Egress VTEP	VLAN	Network to Access [GROUP:decap direction]	Native L2 traffic [GROUP:inner]	NO
    Egress	RACL	Egress VTEP	Tenant L3 SVI	Network to Access [GROUP:decap direction]	Post-decap L3 traffic [GROUP:inner]	YES

  • ACL Options for VXLAN traffic on Cisco Nexus 92160YC-X, 93108TC-EX, 93180LC-EX, and 93180YC-EX switches, Release 7.0(3)I6(1).

  • Support added for MultiAuth Change of Authorization (CoA). For more information, see the Cisco Nexus 9000 Series NX-OS Security Configuration Guide, Release 9.3(x).

• Multicast

  • Beginning with Cisco NX-OS Release 10.3(1)F, the Non-blocking Multicast (NBM) feature and VXLAN can co-exist on the same box but in two different VRFs.

    Note	Make sure that the NBM is not enabled on the default VRF where underlay runs.

  • NLB in the unicast, multicast, and IGMP multicast modes is not supported on Cisco Nexus 9000 switch VXLAN VTEPs. The work-around is to move the NLB cluster behind the intermediary device (which supports NLB in the respective mode) and inject the cluster IP address as an external prefix into the VXLAN fabric.

  • On Cisco Nexus 9500 Series switches, if feature nv overlay is enabled, ensure that the interface NVE is configured and in UP state. Otherwise, multicast traffic may be silently dropped in Fabric Modules when it needs to be forwarded out of sub-interfaces.

  • If multiple VTEPs use the same multicast group address for underlay multicast but have different VNIs, the VTEPs should have at least one VNI in common. Doing so ensures that NVE peer discovery occurs and underlay multicast traffic is forwarded correctly.

    For example, leafs L1 and L4 could have VNI 10 and leafs L2 and L3 or border spine could have VNI 20, and both VNIs could share the same group address. When leaf L1 sends traffic to leaf L4, the traffic could pass through leaf L2 or L3 or border spine. Because NVE peer L1 is not learned on leaf L2 or L3 or border spine , the traffic is dropped. Therefore, VTEPs that share a group address need to have at least one VNI in common so that peer learning occurs and traffic is not dropped. This requirement applies to VXLAN bud-node topologies and border spine cases.

• PIM BiDir

  • PIM BiDir for VXLAN underlay with and without vPC is supported.

    The following features are not supported when PIM BiDir for VXLAN underlay is configured:

    • Flood and Learn VXLAN

    • Tenant Routed Multicast (TRM)

    • VXLAN EVPN Multi-Site

    • VXLAN EVPN Multihoming

    • vPC attached VTEPs

    For redundant RPs, use Phantom RP.

    For transitioning from PIM ASM to PIM BiDir or from PIM BiDir to PIM ASM underlay, we recommend that you use the following example procedure:

    no ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8
    clear ip mroute *
    clear ip mroute date-created *
    clear ip pim route *
    clear ip igmp groups *
    clear ip igmp snooping groups * vlan all

    Wait for all tables to clean up.

    ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8 bidir

  • When entering the no feature pim command, NVE ownership on the route is not removed so the route stays and traffic continues to flow. Aging is done by PIM. PIM does not age out entries having a VXLAN encap flag.

• ARP suppression

  • Beginning with Cisco NX-OS Release 9.3(3), ARP suppression is supported for Cisco Nexus 9300-GX platform switches.

  • Beginning with Cisco NX-OS Release 9.3(5), ARP suppression is supported with reflective relay for Cisco Nexus 9364C, 9300-EX, 9300-FX/FX2/FXP, and 9300-GX platform switches. For information on reflective relay, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide.

  • ARP suppression is supported for a VNI only if the VTEP hosts the First-Hop Gateway (Distributed Anycast Gateway) for this VNI. The VTEP and SVI for this VLAN must be properly configured for the Distributed Anycast Gateway operation (for example, global anycast gateway MAC address configured and anycast gateway with the virtual IP address on the SVI).

  • ARP suppression is a per-L2VNI fabric-wide setting in the VXLAN fabric. Enable or disable this feature consistently across all VTEPs in the fabric. Inconsistent ARP suppression configuration across VTEPs is not supported.

• FCoE/NPV

  Fibre Channel over Ethernet (FCoE) N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-EX and 93180YC-FX switches.

  Fibre Channel N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-FX switches. VXLAN can exist only on the Ethernet front-panel ports and not on the FC front-panel ports.

• Subinterfaces

  • Beginning with Cisco NX-OS Release 9.3(5), the subinterfaces on VXLAN uplinks has the ability to carry non-VXLAN L3 IP traffic for Cisco Nexus 9332C, 9364C, 9300-EX, 9300-FX/FX2/FXP, and 9300-GX platform switches and Cisco Nexus 9500 platform switches with -EX/FX line cards. This feature is supported for VXLAN flood and learn and VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI.

  • Beginning with Cisco NX-OS Release 10.1(1), VXLAN-encapsulated traffic over Parent Interface that Carries Subinterfaces is supported on Cisco Nexus 9300-FX3 platform switches.

  • Beginning with Cisco NX-OS Release 9.3(5), VTEPs support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured. This feature is supported for VXLAN flood and learn, VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI. As shown in the following configuration example, VXLAN traffic is forwarded on the parent interface (eth1/1) in the default VRF, and L3 IP (non-VXLAN) traffic is forwarded on subinterfaces (eth1/1.10) in the tenant VRF.

    interface ethernet 1/1
     description VXLAN carrying interface
     no switchport
     ip address 10.1.1.1/30
    
    interface ethernet 1/1.10
     description NO VXLAN
     no switchport
     vrf member Tenant10
     encapsulation dot1q 10
      ip address 10.10.1.1/30

Restrictions of Cisco Nexus 9504 and 9508 switches with -R line cards

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN Layer 2 Gateway is supported on the 9636C-RX line card. VXLAN and MPLS cannot be enabled on the Cisco Nexus 9508 switch at the same time.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, if VXLAN is enabled, the Layer 2 Gateway cannot be enabled when there is any line card other than the 9636C-RX.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, PIM/ASM is supported in the underlay ports. PIM/Bidir is not supported. For more information, see the Cisco Nexus 9000 Series NX_OS Multicast Routing Configuration Guide, Release 9.3(x).

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, IPv6 hosts routing in the overlay is supported.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, ARP suppression is supported.

• For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN with ingress replication is not supported.

• VXLAN does not support coexistence with MVR and MPLS for Cisco Nexus 9504 and 9508 with -R line cards.

• For Cisco Nexus 9504 and 9508 switches with -R line cards, the L3VNI's VLAN must be added on the vPC peer-link trunk's allowed VLAN list.

Not supported features

• VXLAN is not supported on the Cisco Nexus N9K-C92348GC-X switches.

• MDP is not supported for VXLAN configurations.

• Consistency checkers are not supported for VXLAN tables.

• VTEP connected to FEX host interface ports is not supported.

• Resilient hashing (port-channel load-balancing resiliency) and VXLAN configurations are not compatible with VTEPs using ALE uplink ports.

  Note	Resilient hashing is disabled by default.

• Beginning with Cisco NX-OS Release 10.3(3)F, MHBFD with the new L3VNI mode is not supported on VXLAN.

• Routing protocol adjacencies using Anycast Gateway SVIs is not supported.

• RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs support is not available for de-capsulated packets in the network to access direction on the inner payload.

  As a best practice, use PACLs/VACLs for the access to the network direction.

• The QoS buffer-boost feature is not applicable for VXLAN traffic.

• The following limitations apply to releases prior to Cisco NX-OS Release 9.3(5):

  • VTEPs do not support VXLAN-encapsulated traffic over subinterfaces, regardless of VRF participation or IEEE 802.1Q encapsulation.

  • VTEPs do not support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured, regardless of VRF participation.

  • Mixing subinterfaces for VXLAN and non-VXLAN VLANs is not supported.

• Point-to-multipoint Layer 3 and SVI uplinks are not supported.

• SVI and subinterfaces as uplinks are not supported.

Supported Features of CloudScale switches