Configure VXLAN Flood and Learn with Multicast Core
Available Languages
Contents
Introduction
This document describes how to configure and verify Virtual Extensible LAN (VXLAN) flood and learn mode over IPv4 Multicast transport.
Prerequisites
Requirements
Cisco recommends that you have knowledge of basic IP multicast.
Components Used
The information in this document is based on Nexus platform.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Background Information
VXLAN is designed in order to provide the same Ethernet Layer 2 network services as VLAN. VXLAN encapsulates MAC address over UDP packet which makes a Layer 2 packet carried over a Layer 3 network. So, it is basically a MAC-in-UDP header.
VXLAN introduces an 8 byte VXLAN header that consists of a 24 bit VXLAN Network Identifier (VNID) and a few reserved bits. The VXLAN header together with the original Ethernet frame goes in the UDP payload. The 24 bit VNID is used to identify Layer 2 segments and to maintain Layer 2 isolation between the segments. With all 24 bits in VNID, VXLAN can support 16 million LAN segments. So, it resolves the issue of limitation of VLAN. Without VxLAN you can have only 4094 number of VLAN, with increased demand modern networks need more VLANs, and VXLAN is the solution in order to address the issue.
Since it uses the ethernet frame to encapsulate the packet, so ethernet properties need to remain intact like broadcase, unknown unicast and multicast. In order to address these type of traffic, multicast is used. In this document, VXLAN flood and learn is described. As the name specifies that it floods the packet and learns the remote end. It means that data-plane is not up all the times, as soon as the traffic flow data-plane is built up and it expires as soon as MAC address expires.
Packet Format of VXLAN
As shown in this figure, original frame is encapsulated in VXLAN header which is of 8 byte and VNID is of 24 bit. That is further encapsulated in UDP header and outer header is an IP header.
Source IP address is IP of encapsulating Virtual Terminal End Point (VTEP) and destination IP either can be a multicast or unicast one. VXLAN uses VXLAN tunnel endpoint (VTEP) devices to map tenants's end devices to VXLAN segments and in order to perform VXLAN encapsulation and de-encapsulation. Each VTEP has two interfaces: One is a switch interface on the local LAN segment in order to support local endpoint communication through bridging, and the other is an IP interface to the transport IP network.
Remote VTEP Discovery
When the host starts to send the traffic, the process followed is as explained here. At this moment, VTEP does not know the MAC address of the remote host.
- End station sends Address Resolution Protocol (ARP) packet for remote end station.
- Packet reaches to VTEP-A and since VTEP-A does not know about the VTEP-B, it encapsulates the packet inside VXLAN header. It puts the multicast IP address as the destination IP address. Since the same multicast address is used by all VTEPs, all joins the same multicast group.
- This packet reaches to all VTEP and is decapsulated, in this way remote VTEP learns about the other VTEP. Since the decapsulated VTEP has the VNID, it is forwarded in the VLAN that has the same VNID configured.
- Now, remote end sends the ARP reply packet and it reaches to VTEP-B, since now VTEP-B knows about VTEP-A it again encapsulates the orginal frame but now the destination IP address is of VTEP-B and it is the unicast IP address .
- ARP reply reaches to VTEP-A and now VTEP-A gets to know about VTEP-B it forms the neighbor relationship with VTEP-B.
As shown in the diagram, Host H1 belongs to VLAN 10 and is encapsulated in VNID 10000. As shown here, SMAC with H1 and DMAC with H2 is encapsulated inside VNI 1000 and Source IP and Destination IP could be multicast or unicast described in this section .
Configure
Network Diagram
- 9396-A and 9396-B are the VPC peers considerd as VTEP-1
- 9396-C is the VTEP-2
- The diagram has two host in VLAN 10 i.e. 10.10.10.1 and 10.10.10.2
- VLAN 10 is used with VNID as 10010
- 230.1.1.1 is used as multicast group
In order to enable VXLAN on Nexus, you need to enable this feature.
9396-A Configuration
!
feature vn-segment-vlan-based
feature nv overlay
!
vlan 10
vn-segment 10010 ------> 10010 is VNID
!
interface nve1
no shutdown
source-interface loopback0
member vni 10010 mcast-group 230.1.1.1
!
interface eth1/2
!
ip pim sparse-mode
!
interface loopback0
ip address 10.1.1.1/32
ip address 10.1.1.10/32 secondary
ip router ospf 9k area 0.0.0.0
ip pim sparse-mode
!
!
feature vpc
!
vpc domain 1
peer-switch
peer-keepalive destination 10.31.113.41 source 10.31.113.40
peer-gateway
!
interface port-channel1
vpc peer-link
!
interface port-channel112
vpc 112
!
9396-B Configuration
!
vlan 10
vn-segment 10010 ------> 10010 is VNID
!
interface nve1
no shutdown
source-interface loopback0
member vni 10010 mcast-group 230.1.1.1
!
interface eth1/2
ip pim sparse-mode
!
interface loopback0
ip address 10.1.1.2/32
ip address 10.1.1.10/32 secondary
ip router ospf 9k area 0.0.0.0
ip pim sparse-mode
!
feature vpc
!
vpc domain 1
peer-switch
peer-keepalive destination 10.31.113.40 source 10.31.113.41
peer-gateway
!
interface port-channel1
vpc peer-link
!
interface port-channel112
vpc 112
!
9508-A Configuration
feature pim
ip pim rp-address 10.1.1.5 group-list 224.0.0.0/4
ip pim ssm range 232.0.0.0/8
interface loopback0
ip pim sparse-mode
interface Ethernet5/2
ip pim sparse-mode
interface Ethernet5/3
ip pim sparse-mode
interface Ethernet5/4
ip pim sparse-mode
9396-C Configuration
!
vlan 10
vn-segment 10010
!
interface loopback0
ip address 10.1.1.3/32
ip router ospf 9k area 0.0.0.0
ip pim sparse-mode
!
interface nve1
no shutdown
source-interface loopback0
member vni 10010 mcast-group 230.1.1.1
!
int eth1/2
ip pim sparse-mode
!
Verify
Use this section in order to confirm that your configuration works properly.
As of now host has not started to send the packet stream. Since 9396-A is a VPC holding device, it originates the traffic sourcing from the secondary IP address and acts as a source IP address for multicast stream.
9396-A# sh nve interface
Interface: nve1, State: Up, encapsulation: VXLAN
VPC Capability: VPC-VIP-Only [notified]
Local Router MAC: d8b1.9076.9053
Host Learning Mode: Data-Plane
Source-Interface: loopback0 (primary: 10.1.1.1, secondary: 10.1.1.10)
9396-A# sh ip mroute 230.1.1.1
IP Multicast Routing Table for VRF "default"
(*, 230.1.1.1/32), uptime: 01:09:34, ip pim nve
Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.2
Outgoing interface list: (count: 1)
nve1, uptime: 00:11:20, nve
(10.1.1.3/32, 230.1.1.1/32), uptime: 00:12:19, ip mrib pim nve
Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.2
Outgoing interface list: (count: 1)
nve1, uptime: 00:11:20, nve
(10.1.1.10/32, 230.1.1.1/32), uptime: 00:11:20, nve ip mrib pim
Incoming interface: loopback0, RPF nbr: 10.1.1.10
Outgoing interface list: (count: 1)
Ethernet1/2, uptime: 00:11:20, pim
In *,G entry nve interface is populated in Outgoing Interface List (OIL). As shown here, 10.1.1.10 is source of multicast stream and nve interface is the Last hop router for the multicast stream with eth1/2 that faces towards the core is the outgoing interface.
As there is no traffic flowing from host, there are no nve peers:
9396-A# show mac address-table vlan 10
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 10 8c60.4f93.5ffc dynamic 0 F F Po112 >> This mac is for host 10.10.10.1
9396-A# sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
This output shows you how vPC output must look like:
9396-A# sh vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po1 up 1-10
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
112 Po112 up success success 1-10
9396-A# sh vpc consistency-parameters global
Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
Vlan to Vn-segment Map 1 1 Relevant Map(s) 1 Relevant Map(s)
STP Mode 1 Rapid-PVST Rapid-PVST
STP Disabled 1 None None
STP MST Region Name 1 "" ""
STP MST Region Revision 1 0 0
STP MST Region Instance to 1
VLAN Mapping
STP Loopguard 1 Disabled Disabled
STP Bridge Assurance 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Simulate PVST 1 Enabled Enabled
Nve Admin State, Src Admin 1 Up, Up, 10.1.1.10, DP Up, Up, 10.1.1.10, DP
State, Secondary IP, Host
Reach Mode
Nve Vni Configuration 1 10010 10010
Nve encap Configuration 1 vxlan vxlan
Interface-vlan admin up 2
Interface-vlan routing 2 1 1
capability
Allowed VLANs - 1-10 1-10
Local suspended VLANs - - -
9508-A
Since 9508-A route is core router, it is not aware about the VXLAN, it is aware about the mroute entry only as shown here:
9508-A# sh ip mroute 230.1.1.1
IP Multicast Routing Table for VRF "default"
(*, 230.1.1.1/32), uptime: 01:30:06, pim ip
Incoming interface: loopback0, RPF nbr: 10.1.1.5, uptime: 01:30:06
Outgoing interface list: (count: 3)
Ethernet5/3, uptime: 00:14:11, pim
Ethernet5/2, uptime: 00:14:31, pim
Ethernet5/4, uptime: 00:16:22, pim
(10.1.1.3/32, 230.1.1.1/32), uptime: 00:15:44, pim mrib ip
Incoming interface: Ethernet5/4, RPF nbr: 192.168.10.10, uptime: 00:15:44, internal
Outgoing interface list: (count: 2)
Ethernet5/3, uptime: 00:14:11, pim
Ethernet5/2, uptime: 00:14:31, pim
(10.1.1.10/32, 230.1.1.1/32), uptime: 00:14:31, pim mrib ip
Incoming interface: Ethernet5/2, RPF nbr: 192.168.10.1, uptime: 00:14:31, internal
Outgoing interface list: (count: 1)
Ethernet5/4, uptime: 00:14:31, pim
9396-C
9396-C# show ip mroute
IP Multicast Routing Table for VRF "default"
(*, 230.1.1.1/32), uptime: 01:07:34, ip pim nve
Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.9
Outgoing interface list: (count: 1)
nve1, uptime: 00:10:38, nve
(10.1.1.3/32, 230.1.1.1/32), uptime: 00:10:38, nve ip mrib pim
Incoming interface: loopback0, RPF nbr: 10.1.1.3
Outgoing interface list: (count: 1)
Ethernet1/2, uptime: 00:09:49, pim
(10.1.1.10/32, 230.1.1.1/32), uptime: 00:08:05, ip mrib pim nve
Incoming interface: Ethernet1/2, RPF nbr: 192.168.10.9
Outgoing interface list: (count: 1)
nve1, uptime: 00:08:05, nve
Status After Traffic Flow Starts Between Peers
As soon as host 1 i.e. 10.10.10.1 starts to send the traffic to 10.10.10.2 NVE peer comes up:
9396-A# sh mac address-table dynamic
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 10 8c60.4f93.5ffc dynamic 0 F F Po112
+ 10 8c60.4f93.647c dynamic 0 F F nve1(10.1.1.3)
9396-A# sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 10.1.1.3 Up DP 00:00:14 n/a
9396-A# sh nve peers detail
Details of nve Peers:
----------------------------------------
Peer-Ip: 10.1.1.3
NVE Interface : nve1
Peer State : Up
Peer Uptime : 00:04:49
Router-Mac : n/a
Peer First VNI : 10010
Time since Create : 00:04:49
Configured VNIs : 10010
Provision State : add-complete
Route-Update : Yes
Peer Flags : None
Learnt CP VNIs : --
Peer-ifindex-resp : Yes
----------------------------------------
9396-A sh nve vni 10010 detail
VNI: 10010
NVE-Interface : nve1
Mcast-Addr : 230.1.1.1
VNI State : Up
Mode : data-plane
VNI Type : L2 [10]
VNI Flags :
Provision State : add-complete
Vlan-BD : 10
SVI State : n/a
9396-A# sh nve internal vni 10010
VNI 10010
Ready-State : Ready [L2-vni-flood-learn-ready]
Similarly on 9396-C NVE peers must be up:
9396-C# show mac address-table dynamic
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 10 8c60.4f93.5ffc dynamic 0 F F nve1(10.1.1.10)
* 10 8c60.4f93.647c dynamic 0 F F Eth1/13
9396-C# sh nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- --------------- ----- --------- -------- -----------------
nve1 10.1.1.10 Up DP 00:08:28 n/a
9396-C# sh nve peers detail
Details of nve Peers:
----------------------------------------
Peer-Ip: 10.1.1.10
NVE Interface : nve1
Peer State : Up
Peer Uptime : 00:08:32
Router-Mac : n/a
Peer First VNI : 10010
Time since Create : 00:08:32
Configured VNIs : 10010
Provision State : add-complete
Route-Update : Yes
Peer Flags : None
Learnt CP VNIs : --
Peer-ifindex-resp : Yes
----------------------------------------
9396-C sh nve vni 10010 detail
VNI: 10010
NVE-Interface : nve1
Mcast-Addr : 230.1.1.1
VNI State : Up
Mode : data-plane
VNI Type : L2 [10]
VNI Flags :
Provision State : add-complete
Vlan-BD : 10
SVI State : n/a
9396-C# sh nve internal vni 10010
VNI 10010
Ready-State : Ready [L2-vni-flood-learn-ready]
As shown here, nve peers are based upon data-plane learning and it uses flood and learn mechanism. In case MAC address gets timed out, nve peer goes down.
Troubleshoot
There is currently no specific troubleshooting information available for this configuration.
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)