Extension that allows BGP to transport Layer 2 MAC and Layer 3 IP information is EVPN and uses Multi-Protocol Border Gateway Protocol (MP-BGP) as the protocol to distribute reachability information that pertains to the VXLAN overlay network.
Troubleshoot EVPN VXLANv6 on Catalyst 9000 Series Switches
Available Languages
Contents
Introduction
This document describes how to troubleshoot EVPN VXLANv6 on Catalyst 9000 Series Switches.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
- Unicast EVPN VxLAN feature, BGP and MVPN (Multicast Virtual Private Network).
- IPv4 and IPv6 Unicast
- Multicast concepts and how multicast operates
Components Used
The information in this document is based on these software and hardware versions:
- Catalyst 9000 Series Switches
Note: The 9200, 9500X, and 9600X do not support VXLANv6
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Terminology
|
EVPN |
Ethernet Virtual Private Network |
|
|
VXLAN |
Virtual Extensible LAN (Local Area Network) |
VXLAN is designed to overcome the inherent limitations of VLANs and STP. It is a proposed IETF standard [RFC 7348] to provide the same Ethernet Layer 2 network services as VLANs do, but with greater flexibility. Functionally, it is a MAC-in- UDP encapsulation protocol that runs as a virtual overlay on a Layer 3 underlay network. |
|
VTEP |
Virtual Tunnel Endpoint |
This is the device that does the encapsulation and de-encapsulation |
|
EVI |
EVPN Instance |
The EVPN instance (EVI) is represented by the virtual network identifier (VNI). An EVI represents a VPN on a PE router. It serves the same role of an IP VPN Routing and Forwarding (VRF), and EVIs are assigned import/export Route Targets (RTs) |
|
NVE |
Network Virtual Interface |
Logical interface where the encapsulation and de-encapsulation occur |
|
VNI |
VXLAN network identifier |
Uniquely identifies each Layer 2 subnet or segment. There are two types of VNI: Symmetric (L2VNI): VTEPs have same VNI Asymmetric (L3VNI): VTEPs do not have same VNI and are routed via a single common VNI. |
|
BUM |
Broadcast, Unknown Unicast, Multicast |
BUM traffic is sent via the Mcast group tied to the VNI under the NVE configuration. |
|
TRM |
Tenant Routed Multicast |
BGP-EVPN based solution that enables multicast routing between sources and receivers connected on VTEPS in VxLAN fabric [RFC7432]. There are two types L2TRM (Layer 2 TRM) & L3TRM (Layer 3 TRM) |
|
MDT |
Multicast Distribution Tree |
The multicast tree built between VTEPs for encapsulation and tunnelling of Tenant Multicast Traffic. |
|
PVLAN |
Private VLAN |
Partitions the Ethernet broadcast domain of a VLAN into subdomains, which allows you to isolate the ports on the switch from each other. |
|
MIB |
Management Information Base |
A Simple Network Management Protocol (SNMP) monitor object |
|
PIM-BIDIR |
Protocol Independent Multicast Bi-Directional |
A type of PIM where traffic is only forwarded along a shared tree that is rooted at the rendezvous point (RP) for the group. |
|
VFI |
Virtual Forwarding Instance |
A virtual bridge port that is capable of performing native bridging functions, such as forwarding, based on the destination MAC address, source MAC address learning and aging, and so forth. |
|
IRB |
Integrated Routing and Bridging |
enables a Layer 2 VPN and an Layer 3 VPN overlay that allows end hosts across the overlay to communicate with each other within the same subnet and across different subnets within the VPN. |
|
IMET |
Inclusive Multicast Ethernet Tag |
also called BGP Route Type 3 (RT3), for the auto-discovery of remote peers in order to set up the BUM tunnels over VXLAN. IMET routes carry the remote (egress) VNIs advertised from the remote peers, which can be different from the local VNI. These remote VNIs are called Downstream Assigned VNIs. |
|
DAG |
Distributed Anycast Gateway |
Default gateway function on all VTEPs. The same gateway IP lives on all VTEPs and allows for mobility in the fabric. |
|
IR(B) |
Integrated Routing (& Bridging) |
Enables a Layer 2 VPN and an Layer 3 VPN overlay that allows end hosts across the overlay to communicate with each other within the same subnet and across different subnets within the VPN. |
|
RNH |
Remote Next Hop |
The remote VTEP |
|
RPF |
Reverse Path Forwarding |
The Unicast path back to the Source. Incoming multicast packets are not accepted/forwarded unless they are received the same path as the unicast routing table. ('ip multicast multipath' use cases excluded). |
|
RP |
Rendezvous Point |
A role that a device performs when in PIM Sparse Mode. The common meeting point for Multicast sources and Receivers. |
Network Diagram
Greenfield EVPN VXLANv6 Topology
Verify (Greenfield or Dual-stack)
Before performing per traffic flow troubleshooting, EVPN VXLANv6 topology level information must be verified first to ensure proper operation. This section shows verification commands for both deployments, choose applicable verification commands based on your deployment.
Underlay Connectivity
Verify IPv6 underlay connectivity between VTEPs and Spines (for Greenfield VXLANv6 you only need to verify IPv6 underlay connectivity).
VTEP1#show ipv6 route
IPv6 Routing Table - default - 9 entries
Codes: C - Connected, L - Local, S - Static, U - Per-user Static route
B - BGP, R - RIP, H - NHRP, I1 - ISIS L1
I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary, D - EIGRP
EX - EIGRP external, ND - ND Default, NDp - ND Prefix, DCE - Destination
NDr - Redirect, O - OSPF Intra, OI - OSPF Inter, OE1 - OSPF ext 1
OE2 - OSPF ext 2, ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
la - LISP alt, lr - LISP site-registrations, ld - LISP dyn-eid
lA - LISP away, le - LISP extranet-policy, lp - LISP publications
ls - LISP destinations-summary
LC 2001:DB8:1::1/128 [0/0]
via Loopback0, receive
LC 2001:DB8:1::2/128 [0/0]
via Loopback1, receive
O 2001:DB8:2::1/128 [110/2] <-- VTEP2 BGP update source
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:2::2/128 [110/2] <-- VTEP2 VTEP IP
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:3::1/128 [110/2] <-- VTEP3 BGP update source
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:3::2/128 [110/2] <-- VTEP3 VTEP IP
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:99::99/128 [110/1] <-- SPINE BGP update source
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
Verify both IPv4 and IPv6 underlay connectivity (For Dual-stack you must verify both IPv4 and IPv6 underlay connectivity).
VTEP1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP
n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
H - NHRP, G - NHRP registered, g - NHRP registration summary
o - ODR, P - periodic downloaded static route, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR
& - replicated local route overrides by connected
Gateway of last resort is not set
10.0.0.0/8 is variably subnetted, 14 subnets, 3 masks
C 10.0.1.0/30 is directly connected, GigabitEthernet1/0/3
L 10.0.1.2/32 is directly connected, GigabitEthernet1/0/3
O 10.0.2.0/30 [110/2] via 10.0.1.1, 02:20:01, GigabitEthernet1/0/3
O 10.0.3.0/30 [110/2] via 10.0.1.1, 02:20:01, GigabitEthernet1/0/3
C 10.1.1.1 is directly connected, Loopback0
C 10.1.1.2 is directly connected, Loopback1
O E2 10.1.1.3 [110/20] via 10.0.1.1, 02:19:24, GigabitEthernet1/0/3
O 10.2.2.1 [110/3] via 10.0.1.1, 02:19:34, GigabitEthernet1/0/3 <-- VTEP2 BGP update source
O 10.2.2.2 [110/3] via 10.0.1.1, 02:19:34, GigabitEthernet1/0/3 <-- VTEP2 VTEP IP
O 10.3.3.1 [110/3] via 10.0.1.1, 02:19:24, GigabitEthernet1/0/3 <-- VTEP3 BGP update source
O 10.3.3.2 [110/3] via 10.0.1.1, 02:19:24, GigabitEthernet1/0/3 <-- VTEP3 VTEP IP
O E2 10.5.0.0/24 [110/20] via 10.0.1.1, 02:19:24, GigabitEthernet1/0/3
O E2 10.5.0.1/32 [110/20] via 10.0.1.1, 02:19:24, GigabitEthernet1/0/3
O 10.99.99.99 [110/2] via 10.0.1.1, 02:20:01, GigabitEthernet1/0/3 <-- SPINE BGP update source
VTEP1#show ipv6 route
IPv6 Routing Table - default - 8 entries
Codes: C - Connected, L - Local, S - Static, U - Per-user Static route
B - BGP, R - RIP, H - NHRP, I1 - ISIS L1
I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary, D - EIGRP
EX - EIGRP external, ND - ND Default, NDp - ND Prefix, DCE - Destination
NDr - Redirect, O - OSPF Intra, OI - OSPF Inter, OE1 - OSPF ext 1
OE2 - OSPF ext 2, ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
la - LISP alt, lr - LISP site-registrations, ld - LISP dyn-eid
lA - LISP away, le - LISP extranet-policy, lp - LISP publications
ls - LISP destinations-summary192.
LC 2001:DB8:1::1/128 [0/0]
via Loopback0, receive
LC 2001:DB8:1::2/128 [0/0]
via Loopback1, receive
O 2001:DB8:2::1/128 [110/2] <-- VTEP2 BGP update source
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:2::2/128 [110/2] <-- VTEP2 VTEP IP
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:3::1/128 [110/2] <-- VTEP3 BGP update source
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:3::2/128 [110/2] <-- VTEP3 VTEP IP
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
O 2001:DB8:99::99/128 [110/1] <-- SPINE BGP update
via FE80::250:56FF:FE9A:EE12, GigabitEthernet1/0/3
BGP L2VPN EVPN Neighbors
Confirm neighbors are established on all VTEPs and Spines
VTEP1#show ip bgp l2vpn evpn neighbors
BGP neighbor is 2001:DB8:99::99, remote AS 100, internal link <-- BGP neighbor session with SPINE
BGP version 4, remote router ID 10.99.99.99
BGP state = Established, up for 3d17h <-- BGP neighbor is established
Last read 00:00:27, last write 00:00:50, hold time is 180, keepalive interval is 60 seconds
Last update received: 3d12h
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family L2VPN Evpn: advertised and received
Graceful Restart Capability: advertised and received
Remote Restart timer is 120 seconds
Address families advertised by peer:
L2VPN Evpn (was not preserved)
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 20 39
Keepalives: 5896 5893
Route Refresh: 0 0
Total: 5917 5935
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 0 seconds
NVE Interface Status
Verify Greenfield VXLANv6
VTEP1#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up
Encapsulation: Vxlan IPv6 <-- VXLAN encapsulation is set to IPv6
Multicast BUM encapsulation: Vxlan IPv6 <-- encap for Multicast Replication is also set to IPv6
BGP host reachability: Enabled, VxLAN dport: 4789
VNI number: L3CP 1 L2CP 6 L2DP 0
source-interface: Loopback1 (primary: 2001:DB8:1::2 vrf: 0) <-- VTEP IP
tunnel interface: Tunnel0
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
Verify Dual-stack
VTEP1#show nve interface nve1 detail
Interface: nve1, State: Admin Up, Oper Up
Encapsulation: Vxlan dual stack prefer IPv6 <-- VXLAN encapsulation is set to dual-stack prefer IPv6
Multicast BUM encapsulation: Vxlan IPv4 <-- encap for Multicast Replication is still IPv4
BGP host reachability: Enabled, VxLAN dport: 4789
VNI number: L3CP 1 L2CP 6 L2DP 0
source-interface: Loopback1 (primary: 10.1.1.2 2001:DB8:1::2 vrf: 0) <-- Primary is IPv4, secondary IPv6
tunnel interface: Tunnel0 Tunnel1 <-- Two tunnels are created for both VXLANv4 and VXLANv6
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
VXLAN Tunnel Interface Status
Verify Greenfield VXLANv6 (only VXLANv6 tunnel is created for this deployment)
VTEP1#show interface Tunnel0
Tunnel0 is up, line protocol is up <-- Tunnel is up/up
Hardware is Tunnel
MTU 9216 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel linestate evaluation up
Tunnel source 2001:DB8:1::2
Tunnel protocol/transport MUDP/IPV6 <-- VXLANv6 tunnel
TEID 0x0, sequencing disabled
Checksumming of packets disabled
source_port:4789, destination_port:0
<...snip...>
Verify Dual-stack (both VXLANv4 and VXLANv6 tunnels are created for this deployment)
VTEP1#show interface Tunnel0
Tunnel0 is up, line protocol is up <-- Tunnel is up/up
Hardware is Tunnel
Interface is unnumbered. Using address of Loopback1 (10.1.1.2)
MTU 17864 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel linestate evaluation up
Tunnel source 10.1.1.2
Tunnel protocol/transport MUDP/IP <-- VXLANv4 tunnel
TEID 0x0, sequencing disabled
Checksumming of packets disabled
source_port:4789, destination_port:0
<...snip...>
VTEP1#show interface Tunnel1
Tunnel1 is up, line protocol is up <-- Tunnel is up/up
Hardware is Tunnel
MTU 9216 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel linestate evaluation up
Tunnel source 2001:DB8:1::2
Tunnel protocol/transport MUDP/IPV6 <-- VXLANv6 tunnel
TEID 0x0, sequencing disabled
Checksumming of packets disabled
source_port:4789, destination_port:0
<...snip...>
L2VNI Status in NVE
Verify Greenfield VXLANv6 L2VNI in NVE
VTEP1#show nve vni 20011 detail
Interface VNI Multicast-group VNI state Mode VLAN cfg vrf
nve1 20011 N/A Up L2CP 11 CLI red
L2CP VNI IRB state: IPv4 up, IPv6 up
L2CP VNI local VTEP info:
VLAN: 11
SVI if handler: 0x1A
Local VTEP: 2001:DB8:1::2
Local routing: Disabled
Core IRB info:
L3VNI: 30000
VRF name: red
VLAN: 3
V4TopoID: 0x2
V6TopoID: 0x1E000002
Local VTEP: 2001:DB8:1::2
SVI if handler: 0x19
SVI MAC: 0050.569A.A8BF
IPv4 TRM mdt group: N/A
IPv6 TRM mdt group: N/A
VNI Detailed statistics:
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
Verify Dual-stack L2VNI in NVE
VTEP1#show nve vni 20011 detail
Interface VNI Multicast-group VNI state Mode VLAN cfg vrf
nve1 20011 N/A Up L2CP 11 CLI red
L2CP VNI IRB state: IPv4 up, IPv6 up
L2CP VNI local VTEP info:
VLAN: 11
SVI if handler: 0x1A
Local VTEP: 10.1.1.2 2001:DB8:1::2 <-- Local primary and secondary VTEP IP
Local routing: Disabled
Core IRB info:
L3VNI: 30000
VRF name: red
VLAN: 3
V4TopoID: 0x2
V6TopoID: 0x1E000002
Local VTEP: 10.1.1.2 2001:DB8:1::2
SVI if handler: 0x19
SVI MAC: 0050.569A.A8BF
IPv4 TRM mdt group: N/A
IPv6 TRM mdt group: N/A
VNI Detailed statistics:
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
L3VNI Status in NVE
Verify Greenfield VXLANv6 L3VNI status in NVE
VTEP1#show nve vni 30000 detail
Interface VNI Multicast-group VNI state Mode VLAN cfg vrf
nve1 30000 N/A Up L3CP 3 CLI red
L3CP VNI IRB state: IPv4 up, IPv6 up
L3CP VNI TRM state: IPv4 down, IPv6 down
L3CP VNI local VTEP info:
VRF name: red
VLAN: 3
V4TopoID: 0x2
V6TopoID: 0x1E000002
Local VTEP: 2001:DB8:1::2
SVI if handler: 0x19
SVI MAC: 0050.569A.A8BF
IPv4 TRM mdt group: N/A
IPv6 TRM mdt group: N/A
VNI Detailed statistics:
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
Verify Dual-stack L3VNI in NVE
VTEP1#show nve vni 30000 detail
Interface VNI Multicast-group VNI state Mode VLAN cfg vrf
nve1 30000 N/A Up L3CP 3 CLI red
L3CP VNI IRB state: IPv4 up, IPv6 up
L3CP VNI TRM state: IPv4 down, IPv6 down
L3CP VNI local VTEP info:
VRF name: red
VLAN: 3
V4TopoID: 0x2
V6TopoID: 0x1E000002
Local VTEP: 10.1.1.2 2001:DB8:1::2
SVI if handler: 0x19
SVI MAC: 0050.569A.A8BF
IPv4 TRM mdt group: N/A
IPv6 TRM mdt group: N/A
VNI Detailed statistics:
Pkts In Bytes In Pkts Out Bytes Out
0 0 0 0
NVE Peer Information
Verify the NVE peers with these commands
VTEP1#show nve peers
'M' - MAC entry download flag 'A' - Adjacency download flag
'4' - IPv4 flag '6' - IPv6 flag
Interface VNI Type Peer-IP RMAC/Num_RTs eVNI state flags UP time
nve1 30000 L3CP 2001:DB8:3::2 0050.569a.1db3 30000 UP A/-/4 04:06:28
nve1 30000 L3CP 2001:DB8:2::2 0050.569a.89d8 30000 UP A/-/4 04:07:50
nve1 30000 L3CP 2001:DB8:3::2 0050.569a.1db3 30000 UP A/M/6 04:06:28
nve1 30000 L3CP 2001:DB8:2::2 0050.569a.89d8 30000 UP A/M/6 04:07:50
nve1 20011 L2CP 2001:DB8:2::2 6 20011 UP N/A 04:07:50
nve1 20011 L2CP 2001:DB8:3::2 4 20011 UP N/A 04:06:28
nve1 20012 L2CP 2001:DB8:2::2 7 20012 UP N/A 04:07:50
NVE peer is detected based on remote EVPN routes received from that peer.
- For peers on L2VNI, you can use this show command to get more information from EVPN Manager:
VTEP1#show l2vpn evpn peers vxlan detail
Interface: nve1
Local VNI: 20011
Peer VNI: 20011
Peer IP Address: 2001:DB8:2::2
UP time: 3d19h
Number of routes
EAD per-EVI: 0
MAC: 2
MAC/IP: 4
IMET: 1
Total: 7
Interface: nve1
Local VNI: 20011
Peer VNI: 20011
Peer IP Address: 2001:DB8:3::2
UP time: 3d19h
Number of routes
EAD per-EVI: 0
MAC: 1
MAC/IP: 2
IMET: 1
Total: 4
Interface: nve1
Local VNI: 20012
Peer VNI: 20012
Peer IP Address: 2001:DB8:2::2
UP time: 3d19h
Number of routes
EAD per-EVI: 0
MAC: 2
MAC/IP: 4
IMET: 0
Total: 6
Peer information on L3VNI are coming directly from BGP. These BGP show commands can be used to get more information on L3VNI peers:
show ip bgp l2vpn evpn route-type 5
show bgp vpnv4 unicast vrf <vrfname> <prefix>
show bgp vpnv6 unicast vrf <vrfname> <prefix>
show bgp l2vpn evpn rnh vrf <vrfname>
EVPN Instance in EVPN Manager
Verify Greenfield VXLANv6
VTEP1#show l2vpn evpn evi 1 detail
EVPN instance: 1 (VLAN Based)
RD: 10.1.1.3:1 (auto)
Import-RTs: 100:1
Export-RTs: 100:1
Per-EVI Label: none
State: Established
Replication Type: Ingress
Encapsulation: vxlan
IP Local Learn: Enabled (global)
Adv. Def. Gateway: Enabled (global)
Re-originate RT5: Disabled
Adv. Multicast: Enabled (global)
Vlan: 11
Protected: False
Ethernet-Tag: 0
State: Established
Flood Suppress: Attached
Core If: Vlan3
Access If: Vlan11
NVE If: nve1
RMAC: 0050.569a.a8bf
Core Vlan: 3
L2 VNI: 20011
L3 VNI: 30000
VTEP IP: 2001:DB8:1::2 <-- VTEP IP from NVE Manager
VRF: red
IPv4 IRB: Enabled
IPv6 IRB: Enabled
Pseudoports:
GigabitEthernet1/0/1 service instance 11
Routes: 1 MAC, 2 MAC/IP
Peers:
2001:DB8:2::2
Routes: 2 MAC, 3 MAC/IP, 1 IMET, 0 EAD
2001:DB8:3::2
Routes: 1 MAC, 2 MAC/IP, 1 IMET, 0 EAD
Verify Dual-stack
VTEP1#show l2vpn evpn evi 1 detail
EVPN instance: 1 (VLAN Based)
RD: 10.1.1.3:1 (auto)
Import-RTs: 100:1
Export-RTs: 100:1
Per-EVI Label: none
State: Established
Replication Type: Ingress
Encapsulation: vxlan
IP Local Learn: Enabled (global)
Adv. Def. Gateway: Enabled (global)
Re-originate RT5: Disabled
Adv. Multicast: Enabled (global)
Vlan: 11
Protected: False
Ethernet-Tag: 0
State: Established
Flood Suppress: Attached
Core If: Vlan3
Access If: Vlan11
NVE If: nve1
RMAC: 0050.569a.a8bf
Core Vlan: 3
L2 VNI: 20011
L3 VNI: 30000
VTEP IP: 10.1.1.2 <-- Primary VTEP IP from NVE Manager
Sec. VTEP IP: 2001:DB8:1::2 <-- Secondary VTEP IP from NVE Manager
VRF: red
IPv4 IRB: Enabled
IPv6 IRB: Enabled
Pseudoports:
GigabitEthernet1/0/1 service instance 11
Routes: 1 MAC, 2 MAC/IP
Peers:
2001:DB8:2::2
Routes: 2 MAC, 3 MAC/IP, 1 IMET, 0 EAD
2001:DB8:3::2
Routes: 1 MAC, 2 MAC/IP, 1 IMET, 0 EAD
Topology in L2RIB
Verify Greenfield VXLANv6
VTEP1#show l2rib topologies topology 1 detail
Topology ID : 100000000
EVI : 1
ETAG : 0
Topology Name : BD-11
Type : VxLAN
Producer : L2VPN
BD/VLAN-Id : 11
Layer-2 VNI : 20011
Downstream VNI : Enabled
VTEP Address : 2001:DB8:1::2
Mcast Address : UNKNOWN
Layer-3 VNI : 30000
Core BD/VLAN-Id : 3
Router MAC : 0050.569a.a8bf
VRF Name : red
Access IRB Interface : Vlan11
Core IRB Interface : Vlan3
IPv4 IRB : Enabled
IPv6 IRB : Enabled
Re-originate RT5 : Disabled
VerifyDual-stack
VTEP1#show l2rib topologies topology 1 detail
Topology ID : 100000000
EVI : 1
ETAG : 0
Topology Name : BD-11
Type : VxLAN
Producer : L2VPN
BD/VLAN-Id : 11
Layer-2 VNI : 20011
Downstream VNI : Enabled
VTEP Address : 10.1.1.2
Secondary VTEP Address : 2001:DB8:1::2
Mcast Address : UNKNOWN
Layer-3 VNI : 30000
Core BD/VLAN-Id : 3
Router MAC : 0050.569a.a8bf
VRF Name : red
Access IRB Interface : Vlan11
Core IRB Interface : Vlan3
IPv4 IRB : Enabled
IPv6 IRB : Enabled
Re-originate RT5 : Disabled
Local VTEP Information in BGP
Verify Greenfield VXLANv6
VTEP1#show bgp l2vpn evpn local-vtep vrf red
Local VTEP vrf red:
Protocol: IPv4
RMAC Address: 0050.569A.A8BF
VTEP-IP:2001:DB8:1::2
SEC-VTEP-IP:UNKNOWN
VNI: 30000
BDI:Vlan3
Protocol: IPv6
RMAC Address: 0050.569A.A8BF
VTEP-IP:2001:DB8:1::2
SEC-VTEP-IP:UNKNOWN
VNI: 30000
BDI:Vlan3
Verify Dual-stack
VTEP1#show bgp l2vpn evpn local-vtep vrf red
Local VTEP vrf red:
Protocol: IPv4
RMAC Address: 0050.569A.A8BF
VTEP-IP:10.1.1.2
SEC-VTEP-IP:2001:DB8:1::2
VNI: 30000
BDI:Vlan3
Protocol: IPv6
RMAC Address: 0050.569A.A8BF
VTEP-IP:10.1.1.2
SEC-VTEP-IP:2001:DB8:1::2
VNI: 30000
BDI:Vlan3
Remote RNH Information in BGP
VTEP1#show bgp l2vpn evpn rnh vrf red
Remote VTEP entries for vrf red:
Protocol: ipv4
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / 0050.569A.89D8 / 2001:DB8:2::2 / yes]
[30000 / 0050.569A.1DB3 / 2001:DB8:3::2 / yes]
Protocol: ipv6
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / 0050.569A.1DB3 / 2001:DB8:3::2 / yes]
[30000 / 0050.569A.89D8 / 2001:DB8:2::2 / yes]
Troubleshoot
BUM Traffic Forwarding (Ingress Replication)
This section provides an example to troubleshoot BUM traffic from VTEP1 to VTEP2 in EVI 1 where replication type is configured as ingress.
Verify Replication Type is set to Ingress for the EVI in EVPN Manager on both VTEPs
VTEP1#show l2vpn evpn evi 1 detail
EVPN instance: 1 (VLAN Based)
RD: 10.1.1.3:1 (auto)
Import-RTs: 100:1
Export-RTs: 100:1
Per-EVI Label: none
State: Established
Replication Type: Ingress
<...snip...>
Verify L2RIB has the local IMET route for the EVI from EVPN Manager on egress VTEP
VTEP2#show l2route evpn imet topology 1 producer L2VPN detail
EVPN Instance: 1
Ethernet Tag: 0
Producer Name: L2VPN
Router IP Addr: 10.2.2.3
Route Ethernet Tag: 0
Tunnel Flags: 0
Tunnel Type: Ingress Replication
Tunnel Labels: 20011
Tunnel ID: 2001:DB8:2::2
Multicast Proxy: IGMP,MLD
Next Hop(s): N/A
Verify local IMET route in BGP on egress VTEP
Greenfield VXLANv6
VTEP2#show ip bgp l2vpn evpn route-type 3 0 10.2.2.3
BGP routing table entry for [3][10.2.2.3:1][0][32][10.2.2.3]/17, version 15
Paths: (1 available, best #1, table evi_1)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.2.2.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 16, vni:20011 tunnel identifier: 0000 0000
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.89D8
core-irb interface:Vlan3
vtep-ip:2001:DB8:2::2
sec-vtep-ip:UNKNOWN
rx pathid: 0, tx pathid: 0x0
Updated on Apr 7 2022 23:37:11 UTC
Dual-stack
VTEP2#show ip bgp l2vpn evpn route-type 3 0 10.2.2.3
show ip bgp l2vpn evpn route-type 3 0 10.2.2.3
BGP routing table entry for [3][10.2.2.3:1][0][32][10.2.2.3]/17, version 9
Paths: (1 available, best #1, table evi_1)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.2.2.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute added with secondary VTEP IP
Encap type: 8
Secondary nexthop address 2001:DB8:2::2(active)
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, vni:20011 tunnel identifier: 0000 0000
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.89D8
core-irb interface:Vlan3
vtep-ip:10.2.2.2
sec-vtep-ip:2001:DB8:2::2
rx pathid: 0, tx pathid: 0x0
Verify remote IMET route is received and imported to EVI table in BGP on ingress VTEP
Greenfield VXLANv6
VTEP1#show ip bgp l2vpn evpn route-type 3 0 10.2.2.3
BGP routing table entry for [3][10.1.1.3:1][0][32][10.2.2.3]/17, version 30
Paths: (1 available, best #1, table evi_1)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [3][10.2.2.3:1][0][32][10.2.2.3]/17 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
Originator: 10.2.2.1, Cluster list: 10.99.99.99
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 16, vni:20011 tunnel identifier: < Tunnel Endpoint: 2001:DB8:2::2 >
rx pathid: 0, tx pathid: 0x0
Updated on Apr 7 2022 23:37:17 UTC
BGP routing table entry for [3][10.2.2.3:1][0][32][10.2.2.3]/17, version 23
Paths: (1 available, best #1, table EVPN-BGP-Table)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
Local
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
Originator: 10.2.2.1, Cluster list: 10.99.99.99
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 16, vni:20011 tunnel identifier: < Tunnel Endpoint: 2001:DB8:2::2 >
rx pathid: 0, tx pathid: 0x0
Updated on Apr 7 2022 23:37:17 UTC
Dual-stack
VTEP1#show ip bgp l2vpn evpn route-type 3 0 10.2.2.3
BGP routing table entry for [3][10.2.2.3:1][0][32][10.2.2.3]/17, version 22
Paths: (1 available, best #1, table EVPN-BGP-Table)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
Local
10.2.2.2 (metric 3) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:2::2(active)
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, vni:20011 tunnel identifier: < Tunnel Endpoint: 10.2.2.2 >
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:03:49 UTC
BGP routing table entry for [3][10.1.1.3:1][0][32][10.2.2.3]/17, version 31
Paths: (1 available, best #1, table evi_1)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [3][10.2.2.3:1][0][32][10.2.2.3]/17 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1 ENCAP:8 EVPN Mcast Flags:3
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute received from remote dual-stack VTEP
Encap type: 8
Secondary nexthop address 2001:DB8:2::2
PMSI Attribute: Flags:0x0, Tunnel type:IR, length 4, vni:20011 tunnel identifier: < Tunnel Endpoint: 10.2.2.2 >
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:03:49 UTC
Note: Note: In the case of dual-nexthop IMET routes, PMSI tunnel attribute in the EVI imported route has its primary nexthop address as Tunnel identifier. BGP replaces that with secondary nexthop address from the Tunnel encap attribute if the local preference is that of the tunnel encap nexthop address-family. In this example, though BGP shows Tunnel Endpoint as 10.2.2.2, while installing into L2RIB, it replaces with 2001:DB8:2::2. This is done because BGP does not change the attribute entry in the attribute table.
Verify remote IMET route is received in L2RIB on ingress VTEP
VTEP1#show l2route evpn imet topology 1 producer bgp origin-rtr 10.2.2.3 detail
EVPN Instance: 1
Ethernet Tag: 0
Producer Name: BGP
Router IP Addr: 10.2.2.3
Route Ethernet Tag: 0
Tunnel Flags: 0
Tunnel Type: Ingress Replication
Tunnel Labels: 20011
Tunnel ID: 2001:DB8:2::2
Multicast Proxy: IGMP,MLD
Next Hop(s): V:0 2001:DB8:2::2
Verify entry for the egress VTEP is in the flood list in L2FIB on ingress VTEP
VTEP1#show l2fib bridge-domain 11 detail
Bridge Domain : 11
Reference Count : 12
Replication ports count : 3
Unicast Address table size : 2
IP Multicast Prefix table size : 1
Flood List Information :
Olist: 1035, Ports: 3 <-- Use this value in the output-list command
Port Information :
BD_PORT Gi1/0/1:11
VXLAN_REP PL:2(1) T:VXLAN_REP [IR]20011:2001:DB8:2::2
VXLAN_REP PL:9(1) T:VXLAN_REP [IR]20011:2001:DB8:3::2
Unicast Address table information :
aabb.0000.0021 VXLAN_UC PL:1(1) T:VXLAN_UC [MAC]20011:2001:DB8:2::2
aabb.0000.0031 VXLAN_UC PL:12(1) T:VXLAN_UC [MAC]20011:2001:DB8:3::2
IP Multicast Prefix table information :
Source: *, Group: 239.21.21.21, IIF: Null, Adjacency: Olist: 6156, Ports: 1
VTEP1#show l2fib output-list 1035 <-- From the previous command
ID : 1035
Bridge Domain : 11
Reference Count : 1
Flags : flood list
Port Count : 3
Port(s) : BD_PORT Gi1/0/1:11
: VXLAN_REP PL:2(1) T:VXLAN_REP [IR]20011:2001:DB8:2::2
: VXLAN_REP PL:9(1) T:VXLAN_REP [IR]20011:2001:DB8:3::2
VTEP1#show l2fib path-list 2 detail
VXLAN_REP Pathlist 2: topo 11, 1 paths, none
ESI: 0000.0000.0000.0000.0000
Originator: 10.2.2.3
path 2001:DB8:2::2, type VXLAN, evni 20011, vni 20011, source IR
forwarding oce 0x7F262ED39BF8 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:2::2, cid: 1
output chain:
oce type: evpn_vxlan_encap, sw_handle 0x7F262F4849D0
forwarding oce 0x7F262ED39BF8 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:2::2, cid: 1
Verify Platform Dependent (PD) adjacency on ingress VTEP
VTEP1#show platform software fed switch active matm adjacencies vlan 11
VLAN ADJ_ID ADJ_KEY Encap Link siHandle riHandle l3mriHandle diHandle Obj_type Shared
----------------------------------------------------------------------------------------------------------------------------------------
11 15 0x30000000f VXLAN V6 0x7fb4687f45f8 0x7fb4687e9be8 0x0 0x0 UC Yes
11 18 0x300000012 VXLAN V6 0x7fb4687ee058 0x7fb4687ee7a8 0x0 0x0 UC Yes
11 148 0x200000094 VXLAN V6 0x0 0x7fb4687eb9e8 0x0 0x0 IR No
11 149 0x200000095 VXLAN V6 0x0 0x7fb4687eb158 0x0 0x0 IR No
VTEP1#show platform software fed switch active matm adjacencies vlan 11 | inc IR
11 148 0x200000094 VXLAN V6 0x0 0x7fb4687eb9e8 0x0 0x0 IR No
11 149 0x200000095 VXLAN V6 0x0 0x7fb4687eb158 0x0 0x0 IR No
VTEP1#show plat hard fed sw active fwd abs print 0x7fb4687eb9e8 1 <-- first Value from matm adjacencies command
Handle:0x7fb4687eb9e8 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: index0:0x36 mtu_index/l3u_ri_index0:0x0 index1:0x36 mtu_index/l3u_ri_index1:0x0
Features sharing this resource:58 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 00 00 00 00 07 00 94 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0) <-- First ASIC instance
----------------------------------------
ASIC#:0 RI:54 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2 <-- Source VTEP in hardware
Dst IPv6: 2001:DB8:2::2 <-- Dest VTEP in hardware
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 43 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
Detailed Resource Information (ASIC_INSTANCE# 1) <-- Second ASIC instance
----------------------------------------
ASIC#:1 RI:54 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2 <-- Source VTEP in hardware
Dst IPv6: 2001:DB8:2::2 <-- Dest VTEP in hardware
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 43 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
==============================================================
VTEP1#show plat hard fed sw active fwd abs print 0x7fb4687eb158 1 <-- Second Value from matm adjacencies command
Handle:0x7fb4687eb158 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: index0:0x24 mtu_index/l3u_ri_index0:0x0 index1:0x24 mtu_index/l3u_ri_index1:0x0
Features sharing this resource:58 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 00 00 00 00 07 00 95 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0) <-- First ASIC instance
----------------------------------------
ASIC#:0 RI:36 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2 <-- Source VTEP in hardware
Dst IPv6: 2001:DB8:3::2 <-- Dest VTEP in hardware
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 43 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
Detailed Resource Information (ASIC_INSTANCE# 1) <-- First ASIC instance
----------------------------------------
ASIC#:1 RI:36 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2 <-- Source VTEP in hardware
Dst IPv6: 2001:DB8:3::2 <-- Dest VTEP in hardware
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 43 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
==============================================================
Verify PD Flood list on ingress VTEP
VTEP1#show plat hard fed sw act vlan 11 ingress
VLAN STP State in hardware
vlan id is:: 11 <-- Vlan 11
Interfaces in forwarding state: : Gi1/0/1(Tagged)
flood list: : Gi1/0/1 <-- port present in flood list
REP RI Handle: : 0x00007fb4687ea3c8 <-- Replication Index (used to replicate/flood)
VTEP1#show plat hard fed sw act fwd abs print 0x00007fb4687ea3c8 1
Handle:0x7fb4687ea3c8 Res-Type:ASIC_RSC_RI_REP Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_LISP Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: index0:0x10 mtu_index/l3u_ri_index0:0x0 index1:0x10 mtu_index/l3u_ri_index1:0x0
Cookie length: 56
00 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0) <-- first ASIC instance
----------------------------------------
Replication list RI handle = 7fb4687ea3c8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASIC [0] Replication Expansion Handle [0x7fb4687ea7c8]
Replication list :
Number of RIs = 6
Start RI = 16
Common rewrite = Yes
Replication REP_RI 0x10 [elements = 2]
[0] ri[0]=2 RI_L2 port=78 ri_ref_count:1 dirty=0
[1] ri_list[1]=4 RI_MCAST_BRIDGE_V6 port=88 ri_ref_count:1 dirty=0
RIL first:4 last:4 start:4 ril_total:4 ref_count:0
RI list this:4 num_pairs:4 free:3 next:0 prev:0 ---->
uri1:36 ri_ref_count_1:1 uri0:54 ri_ref_count_0:1 ptr_type:0 last:1 dirty:1
uri1:49151 ri_ref_count_1:0 uri0:49151 ri_ref_count_0:0 ptr_type:1 last:1 dirty:0
uri1:49151 ri_ref_count_1:0 uri0:49151 ri_ref_count_0:0 ptr_type:1 last:1 dirty:0
uri1:49151 ri_ref_count_1:0 uri0:49151 ri_ref_count_0:0 ptr_type:1 last:1 dirty:0
<-- URI represents multiple copies are made with different encap. URI0 always indicate encap index which can be dumped using
“show plat hard fed sw active fwd-asic resource asic 0 rewrite-index range <uri value> <uri value>”
URI1 also indicates a encap index if “last” value is 1. If “last” value is zero, then URI1 just points to next encap list.
Detailed Resource Information (ASIC_INSTANCE# 1) <-- second ASIC instance (if port is on first instance this may not be programmed)
----------------------------------------
Replication list RI handle = 7fb4687ea3c8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASIC [1] Replication Expansion Handle [0x7fb4687ea838]
Replication list :
Number of RIs = 6
Start RI = 16
Common rewrite = Yes
Replication REP_RI 0x10 [elements = 1]
[0] ri[0]=2 RI_L2 port=78 ri_ref_count:1 dirty=0
Verify Decap programming on egress VTEP
- There is one tunnel decap entry per NVE interface programmed on the platform. Entry must have vxlan port “0x12b5” (port 4789) and UDP protocol “0x11”.
VTEP2#show platform software fed switch active fwd-asic resource tcam table tunnel | inc 12b5
V: 00110000 000012b5 00000000 00000000 00000000 00000000 00000000
Value: 00000000000000000000000000000000 12b5 000 11 0 0 00 00 0 <-- ASIC 0
V: 00000000 00000000 000012b5 00000000 00000000
Value: 00000000 00000000 12b5 000 00 0 0 00 00 0
V: 00000000 000012b5 00000000 00000000 00000000 00000000 00000000
Value: 00000000000000000000000000000000 12b5 000 00 0 0 00 00 0 <-- ASIC 1
One VNI-VLAN translation entry per VNI programmed on the platform
VTEP2#show platform hardware fed switch active fwd-asic resource tcam table vnid_trans
Printing entries for region LISP_INST_TRANS (407) type 6 asic 0
========================================================
TAQ-3 Index-512 (A:0,C:0) Valid StartF-1 StartA-1 SkipF-0 SkipA-0
Labels Port Vlan L3If Group
M: 0000 0000 0000 0000
V: 0000 0000 0000 0000
M: 00ff0000 0000ffff ffffff00 00000000
V: 00110000 000012b5 004e8500 00000000
Lisp VRF L3P DPort InstId
Mask 000 ff ffff ffffff00
Value 000 11 12b5 004e8500 <-- hex = UDP port 4789 & VNI 5145856
Action: 00000001 00000000 00000000 00000000 00650000 00000006 000000ed 00000000
00000000 00000000
lvxTranslationValid: 1
lvxTranslatedInstanceIsL3if: 0
l3if_l3_handle: 0
lvxTranslatedInstanceId: 0x065 <- for L2 VNI, this value is the vlan that is mapped to VNI. For L3 vni, this represents internal l3 interface id corresponding to the l3 vni.
priority: 6
SI: 0xed
BUM Traffic Forwarding (Multicast Replication)
This section provides an example to troubleshoot BUM traffic from VTEP1 to VTEP2 in EVI 2 where replication type is configured as static.
Verify Replication Type is set to Static for the EVI in EVPN Manager on both VTEPs
-
Both VTEPs must be configured with the same underlay multicast group for the same EVI. The underlay multicast group information comes from NVE. Check NVE interface configuration if it is not set properly.
VTEP1#show l2vpn evpn evi 2 detail
EVPN instance: 2 (VLAN Based)
RD: 10.1.1.3:2 (auto)
Import-RTs: 100:2
Export-RTs: 100:2
Per-EVI Label: none
State: Established
Replication Type: Static
Encapsulation: vxlan
IP Local Learn: Enabled (global)
Adv. Def. Gateway: Enabled (global)
Re-originate RT5: Disabled
Adv. Multicast: Enabled (global)
Vlan: 12
Protected: False
Ethernet-Tag: 0
State: Established
Flood Suppress: Attached
Core If: Vlan3
Access If: Vlan12
NVE If: nve1
RMAC: 0050.569a.a8bf
Core Vlan: 3
L2 VNI: 20012
L3 VNI: 30000
VTEP IP: 2001:DB8:1::2
MCAST IP: FF0E::11
VRF: red
IPv4 IRB: Enabled
IPv6 IRB: Enabled
Pseudoports:
GigabitEthernet1/0/2 service instance 12
Routes: 1 MAC, 1 MAC/IP
Peers:
2001:DB8:2::2
Routes: 2 MAC, 3 MAC/IP, 0 IMET, 0 EAD
Verify L2RIB has the local IMET route for the EVI from producer Static on ingress VTEP
VTEP1#show l2route evpn imet topology 2 producer static detail
EVPN Instance: 2
Ethernet Tag: 0
Producer Name: Static
Router IP Addr: 10.1.1.3
Route Ethernet Tag: 0
Tunnel Flags: 0
Tunnel Type: No tunnel information present
Tunnel Labels: 20012
Tunnel ID: FF0E::11
Multicast Proxy: IGMP,MLD
Next Hop(s): N/A
Verify entry for the underlay multicast group is in the flood list in L2FIB on ingress VTEP
VTEP1#show l2fib bridge-domain 12 detail
Bridge Domain : 12
Reference Count : 13
Replication ports count : 2
Unicast Address table size : 1
IP Multicast Prefix table size : 3
Flood List Information :
Olist: 1036, Ports: 2 <-- Use in the next output-list command
Port Information :
BD_PORT Gi1/0/2:12
VXLAN_REP PL:12(1) T:VXLAN_REP [SMC]20012:FF0E::11
Unicast Address table information :
aabb.0000.0022 VXLAN_UC PL:26(1) T:VXLAN_UC [MAC]20012:2001:DB8:2::2
IP Multicast Prefix table information :
Source: *, Group: *, IIF: Null, Adjacency: Olist: 6157, Ports: 1
Source: *, Group: 239.21.21.21, IIF: Null, Adjacency: Olist: 6159, Ports: 2
Source: ::, Group: ::, IIF: Null, Adjacency: Olist: 6158, Ports: 1
VTEP1#show l2fib output-list 1036 <-- From the bridge-domain command
ID : 1036
Bridge Domain : 12
Reference Count : 1
Flags : flood list
Port Count : 2
Port(s) : BD_PORT Gi1/0/2:12
: VXLAN_REP PL:12(1) T:VXLAN_REP [SMC]20012:FF0E::11
VTEP1#show l2fib path-list 12 detail
VXLAN_REP Pathlist 12: topo 12, 1 paths, none
ESI: 0000.0000.0000.0000.0000
Originator: 10.1.1.3
path FF0E::11, type VXLAN, evni 20012, vni 20012, source SMC
forwarding oce 0x7FA987EC3F48 type adjacency, IPV6 midchain out of Tunnel0, addr FF0E::11, cid: 1
output chain:
oce type: evpn_vxlan_encap, sw_handle 0x7FA988938778
forwarding oce 0x7FA987EC3F48 type adjacency, IPV6 midchain out of Tunnel0, addr FF0E::11, cid: 1
Verify PIM neighbor on Spine
- In this output the first entry denotes pim neighborship with VTEP1 & the second entry denotes pim neighborship with VTEP2
SPINE#show ipv6 pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, G - GenID Capable
Neighbor Address Interface Uptime Expires Mode DR pri
FE80::822D:BFFF:FE9B:84C6 Te1/1/2 00:50:50 00:01:19 B G DR 1 <-- VTEP1
FE80::C214:FEFF:FEC6:D7C6 Gi1/0/1 00:52:32 00:01:34 B G DR 1 <-- VTEP2
Verify PIM neighbor on ingress VTEP
- In this output, the entry denotes pim neighborship with Spine
VTEP1#show ipv6 pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, G - GenID Capable
Neighbor Address Interface Uptime Expires Mode DR pri
FE80::822D:BFFF:FE7B:1DC8 Te1/1/1 00:53:06 00:01:32 B G 1 <-- SPINE
Verify PIM neighbor on egress VTEP
- In this output, the entry denotes pim neighborship with Spine
VTEP2#show ipv6 pim neighbor
PIM Neighbor Table
Mode: B - Bidir Capable, G - GenID Capable
Neighbor Address Interface Uptime Expires Mode DR pri
FE80::822D:BFFF:FE7B:1DE4 Te1/1/1 00:55:03 00:01:26 B G 1 <-- SPINE
Verify the rpf interface toward the RP on Spine
SPINE#show ipv6 rpf 2001:DB8::99:99
RPF information for 2001:DB8::99:99
RPF interface: Loopback0
RPF neighbor: 2001:DB8::99:99- local
RPF route/mask: 2001:DB8::99:99/128
RPF type: Unicast
RPF recursion count: 0
Metric preference: 0
Metric: 0
Verify the rpf interface toward the RP on ingress VTEP
VTEP1#show ipv6 rpf 2001:DB8::99:99
RPF information for 2001:DB8::99:99
RPF interface: TenGigabitEthernet1/1/1
RPF neighbor: FE80::822D:BFFF:FE7B:1DC8
RPF route/mask: 2001:DB8::99:99/128
RPF type: Unicast
RPF recursion count: 0
Metric preference: 110
Metric: 1
Verify the rpf interface toward the RP on egress VTEP
VTEP2#show ipv6 rpf 2001:DB8::99:99
RPF information for 2001:DB8::99:99
RPF interface: TenGigabitEthernet1/1/1
RPF neighbor: FE80::822D:BFFF:FE7B:1DE4
RPF route/mask: 2001:DB8::99:99/128
RPF type: Unicast
RPF recursion count: 0
Metric preference: 110
Metric: 1
Verify PIM topology entry for the underlay multicast BUM group on the ingress VTEP
VTEP1#show ipv6 pim topology ff05::1
IP PIM Multicast Topology Table
Entry state: (*/S,G)[RPT/SPT] Protocol Uptime Info Upstream Mode
Entry flags: KAT - Keep Alive Timer, AA - Assume Alive, PA - Probe Alive,
RA - Really Alive, LH - Last Hop, DSS - Don't Signal Sources,
RR - Register Received, SR - Sending Registers, E - MSDP External,
DCC - Don't Check Connected, Y - Joined MDT-data group,
y - Sending to MDT-data group
BGS - BGP Signal Sent, !BGS - BGP signal suppressed
SAS - BGP Src-Act Sent, SAR - BGP Src-Act Received
PFA - PFP-SA announced, PFC - PFP-SA cache created
Interface state: Name, Uptime, Fwd, Info
Interface flags: LI - Local Interest, LD - Local Disinterest,
II - Internal Interest, ID - Internal Disinterest,
LH - Last Hop, AS - Assert, AB - Admin Boundary, BS - BGP Signal,
BP - BGP Shared-Tree Prune, BPT - BGP Prune Time
(*,FF05::1)
SM UP: 00:20:56 JP: Join(now) Flags: LH
RP: 2001:DB8::99:99
RPF: TenGigabitEthernet1/1/1,FE80::822D:BFFF:FE7B:1DC8
Tunnel0 , 00:20:56 fwd LI II LH
(2001:DB8::1:1,FF05::1)
SM SPT UP: 00:20:24 JP: Join(never) Flags: KAT(00:00:55) RA SR
RPF: Loopback0,FE80::822D:BFFF:FE9B:8480*
Te1/1/1 , 00:16:37 fwd Join(00:03:01)
Verify the mroute entry for the underlay multicast BUM group on the ingress VTEP
VTEP1#show ipv6 mroute ff05::2
Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group,
C - Connected, L - Local, I - Received Source Specific Host Report,
P - Pruned, R - RP-bit set, F - Register flag, T - SPT-bit set,
J - Join SPT, Y - Joined MDT-data group,
y - Sending to MDT-data group
g - BGP signal originated, G - BGP Signal received,
N - BGP Shared-Tree Prune received, n - BGP C-Mroute suppressed,
q - BGP Src-Active originated, Q - BGP Src-Active received
E - Extranet
Timers: Uptime/Expires
Interface state: Interface, State
(*, FF05::2), 00:04:17/never, RP 2001:DB8::99:99, flags: SCLJ
Incoming interface: TenGigabitEthernet1/1/1
RPF nbr: FE80::822D:BFFF:FE7B:1DC8
Immediate Outgoing interface list:
Tunnel0, Forward, 00:04:17/never
(2001:DB8::1:1, FF05::2), 00:01:25/00:02:04, flags: SFJT
Incoming interface: Loopback0
RPF nbr: FE80::822D:BFFF:FE9B:8480
Immediate Outgoing interface list:
TenGigabitEthernet1/1/1, Forward, 00:01:25/00:03:10
Inherited Outgoing interface list:
Tunnel0, Forward, 00:04:17/never
Verify the mrib entry for the underlay multicast BUM group on ingress VTEP
VTEP1#show ipv6 mrib route ff05::1
IP Multicast Routing Information Base
Entry flags: L - Domain-Local Source, E - External Source to the Domain,
C - Directly-Connected Check, S - Signal, IA - Inherit Accept, D - Drop
ET - Data Rate Exceeds Threshold,K - Keepalive,DDE - Data Driven Event
ME - MoFRR ECMP Flow based, MNE - MoFRR Non-ECMP Flow based,
MP - Primary MoFRR Non-ECMP Flow based entry,
e - Encap helper tunnel flag
Interface flags: F - Forward, A - Accept, IC - Internal Copy,
NS - Negate Signal, DP - Don't Preserve, SP - Signal Present,
II - Internal Interest, ID - Internal Disinterest, LI - Local Interest,
LD - Local Disinterest, MD - mCAC Denied, MI - mLDP Interest
A2 - MoFRR ECMP Backup Accept
(*,FF05::1) RPF nbr: FE80::822D:BFFF:FE7B:1DC8 Flags: C
TenGigabitEthernet1/1/1 Flags: A NS
Tunnel0 Flags: F IC II LI NS
(2001:DB8::1:1,FF05::1) RPF nbr: FE80::822D:BFFF:FE9B:8480 Flags:
Null0 Flags: A
TenGigabitEthernet1/1/1 Flags: F NS
Verify the mfib entry for the underlay multicast BUM group on the ingress VTEP
VTEP1#show ipv6 mfib ff05::1
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
ET - Data Rate Exceeds Threshold, K - Keepalive
DDE - Data Driven Event, HW - Hardware Installed
ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB
MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary
MS - MoFRR Entry in Sync, MC - MoFRR entry in MoFRR Client,
e - Encap helper tunnel flag.
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
NS - Negate Signalling, SP - Signal Present,
A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
MA - MFIB Accept, A2 - Accept backup,
RA2 - MRIB Accept backup, MA2 - MFIB Accept backup
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
I/O Item Counts: HW Pkt Count/FS Pkt Count/PS Pkt Count Egress Rate in pps
Default
(*,FF05::1) Flags: C HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 1/0/146/0, Other: 0/0/0
TenGigabitEthernet1/1/1 Flags: A NS
Tunnel0, VXLAN v6 Decap Flags: F IC NS
Pkts: 0/0/0 Rate: 0 pps
(2001:DB8::1:1,FF05::1) Flags: HW
SW Forwarding: 1/0/116/0, Other: 571/571/0
HW Forwarding: 632/1/175/1, Other: 0/0/0
Null0 Flags: A
TenGigabitEthernet1/1/1 Flags: F NS
Pkts: 0/0/0 Rate: 0 pps
Verify the PIM topology entry for the underlay multicast BUM group on the egress VTEP
VTEP2#show ipv6 pim topology ff05::1
IP PIM Multicast Topology Table
Entry state: (*/S,G)[RPT/SPT] Protocol Uptime Info Upstream Mode
Entry flags: KAT - Keep Alive Timer, AA - Assume Alive, PA - Probe Alive,
RA - Really Alive, LH - Last Hop, DSS - Don't Signal Sources,
RR - Register Received, SR - Sending Registers, E - MSDP External,
DCC - Don't Check Connected, Y - Joined MDT-data group,
y - Sending to MDT-data group
BGS - BGP Signal Sent, !BGS - BGP signal suppressed
SAS - BGP Src-Act Sent, SAR - BGP Src-Act Received
PFA - PFP-SA announced, PFC - PFP-SA cache created
Interface state: Name, Uptime, Fwd, Info
Interface flags: LI - Local Interest, LD - Local Disinterest,
II - Internal Interest, ID - Internal Disinterest,
LH - Last Hop, AS - Assert, AB - Admin Boundary, BS - BGP Signal,
BP - BGP Shared-Tree Prune, BPT - BGP Prune Time
(*,FF05::1)
SM UP: 00:00:42 JP: Join(00:00:16) Flags: LH
RP: 2001:DB8::99:99
RPF: TenGigabitEthernet1/1/1,FE80::822D:BFFF:FE7B:1DE4
Tunnel0 , 00:00:42 fwd LI II LH
(2001:DB8::1:1,FF05::1)
SM SPT UP: 00:00:42 JP: Join(00:00:16) Flags: KAT(00:02:47) RA
RPF: TenGigabitEthernet1/1/1,FE80::822D:BFFF:FE7B:1DE4
No interfaces in immediate olist
Verify the mroute entry for the underlay multicast BUM group on the egress VTEP
VTEP2#show ipv6 mroute ff05::1
Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group,
C - Connected, L - Local, I - Received Source Specific Host Report,
P - Pruned, R - RP-bit set, F - Register flag, T - SPT-bit set,
J - Join SPT, Y - Joined MDT-data group,
y - Sending to MDT-data group
g - BGP signal originated, G - BGP Signal received,
N - BGP Shared-Tree Prune received, n - BGP C-Mroute suppressed,
q - BGP Src-Active originated, Q - BGP Src-Active received
E - Extranet
Timers: Uptime/Expires
Interface state: Interface, State
(*, FF05::1), 00:00:47/never, RP 2001:DB8::99:99, flags: SCLJ
Incoming interface: TenGigabitEthernet1/1/1
RPF nbr: FE80::822D:BFFF:FE7B:1DE4
Immediate Outgoing interface list:
Tunnel0, Forward, 00:00:47/never
(2001:DB8::1:1, FF05::1), 00:00:47/00:02:42, flags: SJT
Incoming interface: TenGigabitEthernet1/1/1
RPF nbr: FE80::822D:BFFF:FE7B:1DE4
Inherited Outgoing interface list:
Tunnel0, Forward, 00:00:47/never
Verify the mrib entry for the underlay multicast BUM group on the egress VTEP
VTEP2#show ipv6 mrib route ff05::1
IP Multicast Routing Information Base
Entry flags: L - Domain-Local Source, E - External Source to the Domain,
C - Directly-Connected Check, S - Signal, IA - Inherit Accept, D - Drop
ET - Data Rate Exceeds Threshold,K - Keepalive,DDE - Data Driven Event
ME - MoFRR ECMP Flow based, MNE - MoFRR Non-ECMP Flow based,
MP - Primary MoFRR Non-ECMP Flow based entry,
e - Encap helper tunnel flag
Interface flags: F - Forward, A - Accept, IC - Internal Copy,
NS - Negate Signal, DP - Don't Preserve, SP - Signal Present,
II - Internal Interest, ID - Internal Disinterest, LI - Local Interest,
LD - Local Disinterest, MD - mCAC Denied, MI - mLDP Interest
A2 - MoFRR ECMP Backup Accept
(*,FF05::1) RPF nbr: FE80::822D:BFFF:FE7B:1DE4 Flags: C
Tunnel0 Flags: F IC II LI NS
TenGigabitEthernet1/1/1 Flags: A NS
(2001:DB8::1:1,FF05::1) RPF nbr: FE80::822D:BFFF:FE7B:1DE4 Flags:
TenGigabitEthernet1/1/1 Flags: A
Tunnel0 Flags: F IC NS
Verify mfib entry for the underlay multicast BUM group on egress VTEP
VTEP2#show ipv6 mfib ff05::1
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
ET - Data Rate Exceeds Threshold, K - Keepalive
DDE - Data Driven Event, HW - Hardware Installed
ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB
MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary
MS - MoFRR Entry in Sync, MC - MoFRR entry in MoFRR Client,
e - Encap helper tunnel flag.
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
NS - Negate Signalling, SP - Signal Present,
A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
MA - MFIB Accept, A2 - Accept backup,
RA2 - MRIB Accept backup, MA2 - MFIB Accept backup
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
I/O Item Counts: HW Pkt Count/FS Pkt Count/PS Pkt Count Egress Rate in pps
Default
(*,FF05::1) Flags: C HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 0/0/0/0, Other: 0/0/0
TenGigabitEthernet1/1/1 Flags: A NS
Tunnel0, VXLAN v6 Decap Flags: F IC NS
Pkts: 0/0/0 Rate: 0 pps
(2001:DB8::1:1,FF05::1) Flags: HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 74/1/170/1, Other: 0/0/0
TenGigabitEthernet1/1/1 Flags: A
Tunnel0, VXLAN v6 Decap Flags: F IC NS
Pkts: 0/0/0 Rate: 0 pps
Verify the pim topolog' entry for the underlay multicast BUM group on the Spine
SPINE#show ipv6 pim topology ff05::1
IP PIM Multicast Topology Table
Entry state: (*/S,G)[RPT/SPT] Protocol Uptime Info Upstream Mode
Entry flags: KAT - Keep Alive Timer, AA - Assume Alive, PA - Probe Alive,
RA - Really Alive, LH - Last Hop, DSS - Don't Signal Sources,
RR - Register Received, SR - Sending Registers, E - MSDP External,
DCC - Don't Check Connected, Y - Joined MDT-data group,
y - Sending to MDT-data group
BGS - BGP Signal Sent, !BGS - BGP signal suppressed
SAS - BGP Src-Act Sent, SAR - BGP Src-Act Received
PFA - PFP-SA announced, PFC - PFP-SA cache created
Interface state: Name, Uptime, Fwd, Info
Interface flags: LI - Local Interest, LD - Local Disinterest,
II - Internal Interest, ID - Internal Disinterest,
LH - Last Hop, AS - Assert, AB - Admin Boundary, BS - BGP Signal,
BP - BGP Shared-Tree Prune, BPT - BGP Prune Time
(*,FF05::1)
SM UP: 00:09:33 JP: Join(00:00:27) Flags:
RP: 2001:DB8::99:99*
RPF: Tunnel1,2001:DB8::99:99*
Te1/1/2 , 00:09:33 fwd Join(00:02:58)
Gi1/0/1 , 00:09:24 fwd Join(00:03:09)
(2001:DB8::1:1,FF05::1)
SM RPT UP: 00:09:33 JP: Prune(never) Flags: KAT(00:03:29) RA RR
RP: 2001:DB8::99:99*
RPF: Tunnel1,2001:DB8::99:99*
Te1/1/2 , 00:09:33 off Prune(00:02:58)
(2001:DB8::1:1,FF05::1)
SM SPT UP: 00:09:33 JP: Join(00:00:38) Flags: KAT(00:03:29) RA RR
RPF: TenGigabitEthernet1/1/2,FE80::822D:BFFF:FE9B:84C6
Gi1/0/1 , 00:09:24 fwd Join(00:03:09)
Verify the mroute entry for the underlay multicast BUM group on the Spine
SPINE#show ipv6 mroute ff05::1
Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group,
C - Connected, L - Local, I - Received Source Specific Host Report,
P - Pruned, R - RP-bit set, F - Register flag, T - SPT-bit set,
J - Join SPT, Y - Joined MDT-data group,
y - Sending to MDT-data group
g - BGP signal originated, G - BGP Signal received,
N - BGP Shared-Tree Prune received, n - BGP C-Mroute suppressed,
q - BGP Src-Active originated, Q - BGP Src-Active received
E - Extranet
Timers: Uptime/Expires
Interface state: Interface, State
(*, FF05::1), 00:09:43/00:03:00, RP 2001:DB8::99:99, flags: S
Incoming interface: Tunnel1
RPF nbr: 2001:DB8::99:99
Immediate Outgoing interface list:
TenGigabitEthernet1/1/2, Forward, 00:09:43/00:02:49
GigabitEthernet1/0/1, Forward, 00:09:34/00:03:00
(2001:DB8::1:1, FF05::1), 00:09:42/00:03:29, RP 2001:DB8::99:99, flags: SR
Incoming interface: Tunnel1
RPF nbr: 2001:DB8::99:99
Immediate Outgoing interface list:
TenGigabitEthernet1/1/2, Null, 00:09:42/00:02:49
Inherited Outgoing interface list:
GigabitEthernet1/0/1, Forward, 00:09:34/00:03:00
(2001:DB8::1:1, FF05::1), 00:09:42/00:03:29, flags: ST
Incoming interface: TenGigabitEthernet1/1/2
RPF nbr: FE80::822D:BFFF:FE9B:84C6
Immediate Outgoing interface list:
GigabitEthernet1/0/1, Forward, 00:09:33/00:03:00
Verify the mrib entry for the underlay multicast BUM group on the Spine
SPINE#show ipv6 mrib route ff05::1
IP Multicast Routing Information Base
Entry flags: L - Domain-Local Source, E - External Source to the Domain,
C - Directly-Connected Check, S - Signal, IA - Inherit Accept, D - Drop
ET - Data Rate Exceeds Threshold,K - Keepalive,DDE - Data Driven Event
ME - MoFRR ECMP Flow based, MNE - MoFRR Non-ECMP Flow based,
MP - Primary MoFRR Non-ECMP Flow based entry,
e - Encap helper tunnel flag
Interface flags: F - Forward, A - Accept, IC - Internal Copy,
NS - Negate Signal, DP - Don't Preserve, SP - Signal Present,
II - Internal Interest, ID - Internal Disinterest, LI - Local Interest,
LD - Local Disinterest, MD - mCAC Denied, MI - mLDP Interest
A2 - MoFRR ECMP Backup Accept
(*,FF05::1) RPF nbr: 2001:DB8::99:99 Flags: C
GigabitEthernet1/0/1 Flags: F NS
TenGigabitEthernet1/1/2 Flags: F NS
Tunnel1 Flags: A
(2001:DB8::1:1,FF05::1) RPF nbr: FE80::822D:BFFF:FE9B:84C6 Flags: L
TenGigabitEthernet1/1/2 Flags: A
GigabitEthernet1/0/1 Flags: F NS
Verify the mfib entry for the underlay multicast BUM group on the Spine
SPINE#show ipv6 mfib ff05::1
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
ET - Data Rate Exceeds Threshold, K - Keepalive
DDE - Data Driven Event, HW - Hardware Installed
ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB
MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary
MS - MoFRR Entry in Sync, MC - MoFRR entry in MoFRR Client,
e - Encap helper tunnel flag.
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
NS - Negate Signalling, SP - Signal Present,
A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
MA - MFIB Accept, A2 - Accept backup,
RA2 - MRIB Accept backup, MA2 - MFIB Accept backup
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
I/O Item Counts: HW Pkt Count/FS Pkt Count/PS Pkt Count Egress Rate in pps
Default
(*,FF05::1) Flags: C HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 0/0/0/0, Other: 0/0/0
Tunnel1 Flags: A NP
TenGigabitEthernet1/1/2 Flags: F NS
Pkts: 0/0/0 Rate: 0 pps
GigabitEthernet1/0/1 Flags: F NS
Pkts: 0/0/0 Rate: 0 pps
(2001:DB8::1:1,FF05::1) Flags: HW
SW Forwarding: 0/0/0/0, Other: 0/0/0
HW Forwarding: 591/1/170/1, Other: 0/0/0
TenGigabitEthernet1/1/2 Flags: A
GigabitEthernet1/0/1 Flags: F NS
Pkts: 0/0/591 Rate: 0 pps
Verify the PD adjacency on the ingress VTEP
VTEP1#show platform software fed switch active matm adjacencies vlan 12 | inc UM
12 124 0x20000007c VXLAN V6 0x0 0x7fb4688f8808 0x0 0x0 UM No
VTEP1#show plat hard fed sw active fwd abs print 0x7fb4688f8808 1
Handle:0x7fb4688f8808 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: index0:0x32 mtu_index/l3u_ri_index0:0x0 index1:0x32 mtu_index/l3u_ri_index1:0x0
Features sharing this resource:58 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 07 00 7c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 11 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0) <-- the first ASIC instance
----------------------------------------
ASIC#:0 RI:50 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2
Dst IPv6: ff0e::11
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20012
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 44 (0)
Port LE: 290 (0)
Vlan LE: 13 (0)
Detailed Resource Information (ASIC_INSTANCE# 1) <-- the second ASIC instance
----------------------------------------
ASIC#:1 RI:50 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_BRIDGING_EPG_MCAST_IPV6_ENCAP(234) Mapped_rii:LVX_ENCAP_L2_PAYLOAD_uIPv6_BRIDGED_EPG(155)
Src IPv6: 2001:DB8:1::2
Dst IPv6: ff0e::11
iVxlan dstMac: 0x00:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20012
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 44 (0)
Port LE: 290 (0)
Vlan LE: 13 (0)
Verify the PD flood list on the ingress VTEP
VTEP1#show plat har fed sw active vlan 12 ingress
VLAN STP State in hardware
vlan id is:: 12
Interfaces in forwarding state: : Gi1/0/2(Tagged)
flood list: : Gi1/0/2 <-- port in flood list
REP RI Handle: : 0x00007fb4689be2d8
VTEP1#show plat hard fed sw active fwd abs print 0x00007fb4689be2d8 1
Handle:0x7fb4689be2d8 Res-Type:ASIC_RSC_RI_REP Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_LISP Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: index0:0xc mtu_index/l3u_ri_index0:0x0 index1:0xc mtu_index/l3u_ri_index1:0x0
Cookie length: 56
00 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Replication list RI handle = 7fb4689be2d8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASIC [0] Replication Expansion Handle [0x7fb4689be738]
Replication list :
Number of RIs = 6
Start RI = 12
Common rewrite = Yes
Replication REP_RI 0xc [elements = 2]
[0] ri[0]=2 RI_L2 port=78 ri_ref_count:1 dirty=0
[1] ri[1]=50 Dynamic port=88 ri_ref_count:1 dirty=0 <-- replication (RI) index
Detailed Resource Information (ASIC_INSTANCE# 1)
----------------------------------------
Replication list RI handle = 7fb4689be2d8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASIC [1] Replication Expansion Handle [0x7fb4689be7a8]
Replication list :
Number of RIs = 6
Start RI = 12
Common rewrite = Yes
Replication REP_RI 0xc [elements = 1]
[0] ri[0]=2 RI_L2 port=78 ri_ref_count:1 dirty=0
==============================================================
Unicast Forwarding Between VTEPs (Same VLAN via L2VNI)
This section provides an example for troubleshooting unicast traffic:
- from host H11 connected to VTEP1 on VLAN 11 (with MAC address aabb.0000.0011 and IP 192.168.11.11)
- to host H31 connected to VTEP3 on VLAN 11 (with MAC address aabb.0000.0031 and IP address 192.168.11.31).
Verify ARP for destination IP address is resolved on source host
-
ARP packets are handled as part of BUM traffic in EVPN fabric. If ARP is not resolved on a host, use the troubleshooting steps for EVPN BUM traffic forwarding for the appropriate configured replication type used in your network.
H11#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.11.11 - aabb.0000.0011 ARPA GigabitEthernet2.11
Internet 192.168.11.31 95 aabb.0000.0031 ARPA GigabitEthernet2.11
Verify the host MAC is learned MAC table on the egress VTEP
VTEP3#show mac address-table dynamic address aabb.0000.0031 vlan 11
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
11 aabb.0000.0031 DYNAMIC Gi1/0/1
Total Mac Addresses for this criterion: 1
Verify the local MAC route is created in EVPN Manager on the egress VTEP
VTEP3#show l2vpn evpn mac address aabb.0000.0031 detail
MAC Address: aabb.0000.0031
EVPN Instance: 1
Vlan: 11
Ethernet Segment: 0000.0000.0000.0000.0000
Ethernet Tag ID: 0
Next Hop(s): V:20011 GigabitEthernet1/0/1 service instance 11
Sequence Number: 0
MAC only present: Yes
MAC Duplication Detection: Timer not running
Verify the local MAC route is created in L2RIB on egress VTEP
VTEP3#show l2route evpn mac mac-address aabb.0000.0031 detail
EVPN Instance: 1
Ethernet Tag: 0
Producer Name: L2VPN <-- Installed from L2VPN
MAC Address: aabb.0000.0031
Num of MAC IP Route(s): 2
Sequence Number: 0
ESI: 0000.0000.0000.0000.0000
Flags: B()
Next Hop(s): Gi1/0/1:11
Verify the local MAC route is created in BGP on the egress VTEP
Greenfield VXLANv6
VTEP3#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0031 *
BGP routing table entry for [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20, version 17
Paths: (1 available, best #1, table evi_1)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.3.3.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.1DB3
core-irb interface:Vlan3
vtep-ip:2001:DB8:3::2
sec-vtep-ip:UNKNOWN
rx pathid: 0, tx pathid: 0x0
Updated on Apr 7 2022 23:38:35 UTC
Dual-stack
VTEP3#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0031 *
BGP routing table entry for [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20, version 51
Paths: (1 available, best #1, table evi_1)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.3.3.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute added with secondary VTEP IP
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.1DB3
core-irb interface:Vlan3
vtep-ip:10.3.3.2
sec-vtep-ip:2001:DB8:3::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:04:13 UTC
Verify the remote MAC route is received and import in EVI table in BGP on the ingress VTEP
Greenfield VXLANv6
VTEP1#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0031 *
BGP routing table entry for [2][10.1.1.3:1][0][48][AABB00000031][0][*]/20, version 34
Paths: (1 available, best #1, table evi_1)
Not advertised to any peer
Refresh Epoch 2
Local, imported path from [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20 (global)
2001:DB8:3::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Originator: 10.3.3.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 14:28:15 UTC
BGP routing table entry for [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20, version 10
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 2
Local
2001:DB8:3::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Originator: 10.3.3.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 14:28:14 UTC
Dual-stack
VTEP1#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0031 *
BGP routing table entry for [2][10.1.1.3:1][0][48][AABB00000031][0][*]/20, version 57
Paths: (1 available, best #1, table evi_1)
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20 (global)
2001:DB8:3::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute received from remote dual-stack VTEP
Encap type: 8
Secondary nexthop address 2001:DB8:3::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:04:13 UTC
BGP routing table entry for [2][10.3.3.3:1][0][48][AABB00000031][0][*]/20, version 56
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 1
Local
10.3.3.2 (metric 3) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20011
Extended Community: RT:100:1 ENCAP:8
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:04:13 UTC
Verify the remote MAC route is received and installed in L2RIB
VTEP1#show l2route evpn mac mac-address aabb.0000.0031 detail
EVPN Instance: 1
Ethernet Tag: 0
Producer Name: BGP <-- Installed from BGP
MAC Address: aabb.0000.0031
Num of MAC IP Route(s): 2
Sequence Number: 0
ESI: 0000.0000.0000.0000.0000
Flags: B()
Next Hop(s): V:20011 2001:DB8:3::2
Verify the remote MAC is in the L2FIB forwarding table
VTEP1#show l2fib bridge-domain 11 detail
Bridge Domain : 11
Reference Count : 12
Replication ports count : 3
Unicast Address table size : 2
IP Multicast Prefix table size : 1
Flood List Information :
Olist: 1035, Ports: 3
Port Information :
BD_PORT Gi1/0/1:11
VXLAN_REP PL:22(1) T:VXLAN_REP [IR]20011:2001:DB8:2::2
VXLAN_REP PL:18(1) T:VXLAN_REP [IR]20011:2001:DB8:3::2
Unicast Address table information :
aabb.0000.0021 VXLAN_UC PL:21(1) T:VXLAN_UC [MAC]20011:2001:DB8:2::2
aabb.0000.0031 VXLAN_UC PL:17(1) T:VXLAN_UC [MAC]20011:2001:DB8:3::2 <-- PL = Path-list. use this value in the next command
IP Multicast Prefix table information :
Source: *, Group: 239.21.21.21, IIF: Null, Adjacency: Olist: 6160, Ports: 1
VTEP1#show l2fib path-list 17 detail
VXLAN_UC Pathlist 17: topo 11, 1 paths, none
ESI: 0000.0000.0000.0000.0000
path 2001:DB8:3::2, type VXLAN, evni 20011, vni 20011, source MAC
oce type: vxlan_header, sw_handle 0x7FA98894B318
forwarding oce 0x7FA988AAE538 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:3::2, cid: 1
output chain:
oce type: evpn_vxlan_encap, sw_handle 0x7FA988938728
oce type: vxlan_header, sw_handle 0x7FA98894B380
forwarding oce 0x7FA988AAE538 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:3::2, cid: 1
Verify the remote MAC is programmed in the PD Mac table
VTEP1#show platform software fed switch active matm macTable vlan 11 mac aabb.0000.0031 detail
VLAN MAC Type Seq# EC_Bi Flags machandle siHandle riHandle diHandle *a_time *e_time ports Con
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
11 aabb.0000.0031 0x1000001 0 0 64 0x7fb4687eefb8 0x7fb4687ee058 0x7fb4687ee7a8 0x0 0 4 No
<-- riHandle = rewrite index handle. This value holds the info on how the ASIC treats this address
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Number of HTM Entries: 1
Entry 0: (handle 0x7fb4687f20b8)
Absolute Index: 2546
Time Stamp: 4
KEY - vlan:12 mac:0xaabb00000031 l3_if:0 gpn:0 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:1
MASK - vlan:0 mac:0x0 l3_if:0 gpn:0 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:0
SRC_AD - need_to_learn:0 lrn_v:0 catchall:0 static_mac:0 chain_ptr_v:0 chain_ptr: 0 static_entry_v:0 auth_state:0 auth_mode:0 auth_behavior_tag:0 traf_m:0 is_src_ce:0
DST_AD - si:0xbf bridge:0 replicate:0 blk_fwd_o:0 v4_rmac:0 v6_rmac:0 catchall:0 ign_src_lrn:0 port_mask_o:0 afd_cli_f:0 afd_lbl:0 prio:3 dest_mod_idx:0 destined_to_us:0 pv_trunk:0 smr:0
==============================================================
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Station Index (SI) [0xbf]
RI = 0x34
DI = 0x5012
stationTableGenericLabel = 0
stationFdConstructionLabel = 0x7
lookupSkipIdIndex = 0
rcpServiceId = 0
dejaVuPreCheckEn = 0
Replication Bitmap: LD
Detailed Resource Information (ASIC_INSTANCE# 1)
----------------------------------------
Station Index (SI) [0xbf]
RI = 0x34
DI = 0x5013
stationTableGenericLabel = 0
stationFdConstructionLabel = 0x7
lookupSkipIdIndex = 0
rcpServiceId = 0
dejaVuPreCheckEn = 0
Replication Bitmap: LD
==============================================================
VTEP1#show plat hard fed sw active fwd abs print 0x7fb4687ee7a8 1
Handle:0x7fb4687ee7a8 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1
priv_ri/priv_si Handle: 0x7fb4687ee358Hardware Indices/Handles: index0:0x34 mtu_index/l3u_ri_index0:0x0 index1:0x34 mtu_index/l3u_ri_index1:0x0
Features sharing this resource:58 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 00 00 00 00 07 00 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
ASIC#:0 RI:52 Rewrite_type:AL_RRM_REWRITE_LVX_IPV6_L2_PAYLOAD_ENCAP_EPG(233) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD_uIPv6_EPG(153)
Src IPv6: 2001:DB8:1::2
Dst IPv6: 2001:DB8:3::2
iVxlan dstMac: 0x120:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 54 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
Detailed Resource Information (ASIC_INSTANCE# 1)
----------------------------------------
ASIC#:1 RI:52 Rewrite_type:AL_RRM_REWRITE_LVX_IPV6_L2_PAYLOAD_ENCAP_EPG(233) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD_uIPv6_EPG(153)
Src IPv6: 2001:DB8:1::2
Dst IPv6: 2001:DB8:3::2
iVxlan dstMac: 0x120:0x00:0x00
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 20011
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 54 (0)
Port LE: 286 (0)
Vlan LE: 12 (0)
==============================================================
Unicast Forwarding Between VTEPs (Different VLANs via L3VNI)
This section provides an example to troubleshoot unicast traffic
- from host H11 connected to VTEP1 on VLAN 11 (with MAC address aabb.0000.0011 and IP 192.168.11.11)
- to host H22 connected to VTEP2 on VLAN 12 (with MAC address aabb.0000.0022 and IP address 192.168.12.22)
Verify ARP for the default gateway is resolved on the host
H11#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.11.11 - aabb.0000.0011 ARPA GigabitEthernet2.11
Internet 192.168.11.254 14 0011.0011.0011 ARPA GigabitEthernet2.11
Verify ARP for the destination IP address is resolved in tenant VRF on the egress VTEP
VTEP2#show arp vrf red
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.0.2.2 - 0050.569a.89d8 ARPA Vlan3
Internet 192.168.12.22 27 aabb.0000.0022 ARPA Vlan12 <-- H22 remote host
Verify the static entry for gateway MAC address is in MAC table on the ingress VTEP
VTEP1#show mac address-table static vlan 11
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
11 0011.0011.0011 STATIC Vl11
Total Mac Addresses for this criterion: 1
Verify the host MAC is learned in MAC table on the egress VTEP
VTEP2#show mac address-table dynamic address aabb.0000.0022 vlan 12
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
12 aabb.0000.0022 DYNAMIC Gi1/0/2
Total Mac Addresses for this criterion: 1
Verify the host MAC/IP is learned in device-tracking database on egress VTEP
VTEP2#show device-tracking database address 192.168.12.22
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
ARP 192.168.12.22 aabb.0000.0022 Gi1/0/2 12 0005 31mn STALE try 0 265 s
Verify the local MAC/IP route is created in EVPN Manager on the egress VTEP
VTEP2#show l2vpn evpn mac ip mac aabb.0000.0022 address 192.168.12.22 detail
IP Address: 192.168.12.22
EVPN Instance: 2
Vlan: 12
MAC Address: aabb.0000.0022
Ethernet Segment: 0000.0000.0000.0000.0000
Ethernet Tag ID: 0
Next Hop(s): V:20012 GigabitEthernet1/0/2 service instance 12
Sequence Number: 0
IP Duplication Detection: Timer not running
Label2 included: Yes
Verify the local MAC/IP route is created in L2RIB on the egress VTEP
VTEP2#show l2route evpn mac ip host-ip 192.168.12.22 mac-address aabb.0000.0022 detail
EVPN Instance: 2
Ethernet Tag: 0
Producer Name: L2VPN
MAC Address: aabb.0000.0022
Host IP: 192.168.12.22
Sequence Number: 0
Label 2: 30000
ESI: 0000.0000.0000.0000.0000
MAC Route Flags: B()
Next Hop(s): Gi1/0/2:12
Verify the local MAC/IP route is created in BGP on the egress VTEP
Greenfield VXLANv6
VTEP2#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0022 192.168.12.22
BGP routing table entry for [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 198
Paths: (1 available, best #1, table evi_2)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.2.2.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.89D8
core-irb interface:Vlan3
vtep-ip:2001:DB8:2::2
sec-vtep-ip:UNKNOWN
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:15:58 UTC
Dual-stack
VTEP2#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0022 192.168.12.22
BGP routing table entry for [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 63
Paths: (1 available, best #1, table evi_2)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via default) from 0.0.0.0 (10.2.2.1)
Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute added with secondary VTEP IP
Encap type: 8
Secondary nexthop address 2001:DB8:2::2(active)
Local irb vxlan vtep:
vrf:red, l3-vni:30000
local router mac:0050.569A.89D8
core-irb interface:Vlan3
vtep-ip:10.2.2.2
sec-vtep-ip:2001:DB8:2::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 19 2022 00:01:58 UTC
Verify the remote MAC/IP route is received and import in EVI table in BGP on the ingress VTEP
Greenfield VXLANv6
VTEP1#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0022 192.168.12.22
BGP routing table entry for [2][10.1.1.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 44
Paths: (1 available, best #1, table evi_2)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 2
Local, imported path from [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:52 UTC
BGP routing table entry for [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 42
Paths: (1 available, best #1, table EVPN-BGP-Table)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 2
Local
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:52 UTC
Dual-stack
VTEP1#show ip bgp l2vpn evpn route-type 2 0 aabb.0000.0022 192.168.12.22
BGP routing table entry for [2][10.1.1.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 87
Paths: (1 available, best #1, table evi_2)
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute: <-- Tunnel encap attribute received from remote dual-stack VTEP
Encap type: 8
Secondary nexthop address 2001:DB8:2::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 19 2022 00:03:18 UTC
BGP routing table entry for [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24, version 67
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 1
Local
10.2.2.2 (metric 3) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Label1 20012, Label2 30000
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:2::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 19 2022 00:01:59 UTC
Verify the host IP address is in BGP VPNv4 table for tenant VRF
Greenfield VXLANv6
VTEP1#show ip bgp vpnv4 vrf red 192.168.12.22/32
BGP routing table entry for 100:101:192.168.12.22/32, version 6
Paths: (1 available, best #1, table red)
Multipath: eiBGP
Flag: 0x100
Not advertised to any peer
Refresh Epoch 2
Local, imported path from [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:3
vtep-ip:2001:DB8:1::2
sec-vtep-ip:UNKNOWN
bdi:Vlan3
Remote VxLAN:
Topoid 0x2(vrf red)
Remote Router MAC:0050.569A.89D8
Encap 8
Egress VNI 30000
RTEP 2001:DB8:2::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:52 UTC
Dual-stack
VTEP1#show ip bgp vpnv4 vrf red 192.168.12.22/32
BGP routing table entry for 100:101:192.168.12.22/32, version 11
Paths: (1 available, best #1, table red)
Multipath: eiBGP
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [2][10.2.2.3:2][0][48][AABB00000022][32][192.168.12.22]/24 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:2 RT:100:100 ENCAP:8
Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:2::2
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:4
vtep-ip:10.1.1.2
sec-vtep-ip:2001:DB8:1::2
bdi:Vlan3
Remote VxLAN:
Topoid 0x2(vrf red)
Remote Router MAC:0050.569A.89D8
Encap 8
Egress VNI 30000
RTEP 2001:DB8:2::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 19 2022 00:01:59 UTC
Verify the IP-Prefix route for the access gateway for the local (imported from vrf) and remote is in the BGP EVPN global table
Greenfield VXLANv6
VTEP1#show bgp l2vpn evpn route-type 5 0 192.168.12.0 24
BGP routing table entry for [5][100:101][0][24][192.168.12.0]/17, version 9
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
Local, imported path from base
0.0.0.0 (via vrf red) from 0.0.0.0 (10.1.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 20
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.A8BF
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:41 UTC
BGP routing table entry for [5][100:102][0][24][192.168.12.0]/17, version 50
Paths: (1 available, best #1, table EVPN-BGP-Table)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 2
Local
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:52 UTC
Dual-stack
VTEP1#show bgp l2vpn evpn route-type 5 0 192.168.12.0 24
BGP routing table entry for [5][100:101][0][24][192.168.12.0]/17, version 10
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
Local, imported path from base
0.0.0.0 (via vrf red) from 0.0.0.0 (10.1.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 20
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.A8BF
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:1::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:03:27 UTC
BGP routing table entry for [5][100:102][0][24][192.168.12.0]/17, version 24
Paths: (1 available, best #1, table EVPN-BGP-Table)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
Local
10.2.2.2 (metric 3) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:2::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:03:49 UTC
Verify the local and remote (imported) IP-Prefix route is in the VRF table in BGP on the ingress VTEP
Greenfield VXLANv6
VTEP1#show bgp vpnv4 unicast vrf red 192.168.12.0
BGP routing table entry for 100:101:192.168.12.0/24, version 4
Paths: (2 available, best #2, table red)
Multipath: eiBGP
Not advertised to any peer
Refresh Epoch 2
Local, imported path from [5][100:102][0][24][192.168.12.0]/17 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:3
vtep-ip:2001:DB8:1::2
sec-vtep-ip:UNKNOWN
bdi:Vlan3
Remote VxLAN:
Topoid 0x2(vrf red)
Remote Router MAC:0050.569A.89D8
Encap 8
Egress VNI 30000
RTEP 2001:DB8:2::2
mpls labels in/out 20/nolabel
rx pathid: 0, tx pathid: 0
Updated on Apr 8 2022 19:25:52 UTC
Refresh Epoch 1
Local
0.0.0.0 (via vrf red) from 0.0.0.0 (10.1.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:100:100
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:3
vtep-ip:2001:DB8:1::2
sec-vtep-ip:UNKNOWN
bdi:Vlan3
mpls labels in/out 20/nolabel(red)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 8 2022 19:25:41 UTC
Dual-stack
VTEP1#show bgp vpnv4 unicast vrf red 192.168.12.0
BGP routing table entry for 100:101:192.168.12.0/24, version 4
Paths: (2 available, best #2, table red)
Multipath: eiBGP
Not advertised to any peer
Refresh Epoch 1
Local, imported path from [5][100:102][0][24][192.168.12.0]/17 (global)
2001:DB8:2::2 (metric 2) (via default) from 2001:DB8:99::99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal
Extended Community: RT:100:100 ENCAP:8 Router MAC:0050.569A.89D8
Originator: 10.2.2.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:2::2
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:4
vtep-ip:10.1.1.2
sec-vtep-ip:2001:DB8:1::2
bdi:Vlan3
Remote VxLAN:
Topoid 0x2(vrf red)
Remote Router MAC:0050.569A.89D8
Encap 8
Egress VNI 30000
RTEP 2001:DB8:2::2
mpls labels in/out 20/nolabel
rx pathid: 0, tx pathid: 0
Updated on Apr 18 2022 18:03:49 UTC
Refresh Epoch 1
Local
0.0.0.0 (via vrf red) from 0.0.0.0 (10.1.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:100:100
Local vxlan vtep:
vrf:red, vni:30000
local router mac:0050.569A.A8BF
encap:4
vtep-ip:10.1.1.2
sec-vtep-ip:2001:DB8:1::2
bdi:Vlan3
mpls labels in/out 20/nolabel(red)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 18 2022 18:03:27 UTC
Verify the adjacency for the core VLAN
-
The adjacency for the core VLAN is created by L3EVPN
VTEP1#show adjacency vlan 3 detail
Protocol Interface Address
IP Vlan3 225.0.0.0(5)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 1
Encap length 14
01005E0000000050569AA8BF0800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
Multicast
IP Vlan3 227.0.0.0(3)
connectionid 1
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 1
Encap length 14
01005E0000000050569AA8BF0800
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ip
Inject p2mp Multicast
IP Vlan3 2001:DB8:2::2(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 4
Encap length 14
0050569A89D80050569AA8BF0800
VXLAN Transport tunnel
IPV6 Vlan3 2001:DB8:2::2(8)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 4
Encap length 14
0050569A89D80050569AA8BF86DD
VXLAN Transport tunnel
IP Vlan3 2001:DB8:3::2(11)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 4
Encap length 14
0050569A1DB30050569AA8BF0800
VXLAN Transport tunnel
IPV6 Vlan3 2001:DB8:3::2(11)
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 4
Encap length 14
0050569A1DB30050569AA8BF86DD
VXLAN Transport tunnel
IPV6 Vlan3 FFFF::(3)
connectionid 1
0 packets, 0 bytes
epoch 0
sourced in sev-epoch 1
Encap length 14
3333000000000050569AA8BF86DD
L2 destination address byte offset 0
L2 destination address byte length 6
Link-type after encap: ipv6
Inject p2mp Multicast
Verify the L2FIB unicast entry in the core VLAN
VTEP1#show l2fib bridge-domain 3 detail
Bridge Domain : 3
Reference Count : 7
Replication ports count : 0
Unicast Address table size : 2
IP Multicast Prefix table size : 0
Flood List Information :
Olist: 1027, Ports: 0
Unicast Address table information :
0050.569a.1db3 VXLAN_UC PL:11(1) T:VXLAN_UC [MAC]30000:2001:DB8:3::2 <-- PL = path-list. Use these values in the next path-list command
0050.569a.89d8 VXLAN_UC PL:7(1) T:VXLAN_UC [MAC]30000:2001:DB8:2::2
VTEP1#show l2fib path-list 11 detail
VXLAN_UC Pathlist 11: topo 3, 1 paths, none
ESI: 0000.0000.0000.0000.0000
path 2001:DB8:3::2, type VXLAN, evni 30000, vni 30000, source MAC
oce type: vxlan_header, sw_handle 0x7F262F466920
forwarding oce 0x7F262F50A448 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:3::2, cid: 1
output chain:
oce type: evpn_vxlan_encap, sw_handle 0x7F262F484840
oce type: vxlan_header, sw_handle 0x7F262F466988
forwarding oce 0x7F262F50A448 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:3::2, cid: 1
VTEP1#show l2fib path-list 7 detail
VXLAN_UC Pathlist 7: topo 3, 1 paths, none
ESI: 0000.0000.0000.0000.0000
path 2001:DB8:2::2, type VXLAN, evni 30000, vni 30000, source MAC
oce type: vxlan_header, sw_handle 0x7F262F466B90
forwarding oce 0x7F262ED39BF8 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:2::2, cid: 1
output chain:
oce type: evpn_vxlan_encap, sw_handle 0x7F262F484930
oce type: vxlan_header, sw_handle 0x7F262F466BF8
forwarding oce 0x7F262ED39BF8 type adjacency, IPV6 midchain out of Tunnel0, addr 2001:DB8:2::2, cid: 1
Verify the host route is installed in IP routing table for the tenant VRF on the ingress VTEP
VTEP1#show ip route vrf red 192.168.12.22
Routing Table: red
Routing entry for 192.168.12.22/32
Known via "bgp 100", distance 200, metric 0, type internal
Last update from 2001:DB8:2::2 on Vlan3, 00:37:49 ago
Routing Descriptor Blocks:
* 2001:DB8:2::2 (red:ipv6), from 2001:DB8:99::99, 00:37:49 ago, via Vlan3
opaque_ptr 0x7FC009408C68
Route metric is 0, traffic share count is 1
AS Hops 0
MPLS label: none
MPLS Flags: NSF
Verify the CEF forwarding chain is built properly for the host route in the tenant VRF on the ingress VTEP
VTEP1#show ip cef vrf red 192.168.12.22 internal
192.168.12.22/32, epoch 1, flags [rnolbl, rlbls], RIB[B], refcnt 6, per-destination sharing
sources: RIB
feature space:
IPRM: 0x00018000
Broker: linked, distributed at 3rd priority
ifnums:
Vlan3(25): 2001:DB8:2::2
path list 7FC008D9C400, 5 locks, per-destination, flags 0x249 [shble, rif, hwcn, bgp]
path 7FC00874B5D8, share 1/1, type attached nexthop, for IPv4
nexthop 2001:DB8:2::2 Vlan3, IP adj out of Vlan3, addr 2001:DB8:2::2 7FC009A11360
output chain:
IP adj out of Vlan3, addr 2001:DB8:2::2 7FC009A11360
Verify the route in the PD forwarding table on the ingress VTEP
VTEP1#show platform software fed swith active ip route vrf red 192.168.12.22/32 detail
vrf dest htm flags SGT DGID MPLS Last-modified SecsSinceHit
--- ---- --- ----- --- ---- ---- ------------------------ ------------
2 192.168.12.22/32 0x7fb4687e5e68 0x0 0 0 2022/04/08 19:25:53.492 14
FIB: prefix_hdl:0xd8000042, mpls_ecr_prefix_hdl:0, sgtOverWrite: 0
========== OCE chain =====
ADJ:objid:138 {link_type:IP ifnum:0x19, adj:0x8c000041, si: 0x7fb4687e1c18 IPv4: 2.0.205.171 }
===============
MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0
===============
Hardware entry details
----------------------
Handle:0x7fb4687e5e68 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1
priv_ri/priv_si Handle:(nil) Hardware Indices/Handles: handle0:0x7fb4687e6078
Features sharing this resource:
Brief Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Number of HTM Entries: 1
Entry #0: (handle 0x7fb4687e6078)
KEY - vrf:2 mtr:0 prefix:192.168.12.22 rcp_redirect_index:0x0
MASK - vrf:0 mtr:0 prefix:0.0.0.0 rcp_redirect_index:0x0
FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:190 destined_to_us:0 hw_stats_idx:1 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0 vpn prefix flag:0
SRC-AD:learning_violation:0 need_to_learn:0 locally_connected:0 staticentryViolation:0
rpfValid:1 rpfLe:45 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:1 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0
sgtValid:0 sgtOverwrite:0 sgt:0 ipClientLabel:0
src_rloc_trusted:0, sgtCacheControl1:0, sgtCacheControl0:0
port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0
==============================================================
Asic SI-Index DI-Index
---- -------- -------
0 190 0x5012
Detailed Resource Information (ASIC# 0)
----------------------------------------
Station Index (SI) [0xbe]
RI = 0x3e
DI = 0x5012
Replication Bitmap: LD
Destination index = 0x5012 DI_RCP_PORT1
pmap = 0x00000000 0x00000000
rcp_pmap = 0x1
Asic SI-Index DI-Index
---- -------- -------
1 190 0x5012
Detailed Resource Information (ASIC# 1)
----------------------------------------
Station Index (SI) [0xbe]
RI = 0x3e
DI = 0x5013
Replication Bitmap: LD
Destination index = 0x5012 DI_RCP_PORT1
pmap = 0x00000000 0x00000000
VTEP1#show plat sof fed sw active ip adj | inc 0x8a
2001:DB8:2::2 Vlan3 0050.569a.89d8 0x7fb4687e1c18 0x7fb4687e2008 0x0 0x8a 2022/04/08 19:25:53.068
VTEP1#show plat hard fed sw active fwd abs print 0x7fb4687e2008 1
Handle:0x7fb4687e2008 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:3
priv_ri/priv_si Handle: 0x7fb4687e52c8Hardware Indices/Handles: index0:0x3e mtu_index/l3u_ri_index0:0x23 index1:0x3e mtu_index/l3u_ri_index1:0x23
Features sharing this resource:66 (1)]
67 (1)]
58 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00 07 00 00 50 56 9a 89 d8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
ASIC#:0 RI:62 Rewrite_type:AL_RRM_REWRITE_LVX_IPV6_L2_PAYLOAD_ENCAP_EPG(233) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD_uIPv6_EPG(153)
Src IPv6: 2001:DB8:1::2
Dst IPv6: 2001:DB8:2::2
iVxlan dstMac: 0x050:0x569a:0x89d8 <-- MAC dest address 0050.569a.89d8
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 30000
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 52 (0)
Port LE: 284 (0)
Vlan LE: 5 (0)
Detailed Resource Information (ASIC_INSTANCE# 1)
----------------------------------------
ASIC#:1 RI:62 Rewrite_type:AL_RRM_REWRITE_LVX_IPV6_L2_PAYLOAD_ENCAP_EPG(233) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD_uIPv6_EPG(153)
Src IPv6: 2001:DB8:1::2
Dst IPv6: 2001:DB8:2::2
iVxlan dstMac: 0x050:0x569a:0x89d8 <-- MAC dest address 0050.569a.89d8
iVxlan srcMac: 0x00:0x00:0x00
IPv6 hlim: 0
iid present: 0
lisp iid: 30000
lisp flags: 0
dst Port: 4789
update only l3if: 0
is Sgt: 0
is TTL Prop: 0
L3if LE: 52 (0)
Port LE: 284 (0)
Vlan LE: 5 (0)
==============================================================
EVPN Handoff Between Border Leaf and MPLS VPN Router
This section is applicable for both L3VPN handoff and Multi VRF handoff (VRF lite/PE-CE)
Verify the IP Prefix on remote VPN Router(R1) advertised to Border-Leaf(vtep3) in the EVPN fabric
- VPN Node R1 has an MPLS underlay, it is a single stack. The Border-leaf node Leaf3 and rest of the EVPN fabric can be in either single stack or dual-stack mode,
Check these things if there is a traffic issue with EVPN Route-type 5 (RT5):
Step 1. Confirm RT5 route is in EVPN global table
Step 2. Verify the route is imported into VRF
- Must have the import RT configured for the VRF.
- Stitching Route-Target (RT) for a handoff scenario on border-leaf/border-spine
- Ensure BGP EVPN config “advertise l2vpn evpn” is present in the address-family vrf
- Ensure correct import configuration is done on the address-family l2vpn evpn and vpnv4/vpnv6 ror L3VPN handoff (on border node) as shown here:
vrf definition red
rd 100:103
!
address-family ipv4
route-target export 10:100
route-target import 10:100
route-target export 100:100 stitching
route-target import 100:100 stitching
exit-address-family
!
address-family ipv6
route-target export 10:200
route-target import 10:200
route-target export 100:200 stitching
route-target import 100:200 stitching
exit-address-family
!
router bgp 100
neighbor 10.5.0.1 remote-as 10
neighbor 10.99.99.99 remote-as 100
neighbor 10.99.99.99 update-source Loopback0
neighbor 2001:DB8:99::99 remote-as 100
neighbor 2001:DB8:99::99 update-source Loopback0
!
address-family vpnv4
import l2vpn evpn re-originate
neighbor 10.5.0.1 activate
neighbor 10.5.0.1 send-community both
exit-address-family
!
address-family vpnv6
import l2vpn evpn re-originate
neighbor 10.5.0.1 activate
neighbor 10.5.0.1 send-community both
exit-address-family
!
address-family l2vpn evpn
import vpnv4 unicast re-originate
import vpnv6 unicast re-originate
neighbor 10.99.99.99 activate
neighbor 10.99.99.99 send-community both
neighbor 2001:DB8:99::99 activate
neighbor 2001:DB8:99::99 send-community both
exit-address-family
!
address-family ipv4 vrf red
advertise l2vpn evpn
redistribute connected
exit-address-family
!
address-family ipv6 vrf red
redistribute connected
advertise l2vpn evpn
exit-address-family
Step 3. Check the local VTEP status with this command
VTEP3#show bgp l2vpn evpn local-vtep vrf red
Local VTEP vrf red:
Protocol: IPv4
RMAC Address: AABB.CC81.F700
VTEP-IP:10.3.3.2
SEC-VTEP-IP:2001:DB8:3::2
VNI: 30000
BDI:Vlan3
Protocol: IPv6
RMAC Address: AABB.CC81.F700
VTEP-IP:10.3.3.2
SEC-VTEP-IP:2001:DB8:3::2
VNI: 30000
BDI:Vlan3
Step 4. Check the Remote Nexthop (RNH) is installed for the VRF.
- RNH ensures the adjacency for the route installed in the RIB.
- BGP installs the RT5 route in the BGP VPN table, which is then installed in the corresponding VRF IP table.
- The installed nexthop’s adjacency is ensured by the remote VTEP’s parameters (such as RMAC, eVNI and RTEP). RTEP would be same as the nexthop address of the route.
- BGP installs the RNH in the L3-EVPN/NVE directly.
When RNH is not correct, CEF entry for the routes would have incomplete adjacency.
- In a dual-stack scenario, you need to check whether the nexthop for the Route and RNH RTEP (remote tunnel endpoint) are the same.
When a route is not installed or installed route has incomplete adjacency, check for the RNH parameters and correlate with configuration in the local node and the value of the parameters in the route.
VTEP3#show bgp l2vpn evpn rnh vrf red
Remote VTEP entries for vrf red:
Protocol: ipv4
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
Protocol: ipv6
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
Step 5: Check the NVE peer status
- NVE peer status is up when BGP receives a route from the remote peer and is installed correctly into the RIB and L3-EVPN/NVE subsystem.
VTEP3#show nve peers
'M' - MAC entry download flag 'A' - Adjacency download flag
'4' - IPv4 flag '6' - IPv6 flag
Interface VNI Type Peer-IP RMAC/Num_RTs eVNI state flags UP time
nve1 30000 L3CP 2001:DB8:1::2 aabb.cc81.f500 30000 UP A/M/4 08:52:46
nve1 30000 L3CP 2001:DB8:2::2 aabb.cc81.f600 30000 UP A/-/4 08:51:41
nve1 30000 L3CP 2001:DB8:1::2 aabb.cc81.f500 30000 UP A/-/6 08:52:46
nve1 30000 L3CP 2001:DB8:2::2 aabb.cc81.f600 30000 UP A/M/6 08:51:41
nve1 20011 L2CP 2001:DB8:1::2 6 20011 UP N/A 08:52:09
nve1 20011 L2CP 2001:DB8:2::2 5 20011 UP N/A 08:52:09
Verify prefix is in VPNv4 table
R1#show ip route vrf red 10.10.10.0 <-- prefix in Routing table
Routing Table: red
Routing entry for 10.10.10.0/24
Known via "connected", distance 0, metric 0 (connected, via interface)
Redistributing via ospfv3 1, bgp 10
Advertised by bgp 10
Routing Descriptor Blocks:
* directly connected, via Loopback0
Route metric is 0, traffic share count is 1
R1#show bgp vpnv4 unicast vrf red 10.10.10.0/24 <-- prefix added to VPNv4 table
BGP routing table entry for 10:100:10.10.10.0/24, version 34
Paths: (1 available, best #1, table red)
Advertised to update-groups:
1
Refresh Epoch 1
Local
0.0.0.0 (via vrf red) from 0.0.0.0 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:10:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0
mpls labels in/out 18/nolabel(red)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 16:54:32 PST
Verify prefix is in VPNv6 table
R1#show ipv6 route vrf red 2001:DB8:10::/128 <-- prefix is in Routing table
Routing entry for 2001:DB8:10::/128
Known via "connected", distance 0, metric 0, type receive, connected
Redistributing via ospf 1, bgp 10
Route count is 1/1, share count 0
Routing paths:
receive via Loopback0
Route metric is 0, traffic share count is 1
Last updated 06:59:56 ago
R1#show bgp vpnv6 unicast vrf red 2001:DB8:10::/128 <-- Prefix added to VPNv6 table
BGP routing table entry for [10:100]2001:DB8:10::/128, version 2
Paths: (1 available, best #1, table red)
Advertised to update-groups:
1
Refresh Epoch 1
Local
:: (via vrf red) from 0.0.0.0 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, weight 32768, valid, sourced, best
Extended Community: RT:10:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0
mpls labels in/out 17/nolabel(red)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:16:12 PST
Verify IP Prefix on Border-Leaf (VTEP3) in the EVPN fabric
(Greenfield: VxLANv6) verify prefix is in VPNv4 table
VTEP3#show bgp vpnv4 unicast vrf red 10.10.10.0/24
BGP routing table entry for 100:103:10.10.10.0/24, version 31
Paths: (1 available, best #1, table red)
Not advertised to any peer
Refresh Epoch 1
10, imported path from 10:100:10.10.10.0/24 (global)
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:10:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F700
encap:5
vtep-ip:2001:DB8:3::2
sec-vtep-ip:UNKNOWN
bdi:Vlan3
mpls labels in/out nolabel/18
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 16:54:50 PST
(Dual-stack: Prefer IPv6) verify prefix is in VPNv4 table
VTEP3#show bgp vpnv4 unicast vrf red 10.10.10.0/24
BGP routing table entry for 100:103:10.10.10.0/24, version 30
Paths: (1 available, best #1, table red)
Not advertised to any peer
Refresh Epoch 1
10, imported path from 10:100:10.10.10.0/24 (global)
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:10:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F700
encap:4
vtep-ip:10.3.3.2
sec-vtep-ip:2001:DB8:3::2
bdi:Vlan3
mpls labels in/out nolabel/18
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:30:45 PST
Verify prefix is in VPNv6 table
VTEP3#show bgp vpnv6 unicast vrf red 2001:DB8:10::/128
BGP routing table entry for [100:103]2001:DB8:10::/128, version 12
Paths: (1 available, best #1, table red)
Not advertised to any peer
Refresh Epoch 1
10, imported path from [10:100]2001:DB8:10::/128 (global)
::FFFF:10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:10:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F700
encap:4
vtep-ip:10.3.3.2
sec-vtep-ip:2001:DB8:3::2
bdi:Vlan3
mpls labels in/out nolabel/17
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:17:32 PST
Verify IP Prefix is imported from VPN into EVPN on Border-Leaf (VTEP3)
VTEP3#show bgp l2vpn evpn rnh vrf red
Remote VTEP entries for vrf red:
Protocol: ipv4
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
Protocol: ipv6
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
(Greenfield: VxLANv6) verify prefix is in EVPNv4 table
VTEP3#show bgp l2vpn evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 167
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
2
Refresh Epoch 1
10, imported path from base
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 18
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 16:54:50 PST
(Dual-stack: Prefer IPv6) verify prefix is in EVPNv4 table
VTEP3#show bgp l2vpn evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 132
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, imported path from base
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 18
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:30:45 PST
Verify prefix is in EVPNv6 table
VTEP3#show bgp l2vpn evpn route-type 5 0 2001:DB8:10::128
BGP routing table entry for [5][100:103][0][128][2001:DB8:10::]/29, version 74
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, imported path from base
::FFFF:10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: ::, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 17
Extended Community: RT:100:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:17:52 PST
Verify IP Prefix imported from VPN into EVPN on Border-Leaf (VTEP 3)
VTEP3#show bgp l2vpn evpn rnh vrf red
Remote VTEP entries for vrf red:
Protocol: ipv4
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
Protocol: ipv6
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
[30000 / AABB.CC81.F500 / 2001:DB8:1::2 / yes]
(Greenfield: VxLANv6) verify prefix is in EVPNv4 table
VTEP3#show bgp l2vpn evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 167
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
2
Refresh Epoch 1
10, imported path from base
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 18
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 16:54:50 PST
(Dual-stack: Prefer IPv6) verify prefix is in EVPNv4 table
VTEP3#show bgp l2vpn evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 132
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, imported path from base
10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 18
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:30:45 PST
Verify prefix is in EVPNv6 table
VTEP3#show bgp l2vpn evpn route-type 5 0 2001:10:: 128
BGP routing table entry for [5][100:103][0][128][2001:10::]/29, version 74
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, imported path from base
::FFFF:10.5.0.1 (via default) from 10.5.0.1 (10.5.1.1)
Origin incomplete, metric 0, localpref 100, valid, external, best
EVPN ESI: 00000000000000000000, Gateway Address: ::, local vtep: 0.0.0.0, VNI Label 30000, MPLS VPN Label 17
Extended Community: RT:100:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:17:52 PST
Verify EVPN IP Prefix Received From Border Leaf and Advertised to Other VTEPs
(Greenfield: VxLANv6) verify prefix is in EVPNv4 table
SPINE#show bgp l2 evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 45
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
2
Refresh Epoch 1
10, (Received from a RR-client)
2001:DB8:3::2 (metric 10) (via default) from 10.3.3.1 (10.3.3.1)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 19:36:59 EST
(Dual-stack: Prefer IPv6) verify prefix is in EVPNv4 table
SPINE#show bgp l2 evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 103
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, (Received from a RR-client)
10.3.3.2 (metric 11) (via default) from 10.3.3.1 (10.3.3.1)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 07:30:45 EST
Verify prefix is in EVPNv6 table
SPINE#show bgp l2 ev route 5 0 2001:10::128
BGP routing table entry for [5][100:103][0][128][2001:10::]/29, version 70
Paths: (1 available, best #1, table EVPN-BGP-Table)
Advertised to update-groups:
1
Refresh Epoch 1
10, (Received from a RR-client)
10.3.3.2 (metric 11) (via default) from 10.3.3.1 (10.3.3.1)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: ::, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 07:17:53 EST
Verify EVPN IP Prefix in the Remote VTEP Global Table
VTEP1#show bgp l2vpn evpn rnh vrf red
Remote VTEP entries for vrf red:
Protocol: ipv4
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
[30000 / AABB.CC81.F700 / 2001:DB8:3::2 / yes]
Protocol: ipv6
[VNI / RMAC ADDRESS / VTEP-IP / Installed]
[30000 / AABB.CC81.F600 / 2001:DB8:2::2 / yes]
[30000 / AABB.CC81.F700 / 2001:DB8:3::2 / yes]
(Greenfield: VxLANv6) verify prefix is in EVPNv4 table
VTEP1#show bgp l2vpn evpn route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 219
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 1
10
2001:DB8:3::2 (metric 20) (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 16:54:50 PST
(Dual-stack: Prefer IPv6) verify prefix is in EVPNv4 table
VTEP1#show bgp l2 ev route-type 5 0 10.10.10.0 24
BGP routing table entry for [5][100:103][0][24][10.10.10.0]/17, version 159
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 1
10
10.3.3.2 (metric 21) (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: 0.0.0.0, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:30:45 PST
Verify prefix is in EVPNv6 table
VTEP1#show bgp l2 ev route-type 5 0 2001:10:: 128
BGP routing table entry for [5][100:103][0][128][2001:10::]/29, version 105
Paths: (1 available, best #1, table EVPN-BGP-Table)
Not advertised to any peer
Refresh Epoch 1
10
10.3.3.2 (metric 21) (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
EVPN ESI: 00000000000000000000, Gateway Address: ::, VNI Label 30000, MPLS VPN Label 0
Extended Community: RT:100:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(active)
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:17:53 PST
Verify Imported VPN Prefix in Remote VTEP VPN/VRF Table
(Greenfield: VxLANv6) verify prefix is in VPNv4 table
VTEP1#show bgp vpnv4 unicast vrf red 10.10.10.0/24
BGP routing table entry for 100:101:10.10.10.0/24, version 64
Paths: (1 available, best #1, table red)
Not advertised to any peer
Refresh Epoch 1
10, imported path from [5][100:103][0][24][10.10.10.0]/17 (global)
2001:DB8:3::2 (metric 20) (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F500
encap:3
vtep-ip:2001:DB8:1::2
sec-vtep-ip:UNKNOWN
bdi:Vlan3
Remote VxLAN:
Topoid 0x1(vrf red)
Remote Router MAC:AABB.CC81.F700
Encap 8
Egress VNI 30000
RTEP 2001:DB8:3::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 4 2022 20:51:55 PST
(Dual-stack: Prefer IPv6) verify prefix is in VPNv4 table
VTEP1#show bgp vpnv4 unicast vrf red 10.10.10.0/24
BGP routing table entry for 100:101:10.10.10.0/24, version 21
Paths: (1 available, best #1, table red)
Not advertised to any peer
Refresh Epoch 1
10, imported path from [5][100:103][0][24][10.10.10.0]/17 (global)
2001:DB8:3::2 (metric 20) (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:100 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F500
encap:4
vtep-ip:10.1.1.2
sec-vtep-ip:ABCC:1::2
bdi:Vlan3
Remote VxLAN:
Topoid 0x1(vrf red)
Remote Router MAC:AABB.CC81.F700
Encap 8
Egress VNI 30000
RTEP 2001:DB8:3::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:30:45 PST
Verify prefix is in VPNv6 table
VTEP1#show bgp vpnv6 unicast vrf red 2001:10::/128
BGP routing table entry for [100:101]2001:10::/128, version 16
Paths: (1 available, best #1, table red)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
10, imported path from [5][100:103][0][128][2001:10::]/29 (global)
2001:DB8:3::2 (via default) from 10.99.99.99 (10.99.99.99)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:200 OSPF ROUTER ID:10.10.10.0:0
OSPF RT:0.0.0.0:2:0 ENCAP:8 Router MAC:AABB.CC81.F700
Originator: 10.3.3.1, Cluster list: 10.99.99.99
Tunnel Encapsulation Attribute:
Encap type: 8
Secondary nexthop address 2001:DB8:3::2(inaccessible)
Local vxlan vtep:
vrf:red, vni:30000
local router mac:AABB.CC81.F500
encap:4
vtep-ip:10.1.1.2
sec-vtep-ip:ABCC:1::2
bdi:Vlan3
Remote VxLAN:
Topoid 0x1E000001(vrf red)
Remote Router MAC:AABB.CC81.F700
Encap 8
Egress VNI 30000
RTEP 2001:DB8:3::2
rx pathid: 0, tx pathid: 0x0
Updated on Apr 25 2022 04:17:53 PST
Verify Imported VPN Prefix in Remote VTEP IP VRF Table
Verify prefix is in VPNv4 routing table
VTEP1#show ip route vrf red 10.10.10.0
Routing Table: red
Routing entry for 10.10.10.0/24
Known via "bgp 100", distance 200, metric 0
Tag 10, type internal
Last update from 2001:DB8:3::2 on Vlan3, 00:26:36 ago
Routing Descriptor Blocks:
* 2001:DB8:3::2 (red:ipv6), from 10.99.99.99, 00:26:36 ago, via Vlan3
opaque_ptr 0x7F555D459A38
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 10
MPLS label: none
Verify prefix is in VPNv6 routing table
VTEP1#show ipv6 route vrf red 2001:10::/128
Routing entry for 2001:10::/128
Known via "bgp 100", distance 200, metric 0
Tag 10, type internal
Route count is 1/1, share count 0
Routing paths:
2001:DB8:3::2%default, Vlan3%default
Route metric is 0, traffic share count is 1
MPLS label: nolabel
From 10.99.99.99
opaque_ptr 0x7F555D499A68
Last updated 08:19:05 ago
Debug Commands
List of relevant debugs that can be enabled when troubleshooting EVPN VXLANv6 issues
BGP
Verify the route exchange between BGP EVPN and L2RIB/EVPNMgr
Route exchange
debug bgp l2vpn evpn evi event detail
EVI context interaction with EVPNMgr/L2RIB
debug bgp l2vpn evpn evi context detail
Verify the BGP EVPN route update, advertisement and receive
Receive
debug bgp l2vpn evpn update in
Transmit
debug bgp l2vpn evpn update out
Verify the BGP NVE/L3-EVPN interaction for vtep status notification
debug bgp l2vpn evpn nve detail
Verify the BGP bestpath computation details
EVPN
debug bgp l2vpn evpn addpath
VPNv4
debug bgp vpnv4 unicast addpath
VPNv6
debug bgp vpnv6 unicast addpath
Verify the BGP route import between EVPN and VPN tables
Events
debug bgp l2vpn evpn import events
Updates
debug bgp l2vpn evpn import updates
Verify dual-nexthop related events
debug bgp l2vpn evpn import events
debug ip bgp events
EVPN Manager
debug l2vpn evpn error
debug l2vpn evpn event
debug l2vpn evpn event detail
L2RIB
debug l2rib error
debug l2rib event
debug l2rib event detail
L2FIB
debug l2fib all
NVE Manager
debug nve all
Multicast
debug ipv6 mld
debug ipv6 mld group ff05::1
debug ipv6 pim
debug ipv6 mrib route
debug ipv6 mrib route ff05::1
debug ipv6 pim group ff05::1
debug ipv6 mrib table
debug ipv6 mfib platform errors
debug ipv6 mfib platform notify
debug ipv6 mfib events
debug ipv6 mfib errors
debug ipv6 mfib pak ff05::1
debug ipv6 mfib ps ff05::1
debug ipv6 mfib fs ff05::1
Platform Dependent Traces
Feature Specific Traces
Elevate these traces to debug level prior to collection of trace archive
set platform software trace fed switch active l2_fib_entry debug
set platform software trace fed switch active l2_fib_adj debug
set platform software trace fed switch active matm debug
set platform software trace fed switch active asic_l2u debug
set platform software trace fed switch active asic_l3u debug
set platform software trace fed switch active efp debug
set platform software trace fed switch active nve debug
set platform software trace fed switch active l3_tunnel debug
set platform software trace fed switch active l3_adj debug
set platform software trace fed switch active l3_fib debug
set platform software trace fed switch active l3_mcast_aal debug
set platform software trace fed switch active l3_mcast_db debug
set platform software trace fed switch active l3_mcast_mif debug
set platform software trace fed switch active l3_mcast_mroute debug
set platform software trace fed switch active asic_l3m debug
set platform software trace fed switch active asic_app debug
set platform software trace fed switch active asic_rrm noise
Fed Traces Since Last Reboot
show logging process fed internal start last boot switch active to-file flash:<file>
Fed Traces Since Last Clear Log
Show logging process fed internal start last clear switch active to-file flash:<file>
Related Information
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
07-Jun-2023 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)