Verify MPLS on Catalyst 9000 Switches
Available Languages
Contents
Introduction
This document describes the how to configure and validate Multiprotocol Label Switching (MPLS) Layer 3 Virtual Private Network (VPN) on Catalyst 9000 series switches.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
- IP Forwarding
- Border Gateway Protocol (BGP)
- MPLS
Components Used
The information in this document is based on these software and hardware versions:
- C9500 on Cisco IOS® XE 16.12.4
- C9300 on Cisco IOS® XE 16.12.4
- C3850 on Cisco IOS® XE 16.9.6
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Background Information
MPLS Layer 3 VPNs (L3VPN) use a peer-to-peer model that uses BGP to distribute VPN-related information. A MPLS VPN consists of a set of sites that are interconnected by means of a MPLS provider core network. At each customer site, one or more customer edge (CE) devices attach to one or more provide edge (PE) devices.
In conventional Layer 3 routing, as a packet traverses the network, each switch extracts all the information relevant to forward the packet from the Layer 3 header. This information is then used as an index for a routing table lookup to determine the next hop for the packet.
In the most common case, the only relevant field in the header is the destination address field, but in some cases, other header fields might also be relevant. As a result, the header analysis must be done independently at each switch through which the packet passes. In addition, a complicated table lookup must also be done at each switch.
In label switching, the analysis of the Layer 3 header is done only once. The Layer 3 header is then mapped into a fixed length, unstructured value called alabel.
Many different headers can map to the same label, as long as those headers always result in the same choice of next hop. In effect, a label represents aforwarding equivalence class (FEC) that is, a set of packets which, however different they can be indistinguishable by the forwarding function.
The initial choice of a label need not be based exclusively on the contents of the Layer 3 packet header; for example, decisions to forward packets at subsequent hops can also be based on other factors.
Once a label is assigned, a short label header is added at the front of the Layer 3 packet. This header is carried across the network as part of the packet. At subsequent hops through each MPLS switch in the network, labels are swapped and decisions are made by means of MPLS forwarding table lookup for the label carried in the packet header. Hence, the packet header does not need to be reevaluated during packet transit through the network. Because the label is of fixed length and unstructured, the MPLS forwarding table lookup process is both straightforward and fast.
Each Label Switching Router (LSR) in the network makes an independent, local decision as to which label value to use to represent a forwarding equivalence class. This association is known as a label binding. Each LSR informs its neighbors of the label bindings it has made. This awareness of label bindings by neighboring switches is facilitated by these protocols:
-
Label Distribution Protocol (LDP) - Enables peer LSRs in an MPLS network to exchange label information to support hop-by-hop forwarding in an MPLS network
-
Border Gateway Protocol (BGP) - Used to support MPLS Virtual Private Networks (VPNs)
When a labeled packet is sent from LSR A to LSR B, the label value carried by the IP packet is the label value that LSR B assigned to represent the forwarding equivalence class of the packet. Thus, the label value changes as the IP packet traverses the network.
How to Use this Guide
The guide is broken into two scenarios, and a hardware scale validation section is presented at the end of the document:
- single-hop adjacency within the MPLS core
- Equal Cost Multi-Path (ECMP) adjacencies within the MPLS core
- how to check TCAM usage for scale issues
Each scenario covers verification of prefixes and labels for each MPLS device.
Terminology
|
MPLS |
Multi-Protocol Label Switching |
A high-performance packet forwarding technology that integrates the performance and traffic management capabilities of data link layer (Layer 2) switching with the scalability, flexibility, and performance of network layer (Layer 3) routing. |
|
PE |
Provider Edge (switch/router) |
The edge device of the provider network that receives IP prefixes from a customer CE, and passes them into the MPLS cloud. |
|
CE |
Customer Edge (switch/router) |
A device at the customer premises that is connected to the provider edge router of a service provider IP/MPLS network. |
|
LDP |
Label Discovery Protocol |
LDP is a protocol that automatically generates and exchanges labels between routers. Each router locally generates labels for its prefixes and then advertises the label values to its neighbors. |
|
LSPA |
Label Switch Path Array |
The set of labels to reach a specific MPLS destination. In a typical L3VPN – you can have IGP + VPN label. If there is a TE tunnel, then you have TE label + IGP + VPN. Catalyst 9000 can support up to 6 labels, and this array of labels is called LSPA. |
|
Label Stack Id |
Label Stack Id |
A unique index to identify a label stack (allows LSPA sharing). |
|
Label |
Label |
The MPLS label used for lookup. Multiple labels make up label stack. |
|
Prefix ID |
Prefix Identifier |
Catalyst 9000 creates a global resource for every prefix (there are as many prefix IDs as there are routes in the case of per-prefix label allocation). |
|
EM |
Exact Match |
An entry in Hash memory that is a 1:1 match (host route, Directly Connected host). |
|
LPM |
Longest Prefix Match |
Any route that is /31 or shorter (/32 routes are EM type). |
|
TCAM |
Ternary Content-Addressable Memory |
A type of memory that stores and queries entries with three different inputs: 0, 1 and X. This type of memory must be used in cases where there can be multiple matches to the same entry, and the resulting Hash for each would not be unique. This table includes a mask or "X" value that allows it to know if it matches or does not match this entry. |
|
CAM |
Content-Addressable Memory |
General term for hardware memory (Hash/TCAM). |
|
RIB |
Routing Information Base |
the routing table seen in 'show ip route' |
|
FIB |
Forwarding Information Base |
simplified table with prefixes added by the RIB and ARP tables with a pointer to the ADJ table |
|
Directly Connected |
Directly Connected Route |
A locally connected host prefix (ARP adjacent) |
|
Indirectly Connected |
Indirectly Connected Route |
A route that is via a remote next hop to reach |
|
ADJ |
Adjacency (table) |
stores next hop information used for packet rewrite |
|
EM |
Exact Match |
Connected hosts, indirect /32 host prefixes |
|
TCAM |
Ternary Content-Addressable Memory |
Indirect prefixes /31 or shorter |
|
FED |
Forward Engine Driver |
The ASIC (hardware) layer |
|
FMAN-FP |
Forward Manager- Forwarding Plane |
FMAN-FP manages software objects that add, delete, or modify FED information |
|
SI |
Station Index |
Station Index = packet rewrite information (RI = Rewrite Index) & outbound interface information (DI = Destination Index) |
|
RI |
Rewrite Index |
MAC address rewrite information for layer 3 forwarding to the next hop adjacency |
|
DI |
Destination Index |
Index that points to the outbound interface |
Configure and Verify
Scenario 1. L3VPN with Single Hop Adjacency in MPLS Core
Reference Topology
For the purpose of this example, Catalyst 9300 switches functions as the PE devices, Catalyst 9500 in Stackwise Virtual function as the P device, and Catalyst 3850 switches function as the CE devices.
Configuration Details
Configuration of C3850-CE-1
hostname C3850-CE-1
!
interface Loopback0
ip address 192.168.3.1 255.255.255.0
!
interface TenGigabitEthernet1/0/1
no switchport
ip address 10.0.0.1 255.255.255.252
!
router eigrp 420
network 10.0.0.0 0.0.0.3
network 192.168.3.0 0.0.0.255
eigrp stub connected summary
!
ip route 0.0.0.0 0.0.0.0 10.0.0.2
Configuration of C9300-PE-1
hostname C9300-PE-1
!
ip vrf RED
rd 69:69
route-target export 69:69
route-target import 69:69
!
mpls ldp explicit-null
!
interface Loopback0
ip address 192.168.1.2 255.255.255.255
!
interface GigabitEthernet1/0/1
no switchport
ip vrf forwarding RED
ip address 10.0.0.2 255.255.255.252
!
interface GigabitEthernet1/0/2
no switchport
ip address 10.0.0.5 255.255.255.252
!
router eigrp 420
!
address-family ipv4 vrf RED
network 10.0.0.0 0.0.0.3
autonomous-system 420
exit-address-family
!
router ospf 420
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
!
router bgp 69420
bgp log-neighbor-changes
neighbor 192.168.1.4 remote-as 69420
neighbor 192.168.1.4 update-source Loopback0
!
address-family vpnv4
neighbor 192.168.1.4 activate
neighbor 192.168.1.4 send-community extended
exit-address-family
!
address-family ipv4 vrf RED
redistribute eigrp 420
exit-address-family
Configuration of C9500-P
hostname C9500-P
!
interface Loopback0
ip address 192.168.1.3 255.255.255.255
!
interface TenGigabitEthernet1/0/1
no switchport
ip address 10.0.0.6 255.255.255.252
!
interface TenGigabitEthernet1/0/2
no switchport
ip address 10.0.0.13 255.255.255.252
!
router ospf 420
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
Configuration of C9300-CE-2
hostname C9300-PE-2
!
ip vrf RED
rd 69:69
route-target export 69:69
route-target import 69:69
!
mpls ldp explicit-null
!
interface Loopback0
ip address 192.168.1.4 255.255.255.255
!
interface GigabitEthernet2/0/1
no switchport
ip vrf forwarding RED
ip address 10.0.0.21 255.255.255.252
!
interface GigabitEthernet2/0/2
no switchport
ip address 10.0.0.14 255.255.255.252
!
router eigrp 400
!
address-family ipv4 vrf RED
network 10.0.0.20 0.0.0.3
autonomous-system 400
exit-address-family
!
router ospf 420
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
!
router bgp 69420
bgp log-neighbor-changes
neighbor 192.168.1.2 remote-as 69420
neighbor192.168.1.2 update-source Loopback0
!
address-family vpnv4
neighbor 192.168.1.2 activate
neighbor 192.168.1.2 send-community extended
exit-address-family
!
address-family ipv4 vrf RED
redistribute eigrp 400
exit-address-family
Configuration of C3850-CE-2
hostname C3850-CE-2
!
interface Loopback0
ip address 192.168.2.1 255.255.255.0
!
interface TenGigabitEthernet2/0/1
no switchport
ip address 10.0.0.22 255.255.255.252
!
router eigrp 400
network 10.0.0.20 0.0.0.3
network 192.168.2.0 0.0.0.255
eigrp stub connected summary
!
ip route 0.0.0.0 0.0.0.0 10.0.0.21
Basic Verification
Before validation of MPLS programming there are base requirements that need to be validated:
- Validate PE to PE connectivity is present
- Validate the label switched path (LSP) between the PEs
- Validate BGPv4 adjacency between PEs
- Validate VPNv4 and LDP labels
- Validate MPLS Forwarding Table
Validate PE to PE Connectivity
You can ping the remote PE loopback and source from the local loopback, but this does not confirm the MPLS label switched path (LSP) is good, since the Loopback IP addresses are advertised in the underlay.
C9300-PE-1#ping 192.168.1.4 source 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.4, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms C9300-PE-1#show ip route 192.168.1.4
Routing entry for 192.168.1.4/32
Known via "ospf 420", distance 110, metric 3, type intra area
Last update from 10.0.0.10 on GigabitEthernet1/0/3, 00:55:58 ago
Routing Descriptor Blocks:
* 10.0.0.6, from 192.168.1.4, 00:55:58 ago, via GigabitEthernet1/0/2
Route metric is 3, traffic share count is 1
Validate the LSP
You can use a MPLS traceroute from PE to PE loopback to validate the LSP and all MPLS LDP labels along the path.
C9300-PE-1#traceroute mpls ipv4 192.168.1.4/32 source 192.168.1.2 Tracing MPLS Label Switched Path to 192.168.1.4/32, timeout is 2 seconds Codes: '!' - success, 'Q' - request not sent, '.' - timeout, 'L' - labeled output interface, 'B' - unlabeled output interface, 'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch, 'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 'P' - no rx intf label prot, 'p' - premature termination of LSP, 'R' - transit router, 'I' - unknown upstream index, 'l' - Label switched with FEC change, 'd' - see DDMAP for return code, 'X' - unknown return code, 'x' - return code 0 Type escape sequence to abort. 0 10.0.0.5 MRU 1500 [Labels: 17 Exp: 0] L 1 10.0.0.6 MRU 1500 [Labels: explicit-null Exp: 0] 8 ms ! 2 10.0.0.14 2 ms
If you do not have access to the CE or a device behind the CE and you want to demonstrate that there is successful VPNv4 and LDP label imposition/disposition you can attempt to ping from the CE-facing interface in the VRF on a PE to the other CE-facing interface in the VRF on the remote PE.
C9300-PE-1#ping vrf RED 10.0.0.21 source 10.0.0.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.0.0.21, timeout is 2 seconds: Packet sent with a source address of 10.0.0.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
Validate BGP VPNv4 adjacency between PEs
C9300-PE-1#show bgp vpnv4 unicast all neighbors 192.168.1.4
BGP neighbor is 192.168.1.4, remote AS 69420, internal link
BGP version 4, remote router ID 192.168.1.4
BGP state = Established, up for 00:57:37
Last read 00:00:41, last write 00:00:41, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Address family VPNv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 6 6
Keepalives: 62 63
Route Refresh: 0 0
Total: 69 70
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 0 seconds
<snip>
C9300-PE-2#show bgp vpnv4 unicast all neighbors 192.168.1.2
BGP neighbor is 192.168.1.2, remote AS 69420, internal link
BGP version 4, remote router ID 192.168.1.2
BGP state = Established, up for 01:01:00
Last read 00:00:13, last write 00:00:37, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Address family VPNv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 6 6
Keepalives: 67 66
Route Refresh: 0 0
Total: 74 73
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 0 seconds
Remote PE VPNv4 adjacency is up, and a prefix has been received
C9300-PE-1#show bgp vpnv4 unicast all summary BGP router identifier 192.168.1.2, local AS number 69420 BGP table version is 7, main routing table version 7 4 network entries using 1024 bytes of memory 4 path entries using 544 bytes of memory 4/4 BGP path/bestpath attribute entries using 1216 bytes of memory 4 BGP extended community entries using 1000 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 3784 total bytes of memory BGP activity 4/0 prefixes, 4/0 paths, scan interval 60 secs 4 networks peaked at 16:19:10 Jun 1 2021 UTC (01:32:00.716 ago) Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.4 4 69420 108 108 7 0 0 01:34:52 2
C9300-PE-2#show bgp vpnv4 unicast all summary BGP router identifier 192.168.1.4, local AS number 69420 BGP table version is 7, main routing table version 7 4 network entries using 1024 bytes of memory 4 path entries using 544 bytes of memory 4/4 BGP path/bestpath attribute entries using 1216 bytes of memory 4 BGP extended community entries using 1000 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 3784 total bytes of memory BGP activity 4/0 prefixes, 4/0 paths, scan interval 60 secs 4 networks peaked at 16:18:31 Jun 1 2021 UTC (01:37:30.404 ago) Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.2 4 69420 114 114 7 0 0 01:40:22 2
Verify what prefixes are exchanged in the particular VRF
C9300-PE-1#show ip bgp vpnv4 vrf RED
BGP table version is 10, local router ID is 192.168.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 69:69 (default for vrf RED)
*> 10.0.0.0/30 0.0.0.0 0 32768 ?
*>i 10.0.0.20/30 192.168.1.4 0 100 0 ?
*> 192.168.1.0 10.0.0.1 130816 32768 ?
*>i 192.168.2.0 192.168.1.4 130816 100 0 ?
C9300-PE-2#show ip bgp vpnv4 vrf RED
BGP table version is 9, local router ID is 192.168.1.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 69:69 (default for vrf RED)
*>i 10.0.0.0/30 192.168.1.2 0 100 0 ?
*> 10.0.0.20/30 0.0.0.0 0 32768 ?
*>i 192.168.1.0 192.168.1.2 130816 100 0 ?
*> 192.168.2.0 10.0.0.22 130816 32768 ?
Validate VPNv4 and LDP Labels:
Verify the VPNv4 label that are used to reach the prefixes in the VRF
C9300-PE-1#show ip bgp vpnv4 vrf RED labels Network Next Hop In label/Out label Route Distinguisher: 69:69 (RED) 10.0.0.0/30 0.0.0.0 20/nolabel(RED) 10.0.0.20/30 192.168.1.4 nolabel/20 192.168.1.0 10.0.0.1 21/nolabel 192.168.2.1/32 192.168.1.4 nolabel/21 <-- VPNv4 label that is imposed to reach 192.168.2.0
C9300-PE-1#show ip route vrf RED 192.168.2.1
Routing Table: RED
Routing entry for 192.168.2.0/24
Known via "bgp 69420", distance 200, metric 130816, type internal
Last update from 192.168.1.4 01:31:56 ago
Routing Descriptor Blocks:
* 192.168.1.4 (default), from 192.168.1.4, 01:31:56 ago
Route metric is 130816, traffic share count is 1
AS Hops 0
MPLS label: 21 <-- VPNv4 label that matches the previous output
MPLS Flags: MPLS Required
C9300-PE-2#show ip bgp vpnv4 vrf RED labels Network Next Hop In label/Out label Route Distinguisher: 69:69 (RED) 10.0.0.0/30 192.168.1.2 nolabel/20 10.0.0.20/30 0.0.0.0 20/nolabel(RED) 192.168.1.0 192.168.1.2 nolabel/21 192.168.2.0. 10.0.0.22 21/nolabel <-- VPNv4 label that is advertised to reach 192.168.2.0 C9300-PE-2#show ip route vrf RED 192.168.2.1 Routing Table: RED Routing entry for 192.168.2.0/24 Known via "eigrp 400", distance 90, metric 130816, precedence routine (0), type internal Redistributing via eigrp 400, bgp 69420 Advertised by bgp 69420 Last update from 10.0.0.22 on GigabitEthernet2/0/1, 01:34:42 ago Routing Descriptor Blocks: * 10.0.0.22, from 10.0.0.22, 01:34:42 ago, via GigabitEthernet2/0/1 <-- CE-facing interface in the VRF Route metric is 130816, traffic share count is 1 Total delay is 5010 microseconds, minimum bandwidth is 1000000 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1
Verify the LDP labels that are utilized
C9300-PE-1#show mpls forwarding-table 192.168.1.4 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6 <-- 17 is the LDP label imposed to reach PE at 192.168.1.4 through Gi1/0/2 C9300-PE-2#show mpls forwarding-table 192.168.1.2 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 17 16 192.168.1.2/32 0 Gi2/0/2 10.0.0.13 <-- 16 is the LDP label imposed to reach PE at 192.168.1.4 through Gi2/0/2
Validate the MPLS Forwarding Table
C9300-PE-1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 192.168.1.3/32 0 Gi1/0/2 10.0.0.6
17 Pop Label 10.0.0.16/30 0 Gi1/0/2 10.0.0.6
18 Pop Label 10.0.0.12/30 0 Gi1/0/2 10.0.0.6
19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6
20 No Label 10.0.0.0/30[V] 1982 aggregate/RED
21 No Label 192.168.3.0/24[V] \
0 Gi1/0/1 10.0.0.1
C9300-PE-2#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 192.168.1.3/32 0 Gi2/0/2 10.0.0.13
Pop Label 192.168.1.3/32 0 Gi2/0/3 10.0.0.17
17 16 192.168.1.2/32 164 Gi2/0/2 10.0.0.13
16 192.168.1.2/32 1224 Gi2/0/3 10.0.0.17
18 Pop Label 10.0.0.4/30 0 Gi2/0/2 10.0.0.13
Pop Label 10.0.0.4/30 0 Gi2/0/3 10.0.0.17
20 No Label 10.0.0.20/30[V] 0 aggregate/RED
21 No Label 192.168.2.0/24[V] \
1440 Gi2/0/1 10.0.0.22
Confirm the inner (VPNv4) and outer (LDP) labels used to reach to each given prefix in the VRF
C9300-PE-1#show ip cef vrf RED 192.168.2.0/24 detail
192.168.2.1/32, epoch 0, flags [rib defined all labels]
recursive via 192.168.1.4 label 21 <-- VPNv4 label
nexthop 10.0.0.6 GigabitEthernet1/0/2 label 17-(local:19) <-- 17 is the LDP label that is be imposed to reach the remote PE,
19 is the local LDP label advertised to the P router
C9300-PE-2#show ip cef vrf RED 192.168.3.0/24 detail
192.168.1.1/32, epoch 0, flags [rib defined all labels]
recursive via 192.168.1.2 label 22 <-- VPNv4 label
nexthop 10.0.0.13 GigabitEthernet2/0/2 label 16-(local:17) <-- 16 is the LDP label that is be imposed to reach the remote PE,
17 is the local LDP label advertised to the P router
Verify Object-Manager Statistics
In ideal scenarios, there are no pending objects
C9300-PE-1#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 491 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
9500-P#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 491 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
C9300-PE-2#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 482 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
Prefix Programming
The next section covers prefix programming on the MPLS routers, C9300-PE-1, C9500-P, and C9300-PE-2.
C9300-PE-1 Prefix Programming
***Software Prefix Programming***
C9300-PE-1#show ip route vrf RED 192.168.2.1 Routing Table: RED Routing entry for 192.168.2.0/24 Known via "bgp 69420", distance 200, metric 130816, type internal Last update from 192.168.1.4 20:21:40 ago Routing Descriptor Blocks: * 192.168.1.4 (default), from 192.168.1.4, 20:21:40 ago <-- Remote PE reachable in the global routing table Route metric is 130816, traffic share count is 1 AS Hops 0 MPLS label: 21 <-- VPNv4 label MPLS Flags: MPLS Required
C9300-PE-1#show ip route 192.168.1.4
Routing entry for 192.168.1.4/32
Known via "ospf 420", distance 110, metric 3, type intra area
Last update from 10.0.0.6 on GigabitEthernet1/0/2, 21:27:11 ago
Routing Descriptor Blocks:
* 10.0.0.6, from 192.168.1.4, 21:27:11 ago, via GigabitEthernet1/0/2 <-- Next-hop 10.0.0.6 via Gi1/0/2 to reach
Route metric is 3, traffic share count is 1
***FMAN RP Prefix Programming***
C9300-PE-1#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID <-- VRF ID is important in subsequent command Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi1/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix
C9300-PE-1#show platform software ip switch active r0 cef table index 2 prefix 192.168.2.0/24 <-- Index value is the VRF ID from previous command Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.2.0/24 OBJ_LABEL 0x14 C9300-PE-1#show platform software mpls switch active r0 label index 0x14 <-- Utilize the Index value from previous command Label OCE 0x14 -> OBJ_LABEL (0x17) <-- Utilized in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0x15 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480636fb0 C9300-PE-1#show platform software mpls switch active r0 label index 0x17 <-- Utilize the OBJ_LABEL value from previous command Label OCE 0x17 -> OBJ_ADJACENCY (0x46) <-- Utilized in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348062f858 C9300-PE-1#show platform software adjacency switch active r0 index 0x46 <-- Utilize the OBJ_ADJACENCY value from previous command Number of adjacency objects: 6 Adjacency id: 0x46 (70) Interface: GigabitEthernet1/0/2, IF index: 54, Link Type: MCP_LINK_TAG <-- Egress interface Encap: d4:ad:71:b5:dd:e4:a0:f8:49:11:d1:d6:88:47 <-- MAC ending in DDE4 is the DMAC, MAC ending in D1D6 is SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.6 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x3480636280
***FMAN FP Prefix Programming*** C9300-PE-1#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID <-- VRF ID is important in subsequent command Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi1/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix C9300-PE-1#show platform software ip switch active f0 cef table index 2 prefix 192.168.2.0/24 detail <-- Index value is the VRF ID from previous command Forwarding Table 192.168.2.0/24 -> OBJ_LABEL (0x14), urpf: 15 <-- Utilized in next command Prefix Flags: unknown aom id: 648, HW handle: (nil) (created) C9300-PE-1#show platform software mpls switch active f0 label index 0x14 <-- Utilize the OBJ_LABEL value from the previous command Label OCE 0x14 -> OBJ_LABEL (0x17) <-- Utilized in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0x15 Backup flags: Pop, UHP, backup label 0x100001 aom id: 647, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software mpls switch active f0 label index 0x17 <-- Utilize the OBJ_LABEL value from the previous command Label OCE 0x17 -> OBJ_ADJACENCY (0x46) <-- Utilized in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 664, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software adjacency switch active f0 index 0x46 <-- Utilize the OBJ_ADJACENCY value from the previous command Number of adjacency objects: 6 Adjacency id: 0x46 (70) Interface: GigabitEthernet1/0/2, IF index: 54, Link Type: MCP_LINK_TAG <-- Egress interface Encap: d4:ad:71:b5:dd:e4:a0:f8:49:11:d1:d6:88:47 <-- MAC ending in DDE4 is the DMAC, MAC ending in D1D6 is SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.6 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 522, HW handle: (nil) (created)
***FED Prefix Programming***
C9300-PE-1#show platform software fed switch active ip route vrf-name RED 192.168.2.0/24 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 2 192.168.2.0/24 0x7feeeca12bb8 0x0 0 0 lspa0x2 2021/06/14 17:13:59.644 <-- HTM value significant for next command FIB: prefix_hdl:0x5000002a, mpls_ecr_prefix_hdl:0 ========== OCE chain ===== LABEL:objid:20 link_type:MPLS local_label:1048577 outlabel:(21, 0) <-- VPNv4 Label flags:0x1:(REAL,) pdflags:0x80:(INSTALL_HW_OK,RECIR_ADJ,) adj_handle:0x5100003d <-- adj_handle and local_adj_hdl values must match unsupported recursion:0 olbl_changed 0 local_adj:1 modify_cnt:1 bwalk_cnt:0 subwalk_cnt:1 collapsed_oce:0 AAL: id:1358954557 lbl:19 smac:0000.0000.0000 dmac:0000.0000.0000 <-- Label 19 matches the local transport label sub_type:0 link_type:0 adj_flags:0x10 label_type:0 rewrite_type:PSH2(121) vlan_id:0 vrf_id:0 ri:0x7feeeca9acf8, ri_id:0x46 phdl:0, ref_cnt:2 <-- ri_id and ri_idx values must match si:0x7feeeca6ab98, si_id:0xb6, di_id:0x5013 LABEL:objid:23 link_type:MPLS local_label:19 outlabel:(17, 0) <-- Label 19 is the local transport label, Label 17 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x50000034 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1342177332 lbl:0 smac:a0f8.4911.d1d6 dmac:d4ad.71b5.dde4 <-- Matches the next-hop information to reach 192.168.2.0/24 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7feeecd6d238, ri_id:0x3e phdl:0x9f00004b, ref_cnt:1 si:0x7feeecd706d8, si_id:0x4013, di_id:0x535f <-- di_id utilized in subsequent commands ADJ:objid:70 {link_type:MPLS ifnum:0x36, si:0x94000021, } =============== MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0xab000003 AAL:LSPA: id:2868903939 num_path:1 prefix_id:0x2 delete_hw_hdl_cnt:0 cookie[64]: 15000000000000000000000000000000000000000000000000000000000000004600000000000000000000000000000000000000000000000000000000000000 status:ok vpn_lbl:21 local_adj_hdl:0x5100003d hw_hdl:0x7feeecab2d48 ri_idx:0x46 <-- vpn_lbl matches the VPNv4 label,adj_handle and local_adj_hdl values must match,ri_id and ri_idx values must match =============== C9300-PE-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7feeeca12bb8 1 <-- Utilize HTM value from previous command Handle:0x7feeeca12bb8 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7feeeca2af28 Features sharing this resource:Cookie length: 12 01 02 a8 c0 00 00 02 d0 07 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7feeeca2af28) Absolute Index: 66036 Time Stamp: 160003 KEY - vrf:2 mtr:0 prefix:192.168.2.0 rcp_redirect_index:0x0 MASK - vrf:0 mtr:0 prefix:0.0.0.255 rcp_redirect_index:0x0 FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:182 destined_to_us:0 hw_stats_idx:0 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0x2 SRC-AD = learning_violation:0 need_to_learn:0 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:0 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:0 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9300-PE-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x535f 0x535f <-- Utilize the di_id from the previous command ASIC#0: index = 0x535f pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x535f pmap = 0x00000000 0x00000002 <-- Looking at 0x00000002, in binary that is 0000 0000 0000 0000 0000 0000 0000 0010 = Port 1 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9300-PE-1#show plat soft fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/2 0x36 1 0 1 1 0 6 7 2 2 NIF Y <-- Port 1 is the egress port, Gi1/0/2
C9500-P Prefix Programming
***Software Prefix Programming***
C9500-P#show ip route 192.168.1.4 Routing entry for 192.168.1.4/32 Known via "ospf 420", distance 110, metric 2, type intra area Last update from 10.0.0.14 on TenGigabitEthernet1/0/2, 1d21h ago Routing Descriptor Blocks: * 10.0.0.14, from 192.168.1.4, 1d21h ago, via TenGigabitEthernet1/0/2 <-- Next-hop to reach 192.168.1.4 Route metric is 2, traffic share count is 1 C9500-P#show ip cef 192.168.1.4 detail 192.168.1.4/32, epoch 4 dflt local label info: global/17 [0x3] nexthop 10.0.0.14 TenGigabitEthernet1/0/2 label explicit-null-(local:17)
***FMAN RP Prefix Programming*** C9500-P#show platform software ip switch active r0 cef prefix 192.168.1.4/32 Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.1.4/32 OBJ_LABEL 0x16 <-- Value used in next command C9500-P#show platform software mpls switch active r0 label index 0x16 <-- Utilize the OBJ_LABEL value from previous command Label OCE 0x16 -> OBJ_ADJACENCY (0x49) <-- Value used in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x34806492f0 C9500-P#show platform software adjacency switch active r0 index 0x49 <-- Utilize OBJ_ADJACENCY value from previous command Number of adjacency objects: 8 Adjacency id: 0x49 (73) Interface: TenGigabitEthernet1/0/2, IF index: 66, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:71:d4:ad:71:b5:dd:d6:88:47 <-- MAC ending in AE71 is the DMAC, MAC ending in DDD6 is the SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.14 <-- Next-hop IP IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x3480647760
***FMAN FP Prefix Programming*** C9500-P#show platform software ip switch active f0 cef prefix 192.168.1.4/32 detail Forwarding Table 192.168.1.4/32 -> OBJ_LABEL (0x16), urpf: 21 <-- Used in subsequent command Prefix Flags: unknown aom id: 567, HW handle: (nil) (created) C9500-P#show platform software mpls switch active f0 label index 0x16 <-- Utilize the OBJ_LABEL value from previous command Label OCE 0x16 -> OBJ_ADJACENCY (0x49) <-- Used in subsequent command Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 589, CPP handle: 0xdeadbeef (created) C9500-P#show platform software adjacency switch active f0 index 0x49 <-- Utilize the OBJ_ADJACENCY from previous command Number of adjacency objects: 8 Adjacency id: 0x49 (73) Interface: TenGigabitEthernet1/0/2, IF index: 66, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:71:d4:ad:71:b5:dd:d6:88:47 <-- MAC ending in AE71 is the DMAC, MAC ending in DDD6 is the SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.14 <-- Next-hop IP IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 535, HW handle: (nil) (created)
*** FED Prefix Programming*** C9500-P#show platform software fed switch active ip route 192.168.1.4/32 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 0 192.168.1.4/32 0x7f790c4cf0e8 0x0 0 0 2021/06/14 22:10:54.150 <-- HTM value significant for next command FIB: prefix_hdl:0x6a000020, mpls_ecr_prefix_hdl:0 ========== OCE chain ===== LABEL:objid:22 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Label 17 is the local transport label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xb9000037 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:3103785015 lbl:0 smac:d4ad.71b5.ddd6 dmac:70d3.79be.ae71 <-- Matches the next-hop information to reach 192.168.1.4/32 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f790c4cdfd8, ri_id:0x38 phdl:0x76000058, ref_cnt:1 si:0x7f790c4c22f8, si_id:0x400b, di_id:0x2 <-- di_id utilized in subsequent commands ADJ:objid:73 {link_type:MPLS ifnum:0x42, si:0x2d000027, } =============== MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0 =============== C9500-P#show platform hardware fwd-asic abstraction print-resource-handle 0x7f790c4cf0e8 1 <-- Utilize the HTM value from previous command Handle:0x7f790c4cf0e8 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f790c4cf2f8 Features sharing this resource:Cookie length: 12 04 01 a8 c0 00 00 00 d0 07 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7f790c4cf2f8) Absolute Index: 126650 Time Stamp: 40 KEY - vrf:0 mtr:0 prefix:192.168.1.4 rcp_redirect_index:0x0 MASK - vrf:0 mtr:0 prefix:0.0.0.0 rcp_redirect_index:0x0 FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:16395 destined_to_us:0 hw_stats_idx:1 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0 SRC-AD = learning_violation:0 need_to_learn:0 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:38 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:1 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9500-P#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x2 0x2 <-- Utilize the di_id value from the previous command ASIC#0: index = 0x2 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x2 pmap = 0x00000000 0x00000002 <-- 0x00000002 in binary is 0000 0000 0000 0000 0000 0000 0000 = Port 1 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9500-P#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active TenGigabitEthernet1/0/2 0x42 1 0 1 1 0 10 1 2 2 NIF Y <-- Port 1 is the egress port, TenGig1/0/2
C9300-PE-2 Prefix Programming
***Software Prefix Programming***
C9300-PE-2#show ip route vrf RED 192.168.2.1 Routing Table: RED Routing entry for 192.168.2.0/24 Known via "eigrp 400", distance 90, metric 130816, precedence routine (0), type internal Redistributing via eigrp 400, bgp 69420 Advertised by bgp 69420 Last update from 10.0.0.22 on GigabitEthernet2/0/1, 1d21h ago Routing Descriptor Blocks: * 10.0.0.22, from 10.0.0.22, 1d21h ago, via GigabitEthernet2/0/1 <-- Next-hop reachable in the VRF Route metric is 130816, traffic share count is 1 Total delay is 5010 microseconds, minimum bandwidth is 1000000 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1 C9300-PE-2#show ip route vrf RED 10.0.0.22 Routing Table: RED Routing entry for 10.0.0.20/30 Known via "connected", distance 0, metric 0 (connected, via interface) Redistributing via eigrp 400, bgp 69420 Advertised by bgp 69420 Routing Descriptor Blocks: * directly connected, via GigabitEthernet2/0/1 <-- Next-hop directly connected Route metric is 0, traffic share count is 1 C9300-PE-2#show ip cef vrf RED 192.168.2.0/24 detail 192.168.2.0/24, epoch 0 QOS: Precedence routine (0) dflt local label info: other/21 [0x2] nexthop 10.0.0.22 GigabitEthernet2/0/1
***FMAN RP Prefix Programming*** C9300-PE-2#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID <-- VRF ID is important in subsequent command Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi2/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix C9300-PE-2#show platform software ip switch active r0 cef table index 2 prefix 192.168.2.0/24 Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.2.0/24 OBJ_ADJACENCY 0x19 C9300-PE-2#show platform software adjacency switch active r0 index 0x19 <-- Utilize the Index value from previous command Number of adjacency objects: 6 Adjacency id: 0x19 (25) Interface: GigabitEthernet2/0/1, IF index: 53, Link Type: MCP_LINK_IP Encap: 0:72:78:c8:c9:c2:70:d3:79:be:ae:42:8:0 <-- MAC ending in C9C2 is DMAC, MAC ending in AE42 is SMAC, 0x800 is the IP ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: no-l3-inject Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.22 IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x348062f118
***FMAN FP Prefix Programming*** C9300-PE-2#show platform software ip switch active f0 cef table index 2 prefix 192.168.2.0/24 detail Forwarding Table 192.168.2.0/24 -> OBJ_ADJACENCY (0x19), urpf: 30 <-- Utilized in next command Prefix Flags: unknown aom id: 665, HW handle: (nil) (created) QPPB precedence: 0 C9300-PE-2#show platform software adjacency switch active f0 index 0x19 <-- Utilize the OBJ_ADJACENCY from previous command Number of adjacency objects: 6 Adjacency id: 0x19 (25) Interface: GigabitEthernet2/0/1, IF index: 53, Link Type: MCP_LINK_IP Encap: 0:72:78:c8:c9:c2:70:d3:79:be:ae:42:8:0 Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: no-l3-inject Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.22 IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 659, HW handle: (nil) (created)
***FED Prefix Programming*** C9300-PE-2#show platform software fed switch active ip route vrf-name RED 192.168.2.0/24 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 2 192.168.2.0/24 0x7f7fb4a25648 0x0 0 0 2021/06/14 17:04:13.460 <-- HTM value significant for next command FIB: prefix_hdl:0x6e00002a, mpls_ecr_prefix_hdl:0 ========== OCE chain ===== ADJ:objid:25 {link_type:IP ifnum:0x35, si:0x3300003e, IPv4: 10.0.0.22 } =============== MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0 =============== C9300-PE-2#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f7fb4a25648 1 <-- Utilize HTM value from previous command Handle:0x7f7fb4a25648 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f7fb4a10e58 Features sharing this resource:Cookie length: 12 01 02 a8 c0 00 00 02 d0 07 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7f7fb4a10e58) Absolute Index: 66036 Time Stamp: 164911 KEY - vrf:2 mtr:0 prefix:192.168.2.0 rcp_redirect_index:0x0 MASK - vrf:0 mtr:0 prefix:0.0.0.255 rcp_redirect_index:0x0 FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:182 destined_to_us:0 hw_stats_idx:1 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0 SRC-AD = learning_violation:0 need_to_learn:0 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:37 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:1 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9300-PE-2#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ----- ------------------------ 10.0.0.22 GigabitEthernet2/0/1 0072.78c8.c9c2 0x7f7fb4a44048 0x7f7fb4b089d8 0x0 0x19 2021/06/14 16:59:43.447 <-- si_hdl used in next command C9300-PE-2#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f7fb4a44048 1 <-- Utilize the si_hdl value from previous command Handle:0x7f7fb4a44048 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f7fb4b089d8Hardware Indices/Handles: index0:0xb6 mtu_index/l3u_ri_index0:0x0 index1:0xb6 mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 25 00 00 00 00 00 00 00 00 00 00 00 08 00 00 72 78 c8 c9 c2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Station Index (SI) [0xb6] RI = 0x2b DI = 0x5338 stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: CD Detailed Resource Information (ASIC# 1) ---------------------------------------- Station Index (SI) [0xb6] RI = 0x2b DI = 0x5338 stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD ============================================================== C9300-PE-2#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x5338 0x5338 <-- Utilize the DI value from previous command ASIC#0: index = 0x5338 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x5338 pmap = 0x00000000 0x00000001 <-- 0x00000001 in binary is 0000 0000 0000 0000 0000 0000 0000 0001 = Port 0 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9300-PE-2#show platform software fed switch active ifm map Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet2/0/1 0x35 1 0 1 0 0 26 6 1 97 NIF Y <-- Port 0 is the egress port, Gi2/0/1
VPNv4 Label Programming
The next section covers VPNv4 label programming on the MPLS PE routers, C9300-PE-1 and C9300-PE-2. The C9500 does not forward on the VPNv4 label so there is no output from the C9500.
C9300-PE-1 VPNv4 Label Programming:
Check the local prefix to the PE, not the remote prefix.
***Software VPNv4 Label Programming***
C9300-PE-1#show ip cef vrf RED 192.168.3.0/24 detail 192.168.3.0/24, epoch 0 QOS: Precedence routine (0) dflt local label info: other/22 [0x2] <-- VPNv4 label associated with the local prefix nexthop 10.0.0.1 GigabitEthernet1/0/1
*** FMAN RP VPNv4 Label Programming*** C9300-PE-1#show platform software mpls switch active r0 eos index 24 <-- Utilize the objid from the FED command EOS Choice 0x18, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x17 OM handle: 0x3480631760
***FMAN FP VPNv4 Label Programming*** C9300-PE-1#show platform software mpls switch active f0 eos index 24 <-- Utilize the objid from the FED command EOS Choice 0x18, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x17 aom id: 5748, CPP handle: 0xdeadbeef (created), flags: 0 <-- Utilized in subsequent command C9300-PE-1#show platform software object-manager switch active f0 object 5748 <-- Utilize the aom id from previous command Object identifier: 5748 Description: EOS Choice 0x18 Status: Done, Epoch: 0, Client data: 0x63150908 C9300-PE-1#show platform software object-manager switch active f0 object 5748 parents <-- Utilize the aom id Object identifier: 7 Description: Special Object adj_drop Status: Done Object identifier: 5746 Description: label 0x17 Status: Done
***FED VPNv4 Label Programming*** C9300-PE-1#show platform software fed switch active mpls forwarding label 22 detail LENTRY:label:22 nobj:(EOS, 24) lentry_hdl:0x800000a modify_cnt:1 backwalk_cnt:0 lspa_handle:0 AAL: id:134217738 lbl:22 eos0:[adj_hdl:0, hw_hdl:0x7fa4c4d72e08] eos1:[adj_hdl:0x6e00003e, hw_hdl:0x7fa4c4d72c58] deagg_vrf_id = 0 lspa_handle:0 EOS:objid:24 local_label:0 flags:0:() pdflags:0 <-- Utilized in previous commands nobj0:(ADJ SPECIAL,DROP 0), nobj1:(LABEL, 23) modify:0 bwalk:0 LABEL:objid:23 link_type:IP local_label:22 outlabel:(1048577, 0) flags:0xc:(UHP,POP,) pdflags:0x2:(INSTALL_HW_OK,) adj_handle:0x6e00003e unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1845493822 lbl:0 smac:a0f8.4911.d1e4 dmac:0072.78c8.06e4 sub_type:0 link_type:0 adj_flags:0x2 label_type:1 rewrite_type:POP2IP(135) vlan_id:0 vrf_id:0 ri:0x7fa4c4a81af8, ri_id:0x44 phdl:0xf1000024, ref_cnt:1 si:0x7fa4c4d83da8, si_id:0x4012, di_id:0x5338 ADJ:objid:113 {link_type:IP ifnum:0x35, si:0x2000003a, IPv4: 10.0.0.1 }
Verify C9300-PE-2 VPNv4 Label:
Check the local prefix to the PE, not the remote prefix
***Software VPNv4 Label Programming***
C9300-PE-2#show ip cef vrf RED 192.168.2.0/24 detail 192.168.2.0/24, epoch 0 QOS: Precedence routine (0) dflt local label info: other/21 [0x2] <-- VPNv4 label associated with local prefix nexthop 10.0.0.22 GigabitEthernet2/0/1
*** FMAN RP VPNv4 Label Programming*** C9300-PE-2#show platform software mpls switch active r0 eos index 61 <-- Use the objid from the FED command EOS Choice 0x3d, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x3b OM handle: 0x348063f2f8
*** FMAN FP VPNv4 Label Programming*** C9300-PE-2#show platform software mpls switch active f0 eos index 61 <-- Use the objid from the FED command EOS Choice 0x3d, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x3b aom id: 3541, CPP handle: 0xdeadbeef (created), flags: 0 <-- Utilized in subsequent command C9300-PE-2#show platform software object-manager switch active f0 object 3541 <-- Use the aom id from previous command Object identifier: 3541 Description: EOS Choice 0x3d Status: Done, Epoch: 0, Client data: 0x11079188 C9300-PE-2#show platform software object-manager switch active f0 object 3541 parents <-- Use the aom id from previous command Object identifier: 7 Description: Special Object adj_drop Status: Done Object identifier: 3540 Description: label 0x3b Status: Done
*** FED VPNv4 Label Programming*** C9300-PE-2#show platform software fed switch active mpls forwarding label 21 detail LENTRY:label:21 nobj:(EOS, 61) lentry_hdl:0x69000009 modify_cnt:3 backwalk_cnt:0 lspa_handle:0 AAL: id:1761607689 lbl:21 eos0:[adj_hdl:0, hw_hdl:0x7fe8f8a71bd8] eos1:[adj_hdl:0x49000040, hw_hdl:0x7fe8f8a72458] deagg_vrf_id = 0 lspa_handle:0 EOS:objid:61 local_label:0 flags:0:() pdflags:0 <-- Utilized in previous commands nobj0:(ADJ SPECIAL,DROP 0), nobj1:(LABEL, 59) modify:0 bwalk:0 LABEL:objid:59 link_type:IP local_label:21 outlabel:(1048577, 0) flags:0xc:(UHP,POP,) pdflags:0x2:(INSTALL_HW_OK,) adj_handle:0x49000040 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1224736832 lbl:0 smac:70d3.79be.ae42 dmac:0072.78c8.c9c2 sub_type:0 link_type:0 adj_flags:0x2 label_type:1 rewrite_type:POP2IP(135) vlan_id:0 vrf_id:0 ri:0x7fe8f8a8ab98, ri_id:0x44 phdl:0xf1000024, ref_cnt:1 si:0x7fe8f8a6ae08, si_id:0x4006, di_id:0x5338 ADJ:objid:25 {link_type:IP ifnum:0x35, si:0x800003e, IPv4: 10.0.0.22 }
LDP Label Programming
The next section covers LDP label programming on the MPLS routers, C9300-PE-1, C9500-P, and C9300-PE-2.
The LDP (outer) label is what the MPLS network label-switches the packets on. Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label.
C9300-PE-1 LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9300-PE-1#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 Pop Label 192.168.1.3/32 0 Gi1/0/2 10.0.0.6 18 Pop Label 10.0.0.12/30 0 Gi1/0/2 10.0.0.6 19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6 <-- LDP Label 19 is advertised to Remote PE 192.168.1.4, validate LDP label 19 20 No Label 10.0.0.0/30[V] 1890 aggregate/RED 22 No Label 192.168.3.0/24[V] \ 1982 Gi1/0/1 10.0.0.1
***FMAN RP LDP Label Programming*** C9300-PE-1#show platform software mpls switch active r0 label index 59 Label OCE 0x3b -> OBJ_ADJACENCY (0x46) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x34805f3dc8 ***FMAN FP LDP Label Programming*** C9300-PE-1#show platform software mpls switch active f0 label index 59 Label OCE 0x3b -> OBJ_ADJACENCY (0x46) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 7065, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software object-manager switch active f0 object 7065 Object identifier: 7065 Description: label 0x3b Status: Done, Epoch: 0, Client data: 0x63152218 C9300-PE-1#show platform software object-manager switch active f0 object 7065 parents Object identifier: 511 Description: adj 0x46, Flags None Status: Done ***FED LDP Label Programming***
C9300-PE-1#show platform software fed switch active mpls forwarding label 19 detail LENTRY:label:19 nobj:(LABEL, 59) lentry_hdl:0xef000007 modify_cnt:7 backwalk_cnt:0 lspa_handle:0 AAL: id:4009754631 lbl:19 eos0:[adj_hdl:0x91000056, hw_hdl:0x7fa4c4d6cae8] eos1:[adj_hdl:0x91000056, hw_hdl:0x7fa4c4d6c8e8] deagg_vrf_id = 0 lspa_handle:0 LABEL:objid:59 link_type:MPLS local_label:19 outlabel:(17, 0) flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x91000056 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:2432696406 lbl:0 smac:a0f8.4911.d1d6 dmac:d4ad.71b5.dde4 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fa4c4d75fa8, ri_id:0x26 phdl:0x9f00004b, ref_cnt:1 si:0x7fa4c4d5f6c8, si_id:0x4013, di_id:0x535f ADJ:objid:70 {link_type:MPLS ifnum:0x36, si:0x25000021, }
C9500 LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9500-P#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 explicit-n 192.168.1.2/32 23409 Te1/0/1 10.0.0.5 <-- LDP label 16 is advertised to reach PE 192.168.1.2 17 explicit-n 192.168.1.4/32 23345 Te1/0/2 10.0.0.14 <-- LDP label 17 is advertised to reach PE 192.168.1.4
***FMAN RP LDP Label Programming*** C9500-P#show platform software mpls switch active r0 label index 23 <-- Use the obj id from the FED command Label OCE 0x17 -> OBJ_ADJACENCY (0x3f) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480645150 ***FMAN FP LDP Label Programming*** C9500-P#show platform software mpls switch active f0 label index 23 <-- Use the obj id from the FED command Label OCE 0x17 -> OBJ_ADJACENCY (0x3f) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 654, CPP handle: 0xdeadbeef (created) C9500-P#show platform software object-manager switch active f0 object 654 <-- Use the aom id from the previous command Object identifier: 654 Description: label 0x17 Status: Done, Epoch: 0, Client data: 0x4b41c08 C9500-P#show platform software object-manager switch active f0 object 654 parents <-- Use the aom id from the previous command Object identifier: 515 Description: adj 0x3f, Flags None Status: Done
***FED LDP Label Programming*** C9500-P#show platform software fed switch active mpls forwarding label 16 detail LENTRY:label:16 nobj:(LABEL, 23) lentry_hdl:0xec000004 modify_cnt:6 backwalk_cnt:0 lspa_handle:0 AAL: id:3959422980 lbl:16 eos0:[adj_hdl:0xc3000055, hw_hdl:0x7f28944be3c8] eos1:[adj_hdl:0xc3000055, hw_hdl:0x7f28944be1b8] deagg_vrf_id = 0 lspa_handle:0 LABEL:objid:23 link_type:MPLS local_label:16 outlabel:(0, 0) <-- Utilized in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xc3000055 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:3271557205 lbl:0 smac:d4ad.71b5.dde4 dmac:a0f8.4911.d1d6 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f289449bf88, ri_id:0x44 phdl:0xe9000057, ref_cnt:1 si:0x7f2894489b58, si_id:0x4009, di_id:0x1 ADJ:objid:63 {link_type:MPLS ifnum:0x41, si:0x57000023, }
***Software LDP Label Programming*** C9500-P#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 explicit-n 192.168.1.2/32 23409 Te1/0/1 10.0.0.5 17 explicit-n 192.168.1.4/32 23345 Te1/0/2 10.0.0.14
***FMAN RP LDP Label Programming*** C9500-P#show platform software mpls switch active r0 label index 64 <-- Use the obj id from the FED command Label OCE 0x40 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480641d08
***FMAN FP LDP Label Programming*** C9500-P#show platform software mpls switch active f0 label index 64 <-- Use the obj id from the FED command Label OCE 0x40 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 657, CPP handle: 0xdeadbeef (created) C9500-P#show platform software object-manager switch active f0 object 657 <-- Use the aom id value from previous command Object identifier: 657 Description: label 0x40 Status: Done, Epoch: 0, Client data: 0x4b523f8 C9500-P#show platform software object-manager switch active f0 object 657 parents<-- Use the aom id value from previous command Object identifier: 535 Description: adj 0x49, Flags None Status: Done
***FED LDP Label Programming*** C9500-P#show platform software fed switch active mpls forwarding label 17 detail LENTRY:label:17 nobj:(LABEL, 64) lentry_hdl:0x8d000005 modify_cnt:6 backwalk_cnt:0 lspa_handle:0 AAL: id:2365587461 lbl:17 eos0:[adj_hdl:0xcc000037, hw_hdl:0x7f2894480438] eos1:[adj_hdl:0xcc000037, hw_hdl:0x7f2894480228] deagg_vrf_id = 0 lspa_handle:0 LABEL:objid:64 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Utilized in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xcc000037 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:3422552119 lbl:0 smac:d4ad.71b5.ddd6 dmac:70d3.79be.ae71 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f2894498008, ri_id:0x38 phdl:0x76000058, ref_cnt:1 si:0x7f2894498478, si_id:0x400b, di_id:0x2 ADJ:objid:73 {link_type:MPLS ifnum:0x42, si:0x3d000027, }
C9300-PE-2 LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9300-PE-2#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 Pop Label 192.168.1.3/32 0 Gi2/0/2 10.0.0.13 17 16 192.168.1.2/32 630 Gi2/0/2 10.0.0.13 <-- LDP label 17 is advertised to Remote PE 192.168.1.2 18 Pop Label 10.0.0.4/30 0 Gi2/0/2 10.0.0.13 20 No Label 10.0.0.20/30[V] 1260 aggregate/RED 21 No Label 192.168.2.0/24[V] \ 2070 Gi2/0/1 10.0.0.22 C9300-PE-2#show platform software mpls switch active r0 label index 82 <-- Utilize the obj id value from the FED Command Label OCE 0x52 -> OBJ_ADJACENCY (0x46) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348063ad00 C9300-PE-2#show platform software mpls switch active f0 label index 82 <-- Utilize the obj id value from the FED Command Label OCE 0x52 -> OBJ_ADJACENCY (0x46) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 aom id: 3624, CPP handle: 0xdeadbeef (created) <-- Used in next commands C9300-PE-2#show platform software object-manager switch active f0 object 3624 <-- Utilize the aom id value Object identifier: 3624 Description: label 0x52 Status: Done, Epoch: 0, Client data: 0x11071668 C9300-PE-2#show platform software object-manager switch active f0 object 3624 parents <-- Utilize the aom id value Object identifier: 496 Description: adj 0x46, Flags None Status: Done C9300-PE-2#show platform software fed switch active mpls forwarding label 17 detail LENTRY:label:17 nobj:(LABEL, 82) lentry_hdl:0x44000005 modify_cnt:6 backwalk_cnt:0 lspa_handle:0 AAL: id:1140850693 lbl:17 eos0:[adj_hdl:0x5f000032, hw_hdl:0x7fe8f8a52798] eos1:[adj_hdl:0x5f000032, hw_hdl:0x7fe8f8a52588] deagg_vrf_id = 0 lspa_handle:0 LABEL:objid:82 link_type:MPLS local_label:17 outlabel:(16, 0) <-- Used in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x5f000032 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1593835570 lbl:0 smac:70d3.79be.ae71 dmac:d4ad.71b5.ddd6 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fe8f8a93c78, ri_id:0x3a phdl:0x9f00004b, ref_cnt:1 si:0x7fe8f8a91188, si_id:0x4011, di_id:0x535f ADJ:objid:70 {link_type:MPLS ifnum:0x36, si:0xaa000021, }
Scenario 2. L3VPN with ECMP Between PEs and P Routers
Reference Topology
For the purpose of this example, Catalyst 3850 switches function as CE devices, Catalyst 9300 switches function as PE devices, Catalyst 9500 in Stackwise Virtual function as the P device. EIGRP runs between the CE and PE devices, OSPF and LDP adjacencies in the MPLS core, with an iBGP VPNv4 adjacency between the PE devices. Within the MPLS core, there is ECMP between the PE and P devices.
Configuration Details
Configuration of C3850-CE-1
hostname C3850-CE-1
!
interface Loopback0
ip address 192.168.3.1 255.255.255.0
!
interface TenGigabitEthernet1/0/1
no switchport
ip address 10.0.0.1 255.255.255.252
!
router eigrp 420
network 10.0.0.0 0.0.0.3
network 192.168.3.0
eigrp stub connected summary
!
ip route 0.0.0.0 0.0.0.0 10.0.0.2
Configuration of C9300-PE-1
hostname C9300-PE-1
!
ip vrf RED
rd 69:69
route-target export 69:69
route-target import 69:69
!
mpls ldp explicit-null
!
interface Loopback0
ip address 192.168.1.2 255.255.255.255
!
interface GigabitEthernet1/0/1
no switchport
ip vrf forwarding RED
ip address 10.0.0.2 255.255.255.252
!
interface GigabitEthernet1/0/2
no switchport
ip address 10.0.0.5 255.255.255.252
!
interface GigabitEthernet1/0/3
no switchport
ip address 10.0.0.9 255.255.255.252
!
router eigrp 420
!
address-family ipv4 vrf RED
network 10.0.0.0 0.0.0.3
autonomous-system 420
exit-address-family
!
router ospf 420
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
!
router bgp 69420
bgp log-neighbor-changes
neighbor 192.168.1.4 remote-as 69420
neighbor 192.168.1.4 update-source Loopback0
!
address-family vpnv4
neighbor 192.168.1.4 activate
neighbor 192.168.1.4 send-community extended
exit-address-family
!
address-family ipv4 vrf RED
redistribute eigrp 420
exit-address-family
Configuration of C9500-P
hostname C9500-P
!
interface Loopback0
ip address 192.168.1.3 255.255.255.255
!
interface TenGigabitEthernet1/0/1
no switchport
ip address 10.0.0.6 255.255.255.252
!
interface TenGigabitEthernet1/0/2
no switchport
ip address 10.0.0.13 255.255.255.252
!
interface TenGigabitEthernet2/0/1
no switchport
ip address 10.0.0.10 255.255.255.252
!
interface TenGigabitEthernet2/0/2
no switchport
ip address 10.0.0.17 255.255.255.252
!
router ospf 420
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
Configuration of C9300-PE-2
hostname C9300-PE-2
!
ip vrf RED
rd 69:69
route-target export 69:69
route-target import 69:69
!
mpls ldp explicit-null
!
interface Loopback0
ip address 192.168.1.4 255.255.255.255
!
interface GigabitEthernet2/0/1
no switchport
ip vrf forwarding RED
ip address 10.0.0.21 255.255.255.252
!
interface GigabitEthernet2/0/2
no switchport
ip address 10.0.0.14 255.255.255.252
!
interface GigabitEthernet2/0/3
no switchport
ip address 10.0.0.18 255.255.255.252
!
router eigrp 400
!
address-family ipv4 vrf RED
network 10.0.0.20 0.0.0.3
autonomous-system 400
exit-address-family
!
router ospf 420
passive-interface GigabitEthernet2/0/24
network 0.0.0.0 255.255.255.255 area 0
mpls ldp autoconfig
!
router bgp 69420
bgp log-neighbor-changes
neighbor 192.168.1.2 remote-as 69420
neighbor 192.168.1.2 update-source Loopback0
!
address-family vpnv4
neighbor 192.168.1.2 activate
neighbor 192.168.1.2 send-community extended
exit-address-family
!
address-family ipv4 vrf RED
redistribute eigrp 400
exit-address-family
Configuration of C3850-CE-2
hostname C3850-CE-2
!
interface Loopback0
ip address 192.168.2.1 255.255.255.0
!
interface TenGigabitEthernet2/0/1
no switchport
ip address 10.0.0.22 255.255.255.252
!
router eigrp 400
network 10.0.0.20 0.0.0.3
network 192.168.2.0
eigrp stub connected summary
!
ip route 0.0.0.0 0.0.0.0 10.0.0.21
Basic Validation
Before validation of MPLS programming there are base requirements that need to be validated:
- Validate PE to PE connectivity is present
- Validate the label switched path (LSP) between the PEs
- Validate BGPv4 adjacency between PEs
- Validate VPNv4 and LDP labels
- Validate MPLS Forwarding Table
Validate PE to PE Connectivity
You can ping the remote PE loopback and source from the local loopback, but this does not confirm the MPLS label switched path (LSP) is good, since the Loopback IP addresses are advertised in the underlay.
C9300-PE-1#ping 192.168.1.4 source 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.4, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
C9300-PE-1#show ip route 192.168.1.4
Routing entry for 192.168.1.4/32
Known via "ospf 420", distance 110, metric 3, type intra area
Last update from 10.0.0.10 on GigabitEthernet1/0/3, 18:39:30 ago
Routing Descriptor Blocks:
10.0.0.10, from 192.168.1.4, 18:39:30 ago, via GigabitEthernet1/0/3
Route metric is 3, traffic share count is 1
* 10.0.0.6, from 192.168.1.4, 18:39:30 ago, via GigabitEthernet1/0/2
Route metric is 3, traffic share count is 1
Validate the LSP
You can use a MPLS traceroute from PE to PE loopback to validate the LSP and all MPLS LDP labels along the path.
C9300-PE-1#traceroute mpls ipv4 192.168.1.4/32 source 192.168.1.2 Tracing MPLS Label Switched Path to 192.168.1.4/32, timeout is 2 seconds Codes: '!' - success, 'Q' - request not sent, '.' - timeout, 'L' - labeled output interface, 'B' - unlabeled output interface, 'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch, 'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 'P' - no rx intf label prot, 'p' - premature termination of LSP, 'R' - transit router, 'I' - unknown upstream index, 'l' - Label switched with FEC change, 'd' - see DDMAP for return code, 'X' - unknown return code, 'x' - return code 0 Type escape sequence to abort. 0 10.0.0.5 MRU 1500 [Labels: 17 Exp: 0] L 1 10.0.0.6 MRU 1500 [Labels: explicit-null Exp: 0] 7 ms ! 2 10.0.0.18 1 ms
If you do not have access to the CE or a device behind the CE and you want to demonstrate that there is successful VPNv4 and LDP label imposition/disposition you can attempt to ping from the CE-facing interface in the VRF on a PE to the other CE-facing interface in the VRF on the remote PE.
C9300-PE-1#ping vrf RED 10.0.0.21 source 10.0.0.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.0.0.21, timeout is 2 seconds: Packet sent with a source address of 10.0.0.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
Validate BGP VPNv4 adjacency between PEs
C9300-PE-1#show bgp vpnv4 unicast all neighbors 192.168.1.4
BGP neighbor is 192.168.1.4, remote AS 69420, internal link
BGP version 4, remote router ID 192.168.1.4
BGP state = Established, up for 18:40:49
Last read 00:00:40, last write 00:00:47, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Address family VPNv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 4 4
Keepalives: 1237 1233
Route Refresh: 0 0
Total: 1242 1238
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 0 seconds
<snip>
C9300-PE-2#show bgp vpnv4 unicast all neighbors 192.168.1.2
BGP neighbor is 192.168.1.2, remote AS 69420, internal link
BGP version 4, remote router ID 192.168.1.2
BGP state = Established, up for 18:41:36
Last read 00:00:42, last write 00:00:32, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Address family VPNv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 4 4
Keepalives: 1234 1238
Route Refresh: 0 0
Total: 1239 1243
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 0 seconds
Remote PE VPNv4 adjacency is up, and a prefix has been received
C9300-PE-1#show bgp vpnv4 unicast all summary BGP router identifier 192.168.1.2, local AS number 69420 BGP table version is 7, main routing table version 7 4 network entries using 1024 bytes of memory 4 path entries using 544 bytes of memory 4/4 BGP path/bestpath attribute entries using 1216 bytes of memory 4 BGP extended community entries using 1000 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 3784 total bytes of memory BGP activity 4/0 prefixes, 4/0 paths, scan interval 60 secs 4 networks peaked at 18:49:56 Jun 23 2021 UTC (18:41:06.070 ago) Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.4 4 69420 1240 1244 7 0 0 18:41:59 2 C9300-PE-2#show bgp vpnv4 unicast all summary BGP router identifier 192.168.1.4, local AS number 69420 BGP table version is 7, main routing table version 7 4 network entries using 1024 bytes of memory 4 path entries using 544 bytes of memory 4/4 BGP path/bestpath attribute entries using 1216 bytes of memory 4 BGP extended community entries using 1000 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 3784 total bytes of memory BGP activity 4/0 prefixes, 4/0 paths, scan interval 60 secs 4 networks peaked at 18:49:37 Jun 23 2021 UTC (18:41:06.851 ago) Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.2 4 69420 1244 1240 7 0 0 18:42:17 2
Verify what prefixes are exchanged in the particular VRF
C9300-PE-1#show ip bgp vpnv4 vrf RED
BGP table version is 7, local router ID is 192.168.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 69:69 (default for vrf RED)
*> 10.0.0.0/30 0.0.0.0 0 32768 ?
*>i 10.0.0.20/30 192.168.1.4 0 100 0 ?
*>i 192.168.2.0 192.168.1.4 130816 100 0 ?
*> 192.168.3.0 10.0.0.1 130816 32768 ?
C9300-PE-2#show ip bgp vpnv4 vrf RED
BGP table version is 7, local router ID is 192.168.1.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 69:69 (default for vrf RED)
*>i 10.0.0.0/30 192.168.1.2 0 100 0 ?
*> 10.0.0.20/30 0.0.0.0 0 32768 ?
*> 192.168.2.0 10.0.0.22 130816 32768 ?
*>i 192.168.3.0 192.168.1.2 130816 100 0 ?
Validate VPNv4 and LDP Labels
C9300-PE-1#show ip bgp vpnv4 vrf RED labels
Network Next Hop In label/Out label
Route Distinguisher: 69:69 (RED)
10.0.0.0/30 0.0.0.0 20/nolabel(RED)
10.0.0.20/30 192.168.1.4 nolabel/20
192.168.2.0 192.168.1.4 nolabel/21 <-- VPNv4 label that is be imposed to reach 192.168.20
192.168.3.0 10.0.0.1 21/nolabel
C9300-PE-1#show ip route vrf RED 192.168.2.1
Routing Table: RED
Routing entry for 192.168.2.0/24
Known via "bgp 69420", distance 200, metric 130816, type internal
Last update from 192.168.1.4 18:41:56 ago
Routing Descriptor Blocks:
* 192.168.1.4 (default), from 192.168.1.4, 18:41:56 ago
Route metric is 130816, traffic share count is 1
AS Hops 0
MPLS label: 21 <-- VPNv4 label that matches the previous output
MPLS Flags: MPLS Required
C9300-PE-2#show ip bgp vpnv4 vrf RED labels
Network Next Hop In label/Out label
Route Distinguisher: 69:69 (RED)
10.0.0.0/30 192.168.1.2 nolabel/20
10.0.0.20/30 0.0.0.0 20/nolabel(RED)
192.168.2.0 10.0.0.22 21/nolabel <-- VPNv4 label that is advertised to reach 192.168.2.0
192.168.3.0 192.168.1.2 nolabel/21
C9300-PE-2#show ip route vrf RED 192.168.2.1
Routing Table: RED
Routing entry for 192.168.2.0/24
Known via "eigrp 400", distance 90, metric 130816, precedence routine (0), type internal
Redistributing via eigrp 400, bgp 69420
Advertised by bgp 69420
Last update from 10.0.0.22 on GigabitEthernet2/0/1, 18:45:04 ago
Routing Descriptor Blocks:
* 10.0.0.22, from 10.0.0.22, 18:45:04 ago, via GigabitEthernet2/0/1 <-- CE-facing interface in the VRF
Route metric is 130816, traffic share count is 1
Total delay is 5010 microseconds, minimum bandwidth is 1000000 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
Verify the LDP labels that are utilized
C9300-PE-1#show mpls forwarding-table 192.168.1.4
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6 <-- 17 is the LDP label imposed to reach PE at 192.168.1.4 through Gi1/0/2
17 192.168.1.4/32 0 Gi1/0/3 10.0.0.10 <-- 17 is the LDP label imposed to reach PE at 192.168.1.4 through Gi1/0/3
C9300-PE-2#show mpls forwarding-table 192.168.1.2
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
17 16 192.168.1.2/32 0 Gi2/0/2 10.0.0.13 <-- 16 is the LDP label imposed to reach PE at 192.168.1.2 through Gi2/0/2
16 192.168.1.2/32 0 Gi2/0/3 10.0.0.17 <-- 16 is the LDP label imposed to reach PE at 192.168.1.2 through Gi2/0/3
Validate the MPLS Forwarding Table
C9300-PE-1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 192.168.1.3/32 0 Gi1/0/2 10.0.0.6
Pop Label 192.168.1.3/32 0 Gi1/0/3 10.0.0.10
17 Pop Label 10.0.0.16/30 0 Gi1/0/2 10.0.0.6
Pop Label 10.0.0.16/30 0 Gi1/0/3 10.0.0.10
18 Pop Label 10.0.0.12/30 0 Gi1/0/2 10.0.0.6
Pop Label 10.0.0.12/30 0 Gi1/0/3 10.0.0.10
19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6
17 192.168.1.4/32 0 Gi1/0/3 10.0.0.10
20 No Label 10.0.0.0/30[V] 630 aggregate/RED
21 No Label 192.168.3.0/24[V] \
0 Gi1/0/1 10.0.0.1
C9300-PE-2#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 192.168.1.3/32 0 Gi2/0/2 10.0.0.13
Pop Label 192.168.1.3/32 0 Gi2/0/3 10.0.0.17
17 16 192.168.1.2/32 0 Gi2/0/2 10.0.0.13
16 192.168.1.2/32 0 Gi2/0/3 10.0.0.17
18 Pop Label 10.0.0.4/30 0 Gi2/0/2 10.0.0.13
Pop Label 10.0.0.4/30 0 Gi2/0/3 10.0.0.17
19 Pop Label 10.0.0.8/30 0 Gi2/0/2 10.0.0.13
Pop Label 10.0.0.8/30 0 Gi2/0/3 10.0.0.17
20 No Label 10.0.0.20/30[V] 630 aggregate/RED
21 No Label 192.168.2.0/24[V] \
0 Gi2/0/1 10.0.0.22
Confirm the inner (VPNv4) and outer (LDP) labels used to reach to each given prefix in the VRF
C9300-PE-1#show ip cef vrf RED 192.168.2.0/24 detail
192.168.2.0/24, epoch 0, flags [rib defined all labels]
recursive via 192.168.1.4 label 21 <-- VPNv4 label
nexthop 10.0.0.6 GigabitEthernet1/0/2 label 17-(local:19) <-- 17 is the LDP label that is imposed to reach the remote PE, 19 is the local LDP label advertised to the P router
nexthop 10.0.0.10 GigabitEthernet1/0/3 label 17-(local:19)<-- 17 is the LDP label that is imposed to reach the remote PE, 19 is the local LDP label advertised to the P router
C9300-PE-2#show ip cef vrf RED 192.168.3.0/24 detail
192.168.3.0/24, epoch 0, flags [rib defined all labels]
recursive via 192.168.1.2 label 21 <-- VPNv4 label
nexthop 10.0.0.13 GigabitEthernet2/0/2 label 16-(local:17) <-- 16 is the LDP label that is imposed to reach the remote PE, 17 is the local LDP label advertised to the P router
nexthop 10.0.0.17 GigabitEthernet2/0/3 label 16-(local:17) <-- 16 is the LDP label that is imposed to reach the remote PE, 17 is the local LDP label advertised to the P router
Verify Object-Manager Statistics:
In ideal scenarios, there are no pending objects
C9300-PE-1#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 491 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
9500-P#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 491 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
C9300-PE-2#show platform software object-manager switch active f0 statistics Forwarding Manager Asynchronous Object Manager Statistics Object update: Pending-issue: 0, Pending-acknowledgement: 0 Batch begin: Pending-issue: 0, Pending-acknowledgement: 0 Batch end: Pending-issue: 0, Pending-acknowledgement: 0 Command: Pending-acknowledgement: 0 Total-objects: 482 Stale-objects: 0 Resolve-objects: 0 Childless-delete-objects: 0 Error-objects: 0 Paused-types: 0
Prefix Programming
The next section covers prefix programming on the MPLS routers, C9300-PE-1, C9500-P, and C9300-PE-2.
C9300-PE-1 Prefix Programming
***Software Prefix Programming***
C9300-PE-1#show ip route vrf RED 192.168.2.1 Routing Table: RED Routing entry for 192.168.2.0/24 Known via "bgp 69420", distance 200, metric 130816, type internal Last update from 192.168.1.4 19:21:45 ago Routing Descriptor Blocks: * 192.168.1.4 (default), from 192.168.1.4, 19:21:45 ago <-- Remote PE reachable in the global routing table Route metric is 130816, traffic share count is 1 AS Hops 0 MPLS label: 21 <-- VPNv4 label MPLS Flags: MPLS Required C9300-PE-1#show ip route 192.168.1.4 Routing entry for 192.168.1.4/32 Known via "ospf 420", distance 110, metric 3, type intra area Last update from 10.0.0.10 on GigabitEthernet1/0/3, 19:23:17 ago Routing Descriptor Blocks: 10.0.0.10, from 192.168.1.4, 19:23:17 ago, via GigabitEthernet1/0/3 <-- Next-hop to reach 192.168.1.4 Route metric is 3, traffic share count is 1 * 10.0.0.6, from 192.168.1.4, 19:23:17 ago, via GigabitEthernet1/0/2 <-- Next-hop to reach 192.168.1.4 Route metric is 3, traffic share count is 1
***FMAN RP Prefix Programming*** C9300-PE-1#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID <-- VRF ID is important in subsequent command Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi1/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix C9300-PE-1#show platform software ip switch active r0 cef table index 2 prefix 192.168.2.0/24 <-- Index value is the VRF ID from previous command Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.2.0/24 OBJ_LABEL 0x78 C9300-PE-1#show platform software mpls switch active r0 label index 0x78 <-- Utilize the Index value from previous command Label OCE 0x78 -> OBJ_LOADBALANCE (0x70) <-- Utilized in next command Flags: Real, Number of labels in the OCE: 1 Label values: 0x15 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480644d88 C9300-PE-1#show platform software loadinfo switch active r0 index 0x70 <-- Utilize the OBJ_LOADBALANCE value from previous command Number of loadinfo objects: 8 Index: 0x70, Flags: unknown, Hash Algorithm: , Number of Paths: 2, Number of buckets: 16 Anti-polarising Factor: 0xf4a19ba0 Next Object Type: OBJ_LABEL, OBJ_LABEL Next obj handle: 0x6e, 0x6f Hash Buckets: 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1 Color Buckets Map: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 OM handle: 0x3480641fb8 C9300-PE-1#show platform software mpls switch active r0 label index 0x6e <-- Utilize the obj handle value from previous command Label OCE 0x6e -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x34806420d0 C9300-PE-1#show platform software mpls switch active r0 label index 0x6f <-- Utilize the obj handle value from previous command Label OCE 0x6f -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480642268 C9300-PE-1#show platform software adjacency switch active r0 index 0x4b <-- Utilize the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x4b (75) Interface: GigabitEthernet1/0/2, IF index: 54, Link Type: MCP_LINK_TAG Encap: d4:ad:71:b5:dd:e4:a0:f8:49:11:d1:d6:88:47 <-- MAC ending in DDE4 is the DMAC, MAC ending in D1D6 is SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.6 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x34806375f8 C9300-PE-1#show platform software adjacency switch active r0 index 0x4e <-- Utilize the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x4e (78) Interface: GigabitEthernet1/0/3, IF index: 55, Link Type: MCP_LINK_TAG Encap: d4:ad:71:b5:dd:c2:a0:f8:49:11:d1:d8:88:47 <-- MAC ending DDC2 is the DMAC, MAC ending in D1D8 is the SMAC, 8847 is the MPLS ETPYE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.10 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x3480638200
***FMAN FP Prefix Programming*** C9300-PE-1#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi1/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix C9300-PE-1#show platform software ip switch active f0 cef table index 2 prefix 192.168.2.0/24 detail <-- Index value is the VRF ID from previous command Forwarding Table 192.168.2.0/24 -> OBJ_LABEL (0x78), urpf: 118 Prefix Flags: unknown aom id: 618, HW handle: (nil) (created) C9300-PE-1#show platform software mpls switch active f0 label index 0x78 <-- Use the OBJ_LABEL value from previous command Label OCE 0x78 -> OBJ_LOADBALANCE (0x70) Flags: Real, Number of labels in the OCE: 1 Label values: 0x15 Backup flags: Pop, UHP, backup label 0x100001 aom id: 617, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software object-manager switch active f0 object 617 parents <-- Use the aom id from previous command Object identifier: 600 Description: LB 0x70 Status: Done C9300-PE-1#show platform software loadinfo switch active f0 index 0x70 <-- Use the LB value from previous command Number of loadinfo objects: 8 Index: 0x70, Flags: unknown, Hash Algorithm: , Number of Paths: 2, Number of buckets: 16 Anti-polarising Factor: 0xf4a19ba0 Next Object Type: OBJ_LABEL, OBJ_LABEL Next obj handle: 0x6e, 0x6f Hash Buckets: 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1 Color Buckets Map: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 aom id: 600, HW handle: (nil) C9300-PE-1#show platform software mpls switch active f0 label index 0x6e <-- Use the obj handle values from previous commands Label OCE 0x6e -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 598, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software mpls switch active f0 label index 0x6f <-- Use the obj handle values from previous command Label OCE 0x6f -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 599, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software adjacency switch active f0 index 0x4b <-- Use the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x4b (75) Interface: GigabitEthernet1/0/2, IF index: 54, Link Type: MCP_LINK_TAG Encap: d4:ad:71:b5:dd:e4:a0:f8:49:11:d1:d6:88:47 Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.6 IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 531, HW handle: (nil) (created) C9300-PE-1#show platform software adjacency switch active f0 index 0x4e <-- Use the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x4e (78) Interface: GigabitEthernet1/0/3, IF index: 55, Link Type: MCP_LINK_TAG Encap: d4:ad:71:b5:dd:c2:a0:f8:49:11:d1:d8:88:47 Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.10 IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 535, HW handle: (nil) (created)
***FED Prefix Programming*** C9300-PE-1#show platform software fed switch active ip route vrf-name RED 192.168.2.0/24 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 2 192.168.2.0/24 0x7fbae8d86228 0x0 0 0 lspa0x2 2021/06/23 18:50:13.079 <-- HTM value significant for next command FIB: prefix_hdl:0x50000026, mpls_ecr_prefix_hdl:0 ========== OCE chain ===== LABEL:objid:120 link_type:IP local_label:1048577 outlabel:(21, 0) <-- VPNv4 label flags:0x1:(REAL,) pdflags:0x80:(INSTALL_HW_OK,RECIR_ADJ,) adj_handle:0xcb00003c <-- adj_handle and local_adj_hdl values must match unsupported recursion:0 olbl_changed 0 local_adj:1 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:3405774908 lbl:19 smac:0000.0000.0000 dmac:0000.0000.0000 <-- Label 19 matches the local transport label sub_type:0 link_type:0 adj_flags:0x10 label_type:0 rewrite_type:PSH2(121) vlan_id:0 vrf_id:0 ri:0x7fbae8d73648, ri_id:0x46 phdl:0, ref_cnt:2 <-- ri_id and ri_idx values must match si:0x7fbae8d834d8, si_id:0xb6, di_id:0x5013 LB:obj_id:112 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:19 path_inhw:2 ecrh:0x7d000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:2097152002 af:0 ecr_type:0 ref:7 ecrh:0x7fbae8a99268(28:2) hwhdl:3903427176 ::0x7fbae8a98b98,0x7fbae8a9ad48,0x7fbae8a98b98,0x7fbae8a9ad48 Sw Enh ECR scale: objid:112 llabel:19 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0x7d000002 ecr_hwhdl:0x7fbae8a99268 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:4278190135 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1744830509 sih:0x7fbae8a98b98(179) di_id:20499 rih:0x7fbae8a985d8(33) adj_lentry [eos0:0x7fbae8d7bf48 eos1:0x7fbae8d76e88] ecr_adj: id:1392508984 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:2013265966 sih:0x7fbae8a9ad48(180) di_id:20499 rih:0x7fbae8a9a788(46) adj_lentry [eos0:0x7fbae8d7c1b8 eos1:0x7fbae8d77158] ecr_prefix_adj: id:2164260921 (ref:1) sih:0x7fbae8d7df08(181) di_id:20499 rih:0x7fbae8d7db98(68) LABEL:objid:110 link_type:MPLS local_label:19 outlabel:(17, 0) <-- Label 19 is the local transport label, Label 17 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xff000037 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:4278190135 lbl:0 smac:a0f8.4911.d1d6 dmac:d4ad.71b5.dde4 <-- Matches next-hop information to reach 192.168.2.0/24 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fbae8d78c48, ri_id:0x40 phdl:0x9f00004b, ref_cnt:1 si:0x7fbae8d78fd8, si_id:0x4013, di_id:0x535f <-- di_id utilized in subsequent commands ADJ:objid:75 {link_type:MPLS ifnum:0x36, si:0x22000023, } LABEL:objid:111 link_type:MPLS local_label:19 outlabel:(17, 0) <-- Label 19 is the local transport label, Label 17 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x53000038 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1392508984 lbl:0 smac:a0f8.4911.d1d8 dmac:d4ad.71b5.ddc2 <-- Matches next-hop information to reach 192.168.2.0/24 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fbae8d7d0a8, ri_id:0x42 phdl:0x8400004c, ref_cnt:1 si:0x7fbae8d7a908, si_id:0x4014, di_id:0x5360 <-- di_id utilized in subsequent commands ADJ:objid:78 {link_type:MPLS ifnum:0x37, si:0x74000026, } =============== MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0xa7000002 AAL:LSPA: id:2801795074 num_path:1 prefix_id:0x2 delete_hw_hdl_cnt:0 cookie[64]: 15000000000000000000000000000000000000000000000000000000000000004600000000000000000000000000000000000000000000000000000000000000 status:ok vpn_lbl:21 local_adj_hdl:0xcb00003c hw_hdl:0x7fbae8d86018 ri_idx:0x46 <-- vpn_lbl matches the VPNv4 label, adj_handle and local_adj_hdl values must match, ri_id and ri_idx must match =============== C9300-PE-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7fbae8d86228 1 <-- Utilize the HTM value from previous command Handle:0x7fbae8d86228 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7fbae8d71f58 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7fbae8d71f58) Absolute Index: 92181 Time Stamp: 1 KEY - vrf:2 mtr:0 prefix:192.168.2.0 rcp_redirect_index:0x0 MASK - vrf:255 mtr:0 prefix:255.255.255.0 rcp_redirect_index:0x0 FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:182 destined_to_us:0 hw_stats_idx:0 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0x2 SRC-AD = learning_violation:1 need_to_learn:1 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:0 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:0 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9300-PE-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x535f 0x535f <-- Utilize the di_id from the previous command ASIC#0: index = 0x535f pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x535f pmap = 0x00000000 0x00000002 <-- Looking at 0x00000002, in binary that is 0000 0000 0000 0000 000 0000 0000 0010 = Port 1 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9300-PE-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x5360 0x5360 <-- Utilize the di_id from the previous command ASIC#0: ASIC#0: index = 0x5360 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x5360 pmap = 0x00000000 0x00000004 <-- Looking at 0x00000004, in binary that is 0000 0000 0000 0000 0000 0000 0000 0100 = Port 2 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9300-PE-1#show platform software fed switch active ifm map Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/2 0x36 1 0 1 1 0 6 7 2 2 NIF Y <-- Port 1 is an egress port, Gi1/0/2 GigabitEthernet1/0/3 0x37 1 0 1 2 0 28 8 3 3 NIF Y <-- Port 2 is an egress port, Gi1/0/3
C9500 Prefix Programming
***Software Prefix Programming***
C9500-P#show ip route 192.168.1.4 Routing entry for 192.168.1.4/32 Known via "ospf 420", distance 110, metric 2, type intra area Last update from 10.0.0.18 on TenGigabitEthernet2/0/2, 20:15:25 ago Routing Descriptor Blocks: 10.0.0.18, from 192.168.1.4, 20:15:25 ago, via TenGigabitEthernet2/0/2 <-- Next-hop towards 192.168.1.4 Route metric is 2, traffic share count is 1 * 10.0.0.14, from 192.168.1.4, 20:15:25 ago, via TenGigabitEthernet1/0/2 <-- Next-hop towards 192.168.1.4 Route metric is 2, traffic share count is 1 C9500-P#show ip cef 192.168.1.4 detail 192.168.1.4/32, epoch 4, per-destination sharing dflt local label info: global/17 [0x3] nexthop 10.0.0.14 TenGigabitEthernet1/0/2 label explicit-null-(local:17) <-- Explicit null to reach 192.168.1.4 nexthop 10.0.0.18 TenGigabitEthernet2/0/2 label explicit-null-(local:17) <-- Explicit null to reach 192.168.1.4
***FMAN RP Prefix Programming*** C9500-P#show platform software ip switch active r0 cef prefix 192.168.1.4/32 Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.1.4/32 OBJ_LOADBALANCE 0x6a C9500-P#show platform software loadinfo switch active r0 index 0x6a <-- Use the OBJ_LOADBALANCE value from previous command Number of loadinfo objects: 4 Index: 0x6a, Flags: unknown, Hash Algorithm: , Number of Paths: 2, Number of buckets: 16 Anti-polarising Factor: 0x57a70068 Next Object Type: OBJ_LABEL, OBJ_LABEL Next obj handle: 0x68, 0x69 Hash Buckets: 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1 Color Buckets Map: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 OM handle: 0x348064de58 C9500-P#show platform software mpls switch active r0 label index 0x68 <-- Use the obj handle values from the previous command Label OCE 0x68 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064df70 C9500-P#show platform software mpls switch active r0 label index 0x69 Label OCE 0x69 -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064e108 C9500-P#show platform software adjacency switch active r0 index 0x49 <-- Use the OBJ_ADJACENCY values from previous commands Number of adjacency objects: 16 Adjacency id: 0x49 (73) Interface: TenGigabitEthernet1/0/2, IF index: 66, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:71:d4:ad:71:b5:dd:d6:88:47 <-- MAC ending in AE71 is the DMAC, MAC ending is B5DD is SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.14 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x3480647700 C9500-P#show platform software adjacency switch active r0 index 0x4e <-- Use the OBJ_ADJACENCY values from previous commands Number of adjacency objects: 16 Adjacency id: 0x4e (78) Interface: TenGigabitEthernet2/0/2, IF index: 68, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:61:d4:ad:71:b5:dd:f1:88:47 <-- MAC ending in AE61 is DMAC, MAC ending in B5DD is SMAC, 8847 is MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.18 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x3480648f68
***FMAN FP Prefix Programming*** C9500-P#show platform software ip switch active f0 cef prefix 192.168.1.4/32 Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.1.4/32 OBJ_LOADBALANCE 0x6a C9500-P#show platform software loadinfo switch active f0 index 0x6a <-- Use the OBJ_LOADBALANCE value from previous command Number of loadinfo objects: 4 Index: 0x6a, Flags: unknown, Hash Algorithm: , Number of Paths: 2, Number of buckets: 16 Anti-polarising Factor: 0x57a70068 Next Object Type: OBJ_LABEL, OBJ_LABEL Next obj handle: 0x68, 0x69 Hash Buckets: 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1 Color Buckets Map: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 aom id: 578, HW handle: (nil) C9500-P#show platform software mpls switch active f0 label index 0x68 <-- Use the obj handle values from previous command Label OCE 0x68 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 576, CPP handle: 0xdeadbeef (created) C9500-P#show platform software mpls switch active f0 label index 0x69 <-- Use the obj handle values from previous command Label OCE 0x69 -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 577, CPP handle: 0xdeadbeef (created) C9500-P#show platform software adjacency switch active f0 index 0x49 <-- Use the OBJ_ADJACENCY values from previous commands Number of adjacency objects: 16 Adjacency id: 0x49 (73) Interface: TenGigabitEthernet1/0/2, IF index: 66, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:71:d4:ad:71:b5:dd:d6:88:47 <-- MAC ending in AE71 is the DMAC, MAC ending in DDD6 is the SMAC, 8847 is the MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.14 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 536, HW handle: (nil) (created) C9500-P#show platform software adjacency switch active f0 index 0x4e <-- Use the OBJ_ADJACENCY values from previous commands Number of adjacency objects: 16 Adjacency id: 0x4e (78) Interface: TenGigabitEthernet2/0/2, IF index: 68, Link Type: MCP_LINK_TAG Encap: 70:d3:79:be:ae:61:d4:ad:71:b5:dd:f1:88:47 <-- MAC ending in AE61 is the DMAC, MAC ending in DDF1 is the SMAC, 8847 is the MPLS ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: unknown Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.18 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 545, HW handle: (nil) (created)
***FED Prefix Programming*** C9500-P#show platform software fed switch active ip route 192.168.1.4/32 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 0 192.168.1.4/32 0x7f0b284c1118 0x0 0 0 2021/06/23 18:47:01.761 <-- HTM value important for subsequent command FIB: prefix_hdl:0x9b000020, mpls_ecr_prefix_hdl:0xdd00003a ========== OCE chain ===== LB:obj_id:106 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:17 path_inhw:2 ecrh:0x44000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:1140850690 af:0 ecr_type:0 ref:2 ecrh:0x7f0b284a3998(28:2) hwhdl:675953048 ::0x7f0b284b4268,0x7f0b284a1d78,0x7f0b284b4268,0x7f0b284a1d78 Sw Enh ECR scale: objid:106 llabel:17 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0x44000002 ecr_hwhdl:0x7f0b284a3998 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:4127195192 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1207959601 sih:0x7f0b284b4268(181) di_id:23709 rih:0x7f0b284b3ca8(31) adj_lentry [eos0:0x7f0b284c38e8 eos1:0x7f0b284cd858] ecr_adj: id:1157627961 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:67108914 sih:0x7f0b284a1d78(182) di_id:23709 rih:0x7f0b284b47d8(44) adj_lentry [eos0:0x7f0b284c3af8 eos1:0x7f0b284cdb28] ecr_prefix_adj: id:3707764794 (ref:1) sih:0x7f0b284c5028(184) di_id:23709 rih:0x7f0b284c4c48(60) LABEL:objid:104 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Label 17 is the local transport label, 0 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xf6000038 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:4127195192 lbl:0 smac:d4ad.71b5.ddd6 dmac:70d3.79be.ae71 <-- Matches the next-hop information to reach 192.168.1.4/32 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284ceaa8, ri_id:0x38 phdl:0x76000058, ref_cnt:1 si:0x7f0b284ceeb8, si_id:0x400b, di_id:0x2 <-- Used in subsequent commands ADJ:objid:73 {link_type:MPLS ifnum:0x42, si:0x1f000028, } LABEL:objid:105 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Label 17 is the local transport label, 0 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x45000039 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1157627961 lbl:0 smac:d4ad.71b5.ddf1 dmac:70d3.79be.ae61 <-- Matches the next-hop information to reach 192.168.1.4/32 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284c4588, ri_id:0x3a phdl:0x5500005a, ref_cnt:1 si:0x7f0b284d0548, si_id:0x400c, di_id:0x62 <-- Used in subsequent commands ADJ:objid:78 {link_type:MPLS ifnum:0x44, si:0x4900002a, } =============== MPLS info: mpls_ecr_scale_prefix_adj:0xdd00003a, mpls_lspa_hdl:0 =============== C9500-P#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f0b284c1118 1 <-- Use the HTM value from previous command Handle:0x7f0b284c1118 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f0b284c1328 Features sharing this resource:Cookie length: 12 04 01 a8 c0 00 00 00 d0 07 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7f0b284c1328) Absolute Index: 126650 Time Stamp: 1 KEY - vrf:0 mtr:0 prefix:192.168.1.4 rcp_redirect_index:0x0 MASK - vrf:0 mtr:0 prefix:0.0.0.0 rcp_redirect_index:0x0 FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:184 destined_to_us:0 hw_stats_idx:1 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0 SRC-AD = learning_violation:0 need_to_learn:0 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:2 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:1 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:1 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9500-P#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x2 0x2 <-- Use the di_id values from previous command ASIC#0: index = 0x2 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x2 pmap = 0x00000000 0x00000002 <-- Looking at 0x00000002, in binary that is 0000 0000 0000 0000 0000 0000 0000 0010 = Port 1 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9500-P#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x62 0x62 ASIC#0: index = 0x62 pmap = 0x00000000 0x00008000 <-- Looking at 0x00008000, in binary that is 0000 0000 0000 0000 1000 0000 0000 0000 = Port 15 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x62 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9500-P#show platform software fed switch standby ip route 192.168.1.4/32 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 0 192.168.1.4/32 0x7f57c0545938 0x0 0 0 2021/06/23 18:46:51.399 <-- HTM value used in subsequent command FIB: prefix_hdl:0x29000020, mpls_ecr_prefix_hdl:0x8f000039 ========== OCE chain ===== LB:obj_id:106 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:17 path_inhw:2 ecrh:0xf1000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:4043309058 af:0 ecr_type:0 ref:2 ecrh:0x7f57c04d2148(28:2) hwhdl:3226280264 ::0x7f57c0547538,0x7f57c05497d8,0x7f57c0547538,0x7f57c05497d8 Sw Enh ECR scale: objid:106 llabel:17 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0xf1000002 ecr_hwhdl:0x7f57c04d2148 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:201326647 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:3925868592 sih:0x7f57c0547538(181) di_id:23717 rih:0x7f57c0546f18(31) adj_lentry [eos0:0x7f57c04c8a08 eos1:0x7f57c04d07f8] ecr_adj: id:738197560 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:3070230577 sih:0x7f57c05497d8(182) di_id:23717 rih:0x7f57c0547838(44) adj_lentry [eos0:0x7f57c04c8c18 eos1:0x7f57c04d0ac8] ecr_prefix_adj: id:2399141945 (ref:1) sih:0x7f57c04c8788(184) di_id:23717 rih:0x7f57c04c8508(60) LABEL:objid:104 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Label 17 is the local transport label, 0 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xc000037 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:201326647 lbl:0 smac:d4ad.71b5.ddd6 dmac:70d3.79be.ae71 <-- Matches next-hop information to reach 192.168.1.4/32 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f57c04d18e8, ri_id:0x38 phdl:0x76000058, ref_cnt:1 si:0x7f57c04d1b18, si_id:0x400b, di_id:0x2 <-- di_id utilized in subsequent commands ADJ:objid:73 {link_type:MPLS ifnum:0x42, si:0xdf000027, } LABEL:objid:105 link_type:MPLS local_label:17 outlabel:(0, 0) <-- Label 17 is the local transport label, 0 is the LDP label flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x2c000038 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:738197560 lbl:0 smac:d4ad.71b5.ddf1 dmac:70d3.79be.ae61 <-- Matches next-hop information to reach 192.168.1.4/32 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f57c04da418, ri_id:0x3a phdl:0x5500005a, ref_cnt:1 si:0x7f57c04da838, si_id:0x400c, di_id:0x62 <-- di_id utilized in subsequent commands ADJ:objid:78 {link_type:MPLS ifnum:0x44, si:0xfa000029, } =============== MPLS info: mpls_ecr_scale_prefix_adj:0x8f000039, mpls_lspa_hdl:0 =============== C9500-P#show platform hardware fed switch standby fwd-asic resource asic all destination-index range 0x62 0x62 ASIC#0: index = 0x62 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x62 pmap = 0x00000000 0x00000002 <-- Looking at 0x000000002, in binary that is 0000 0000 0000 0000 0000 0000 0000 0010 = Port 1 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9500-P#show platform hardware fed switch standby fwd-asic resource asic all destination-index range 0x2 0x2 ASIC#0: index = 0x2 pmap = 0x00000000 0x00008000 <-- Looking at 0x00008000, in binary that is 0000 0000 0000 0000 1000 0000 0000 0000 = Port 15 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x2 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9500-P#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active TenGigabitEthernet1/0/2 0x42 1 0 1 1 0 10 1 2 2 NIF Y <-- Port 1 is an egres port, TenGi1/0/2 TenGigabitEthernet1/0/16 0x18 0 0 0 15 0 8 11 16 2360 NIF Y <-- Port 15 is the SVL C9500-P#show platform software fed switch standby ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active TenGigabitEthernet2/0/2 0x44 1 0 1 1 0 10 1 2 98 NIF Y <-- Port 1 is an egres port, TenGi2/0/2 TenGigabitEthernet2/0/16 0x33 0 0 0 15 0 8 11 16 2360 NIF Y <-- Port 15 is the SVL
Verify C9300-PE-2 Prefixes
***Software Prefix Programming***
C9300-PE-2#show ip route vrf RED 192.168.2.0 Routing Table: RED Routing entry for 192.168.2.0/24 Known via "eigrp 400", distance 90, metric 130816, precedence routine (0), type internal Redistributing via eigrp 400, bgp 69420 Advertised by bgp 69420 Last update from 10.0.0.22 on GigabitEthernet2/0/1, 21:35:22 ago Routing Descriptor Blocks: * 10.0.0.22, from 10.0.0.22, 21:35:22 ago, via GigabitEthernet2/0/1 <-- Next-hop to reach 192.168.2.0/24 Route metric is 130816, traffic share count is 1 Total delay is 5010 microseconds, minimum bandwidth is 1000000 Kbit Reliability 255/255, minimum MTU 1500 bytes Loading 1/255, Hops 1 C9300-PE-2#show ip route vrf RED 10.0.0.22 Routing Table: RED Routing entry for 10.0.0.20/30 Known via "connected", distance 0, metric 0 (connected, via interface) Redistributing via eigrp 400, bgp 69420 Advertised by bgp 69420 Routing Descriptor Blocks: * directly connected, via GigabitEthernet2/0/1 Route metric is 0, traffic share count is 1 C9300-PE-2#show ip cef vrf RED 192.168.2.0/24 detail 192.168.2.0/24, epoch 0 QOS: Precedence routine (0) dflt local label info: other/21 [0x2] <-- VPNv4 Label nexthop 10.0.0.22 GigabitEthernet2/0/1
***FMAN RP Prefix Programming*** C9300-PE-2#show ip vrf detail VRF RED (VRF Id = 2); default RD 69:69; default VPNID <-- VRF ID used in next command Old CLI format, supports IPv4 only Flags: 0xC Interfaces: Gi2/0/1 Address family ipv4 unicast (Table ID = 0x2): Flags: 0x0 Export VPN route-target communities RT:69:69 Import VPN route-target communities RT:69:69 No import route-map No global export route-map No export route-map VRF label distribution protocol: not configured VRF label allocation mode: per-prefix C9300-PE-2#show platform software ip switch active r0 cef table index 2 prefix 192.168.2.0/24 <-- Use the VRF ID from previous command Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.2.0/24 OBJ_ADJACENCY 0x3a C9300-PE-2#show platform software adjacency switch active r0 index 0x3a <-- Use the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x3a (58) Interface: GigabitEthernet2/0/1, IF index: 53, Link Type: MCP_LINK_IP Encap: 0:72:78:c8:c9:c2:70:d3:79:be:ae:42:8:0 <-- MAC ending in C9C2 is the DMAC, MAC ending in AE42 is SMAC, 0800 is IP ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: no-l3-inject Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.22 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 OM handle: 0x348062b578
***FMAN FP Prefix Programming*** C9300-PE-2#show platform software ip switch active f0 cef table index 2 prefix 192.168.2.0/24 Forwarding Table Prefix/Len Next Object Index ---------------------------------------------------------------- 192.168.2.0/24 OBJ_ADJACENCY 0x3a C9300-PE-2#show platform software adjacency switch active f0 index 0x3a <-- Use the OBJ_ADJACENCY value from previous command Number of adjacency objects: 10 Adjacency id: 0x3a (58) Interface: GigabitEthernet2/0/1, IF index: 53, Link Type: MCP_LINK_IP Encap: 0:72:78:c8:c9:c2:70:d3:79:be:ae:42:8:0 <-- MAC ending in C9C2 is the DMAC, MAC ending in AE42 is SMAC, 0800 is IP ETYPE Encap Length: 14, Encap Type: MCP_ET_ARPA, MTU: 1500 Flags: no-l3-inject Incomplete behavior type: None Fixup: unknown Fixup_Flags_2: unknown Nexthop addr: 10.0.0.22 <-- Next-hop IP address IP FRR MCP_ADJ_IPFRR_NONE 0 aom id: 477, HW handle: (nil) (created)
***FED Prefix Programming*** C9300-PE-2#show platform hardware fed switch active ip route vrf-name RED 192.168.2.0/24 vrf dest htm flags SGT DGID MPLS Last-modified --- ---- --- ----- --- ---- ---- ------------------------ 2 192.168.2.0/24 0x7f0650a7e3e8 0x0 0 0 2021/06/23 18:46:56.801 <-- HTM value used in subsequent command FIB: prefix_hdl:0x38000016, mpls_ecr_prefix_hdl:0 ========== OCE chain ===== ADJ:objid:58 {link_type:IP ifnum:0x35, si:0x9700001b, IPv4: 10.0.0.22 } <-- objid relevant in subsequent command, 10.0.0.22 is the next-hop IP =============== MPLS info: mpls_ecr_scale_prefix_adj:0, mpls_lspa_hdl:0 =============== C9300-PE-2#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f0650a7e3e8 1 <-- Use the HTM value from previous command Handle:0x7f0650a7e3e8 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_IPV4_L3_UNICAST ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f0650ba4028 Detailed Resource Information (ASIC# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7f0650ba4028) Absolute Index: 92180 Time Stamp: 1 KEY - vrf:2 mtr:0 prefix:192.168.2.0 rcp_redirect_index:0x0 MASK - vrf:255 mtr:0 prefix:255.255.255.0 rcp_redirect_index:0x0 (SI value used later) FWD-AD = afd_label_flag:0 icmp_redir_enable:1 lvx_smr_enabled:0, dstNatType:0 priority:5 afdLabelOrDestClientId:0 SI:173 destined_to_us:0 hw_stats_idx:1 stats_id:0 redirectSetRouterMac:0 dgtIdx:0 destModIndex:0 dstNatTypeOrVpnPrefixPtrMsb:0 vpnPrefixPtr:0 SRC-AD = learning_violation:1 need_to_learn:1 locally_connected:0 staticentryViolation:0 rpfValid:1 rpfLe:37 rpfLePointer:0 rpfForcePass:0 rpfForceFail:0 reachableviaSome:1 rpfCheckIncomplete:0 defaultRoute:0 ChainPtrValid:0 ChainPtrOrPortLeIndex:72 UseRpfmatchTable:0 rpfIncomplete:0 is_src_ce:0 sgtValid:0 sgt:0 src_rloc_trusted:0,sgtCacheControl1 = 0, sgtCacheControl0 = 0 port_label:0x0 port_mask:0x0 vlan_label:0x0 vlan_mask:0x0 l3if_label:0x0 l3if_mask:0x0 group_label:0x0 group_mask:0x0 ============================================================== C9300-PE-2#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ----- ------------------------ 10.0.0.22 GigabitEthernet2/0/1 0072.78c8.c9c2 0x7f0650a32858 0x7f0650a1af48 0x0 0x3a 2021/06/23 18:46:52.956 C9300-PE-2#show ip arp vrf RED 10.0.0.22 Protocol Address Age (min) Hardware Addr Type Interface Internet 10.0.0.22 131 0072.78c8.c9c2 ARPA GigabitEthernet2/0/1 <-- dst_mac matches the ARP entry C9300-PE-2#show platform hardware fed fwd-asic abstraction print-resource-handle 0x7f0650a32858 1 <-- Use the HTM value from previous command Handle:0x7f0650a32858 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f0650a1af48Hardware Indices/Handles: index0:0xad mtu_index/l3u_ri_index0:0x0 index1:0xad mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 25 00 00 00 00 00 00 00 00 00 00 00 08 00 00 72 78 c8 c9 c2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC# 0) ---------------------------------------- Station Index (SI) [0xad] RI = 0x18 DI = 0x5338 stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: CD Detailed Resource Information (ASIC# 1) ---------------------------------------- Station Index (SI) [0xad] RI = 0x18 DI = 0x5338 stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD ============================================================== C9300-PE-2#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x5338 0x5338 <-- Use the DI value from previous command ASIC#0: index = 0x5338 pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: index = 0x5338 pmap = 0x00000000 0x00000001 <-- Looking at 0x00000001, in binary that is 0000 0000 0000 0000 0000 0000 0000 0001 = Port 0 (Zero based, count right to left) cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 C9300-PE-2#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet2/0/1 0x35 1 0 1 0 0 26 6 1 97 NIF Y <-- Port 0 is the egress port, Gi2/0/1
VPNv4 Label Programming
The next section covers VPNv4 label programming on the MPLS PE routers, C9300-PE-1 and C9300-PE-2. The C9500=P does not forward on the VPNv4 label so there is no output from C9500-P.
C9300-PE-1 VPNv4 Label Programming:
Check the local prefix to the PE, not the remote prefix. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software VPNv4 Label Programming***
C9300-PE-1#show ip cef vrf RED 192.168.3.0/24 detail 192.168.3.0/24, epoch 0 QOS: Precedence routine (0) dflt local label info: other/21 [0x2] <-- VPNv4 label assocaited with the local prefix nexthop 10.0.0.1 GigabitEthernet1/0/1
***FMAN RP VPNv4 Label Programming*** C9300-PE-1#show platform software mpls switch active r0 eos index 117 <-- Utilize the objid from the FED command EOS Choice 0x75, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x74 OM handle: 0x3480644470 ***FMAN FP VPNv4 Label Programming*** C9300-PE-1#show platform software mpls switch active f0 eos index 117 <-- Utilize the objid from the FED command EOS Choice 0x75, Number of paths: 2 Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL Next Object Index: 0,0x74 aom id: 612, CPP handle: 0xdeadbeef (created), flags: 0 C9300-PE-1#show platform software object-manager switch active f0 object 612 <-- Use the aom id from previous command Object identifier: 612 Description: EOS Choice 0x75 Status: Done, Epoch: 0, Client data: 0xe05e9318 C9300-PE-1#show platform software object-manager switch active f0 object 612 parents <-- Use the aom id from previous command Object identifier: 7 Description: Special Object adj_drop Status: Done Object identifier: 611 Description: label 0x74 Status: Done
***FED VPNv4 Label Programming*** C9300-PE-1#show platform software fed switch active mpls forwarding label 21 detail LENTRY:label:21 nobj:(EOS, 117) lentry_hdl:0x8b000009 modify_cnt:0 backwalk_cnt:0 lspa_handle:0 AAL: id:2332033033 lbl:21 eos0:[adj_hdl:0, hw_hdl:0x7fbae8d87428] eos1:[adj_hdl:0x4300003b, hw_hdl:0x7fbae8d87278] deagg_vrf_id = 0 lspa_handle:0 EOS:objid:117 local_label:0 flags:0:() pdflags:0 <-- Utilized in previous commands nobj0:(ADJ SPECIAL,DROP 0), nobj1:(LABEL, 116) modify:0 bwalk:0 LABEL:objid:116 link_type:IP local_label:21 outlabel:(1048577, 0) flags:0xc:(UHP,POP,) pdflags:0x2:(INSTALL_HW_OK,) adj_handle:0x4300003b unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1124073531 lbl:0 smac:a0f8.4911.d1e4 dmac:0072.78c8.06e4 sub_type:0 link_type:0 adj_flags:0x2 label_type:1 rewrite_type:POP2IP(135) vlan_id:0 vrf_id:0 ri:0x7fbae8d811b8, ri_id:0x3e phdl:0xf1000024, ref_cnt:1 si:0x7fbae8d72078, si_id:0x4012, di_id:0x5338 ADJ:objid:58 {link_type:IP ifnum:0x35, si:0x1900001b, IPv4: 10.0.0.1 }
Verify C9300-PE-2 VPNv4 Labels
Check the local prefix to the PE, not the remote prefix. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
C9300-PE-2#show ip cef vrf RED 192.168.2.0/24 detail
192.168.2.0/24, epoch 0
QOS: Precedence routine (0)
dflt local label info: other/21 [0x2] <-- VPNv4 label associated with the local prefix
nexthop 10.0.0.22 GigabitEthernet2/0/1
C9300-PE-2#show platform software mpls switch active r0 eos index 118 <-- Utilize the objid value from the FED command
EOS Choice 0x76, Number of paths: 2
Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL
Next Object Index: 0,0x75
OM handle: 0x34806402d0
C9300-PE-2#show platform software mpls switch active f0 eos index 118 <-- Utilize the objid value from the FED command
EOS Choice 0x76, Number of paths: 2
Next Object Type: OBJ_ADJ_DROP,OBJ_LABEL
Next Object Index: 0,0x75
aom id: 589, CPP handle: 0xdeadbeef (created), flags: 0
C9300-PE-2#show platform software object-manager switch active f0 object 589 <-- Utilize the aom id from the previous command
Object identifier: 589
Description: EOS Choice 0x76
Status: Done, Epoch: 0, Client data: 0x248cac8
C9300-PE-2#show platform software object-manager switch active f0 object 589 parents <-- Utilize the aom id from the previous command
Object identifier: 7
Description: Special Object adj_drop
Status: Done
Object identifier: 588
Description: label 0x75
Status: Done
C9300-PE-2#show platform software fed switch active mpls forwarding label 21 detail
LENTRY:label:21 nobj:(EOS, 118) lentry_hdl:0x63000009
modify_cnt:0 backwalk_cnt:0
lspa_handle:0
AAL: id:1660944393 lbl:21
eos0:[adj_hdl:0, hw_hdl:0x7f0650a40408]
eos1:[adj_hdl:0xcb00003a, hw_hdl:0x7f0650a401f8]
deagg_vrf_id = 0 lspa_handle:0
EOS:objid:118 local_label:0 flags:0:() pdflags:0
nobj0:(ADJ SPECIAL,DROP 0), nobj1:(LABEL, 117) modify:0 bwalk:0
LABEL:objid:117 link_type:IP local_label:21 outlabel:(1048577, 0)
flags:0xc:(UHP,POP,) pdflags:0x2:(INSTALL_HW_OK,) adj_handle:0xcb00003a
unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0
bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0
AAL: id:3405774906 lbl:0 smac:70d3.79be.ae42 dmac:0072.78c8.c9c2
sub_type:0 link_type:0 adj_flags:0x2 label_type:1 rewrite_type:POP2IP(135)
vlan_id:0 vrf_id:0 ri:0x7f0650a3f2a8, ri_id:0x48 phdl:0xf1000024, ref_cnt:1
si:0x7f0650a3d5e8, si_id:0x400a, di_id:0x5338
ADJ:objid:58 {link_type:IP ifnum:0x35, si:0x9700001b, IPv4: 10.0.0.22 }
LDP Label Programming
The next section covers LDP label programming on the MPLS routers, C9300-PE-1, C9500-P, and C9300-PE-2.
The LDP (outer) label is what the MPLS network label-switches the packets on. Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label.
C9300-PE-1 LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9300-PE-1#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 Pop Label 192.168.1.3/32 0 Gi1/0/2 10.0.0.6 Pop Label 192.168.1.3/32 0 Gi1/0/3 10.0.0.10 17 Pop Label 10.0.0.16/30 0 Gi1/0/2 10.0.0.6 Pop Label 10.0.0.16/30 0 Gi1/0/3 10.0.0.10 18 Pop Label 10.0.0.12/30 0 Gi1/0/2 10.0.0.6 Pop Label 10.0.0.12/30 0 Gi1/0/3 10.0.0.10 19 17 192.168.1.4/32 0 Gi1/0/2 10.0.0.6 <-- LDP label 19 is advertised to reach PE 192.168.1.4 17 192.168.1.4/32 0 Gi1/0/3 10.0.0.10 20 No Label 10.0.0.0/30[V] 630 aggregate/RED 21 No Label 192.168.3.0/24[V] \ 0 Gi1/0/1 10.0.0.1
***FMAN RP LDP Label Programming*** C9300-PE-1#show platform software mpls switch active r0 label index 110 <-- Use the objid value from the FED commands Label OCE 0x6e -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x34806420d0 C9300-PE-1#show platform software mpls switch active r0 label index 111 <-- Use the objid value from the FED commands Label OCE 0x6f -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480642268
***FMAN FP LDP Label Programming*** C9300-PE-1#show platform software mpls switch active f0 label index 110 <-- Use the objid value from the FED commands Label OCE 0x6e -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 598, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software mpls switch active f0 label index 111 <-- Use the objid value from the FED commands Label OCE 0x6f -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x11 Backup flags: Pop, UHP, backup label 0x100001 aom id: 599, CPP handle: 0xdeadbeef (created) C9300-PE-1#show platform software object-manager switch active f0 object 598 <-- Utilize the aom id from previous commands Object identifier: 598 Description: label 0x6e Status: Done, Epoch: 0, Client data: 0xe05e6d78 C9300-PE-1#show platform software object-manager switch active f0 object 598 parents <-- Utilize the aom id from previous commands Object identifier: 531 Description: adj 0x4b, Flags None Status: Done C9300-PE-1#show platform software object-manager switch active f0 object 599 <-- Utilize the aom id from previous commands Object identifier: 599 Description: label 0x6f Status: Done, Epoch: 0, Client data: 0xe05e6f78 C9300-PE-1#show platform software object-manager switch active f0 object 599 parents <-- Utilize the aom id from previous commands Object identifier: 535 Description: adj 0x4e, Flags None Status: Done C9300-PE-1#show platform software fed switch active mpls forwarding label 19 detail LENTRY:label:19 nobj:(LB, 112) lentry_hdl:0x9000007 modify_cnt:1 backwalk_cnt:0 lspa_handle:0 AAL: id:150994951 lbl:19 eos0:[adj_hdl:0x7d000002, hw_hdl:0x7fbae8d778b8] eos1:[adj_hdl:0x7d000002, hw_hdl:0x7fbae8d776a8] deagg_vrf_id = 0 lspa_handle:0 LB:obj_id:112 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:19 path_inhw:2 ecrh:0x7d000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:2097152002 af:0 ecr_type:0 ref:7 ecrh:0x7fbae8a99268(28:2) hwhdl:3903427176 ::0x7fbae8a98b98,0x7fbae8a9ad48,0x7fbae8a98b98,0x7fbae8a9ad48 Sw Enh ECR scale: objid:112 llabel:19 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0x7d000002 ecr_hwhdl:0x7fbae8a99268 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:4278190135 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1744830509 sih:0x7fbae8a98b98(179) di_id:20499 rih:0x7fbae8a985d8(33) adj_lentry [eos0:0x7fbae8d7bf48 eos1:0x7fbae8d76e88] ecr_adj: id:1392508984 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:2013265966 sih:0x7fbae8a9ad48(180) di_id:20499 rih:0x7fbae8a9a788(46) adj_lentry [eos0:0x7fbae8d7c1b8 eos1:0x7fbae8d77158] ecr_prefix_adj: id:2164260921 (ref:1) sih:0x7fbae8d7df08(181) di_id:20499 rih:0x7fbae8d7db98(68) LABEL:objid:110 link_type:MPLS local_label:19 outlabel:(17, 0) <-- Used in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xff000037 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:4278190135 lbl:0 smac:a0f8.4911.d1d6 dmac:d4ad.71b5.dde4 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fbae8d78c48, ri_id:0x40 phdl:0x9f00004b, ref_cnt:1 si:0x7fbae8d78fd8, si_id:0x4013, di_id:0x535f ADJ:objid:75 {link_type:MPLS ifnum:0x36, si:0x22000023, } LABEL:objid:111 link_type:MPLS local_label:19 outlabel:(17, 0) <-- Used in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x53000038 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1392508984 lbl:0 smac:a0f8.4911.d1d8 dmac:d4ad.71b5.ddc2 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7fbae8d7d0a8, ri_id:0x42 phdl:0x8400004c, ref_cnt:1 si:0x7fbae8d7a908, si_id:0x4014, di_id:0x5360 ADJ:objid:78 {link_type:MPLS ifnum:0x37, si:0x74000026, }
C9500-P LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Check the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9500-P#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 explicit-n 192.168.1.2/32 1240 Te1/0/1 10.0.0.5 <-- LDP Label 16 advertised to reach PE 192.168.1.2 explicit-n 192.168.1.2/32 226537 Te2/0/1 10.0.0.9 17 explicit-n 192.168.1.4/32 610 Te1/0/2 10.0.0.14 <-- LDP LAbel 17 advertised to reach PE 192.168.1.4 explicit-n 192.168.1.4/32 227592 Te2/0/2 10.0.0.18
***FMAN RP LDP Label Programming*** C9500-P#show platform software mpls switch active r0 label index 94 Label OCE 0x5e -> OBJ_ADJACENCY (0x3f) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064c530 C9500-P#show platform software mpls switch active r0 label index 95 Label OCE 0x5f -> OBJ_ADJACENCY (0x44) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064c6c8 C9500-P#show platform software mpls switch active r0 label index 104 Label OCE 0x68 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064df70 C9500-P#show platform software mpls switch active r0 label index 105 Label OCE 0x69 -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x348064e108
***FMAN FP LDP Label Programming*** C9500-P#show platform software mpls switch active f0 label index 94 Label OCE 0x5e -> OBJ_ADJACENCY (0x3f) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 564, CPP handle: 0xdeadbeef (created) C9500-P#show platform software mpls switch active f0 label index 95 Label OCE 0x5f -> OBJ_ADJACENCY (0x44) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 565, CPP handle: 0xdeadbeef (created) C9500-P#show platform software mpls switch active f0 label index 104 Label OCE 0x68 -> OBJ_ADJACENCY (0x49) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 576, CPP handle: 0xdeadbeef (created) C9500-P#show platform software mpls switch active f0 label index 105 Label OCE 0x69 -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0 Backup flags: Pop, UHP, backup label 0x100001 aom id: 577, CPP handle: 0xdeadbeef (created) C9500-P#show platform software object-manager switch active f0 object 564 Object identifier: 564 Description: label 0x5e Status: Done, Epoch: 0, Client data: 0x4f737108 C9500-P#show platform software object-manager switch active f0 object 564 parents Object identifier: 515 Description: adj 0x3f, Flags None Status: Done C9500-P#show platform software object-manager switch active f0 object 565 Object identifier: 565 Description: label 0x5f Status: Done, Epoch: 0, Client data: 0x4f737448 C9500-P#show platform software object-manager switch active f0 object 565 parents Object identifier: 525 Description: adj 0x44, Flags None Status: Done C9500-P#show platform software object-manager switch active f0 object 576 Object identifier: 576 Description: label 0x68 Status: Done, Epoch: 0, Client data: 0x4f6d4bf8 C9500-P#show platform software object-manager switch active f0 object 576 parents Object identifier: 536 Description: adj 0x49, Flags None Status: Done C9500-P#show platform software object-manager switch active f0 object 577 Object identifier: 577 Description: label 0x69 Status: Done, Epoch: 0, Client data: 0x4f737f78 C9500-P#show platform software object-manager switch active f0 object 577 parents Object identifier: 545 Description: adj 0x4e, Flags None Status: Done
***FED LDP Label Programming*** C9500-P#show platform software fed switch active mpls forwarding label 16 detail LENTRY:label:16 nobj:(LB, 96) lentry_hdl:0xeb000004 modify_cnt:2 backwalk_cnt:0 lspa_handle:0 AAL: id:3942645764 lbl:16 eos0:[adj_hdl:0x44000002, hw_hdl:0x7f0b284b4d98] eos1:[adj_hdl:0x44000002, hw_hdl:0x7f0b284b4be8] deagg_vrf_id = 0 lspa_handle:0 LB:obj_id:96 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:16 path_inhw:2 ecrh:0x44000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:1140850690 af:0 ecr_type:0 ref:2 ecrh:0x7f0b284a3998(28:2) hwhdl:675953048 ::0x7f0b284b4268,0x7f0b284a1d78,0x7f0b284b4268,0x7f0b284a1d78 Sw Enh ECR scale: objid:96 llabel:16 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0x44000002 ecr_hwhdl:0x7f0b284a3998 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:1610612787 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1207959601 sih:0x7f0b284b4268(181) di_id:23709 rih:0x7f0b284b3ca8(31) adj_lentry [eos0:0x7f0b284a32d8 eos1:0x7f0b284a3cc8] ecr_adj: id:805306420 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:67108914 sih:0x7f0b284a1d78(182) di_id:23709 rih:0x7f0b284b47d8(44) adj_lentry [eos0:0x7f0b284c1608 eos1:0x7f0b284a2138] ecr_prefix_adj: id:3976200245 (ref:1) sih:0x7f0b284c2bf8(183) di_id:23709 rih:0x7f0b284c2888(50) LABEL:objid:94 link_type:MPLS local_label:16 outlabel:(0, 0) flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x60000033 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1610612787 lbl:0 smac:d4ad.71b5.dde4 dmac:a0f8.4911.d1d6 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284a2cd8, ri_id:0x2e phdl:0xe9000057, ref_cnt:1 si:0x7f0b284a3048, si_id:0x4009, di_id:0x1 ADJ:objid:63 {link_type:MPLS ifnum:0x41, si:0x2d000023, } LABEL:objid:95 link_type:MPLS local_label:16 outlabel:(0, 0) flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x30000034 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:805306420 lbl:0 smac:d4ad.71b5.ddc2 dmac:a0f8.4911.d1d8 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284a57c8, ri_id:0x30 phdl:0x67000059, ref_cnt:1 si:0x7f0b284a6008, si_id:0x400a, di_id:0x61 ADJ:objid:68 {link_type:MPLS ifnum:0x43, si:0xef000026, } C9500-P#show platform software fed switch active mpls forwarding label 17 detail LENTRY:label:17 nobj:(LB, 106) lentry_hdl:0xf6000005 modify_cnt:1 backwalk_cnt:0 lspa_handle:0 AAL: id:4127195141 lbl:17 eos0:[adj_hdl:0x44000002, hw_hdl:0x7f0b284ce2f8] eos1:[adj_hdl:0x44000002, hw_hdl:0x7f0b284ce0e8] deagg_vrf_id = 0 lspa_handle:0 LB:obj_id:106 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:17 path_inhw:2 ecrh:0x44000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:1140850690 af:0 ecr_type:0 ref:2 ecrh:0x7f0b284a3998(28:2) hwhdl:675953048 ::0x7f0b284b4268,0x7f0b284a1d78,0x7f0b284b4268,0x7f0b284a1d78 Sw Enh ECR scale: objid:106 llabel:17 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0x44000002 ecr_hwhdl:0x7f0b284a3998 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:4127195192 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1207959601 sih:0x7f0b284b4268(181) di_id:23709 rih:0x7f0b284b3ca8(31) adj_lentry [eos0:0x7f0b284c38e8 eos1:0x7f0b284cd858] ecr_adj: id:1157627961 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:67108914 sih:0x7f0b284a1d78(182) di_id:23709 rih:0x7f0b284b47d8(44) adj_lentry [eos0:0x7f0b284c3af8 eos1:0x7f0b284cdb28] ecr_prefix_adj: id:3707764794 (ref:1) sih:0x7f0b284c5028(184) di_id:23709 rih:0x7f0b284c4c48(60) LABEL:objid:104 link_type:MPLS local_label:17 outlabel:(0, 0) flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xf6000038 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:4127195192 lbl:0 smac:d4ad.71b5.ddd6 dmac:70d3.79be.ae71 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284ceaa8, ri_id:0x38 phdl:0x76000058, ref_cnt:1 si:0x7f0b284ceeb8, si_id:0x400b, di_id:0x2 ADJ:objid:73 {link_type:MPLS ifnum:0x42, si:0x1f000028, } LABEL:objid:105 link_type:MPLS local_label:17 outlabel:(0, 0) flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x45000039 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:1157627961 lbl:0 smac:d4ad.71b5.ddf1 dmac:70d3.79be.ae61 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0b284c4588, ri_id:0x3a phdl:0x5500005a, ref_cnt:1 si:0x7f0b284d0548, si_id:0x400c, di_id:0x62 ADJ:objid:78 {link_type:MPLS ifnum:0x44, si:0x4900002a, }
C9300-PE-2 LDP Label Programming:
Validate the local LDP label that is advertised to the remote PE, do not validate the remote LDP label. Start by checking the label from a FED perspective and then backtrack to FMAN RP and FMAN FP.
***Software LDP Label Programming***
C9300-PE-2#show mpls forwarding-table Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 Pop Label 192.168.1.3/32 0 Gi2/0/2 10.0.0.13 Pop Label 192.168.1.3/32 0 Gi2/0/3 10.0.0.17 17 16 192.168.1.2/32 0 Gi2/0/2 10.0.0.13 <-- LDP Label 17 is advertised to Remote PE 192.168.1.2 16 192.168.1.2/32 0 Gi2/0/3 10.0.0.17 18 Pop Label 10.0.0.4/30 0 Gi2/0/2 10.0.0.13 Pop Label 10.0.0.4/30 0 Gi2/0/3 10.0.0.17 19 Pop Label 10.0.0.8/30 0 Gi2/0/2 10.0.0.13 Pop Label 10.0.0.8/30 0 Gi2/0/3 10.0.0.17 20 No Label 10.0.0.20/30[V] 630 aggregate/RED 21 No Label 192.168.2.0/24[V] \ 0 Gi2/0/1 10.0.0.22 ***FMAN RP Label Programming***
C9300-PE-2#show platform software mpls switch active r0 label index 106 <-- Use the objid values from the FED commands Label OCE 0x6a -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480637358 C9300-PE-2#show platform software mpls switch active r0 label index 107 <-- Use the objid values from the FED commands Label OCE 0x6b -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 OM handle: 0x3480638c10
***FMAN FP LDP Label Programming*** C9300-PE-2#show platform software mpls switch active f0 label index 106 Label OCE 0x6a -> OBJ_ADJACENCY (0x4b) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 aom id: 548, CPP handle: 0xdeadbeef (created) C9300-PE-2#show platform software mpls switch active f0 label index 107 Label OCE 0x6b -> OBJ_ADJACENCY (0x4e) Flags: Real, Number of labels in the OCE: 1 Label values: 0x10 Backup flags: Pop, UHP, backup label 0x100001 aom id: 549, CPP handle: 0xdeadbeef (created) C9300-PE-2#show platform software object-manager switch active f0 object 548 <-- Use the aom id value from the previous commands Object identifier: 548 Description: label 0x6a Status: Done, Epoch: 0, Client data: 0x24843d8 C9300-PE-2#show platform software object-manager switch active f0 object 548 parents <-- Use the aom id value from the previous commands Object identifier: 509 Description: adj 0x4b, Flags None Status: Done C9300-PE-2#show platform software object-manager switch active f0 object 549 <-- Use the aom id value from the previous commands Object identifier: 549 Description: label 0x6b Status: Done, Epoch: 0, Client data: 0x2484518 C9300-PE-2#show platform software object-manager switch active f0 object 549 parents <-- Use the aom id value from the previous commands Object identifier: 513 Description: adj 0x4e, Flags None Status: Done
***FED LDP Label Programming*** C9300-PE-2#show platform software fed switch active mpls forwarding label 17 detail LENTRY:label:17 nobj:(LB, 108) lentry_hdl:0x64000005 modify_cnt:1 backwalk_cnt:0 lspa_handle:0 AAL: id:1677721605 lbl:17 eos0:[adj_hdl:0xa0000002, hw_hdl:0x7f0650a5c8e8] eos1:[adj_hdl:0xa0000002, hw_hdl:0x7f0650a5b908] deagg_vrf_id = 0 lspa_handle:0 LB:obj_id:108 link_type:IP num_choices:2 Flags:0 mpls_ecr:1 local_label:17 path_inhw:2 ecrh:0xa0000002 old_ecrh:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 finish_cnt:0 bwalk:[req:0 in_prog:0 nested:0] AAL: ecr:id:2684354562 af:0 ecr_type:0 ref:7 ecrh:0x7f0650a62888(28:2) hwhdl:1353066632 ::0x7f0650a60998,0x7f0650a630d8,0x7f0650a60998,0x7f0650a630d8 Sw Enh ECR scale: objid:108 llabel:17 eos:1 #adjs:2 mixed_adj:0 reprogram_hw:0 ecrhdl:0xa0000002 ecr_hwhdl:0x7f0650a62888 mod_cnt:0 prev_npath:0 pmismatch:0 pordermatch:0 ecr_adj: id:436207667 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:2113929262 sih:0x7f0650a60998(178) di_id:20507 rih:0x7f0650a60378(50) adj_lentry [eos0:0x7f0650a877d8 eos1:0x7f0650a1cf78] ecr_adj: id:3976200246 is_mpls_adj:1 l3adj_flags:0x100000 recirc_adj_id:1509949487 sih:0x7f0650a630d8(179) di_id:20507 rih:0x7f0650a62b18(51) adj_lentry [eos0:0x7f0650a87a48 eos1:0x7f0650a1d188] ecr_prefix_adj: id:2919235640 (ref:1) sih:0x7f0650a87558(180) di_id:20507 rih:0x7f0650a871d8(68) LABEL:objid:106 link_type:MPLS local_label:17 outlabel:(16, 0) <-- Used in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0x1a000033 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:436207667 lbl:0 smac:70d3.79be.ae71 dmac:d4ad.71b5.ddd6 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0650a67d48, ri_id:0x3a phdl:0x9f00004b, ref_cnt:1 si:0x7f0650a65408, si_id:0x4010, di_id:0x535f ADJ:objid:75 {link_type:MPLS ifnum:0x36, si:0x35000023, } LABEL:objid:107 link_type:MPLS local_label:17 outlabel:(16, 0) <-- Used in previous commands flags:0x1:(REAL,) pdflags:0:(INSTALL_HW_OK,) adj_handle:0xed000036 unsupported recursion:0 olbl_changed 0 local_adj:0 modify_cnt:0 bwalk_cnt:0 subwalk_cnt:0 collapsed_oce:0 AAL: id:3976200246 lbl:0 smac:70d3.79be.ae61 dmac:d4ad.71b5.ddf1 sub_type:0 link_type:2 adj_flags:0 label_type:2 rewrite_type:PSH1(119) vlan_id:0 vrf_id:0 ri:0x7f0650a6f4f8, ri_id:0x40 phdl:0x8400004c, ref_cnt:1 si:0x7f0650a73088, si_id:0x4013, di_id:0x5360 ADJ:objid:78 {link_type:MPLS ifnum:0x37, si:0xa2000025, }
Troubleshoot Hardware Scale
This section provides information you can use in order to troubleshoot your configuration.
MPLS Hardware Syslogs
If you run out of a particular resource, such as MPLS labels, SYSLOG message are generated by the system.
Key points to remember
- MPLS LABEL is used for label disposition. (This resource is consumed when prefixes are learnt from a local CE)
- LSPA is used for label imposition. (This resource is consumed when prefixes are learnt from a remote PE)
| MPLS Log Message | Definition | Recovery Action |
|
%FED_L3_ERRMSG-3-RSRC_ERR: Switch 1 R0/0: fed: Failed to allocate hardware resource for fib entry due to hardware resource exhaustion |
Hardware reserved for IP prefixes has run out of space (EM or TCAM) |
Take one of these actions to reduce the number of prefixes learned by the local or remote PE: 1. Summarize prefixes at CE 2. Change label allocation mode from per-prefix to per-vrf |
|
%FED_L3_ERRMSG-3-mpls_out_of_resource: Switch 1 R0/0: fed: Out of resource for MPLS LABEL ENTRY. Failed to program local label:8205 (8192/8192) in hardware |
Local label Allocation: Hardware reserved for MPLS local labels has run out of space (EM or TCAM) |
Take one of these actions to reduce number of lables used on local PE: 1. Summarize prefixes at local CE or local PE 2. Change label allocation mode from per-prefix to per-vrf on the local PE |
|
%FED_L3_ERRMSG-3-MPLS_LENTRY_PAUSE: Switch 1 R0/0: fed: Critical limit reached for MPLS LABEL ENTRY resource. Lentry Create PAUSED. |
Local label Allocation: Hardware reserved for MPLS local labels has run out of space (EM or TCAM) |
Take one of these actions to reduce number of lables used on local PE: 1. Summarize prefixes at local CE or local PE 2. Change label allocation mode from per-prefix to per-vrf on the local PE |
|
%FED_L3_ERRMSG-3-mpls_out_of_resource: Switch 1 R0/0: fed: Out of resource for MPLS LSPA. Failed to program in hardware |
Remote label allocation: Hardware reserved for LSPA remote labels has run out of space |
Take one of these actions to reduce number of lables used on remote PE: 1. Summarize prefixes at remote CE or remote PE 2. Change label allocation mode from per-prefix to per-vrf on the remote PE |
Hardware Validation Commands
show platform hardware fed active fwd-asic resource tcam utilizationcommand is the first place you want to look to evaluate if you have a hardware scale issue. It displays information on a per-ASIC basis.
This section shows a PE learning prefixes from BGP in vrf MPLS with the parameters described here:
- The default per-prefix label allocation is used
- PE is C9300-48U with Cisco IOS-XE 17.3.4
- CE is Ixia as a BGP neighbor that advertises prefixes to an interface in vrf MPLS
- Prefix length used is /28. Thus platform uses TCAM for prefix lengths /31 or shorter
- This platform uses EM memory for MPLS/BGP labels first, then overflows to TCAM if EM becomes full
Topology
Baseline Resource Usage
Prior to the addition of any prefixes, there is some base usage:
- This baseline was taken after MPLS LDP neighbors were formed in global table
- From this baseline, VPNv4 prefixes are added in VRF MPLS
- Your baseline numbers can vary. It depends on what is already programmed on the switch
C9300-48U#show version | inc IOS
Cisco IOS XE Software, Version 17.03.04
Cisco IOS Software [Amsterdam], Catalyst L3 Switch Software (CAT9K_IOSXE), Version 17.3.4, RELEASE SOFTWARE (fc3)
C9300-48U#show platform hardware fed switch active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ Mac Address Table EM I 32768 20 0.06% 0 0 0 20 Mac Address Table TCAM I 1024 21 2.05% 0 0 0 21 L3 Multicast EM I 8192 0 0.00% 0 0 0 0 L3 Multicast TCAM I 512 9 1.76% 3 6 0 0 L2 Multicast EM I 8192 0 0.00% 0 0 0 0 L2 Multicast TCAM I 512 11 2.15% 3 8 0 0 IP Route Table EM I 24576 23 0.09% 14 0 9 0 <-- 23 EM (hash) base usage IP Route Table TCAM I 8192 25 0.31% 12 10 2 1 <-- 25 TCAM base usage
C9300-48U#show platform software fed switch active mpls summary | b Resource shar
Resource sharing info:
SI: 4/65536
RI: 10/65536
Well Known Index: 49/2048
Tcam: 21/57344
lv1_ecr: 0/64
lv2_ecr: 0/256
lspa: 0/16385
label_stack_id: 2/65537
vpn_spoke_id: 0/255
indirect_si: 0/255
RSM resource database stats:
Num of (L3+mpls) ADJ entries allocated: 36/131072
Num of LABEL entries allocated: 4/8192 <-- Baseline label usage = 4 (label entries allocated on local PE-CE side)
Num of LSPA entries allocated: 0/8192 <-- LSPA resource used when prefix learnt from another PE, not from a local CE (The SDM template determines max value)
Num of local adjs in mpls adjs: 3
Num of SI stats allocated: 6/49152
Adjs stats allocated by MPLS:
Num of mpls adjs: 11
Num of L3 adjs: 0
Num of VPN prefix_id: 0
<...snip...>
Other MPLS resource alloc error stats: <-- reported resource allocation issues shown here
LENTRY out-of-resource errors: 0
LENTRY general errors: 0
LSPA out-of-resource errors: 0
LSPA general errors: 0
ADJ out-of-resource errors: 0
SI stats alloc error: 0
MPLS ADJ stats error: 0
MPLS ADJ stats last error rc: 0
Add 1000 BGP VPNv4 Prefixes
Neighbor (Ixia) brought up with 1000 prefixes added to VRF MPLS from CE
9300 Local PE (connected to CE)
C9300-48U#show bgp vpnv4 unicast all summary
BGP router identifier 10.0.0.1, local AS number 65000
<...snip...> Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.0.0.5 4 65000 102 304 3001 0 0 01:28:23 0 192.168.1.2 4 65005 102 5 3001 0 0 00:00:58 1000 <-- PE learns 1000 prefixes from CE device
C9300-48U#show bgp vpnv4 unicast all | count /28
Number of lines which match regexp = 1000 <-- All 1000 prefixes are /28
C9300-48U#show platform hardware fed switch active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ Mac Address Table EM I 32768 20 0.06% 0 0 0 20 Mac Address Table TCAM I 1024 21 2.05% 0 0 0 21 L3 Multicast EM I 8192 0 0.00% 0 0 0 0 L3 Multicast TCAM I 512 9 1.76% 3 6 0 0 L2 Multicast EM I 8192 0 0.00% 0 0 0 0 L2 Multicast TCAM I 512 11 2.15% 3 8 0 0 IP Route Table EM I 24576 2023 8.23% 14 0 2009 0 IP Route Table TCAM I 8192 1025 12.51% 1012 10 2 1
<-- 25 base + 1000 /28 prefixes = 1025 TCAM entries
<-- MPLS labels are added to EM, and each MPLS label uses 2 entries (one IPv4 prefix, and one MPLS label results in 3 entries used in hardware)
C9300-48U#show platform software fed switch active mpls summary | b Resource shar Resource sharing info: SI: 4/65536 RI: 1010/65536 Well Known Index: 49/2048 Tcam: 1021/57344 lv1_ecr: 0/64 lv2_ecr: 0/256 lspa: 0/16385 label_stack_id: 1002/65537 vpn_spoke_id: 0/255 indirect_si: 0/255 RSM resource database stats: Num of (L3+mpls) ADJ entries allocated: 1036/131072 Num of LABEL entries allocated: 1004/8192 <-- Increased by 1000 on local PE Num of LSPA entries allocated: 0/8192 <-- No prefixes learnt from remote PE, no LSPA allocated Num of local adjs in mpls adjs: 3 Num of SI stats allocated: 1006/49152 Adjs stats allocated by MPLS: Num of mpls adjs: 1011 Num of L3 adjs: 0 Num of VPN prefix_id: 0 <...snip...> Other MPLS resource alloc error stats: <-- no resource allocation issues LENTRY out-of-resource errors: 0 LENTRY general errors: 0 LSPA out-of-resource errors: 0 LSPA general errors: 0 ADJ out-of-resource errors: 0 SI stats alloc error: 0 MPLS ADJ stats error: 0 MPLS ADJ stats last error rc: 0
<-- Resources shown in baseline outputs are now increased by 1000
9500H Remote PE (Learned over MPLS)
C9500-24Y4C#show platform hardware fed active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ Mac Address Table EM I 32768 19 0.06% 0 0 0 19 Mac Address Table TCAM I 768 21 2.73% 0 0 0 21 L3 Multicast EM I 32768 0 0.00% 0 0 0 0 L3 Multicast TCAM I 768 6 0.78% 3 3 0 0 L2 Multicast TCAM I 2304 7 0.30% 3 4 0 0 IP Route Table EM/LPM I 212992 1012 0.48% 1003 0 9 0 IP Route Table TCAM I 1536 28 1.82% 23 3 2 0
CTS Cell Matrix/VPN
Label EM O 32768 992 3.03% 0 0 992 0 <-- MPLS VPN used 992 entries
CTS Cell Matrix/VPN
Label TCAM O 768 9 1.17% 0 0 8 1
<-- 1000 /28 IPv4 prefixes learned from remote PE (On the 9500HP these /28 prefixes are be stored in EM/LPM memory, not TCAM)
<-- Hardware shared between CTS and VPN (resource is used when prefixes learned PE-PE, label imposition)
C9500-24Y4C#show platform software fed active mpls summary | b Resource shar Resource sharing info: SI: 4/131072 RI: 11/98304 Well Known Index: 48/2048 Tcam: 20/245760 lv1_ecr: 0/64 lv2_ecr: 0/256 lspa: 1000/65536 label_stack_id: 2/65537 vpn_spoke_id: 0/255 indirect_si: 0/255 RSM resource database stats: Num of (L3+mpls) ADJ entries allocated: 37/196608 Num of LABEL entries allocated: 4/45056 <-- LABEL does not increase (no prefixes learnt from a local CE) Num of LSPA entries allocated: 1000/32768 <-- LSPA usage increased by 1000 (these prefixes require label stack to reach) Num of local adjs in mpls adjs: 4 Num of SI stats allocated: 6/49152 Adjs stats allocated by MPLS: Num of mpls adjs: 12 Num of L3 adjs: 0 Num of VPN prefix_id: 1000 AL MPLS SI/RI resource alloc stats: SI allocated: 1 RI allocated: 6 SI_STATS allocated: 6 Unknowns allocs: 0 Alloc no resource: 0 Alloc errors: 0 Free errors: 0 Invalid free: 0 Free unknown: 0 Other MPLS resource alloc error stats: <-- no resource allocation issues LENTRY out-of-resource errors: 0 LENTRY general errors: 0 LSPA out-of-resource errors: 0 LSPA general errors: 0 ADJ out-of-resource errors: 0 SI stats alloc error: 0 MPLS ADJ stats error: 0 MPLS ADJ stats last error rc: 0
<-- Different resources are allocated to reach a local prefix (LABEL) versus a remote prefix (LSPA)
MPLS Label and IPv4 Scale Limit and Remediation
In most cases when the MPLS feature is used, and too many hardware resources are consumed, a change to the label allocation from (default) per-prefix to per-vrf can help. In this example consider the resource allocation before and after (in this case, the 9500 is the CE-PE device).
### Usage with per-prefix label allocation ###
C9500-24Y4C#show platform hardware fed active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ Mac Address Table EM I 32768 19 0.06% 0 0 0 19 Mac Address Table TCAM I 768 21 2.73% 0 0 0 21 L3 Multicast EM I 32768 0 0.00% 0 0 0 0 L3 Multicast TCAM I 768 6 0.78% 3 3 0 0 L2 Multicast TCAM I 2304 7 0.30% 3 4 0 0 IP Route Table EM/LPM I 212992 3023 1.42% 1014 0 2009 0 <-- 1 IPv4 prefix entry + 2 entries for labels (2 labels created per every 1 IPv4 prefix) IP Route Table TCAM I 1536 17 1.11% 12 3 2 0
### New usage after change to per-vrf lable allocation ###
C9500-24Y4C(config)#mpls label mode vrf MPLS protocol all-afs per-vrf
C9500-24Y4C#show bgp vpnv4 unicast all BGP table version is 164901, local router ID is 10.0.0.5 Network Next Hop Metric LocPrf Weight Path Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, t secondary path, L long-lived-stale, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf MPLS) *> 172.30.0.0/24 192.168.3.2 2219 0 65100 65101 65102 65103 {65104} e <...snip...>
C9500-24Y4C#show bgp vpnv4 unicast all 172.30.0.0 BGP routing table entry for 1:1:172.30.0.0/24, version 163902 Paths: (1 available, best #1, table MPLS) Advertised to update-groups: 8 Refresh Epoch 1 65100 65101 65102 65103 {65104} 192.168.3.2 (via vrf MPLS) from 192.168.3.2 (192.168.3.2) Origin EGP, metric 2219, localpref 100, valid, external, best Extended Community: RT:1:1 mpls labels in/out IPv4 VRF Aggr:18116/nolabel <-- Verify you see a 'VRF Aggr' label type rx pathid: 0, tx pathid: 0x0 Updated on Dec 9 2021 19:50:22 UTC
### Usage with per-vrf label allocation ###
Allocation on both local and remote PE is dramatically reduced via change to label allocation mode
local switch (PE-CE) C9500-24Y4C#show platform hardware fed active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ Mac Address Table EM I 32768 19 0.06% 0 0 0 19 Mac Address Table TCAM I 768 21 2.73% 0 0 0 21 L3 Multicast EM I 32768 0 0.00% 0 0 0 0 L3 Multicast TCAM I 768 6 0.78% 3 3 0 0 L2 Multicast TCAM I 2304 7 0.30% 3 4 0 0 IP Route Table EM/LPM I 212992 1025 0.48% 1014 0 11 0 <-- one local LABEL used to reach the CE learnt prefixes IP Route Table TCAM I 1536 17 1.11% 12 3 2 0 QOS ACL TCAM I 1024 45 4.39% 15 20 0 10
remote switch (PE-PE) C9300-48U#show platform hardware fed switch active fwd-asic resource tcam utilization Codes: EM - Exact_Match, I - Input, O - Output, IO - Input & Output, NA - Not Applicable CAM Utilization for ASIC [0] Table Subtype Dir Max Used %Used V4 V6 MPLS Other ------------------------------------------------------------------------------------------------------ <...snip...> IP Route Table EM I 24576 23 0.09% 14 0 9 0 IP Route Table TCAM I 8192 1025 12.51% 1012 10 2 1 <-- Still 1:1 usage for IPv4 prefixes <...snip...> CTS Cell Matrix/VPN Label EM O 8192 1 0.01% 0 0 1 0 <-- one remote LSPA used to reach the PE learnt prefixes
Commands to Collect for TAC
The most common hardware resource problems related to MPLS are covered in this guide, with appropriate remediation steps. However, in the event that this guide did not resolve your issue please collect the command list shown and attach them to the service request.
show ip route summary
show ip bgp vpnv4 all | redirect flash:bgp_vpnv4_all
show ip bgp vpnv4 all summary
show ip route vrf <vrf-name> summary
show mpls forwarding-table summary
show ip cef vrf <name> | redirect flash:sh_ip_cef_vrf_<name>
show ip cef vrf <name> summary
show platform software fed switch active ip route summary
show platform software mpls switch <all switches> f0 forwarding-table
show platform software mpls switch <all switches> f0 label
show platform software mpls switch <all switches> f0 eos
show platform software object-manager switch <all switches> f0 error-object
show platform software object-manager switch <all switches> f0 pending-issue-update
show platform software fed switch <all switches> mpls label_oce all detail
show platform software fed switch <all switches> mpls eos all det
show platform software fed switch <all switches> mpls summary
show platform software fed switch active mpls forwarding all detail
show platform software object-manager switch 1 f0 statistics
show tech-support mpls | redirect flash:sh_tech_mpls
show logging | redirect flash:sh_logging_console
show platform hard fed switch active fwd resource tcam table sghash asic 0 format 0 | redirect flash:vpn_lspa
request platform software trace archive last 30 days target flash
Related Information
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
3.0 |
08-Mar-2022 |
Adjusted table formatting |
2.0 |
03-Mar-2022 |
One more title revision |
1.0 |
25-Feb-2022 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)