Troubleshoot RIB Error Message "ROUTING-RIB-3-LABEL_ERR_ADD: Add local-label"

Introduction

This document describes how to troubleshoot the Cisco IOS® XR Routing Information Base (RIB) message "%ROUTING-RIB-3-LABEL_ERR_ADD: Add local-label".

Message Description

In order to clarify its reason and purpose, an example is used, and it is divided into its different components:

RP/0/0/CPU0:Feb 17 11:46:25.663 : ipv4_rib[1148]: %ROUTING-RIB-3-LABEL_ERR_ADD : Add local-label 16111 (2) for table 0xe0000000, prefix 10.0.0.111/32, by proto isis client 17 isis node0_0_CPU0 - existing label 24005 added by proto-id 2 client 16

As per the message details process, ipv4_rib generated an error upon attempt from protocol Intermediate System-to-Intermediate System (IS-IS) - rib client 17 - to add a local-label 16111 for prefix 10.0.0.111/32. The label 24005 already exists in the RIB database for the same prefix previously added by another protocol - rib client 16. In summary, two distinct protocols added a different local-label for the same prefix in the RIB database. This is an error condition and label conflict must be corrected and resolved.

Understand Message Origin

An error condition can occur in a scenario where multiple protocols exchange labels such as when you simultaneously use Border Gateway Protocol - Labelled Unicast (BGP-LU) and Segment-Routing (SR).

Consider the scenario where BGP-LU is used between hosts within the same Autonomous System (AS) to advertise selected prefixes and associated labels. The prefix 10.0.0.111/32 which is known via IS-IS has a local-label assigned by BGP as shown:

RP/0/0/CPU0:XRV4#show route 10.0.0.111/32 detail private
Routing entry for 10.0.0.111/32
  Known via "isis core", distance 115, metric 30, type level-2
  Installed Feb 17 10:56:08.900 for 00:34:45
  Routing Descriptor Blocks
    10.3.4.3, from 10.0.0.111, via GigabitEthernet0/0/0/0
      Route metric is 30
      Label: None
      Tunnel ID: None
      Binding Label: None
      Extended communities count: 0
      Path id:1       Path ref count:0
      NHID:0x1(Ref:8)
      Path flags: 0x0 ()
      Private flags: 0x40 (rib_encap_id)
  Route version is 0x17 (23)
  Local Label: 0x5dc5 (24005)
  IP Precedence: Not Set
  QoS Group ID: Not Set
  Flow-tag: Not Set
  Fwd-class: Not Set
  Route Priority: RIB_PRIORITY_NON_RECURSIVE_MEDIUM (7) SVD Type RIB_SVD_TYPE_LOCAL
  Download Priority 1, Download Version 287
  Client-id: 17
  Route flags: 0x0 ()
  Route Extended flags: 0x0 ()
  Route private flags: 0x20 (priority_promotion)
  Route head pointer: 0x1201f5bc
  Local Label List
    B 24005 (Client ID: 16, Distance: 200,)
  No advertising protos.

The protocol owner of the assigned local-label 24005 can be confirmed and verified from the MPLS label table:

RP/0/0/CPU0:XRV4#show mpls label table label 24005 detail private
Table Label   Owner                           State  Rewrite
----- ------- ------------------------------- ------ -------
0     24005   LDP(A)                          InUse  Yes
              BGP-VPNv4(A):bgp-default        InUse  No
  (IPv4, vers:0, 'default':4U, 10.0.0.111/32)

The moment SR is enabled in the network, then the error condition is triggered as IS-IS adds its own label in the RIB database. This causes the label conflict with the already present and previously allocated label from BGP-LU. At this stage the error message is generated by process ipv4_rib and alerts for the attempt from protocol IS-IS - rib client 17 - to add a local-label 16111 for prefix 10.0.0.111/32 for which there is already a local-label 24005 previously added by protocol BGP - rib client 16:

RP/0/0/CPU0:Feb 17 11:46:25.663 : ipv4_rib[1148]: %ROUTING-RIB-3-LABEL_ERR_ADD : Add local-label 16111 (2) for table 0xe0000000, prefix 10.0.0.111/32, by proto isis client 17 isis node0_0_CPU0 - existing label 24005 added by proto-id 2 client 16

The label conflict condition can be seen and confirmed from prefix route local label list details as shown:

RP/0/0/CPU0:XRV4#show route 10.0.0.111/32 detail private
Routing entry for 10.0.0.111/32
  Known via "isis core", distance 115, metric 30, labeled SR, type level-2
  Installed Feb 17 11:46:25.663 for 00:21:45
  Routing Descriptor Blocks
    10.3.4.3, from 10.0.0.111, via GigabitEthernet0/0/0/0
      Route metric is 30
      Label: 0x3eef (16111)
      Tunnel ID: None
      Binding Label: None
      Extended communities count: 0
      Path id:1       Path ref count:0
      NHID:0x1(Ref:8)
      Path flags: 0x0 ()
      Private flags: 0x42 (has_vpn_data, rib_encap_id)
  Route version is 0x1e (30)
  Local Label: 0x3eef (16111)
  IP Precedence: Not Set
  QoS Group ID: Not Set
  Flow-tag: Not Set
  Fwd-class: Not Set
  Route Priority: RIB_PRIORITY_NON_RECURSIVE_MEDIUM (7) SVD Type RIB_SVD_TYPE_LOCAL
  Download Priority 1, Download Version 309
  Client-id: 17
  Route flags: 0x8800000 (ldp_sr_merge_request, label_sr)
  Route Extended flags: 0x8 (ldp_sr_merge_request)
  Route private flags: 0x20 (priority_promotion)
  Route head pointer: 0x1201f5bc
  Local Label List
    i 16111 (Client ID: 17, Distance: 115,)
    B 24005 (Client ID: 16, Distance: 200,)
  No advertising protos. 

With label conflict in place, both BGP-LU and IS-IS-related outputs provide additional details on the local-label assigned by each specific protocol. The next outputs highlight each protocol specifics with regards to both prefix and assigned label.

While in a label conflict scenario, note prefix, and label details, as seen from protocol BGP-LU:

RP/0/0/CPU0:XRV4#show bgp ipv4 labeled-unicast 10.0.0.111/32 detail       
BGP routing table entry for 10.0.0.111/32
Versions:
  Process           bRIB/RIB  SendTblVer
  Speaker                 69          69
    Local Label: 24005 (no rewrite);
    Flags: 0x01003001+0x00000200;
Last Modified: Feb 17 11:46:31.984 for 00:22:30
Paths: (1 available, best #1)
  Not advertised to any peer
  Path #1: Received by speaker 0
  Flags: 0x4080000009060005, import: 0x20
  Not advertised to any peer
  Local
    10.0.0.111 (metric 30) from 10.0.0.111 (10.0.0.111)
      Received Label 3
      Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, labeled-unicast
      Received Path ID 0, Local Path ID 0, version 69
      Prefix SID Attribute Size: 10
      Label Index: 111

RP/0/0/CPU0:XRV4#show bgp ipv4 labeled-unicast labels 
<snip>
   Network            Next Hop        Rcvd Label      Local Label
*> 10.0.0.4/32        0.0.0.0         nolabel         3
*>i10.0.0.111/32      10.0.0.111      3               24005
Processed 2 prefixes, 2 paths

RP/0/0/CPU0:XRV4#show mpls label table label 24005 detail internal
Table Label   Owner                           State  Rewrite
----- ------- ------------------------------- ------ -------
0     24005   LDP(A)                          InUse  Yes
              BGP-VPNv4(A):bgp-default        InUse  No
  (IPv4, vers:0, 'default':4U, 10.0.0.111/32)

While in a label conflict scenario, note prefix, and label details, as seen from protocol IS-IS:

RP/0/0/CPU0:XRV4#show isis ipv4 route 10.0.0.111/32 detail       
L2 10.0.0.111/32 [30/115] medium priority
     via 10.3.4.3, GigabitEthernet0/0/0/0, XRV3, SRGB Base: 16000, Weight: 0
     src XRV111.00-00, 10.0.0.111, prefix-SID index 111, R:0 N:1 P:0 E:0 V:0 L:0

RP/0/0/CPU0:XRV4#show isis segment-routing label 16111
IS-IS core IS Label Table
Label         Prefix/Interface
----------    ----------------
16111         10.0.0.111/32

RP/0/0/CPU0:XRV4#show mpls label table label 16111 detail internal
Table Label   Owner                           State  Rewrite
----- ------- ------------------------------- ------ -------
0     16111   ISIS(A):core                    InUse  No
  (Lbl-blk SRGB, vers:0, (start_label=16000, size=8000)

Both protocols and associated labels are now programmed and can be observed from MPLS forwarding details:

RP/0/0/CPU0:XRV4#show mpls forwarding labels 24005 detail private
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes      
Label  Label       or ID              Interface                    Switched   
------ ----------- ------------------ ------------ --------------- ------------
24005  24004       10.0.0.111/32      Gi0/0/0/0    10.3.4.3        3055       
     Updated: Feb 17 11:46:25.703
     Version: 217, Priority: 3
     Label Stack (Top -> Bottom): { 24004 }
     NHID: 0x0, Encap-ID: N/A, Path idx: 0, Backup path idx: 0, Weight: 0
     MAC/Encaps: 14/18, MTU: 1500
     Outgoing Interface: GigabitEthernet0/0/0/0 (ifhandle 0x00000040)
     Packets Switched: 56
     Traffic-Matrix Packets/Bytes Switched: 0/0
  Traffic-Matrix Packets/Bytes Switched: 0/0

RP/0/0/CPU0:XRV4#show mpls forwarding labels 16111 detail private
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes      
Label  Label       or ID              Interface                    Switched   
------ ----------- ------------------ ------------ --------------- ------------
16111  16111       SR Pfx (idx 111)   Gi0/0/0/0    10.3.4.3        0          
     Updated: Feb 17 11:46:25.703
     Version: 309, Priority: 15
     Label Stack (Top -> Bottom): { 16111 }
     NHID: 0x0, Encap-ID: N/A, Path idx: 0, Backup path idx: 0, Weight: 0
     MAC/Encaps: 14/18, MTU: 1500
     Outgoing Interface: GigabitEthernet0/0/0/0 (ifhandle 0x00000040)
     Packets Switched: 0
     Traffic-Matrix Packets/Bytes Switched: 0/0
  Traffic-Matrix Packets/Bytes Switched: 0/0

However, from prefix 10.0.0.111/32 forwarding details you can observe that the initial assigned label 24005 is still in use as shown:

RP/0/0/CPU0:XRV4#show mpls forwarding prefix 10.0.0.111/32 detail private
Local  Outgoing    Prefix             Outgoing     Next Hop        Bytes      
Label  Label       or ID              Interface                    Switched   
------ ----------- ------------------ ------------ --------------- ------------
24005  24004       10.0.0.111/32      Gi0/0/0/0    10.3.4.3        3225       
     Updated: Feb 17 11:46:25.703
     Version: 217, Priority: 3
     Label Stack (Top -> Bottom): { 24004 }
     NHID: 0x0, Encap-ID: N/A, Path idx: 0, Backup path idx: 0, Weight: 0
     MAC/Encaps: 14/18, MTU: 1500
     Outgoing Interface: GigabitEthernet0/0/0/0 (ifhandle 0x00000040)
     Packets Switched: 59
     Traffic-Matrix Packets/Bytes Switched: 0/0

RP/0/0/CPU0:XRV4#show cef 10.0.0.111/32 detail       
10.0.0.111/32, version 217, labeled SR, internal 0x1000001 0x81 (ptr 0xa12dc0ec) [1], 0x0 (0xa12c1638), 0xa28 (0xa1527348)
 Updated Feb 17 11:46:31.652
 local adjacency 10.3.4.3
 Prefix Len 32, traffic index 0, precedence n/a, priority 3
 Extensions: context-label:16111
  gateway array (0xa12264f0) reference count 9, flags 0x68, source lsd (5), 1 backups
                [4 type 5 flags 0x8401 (0xa154153c) ext 0x0 (0x0)]
  LW-LDI[type=5, refc=3, ptr=0xa12c1638, sh-ldi=0xa154153c]
  gateway array update type-time 1 Feb 17 11:46:25.702
 LDI Update time Feb 17 11:46:25.702
 LW-LDI-TS Feb 17 11:46:25.702
   via 10.3.4.3/32, GigabitEthernet0/0/0/0, 11 dependencies, weight 0, class 0 [flags 0x0]
    path-idx 0 NHID 0x0 [0xa168816c 0x0]
    next hop 10.3.4.3/32
    local adjacency
     local label 24005      labels imposed {24004}

    Load distribution: 0 (refcount 4)

    Hash  OK  Interface                 Address
    0     Y   GigabitEthernet0/0/0/0    10.3.4.3      

In the above scenario "%ROUTING-RIB-3-LABEL_ERR_ADD: Add local-label" occurrence can be avoided by configuring Segment Routing for BGP (SR-BGP) and make use of BGP Prefix-SID.

Please refer to Segment Routing Configuration Guide for Cisco ASR 9000 Series Routers for detail information on SR-BGP. Quick reference of required steps follows.

On all nodes configure a global Segment Routing Global Block (SRGB) which will automatically enable SR-BGP:

!
segment-routing
 global-block 16000 23999
!

Upon configuration then BGP will use the global SRGB for label allocation as shown from 'show mpls label table detail' command output:

!
! Note: If SR BGP was enabled after configuring BGP then you may need to process restart BGP for global SRGB to be picked up.
!
RP/0/0/CPU0:XRV111#show mpls label table detail     
Table Label   Owner                           State  Rewrite
----- ------- ------------------------------- ------ -------
<snip>
0     16000   ISIS(A):core                    InUse  No
              BGP-VPNv4(A):bgp-default        InUse  No
  (Lbl-blk SRGB, vers:0, (start_label=16000, size=8000)
<snip>

At the BGP node originating the route configure the BGP prefix-SID via a route-policy. A configuration example for the node advertising prefix 10.0.0.111/32 is shown:

!
route-policy SID($SID)
  set label-index $SID
end-policy
!
router bgp 65000
 address-family ipv4 unicast
  network 10.0.0.111/32 route-policy SID(111)
  allocate-label all
!

With SR-BGP in place then "ROUTING-RIB-3-LABEL_ERR_ADD: Add local-label" condition will no longer occur. As shown below BGP will use the BGP-Prefix SID index received with prefix advertisement as a hint to allocate the local label from global SRGB.

RP/0/0/CPU0:XRV4#show route 10.0.0.111/32 detail private
Routing entry for 10.0.0.111/32
  Known via "isis core", distance 115, metric 30, labeled SR, type level-2
  Installed Feb 17 14:48:26.512 for 02:59:18
  Routing Descriptor Blocks
    10.3.4.3, from 10.0.0.111, via GigabitEthernet0/0/0/0
      Route metric is 30
      Label: 0x3eef (16111)
      Tunnel ID: None
      Binding Label: None
      Extended communities count: 0
      Path id:1       Path ref count:0
      NHID:0x2(Ref:8)
      Path flags: 0x0 ()
      Private flags: 0x42 (has_vpn_data, rib_encap_id)
  Route version is 0xd (13)
  Local Label: 0x3eef (16111)
  IP Precedence: Not Set
  QoS Group ID: Not Set
  Flow-tag: Not Set
  Fwd-class: Not Set
  Route Priority: RIB_PRIORITY_NON_RECURSIVE_MEDIUM (7) SVD Type RIB_SVD_TYPE_LOCAL
  Download Priority 1, Download Version 438
  Client-id: 19
  Route flags: 0x9800000 (ldp_sr_merge_request, rib_precedence_over_ldp, label_sr)
  Route Extended flags: 0x48 (ldp_sr_merge_request, rib_precedence_over_ldp)
  Route private flags: 0x0 ()
  Route head pointer: 0x1201f32c
  Local Label List
    i 16111 (Client ID: 19, Distance: 115,)
    B 16111 (Client ID: 18, Distance: 200,)
  No advertising protos.

Conclusion

The message is triggered when two distinct protocols added a different local-label for the same prefix in the RIB database and the important aspect to retain is that this label conflict error condition must be avoided and its origin must be understood and corrected.

This behaviour can be avoided with the use of SR-BGP and BGP Prefix-SID.

For helpful and useful outputs to proceed with triage and understand the message "%ROUTING-RIB-3-LABEL_ERR_ADD: Add local-label" occurrence please refer to the commands list shown:

show rib clients
show rib clients redistribution history all
show route <prefix> detail private
show isis ipv4 route <prefix> detail       
show bgp ipv4 labeled-unicast <prefix> detail  
show bgp ipv4 labeled-unicast labels     
show cef <> detail       
show mpls label table label <prefix> detail private
show mpls label table label <prefix> history
show mpls forwarding labels <prefix> detail private
show mpls forwarding prefix <prefix> detail private
show mpls lsd forwarding labels <prefix> detail
show mpls ldp forwarding detail
show isis segment-routing label table
show isis database verbose detail internal