Understand Next Hop Set in iBGP Advertisements on Nexus NX-OS vs Cisco IOS

Available Languages

Updated:June 12, 2018
Document ID:213402

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.


    Introduction

    This document describes the behavior of the NEXT_HOP path attribute when set for Interior Border Gateway Protocol (iBGP) advertisements on Nexus NX-OS vs Cisco IOS (this includes Cisco IOS-XE) based platforms. This is for advertisements of non locally originated routes.

    Prerequisites

    Requirements

    Cisco recommends that you have knowledge of these topics:

    • Border Gateway Protocol (BGP)
    • Routing Protocols Redistribution

    Components Used

    This document is not restricted to specific software and hardware versions:

    • Nexus 7000 that runs NX-OS version 7.3(0)D1(1)
    • Cisco Router that runs Cisco IOS version 15.6(2)T

    The outputs in this document were taken from devices in a specific lab environment. All devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.


    Background Information

    • On Nexus NX-OS based platforms, for non locally originated routes, iBGP advertisements modifies the NEXT_HOP attribute and sets it with its own local interface IP address.
    • On Cisco IOS based platforms, for non locally originated routes, iBGP advertisements maintains the NEXT_HOP attribute of the original route as it is.

    The behavior on Nexus NX-OS can match the one seen on Cisco IOS if desired thanks to the code changes introduced by defect CSCud20941.

    Note: This is only applicable for iBGP advertisements and not eBGP.


    Note: Applicable for non locally originated routes configured as Static Routes or received via any Interior Gateway Protocol (IGP) like Enhanced Interior Gateway Routing Protocol (EIGRP), Open Shortest Path First (OSPF) or Routing Information Protocol (RIP).


    Understand iBGP Advertisements

    In order to understand the NEXT_HOP set in iBGP advertisements, take as an example the network topology diagrams shown in the images.

    Topology for Nexus NX-OS case


     Topology for Cisco IOS case


    Nexus NX-OS Case

    In Topology for Nexus NX-OS case, R2 (Nexus NX-OS) receives the 1.1.1.1/32 route via EIGRP from Router 1 and advertises it with the use of iBGP to Router 3 as shown in the image.


    R2 (Nexus NX-OS) Routing Table shows the route 1.1.1.1/32 received via EIGRP and with the original next-hop IP of 10.1.2.1

    R2 (Nexus NX-OS) 
    R2# show ip route 1.1.1.1/32
    IP Route Table for VRF "default"
    '*' denotes best ucast next-hop
    '**' denotes best mcast next-hop
    '[x/y]' denotes [preference/metric]
    '%<string>' in via output denotes VRF <string>

    1.1.1.1/32, ubest/mbest: 1/0
        *via 10.1.2.1, Eth2/1, [90/130816], 00:02:28, eigrp-1, internal


    In BGP configuration section you can see the commands in place to advertise 1.1.1.1/32 via iBGP to Router 3.

    R2 (Nexus NX-OS) 
    R2# show running-config bgp

!Command: show running-config bgp
!Time: -

version -
feature bgp

router bgp 2
  address-family ipv4 unicast
    network 1.1.1.1/32
  neighbor 10.2.3.3 remote-as 2
    address-family ipv4 unicast


    On Router 3, the 1.1.1.1/32 route is received via iBGP with the next-hop now set to the IP address of R2 (Nexus NX-OS) which is 10.2.3.2

    - Router 3 BGP table entry for 1.1.1.1/32

    R3 
    R3# show bgp ipv4 unicast 1.1.1.1/32
BGP routing table entry for 1.1.1.1/32, version 8
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 1
  Local
    10.2.3.2 from 10.2.3.2 (2.2.2.2)
      Origin IGP, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0


    - Router 3 Routing Table entry for 1.1.1.1/32

    R3 
    R3# show ip route bgp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      1.0.0.0/32 is subnetted, 1 subnets
B        1.1.1.1 [200/0] via 10.2.3.2, 00:07:17


    Cisco IOS Case

    In Topology for Cisco IOS case, R2 (Cisco IOS) receives the 1.1.1.1/32 route via EIGRP from Router 1 and advertises it with the use of iBGP to Router 3 as shown in the image.

    R2 (Cisco IOS) Routing Table shows the route 1.1.1.1/32 received via EIGRP and with the original next-hop IP of 10.1.2.1

    R2 (Cisco IOS) 
    R2# show ip route 1.1.1.1 255.255.255.255 longer-prefixes
    Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
           D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
           N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
           E1 - OSPF external type 1, E2 - OSPF external type 2
           i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
           ia - IS-IS inter area, * - candidate default, U - per-user static route
           o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
           a - application route
           + - replicated route, % - next hop override, p - overrides from PfR

    Gateway of last resort is not set

          1.0.0.0/32 is subnetted, 1 subnets
    D        1.1.1.1 [90/130816] via 10.1.2.1, 00:00:06, GigabitEthernet0/1


    In BGP configuration section you can see the commands in place to advertise 1.1.1.1/32 via iBGP to Router 3

    R2 (Cisco IOS) 
    R2# show running-config partition router bgp 2
    Building configuration...

    Current configuration : 210 bytes
    !
    ! Last configuration change at -
    !
    !
    !
    !
    router bgp 2
     bgp router-id 2.2.2.2
     bgp log-neighbor-changes
    network 1.1.1.1 mask 255.255.255.255
     neighbor 10.2.3.3 remote-as 2
    !
    !
    end


    On Router 3, you can see the 1.1.1.1/32 route received via iBGP with the original next-hop set to the IP on Router 1 which is 10.1.2.1.

    - Router 3 BGP table entry for 1.1.1.1/32

    R3 
    R3# show bgp ipv4 unicast 1.1.1.1/32
BGP routing table entry for 1.1.1.1/32, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  Refresh Epoch 1
  Local
    10.1.2.1 (inaccessible) from 10.2.3.2 (2.2.2.2)
      Origin IGP, metric 130816, localpref 100, valid, internal
      rx pathid: 0, tx pathid: 0


    In this specific scenario, Router 3 must have a path to 10.1.2.1 (the Next-Hop) so BGP can consider the path as valid. Otherwise, BGP shows the path as (inaccessible).


    Note: This is a basic check described in BGP Best Path Selection Algorithm in order to accept routes from BGP into the Routing Table.


    The command debug ip bgp update shows the reason Router 3 does not install the route is because there is no entry in its Routing Table for the next-hop, in this case the next-hop is 10.1.2.1

    R3 
    R3# debug ip bgp update
*-: BGP(0): 10.2.3.2 rcvd UPDATE w/ attr: nexthop 10.1.2.1, origin i, localpref 100, metric 130816
*-: BGP(0): 10.2.3.2 rcvd 1.1.1.1/32
*-: BGP(0): no valid path for 1.1.1.1/32


    One approach to make the next-hop accessible is:

    - Step 1. One static route in the Routing Table of Router 3 is configured in order to create an entry for the next-hop.

    R3 
    R3# configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
R3(config)# ip route 10.1.2.1 255.255.255.255 10.2.3.2


    - Step 2. The same debug command shows that the route is now accepted.

    R3 
    R3# debug ip bgp update
R3#
*Mar 29 16:08:42.888: BGP(0): 10.2.3.2 rcvd UPDATE w/ attr: nexthop 10.1.2.1, origin i, localpref 100, metric 130816
*Mar 29 16:08:42.890: BGP(0): 10.2.3.2 rcvd 1.1.1.1/32
*Mar 29 16:08:42.892: BGP(0): Revise route installing 1 of 1 routes for 1.1.1.1/32 -> 10.1.2.1(global) to main IP table
R3#


    - Step 3. The BGP table has removed the (inaccessible) status.

    R3 
    R3# show bgp ipv4 unicast 1.1.1.1/32
BGP routing table entry for 1.1.1.1/32, version 6
Paths: (1 available, best #1, table default)
  Not advertised to any peer
  Refresh Epoch 2
  Local
    10.1.2.1 from 10.2.3.2 (2.2.2.2)
      Origin IGP, metric 130816, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0


    - Step 4. Routing Table now installs the route to 1.1.1.1/32

    R3 
    R3# show ip route bgp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      1.0.0.0/32 is subnetted, 1 subnets
B        1.1.1.1 [200/130816] via 10.1.2.1, 00:11:37


    Use of set ip next-hop redist-unchanged Command

    Since version 6.2(12), the commands set ip next-hop redist-unchanged and set ipv6 next-hop redist-unchanged were introduced by defect CSCud20941 in order to make Nexus NX-OS mirror the behavior of Cisco IOS.

    Note: These commands only work when used as parameters in a route-map and are used in combination with the redistribution command.


    In Topology for Nexus NX-OS case, R2 (Nexus NX-OS) receives the 1.1.1.1/32 route via EIGRP from Router 1 and advertises it with iBGP to Router 3 as shown in the image:


    R2 (Nexus NX-OS) Routing Table shows the route 1.1.1.1/32 received via EIGRP and with the original next-hop IP of 10.1.2.1

    R2 (Nexus NX-OS) 
    R2# show ip route 1.1.1.1/32
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

1.1.1.1/32, ubest/mbest: 1/0
    *via 10.1.2.1, Eth2/1, [90/130816], 04:38:21, eigrp-1, internal


    The command set ip next-hop redist-unchanged is available under 'route-map' configuration mode.

    R2 (Nexus NX-OS) 
    R2(config)# route-map REDIST-UNCHANGED
    R2(config-route-map)# set ip next-hop ?
      A.B.C.D              IP address of next hop
      load-share           Enables load sharing
      peer-address         Use peer address (for BGP only)
      redist-unchanged     Use unchanged address during redistribution (for BGP
                           session only) 
      unchanged            Use unchanged address (for eBGP session only)
      verify-availability  Verify the reachability of the tracked object

    R2(config-route-map)# set ip next-hop redist-unchanged


    The route-map REDIST-UNCHANGED is applied as a parameter for the redistribute configuration statement in BGP.

    R2 (Nexus NX-OS) 
    R2#

!
    route-map REDIST-UNCHANGED permit 10
  set ip next-hop redist-unchanged
!    

R2# show running-config bgp

!Command: show running-config bgp
!Time: -

version -
feature bgp

router bgp 2
  address-family ipv4 unicast
    redistribute eigrp 1 route-map REDIST-UNCHANGED
  neighbor 10.2.3.3 remote-as 2
    address-family ipv4 unicast


    Now Router 3 receives the BGP UPDATE with the original NEXT_HOP set similar to Cisco IOS.

    R3 
    R3# show ip bgp 
BGP table version is 15, local router ID is 10.2.3.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
 * i 1.1.1.1/32       10.1.2.1            130816    100      0 ?


    This document describes the difference of how Nexus NX-OS and Cisco IOS handles iBGP advertisements of non locally generated routes.

    The behavior described in this document is for most case scenarios and it is not one that impacts usual network routing operations.

    The optional commands set ip next-hop redist-unchanged and set ipv6 next-hop redist-unchanged are available to maintain BGP routing compliant with RFC 4271 on Nexus NX-OS


    Initial Devices Configuration

    R1 
    hostname R1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
 ip ospf 1 area 0
!
interface GigabitEthernet0/1
 ip address 10.1.2.1 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
router ospf 1
    !


    R2 (Nexus NX-OS) 
    hostname R2
!
feature ospf
feature bgp
!
interface Ethernet2/1
  no switchport
  ip address 10.1.2.2/24
  ip ospf network point-to-point
  ip router ospf 1 area 0.0.0.0
  no shutdown
!
interface Ethernet2/2
  no switchport
  ip address 10.2.3.2/24
  no shutdown
!
router ospf 1
!
router bgp 2
  address-family ipv4 unicast
    network 1.1.1.1/32
  neighbor 10.2.3.3 remote-as 2
    address-family ipv4 unicast
!


    R2 (Cisco IOS) 
    hostname R2
!
interface GigabitEthernet0/1
 ip address 10.1.2.2 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
interface GigabitEthernet0/2
 ip address 10.2.3.2 255.255.255.0
!
router ospf 1
!
router bgp 2
 bgp log-neighbor-changes
 network 1.1.1.1 mask 255.255.255.255
 neighbor 10.2.3.3 remote-as 2
!


    R3 
    hostname R3
!
interface GigabitEthernet0/1
 ip address 10.2.3.3 255.255.255.0
!
router bgp 2
 bgp log-neighbor-changes
 neighbor 10.2.3.2 remote-as 2
!


    TAC Authored

    Contributed by Cisco Engineers

    • Hector Gustavo Serrano Gutierrez
      Cisco TAC Engineer

    Was this Document Helpful?

    FeedbackFeedback

    Contact Cisco