Contents

Introduction

Background Information

Topology Diagram

Considerations for Cisco SD-WAN Multicast Deployment

Observations

Possible Workarounds

Conclusion

Introduction

This document describes why the secondary branch router fails to learn Auto-RP (Rendezvous Point) mapping and provides steps to resolve the issue.

Background Information

In a dual border router setup at a SD-WAN branch site, primary SD-WAN router can learn the Auto-RP mapping messages while the secondary SD-WAN router does not. If the SD-WAN router elected as Protocol Independent Multicast (PIM) Assert Forwarder did not learn the Auto-RP mappings, the downstream switches at the branch will also not receive these mappings.

Topology Diagram

1. In this topology, at branch site SD-WAN routers Branch-C8300-01 and Branch-C8300-02 have Open Shortest Path First (OSPF) neighborship with switch Branch-9k on service VRF/VPN 10. Branch router Branch-C8300-01 is the primary router with OSPF cost of 10, whereas on secondary router Branch-C8300-02, the OSPF cost is 15.

2. Switch DC-9k at datacentre site is the RP. Here are the configurations on this switch:

ip pim rp-address 169.1.224.1 override

ip pim autorp listener

ip pim send-rp-announce Loopback1 scope 30 group-list RP-Groups

ip pim send-rp-discovery Loopback0 scope 30

ip pim ssm range PIM-SSM-Range

dc1-9k-01#sh ip access-lists RP-Groups

Standard IP access list RP-Groups

30 permit 239.1.0.0, wildcard bits 0.0.255.255

Considerations for Cisco SD-WAN Multicast Deployment

- You can configure multicast Rendezvous Point and replicator node on Datacentre-site devices only. Replicator cannot be configured on branch-site devices.

- Mapping Agent and Candidate RP can be deployed on Datacentre LAN side.

- Cisco Auto-RP cannot co-exist with PIM BSR. Cisco Auto-RP mode must be disabled with spt-only mode.

- If you have two Cisco IOS XE Catalyst SD-WAN devices in the same site, every  Cisco IOS XE Catalyst SD-WAN device needs to be configured as a replicator for traffic to flow.

- Loopback is one of many interface types that can be used for configuring an RP candidate.

Observations

- When tracing Multicast flows, you must start from receiver end (last hop router) and move towards source end (first hop router). Here Branch-9k is the Last-Hop Router (LHR).

- When checked on the branch side, the switch was not receiving the AutoRP mapping.

Branch-9k#sh ip pim rp mapping

PIM Group-to-RP Mappings

Branch-9k#

PIM configurations on the switch:

Branch-9k#sh run | in pim

ip pim sparse-mode

ip pim sparse-mode

ip pim sparse-mode

ip pim autorp listener

ip pim ssm range PIM-SSM-Range

Branch-9k#

- When the Multicast Forwarding Information Base (MFIB) was checked on the branch routers, it was noticed that on the branch Router 01, the RP interface was pruned. AutoRP uses the group 224.0.1.40 to advertise the RP information. There was no F flag set for RP interface in the output of show ip pim mfib of branch router 01. The F flag was set on branch router 02.

Branch-C8300-01#sh ip mfib vrf 10 224.0.1.40

Entry Flags:    C - Directly Connected, S - Signal, IA - Inherit A flag,

ET - Data Rate Exceeds Threshold, K - Keepalive

DDE - Data Driven Event, HW - Hardware Installed

ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB

MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary

MS  - MoFRR  Entry in Sync, MC - MoFRR entry in MoFRR Client,

e   - Encap helper tunnel flag.

I/O Item Flags: IC - Internal Copy, NP - Not platform switched,

NS - Negate Signalling, SP - Signal Present,

A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,

MA - MFIB Accept, A2 - Accept backup,

RA2 - MRIB Accept backup, MA2 - MFIB Accept backup

Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second

Other counts:      Total/RPF failed/Other drops

I/O Item Counts:   HW Pkt Count/FS Pkt Count/PS Pkt Count   Egress Rate in pps

VRF 10

(*,224.0.1.40) Flags: C HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 1741/1741/0

Lspvif0, LSM/0, RPF-ID: *, Flags: F NS

Pkts: 0/0/0    Rate: 0 pps

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.10.3,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 29642/29642/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.1.10.4,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 29939/29939/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A NS

GigabitEthernet0/0/0.100 Flags: IC

(169.1.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47783/47783/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.1.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47720/47720/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.1.20.10,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47784/47784/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.1.20.14,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47724/47724/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.1.100.7,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 60088/60088/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.2.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47680/47680/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

(169.2.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 47640/47640/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: IC

Branch-C8300-01#

Branch-C8300-02#sh ip mfib vrf 10 224.0.1.40

Entry Flags:    C - Directly Connected, S - Signal, IA - Inherit A flag,

ET - Data Rate Exceeds Threshold, K - Keepalive

DDE - Data Driven Event, HW - Hardware Installed

ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB

MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary

MS  - MoFRR  Entry in Sync, MC - MoFRR entry in MoFRR Client,

e   - Encap helper tunnel flag.

I/O Item Flags: IC - Internal Copy, NP - Not platform switched,

NS - Negate Signalling, SP - Signal Present,

A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,

MA - MFIB Accept, A2 - Accept backup,

RA2 - MRIB Accept backup, MA2 - MFIB Accept backup

Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second

Other counts:      Total/RPF failed/Other drops

I/O Item Counts:   HW Pkt Count/FS Pkt Count/PS Pkt Count   Egress Rate in pps

VRF 10

(*,224.0.1.40) Flags: C HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 10549/10549/0

Lspvif0, LSM/0, RPF-ID: *, Flags: F NS

Pkts: 0/0/0    Rate: 0 pps

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.10.3,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A NS

GigabitEthernet0/0/0.100 Flags: F IC NS        <==

Pkts: 0/0/0    Rate: 0 pps

(169.1.10.4,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A NS

GigabitEthernet0/0/0.100 Flags: F IC NS        <==

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.10,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A NS

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.14,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A NS

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.2.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

(169.2.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

Lspvif0, LSM/0, RPF-ID: *, Flags: A

GigabitEthernet0/0/0.100 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

Branch-C8300-02#

The F flag being set indicates the router Branch-C8300-02 is the Designated Forwarder for AutoRP and for the multicast traffic. When the PIM neighbours are in the same broadcast domain, the PIM Assert forwarder/Designated Forwarder is elected. The router with highest IP address is chosen as PIM Assert forwarder/Designated Forwarder. (The Active Directory (AD) and Metric were same.) In this scenario, Branch router 2 has higher IP address compared to Branch router 1:

Branch-C8300-01#sh run interface Gi0/0/0.100

Building configuration...

Current configuration : 336 bytes

!

interface GigabitEthernet0/0/0.100

description OSPF peering interface1

encapsulation dot1Q 100

vrf forwarding 10

ip address 169.101.10.1 255.255.255.240

no ip redirects

ip pim sparse-mode

ip nbar protocol-discovery

ip ospf network broadcast

ip ospf dead-interval 40

ip ospf 10 area 0

ip ospf cost 10

arp timeout 1200

end

Branch-C8300-02#sh run interface Gi0/0/0.100

Building configuration...

Current configuration : 336 bytes

!

interface GigabitEthernet0/0/0.100

description OSPF peering interface1

encapsulation dot1Q 100

vrf forwarding 10

ip address 169.101.10.2 255.255.255.240

no ip redirects

ip pim sparse-mode

ip nbar protocol-discovery

ip ospf network broadcast

ip ospf dead-interval 40

ip ospf 10 area 0

ip ospf cost 15

arp timeout 1200

end

- It was noticed that the Branch router 01 was receiving the AutoRP messages, but Branch router 02 was not receiving the AutoRP messages:

Branch-C8300-01#sh ip pim vrf 10 rp mapping

PIM Group-to-RP Mappings

Group(s) 239.195.0.0/16

RP 10.125.125.1 (?), v2v1

Info source: 169.254.100.9 (?), elected via Auto-RP

Uptime: 1w0d, expires: 00:02:31

Branch-C8300-01#

Branch-C8300-02#sh ip pim vrf 10 rp mapping

PIM Group-to-RP Mappings

Branch-C8300-02#

- Since the Branch router 01 is not Assert winner, the S,G is pruned, hence it cannot forward the AutoRP mapping to the downstream switch, and on the Branch router 02, even though it is Designated Forwarder, it has not learned the AutoRP mapping message, and it cannot forward the AutoRP to downstream switch. Now why the Auto RP message was not learnt by the Branch router 02 is the question.

- The Branch routers learn AutoRP mapping from the DC Border routers. DC SD-WAN Routers act as replicators and mapping agents. Mapping agent decides the candidate RP for a given Multicast group.

- On the DC SD-WAN Router, the MFIB was checked and it was noticed that there is a Forwarding flag set for S,G entry:

DC-8500-01#sh ip mfib vrf 10 224.0.1.40

Entry Flags:    C - Directly Connected, S - Signal, IA - Inherit A flag,

ET - Data Rate Exceeds Threshold, K - Keepalive

DDE - Data Driven Event, HW - Hardware Installed

ME - MoFRR ECMP entry, MNE - MoFRR Non-ECMP entry, MP - MFIB

MoFRR Primary, RP - MRIB MoFRR Primary, P - MoFRR Primary

MS  - MoFRR  Entry in Sync, MC - MoFRR entry in MoFRR Client,

e   - Encap helper tunnel flag.

I/O Item Flags: IC - Internal Copy, NP - Not platform switched,

NS - Negate Signalling, SP - Signal Present,

A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,

MA - MFIB Accept, A2 - Accept backup,

RA2 - MRIB Accept backup, MA2 - MFIB Accept backup

Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second

Other counts:      Total/RPF failed/Other drops

I/O Item Counts:   HW Pkt Count/FS Pkt Count/PS Pkt Count   Egress Rate in pps

VRF 10

(*,224.0.1.40) Flags: C HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 294/294/0

TenGigabitEthernet0/0/0.10 Flags: F IC NS

Pkts: 0/0/0    Rate: 0 pps

Lspvif0, LSM/1, RPF-ID: *, Flags: F NS

Pkts: 0/0/0    Rate: 0 pps

(169.1.10.4,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.10,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.1.20.14,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.1.100.7,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F          <==

Pkts: 0/0/0    Rate: 0 pps

(169.2.20.2,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

(169.2.20.6,224.0.1.40) Flags: HW

SW Forwarding: 0/0/0/0, Other: 0/0/0

HW Forwarding:   0/0/0/0, Other: 0/0/0

TenGigabitEthernet0/0/0.10 Flags: A IC

Lspvif0, LSM/1, RPF-ID: *, Flags: F

Pkts: 0/0/0    Rate: 0 pps

Lspvif0 is the virtual tunnel interface like Point-to-Multipoint (P2MP) tunnel, that is, one endpoint to multiple remote endpoints. Lspvif0 tunnel is used in SD-WAN Multicast. Lspvif0 is the virtual tunnel indicating the packets are transmitted through SD-WAN overlay.

DC-8500-01#sh interfaces Lspvif0

Lspvif0 is up, line protocol is up

Hardware is

Interface is unnumbered. Using address of SD-WAN-system-intf (169.1.100.3)

MTU 17892 bytes, BW 10000000 Kbit/sec, DLY 5000 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation LOOPBACK, loopback not set

Keepalive set (10 sec)

Last input never, output 00:00:10, output hang never

Last clearing of "show interface" counters never

Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/0 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

3 packets input, 210 bytes, 0 no buffer

Received 0 broadcasts (3 IP multicasts)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort

1628856 packets output, 71967520 bytes, 0 underruns

Output 0 broadcasts (1289488 IP multicasts)

0 output errors, 0 collisions, 0 interface resets

0 unknown protocol drops

0 output buffer failures, 0 output buffers swapped out

LSM/1 in the MFIB output indicates the Replication list attached to Lspvif0 tunnel.

As per the design of Cisco SD-WAN Multicast, DC router learns the PIM enabled Branch routers through Overlay Management Protocol (OMP) and the system-IP of these Branch routers are added to replication list.

DC-8500-01#sh SD-WAN omp multicast-auto-discover

Code:

C   -> chosen

I   -> installed

Red -> redistributed

Rej -> rejected

L   -> looped

R   -> resolved

S   -> stale

Ext -> extranet

Stg -> staged

IA  -> On-demand inactive

Inv -> invalid

BR-R -> border-router reoriginated

TGW-R -> transport-gateway reoriginated

ADDRESS                 SOURCE         FROM

FAMILY   TENANT    VPN  ORIGINATOR     PEER     STATUS

---------------------------------------------------------

ipv4     0         10   169.1.100.3    0.0.0.0  C,Red,R

0         10   169.1.100.4    2.2.2.1  Inv,U

2.2.2.2  Inv,U

0         10   169.101.100.1  2.2.2.1  C,R

2.2.2.2  C,I,R

0         10   169.101.100.2  2.2.2.1  C,R

2.2.2.2  C,I,R

Note: The AutoRP packets from the DC router will be sent to the Branch routers through the SD-WAN BFD tunnel (Dataplane). The DC router will encapsulate AutoRP packets to IPsec tunnel and forward it to Branch routers.

- When you see Forwarding flag set in the MFIB of DC Router, it indicates the replication list is built for the system-ips of the Branch routers and will forward the Multicast traffic based on the replication list. In the replication list, you are able to see entries for both the branch routers. Here is the command output of the replication list:

DC-8500-01#sh mvpn replication lsm-id 1

Repl ID : 1FFFFF   LSM ID : 1   Uptime : 1w3d

Path Set ID         : 25

Replication branches: 2

IR (169.101.100.1)

Uptime         : 1w3d          Refcount  : 2

Remote Label   : 1006

IR (169.101.100.2)

Uptime         : 1w3d          Refcount  : 2

Remote Label   : 1004

- Once the system-ips are present in the replication list, check whether indirect Operational Cloud Environment (OCE) chain/forwarding chain is established to the given branch router. As you see in the next output, the indirect OCE is available only for Branch Router 01.

DC-8500-01#sh platform software SD-WAN f0 next-hop indirect all

Show SD-WAN next-hop oce all :

OCE ID: 0xf8000d9f, OCE Type: SD-WAN_NH_INDIRECT

Indirect: client_handle 0x5649f38aaa80, ppe addr 418b02c0

nhobj_type: SD-WAN_NH_LOCAL_SLA_CLASS, nhobj_handle: 0xf80805cf

label: 1006, dst_vpn: 10, nexthop sys_ip: 169.101.100.1, sla_class: 1

The indirect OCE chain is the internal forwarding chain built when the DC router learns the unicast routes from the respective branch router. This is as per design of SD-WAN Multicast, where Multicast will leverage unicast routing to forward Multicast RP information.

The reason for DC router not forwarding the AutoRP mapping to branch router 02 is because the Indirect OCE is built only for Branch Router 01, but not for Branch Router 02. Only when the internal forwarding chain is established to the respective branch router, the DC router will forward the AutoRP mapping to that branch router.

- Since the DC router did not have any unicast routes in Routing Information Base (RIB) learnt from branch router 02, the forwarding chain is not built to branch router 02.

DC-8500-01#sh ip route vrf 10 omp

Routing Table: 10

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP

n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

H - NHRP, G - NHRP registered, g - NHRP registration summary

o - ODR, P - periodic downloaded static route, l - LISP

a - application route

+ - replicated route, % - next hop override, p - overrides from PfR

& - replicated local route overrides by connected

Gateway of last resort is 169.1.10.1 to network 0.0.0.0

m     169.101.0.0/16 [251/0] via 169.101.100.1, 2w0d, SD-WAN-system-intf

DC-8500-01#

- As you see in the next OMP routes output, the route learnt from Branch Router 02 is not installed to VRF 10 RIB of DC router.

DC-8500-01#sh SD-WAN omp routes 169.101.0.0/16

Code:

C   -> chosen

I   -> installed

Red -> redistributed

Rej -> rejected

L   -> looped

R   -> resolved

S   -> stale

Ext -> extranet

Inv -> invalid

Stg -> staged

IA  -> On-demand inactive

U   -> TLOC unresolved

BR-R -> border-router reoriginated

TGW-R -> transport-gateway reoriginated

AFFINITY

PATH                      ATTRIBUTE                                                       GROUP

TENANT    VPN    PREFIX              FROM PEER        ID     LABEL    STATUS    TYPE       TLOC IP          COLOR            ENCAP  PREFERENCE  NUMBER      REGION ID   REGION PATH

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

0         10     169.101.0.0/16      2.2.2.1          15     1004     R         installed  169.101.100.2    public-internet  ipsec  -           None        None        -

2.2.2.1          19     1006     C,I,R     installed  169.101.100.1    biz-internet     ipsec  -           None        None        -

2.2.2.2          16     1004     R         installed  169.101.100.2    public-internet  ipsec  -           None        None        -

2.2.2.2          21     1006     C,R       installed  169.101.100.1    biz-internet     ipsec  -           None        None        -

DC-8500-01#

The reason for routes learnt from secondary router is not installed to RIB is because routes learnt from secondary router have higher OSPF cost compared to primary router:

DC-8500-01#sh SD-WAN omp routes 169.101.0.0/16 detail

--------------------------------------------------------------------------

omp route entries for tenant-id 0 vpn 10 route 169.101.0.0/16

--------------------------------------------------------------------------

RECEIVED FROM:

peer            2.2.2.1

path-id         15

label           1004

status          R

loss-reason     origin-metric

lost-to-peer    2.2.2.1

lost-to-path-id 19

Attributes:

originator       169.101.100.2

type             installed

tloc             169.101.100.2, public-internet, ipsec

ultimate-tloc    not set

domain-id        not set

overlay-id        1

site-id          10

preference       not set

affinity-group  None

region-id        None

region-path      not set

route-reoriginator      not set

tag              not set

origin-proto     OSPF-external-1

origin-metric    35

as-path          not set

community        not set

unknown-attr-len not set

RECEIVED FROM:

peer            2.2.2.1

path-id         19

label           1006

status          C,I,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

Attributes:

originator       169.101.100.1

type             installed

tloc             169.101.100.1, biz-internet, ipsec

ultimate-tloc    not set

domain-id        not set

overlay-id        1

site-id          10

preference       not set

affinity-group  None

region-id        None

region-path      not set

route-reoriginator      not set

tag              not set

origin-proto     OSPF-external-1

origin-metric    30

as-path          not set

community        not set

unknown-attr-len not set

RECEIVED FROM:

peer            2.2.2.2

path-id         16

label           1004

status          R

loss-reason     origin-metric

lost-to-peer    2.2.2.2

lost-to-path-id 21

Attributes:

originator       169.101.100.2

type             installed

tloc             169.101.100.2, public-internet, ipsec

ultimate-tloc    not set

domain-id        not set

overlay-id        1

site-id          10

preference       not set

affinity-group  None

region-id        None

region-path      not set

route-reoriginator      not set

tag              not set

origin-proto     OSPF-external-1

origin-metric    35

as-path          not set

community        not set

unknown-attr-len not set

RECEIVED FROM:

peer            2.2.2.2

path-id         21

label           1006

status          C,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

Attributes:

originator       169.101.100.1

type             installed

tloc             169.101.100.1, biz-internet, ipsec

ultimate-tloc    not set

domain-id        not set

overlay-id        1

site-id          10

preference       not set

affinity-group  None

region-id        None

region-path      not set

route-reoriginator      not set

tag              not set

origin-proto     OSPF-external-1

origin-metric    30

as-path          not set

community        not set

unknown-attr-len not set

DC-8500-01#

- Also, only OSPF routes were distributed to OMP on branch routers. Here is the OMP configurations from secondary branch router:

omp

no shutdown

overlay-as       65376

send-path-limit  16

ecmp-limit       16

graceful-restart

no as-dot-notation

timers

holdtime               60

advertisement-interval 1

graceful-restart-timer 43200

eor-timer              300

exit

address-family ipv4 vrf 10

advertise ospf external              <==

!

address-family ipv6

advertise connected

advertise static

Possible Workarounds

1. You can configure a Loopback interface on branch router 02 and advertise the prefix through OMP to DC router.

Branch-C8300-02#sh run interface Lo0

Building configuration...

Current configuration : 151 bytes

!

interface Loopback0

description Management loopback

vrf forwarding 10

ip address 169.101.100.2 255.255.255.255

no ip redirects

ip mtu 1500

end

OMP configurations on branch router 02 to advertise connected routes:

omp

no shutdown

overlay-as       65376

send-path-limit  16

ecmp-limit       16

graceful-restart

no as-dot-notation

timers

holdtime               60

advertisement-interval 1

graceful-restart-timer 43200

eor-timer              300

exit

address-family ipv4 vrf 10

advertise ospf external

advertise connected                  <==

!

address-family ipv6

advertise connected

advertise static

!

!

Now you see AutoRP mapping learnt on branch router 02:

Branch-C8300-02# sh ip pim vrf 10 rp mapping

PIM Group-to-RP Mappings

Group(s) 239.195.0.0/16

RP 10.125.125.1 (?), v2v1

Info source: 169.1.10.4 (terin.net.afrihost.co.za), elected via Auto-RP

Uptime: 00:02:18, expires: 00:02:47

Branch-C8300-02#

Similarly, you can see AutoRP mapping learnt on the branch switch as well:

Branch-9k#sh ip pim rp mapping

PIM Group-to-RP Mappings

Group(s) 239.195.0.0/16

RP 10.125.125.1 (?), v2v1

Info source: 169.254.100.9 (?), elected via Auto-RP

Uptime: 00:03:36, expires: 00:02:46

Acl: RP-Region-Ent-Sites, Static-Override

RP: 10.125.125.1 (?)

When you inspect the Indirect OCE/Forwarding chain output on the DC router, it has entries for system-ip of both primary and secondary branch routers:

DC-8500-01#sh platform software SD-WAN f0 next-hop indirect all

Show SD-WAN next-hop oce all :

OCE ID: 0xf80009bf, OCE Type: SD-WAN_NH_INDIRECT

Indirect: client_handle 0x5649f389fbc0, ppe addr 418b05c0

nhobj_type: SD-WAN_NH_LOCAL_SLA_CLASS, nhobj_handle: 0xf808044f

label: 1006, dst_vpn: 10, nexthop sys_ip: 169.101.100.1, sla_class: 1

OCE ID: 0xf80009df, OCE Type: SD-WAN_NH_INDIRECT

Indirect: client_handle 0x5649f38a11f0, ppe addr 418b06d0

nhobj_type: SD-WAN_NH_LOCAL_SLA_CLASS, nhobj_handle: 0xf808045f

label: 1004, dst_vpn: 10, nexthop sys_ip: 169.101.100.2, sla_class: 1

Now you see DC router RIB has prefixes learnt from branch router 02 also installed.

DC-8500-01#sh ip route vrf 10 omp

Routing Table: 10

Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2, m - OMP

n - NAT, Ni - NAT inside, No - NAT outside, Nd - NAT DIA

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

H - NHRP, G - NHRP registered, g - NHRP registration summary

o - ODR, P - periodic downloaded static route, l - LISP

a - application route

+ - replicated route, % - next hop override, p - overrides from PfR

& - replicated local route overrides by connected

Gateway of last resort is 169.1.10.1 to network 0.0.0.0

169.101.0.0/16 is variably subnetted, 3 subnets, 3 masks

m        169.101.0.0/16 [251/0] via 169.101.100.1, 13:59:47, SD-WAN-system-intf

m        169.101.10.0/28

[251/0] via 169.101.100.2, 00:07:50, SD-WAN-system-intf

m        169.101.100.2/32

[251/0] via 169.101.100.2, 00:07:50, SD-WAN-system-intf

2. On the branch routers interface connecting to the LAN switch, configure higher IP address on the primary branch router compared to that of secondary router. With this primary branch router will become Designated forwarder, thereby it will be able to forward the AutoRP information to the switch as primary branch router had learnt AutoRP information.

Conclusion

While implementing Multicast over Cisco SD-WAN, you must ensure, all the remote routers (including primary and secondary routers) are advertising unicast prefix to the SD-WAN router closer to RP through OMP. The SD-WAN Multicast leverages unicast routing to build forwarding chain which is required for Multicast control plane information to be transmitted.