Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Network Diagram

Specification

Solution

Configure

Branch-1 Configuration

Branch-2 Configuration

DC-Router Configuration

vSmart Policy

Failover Scenarios

Branch-1 Traffic Flow Normal Scenario

Branch-2 Traffic Flow Normal Scenario

Failure Scenarios

Branch-1 Failure Scenario

Branch-2 Failure Scenario

Verify

Troubleshoot

Additional Information

Scenario-1

Scenario-2

Requirement (Service Side NAT (SS-NAT) with UTD Inspection)

Workaround

Introduction

This document describes the scenario with overlapping address spaces in the same VPN across multiple sites in the SD-WAN overlay. It depicts the sample network, traffic behavior in normal/failover scenarios, configuration, and verification.

Prerequisites

Requirements

Cisco recommends that you have knowledge of SD-WAN.

Components Used

The information in this document is based on these software and hardware versions:

- SD-WAN Controller Version 20.6.3

- Cisco IOS® XE (run in controller mode) 17.6.3a

- Host Devices (CSR1000V) 17.3.3

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Here you can find a list of acronyms used in this article.

- Secure Internet Gateway - SIG

- Virtual Routing and Forwarding - VRF

- Virtual Private Network - VPN

- Direct Internet Access - DIA

- Network Address Translation - NAT

- Multi-Protocol Label Switching - MPLS

- Service Side Network Address Translation - SS-NAT

- Data Center - DC

- Overlay Management Protocol - OMP

- Internet Protocol - IP

Refer to the Cisco document for more details on the service side NAT: Service-side NAT

Network Diagram

Note: In this topology, devices hosted in service VPN 10 of each branch router have overlapping IP 192.168.10.0/24 configured.

In this specific topology, there is 1 DC (DC has only MPLS transport, however in a real scenario there can be multiple transports) and 2 Branch locations that have connectivity to SD-WAN overlay over MPLS and Internet transport. Service VPN 10 is configured in all the locations. Branches have SIG tunnel (Primary and backup) configured to Zscaler. DIA is configured for certain specific destination IPs to bypass the Zscaler. In case of Internet link failure at branches, it is expected that all the traffic must be sent to DC via MPLS transport.

eBGP is configured on service VPN 10 with the Zscaler router at the DC end. DC router receives default route from Zscaler router and is redistributed into OMP.

Note: Public IP addresses mentioned in this lab scenario are taken from documentation RFC5737.

Specification

- Leverage overlapping IP addresses for Branch-1 and Branch-2 on the service side VPN 10.

- In a typical scenario, when MPLS and Internet transport are up, Traffic from VPN 10 must exit via SIG Tunnel.

- For specific IP destination prefixes, traffic must bypass the SIG Tunnel and exit via DIA.

- In case of Internet link failure, All/Internet-bound traffic from VPN 10 must exit via DC.

Solution

To achieve the requirement, SD-WAN features Service Side NAT and DIA with Data policy is used.

- Service Side NAT is configured on each branch router with different NAT pool IP addresses.

- In case of internet link failure when traffic is sent to SD-WAN overlay, the source IP is NATed to the IP address from the configured NAT pool.

- DC router sees the post-NAT address for overlapping subnets.

Note: To depict normal traffic via SIG Tunnel from VPN 10, Public IP 192.0.2.100 is used and for a specific destination, via DIA, 192.0.2.1 is used. Corresponding configurations are shown in the configuration section.

Configure

Branch-1 Configuration

Branch-1 Router configuration is as follows.

vrf definition 10

rd 1:10

!

address-family ipv4

route-target export 1:10

route-target import 1:10

exit-address-family

!

interface GigabitEthernet2

description "Internet TLOC"

ip address 198.51.100.1 255.255.255.0

ip nat outside

!

interface GigabitEthernet3

description "MPLS TLOC"

ip address 172.20.1.10 255.255.255.0

!

interface GigabitEthernet4

description "Service Side VPN 10"

vrf forwarding 10

ip address 192.168.10.1 255.255.255.0

!

interface Tunnel2

ip unnumbered GigabitEthernet2

tunnel source GigabitEthernet2

tunnel mode sdwan

!

interface Tunnel3

ip unnumbered GigabitEthernet3

tunnel source GigabitEthernet3

tunnel mode sdwan

!

interface Tunnel100512

ip address 10.10.1.1 255.255.255.252

tunnel source GigabitEthernet2

tunnel destination 203.0.113.1

tunnel vrf multiplexing

!

interface Tunnel100513

ip address 10.10.1.5 255.255.255.252

tunnel source GigabitEthernet2

tunnel destination 203.0.113.2

tunnel vrf multiplexing

!

ip sdwan route vrf 10 0.0.0.0/0 tunnel active Tunnel100512 backup Tunnel100513

ip nat pool natpool1 172.16.2.1 172.16.2.2 prefix-length 30

ip nat inside source list nat-dia-vpn-hop-access-list interface GigabitEthernet2 overload

ip nat inside source list global-list pool natpool1 vrf 10 match-in-vrf overload

ip nat route vrf 10 192.0.2.1 255.255.255.255 global

!

ip route 0.0.0.0 0.0.0.0 198.51.100.100

ip route 0.0.0.0 0.0.0.0 172.20.1.100

!

Branch-2 Configuration

Branch-2 Router configuration is as follows.

vrf definition 10

rd 1:10

!

address-family ipv4

route-target export 1:10

route-target import 1:10

exit-address-family

!

address-family ipv6

exit-address-family

!

interface GigabitEthernet2

description "Internet TLOC"

ip address 198.51.100.2 255.255.255.0

ip nat outside

!

!

interface GigabitEthernet3

description "MPLS TLOC"

ip address 172.20.1.20 255.255.255.0

!

interface GigabitEthernet4

description "Service Side VPN 10"

vrf forwarding 10

ip address 192.168.10.1 255.255.255.0

!

interface Tunnel2

ip unnumbered GigabitEthernet2

tunnel source GigabitEthernet2

tunnel mode sdwan

!

interface Tunnel3

ip unnumbered GigabitEthernet3

tunnel source GigabitEthernet3

tunnel mode sdwan

!

interface Tunnel100512

ip address 10.10.2.1 255.255.255.252

tunnel source GigabitEthernet2

tunnel destination 203.0.113.1

tunnel vrf multiplexing

!

interface Tunnel100513

ip address 10.10.2.5 255.255.255.252

tunnel source GigabitEthernet2

tunnel destination 203.0.113.2

tunnel vrf multiplexing

!

!

ip sdwan route vrf 10 0.0.0.0/0 tunnel active Tunnel100512 backup Tunnel100513

ip nat route vrf 10 192.0.2.1 255.255.255.255 global

ip nat pool natpool1 172.16.2.9 172.16.2.10 prefix-length 30

ip nat inside source list nat-dia-vpn-hop-access-list interface GigabitEthernet2 overload

ip nat inside source list global-list pool natpool1 vrf 10 match-in-vrf overload

!

ip route 0.0.0.0 0.0.0.0 198.51.100.100

ip route 0.0.0.0 0.0.0.0 172.20.1.100

!

DC-Router Configuration

DC Router configuration is as follows.

vrf definition 10

rd 1:10

!

address-family ipv4

route-target export 10:10

route-target import 10:10

exit-address-family

!

interface Tunnel2

ip unnumbered GigabitEthernet2

tunnel source GigabitEthernet2

tunnel mode sdwan

!

interface GigabitEthernet2

ip address 172.20.1.30 255.255.255.0

description "MPLS TLOC"

!

interface GigabitEthernet4

description "Service Side VPN 10"

vrf forwarding 10

ip address 172.31.19.19 255.255.255.252

!

router bgp 10

bgp log-neighbor-changes

distance bgp 20 200 20

!

address-family ipv4 vrf 10

redistribute omp

neighbor 172.31.19.20 remote-as 100

neighbor 172.31.19.20 activate

neighbor 172.31.19.20 send-community both

exit-address-family

!

!

ip route 0.0.0.0 0.0.0.0 172.20.1.100

!

vSmart Policy

vSmart policy configuration is as follows.

Note: Please note that nat pool 1 is called in policy for both branches, however, there are two different IP pools configured for each branch (172.16.2.0/30 for Branch-1 and 172.16.2.8/30 for Branch-2).

data-policy _VPN10-VPN20_1-Branch-A-B-Central-NAT-DIA

vpn-list VPN10

sequence 1

match

source-ip 192.168.10.0/24

!

action accept

nat pool 1

!

default-action accept

!

site-list BranchA-B

site-id 11

site-id 22

!

site-list DC

site-id 33

!

vpn-list VPN10

vpn 10

!

prefix-list _AnyIpv4PrefixList

ip-prefix 0.0.0.0/0 le 32

!

apply-policy

site-list BranchA-B

data-policy _VPN10_1-Branch-A-B-Central-NAT-DIA from-service

!

Failover Scenarios

Branch-1 Traffic Flow Normal Scenario

When both transports are up as shown in the output, by default traffic exits via the primary SIG tunnel Tunnel100512. When the primary tunnel goes down traffic switches to the backup tunnel Tunnel100513.

Branch-1#show ip route vrf 10

Routing Table: 10

<SNIP>

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

S*     0.0.0.0/0 [2/0], Tunnel100512

192.0.2.0/32 is subnetted, 1 subnets

n Nd   192.0.2.1 [6/0], 3d02h, Null0

n Ni   172.16.2.0 [7/0], 3d04h, Null0

m      172.16.2.8 [251/0] via 172.31.31.2, 3d01h, Sdwan-system-intf

Branch-1#

Traceroute shows that traffic takes the SIG Tunnel.

Host-BR-1#ping 192.0.2.100

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.100, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 8/49/101 ms

Host-BR-1#

Host-BR-1#traceroute 192.0.2.100 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.100

VRF info: (vrf in name/id, vrf out name/id)

1 192.168.10.1 38 msec 7 msec 4 msec

2 203.0.113.1 79 msec * 62 msec

Host-BR-1#

Traffic to a specific destination 192.0.2.1 takes exit via DIA (NATed to WAN IP address).

Host-BR-1#ping 192.0.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 8/49/101 ms

Host-BR-1#

Branch-1#sh ip nat translation

Pro    Inside global     Inside local      Outside local     Outside global

icmp   198.51.100.1:1    192.168.10.10:1    192.0.2.1:1       192.0.2.1:1

Total number of translations: 1

Branch-1#

Branch-2 Traffic Flow Normal Scenario

Similar behavior is observed on the Branch-2 router as well.

Branch-2#show ip route vrf 10

Routing Table: 10

<SNIP>

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

S*    0.0.0.0/0 [2/0], Tunnel100512

192.0.2.0/32 is subnetted, 1 subnets

n Nd  192.0.2.1 [6/0], 00:00:08, Null0

m     172.16.2.0 [251/0] via 172.31.31.1, 3d01h, Sdwan-system-intf

n Ni  172.16.2.8 [7/0], 3d04h, Null0

Branch-2#

Host-BR-2#ping 192.0.2.100

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.100, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 8/49/101 ms

Host-BR-2#

Host-BR-2#traceroute 192.0.2.100 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.100

VRF info: (vrf in name/id, vrf out name/id)

1 192.168.10.1 38 msec 7 msec 4 msec

2 203.0.113.1 79 msec * 62 msec

Host-BR-2#

Host-BR-2#ping 192.0.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 8/49/101 ms

Host-BR-2#

Branch-2#show ip nat translation

Pro   Inside global     Inside local     Outside local     Outside global

icmp  198.51.100.2:1    192.168.10.10:1   192.0.2.1:1         192.0.2.1:1

Total number of translations: 1

Branch-2#

Failure Scenarios

Branch-1 Failure Scenario

This section describes the behavior during the Internet failure.

The internet link is administratively shut down to simulate an internet failure link.

Branch-1#show sdwan control local-properties

<SNIP>

PUBLIC           PUBLIC     PRIVATE   PRIVATE        PRIVATE                           MAX

INTERFACE               IPv4             PORT       IPv4      IPv6           PORT   VS/VM  COLOR         STATE CNTRL

---------------------------------------------------------------------------------------------------------------------

GigabitEthernet2          198.51.100.1    12346    198.51.100.1   ::         12346    1/0  biz-internet      down

GigabitEthernet3          172.20.1.10     12346    172.20.1.10    ::         12346    1/1  mpls               up

Branch-1#

Outputs show that during the internet link failure scenario, the Branch-1 router receives the default route from the DC router via OMP. 172.31.31.3 is the system-IP for the DC router.

Branch-1#show ip route vrf 10

<SNIP>

Gateway of last resort is 172.31.31.3 to network 0.0.0.0

m* 0.0.0.0/0 [251/0] via 172.31.31.3, 00:01:17, Sdwan-system-intf

<SNIP>

Traffic destined to 192.0.2.100  gets NATed to the service side NAT pool and exits via DC.

Host-BR-1#ping 192.0.2.100

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.100, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 10/37/103 ms

Host-BR-1#

Branch-1#show ip nat translations

Pro   Inside global   Inside local    Outside local    Outside global

icmp  172.16.2.1:3    192.168.10.1:3   192.0.2.100:3        192.0.2.100:3

Total number of translations: 1

Branch-1#

Traceroute results show traffic takes the DC path. 172.20.1.30 is the MPLS transport WAN IP of the DC router.

Host-BR-1#traceroute 192.0.2.100 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.100

1 192.168.10.1 26 msec 5 msec 3 msec

2 172.20.1.30 10 msec 5 msec 27 msec

<SNIP>

Branch-1#show sdwan bfd sessions

SOURCE TLOC     REMOTE TLOC                DST PUBLIC      DST PUBLIC          DETECT TX

SYSTEM IP       SITE ID   STATE    COLOR           COLOR        SOURCE IP     IP              PORT        ENCAP   MULTIPLIER  INTERVAL(msec) UPTIME TRANSITIONS

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

172.31.31.2     22         up       mpls           mpls         172.20.1.10    172.20.1.20    12406       ipsec    7           1000            0:14:56:54 0

172.31.31.3     33         up       mpls           mpls         172.20.1.10    172.20.1.30    12406       ipsec    7           1000            0:14:56:57 0

Branch-1#

Traffic destined to specific IP 192.0.2.1 also gets NATed to the service side NAT pool and exits via DC.

Host-BR-1#ping 192.0.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 10/37/103 ms

Host-BR-1#

Branch-1#show ip nat translations

Pro    Inside global    Inside local      Outside local     Outside global

icmp   172.16.2.1:4    192.168.10.10:4    192.0.2.1:4         192.0.2.1:4

Total number of translations: 1

Branch-1#

Host-BR-1#traceroute 192.0.2.1 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.1

1 192.168.10.1 26 msec 5 msec 3 msec

2 172.20.1.30  10 msec 5 msec 27 msec

<SNIP>

Data policy configuration pushed from vSmart:

Branch-1#show sdwan policy from-vsmart

from-vsmart data-policy _VPN10-VPN20_1-Branch-A-B-Central-NAT-DIA

direction from-service

vpn-list VPN10

sequence 1

match

source-ip 192.168.10.0/24

action accept

count NAT_VRF10_BRANCH_A_B_-968382210

nat pool 1

!

from-vsmart lists vpn-list VPN10

vpn 10

!

Branch-1#

Branch-1#show run | sec "natpool1"

<SNIP>

ip nat pool natpool1 172.16.2.1 172.16.2.2 prefix-length 30

Branch-2 Failure Scenario

Similar behavior is also observed in the Branch-2 routers when there is an internet failover.

Branch-2#show sdwan control local-properties

<SNIP>

PUBLIC        PUBLIC      PRIVATE   PRIVATE       PRIVATE                              MAX

INTERFACE             IPv4          PORT        IPv4      IPv6          PORT    VS/VM  COLOR          STATE CNTRL

---------------------------------------------------------------------------------------------------------------------

GigabitEthernet2      198.51.100.2  12346       198.51.100.2  ::        12346    1/0   biz-internet       down

GigabitEthernet3      172.20.1.20   12346       172.20.1.20   ::        12346    1/1   mpls                up

Branch-2#

Branch-2#show ip route vrf 10

<SNIP>

Gateway of last resort is 172.31.31.3 to network 0.0.0.0

m* 0.0.0.0/0 [251/0] via 172.31.31.3, 00:10:17, Sdwan-system-intf

<SNIP>

Host-BR-2#ping 192.0.2.100

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.100, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 10/37/103 ms

Host-BR-2#

Branch-2#show ip nat translations

Pro    Inside global         Inside local          Outside local         Outside global

icmp   172.16.2.9:3         192.168.10.1:3         192.0.2.100:3             192.0.2.100:3

Total number of translations: 1

Branch-2#

Host-BR-2#traceroute 192.0.2.100 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.100

1 192.168.10.1 26 msec 5 msec 3 msec

2 172.20.1.30  10 msec 5 msec 27 msec

<SNIP>

Host-BR-2#ping 192.0.2.1

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.1, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 10/37/103 ms

Host-BR-2#

Branch-2#show ip nat translations

Pro  Inside global         Inside local          Outside local         Outside global

icmp 172.16.2.9:4         192.168.10.10:4         192.0.2.1:4             192.0.2.1:4

Total number of translations: 1

Branch-2#

Host-BR-2#traceroute 192.0.2.1 numeric

Type escape sequence to abort.

Tracing the route to 192.0.2.1

1 192.168.10.1 26 msec 5 msec 3 msec

2 172.20.1.30  10 msec 5 msec 27 msec

<SNIP>

Branch-2#show sdwan policy from-vsmart

from-vsmart data-policy _VPN10-VPN20_1-Branch-A-B-Central-NAT-DIA

direction from-service

vpn-list VPN10

sequence 1

match

source-ip 192.168.10.0/24

action accept

count NAT_VRF10_BRANCH_A_B_-968382210

nat pool 1

!

from-vsmart lists vpn-list VPN10-VPN20

vpn 10

!

Branch-2#

Branch-2#show run | sec "natpool1"

<SNIP>

ip nat pool natpool1 172.16.2.9 172.16.2.9 prefix-length 30

DC Router Routing Status

The routing table captures from the DC router.

As shown in the output, the DC router is able to differentiate overlapping IP addresses from both branches with the post-NAT IPderived from the SS-NAT pool (172.16.2.0 and 172.16.2.8) instead of the actual LAN IP 192.168.10.0/24172.31.31.1 and 172.31.31.2 are the system-ip configured for Branch-1/Branch-2. System-IP 172.31.31.10 belongs to vSmart.

DC-RTR#show ip route vrf 10

Routing Table: 10

<SNIP>

m        172.16.2.0  [251/0] via 172.31.31.1, 02:44:25, Sdwan-system-intf

m        172.16.2.8  [251/0] via 172.31.31.2, 02:43:33, Sdwan-system-intf

m        192.168.10.0[251/0] via 172.31.31.2, 03:01:35, Sdwan-system-intf

[251/0] via 172.31.31.1, 03:01:35, Sdwan-system-intf

DC-RTR#show sdwan omp routes

<SNIP>                                     PATH                        ATTRIBUTE

VPN    PREFIX          FROM PEER      ID    LABEL      STATUS     TYPE     TLOC IP     COLOR            ENCAP    PREFERENCE

--------------------------------------------------------------------------------------------------------------------------------------

10     172.16.2.0/30   172.31.31.10     6     1002       C,I,R    installed 172.31.31.1   mpls            ipsec -

172.31.31.10     10    1002       Inv,U    installed 172.31.31.1   biz-internet    ipsec -

10     172.16.2.8/30   172.31.31.10     8     1002       C,I,R    installed 172.31.31.2   mpls            ipsec -

10     192.168.10.0/24 172.31.31.10     1     1002       C,I,R    installed 172.31.31.1   mpls            ipsec -

172.31.31.10     2     1002       C,I,R    installed 172.31.31.2   mpls            ipsec -

172.31.31.10     12    1002       Inv,U    installed 172.31.31.1   biz-internet    ipsec -

Verify

There is currently no specific verification procedure available for this configuration.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.

Additional Information

Scenario-1

In scenarios where Controllers are on version 20.3.4, and cEdge runs 17.3.3a or lower versions with the same configurations, it is observed, that in normal/fail-over scenarios traffic gets NATed to the service side NAT pool and breaks the flow.

cEdge captures:

Host-BR-1#ping 192.0.2.100

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.0.2.100, timeout is 2 seconds:

U.U.U

Success rate is 0 percent (0/5)

Host-BR-1#

Branch-1#show ip nat translations

Pro    Inside global   Inside local    Outside local     Outside global

icmp   172.16.2.1:3    192.168.10.1:3   192.0.2.100:3         192.0.2.100:3

Total number of translations: 1

Branch-1#

WOW-Branch-1#show run | sec "natpool1"

<SNIP>

ip nat pool natpool1 172.16.2.1 172.16.2.2 prefix-length 30

Output is captured from cEdge runs on the 17.3.3a version. The traffic destined via the SIG Tunnel gets NATed to the SS-NAT pool and is dropped. A fix is available from version 17.3.6 onwards.

Scenario-2

Requirement (Service Side NAT (SS-NAT) with UTD Inspection)

Suppose the user has requested these requirements:

1. When both the internet and MPLS transports are operational, wireless clients in VPN 10 can be directed to ISE in the Data Center for authentication. Additionally, VPN 10 traffic traveling via the SD-WAN overlay can undergo inspection. As this traffic is part of the overlay, VPN 10 utilizes the SS-NAT feature. [UTD + SS-NAT]

2. If the internet transport becomes unavailable, all traffic from VPN 10, including both wireless and wired traffic, can be routed via the overlay using the MPLS transport. This traffic can also be subject to inspection. [UTD + SS-NAT]

These requirements aim to ensure secure and monitored traffic flow for VPN 10 in Branch-1 under different network conditions.

In both previously mentioned scenarios, you have UTD inspection with an SS-NAT combination. Here is the sample UTD configuration for this scenario.

policy utd-policy-vrf-10

all-interfaces

vrf 10

threat-inspection profile TEST_IDS_Policy

exit

Warning: Please note that currently, the combination of UTD with SS-NAT is not supported. Therefore, this combination does not work as expected. A fix for this issue might be included in future releases.

Workaround

The workaround is to disable the UTD policy on Overlapping IP VPN (in this case VPN 10) and enable Global VPN.

Note: This configuration is tested and verified in the 17.6 version.

policy utd-policy-vrf-global

all-interfaces

vrf global

threat-inspection profile TEST_IDS_Policy

exit