Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Configure

Network Diagram

Configurations

Verify

Troubleshoot

Related Information

Introduction

This document describes how to configure the Internet Protocol Service Level Agreement (IPSLA) in Cisco Application Centric Infrastructure (ACI) to track static route learn from one L3out and advertise to another L3out only if the subnet is reachable from the first L3out.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

- ACI software version 4.1 and later

- L3out toward External device or Server

- EX and -FX chassis

- Track the route to use Internet Control Message Protocol (ICMP) and TCP probes (in this example ICMP probe is used)

Note: ACI image IP SLA is supported in all Cisco Nexus second-generation switches, which includes -EX and -FX chassis. Please read Guidelines and Limitations for IP SLA.

Components Used

The information in this document is based on these software and hardware versions:

- ACI version 5.2(2f)

- N9K-C93180YC-FX

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Some servers have multiple interfaces (like a  loopback) which is reachable from ACI via the physical IP address of the server. In such a case you can have a requirement to add a static route and advertise externally but only if the physical IP of the server is reachable. Hence IP SLA track feature is an unavoidable configuration that can only achieve through L3out configuration toward those servers. At this moment IP SLA track features are not supported for the static route on a Bridge Domain. In this document, we will look for server examples and transit route configurations that use IP SLA.

Configure

- L3out toward Server and toward N3K devices.

- Configure IP SLA track for the physical IP address of the server.

- Configure static route under L3out toward server which use IP SLA track and advertise from another L3out toward N3K.

Network Diagram

ACI Lab Topology

Configurations

Summary Steps:

ACI fabric policies:

- Create Contract (for this example, a common default filter that allows all traffic to be used, but you can use a specific filter locally created in the same Tenant to allow specific traffic. in such a case please ensure you allow protocol that we are used for IP SLA track).

- Create new L3out toward server 10.100.0.100/24 (ACI side SVI 550 with IP address 10.100.0.254)

- Create IP SLA Track Policies (IP SLA monitor policy, Track Members policy, Track List policy)

- Add static route under L3out toward server with IP SLA tracklist.

- Create a new L3out toward the N3K device which use BGP. (EBGP) ACI AS 65535 and N3K AS 65536

- Export static route from L3out toward N3K.

- Verify configuration and reachability.

1. Create Contract (for this example, use a common default filter that allows all traffic, however, you can use a specific filter locally created in the same tenant to allow specific traffic but in such case please ensure you allow protocol which we are used for IP SLA track).

Create Contract

2. Create a new L3out toward server 10.100.0.100/24 (ACI side SVI 550 with IP address 10.100.0.254).

Create L3out

Attaching node to L3out

Attaching interface to L3out

Configure External EPG

Attaching Contract to L3out

3. Create IP SLA Track Policies (IP SLA monitor policy, Track Members policy, Track List policy).

IP SLA Monitor Policy:

Configure IP SLA monitor policy

IP SLA Track Members:

Adding IP to monitor policy

Track List Policy:

Configure Track List

4. Configure static route under L3out toward server with newly created IP SLA tracklist policy.

Configure Static route under L3out

5. Create a new L3out toward the N3K device which uses Border Gateway Protocol (BGP). (EBGP) ACI AS 65535 and N3K AS 65536.

Configure BGP protocol

BGP Peer Profile

Configure BGP peer policy

Configure Logical interface profile under L3out

External EPG export subnet in transit L3out

Attaching Contract to External EPG

6. Export static route from L3out toward N3K.

switchname N3K

feature bgp

feature interface-vlan

interface Vlan550

no shutdown

vrf member BGP_L3out

ip address 100.0.0.2/30

interface loopback200

vrf member BGP_L3out

ip address 30.30.30.1/32

interface Ethernet1/1

switchport mode trunk

router bgp 65536

address-family ipv4 unicast

neighbor 100.0.0.1

vrf BGP_L3out

router-id 3.3.3.3

address-family ipv4 unicast

network 30.30.30.1/32

neighbor 100.0.0.1

remote-as 65535

update-source Vlan550

address-family ipv4 unicast

Verify

Use this section in order to confirm that your configuration works properly.

Nexus3K.

Transit route advertisement explained by topology

N3K# routing vrf BGP_L3out

N3K%BGP_L3out# show ip route

IP Route Table for VRF "BGP_L3out"

'*' denotes best ucast next-hop

'**' denotes best mcast next-hop

'[x/y]' denotes [preference/metric]

'%' in via output denotes VRF

30.30.30.1/32, ubest/mbest: 2/0, attached

*via 30.30.30.1, Lo200, [0/0], 02:35:27, local

*via 30.30.30.1, Lo200, [0/0], 02:35:27, direct

100.0.0.0/30, ubest/mbest: 1/0, attached

*via 100.0.0.2, Vlan550, [0/0], 05:52:18, direct

100.0.0.2/32, ubest/mbest: 1/0, attached

*via 100.0.0.2, Vlan550, [0/0], 05:52:18, local

200.0.0.1/32, ubest/mbest: 1/0

*via 100.0.0.1, [20/0], 02:32:36, bgp-65536, external, tag 65535

Server Loopback is reachable with source as N3K loopback address.

N3K

interface loopback200

vrf member BGP_L3out

ip address 30.30.30.1/32

N3K# ping 200.0.0.1 vrf BGP_L3out source 30.30.30.1

PING 200.0.0.1 (200.0.0.1): 56 data bytes

64 bytes from 200.0.0.1: icmp_seq=0 ttl=252 time=0.94 ms

64 bytes from 200.0.0.1: icmp_seq=1 ttl=252 time=0.729 ms

64 bytes from 200.0.0.1: icmp_seq=2 ttl=252 time=0.658 ms

64 bytes from 200.0.0.1: icmp_seq=3 ttl=252 time=0.706 ms

64 bytes from 200.0.0.1: icmp_seq=4 ttl=252 time=0.655 ms

--- 200.0.0.1 ping statistics ---

5 packets transmitted, 5 packets received, 0.00% packet loss

round-trip min/avg/max = 0.655/0.737/0.94 ms

ACI Leaf 102 route table (which has L3out toward Nexus 3K).

Leaf102# show ip route vrf TN_D:VRF_S

IP Route Table for VRF "TN_D:VRF_S"

'*' denotes best ucast next-hop

'**' denotes best mcast next-hop

'[x/y]' denotes [preference/metric]

'%' in via output denotes VRF

10.100.0.0/24, ubest/mbest: 1/0

*via 10.0.96.64%overlay-1, [200/0], 02:56:36, bgp-65535, internal, tag 65535

30.30.30.1/32, ubest/mbest: 1/0                                                      <<address of N3K.

*via 100.0.0.2%TN_D:VRF_S, [20/0], 02:44:34, bgp-65535, external, tag 65536

100.0.0.0/30, ubest/mbest: 1/0, attached, direct

*via 100.0.0.1, vlan19, [0/0], 05:09:37, direct

100.0.0.1/32, ubest/mbest: 1/0, attached

*via 100.0.0.1, vlan19, [0/0], 05:09:37, local, local

101.101.101.101/32, ubest/mbest: 1/0

*via 10.0.96.64%overlay-1, [1/0], 02:56:36, bgp-65535, internal, tag 65535

102.102.102.102/32, ubest/mbest: 2/0, attached, direct

*via 102.102.102.102, lo5, [0/0], 16:49:13, local, local

*via 102.102.102.102, lo5, [0/0], 16:49:13, direct

200.0.0.1/32, ubest/mbest: 1/0

*via 10.0.96.64%overlay-1, [1/0], 02:42:15, bgp-65535, internal, tag 65535

Leaf 101 IP SLA configuration verification from CLI.

Leaf101# show ip sla configuration

IP SLAs Infrastructure Engine-III

Entry number: 2000

Owner: owner-icmp-echo-dme

Tag:

Operation timeout (milliseconds): 900

Type of operation to perform: icmp-echo

Target address/Source address: 10.100.0.100/0.0.0.0

Traffic-Class parameter: 0x0

Type Of Service parameter: 0x0

Request size (ARR data portion): 28

Verify data: No

Vrf Name: TN_D:VRF_S

Schedule:

Operation frequency (seconds): 5  (not considered if randomly scheduled)

Next Scheduled Start Time: Start Time already passed

Group Scheduled : FALSE

Randomly Scheduled : FALSE

Life (seconds): Forever

Entry Ageout (seconds): 3600

Recurring (Starting Everyday): FALSE

Status of entry (SNMP RowStatus): Active

Threshold (milliseconds): 900

Distribution Statistics:

Number of statistic hours kept: 2

Number of statistic distribution buckets kept: 1

Statistic distribution interval (milliseconds): 20

History Statistics:

Number of history Lives kept: 0

Number of history Buckets kept: 15

History Filter Type: None

Leaf101# show track brief

TrackId  Type     Instance       Parameter            State      Last Change

4        IP SLA   2000           reachability         up         2021-09-16T18:08:42.364+00:00

3        List     ---            percentage           up         2021-09-16T18:08:42.365+00:00

Leaf101# show track

Track 1

List Threshold percentage

Threshold percentage is up

6 changes, last change 2021-09-16T00:01:50.339+00:00

Threshold percentage up 1% down 0%

Tracked List Members:

Object 2 (100)% up

Attached to:

Route prefix 200.0.0.1/32

Track 2

IP SLA 2000

reachability is up

6 changes, last change 2021-09-16T00:01:50.338+00:00

Tracked by:

Track List 1

Verification with Managed Object Query (Moquery) command:

apic1# moquery -c fvIPSLAMonitoringPol -f 'fv.IPSLAMonitoringPol.name=="ICMP_Monitor"'

Total Objects shown: 1

# fv.IPSLAMonitoringPol

name                 : ICMP_Monitor

annotation           :

childAction          :

descr                :

dn                   : uni/tn-TN_D/ipslaMonitoringPol-ICMP_Monitor

extMngdBy            :

httpMethod           : get

httpUri              : /

httpVersion          : HTTP10

ipv4Tos              : 0

ipv6TrfClass         : 0

lcOwn                : local

modTs                : 2021-09-15T21:18:48.195+00:00

monPolDn             : uni/tn-common/monepg-default

nameAlias            :

ownerKey             :

ownerTag             :

reqDataSize          : 28

rn                   : ipslaMonitoringPol-ICMP_Monitor

slaDetectMultiplier  : 3

slaFrequency         : 5

slaPort              : 0

slaType              : icmp

status               :

threshold            : 900

timeout              : 900

uid                  : 15374

userdom              : :all:

apic1# moquery -c fvTrackMember -f 'fv.TrackMember.name=="Server_Physical_IP"'

Total Objects shown: 1

# fv.TrackMember

name         : Server_Physical_IP

annotation   :

childAction  :

descr        :

dn           : uni/tn-TN_D/trackmember-Server_Physical_IP

dstIpAddr    : 10.100.0.100

extMngdBy    :

id           : 2000

lcOwn        : local

modTs        : 2021-09-15T21:16:22.992+00:00

monPolDn     : uni/tn-common/monepg-default

nameAlias    :

ownerKey     :

ownerTag     :

rn           : trackmember-Server_Physical_IP

scopeDn      : uni/tn-TN_D/out-L3out_Static_server

status       :

uid          : 15374

userdom      : :all:

apic1# moquery -c fvTrackList -f 'fv.TrackList.name=="Tracking_Server_Physical_IP"'

Total Objects shown: 1

# fv.TrackList

name           : Tracking_Server_Physical_IP

annotation     :

childAction    :

descr          :

dn             : uni/tn-TN_D/tracklist-Tracking_Server_Physical_IP

extMngdBy      :

lcOwn          : local

modTs          : 2021-09-15T07:41:15.958+00:00

monPolDn       : uni/tn-common/monepg-default

nameAlias      :

ownerKey       :

ownerTag       :

percentageDown : 0

percentageUp   : 1

rn             : tracklist-Tracking_Server_Physical_IP

status         :

type           : percentage

uid            : 15374

userdom        : :all:

weightDown     : 0

weightUp       : 1

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.

In event of link disconnection or physical IP address is unreachable, ACI IP SLA shows destination IP 'timeout' after configured threshold reaches.

L3out interface down

IP SLA monitor link status after link down

Leaf 101 CLI verification (You can see timeout for "Last Operation return code").

Leaf101# show ip sla statistics

IPSLAs Latest Operation Statistics

IPSLA operation id: 2000

Latest RTT: NoConnection/Busy/Timeout

Latest operation start time: 23:54:30 UTC Wed Sep 15 2021

Latest operation return code: Timeout

Number of successes: 658

Number of failures: 61

Operation time to live: forever

As soon as the server is reachable, it shows status OK.

IP SLA monitor status after link brought up

Leaf101# show ip sla statistics

IPSLAs Latest Operation Statistics

IPSLA operation id: 2000

Latest RTT: 1 milliseconds

Latest operation start time: 00:03:15 UTC Thu Sep 16 2021

Latest operation return code: OK

Number of successes: 18

Number of failures: 86

Operation time to live: forever

Related Information

- Cisco APIC Layer 3 Networking Configuration Guide, Release 5.2(x)

- Technical Support & Documentation - Cisco Systems