The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes the configuration walkthrough of L4-L7 Service Graph with Route Peering, where both the consumer and the provider are external to the Application Centric Infrastructure (ACI) fabric.
Contributed by Zahid Hassan, Cisco Advanced Services Engineer.
Cisco recommends that you have knowledge of these topics:
The preceding Fabric Access and L3Out configurations steps are not covered in this document and have been assumed that these have already been completed.
The information in this document is based on these software versions:
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Route Peering is a feature which enables a service appliance such as a load balancer or a firewall to advertise it’s reachability through the ACI fabric to all the way to an external network.
The use case presented here is a physical firewall which is deployed as a two-arm Service Graph, in between two L3Outs or external End Point Groups (EPGs). The Service Graph is associated with a contract between the external EPG on Leaf 101 (N3K-1) and the external EPG on Leaf 102 (N3K-2). The ACI fabric is providing a transit ervice for the routers (N3K-1 and N3K-2) and Route Peering is used, with Open Shortest Path First (OSPF) as the routing protocol, to exchange routes between the firewall and the ACI fabric.
The following image shows how Route Peering works end-to-end:


Step 1. Configure the Virtual Routing and Forwarding1 (VRF1), VRF2, Bridge Domain1 (BD1) and BD2. Associate BD1 to VRF1 and BD2 to VRF2, as shown in the image:

Step 2. Upload the ASA Device Package under L4-L7 Device, as shown in the image, :

Configure L4-L7 Device for physical ASA 5585 (Routed), as shown in the image:

Step 3. Configure L3Out for N3K-1 and associate with BD1 and VRF1.
External routed network is used to specify the routing configuration in the ACI fabric for Route Peering, as shown in the image:

Note: All L3Out interfaces which are used for Route Peering, are required to be configured as a Switch Virtual Interface (SVI) with VLAN encap accordingly.

Configure Import/Export Route Control on Subnets for N3K-1 L3Out External EPG, as shown in the image:

Configure L3Out for ASA-External Interface and associate with BD1 and VRF1, as shown in the image:


Configure Import/Export Route Control on Subnets for ASA-External L3Out External EPG, as shown in the image:

Configure L3out for ASA-Internal and associate with BD2 and VRF2, as shown in the image:


Configure Import/Export Route Control on Subnets for ASA-Internal L3Out External EPG, as shown in the image:

Configure L3Out for N3K-2 and associate with BD2 and VRF2, as shown in the image:


Configure Import/Export Route Control on Subnets for N3K-2 L3Out for External EPG, as shown in the image:

Step 4. Create Function Profile Group and configure Function Profile from existing template, as shown in the image:


Step 5. Create a Contract and modify the Scope field to Tenant, as shown in the image:

Step 6. As shown in the image, create L4-L7 Service Graph Template where Service Graph association involves the association of an external routed network policy and router configuration with a Device Selection Policy.
:


Router configuration to specify the Router ID that will be used on the Service Appliance (ASA 5585), as shown in the image:

Change Adjacency Type from L2 to L3, as shown in the image:

Apply Service Graph Template, as shown in the image:

Attach the Service Graph to Contract, as shown in the image:


Add/Change L4-L7 Parameter if needed, as shown in the image:

Step 7: Route-tag Policy, configure Route-tag Policy for VRF1 (Tag:100), as shown in the image:

Configure Route-tag Policy for VRF2 (Tag:200), as shown in the image:

Step 8: Check the status and verify Device Selection Policy, as shown in the image:


Verify Deployed Graph instance, as shown in the image:



APIC configuration for Tenant:
apic1# sh running-config tenant T1
# Command: show running-config tenant T1
# Time: Thu Feb 25 16:05:14 2016
tenant T1
access-list PERMIT_ALL
match ip
exit
contract PERMIT_ALL
scope tenant
subject PERMIT_ALL
access-group PERMIT_ALL both
l4l7 graph ASA5585_SGT
exit
exit
vrf context VRF1
exit
vrf context VRF2
exit
l3out ASA_IN_L3OUT
vrf member VRF2
exit
l3out ASA_OUT_L3OUT
vrf member VRF1
exit
l3out N3K-1_L3OUT
vrf member VRF1
exit
l3out N3K-2_L3OUT
vrf member VRF2
exit
bridge-domain BD1
vrf member VRF1
exit
bridge-domain BD2
vrf member VRF2
exit
application AP1
epg EPG1
bridge-domain member BD1
exit
epg EPG2
bridge-domain member BD2
exit
exit
external-l3 epg ASA_IN_EXT_NET l3out ASA_IN_L3OUT
vrf member VRF2
match ip 10.10.10.0/24
exit
external-l3 epg ASA_OUT_EXT_NET l3out ASA_OUT_L3OUT
vrf member VRF1
match ip 20.20.20.0/24
exit
external-l3 epg N3K-1_EXT_NET l3out N3K-1_L3OUT
vrf member VRF1
match ip 10.10.10.0/24
contract consumer PERMIT_ALL
exit
external-l3 epg N3K-2_EXT_NET l3out N3K-2_L3OUT
vrf member VRF2
match ip 20.20.20.0/24
contract provider PERMIT_ALL
exit
interface bridge-domain BD1
exit
interface bridge-domain BD2
exit
l4l7 cluster name ASA5585 type physical vlan-domain T1_PHY service FW function go-to
cluster-device ASA5585_Device_1
cluster-interface inside
member device ASA5585_Device_1 device-interface GigabitEthernet0/1
interface ethernet 1/2 leaf 106
exit
exit
cluster-interface outside
member device ASA5585_Device_1 device-interface GigabitEthernet0/0
interface ethernet 1/2 leaf 105
exit
exit
exit
l4l7 graph ASA5585_SGT contract PERMIT_ALL
service N1 device-cluster-tenant T1 device-cluster ASA5585 mode FW_ROUTED
connector consumer cluster-interface outside
l4l7-peer tenant T1 out ASA_OUT_L3OUT epg ASA_OUT_EXT_NET redistribute bgp,ospf
exit
connector provider cluster-interface inside
l4l7-peer tenant T1 out ASA_IN_L3OUT epg ASA_IN_EXT_NET redistribute bgp,ospf
exit
rtr-cfg ASA5585
exit
connection C1 terminal consumer service N1 connector consumer
connection C2 terminal provider service N1 connector provider
exit
rtr-cfg ASA5585
router-id 3.3.3.3
exit
exit
apic1#
[an error occurred while processing this directive]
Verify OSPF neighbor relationship and routing table on leaf 101:
leaf101# show ip ospf neighbors vrf T1:VRF1 OSPF Process ID default VRF T1:VRF1 Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 1.1.1.1 1 FULL/BDR 02:07:19 192.168.1.1 Vlan8 3.3.3.3 1 FULL/BDR 00:38:35 192.168.1.5 Vlan9[an error occurred while processing this directive]
leaf101# show ip route vrf T1:VRF1
IP Route Table for VRF "T1:VRF1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.10.10.0/24, ubest/mbest: 1/0
*via 192.168.1.1, vlan8, [110/8], 01:59:50, ospf-default, intra
20.20.20.0/24, ubest/mbest: 1/0
*via 192.168.1.5, vlan9, [110/22], 00:30:20, ospf-default, inter
100.100.100.100/32, ubest/mbest: 2/0, attached, direct
*via 100.100.100.100, lo1, [1/0], 02:21:22, local, local
*via 100.100.100.100, lo1, [1/0], 02:21:22, direct
192.168.1.0/30, ubest/mbest: 1/0, attached, direct
*via 192.168.1.2, vlan8, [1/0], 02:35:53, direct
192.168.1.2/32, ubest/mbest: 1/0, attached
*via 192.168.1.2, vlan8, [1/0], 02:35:53, local, local
192.168.1.4/30, ubest/mbest: 1/0, attached, direct
*via 192.168.1.6, vlan9, [1/0], 02:20:53, direct
192.168.1.6/32, ubest/mbest: 1/0, attached
*via 192.168.1.6, vlan9, [1/0], 02:20:53, local, local
192.168.1.8/30, ubest/mbest: 1/0
*via 192.168.1.5, vlan9, [110/14], 00:30:20, ospf-default, intra
200.200.200.200/32, ubest/mbest: 1/0
*via 192.168.1.5, vlan9, [110/15], 00:30:20, ospf-default, intra
Verify OSPF neighbor relationship and routing table on leaf 102:
leaf102# show ip ospf neighbors vrf T1:VRF2
OSPF Process ID default VRF T1:VRF2
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
3.3.3.3 1 FULL/BDR 00:37:07 192.168.1.9 Vlan14
2.2.2.2 1 FULL/BDR 02:09:59 192.168.1.13 Vlan15
leaf102# show ip route vrf T1:VRF2
IP Route Table for VRF "T1:VRF2"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.10.10.0/24, ubest/mbest: 1/0
*via 192.168.1.9, vlan14, [110/22], 00:35:22, ospf-default, inter
20.20.20.0/24, ubest/mbest: 1/0
*via 192.168.1.13, vlan15, [110/8], 02:08:13, ospf-default, intra
192.168.1.4/30, ubest/mbest: 1/0
*via 192.168.1.9, vlan14, [110/14], 00:35:22, ospf-default, intra
192.168.1.8/30, ubest/mbest: 1/0, attached, direct
*via 192.168.1.10, vlan14, [1/0], 02:14:29, direct
192.168.1.10/32, ubest/mbest: 1/0, attached
*via 192.168.1.10, vlan14, [1/0], 02:14:29, local, local
192.168.1.12/30, ubest/mbest: 1/0, attached, direct
*via 192.168.1.14, vlan15, [1/0], 02:09:04, direct
192.168.1.14/32, ubest/mbest: 1/0, attached
*via 192.168.1.14, vlan15, [1/0], 02:09:04, local, local
200.200.200.200/32, ubest/mbest: 2/0, attached, direct
*via 200.200.200.200, lo4, [1/0], 02:10:02, local, local
*via 200.200.200.200, lo4, [1/0], 02:10:02, direct
[an error occurred while processing this directive]
Verify configuration, OSPF neighbor relationship and routing table on ASA 5585:
ASA5585# sh run interface
!
interface GigabitEthernet0/0
no nameif
security-level 0
no ip address
!
interface GigabitEthernet0/0.101
nameif externalIf
security-level 50
ip address 192.168.1.5 255.255.255.252
!
interface GigabitEthernet0/1
no nameif
security-level 100
no ip address
!
interface GigabitEthernet0/1.102
nameif internalIf
security-level 100
ip address 192.168.1.9 255.255.255.252
!
interface Management0/0
management-only
nameif management
security-level 0
ip address 172.23.97.1 255.255.254.0
ASA5585# sh run router
router ospf 1
router-id 3.3.3.3
network 192.168.1.4 255.255.255.252 area 0
network 192.168.1.8 255.255.255.252 area 0
area 0
log-adj-changes
!
ASA5585# sh ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
100.100.100.100 1 FULL/DR 0:00:38 192.168.1.6 externalIf
200.200.200.200 1 FULL/DR 0:00:33 192.168.1.10 internalIf
ASA5585# sh route ospf
Routing Table: T1
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, + - replicated route
Gateway of last resort is not set
O IA 10.10.10.0 255.255.255.0
[110/18] via 192.168.1.6, 00:22:57, externalIf
O IA 20.20.20.0 255.255.255.0
[110/18] via 192.168.1.10, 00:22:47, internalIf
O 200.200.200.200 255.255.255.255
[110/11] via 192.168.1.10, 00:22:47, internalIf
ASA5585# sh access-list
access-list cached ACL log flows: total 0, denied 0 (deny-flow-max 4096)
alert-interval 300
access-list access-list-inbound; 3 elements; name hash: 0xcb5bd6c7
access-list access-list-inbound line 1 extended permit tcp any any eq www (hitcnt=0) 0xc873a747
access-list access-list-inbound line 2 extended permit tcp any any eq https (hitcnt=0) 0x48bedbdd
access-list access-list-inbound line 3 extended permit icmp any any (hitcnt=6) 0xe4b5a75d[an error occurred while processing this directive]
Verify configuration, OSPF neighbor relationship and routing table on N3K-1:
N3K-1# sh run ospf
!Command: show running-config ospf
!Time: Thu Feb 25 15:40:55 2016
version 6.0(2)U3(7)
feature ospf
router ospf 1
router-id 1.1.1.1
interface Ethernet1/21
ip router ospf 1 area 0.0.0.1
interface Ethernet1/47
ip router ospf 1 area 0.0.0.1
N3K-1# sh ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
100.100.100.100 1 FULL/DR 01:36:24 192.168.1.2 Eth1/47
N3K-1# sh ip ospf route
OSPF Process ID 1 VRF default, Routing Table
(D) denotes route is directly attached (R) denotes route is in RIB
10.10.10.0/24 (intra)(D) area 0.0.0.1
via 10.10.10.0/Eth1/21* , cost 4
20.20.20.0/24 (inter)(R) area 0.0.0.1
via 192.168.1.2/Eth1/47 , cost 62
100.100.100.100/32 (intra)(R) area 0.0.0.1
via 192.168.1.2/Eth1/47 , cost 41
192.168.1.0/30 (intra)(D) area 0.0.0.1
via 192.168.1.1/Eth1/47* , cost 40
[an error occurred while processing this directive]
Verify configuration, OSPF neighbor relationship and routing table on N3K-2:
N3K-2# sh run ospf
!Command: show running-config ospf
!Time: Thu Feb 25 15:44:47 2016
version 6.0(2)U3(7)
feature ospf
router ospf 1
router-id 2.2.2.2
interface loopback0
ip ospf network point-to-point
ip router ospf 1 area 0.0.0.0
interface Ethernet1/21
ip router ospf 1 area 0.0.0.1
interface Ethernet1/47
ip router ospf 1 area 0.0.0.1
N3K-2# sh ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 1
Neighbor ID Pri State Up Time Address Interface
200.200.200.200 1 FULL/DR 01:43:50 192.168.1.14 Eth1/47
N3K-2# sh ip ospf route
OSPF Process ID 1 VRF default, Routing Table
(D) denotes route is directly attached (R) denotes route is in RIB
2.2.2.0/30 (intra)(D) area 0.0.0.0
via 2.2.2.0/Lo0* , cost 1
10.10.10.0/24 (inter)(R) area 0.0.0.1
via 192.168.1.14/Eth1/47 , cost 62
20.20.20.0/24 (intra)(D) area 0.0.0.1
via 20.20.20.0/Eth1/21* , cost 4
192.168.1.12/30 (intra)(D) area 0.0.0.1
via 192.168.1.13/Eth1/47* , cost 40
[an error occurred while processing this directive]
Verify contract filter rules on leaf and the packet hit counts:.
leaf101# show system internal policy-mgr stats Requested Rule Statistics [CUT] Rule (4107) DN (sys/actrl/scope-3112964/rule-3112964-s-32773-d-49158-f-33) Ingress: 1316, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4108) DN (sys/actrl/scope-3112964/rule-3112964-s-49158-d-32773-f-33) Ingress: 1317, Egress: 0, Pkts: 0 RevPkts: 0 leaf101# show system internal policy-mgr stats Requested Rule Statistics [CUT] Rule (4107) DN (sys/actrl/scope-3112964/rule-3112964-s-32773-d-49158-f-33) Ingress: 2317, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4108) DN (sys/actrl/scope-3112964/rule-3112964-s-49158-d-32773-f-33) Ingress: 2317, Egress: 0, Pkts: 0 RevPkts: 0[an error occurred while processing this directive]
leaf102# show system internal policy-mgr stats Requested Rule Statistics [CUT] Rule (4103) DN (sys/actrl/scope-2752520/rule-2752520-s-49156-d-6019-f-default) Ingress: 3394, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4104) DN (sys/actrl/scope-2752520/rule-2752520-s-6019-d-49156-f-default) Ingress: 3394, Egress: 0, Pkts: 0 RevPkts: 0 [CUT] leaf102# show system internal policy-mgr stats Requested Rule Statistics [CUT] Rule (4103) DN (sys/actrl/scope-2752520/rule-2752520-s-49156-d-6019-f-default) Ingress: 4392, Egress: 0, Pkts: 0 RevPkts: 0 Rule (4104) DN (sys/actrl/scope-2752520/rule-2752520-s-6019-d-49156-f-default) Ingress: 4392, Egress: 0, Pkts: 0 RevPkts: 0 [CUT]
Reachability test between N3K-1 and N3K-2:
N3K-1# ping 20.20.20.1 source 10.10.10.1 PING 20.20.20.1 (20.20.20.1) from 10.10.10.1: 56 data bytes 64 bytes from 20.20.20.1: icmp_seq=0 ttl=250 time=2.098 ms 64 bytes from 20.20.20.1: icmp_seq=1 ttl=250 time=0.922 ms 64 bytes from 20.20.20.1: icmp_seq=2 ttl=250 time=0.926 ms 64 bytes from 20.20.20.1: icmp_seq=3 ttl=250 time=0.893 ms 64 bytes from 20.20.20.1: icmp_seq=4 ttl=250 time=0.941 ms --- 20.20.20.1 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.893/1.156/2.098 ms N3K-2# ping 10.10.10.1 source 20.20.20.1 PING 10.10.10.1 (10.10.10.1) from 20.20.20.1: 56 data bytes 64 bytes from 10.10.10.1: icmp_seq=0 ttl=250 time=2.075 ms 64 bytes from 10.10.10.1: icmp_seq=1 ttl=250 time=0.915 ms 64 bytes from 10.10.10.1: icmp_seq=2 ttl=250 time=0.888 ms 64 bytes from 10.10.10.1: icmp_seq=3 ttl=250 time=1.747 ms 64 bytes from 10.10.10.1: icmp_seq=4 ttl=250 time=0.828 ms --- 10.10.10.1 ping statistics --- 5 packets transmitted, 5 packets received, 0.00% packet loss round-trip min/avg/max = 0.828/1.29/2.075 ms[an error occurred while processing this directive]
Attached is the XML configuration file for the Tenant and the ASA Function Profile, used for this demonstration.
Feedback