Configuration and Verification of SDWAN Integration with ACI

Available Languages

Download Options

  • PDF     (1.3 MB)
    View with Adobe Reader on a variety of devices
  • ePub     (1.2 MB)
    View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
  • Mobi (Kindle)     (1.0 MB)
    View on Kindle device or Kindle app on multiple devices
Updated:August 6, 2021
Document ID:216942

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

    Acronyms

    ACI - Application Centric Infrastructure 

    EPG - EndPoint Group

    L3out - Layer 3 Out

    AAR - Application Aware Routing

    SLA - Service Level Agreements

    DC -  Data Center

    WAN - Wide Area Network

    SDN - Software Defined Networking

    SD DC - Software Defined Data Center

    SD WAN -  Software Defined Wide Area Network

    QOS - Quality of Service

    VRF - Virtual Routing and Forwarding 

    Introduction

    This document describes configuration steps to integrate Application Centric Infrastructure (ACI), Cisco's Software Defined - Data Center (SD-DC) solution with  Software Defined - Wide Area Network (SD-WAN) and its verification.

    Software Defined Networking (SDN) have been enhanced to accommodate specific network segment:

    1. Software Defined -Data Center (SD-DC) 
    2. Software Defined - Wide Area Network (SD-WAN)

    Cisco solution provides robust feature of QoS (Qualtiy of Service) in SD-DC (Application Centric Infrastructure ACI) and AAR (Application Aware Routing)/SLA  (Service Level Agreements) profiles in SD-WAN.

    As more and more customers are planning to integrate and want to have seamless traffic treatment across the path, Cisco has come up with SD-DC and SD-WAN Integration.

    The integration focuses on two use cases:

    1. Traffic from ACI (DC) to SDWAN (non ACI Branch)
    2. Traffic from SDWAN (non ACI Branch)  to ACI (DC) 

    Prerequisites

    Requirements

    As the integration with SD-WAN happens over the L3 out configured in ACI, so L3out with supported protocol must be configured.

    Integration takes place over management network so Management reachability between ACI (APIC controllers) and vManage is required.

    Components Used

    ACI Fabric, SDWAN (vManage, vSmart Controller, vEdge)

    This document is based on ACI version 4.2(3l)

    Configuration

    Network Diagram

    Topology for reference:

    In our topology, consider only ACI site A as DC and non-ACI Site C as SDWAN Branch site.

    Topology

    Configurations

    Section A: Integration Configuration

    1. Open the APIC Graphical User Interface (GUI) and navigate to Integrations tab under System tab. 

    Integrations

    1. Create Integration Group

    Create Group

    1. Navigate to newly created Integration Group “SDWAN2” and right click on vManage

    vManage

    1. Right click on vManage and select Create Integration Manager

    Create Integration Manager

    1. Fill in the appropriate details like integration manager name, Device IP/FQDN, username, password

    Create Integration

    1. Ensure that registration is successful from status field. If it is not successfull or if any errors observed, verify if provided information is correct. Partner ID is identifier of vManage controller. You can navigate to Integrations -><Group Name>->vManage -> <Integration Manager Name> -> System info to verify the  status. 

    System Info

    Section B: Configuration of WAN SLA policy

    Pre-configured WAN SLA profiles can be found under Tenants->common->Policies->Protocols->WAN SLA

    This can be inherited in other tenant while configuring the contract using WAN SLA policy.

    These are pre-configured SLAs and cannot be changed.

    WAN SLA

    VPN configured on SD-WAN side which is mapped to this ACI integration will also be reflected under Tenants->common->Policies->Protocols->WAN SLA

    WAN VPN

    1. Create the contract under the tenant/VRF where you want to map the WAN services.

    The QoS Priority value must be set to any value other than Unspecified. The WAN SLA policies will not work if the QoS Priority value is set to Unspecified.

    Please navigate to Tenants-><tenant name>->Contracts->Standard

    Create Contract

    1. Create the Contract Subject and Under Contract Subject, specify WAN SLA Policy.

    The QoS Priority value must be set to any value other than Unspecified. The WAN SLA policies will not work if the QoS Priority value is set to Unspecified.

            

    Create Contract Subject

    1. Provide the contract from EPG.

    Please navigate to Tenants-><tenant name>->Application Profiles->Application EPG->Contracts

    Add Provided Contract

    1. Consume the contract at L3out configured for SD-WAN

    Please navigate to Tenants-><tenant name>->L3outs->External EPG->Consumed Contracts. It is also possible and valid to have contract provided by L3out External EPG and consumed by EPGs

    Consumed Contracts

    1. Match a WAN VPN to a tenant VRF 

    Please navigate to Tenants-><tenant name>->VRFs->Policy->WAN VPN

    Policy

    Verify

    Section 3: Verification

    1. Configuration verification

    Configuration is pushed to both SDWAN devices according to configuration in ACI

    DC end (connected to L3out) SDWAN route

    ASR1001-X-DC#show sdwan policy from-vsmart
    -->>> SLA Policy (parameters)
    from-vsmart sla-class Bulk-Data
     loss    10
     latency 300
     jitter  100

    from-vsmart sla-class Default
     loss    25
     latency 300
     jitter  100

    from-vsmart sla-class Transactional-Data
     loss    5
     latency 50
     jitter  100

    from-vsmart sla-class Voice-And-Video
     loss    2
     latency 45
     jitter  100

    from-vsmart data-policy _vpn-10_data_policy
     direction from-service
     vpn-list vpn-10
      default-action accept

    -->>> DSCP to SLA Mapping
    from-vsmart app-route-policy _412898115_vpn_412898115
     vpn-list 412898115_vpn
      sequence 10
       match
        dscp 14
       action
        sla-class Default
        no sla-class strict
      sequence 20
       match
        dscp 18
       action
        sla-class Voice-And-Video
        no sla-class strict
      sequence 30
       match
        dscp 12
       action
        sla-class Transactional-Data
        no sla-class strict
      sequence 40
       match
        dscp 10
       action
        sla-class Bulk-Data
        no sla-class strict

    from-vsmart lists vpn-list 412898115_vpn
     vpn 10

    from-vsmart lists vpn-list vpn-10
     vpn 10

    ASR1001-X-DC#

    Branch end SDWAN router

    ASR1001-X-Branch#show sdwan policy from-vsmart
    -->>> SLA Policy (parameters)
    from-vsmart sla-class Bulk-Data
     loss    10
     latency 300
     jitter  100

    from-vsmart sla-class Default
     loss    25
     latency 300
     jitter  100

    from-vsmart sla-class Transactional-Data
     loss    5
     latency 50
     jitter  100

    from-vsmart sla-class Voice-And-Video
     loss    2
     latency 45
     jitter  100

    -->>> DSCP to SLA Mapping
    from-vsmart app-route-policy _412898115_vpn_412898115
     vpn-list 412898115_vpn
      sequence 10
       match
        dscp 14
       action
        sla-class Default
        no sla-class strict
      sequence 20
       match
        dscp 18
       action
        sla-class Voice-And-Video
        no sla-class strict
      sequence 30
       match
        dscp 12
       action
        sla-class Transactional-Data
        no sla-class strict
      sequence 40
       match
        dscp 10
       action
        sla-class Bulk-Data
        no sla-class strict

    from-vsmart lists vpn-list 412898115_vpn
     vpn 10

    ASR1001-X-Branch#
    1. QoS verification

    Example 1

    WAN SLA Policy "Transactional-Data". Please navigate to Tenants-><tenant name>->Contracts->Standard-><Contract Name>-><Contract Subject>-> General- WAN SLA Policy

    WAN SLA Policy 1

      sequence 30
       match
        dscp 12
       action
        sla-class Transactional-Data
        no sla-class strict

         

    Direction:

    1. Traffic from DC to SDWAN.

    As can be seen in below captures, traffic originated from DC is with dscp 00 but the traffic reaching to SDWAN is with DSCP 12 (hex 0x0c).

    This indicates DSCP value change according to WAN SLA Policy.

    Packet capture performed at source (DC) reflecting original DSCP value to 00.

    Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

            0000 00.. = Differentiated Services Codepoint: Default (0x00)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa0d5 (41173)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 255

        Protocol: ICMP (0x01)

        Header checksum: 0x9016 [correct]

            [Good: True]

            [Bad : False]

        Source: 192.168.10.2 (192.168.10.2)

        Destination: 172.16.20.2 (172.16.20.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0xc16a [correct]

        Identifier: 0x4158

        Sequence number: 768 (0x0300)

        Data (56 bytes)

               

    Packet capture on destination (SDWAN Branch site) reflecting change in DSCP 12 (hex 0x0c) value according to WAN SLA Policy.

    Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x30 (DSCP 0x0c: Assured Forwarding 12; ECN: 0x00)

            0011 00.. = Differentiated Services Codepoint: Assured Forwarding 12 (0x0c)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa0d1 (41169)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 251

        Protocol: ICMP (0x01)

        Header checksum: 0x93ea [correct]

            [Good: True]

            [Bad : False]

        Source: 192.168.10.2 (192.168.10.2)

        Destination: 172.16.20.2 (172.16.20.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0x6e30 [correct]

        Identifier: 0xc057

        Sequence number: 1024 (0x0400)

        Data (56 bytes)

    2. Traffic from SDWAN to DC

    As can be seen in below captures, traffic originated from SDWAN Branch site is with dscp 00 but the traffic reaching to DC is with DSCP 12 (hex 0x0c) reflecting the change in DSCP value according to WAN SLA Policy applied.

    Packet capture performed at source (SDWAN Branch) reflecting original DSCP value to 00.

    Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

            0000 00.. = Differentiated Services Codepoint: Default (0x00)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa0c8 (41160)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 255

        Protocol: ICMP (0x01)

        Header checksum: 0x9023 [correct]

            [Good: True]

            [Bad : False]

        Source: 172.16.20.2 (172.16.20.2)

        Destination: 192.168.10.2 (192.168.10.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0xd3ff [correct]

        Identifier: 0x5c79

        Sequence number: 1 (0x0001)

        Data (56 bytes)

    Packet capture on destination (DC) reflecting change in DSCP 12 (hex 0x0c) value according to WAN SLA Policy.

    Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x30 (DSCP 0x0c: Assured Forwarding 12; ECN: 0x00)

            0011 00.. = Differentiated Services Codepoint: Assured Forwarding 12 (0x0c)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa073 (41075)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 251

        Protocol: ICMP (0x01)

        Header checksum: 0x9448 [correct]

            [Good: True]

            [Bad : False]

        Source: 172.16.20.2 (172.16.20.2)

        Destination: 192.168.10.2 (192.168.10.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0x741a [correct]

        Identifier: 0x5c79

        Sequence number: 43776 (0xab00)

        Data (56 bytes)

    Example 2

    WAN SLA Policy "Voice-And-Video"  Please navigate to Tenants-><tenant name>->Contracts->Standard-><Contract Name>-><Contract Subject>-> General- WAN SLA Policy

    WAN SLA Policy 2

      sequence 20
       match
        dscp 18
       action
        sla-class Voice-And-Video
        no sla-class strict

    1. Traffic from DC to SDWAN.

    As can be seen in below captures, traffic originated from DC is with DSCP 00 but the traffic reaching to SDWAN is with DSCP 18 (hex 0x12).

    This indicates DSCP value change according to WAN SLA Policy.

    Packet capture performed at source (DC) reflecting original DSCP value to 00.

    Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

            0000 00.. = Differentiated Services Codepoint: Default (0x00)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa2b6 (41654)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 255

        Protocol: ICMP (0x01)

        Header checksum: 0x8e35 [correct]

            [Good: True]

            [Bad : False]

        Source: 192.168.10.2 (192.168.10.2)

        Destination: 172.16.20.2 (172.16.20.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0x3614 [correct]

        Identifier: 0x8c5f

        Sequence number: 512 (0x0200)

        Data (56 bytes)

               

    Packet capture on destination (SDWAN Branch site) reflecting change in DSCP value 18 (0x12) matching it with WAN SLA Policy.

    Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x48 (DSCP 0x12: Assured Forwarding 21; ECN: 0x00)

            0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (0x12)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa2b8 (41656)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 255

        Protocol: ICMP (0x01)

        Header checksum: 0x8deb [correct]

            [Good: True]

            [Bad : False]

        Source: 172.16.20.2 (172.16.20.2)

        Destination: 192.168.10.2 (192.168.10.2)

    Internet Control Message Protocol

        Type: 0 (Echo (ping) reply)

        Code: 0 ()

        Checksum: 0x8a13 [correct]

        Identifier: 0x8c5f

        Sequence number: 1024 (0x0400)

        Data (56 bytes)

    2. Traffic from SDWAN to DC.

    Packet capture on source (SDWAN Branch) showing the original DSCP value (00).

    Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

            0000 00.. = Differentiated Services Codepoint: Default (0x00)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa1bb (41403)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 255

        Protocol: ICMP (0x01)

        Header checksum: 0x8f30 [correct]

            [Good: True]

            [Bad : False]

        Source: 172.16.20.2 (172.16.20.2)

        Destination: 192.168.10.2 (192.168.10.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0x68e5 [correct]

        Identifier: 0x1d03

        Sequence number: 2048 (0x0800)

        Data (56 bytes)

               

    Packet capture on destination (DC) reflecting change in DSCP value 18 (0x12) according to WAN SLA Policy.

    Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x48 (DSCP 0x12: Assured Forwarding 21; ECN: 0x00)

            0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (0x12)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0xa1bb (41403)

        Flags: 0x00

            0.. = Reserved bit: Not Set

            .0. = Don't fragment: Not Set

            ..0 = More fragments: Not Set

        Fragment offset: 0

        Time to live: 251

        Protocol: ICMP (0x01)

        Header checksum: 0x92e8 [correct]

            [Good: True]

            [Bad : False]

        Source: 172.16.20.2 (172.16.20.2)

        Destination: 192.168.10.2 (192.168.10.2)

    Internet Control Message Protocol

        Type: 8 (Echo (ping) request)

        Code: 0 ()

        Checksum: 0x68e5 [correct]

        Identifier: 0x1d03

        Sequence number: 2048 (0x0800)

        Data (56 bytes)

    Troubleshoot

    Following log files are useful from troubleshooting perspective. .

            Control path debugging

    APIC techsupport files
    PolicyDistributor Logs, PolicyManager Logs, PolicyElement, Edmgr logs can provide insight about relevant configuration getting pushed to leaves and spines.

    Data path debugging 

    Packet captures on L3out interface and interfaces on vEdge routers.

    ELAM can also help.

    Revision History

    Revision Publish Date Comments
    1.0
    06-Aug-2021
    Initial Release
    TAC Authored

    Contributed by Cisco Engineers

    • Bharatkumar Chaudhari
      ACI TAC

    Was this Document Helpful?

    FeedbackFeedback

    Contact Cisco

    This Document Applies to These Products