Enhanced Application-Aware Routing

Enhanced Application-Aware Routing in Cisco SD-WAN

A Enhanced Application-Aware Routing is a Cisco Catalyst SD-WAN capability that

  • speeds detection of tunnel performance issues

  • enables faster traffic redirection to meet SLA thresholds, and

  • reduces switchover time compared to default behavior.

Role of BFD in Performance Monitoring

Bidirectional Forwarding Detection (BFD) detects link failures and collects performance routing (PfR) data for SD-WAN tunnels (IPsec and GRE).

Each BFD hello packet gathers:

  • Latency: Round-trip time between BFD echo request and reply.

  • Jitter: Variation in packet arrival delay (irregular timing of packets).

  • Loss: Number of echo requests that do not receive replies.

With the default BFD hello timer of 1 second, the system collects one PfR sample every second.

Poll Interval and Data Averaging

The system collects PfR samples throughout the poll interval (default: 10 minutes).

During each poll interval, it calculates the average for latency, jitter, and loss.

To make dynamic path decisions based on application-aware routing SLAs, the system uses a default multiplier of 6.

This multiplier evaluates multiple poll-interval averages before making routing decisions.

A poll interval average represents the time between consecutive data-collection events and indicates how frequently the system samples network performance.

Convergence Time Considerations

Convergence time is the time required for the network to recover and resume normal operation after a disruption.

  • Default detection of slowly degrading WAN circuits: 10 minutes to 1 hour

  • With the lowest recommended settings (2-minute poll interval × 6 intervals): 2 to 12 minutes

Using very low poll intervals can cause false positives and traffic instability due to insufficient performance data.

Without enhanced application-aware routing enabled, Cisco IOS XE Catalyst SD-WAN devices require several minutes to switch traffic from one network path to another to meet SLA requirements when the loss, latency, and jitter exceed specific threshold values. Enabling enhanced application-aware routing speeds the detection of tunnel performance issues. This enables Cisco IOS XE Catalyst SD-WAN device to redirect traffic away from tunnels that do not meet SLA requirements.

Pfr measurements

PfR measurements provide insight into network performance by tracking loss, latency, and jitter using BFD.
Table 1. Pfr measurements

Metric

Source

Description

Loss

BFD

Measured as loss of BFD packet at 1pps or one packet in n_app_probe_class (n-apc) sec​

If the application probe class (APC) configuration is not set, BFD packet loss occurs at a rate of 1 packet per second (1 pps). With the APC configuration, packet loss reduces to 1 packet every N seconds.

For more information see, Application Probe Class.

Latency

BFD

RTT measurements 1 pps or one packet in n-apc sec​

Without the application probe class (APC) configuration, RTT packet loss occurs at a rate of 1 packet per second (1 pps). With the APC configuration, packet loss reduces to 1 packet every N seconds.

Jitter

BFD

Variation in RTT​

Application-aware routing design and measurement intervals

This section provides information about default intervals, calculation methods, and expert recommendations for application-aware routing design and measurement accuracy.
  • The default BFD hello-interval is 1 second, and the app-route/SLA poll-interval is 10 minutes.

  • The BFD hello-interval defines how often the BFD (Bidirectional Forwarding Detection) protocol sends hello packets to detect the liveliness of a network path. The app-route/SLA poll-interval defines how often the network monitoring system collects data or measures network metrics related to application routes or Service Level Agreements (SLAs).

  • By default, the system calculates to 60 minutes using 1 pps x 600 sec x 6 buckets. This calculation multiplies 1 packet per second (pps) by 600 seconds (10 minutes) and then by 6 buckets, resulting in a default poll-interval of 60 minutes.

  • Experts suggest using a poll-interval of 120 seconds (2 minutes) and a multiplier of 5, which produces a 10-minute interval. This recommendation helps achieve a specific monitoring frequency.

  • Reducing the poll-interval or multiplier improves detection time but can create false positives because PfR metrics rely on a small number of samples. Decreasing these values speeds up detection of network performance issues but increases the likelihood of false positives due to fewer data samples. You must balance detection time with the accuracy of PfR (Performance Routing) metrics.

  • To improve measurement accuracy at a faster rate, reduce the BFD hello-interval. Decreasing the interval at which hello packets are exchanged allows more frequent detection of network path liveliness and improves measurement accuracy.

Benefits of enhanced application-aware routing

This list provides the key benefits of enhanced application-aware routing:
  • Improved the PfR metrics: Inline data now improves PfR metric measurements (loss, latency, and jitter) by providing more accurate and detailed analysis. Cisco IOS XE Catalyst SD-WAN devices process and inspect this data directly at the network edge, enabling real-time inspection and decision-making.

  • Quick Enhanced-App-Route Detection and SLA Enforcement: Reduced the PfR poll interval to a minimum of 10 seconds to enhance App-Route detection and SLA enforcement. Cisco IOS XE Catalyst SD-WAN devices can now quickly detect circuit degradation and remove tunnels from SLA forwarding when thresholds are not met, ensuring efficient and reliable performance.

  • Speed: Increased the speed of SLA switch-over.

  • SLA Dampening: Introduced SLA dampening to enable smoother transitions back to SLA forwarding. Tunnels now undergo a dampening period before re-entering SLA forwarding to prevent disruptions and instability and to minimize negative effects on network performance.

  • Measurements: Enhanced measurement of loss, latency, and jitter.

Guidelines of enhanced application-aware routing

Guidelines of enhanced application-aware routing

  • Both GRE and IPSEC tunnels are supported.

  • All existing TLOCs and WAN interface types, including physical, sub interface, loopback bind, dialer, and LTE interfaces, are supported.

  • TLOC Extension tunnels are supported.

  • Both IPv4 and IPv6 underlay tunnels are supported.

  • SLA update and switchover occur at a minimum interval of 10 seconds.

  • Does not impact tunnel scale and maintains minimal impact on memory and performance.

  • Provides support with and without app-probe class configuration in SLA classes. For more information on app-probe class, see Application Probe Class.

  • SLA dampening is supported.

Compatibility with Cisco IOS XE catalyst SD-WAN devices without enhanced application-aware routing

Explains compatibility for Cisco IOS XE Catalyst SD-WAN devices when enhanced application-aware routing (EAAR) is not enabled or not available.
  1. In these scenarios:

    • On the local side: When the Cisco IOS XE Catalyst SD-WAN device is upgraded to Cisco IOS XE Catalyst SD-WAN Release 17.12.1a and later and has EAAR (Enhanced Application-Aware Routing) enabled.

    • On the remote side: When the Cisco IOS XE Catalyst SD-WAN device is not upgraded to Cisco IOS XE Catalyst SD-WAN Release 17.12.1a and the EAAR is not enabled.

    It falls back to using BFD-based measurements where support for older releases or disabled features is required.

  2. If both the local and remote sides are using Cisco IOS XE Catalyst SD-WAN Release 17.12.1a but the EAAR feature is not enabled, it reverts to using BFD-based measurements.


Note


The EAAR feature is disabled by default to support existing deployments.


Restrictions for enhanced application-aware routing

Restrictions for enhanced application-aware routing

  • The branch device on which you enable this feature does not support loopback unbind mode. The loopback unbind mode refers to a network interface configuration in which the loopback device is disconnected from the network stack.

  • GRE tunnels do not support per-queue measurement. Per-queue measurement monitors and analyzes network traffic on a per-queue basis by measuring and collecting metrics and statistics for each individual queue in a network device or system. A queue acts as a buffer where packets are stored before transmission or processing.

Prerequisites for enhanced application-aware routing

To enable application-aware routing on a Cisco IOS XE Catalyst SD-WAN device, enable enhanced application-aware routing on both the Cisco IOS XE Catalyst SD-WAN devices.

Supported devices for enhanced application-aware routing

Cisco IOS XE Catalyst SD-WAN device

Configure enhanced application-aware routing

The procedures in this section describe how to deploy the enhanced app-aware routing configurations from Cisco Catalyst SD-WAN Manager to Cisco IOS XE Catalyst SD-WAN devices.

Use one of these methods to configure enhanced application-aware routing:

Configure enhanced application-aware routing using a configuration group

Use these steps to configure enhanced application-aware routing (EAAR) using a configuration group in Cisco SD-WAN Manager.

Procedure


Step 1

From the Cisco SD-WAN Manager menu, choose Configuration > Configuration Groups.

Step 2

Choose a configuration group.

Step 3

Under Actions, click Edit.

Step 4

Under Feature Profiles, click System Profile.

Step 5

Choose basic and under Actions, click Edit Feature.

Step 6

In the Edit Basic Feature page, use the Enhanced App-Route field and choose one of the modes below:

Table 2. Enhanced app-route modes and parameters

Mode

EAAR poll interval

EAAR poll multiplier

EAAR poll window

SLA dampening multiplier

SLA dampening window

Aggressive

10s

6

10s - 60s

120

20 mins

Moderate

60s

5

60s - 300s

40

40 mins

Conservative

300s

6

300s - 1800s

12

60 mins

Step 7

Click Save.


Configure enhanced application-aware routing using CLI commands

Use these steps to configure enhanced application-aware routing using CLI commands to enforce SLAs and optimize routing decisions based on application performance metrics.

Procedure


Step 1

Enable enhanced PfR measurements for SLA enforcement.

Example:


bfd enhanced-app-route enable
          

Enabling the application-aware routing feature on a Cisco IOS XE Catalyst SD-WAN device requires you to enable the PfR CLI on both the remote and local routers.

  • The remote Cisco IOS XE Catalyst SD-WAN device must provide loss statistics to the local Cisco IOS XE Catalyst SD-WAN device.

  • The local Cisco IOS XE Catalyst SD-WAN device utilizes these metrics to enforce SLAs.

Step 2

Modify poll interval and multiplier settings for SLA enforcement and switchover.

Example:


bfd enhanced-app-route pfr-poll-interval
bfd enhanced-app-route pfr-multiplier <number>
          

The default poll-interval is 10 seconds and multiplier is 6 for SLA enforcement and switchover. The aggressive mode setting for app route pfr multiplier is 6 by default; it is 5 for moderate mode.

Use the bfd enhanced-app-route ignore-local-loss command to ignore local loss on the router caused by QoS queues.

By default, loss includes both local and WAN loss.

Step 3

Configure SLA dampening time and multiplier.

Example:


bfd sla-dampening enable
bfd sla-dampening multiplier <number>
          

SLA dampening time is the waiting period before returning the tunnel to SLA buckets after meeting the SLA. The default time is 120 seconds. The aggressive mode setting for dampening multiplier is 120 by default.


Configure enhanced application-aware routing using templates

Procedure


Step 1

From the Cisco SD-WAN Manager menu, choose Configuration > Templates .

Step 2

Click Feature Templates.

Step 3

Click Add Template.

Step 4

Choose a device and click the Cisco System template under Basic Information.

Step 5

In the Enhanced App-Aware Routing field, click Global from the drop-down list and choose one of the modes:

Table 3. Enhanced app-aware routing modes and parameters

Mode

EAAR poll interval

EAAR poll multiplier

EAAR poll window

SLA dampening multiplier

SLA dampening window

Aggressive

10s

6

10s - 60s

120

20 mins

Moderate

60s

5

60s - 300s

40

40 mins

Conservative

300s

6

300s - 1800s

12

60 mins

Note

 

You can configure the enhanced application aware routing (EAAR) poll interval, poll multiplier, and SLA dampening multiplier only through a CLI template.

Step 6

Click Save.


Verify enhanced application-aware routing configuration

To verify the enhanced application-aware routing configuration and display the configured parameters for EAAR, use the show sdwan app-route params command.


Device# show sdwan app-route params
*EAAR = Enhanced Application-Aware Routing
Config:                  :Enabled   
Poll interval:           :10000     
Poll multiplier:         :6         

App route 
Poll interval:           :600000    
Poll multiplier:         :6         

SLA dampening  
Config:                  :Enabled   
Multiplier:              :120   
      

You can use the show sdwan bfd sessions alt command to highlight the flags for EAAR.


Device# show sdwan bfd sessions alt
*Sus = Suspend
*GREinUDP = GREinUDP encap
*EAAR  = Enhanced Application-Aware Routing
*NA  = Flag Not Set
 
                                                      SOURCE TLOC      REMOTE TLOC                                  DST PUBLIC     DST PUBLIC                                               
SYSTEM IP        SITE ID   STATE  COLOR                  COLOR                   SOURCE IP         IP                      PORT        ENCAP  BFD-LD    FLAGS       UPTIME          
-------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.0.0    100       up          lte                           lte                           10.0.0.0            10.0.0.1         12367       ipsec      20013     NA            0:07:48:38      
172.16.0.1    100       up          lte                           lte                           10.0.0.0            10.0.0.1         12377       ipsec      20014     NA            0:07:48:39      
172.16.0.0    400       up          lte                           lte                           10.0.0.0            10.0.0.1         12366       ipsec      20015     NA            0:07:48:39      
172.16.0.1    500       up          lte                           lte                           10.0.0.0            10.0.0.1         12366       ipsec      20016     EAAR          0:07:48:39
      

You can use show sdwan app-route stats summary command to display the app-route (PfR) statistics details for each tunnel, across different intervals of measurements, for every configured APC.


Note


The total-packets field in the show sdwan app-route stats summary output displays values in multiples of 64 or 0 and includes both application packets and BFD probes. For per-interval application packet counts, refer to the tx-data-pkts and rx-data-pkts fields.



Device# show sdwan app-route stats summary   
app-route statistics 10.0.0.0 10.0.0.0 ipsec 12366 12367
 remote-system-ip         172.16.0.0
 local-color              lte
 remote-color             lte
 sla-class-index          0,1,2,3
 fallback-sla-class-index None
 enhanced-app-route       Enabled
 sla-dampening-index      4,5
 app-probe-class-list None
  mean-loss    0
  mean-latency 0
  mean-jitter  0
              TOTAL                      AVERAGE  AVERAGE     TX DATA     RX DATA    IPV6 TX        IPV6 RX
INDEX   PACKETS  LOSS     LATENCY  JITTER            PKTS          PKTS          DATA PKTS  DATA PKTS
--------------------------------------------------------------------------------------------------------------------------------------------------
0           664              0             0                0                       0                 0                 0                       0
1           663              0             0                0                       0                 0                 0                       0
2           666              0             0                0                       0                 0                 0                       0
3           664              0             0                0                       0                 0                 0                       0
4           662              0             0                0                       0                 0                 0                       0
5           664              0             0                0                       0                 0                 0                       0
      

Monitor enhanced application-aware routing using Cisco Catalyst SD-WAN Manager

Procedure


Step 1

From the Cisco Catalyst SD-WAN Manager menu, choose Monitor > Devices.

Step 2

Under Devices, choose a device.

Step 3

Click Real Time in the left pane.

Step 4

In the Device Options field, choose App Routes Statistics.

The following image shows the App Routes Statistics view:


Enhanced application-aware routing troubleshooting

This section provides troubleshooting steps and example command outputs for enhanced application-aware routing.

From the device:


Device# show sdwan run | include enhanced-app-route
bfd enhanced-app-route enable
bfd enhanced-app-route pfr-poll-interval 10000
bfd enhanced-app-route pfr-multiplier 6

show sdwan run | inc sla-dampening
bfd sla-dampening enable
bfd sla-dampening multiplier 12
      

Device# show sdwan app-route params
Enhanced app route
  Config:                  :Enabled <<< Enhanced app-aware routing enabled 
    Poll interval:           :10000
    Poll multiplier:         :6
App route
    Poll interval:           :600000
    Poll multiplier:         :6
SLA dampening
    Config:                  :Enabled
    Multiplier:              :120
      

Device# show platform hardware qfp active feature sdwan datapath pathmon summary
Src IP          Dst IP          Src Port Dst Port   Encap    Uidb     Bfd Discrim PathMon
------          ------          -------- -------    ------   -------  ----------- ------
10.0.0.0      10.0.0.1      12346    12366      IPSEC    65527    20003       in/out
      

Device# show sdwan bfd sessions alt
*Sus = Suspend
*GREinUDP = GREinUDP encap
*EAAR  = Enhanced Application-Aware Routing
*NA  = Flag Not Set
                                       SOURCE TLOC      REMOTE TLOC                                      DST PUBLIC                      DST PUBLIC
SYSTEM IP        SITE ID   STATE       COLOR            COLOR            SOURCE IP                       IP                              PORT        ENCAP  BFD-LD    FLAGS             UPTIME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.0.0        100       down        private1         lte              10.0.0.0                      10.0.0.1                      12367       ipsec  20011     EAAR                NA
172.16.0.1        500       down        private1         3g               10.0.0.0                      10.0.0.1                      12366       ipsec  20013     EAAR                NA
172.16.0.0        600       down        private1         3g               10.0.0.0                      10.0.0.1                      12366       ipsec  20007     EAAR                NA
      

Device# show sdwan app-route stats remote-system-ip 172.16.0.0 app-route statistics 10.0.0.0 10.0.0.1 ipsec 12366 12366
remote-system-ip         172.16.0.0        
local-color              private1
remote-color             3g
sla-class-index          0
fallback-sla-class-index None
enhanced-app-route       Enabled
sla-dampening-index      None