Contents

Introduction

Prerequisites

Requirements

Components Used

Background Information

Passive Monitoring

Active Monitoring

Hybrid Mode

Configure

Network Diagram

Relevant Configuration

Verify

Passive Mode

Active Mode

Hybrid Mode

Troubleshoot

Introduction

This document describes the methods used in Performance Routing version 2 (PfRv2) to monitor the performance of the Wide Area Network (WAN) links on the branch routers.

Prerequisites

Requirements

Cisco recommends that you have basic knowledge of Performance Routing (PfR).

Components Used

This document is not restricted to specific software and hardware versions.

Note: PFRv2 is not supported in Polaris code 16.x.x.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Background Information

PfRv2 uses three methods to measure performance of the Border Routers (BRs) links. The information collected is used by Master Controller (MC) for PfR policy implementation. The three methods are passive monitoring, active monitoring, and hybrid mode.

Passive Monitoring

In this mode, Netflow enabled (by default with PfR) on the BR collects this information regarding the traffic class and sends it back to the Master controller.

This information is applicable for TCP flows passing through BR:

- Reachability: This is calculated based on TCP SYN for which corresponding TCP ACK have not being received.

- Delay: Time calculated between TCP SYN and TCP ACK messages during TCP 3-way handshake. The total value is then divided by two.

- Loss: Measured on the basis of TCP sequence numbers. For example, when the received TCP sequence number is higher or lower than the expected, loss is reported.

This information is applicable for all flows (which include TCP) passing through BR:

- Egress Bandwidth: Throughput for traffic class egressing the BRs (calculated in bits per second using Netflow).

- Ingress Bandwidth: Throughput for traffic class ingressing the BRs (calculated in bits per second using Netflow).

Active Monitoring

In this mode, the BR sends out IP SLA probes over its WAN interface to measure several parameters regarding the traffic class. Information collected is sent back to the Master controller. These parameters are measured:

- Reachability

- Delay

- Loss

- Egress Bandwidth

- Ingress Bandwidth

These probes are generated automatically when the monitoring method configured on the Master controller is Active and can also be configured manually. By default, the probes sent are ICMP echo but can be changed to TCP or UDP probes depending upon the type of traffic being sent over the WAN links.

While the Exit BR selection is ongoing, all the BRs will send active probes for the Netflow-learned prefixes. Upon selection of the Exit BR, other BRs will stop sending active probes. The selected BR will continue to send active probes.

Hybrid Mode

Hybrid mode uses both the Netflow statistics and IP Service Level Agreement (SLA) to decide on exit point (BR) and link monitoring. In this mode, IP SLA probe information is used to select the exit point and then Netflow statistics are used to monitor that BR's WAN connection towards the destination.

While PfR is in the learning state and has not yet moved into the “INPOLICY” state, all the BRs will send active probes for the prefixes collected from Netflow. This is to determine respective link conditions. When an MC state changes to “INPOLICY”, all the BRs will stop sending active probes and now monitoring will be done passively (using Netflow).

Configure

This image can be used as a sample topology for rest of the document:

Network Diagram

Relevant Configuration

This basic configuration is required for using different modes. R3 is configured as MC so these configurations will have to be completed on R3:

Passive Mode

pfr master

!

border 10.4.4.4 key-chain pfr

interface Ethernet0/1 external

interface Ethernet0/0 internal

!

border 10.5.5.5 key-chain pfr

interface Ethernet0/0 internal

interface Ethernet0/1 external

!

mode monitor passive

Active Mode

pfr master

!

border 10.4.4.4 key-chain pfr

interface Ethernet0/1 external

interface Ethernet0/0 internal

!

border 10.5.5.5 key-chain pfr

interface Ethernet0/0 internal

interface Ethernet0/1 external

!

mode monitor active

Hybrid Mode

This is the default mode. If no mode command is mentioned, Hybrid mode will be activated or the command mode monitor both can be used to enable it.

pfr master

!

border 10.4.4.4 key-chain pfr

interface Ethernet0/1 external

interface Ethernet0/0 internal

!

border 10.5.5.5 key-chain pfr

interface Ethernet0/0 internal

interface Ethernet0/1 external

Note: If the command mode monitor both is given manually, then it will not be shown in the configuration as it is a default command.

Verify

Most verification commands are executed on MC. These commands can be used to verify working of different modes.

Passive Mode

R3#show pfr master

<Output suppressed>

Default Policy Settings:

backoff 90 900 90

delay relative 50

holddown 90

periodic 0

probe frequency 56

number of jitter probe packets 100

mode route control

mode monitor passive

loss relative 10

jitter threshold 20

mos threshold 3.60 percent 30

unreachable relative 50

trigger-log percentage 30

Test 1 - Initiate TCP Stream From the Server

R3#show pfr master traffic-class

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix

Flags             State     Time            CurrBR  CurrI/F Protocol

PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw

ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos

--------------------------------------------------------------------------------

10.20.20.0/24              N    N    N           N           N N

INPOLICY        0          10.4.4.4 Et0/1           BGP

46       46        0        0    35502    35502        2        1

N        N        N        N        N        N

10.30.30.0/24              N    N    N           N           N N

INPOLICY        0          10.5.5.5 Et0/1           BGP

1        1        0        0        0        0       14        1

N        N        N        N        N        N

Test 2 - Initiate UDP Stream From the Server

R3#show pfr master traffic-class

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix

Flags             State     Time            CurrBR  CurrI/F Protocol

PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw

ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos

--------------------------------------------------------------------------------

10.20.20.0/24               N    N    N           N           N N

INPOLICY        0          10.5.5.5 Et0/1           BGP

U        U        0        0        0        0        13        0

N        N        N        N        N        N

10.30.30.0/24               N    N    N           N           N N

INPOLICY        0          10.5.5.5 Et0/1           BGP

U        U        0        0        0        0        14        0

N        N        N        N        N        N

As shown previously, for TCP traffic, you can see Delay and Unreachable counters also getting populated but in case of UDP stream you can only see Bandwidth counters getting populated.

Active Mode

R3#show pfr master

<Output suppressed>

Default Policy Settings:

backoff 90 900 90

delay relative 50

holddown 90

periodic 0

probe frequency 56

number of jitter probe packets 100

mode route control

mode monitor active

loss relative 10

jitter threshold 20

mos threshold 3.60 percent 30

unreachable relative 50

trigger-log percentage 30

Test - Initiate TCP Stream From the Server

On Master Controller:

R3#show pfr master traffic-class

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix

Flags             State     Time            CurrBR  CurrI/F Protocol

PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw

ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos

--------------------------------------------------------------------------------

10.10.20.0/24              N    N    N           N           N N

INPOLICY        0          10.4.4.4 Et0/1           BGP

N        N        N        N        N        N        N        N

54       54        0        0        N        N        N        N

10.30.30.0/24              N    N    N           N           N N

INPOLICY        0         10.4.4.4 Et0/1           BGP

N        N        N        N        N        N        N        N

54       54        0     1000        N        N        N        N

On BR1:

R4#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

echo    10.10.20.11         N 192.168.1.1      Et0/1                 3       3

0

echo    10.30.30.12         N 192.168.1.1      Et0/1                 3       3

0

On BR2:

R5#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source        Interface           Att   Comps

DSCP

echo     10.10.20.11       N 192.168.2.1      Et0/1                 3       3

0

echo     10.30.30.12       N 192.168.2.1      Et0/1                 3       3

0

Once the Traffic classes on MC move into “INPOLICY” state and BR1 is selected as the BR for sending all traffic, BR2 will stop sending probes :

R4#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

echo    10.10.20.11         N 192.168.1.1      Et0/1                10      10

0

echo    10.30.30.12        N 192.168.1.1      Et0/1                10      10

0

R5#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

<No Active Probes>

Hybrid Mode

R3#show pfr master

OER state: ENABLED and ACTIVE

<Output Suppressed>

Default Policy Settings:

backoff 90 900 90

delay relative 50

holddown 90

periodic 0

probe frequency 56

number of jitter probe packets 100

mode route control

mode monitor both

loss relative 10

jitter threshold 20

mos threshold 3.60 percent 30

unreachable relative 50

trigger-log percentage 30

Test - Initiate TCP Stream From the Server

While Traffic Class (TC) is being measured and the state is not yet “INPOLICY”, both the BRs will send active probes to the prefixes collected from Netflow. This is to determine respective link conditions.

On MC:

R3#show pfr mas traffic-class

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix

Flags             State     Time            CurrBR  CurrI/F Protocol

PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw

ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos

--------------------------------------------------------------------------------

10.20.20.0/24              N    N    N           N           N N

HOLDDOWN       61          10.5.5.5Et0/1           BGP

1        1        0        0        0        0       16        1

1        1        0        0        N        N        N        N

10.30.30.0/24             N    N    N           N           N N

HOLDDOWN       61         10.5.5.5 Et0/1           BGP

1        1        0        0        0        0       16        1

4        4        0        0        N        N        N        N

On BR1:

R4#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

echo     10.20.20.1          N 192.168.1.1      Et0/1                 1       1

0

echo     10.30.30.1          N 192.168.1.1     Et0/1                 1       1

0

On BR2:

R5#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

echo     10.20.20.1          N 192.168.2.1      Et0/1                 1       1

0

echo     10.30.30.1          N 192.168.2.1      Et0/1                 1       1

When on MC the state changes to “INPOLICY”, both the BRs will stop sending active probes and the respective monitoring will switch to passive mode (using Netflow).

R3#show pfr master traffic-class

OER Prefix Statistics:

Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),

P - Percentage below threshold, Jit - Jitter (ms),

MOS - Mean Opinion Score

Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),

E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable

U - unknown, * - uncontrolled, + - control more specific, @ - active probe all

# - Prefix monitor mode is Special, & - Blackholed Prefix

% - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix

Flags             State     Time            CurrBR  CurrI/F Protocol

PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw

ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos

--------------------------------------------------------------------------------

10.20.20.0/24              N    N    N           N           N N

INPOLICY        0          10.5.5.5 Et0/1           BGP

1        1        0        0        0        0        3        1

1        1        0        0       N        N        N        N

10.30.30.0/24           N    N    N           N           N N

INPOLICY        0          10.5.5.5 Et0/1           BGP

1        1        0        0        0        0       14        1

1        1        0        0        N        N        N        N

As shown, you can see counters for both the Passive and Active components. Also, the probes will stop on BRs once the TCs move to the "INPOLICY" state.

R4#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

<No Active Probes>

R5#show pfr border active-probes

OER Border active-probes

Type      = Probe Type

Target    = Target IP Address

TPort     = Target Port

Source    = Send From Source IP Address

Interface = Exit interface

Att       = Number of Attempts

Comps   = Number of completions

N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps

DSCP

<No Active Probes>

Note: PfRv2 is not supported in Releases 15.6(3)M, 15.7(3)M, and later T-train releases. Also Release 16.3.1 has PfRv2 CLIs, but functionality is not supported. Functionality was broken when the code moved from MCP to Polaris, and this will not be fixed in Polaris releases.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.