Configure PfRv2 Performance Monitoring Methods

Introduction

This document describes the methods used in Performance Routing version 2 (PfRv2) to monitor the performance of the Wide Area Network (WAN) links on the branch routers.

Prerequisites

Requirements

Cisco recommends that you have basic knowledge of Performance Routing (PfR).

Components Used

This document is not restricted to specific software and hardware versions.

Note: PFRv2 is not supported in Polaris code 16.x.x.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Background Information

PfRv2 uses three methods to measure performance of the Border Routers (BRs) links. The information collected is used by Master Controller (MC) for PfR policy implementation. The three methods are passive monitoring, active monitoring, and hybrid mode.

Passive Monitoring

In this mode, Netflow enabled (by default with PfR) on the BR collects this information regarding the traffic class and sends it back to the Master controller.

This information is applicable for TCP flows passing through BR:

• Reachability: This is calculated based on TCP SYN for which corresponding TCP ACK have not being received.
• Delay: Time calculated between TCP SYN and TCP ACK messages during TCP 3-way handshake. The total value is then divided by two.
• Loss: Measured on the basis of TCP sequence numbers. For example, when the received TCP sequence number is higher or lower than the expected, loss is reported.

This information is applicable for all flows (which include TCP) passing through BR:

• Egress Bandwidth: Throughput for traffic class egressing the BRs (calculated in bits per second using Netflow).
• Ingress Bandwidth: Throughput for traffic class ingressing the BRs (calculated in bits per second using Netflow).

Active Monitoring

In this mode, the BR sends out IP SLA probes over its WAN interface to measure several parameters regarding the traffic class. Information collected is sent back to the Master controller. These parameters are measured:

• Reachability
• Delay
• Loss
• Egress Bandwidth
• Ingress Bandwidth

These probes are generated automatically when the monitoring method configured on the Master controller is Active and can also be configured manually. By default, the probes sent are ICMP echo but can be changed to TCP or UDP probes depending upon the type of traffic being sent over the WAN links.

While the Exit BR selection is ongoing, all the BRs will send active probes for the Netflow-learned prefixes. Upon selection of the Exit BR, other BRs will stop sending active probes. The selected BR will continue to send active probes.

Hybrid Mode

Hybrid mode uses both the Netflow statistics and IP Service Level Agreement (SLA) to decide on exit point (BR) and link monitoring. In this mode, IP SLA probe information is used to select the exit point and then Netflow statistics are used to monitor that BR's WAN connection towards the destination.

While PfR is in the learning state and has not yet moved into the “INPOLICY” state, all the BRs will send active probes for the prefixes collected from Netflow. This is to determine respective link conditions. When an MC state changes to “INPOLICY”, all the BRs will stop sending active probes and now monitoring will be done passively (using Netflow).

Configure

This image can be used as a sample topology for rest of the document:

Network Diagram

Relevant Configuration

This basic configuration is required for using different modes. R3 is configured as MC so these configurations will have to be completed on R3:

Passive Mode

pfr master
 !
 border 10.4.4.4 key-chain pfr
 interface Ethernet0/1 external
 interface Ethernet0/0 internal
 !
 border 10.5.5.5 key-chain pfr
 interface Ethernet0/0 internal
 interface Ethernet0/1 external
 !
 mode monitor passive

Active Mode

pfr master
 !
 border 10.4.4.4 key-chain pfr
 interface Ethernet0/1 external
 interface Ethernet0/0 internal
 !
 border 10.5.5.5 key-chain pfr
 interface Ethernet0/0 internal
 interface Ethernet0/1 external
 !
 mode monitor active

Hybrid Mode

This is the default mode. If no mode command is mentioned, Hybrid mode will be activated or the command mode monitor both can be used to enable it.

pfr master
 !
 border 10.4.4.4 key-chain pfr
 interface Ethernet0/1 external
 interface Ethernet0/0 internal
 !
 border 10.5.5.5 key-chain pfr
 interface Ethernet0/0 internal
 interface Ethernet0/1 external 

Note: If the command mode monitor both is given manually, then it will not be shown in the configuration as it is a default command.

Verify

Most verification commands are executed on MC. These commands can be used to verify working of different modes.

Passive Mode

R3#show pfr master
<Output suppressed>
Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 0
  probe frequency 56
  number of jitter probe packets 100
  mode route control 
  mode monitor passive
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30

Test 1 - Initiate TCP Stream From the Server

R3#show pfr master traffic-class
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.20.20.0/24              N    N    N           N           N N                 
                          INPOLICY        0          10.4.4.4 Et0/1           BGP     
               46       46        0        0    35502    35502        2        1
               N        N        N        N        N        N

10.30.30.0/24              N    N    N           N           N N                 
                          INPOLICY        0          10.5.5.5 Et0/1           BGP     
               1        1        0        0        0        0       14        1
               N        N        N        N        N        N

Test 2 - Initiate UDP Stream From the Server

R3#show pfr master traffic-class 
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.20.20.0/24               N    N    N           N           N N                 
                          INPOLICY        0          10.5.5.5 Et0/1           BGP     
               U        U        0        0        0        0        13        0
               N        N        N        N        N        N

10.30.30.0/24               N    N    N           N           N N                 
                          INPOLICY        0          10.5.5.5 Et0/1           BGP     
               U        U        0        0        0        0        14        0
               N        N        N        N        N        N

As shown previously, for TCP traffic, you can see Delay and Unreachable counters also getting populated but in case of UDP stream you can only see Bandwidth counters getting populated.

Active Mode

R3#show pfr master
<Output suppressed>
Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 0
  probe frequency 56
  number of jitter probe packets 100
  mode route control 
  mode monitor active
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30

Test - Initiate TCP Stream From the Server

On Master Controller:

R3#show pfr master traffic-class
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.10.20.0/24              N    N    N           N           N N                 
                          INPOLICY        0          10.4.4.4 Et0/1           BGP     
               N        N        N        N        N        N        N        N
              54       54        0        0        N        N        N        N

10.30.30.0/24              N    N    N           N           N N                 
                          INPOLICY        0         10.4.4.4 Et0/1           BGP     
               N        N        N        N        N        N        N        N
               54       54        0     1000        N        N        N        N

On BR1:

R4#show pfr border active-probes                 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP 
echo    10.10.20.11         N 192.168.1.1      Et0/1                 3       3
0     
echo    10.30.30.12         N 192.168.1.1      Et0/1                 3       3
0     

On BR2:

R5#show pfr border active-probes                 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source        Interface           Att   Comps
DSCP 
echo     10.10.20.11       N 192.168.2.1      Et0/1                 3       3
0     
echo     10.30.30.12       N 192.168.2.1      Et0/1                 3       3
0 

Once the Traffic classes on MC move into “INPOLICY” state and BR1 is selected as the BR for sending all traffic, BR2 will stop sending probes :

R4#show pfr border active-probes                 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP 
echo    10.10.20.11         N 192.168.1.1      Et0/1                10      10
0     
echo    10.30.30.12        N 192.168.1.1      Et0/1                10      10
0     

R5#show pfr border active-probes                 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP

<No Active Probes>

Hybrid Mode

R3#show pfr master 
OER state: ENABLED and ACTIVE
<Output Suppressed>
Default Policy Settings:
  backoff 90 900 90
  delay relative 50
  holddown 90
  periodic 0
  probe frequency 56
  number of jitter probe packets 100
  mode route control 
  mode monitor both
  loss relative 10
  jitter threshold 20
  mos threshold 3.60 percent 30
  unreachable relative 50
  trigger-log percentage 30

Test - Initiate TCP Stream From the Server

While Traffic Class (TC) is being measured and the state is not yet “INPOLICY”, both the BRs will send active probes to the prefixes collected from Netflow. This is to determine respective link conditions.

On MC:

R3#show pfr mas traffic-class 
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.20.20.0/24              N    N    N           N           N N                 
                         HOLDDOWN       61          10.5.5.5Et0/1           BGP     
               1        1        0        0        0        0       16        1
               1        1        0        0        N        N        N        N

10.30.30.0/24             N    N    N           N           N N                 
                          HOLDDOWN       61         10.5.5.5 Et0/1           BGP     
               1        1        0        0        0        0       16        1
               4        4        0        0        N        N        N        N

On BR1:

R4#show pfr border active-probes 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP 
echo     10.20.20.1          N 192.168.1.1      Et0/1                 1       1
0     
echo     10.30.30.1          N 192.168.1.1     Et0/1                 1       1
0     

On BR2:

R5#show pfr border active-probes 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP 
echo     10.20.20.1          N 192.168.2.1      Et0/1                 1       1
0     
echo     10.30.30.1          N 192.168.2.1      Et0/1                 1       1

When on MC the state changes to “INPOLICY”, both the BRs will stop sending active probes and the respective monitoring will switch to passive mode (using Netflow).

R3#show pfr master traffic-class
OER Prefix Statistics:
 Pas - Passive, Act - Active, S - Short term, L - Long term, Dly - Delay (ms),
 P - Percentage below threshold, Jit - Jitter (ms), 
 MOS - Mean Opinion Score
 Los - Packet Loss (percent/10000), Un - Unreachable (flows-per-million),
 E - Egress, I - Ingress, Bw - Bandwidth (kbps), N - Not applicable
 U - unknown, * - uncontrolled, + - control more specific, @ - active probe all
 # - Prefix monitor mode is Special, & - Blackholed Prefix
 % - Force Next-Hop, ^ - Prefix is denied

DstPrefix           Appl_ID Dscp Prot     SrcPort     DstPort SrcPrefix         
           Flags             State     Time            CurrBR  CurrI/F Protocol
         PasSDly  PasLDly   PasSUn   PasLUn  PasSLos  PasLLos      EBw      IBw
         ActSDly  ActLDly   ActSUn   ActLUn  ActSJit  ActPMOS  ActSLos  ActLLos
--------------------------------------------------------------------------------
10.20.20.0/24              N    N    N           N           N N                 
                          INPOLICY        0          10.5.5.5 Et0/1           BGP     
                1        1        0        0        0        0        3        1
              1        1        0        0       N        N        N        N

10.30.30.0/24           N    N    N           N           N N                 
                          INPOLICY        0          10.5.5.5 Et0/1           BGP     
               1        1        0        0        0        0       14        1
               1        1        0        0        N        N        N        N

As shown, you can see counters for both the Passive and Active components. Also, the probes will stop on BRs once the TCs move to the "INPOLICY" state.

R4#show pfr border active-probes 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP 

<No Active Probes>

R5#show pfr border active-probes 
        OER Border active-probes
Type      = Probe Type
Target    = Target IP Address
TPort     = Target Port
Source    = Send From Source IP Address
Interface = Exit interface
Att       = Number of Attempts
Comps   = Number of completions
N - Not applicable

Type     Target          TPort Source          Interface           Att   Comps
DSCP

<No Active Probes>

Note: PfRv2 is not supported in Releases 15.6(3)M, 15.7(3)M, and later T-train releases. Also Release 16.3.1 has PfRv2 CLIs, but functionality is not supported. Functionality was broken when the code moved from MCP to Polaris, and this will not be fixed in Polaris releases.

Troubleshoot

There is currently no specific troubleshooting information available for this configuration.