Contents

Introduction

Background Information

Problem: ASR1002 platform limitation with IPSec, Netflow, NBAR

Configuration

Observations

Solution

Introduction

This document descibes the problem with throughput on ASR1002 platform with Application Visibility and Control (AVC) configured along with IPSec feature on the router.

Background Information

As per CCO documentation, ASR10002 provides 10 gbps throughput for normal data traffic, 4 Gbps with IPSec feature enabled. But there is a caveat attached to the throughput on ASR1002 platform. Netflow and NBAR are two features which consumes a lot of resources from Quantum Flow Processor (QFP) and thus reduces the cabability of the Encapsulating Security Payload (ESP) card to process more traffic and thus reducing the overall system throughput. With AVC configuration along with IPSec, the overall platform throughput can be severely degraded and can face huge traffic loss.

Problem: ASR1002 platform limitation with IPSec, Netflow, NBAR

The problem was initial noticed when the bandwidth was upgraded with the provider and bandwidth testing was being perform. Initially 1000 bytes packet was sent, which went perfectly fine, then the testing was performed with 512 byte packets after which they nearly noticed 80% traffic loss. Refer to this lab testing topology:

Run these features:

- DMVPN over IPSec

- Netflow

- NBAR (as part of QoS policy match statement)

Configuration

crypto isakmp policy 1

encr 3des

group 2

crypto isakmp policy 2

encr 3des

authentication pre-share

group 2

crypto isakmp key cisco123 address 0.0.0.0

crypto ipsec security-association replay disable

crypto ipsec transform-set remoteoffice-vpn esp-3des esp-sha-hmac

mode tunnel

crypto ipsec transform-set IPTerm-TransSet esp-3des esp-sha-hmac

mode tunnel

crypto ipsec profile IPTerminals-VPN

set transform-set IPTerm-TransSet

crypto ipsec profile vpn-dmvpn

set transform-set remoteoffice-vpn

!

<snip>

class-map match-any Test

match ip precedence 2

match ip dscp af21

match ip dscp af22

match ip dscp af23

match access-group name test1

match protocol ftp

match protocol secure-ftp

!

policy-map test

<snip>

!

interface Tunnel0

bandwidth 512000

ip vrf forwarding CorpnetVPN

ip address 10.1.1.1 255.255.255.0

no ip redirects

ip mtu 1350

ip flow ingress

ip nhrp authentication 1dcBb

ip nhrp map multicast dynamic

ip nhrp network-id 1000

ip nhrp holdtime 600

ip nhrp shortcut

ip nhrp redirect

ip virtual-reassembly max-reassemblies 256

ip tcp adjust-mss 1310

ip ospf network point-to-multipoint

ip ospf hello-interval 3

ip ospf prefix-suppression

load-interval 30

qos pre-classify

tunnel source Loopback0

tunnel mode gre multipoint

tunnel key 1234

tunnel protection ipsec profile vpn-dmvpn

!

int gi 0/1/0

bandwidth 400000

ip address 12.12.12.1 255.255.255.252

load-interval 30

negotiation auto

ip flow ingress

service-policy output PM-1DC-AGGREGATE

!

The Dynamic Multipoint VPN (DMVPN) is between the two ASR1k routers. Traffic was generated from IXIA to IXIA across the DMVPN cloud with packet size of 512 bytes @ 50000 pps. Another stream is configured for Expedited Forwarding (EF) traffic from IXIA to IXIA

With the above stream, we noticed traffic loss in both streams for upto nearly 30000 pps.

Observations

There were not much output drops incrementing and not much drops seen in the EF class or other classes except from default class of the service-policy.

Found drops in QFP using show platform hardware qfp active statistics drops and noticed those drops were incrementing rapidly.

RTR-1#show platform hardware qfp active statistics drop

-------------------------------------------------------------------------

Global Drop Stats Packets Octets

-------------------------------------------------------------------------

IpsecInput 300010 175636790

IpsecOutput 45739945 23690171340

TailDrop 552830109 326169749399

RTR-1#

RTR-1#show platform hardware qfp active statistics drop

-------------------------------------------------------------------------

Global Drop Stats Packets Octets

-------------------------------------------------------------------------

IpsecInput 307182 179835230

IpsecOutput 46883064 24282257670

TailDrop 552830109 326169749399

RTR-1#

Further IPSec drops were checked for QFP using command show platform hardware qfp active feature ipsec data drops

RTR-1#show platform hardware qfp active feature ipsec data drops

------------------------------------------------------------------------

Drop Type Name Packets

------------------------------------------------------------------------

28 IN_PSTATE_CHUNK_ALLOC_FAIL 357317

54 OUT_PSTATE_CHUNK_ALLOC_FAIL 51497757

66 N2_GEN_NOTIFY_SOFT_EXPIRY 4023610

RTR-1#

It was noticed that drop counter for  IN_PSTATE_CHUNK_ALLOC_FAIL counter was matching the value IpsecInput counter in the QFP drops and same with IpsecOutput matching with OUT_PSTATE_CHUNK_ALLOC_FAIL counter.

This issue is seen due to the software defect# CSCuf25027 .

Solution

Workaround to this problem is to disable Netflow and Network Based Application Recognition (NBAR) feature on the router. If you want to run all the features and have better throughput, then better option is to upgrade to ASR1002-X or ASR1006 with ESP-100.