Performance and Scale

This chapter includes the following major topics:

Two types of scalability numbers exist for L3VPN: 1-Dimensional (1D) and Multi-Dimensional (MD). The 1D scale numbers only show scale of L3VPN as a single service running on ASR9000, which is not realistic from a deployment standpoint because a L3VPN PE in an Enterprise or service provider network would usually have mixed services and features, hence we tested and certified the MD scale profile for L3VPN PE.

Table 6-1 captures the MD scale numbers of L3VPN PE Profile with all services and features enabled simultaneously on a PE in a realistic deployment environment.

Table 6-1 ASR9k L3VPN PE Profile Multi-Dimensional Scale Numbers

Feature
Parameters
Scale

L3 Interfaces

Qot1q, qinq, Ethernet

4k

ATM, POS, FR, CE, TDM, HDLC, etc.

6k

MPLS VPNv4

IPv4 VRF Sessions (2 to 3 interfaces per VRF)

4k

VPNv4 Prefixes

2M

PE-CE Routing

 

eBGP with NSR, MD5, and lower KA-HT

4k

OSPF with MD5 and sham links

1k

Staticv4

4750

EIGRPv4

250

MPLS VPNv6

IPv6 VRF Sessions (2 interfaces per VRF)

4k

VPNv6 Prefixes

500k

PE-CE Routing

 

eBGP with NSR, MD5, and lower KA-HT

4k

OSPF with MD5 and sham links

1k

Staticv6

4750

EIGRPv6

250

MVPN

MVPN IPv4/IPv6

500

IPv4 Mroutes, IPv6 Mroutes

32k, 16k

P2MP-TE

Headend LSP

1k

uRPF

Ipv4, IPv6

10k, 10k

IGMP Snooping

BDs, Snooping Entries

1k, 32k

MLD Snooping

BDs, Snooping Entries

1k, 32k

L2 Interfaces

Ethernet (Phy, Bundle-Ether, BVI, PW-HE)

 

POS and Serial

 

L2VPN

AToM VPWS

1k

FRoMPLS

1k

FR to Eth IWoMPLS

1k

VPWS PWs

15k

VPWS ACs (1000 each on Eth, BE, PW-HE)

3k

VPLS PWs (w/ 5 neighoring PEs)

15k

VPLS ACs (1000 each on 10GigE, BE, PW-HE)

3k

VPLS PWs to Simulated PEs

34k

VPLS ACs for Simulated PEs (GigE, 10GigE)

2k

MAC address

2M

QoS

Interfaces w/ Ingress Policy

10k

Interfaces w/ Egress Policy

10k

ACLs

IPv4 ACLs on interface

10k

IPv6 ACLs on interface

10k

MPLS TE

Headend LSP with FRR

3k

Midpoint LSP

50k

BFD

IPv4 echo

10k

IPv6 Async

10k

Internet Peering Application

ASR9K is used extensively in Internet Peering, Inter-Connect and RR applications because of its rich BGP features, stability of XR SW, and high scale. We have designed and tested the following profiles:

  • ASR 9001 as RR
  • ASR9k as peering and Enterprise, DC or SP inter-connect platform

ASR 9001 RR-tested scalability numbers are summarized in Table 6-2 .

Table 6-2 ASR9k Route Reflector Scale Numbers

Feature
Scale

eBGP sessions with 3 BGP instances

5k

eBGP routes with 3 BGP instances

Total Route Scale = 14M routes

IPv4 = 6M

VPNv4 = 5M

IPv6 = 1.5M

VPNv6 = 1.5M

iBGP sessions with 2 BGP instances

5k

iBGP routes with 2 BGP instances

Total Route Scale = 10M

Ipv4 = 402k

VPNv4 = 7.6M

VPNv6 = 2M

In the Internet Peering and Inter-Connect profile, we used the topology described in Figure 6-1 to test Enterprise, Data Center and SP peering and inter-connect use cases with scalability. The following key features were tested in this profile:

  • Inter-AS option B and C Unicast Routing
  • BGP Flowspec
  • NetFlow 1:10k Sampling for IPv4, IPv6 and MPLS
  • VXLAN L3VPN/L2VPN Gateway handoff between Inter-AS Core
  • RFC 3107 PIC, BGP PIC edge for VPNv4, 6VPE, 6PE etc.
  • LFA, rLFA
  • Inter-AS option C L2VPN VPWS/VPLS with BGP AD, Inter-AS MS-PW, FAT-PW
  • Inter-Area/Inter-AS MPLS TE, P2MP TE
  • Inter-AS Native IPv4/v6 Multicast, Rosen-mGRE-MVPNv4/v6, mLDP-MVPNv4/v6
  • Native IPv4/v6, VPNv4/v6, VPWS/VPLS, Native IPv4/v6 Multicast, mGRE-MVPNv4/v6, PBB-EVPN over CsC
  • Next-generation Routing LISP, LISP-MPLS Gateway
  • Next-generation MVPN LSM with BGP C-mcast, Dynamic P2MP-TE MVPN, BGP SAFI 2, 129, 5
  • Next-generation L2VPN PBB-EVPN
  • Next-generation L2 Multicast: VPLS LSM
  • TI-MoFRR, MPLS-TP, Bi-Directional TE LSPs (aka. Flex-LSP)

Figure 6-1 ASR9k Internet Peering and Inter-Connect Profile Topology

 

The ASR9k scalability test results of Internet Peering and Inter-Connect Profile are shown in Table 6-3 .

Table 6-3 ASR9k Internet Peering and Inter-Connect Profile Scale Numbers

Feature
PE1
PE2
PE3
ASBR1
ASBR2
ASBR3
PE8

Global FIB v4

512k

512k

512k

512k

512k

512k

512k

Global FIB v6

18k

128k

128k

128k

128k

128k

128k

VRF (v4+v6)

4k

4k

4k

 

 

 

4k

VRF FIB v4

2M

2M

2M

 

 

 

2M

VRF FIB v6

256k

256k

256k

 

 

 

256k

LFIB

 

 

 

 

512k

512k

512k

L3 interfaces

8k

8k

8k

 

 

 

8k

ARP Adjacencies

32k

32k

32k

32k

32k

32k

32k

BGP session V4

3k

3k

3k

3k

3k

3k

3k

BGP session V6

256k

256k

256k

256k

256k

256k

256k

Labeled-BGP routes

10k

10k

10k

10k

10k

10k

10k

OSPFv2 adjacency

32k

32k

32k

32k

32k

32k

32k

OSPFv3 adjacency

32k

32k

32k

32k

32k

32k

32k

OSPFv2 routes

5k

5k

5k

5k

5k

5k

5k

OSPFv3 routes

10k

10k

10k

10k

10k

10k

10k

ISISv4 adjacency

32k

32k

32k

32k

32k

32k

32k

ISISv6 adjacency

32k

32k

32k

32k

32k

32k

32k

ISISv4 routes

5k

5k

5k

5k

5k

5k

5k

ISISv6 routes

10k

10k

10k

10k

10k

10k

10k

IGP LFA

10k

10k

10k

10k

10k

10k

10k

VRRP/HSRP

400k

400k

400k

400k

400k

400k

400k

ECMP

8k

8k

8k

8k

8k

8k

8k

MPLS label

512k

512k

512k

512k

512k

512k

512k

Intra-Area MPLS TE

1k

1k

1k

1k

1k

1k

1k

Inter-Area MPLS TE

1k

1k

1k

1k

1k

1k

1k

Intra-AS MPLS TE

1k

1k

1k

1k

1k

1k

1k

ACL

10k

10k

10k

10k

10k

10k

10k

L2 interfaces (physical)

16k

16k

16k

 

 

 

16k

L2 interfaces (bundle)

16k

16k

16k

 

 

 

16k

PW

32k

32k

32k

32k

32k

32k

32k

MS-PW

4k

4k

4k

4k

4k

4k

4k

BD/VFI

4k

4k

4k

4k

4k

4k

4k

MAC

512k

512k

512k

512k

512k

512k

512k

CFM MEP

4k

4k

4k

4k

4k

4k

4k

CFM MIP

4k

4k

4k

4k

4k

4k

4k

MPLS-TP

1k

1k

1k

1k

1k

1k

1k

Policy-map

1k

1k

1k

1k

1k

1k

1k

Class-map

1k

1k

1k

1k

1k

1k

1k

Policers

32k

32k

32k

32k

32k

32k

32k

Ingress Queue

64k

64k

64k

64k

64k

64k

64k

Egress Queue

64k

64k

64k

64k

64k

64k

64k

100G Edge and Core-Facing Ports

ASR9k is being positioned as the 100G routing platform in Enterprise, SP, Data Center, and Public Sector segments as the de facto platform for UNI or edge services and NNI or core-facing connectivity.

Table 6-4 describes 100G density and performance testing results based on UNI and NNI testing configurations of ASR9k.

Table 6-4 Summary of 100G Support for UNI and NNI on ASR9K

Parameter
Typhoon

No. of 100G ports per slot

2X100G line rate

SW support

XR 4.2.1

No. of 100G ports per slice

1x100G

Bi-directional bandwidth

200Gbps 100Gbps per NPU

Bi-directional PPS

90Mpps/direction

UNI or Edge-facing service termination on 100G

Yes

NNI or Core-facing for 100G transport

Yes

nV cluster

Yes

nV satellite

Yes

MACSEC Suite B+

No

MACSEC over Cloud

No

100G Pro-active Protection

Yes

CPAK Optics

No

L2FIB MAC address

2M

L3FIB IPv4/IPv6 address

4M/2M

Bridge domain

64k

We have validated the 100G line card throughput and latency of ASR9k Typhoon line cards in the following two roles and summarized the performance in Table 6-5 .

  • UNI or edge-facing L2/L3/Multicast VPN services with features
  • NNI or core-facing transport with features

The 100G deployment profiles we covered included MPLS, IPv4 and IPv6 in these applications: Internet Peering, DCI PE, SP Edge PE, Metro-Ethernet PE and P, Wan-Core PE and P router and general purpose Core P router.

Table 6-5 Typhoon 100G Forwarding Chain Performance

SW Ver
Feature
UNI/Edge or NNI/Core Facing Role
Sub-Feature
Linecard
Linerate Packet Size (bytes)
Min Latency (us)

5.1.0

MPLS

NNI/Core

mpls_swap

A9K-2x100GE-SE

130

15

5.1.0

MPLS

NNI/Core

mpls_depo

A9K-2x100GE-SE

176

14

5.1.0

MPLS

NNI/Core

mpls_impo

A9K-2x100GE-SE

175

14

5.1.0

IPv4

NNI/Core

IPv4 10K BGP route

A9K-2x100GE-SE

136

14

5.1.0

IPv4

NNI/Core

IPv4 500K BGP+uRPF

A9K-2x100GE-SE

212

15

5.1.0

IPv4

NNI/Core

IPv4 non recursive

A9K-2x100GE-SE

114

14

5.1.0

IPv4

NNI/Core

IPv4 500K BGP route

A9K-2x100GE-SE

160

16

5.1.0

IPv6

NNI/Core

IPv6_50K BGP route + QoS

A9K-2x100GE-SE

384

18

5.1.0

IPv6

NNI/Core

IPv6_nonrcur udp NH

A9K-2x100GE-SE

196

14

5.1.0

IPv6

NNI/Core

IPv6_50K BGP route

A9K-2x100GE-SE

361

17

5.1.0

IPv6

NNI/Core

IPv6_10K BGP route + QoS

A9K-2x100GE-SE

359

17

5.1.0

IPv6

NNI/Core

IPv6_50K BGP route + QoS

A9K-2x100GE-SE

384

18

5.1.0

L3VPN

NNI/Edge

L3VPN_30vrf

A9K-2x100GE-SE

232

15

5.1.0

IPv4 ACL

UNI/Edge

output_acl

A9K-2x100GE-SE

140

15

5.1.0

IPv4 ACL

NNI/Core

input_acl

A9K-2x100GE-SE

199

15

5.1.0

IPv4 ACL

NNI/Core

in+out_acl

A9K-2x100GE-SE

333

16

5.1.0

IPv4 QoS

NNI/Core

in+out_policy

A9K-2x100GE-SE

230

16

5.1.0

IPv4 QoS

NNI/Core

out shaper

A9K-2x100GE-SE

168

15

5.1.0

IPv4 QoS

NNI/Core

inpol+outshap

A9K-2x100GE-SE

218

16

5.1.0

IPv4 QoS

NNI/Core

IPv4 500K BGP route_inpol+outshap

A9K-2x100GE-SE

264

16

5.1.0

IPv4 QoS

NNI/Core

input_policy

A9K-2x100GE-SE

223

16

5.1.0

IPv4 QoS

NNI/Core

output_policy

A9K-2x100GE-SE

209

15

5.1.0

L2

UNI/Edge

Bridge

A9K-2x100GE-SE

129

14

5.1.0

L2

UNI/Edge

xconnect

A9K-2x100GE-SE

113

13

5.1.0

Multicast

UNI/Edge

mcast_IPv4

A9K-2x100GE-SE

277

15

5.1.0

Multicast

UNI/Edge

mcast_IPv6

A9K-2x100GE-SE

516

14

5.1.0

BVI

UNI/Edge

L2 EFP BVI L3_2K BVI

A9K-2x100GE-SE

592

17

5.1.0

mVPN

UNI/Edge

mVPN 12vrf_100mroute

A9K-2x100GE-SE

507

15

5.1.0

L2VPN

UNI/Edge

VPLS+qos

A9K-2x100GE-SE

596

17

5.1.0

L2VPN

UNI/Edge

VPWS 3ac+3pw

A9K-2x100GE-SE

319

15

5.1.0

L2VPN

UNI/Edge

VPLS_9BD+9ac+27pw

A9K-2x100GE-SE

374

16

5.1.0

L2VPN

UNI/Edge

VPWS_3ac+3pw+inpol+outshap

A9K-2x100GE-SE

326

15