Performance and Scale
This chapter includes the following major topics:
Two types of scalability numbers exist for L3VPN: 1-Dimensional (1D) and Multi-Dimensional (MD). The 1D scale numbers only show scale of L3VPN as a single service running on ASR9000, which is not realistic from a deployment standpoint because a L3VPN PE in an Enterprise or service provider network would usually have mixed services and features, hence we tested and certified the MD scale profile for L3VPN PE.
Table 6-1 captures the MD scale numbers of L3VPN PE Profile with all services and features enabled simultaneously on a PE in a realistic deployment environment.
Table 6-1 ASR9k L3VPN PE Profile Multi-Dimensional Scale Numbers
|
|
|
L3 Interfaces |
Qot1q, qinq, Ethernet |
4k |
ATM, POS, FR, CE, TDM, HDLC, etc. |
6k |
MPLS VPNv4 |
IPv4 VRF Sessions (2 to 3 interfaces per VRF) |
4k |
VPNv4 Prefixes |
2M |
PE-CE Routing |
|
eBGP with NSR, MD5, and lower KA-HT |
4k |
OSPF with MD5 and sham links |
1k |
Staticv4 |
4750 |
EIGRPv4 |
250 |
MPLS VPNv6 |
IPv6 VRF Sessions (2 interfaces per VRF) |
4k |
VPNv6 Prefixes |
500k |
PE-CE Routing |
|
eBGP with NSR, MD5, and lower KA-HT |
4k |
OSPF with MD5 and sham links |
1k |
Staticv6 |
4750 |
EIGRPv6 |
250 |
MVPN |
MVPN IPv4/IPv6 |
500 |
IPv4 Mroutes, IPv6 Mroutes |
32k, 16k |
P2MP-TE |
Headend LSP |
1k |
uRPF |
Ipv4, IPv6 |
10k, 10k |
IGMP Snooping |
BDs, Snooping Entries |
1k, 32k |
MLD Snooping |
BDs, Snooping Entries |
1k, 32k |
L2 Interfaces |
Ethernet (Phy, Bundle-Ether, BVI, PW-HE) |
|
POS and Serial |
|
L2VPN |
AToM VPWS |
1k |
FRoMPLS |
1k |
FR to Eth IWoMPLS |
1k |
VPWS PWs |
15k |
VPWS ACs (1000 each on Eth, BE, PW-HE) |
3k |
VPLS PWs (w/ 5 neighoring PEs) |
15k |
VPLS ACs (1000 each on 10GigE, BE, PW-HE) |
3k |
VPLS PWs to Simulated PEs |
34k |
VPLS ACs for Simulated PEs (GigE, 10GigE) |
2k |
MAC address |
2M |
QoS |
Interfaces w/ Ingress Policy |
10k |
Interfaces w/ Egress Policy |
10k |
ACLs |
IPv4 ACLs on interface |
10k |
IPv6 ACLs on interface |
10k |
MPLS TE |
Headend LSP with FRR |
3k |
Midpoint LSP |
50k |
BFD |
IPv4 echo |
10k |
IPv6 Async |
10k |
Internet Peering Application
ASR9K is used extensively in Internet Peering, Inter-Connect and RR applications because of its rich BGP features, stability of XR SW, and high scale. We have designed and tested the following profiles:
- ASR 9001 as RR
- ASR9k as peering and Enterprise, DC or SP inter-connect platform
ASR 9001 RR-tested scalability numbers are summarized in Table 6-2 .
Table 6-2 ASR9k Route Reflector Scale Numbers
|
|
eBGP sessions with 3 BGP instances |
5k |
eBGP routes with 3 BGP instances |
Total Route Scale = 14M routes |
IPv4 = 6M |
VPNv4 = 5M |
IPv6 = 1.5M |
VPNv6 = 1.5M |
iBGP sessions with 2 BGP instances |
5k |
iBGP routes with 2 BGP instances |
Total Route Scale = 10M |
Ipv4 = 402k |
VPNv4 = 7.6M |
VPNv6 = 2M |
In the Internet Peering and Inter-Connect profile, we used the topology described in Figure 6-1 to test Enterprise, Data Center and SP peering and inter-connect use cases with scalability. The following key features were tested in this profile:
- Inter-AS option B and C Unicast Routing
- BGP Flowspec
- NetFlow 1:10k Sampling for IPv4, IPv6 and MPLS
- VXLAN L3VPN/L2VPN Gateway handoff between Inter-AS Core
- RFC 3107 PIC, BGP PIC edge for VPNv4, 6VPE, 6PE etc.
- LFA, rLFA
- Inter-AS option C L2VPN VPWS/VPLS with BGP AD, Inter-AS MS-PW, FAT-PW
- Inter-Area/Inter-AS MPLS TE, P2MP TE
- Inter-AS Native IPv4/v6 Multicast, Rosen-mGRE-MVPNv4/v6, mLDP-MVPNv4/v6
- Native IPv4/v6, VPNv4/v6, VPWS/VPLS, Native IPv4/v6 Multicast, mGRE-MVPNv4/v6, PBB-EVPN over CsC
- Next-generation Routing LISP, LISP-MPLS Gateway
- Next-generation MVPN LSM with BGP C-mcast, Dynamic P2MP-TE MVPN, BGP SAFI 2, 129, 5
- Next-generation L2VPN PBB-EVPN
- Next-generation L2 Multicast: VPLS LSM
- TI-MoFRR, MPLS-TP, Bi-Directional TE LSPs (aka. Flex-LSP)
Figure 6-1 ASR9k Internet Peering and Inter-Connect Profile Topology
The ASR9k scalability test results of Internet Peering and Inter-Connect Profile are shown in Table 6-3 .
Table 6-3 ASR9k Internet Peering and Inter-Connect Profile Scale Numbers
|
|
|
|
|
|
|
|
Global FIB v4 |
512k |
512k |
512k |
512k |
512k |
512k |
512k |
Global FIB v6 |
18k |
128k |
128k |
128k |
128k |
128k |
128k |
VRF (v4+v6) |
4k |
4k |
4k |
|
|
|
4k |
VRF FIB v4 |
2M |
2M |
2M |
|
|
|
2M |
VRF FIB v6 |
256k |
256k |
256k |
|
|
|
256k |
LFIB |
|
|
|
|
512k |
512k |
512k |
L3 interfaces |
8k |
8k |
8k |
|
|
|
8k |
ARP Adjacencies |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
BGP session V4 |
3k |
3k |
3k |
3k |
3k |
3k |
3k |
BGP session V6 |
256k |
256k |
256k |
256k |
256k |
256k |
256k |
Labeled-BGP routes |
10k |
10k |
10k |
10k |
10k |
10k |
10k |
OSPFv2 adjacency |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
OSPFv3 adjacency |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
OSPFv2 routes |
5k |
5k |
5k |
5k |
5k |
5k |
5k |
OSPFv3 routes |
10k |
10k |
10k |
10k |
10k |
10k |
10k |
ISISv4 adjacency |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
ISISv6 adjacency |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
ISISv4 routes |
5k |
5k |
5k |
5k |
5k |
5k |
5k |
ISISv6 routes |
10k |
10k |
10k |
10k |
10k |
10k |
10k |
IGP LFA |
10k |
10k |
10k |
10k |
10k |
10k |
10k |
VRRP/HSRP |
400k |
400k |
400k |
400k |
400k |
400k |
400k |
ECMP |
8k |
8k |
8k |
8k |
8k |
8k |
8k |
MPLS label |
512k |
512k |
512k |
512k |
512k |
512k |
512k |
Intra-Area MPLS TE |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
Inter-Area MPLS TE |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
Intra-AS MPLS TE |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
ACL |
10k |
10k |
10k |
10k |
10k |
10k |
10k |
L2 interfaces (physical) |
16k |
16k |
16k |
|
|
|
16k |
L2 interfaces (bundle) |
16k |
16k |
16k |
|
|
|
16k |
PW |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
MS-PW |
4k |
4k |
4k |
4k |
4k |
4k |
4k |
BD/VFI |
4k |
4k |
4k |
4k |
4k |
4k |
4k |
MAC |
512k |
512k |
512k |
512k |
512k |
512k |
512k |
CFM MEP |
4k |
4k |
4k |
4k |
4k |
4k |
4k |
CFM MIP |
4k |
4k |
4k |
4k |
4k |
4k |
4k |
MPLS-TP |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
Policy-map |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
Class-map |
1k |
1k |
1k |
1k |
1k |
1k |
1k |
Policers |
32k |
32k |
32k |
32k |
32k |
32k |
32k |
Ingress Queue |
64k |
64k |
64k |
64k |
64k |
64k |
64k |
Egress Queue |
64k |
64k |
64k |
64k |
64k |
64k |
64k |
100G Edge and Core-Facing Ports
ASR9k is being positioned as the 100G routing platform in Enterprise, SP, Data Center, and Public Sector segments as the de facto platform for UNI or edge services and NNI or core-facing connectivity.
Table 6-4 describes 100G density and performance testing results based on UNI and NNI testing configurations of ASR9k.
Table 6-4 Summary of 100G Support for UNI and NNI on ASR9K
|
|
No. of 100G ports per slot |
2X100G line rate |
SW support |
XR 4.2.1 |
No. of 100G ports per slice |
1x100G |
Bi-directional bandwidth |
200Gbps 100Gbps per NPU |
Bi-directional PPS |
90Mpps/direction |
UNI or Edge-facing service termination on 100G |
Yes |
NNI or Core-facing for 100G transport |
Yes |
nV cluster |
Yes |
nV satellite |
Yes |
MACSEC Suite B+ |
No |
MACSEC over Cloud |
No |
100G Pro-active Protection |
Yes |
CPAK Optics |
No |
L2FIB MAC address |
2M |
L3FIB IPv4/IPv6 address |
4M/2M |
Bridge domain |
64k |
We have validated the 100G line card throughput and latency of ASR9k Typhoon line cards in the following two roles and summarized the performance in Table 6-5 .
- UNI or edge-facing L2/L3/Multicast VPN services with features
- NNI or core-facing transport with features
The 100G deployment profiles we covered included MPLS, IPv4 and IPv6 in these applications: Internet Peering, DCI PE, SP Edge PE, Metro-Ethernet PE and P, Wan-Core PE and P router and general purpose Core P router.
Table 6-5 Typhoon 100G Forwarding Chain Performance
|
|
UNI/Edge or NNI/Core Facing Role
|
|
|
Linerate Packet Size (bytes)
|
|
5.1.0 |
MPLS |
NNI/Core |
mpls_swap |
A9K-2x100GE-SE |
130 |
15 |
5.1.0 |
MPLS |
NNI/Core |
mpls_depo |
A9K-2x100GE-SE |
176 |
14 |
5.1.0 |
MPLS |
NNI/Core |
mpls_impo |
A9K-2x100GE-SE |
175 |
14 |
5.1.0 |
IPv4 |
NNI/Core |
IPv4 10K BGP route |
A9K-2x100GE-SE |
136 |
14 |
5.1.0 |
IPv4 |
NNI/Core |
IPv4 500K BGP+uRPF |
A9K-2x100GE-SE |
212 |
15 |
5.1.0 |
IPv4 |
NNI/Core |
IPv4 non recursive |
A9K-2x100GE-SE |
114 |
14 |
5.1.0 |
IPv4 |
NNI/Core |
IPv4 500K BGP route |
A9K-2x100GE-SE |
160 |
16 |
5.1.0 |
IPv6 |
NNI/Core |
IPv6_50K BGP route + QoS |
A9K-2x100GE-SE |
384 |
18 |
5.1.0 |
IPv6 |
NNI/Core |
IPv6_nonrcur udp NH |
A9K-2x100GE-SE |
196 |
14 |
5.1.0 |
IPv6 |
NNI/Core |
IPv6_50K BGP route |
A9K-2x100GE-SE |
361 |
17 |
5.1.0 |
IPv6 |
NNI/Core |
IPv6_10K BGP route + QoS |
A9K-2x100GE-SE |
359 |
17 |
5.1.0 |
IPv6 |
NNI/Core |
IPv6_50K BGP route + QoS |
A9K-2x100GE-SE |
384 |
18 |
5.1.0 |
L3VPN |
NNI/Edge |
L3VPN_30vrf |
A9K-2x100GE-SE |
232 |
15 |
5.1.0 |
IPv4 ACL |
UNI/Edge |
output_acl |
A9K-2x100GE-SE |
140 |
15 |
5.1.0 |
IPv4 ACL |
NNI/Core |
input_acl |
A9K-2x100GE-SE |
199 |
15 |
5.1.0 |
IPv4 ACL |
NNI/Core |
in+out_acl |
A9K-2x100GE-SE |
333 |
16 |
5.1.0 |
IPv4 QoS |
NNI/Core |
in+out_policy |
A9K-2x100GE-SE |
230 |
16 |
5.1.0 |
IPv4 QoS |
NNI/Core |
out shaper |
A9K-2x100GE-SE |
168 |
15 |
5.1.0 |
IPv4 QoS |
NNI/Core |
inpol+outshap |
A9K-2x100GE-SE |
218 |
16 |
5.1.0 |
IPv4 QoS |
NNI/Core |
IPv4 500K BGP route_inpol+outshap |
A9K-2x100GE-SE |
264 |
16 |
5.1.0 |
IPv4 QoS |
NNI/Core |
input_policy |
A9K-2x100GE-SE |
223 |
16 |
5.1.0 |
IPv4 QoS |
NNI/Core |
output_policy |
A9K-2x100GE-SE |
209 |
15 |
5.1.0 |
L2 |
UNI/Edge |
Bridge |
A9K-2x100GE-SE |
129 |
14 |
5.1.0 |
L2 |
UNI/Edge |
xconnect |
A9K-2x100GE-SE |
113 |
13 |
5.1.0 |
Multicast |
UNI/Edge |
mcast_IPv4 |
A9K-2x100GE-SE |
277 |
15 |
5.1.0 |
Multicast |
UNI/Edge |
mcast_IPv6 |
A9K-2x100GE-SE |
516 |
14 |
5.1.0 |
BVI |
UNI/Edge |
L2 EFP BVI L3_2K BVI |
A9K-2x100GE-SE |
592 |
17 |
5.1.0 |
mVPN |
UNI/Edge |
mVPN 12vrf_100mroute |
A9K-2x100GE-SE |
507 |
15 |
5.1.0 |
L2VPN |
UNI/Edge |
VPLS+qos |
A9K-2x100GE-SE |
596 |
17 |
5.1.0 |
L2VPN |
UNI/Edge |
VPWS 3ac+3pw |
A9K-2x100GE-SE |
319 |
15 |
5.1.0 |
L2VPN |
UNI/Edge |
VPLS_9BD+9ac+27pw |
A9K-2x100GE-SE |
374 |
16 |
5.1.0 |
L2VPN |
UNI/Edge |
VPWS_3ac+3pw+inpol+outshap |
A9K-2x100GE-SE |
326 |
15 |