Verified Scalability Limits

This document describes the Cisco NX-OS configuration limits for Cisco Nexus 9000 Series switches and Cisco Nexus 9508 switch with an X9636C-R, X9636C-RX, or X9636Q-R line card or a C9508-FM-R fabric module and Cisco Nexus 9504 with -R line cards.

Introduction

The values provided in this guide should not be interpreted as theoretical system limits for Cisco Nexus 9000 Series hardware or Cisco NX-OS software. These limits refer to values that have been validated by Cisco. They can increase over time as more testing and validation is done.

Verified Scalability Limits for Cisco Nexus 9000 Series Switches

Verified Scalability Limits

The tables in this section list the verified scalability limits for Cisco Nexus 9000 Series switches for Cisco NX-OS Release 9.2(1). These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.

Table 1. Cisco Nexus 2000 Series Fabric Extenders (FEX) Straight Through Mode Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit¹	9300 Platform Verified Limit²	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit
Fabric Extenders³ and Fabric Extender server interfaces	32 and 1536	16 and 768	Not applicable	16 and 768	16 and 768
VLANs across all Fabric Extenders	2000	2000	Not applicable	562	562
VLANs per Fabric Extender server interface⁴	75	75	Not applicable	75	75
Port channels	426	256	Not applicable	232	232
Unique Fabric Extenders per Cisco Nexus 9500 Series supported line card	12	Not applicable	Not applicable	Not applicable	Not applicable

¹ The Cisco Nexus 2200 Series and B22 Series Fabric Extenders are supported with X9464PX and X9564PX line cards on Cisco Nexus 9500 Series switches. The Cisco Nexus 2300 Series Fabric Extenders are supported with X9432PQ, X9464PX, X9464TX, X9536PQ, X9564PX, X9564TX, and X9636PQ line cards on Cisco Nexus 9500 Series switches.

² The Cisco Nexus 2200 Series and B22 Series Fabric Extenders are supported with the Cisco Nexus 9396PX, 9372PX, and 9372PX-E chassis. The Cisco Nexus 2300 Series Fabric Extenders are supported with the Cisco Nexus 9332PQ, 9396PX, 9372PX, and 9372PX-E chassis.

³ When FEX configured using "AA" mode, then the maximum number of 6 FEX on NFE base ToR and 16 FEX for LSE base ToR are supported.

⁴ For FEX HIF port channels, Cisco recommends that you enable STP port type edge using the spanning tree port type edge [trunk] command.

Table 2. FCoE Verified Scalability Limits (Unidimensional)
Feature ⁵	Verified Limit ⁶
FLOGI per port	256
FLOGI per switch	1000
Port channels	8⁷
Member ports in a port channel	16
VFCs	544
VSANs	32

⁵ Feature Verified Limit is 5.

⁶ All above numbers are verified against Platform — Nexus 93180YC-FX

⁷ The number of SAN port channels and virtual FC port channels, together, can be only 8 on the Cisco Nexus 9000 Series switch.

Note

For a list of platforms on which FCoE is supported, see the Cisco Nexus 9000 Series NX-OS FCoE Configuration Guide.

Table 3. FC Verified Scalability Limits (Unidimensional)
Feature ⁸	Verified Limit ⁹
FLOGI per port	256
FLOGI per switch	1000
Port channels	8¹⁰
Member ports in a port channel	16
FC ports	48
VSANs	32

⁸ Feature Verified Limit is 6.

⁹ All above numbers are verified against Platform — Nexus 93180YC-FX

¹⁰ The number of SAN port channels and virtual FC port channels, together, can be only 8 on the Cisco Nexus 9000 Series switch.

Table 4. Intelligent Traffic Director Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit
Nodes per device group	32	32	32
Nodes across all device groups	256	256	256
Device groups per switch	48	48	48
ITD services per switch	64	64	64
Ingress interfaces per ITD service	8	8	8
Virtual IP addresses per ITD service	255	255	255
Device groups per ITD service	48	48	48

Note

For a list of platforms on which ITD is supported, see the Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide.

Table 5. Interfaces Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
DHCP clients per switch	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)	10 (IPv4) + 10 (IPv6)
IP DHCP relay addresses (helper addresses) per L3 interface	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)	32 (IPv4) + 32 (IPv6)
Generic routing encapsulation (GRE) tunnels	8	8	8	8	8	8	8
Port channel links	32	32	32	32	32	32	32
SVIs	490 (with HSRP), 1500 (without HSRP)	450 (with HSRP)	490	450 (with HSRP)	450 (with HSRP)	450 (with HSRP)	490 (with HSRP), 1500 (without HSRP)
SVI Unnumbered	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs	Primary (50) Secondary (450) 1 primary SVI can have a maximum of 50 secondary SVIs
vPCs	300	80	80	48	80	80	300
Static network address translation (NAT)	Not applicable	1023	1023	1023	1023	1023	Not applicable
Dynamic network address translation (NAT)	Not applicable	1023	1023	1023	1023	1023	Not applicable
Static twice network address translation (NAT)	Not applicable	768	768	768	768	768	Not applicable
Dynamic twice network address translation (NAT)	Not applicable	1023	1023	1023	1023	1023	Not applicable

Table 6. Label Switching Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX/FX Verified Limit
Forwarding Equivalence Classes (FECs) (Node /Prefix /Adj / Binding SID)	128	128	MPLS Heavy Template: 512 Default: 128	MPLS Heavy Template: 4096 Default: 1024	MPLS Heavy Template: 4096 Default: 1024	Not supported	MPLS Heavy Template: 4096 Default: 1024
Equal-cost multipaths (ECMPs)	16	16	16	32	32	Not supported	32
Equal-cost multipaths Groups (ECMPs)	Not applicable	Not applicable	Not applicable	MPLS Heavy Template: 4096 Default: 1024	MPLS Heavy Template: 4096 Default: 1024	Not supported	MPLS Heavy Template: 4096 Default: 1024
FECs * ECMPs	1000	1000	1000	Not applicable	Not applicable	Not supported	Not applicable
Flex counters for segment-routing in ingress direction	4000 (includes ingress and egress)	4000 (includes ingress and egress)	4000 (includes ingress and egress) (MPLS Heavy Template)	Total ingress label stats: 4000 VRF ingress label stats: 1000 (MPLS Heavy Template)	Total ingress label stats: 4000 VRF ingress label stats: 1000 (MPLS Heavy Template)	Not supported	Total ingress label stats: 4000 VRF ingress label stats: 1000 (MPLS Heavy Template)
Flex counters for segment-routing in Egress direction	4000 (includes ingress and egress)	4000 (includes ingress and egress)	4000 (includes ingress and egress) (MPLS Heavy Template)	Total ingress label stats: 48K (MPLS Heavy Template)	Total ingress label stats: 48K (MPLS Heavy Template)	Not supported	Total ingress label stats: 48K (MPLS Heavy Template)
Egress Peer Engineering	64	64	64	64	64	Not supported	64
Label-switched paths (LSPs) for label stack imposition¹¹	128 (with 4-way ECMP and 3 label stack push)	128 (with 4-way ECMP and 3 label stack push)	256 (with 32-way ECMP and 5 label stack push)	256 (with 32-way ECMP and 5 label stack push)	256 (with 32-way ECMP and 5 label stack push)	Not supported	256 (with 32-way ECMP and 5 label stack push)
Layer 3 EVPN Labels	128	128	128 (With MPLS Heavy Template)	1000 (With MPLS Heavy Template)	1000 (With MPLS Heavy Template)	Not supported	1000 (With MPLS Heavy Template)
Node Sid/Prefix SID Scale	128	128	128	4000	4000	Not supported	4000
Adjacency SID Scale	128	128	128	600	600	Not supported	600
Binding SID Scale	50	50	50	1000	1000	Not supported	1000

Private VLANs (PVLANs)
Primary VLANs ¹²	16	16	Not applicable	400	400	16	16
Secondary VLANs ¹³	20	20	Not applicable	400	400	20	20
Ports in Community host mode	40	40	Not applicable	40	40	40	40
Ports in isolated host mode	20	40	Not applicable	40	40	40	40
Ports in isolated trunk host mode	22	40	Not applicable	40	40	40	40
Ports in promiscuous mode	48	5	Not applicable	10	10	5	5
Ports in promiscuous trunk mode	80	5	Not applicable	10	10	5	5
PVLANs allowed on a PVLAN port ¹⁴	16	16	Not applicable	400	400	16	16

¹¹ For Cisco Nexus 9300 and 9500 Series switches, LSPs *ECMP* label stack push cannot exceed 1500.

¹² The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port

¹³ The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port

¹⁴ The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port

Note

For network scalability, Cisco recommends using a hierarchical routing design with multi-hop BGP for advertising the attached prefixes from a top-of-rack (ToR) or border leaf switch.

Table 7. Layer 2 Switching Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
MAC addresses	90,000	90,000	92,000	92,000	74,000	74,000	92,000
MST instances	64	64	64	64	64	64	64
MST PV count with single instances 0	Not applicable	Not applicable	190000	Not applicable	190000	190000	Not applicable
MST virtual ports with more than 1 MST instance	85,000	48,000	48,000	48,000	48,000	48,000	85,000
RPVST virtual ports	22,000	12,000	12,000	12,000	12,000	12,000	22,000
VLANs	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)	3967 (the remaining 127 VLANs are reserved)
VLANs in RPVST mode	500	500	3967	3967	3967	3967	3967¹⁵
Total number of VLANs × ports with switchport isolated (3967 VLANs x 48 ports)	190,000	190,000	190,000	190,000	190,000	190,000	190,000
Private VLANs (PVLANs)
Primary VLANs	16	16	Not applicable	16	16	16	16
Secondary VLANs	20	20	Not applicable	20	20	20	20
Ports in Community host mode	40	40	Not applicable	40	40	40	40
Ports in isolated host mode	20	40	Not applicable	40	40	40	40
Ports in isolated trunk host mode	22	40	Not applicable	40	40	40	40
Ports in promiscuous mode	48	5	Not applicable	5	5	5	5
Ports in promiscuous trunk mode	80	5	Not applicable	5	5	5	5
PVLANs allowed on a PVLAN port	16	16	Not applicable	16	16	16	16

¹⁵ On EOR, support is for 12000 PV count with 3967 vlans and RPVST with default timers. If 22000 PV count is needed with 3968 vlans and RPVST, recommended hello timer value is 4 or higher. It is also recommended to tune forward delay and max age accordingly

Note

The number of supported VLANs per vPC should be within the MST or RPVST virtual port count specified in this table, depending on the topology.
The number of supported STP VLAN port instances for Fabric Extender host interface ports should be less than 13,000.

Table 8. Multicast Routing Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
IPv4 multicast routes	32,000 (Layer 2 + Layer 3)	8000 (Layer 2 + Layer 3)	8000 (Layer 2 + Layer 3) Cisco Nexus 9200 switches do not support the system routing template -Ipm - heavy mode for IPv4 multicast routes. Make sure to reset the LPM maximum limit to 0.	8000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode)	8000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode) 128000 (with system routing template -multicast - ext - heavy mode)	8000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode) 128000 (with system routing template -multicast - ext - heavy mode)	8000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode); 8000 (with system routing template -Ipm - heavy mode)
Outgoing interfaces (OIFs)	40 (SVI + physical Layer 3) or 256 (physical Layer 3)	40 (SVI + physical Layer 3)	40 (SVI + physical Layer 3)	40 (SVI + physical Layer 3)	40 (SVI + physical Layer 3)	40 (SVI + physical Layer 3)	40 (SVI + physical Layer 3) or 256 (physical Layer 3)
IGMP snooping groups	32,000	8000	8000	8000	16000	16000	32,000
PIM neighbors	500	250	250	250	250	250	500

Note

The IPv4 multicast routes and the IPv4/IPv6 host routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.
High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Table 9. IP Fabric for Media Solution Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
Routes	32000
Host Policy
Sender	8000
Receiver	8000
PIM	512
FlowPolicy	2000
ASM group-range	20

Note

For a list of supported platforms on which IP Fabric for Media Solution, see Cisco Nexus 9000 Series NX-OS IP Fabric for Media Solution Guide.

Table 10. Programmability Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
OpenFlow
OpenFlow ports	Not applicable	96	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
OpenFlow Layer 2 flows	Not applicable	32,000	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
OpenFlow Layer 3 flows	Not applicable	3000	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
OpenFlow IPv6 Layer 3 flows	Not applicable	1500	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable

Table 11. Security Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
DHCP snooping bindings	2048	2048	2048	2048	2048	2048	2048
IPv4 ingress TCAM entries	3072 (per network forwarding engine)	3072 (per network forwarding engine)	3582 (per slice of the forwarding engine)	3582 (per slice of the forwarding engine)	3582 (per slice of the forwarding engine)	3582	3582 (per slice of the forwarding engine)
IPv4 egress TCAM entries	768 (per network forwarding engine)	768 (per network forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)
IPv6 ingress TCAM entries	1536 (per network forwarding engine)	1536 (per network forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)	1792 (per slice of the forwarding engine)
IPv6 egress TCAM entries	256 (per network forwarding engine)	256 (per network forwarding engine)	896 (per slice of the forwarding engine)	896 (per slice of the forwarding engine)	896 (per slice of the forwarding engine)	896 (per slice of the forwarding engine)	896 (per slice of the forwarding engine)

Note

The TCAM entries scalability limits also apply to policy-based TCAM entries (PBACLs).

Table 12. System Management Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX Verified Limit
MPLS Stripping
Labels	12,000	12,000	No limit	Not applicable	Not applicable	Not applicable	Not applicable
Ingress interfaces	400	48	48	Not applicable	Not applicable	Not applicable	Not applicable
Egress interfaces	64	16	16	Not applicable	Not applicable	Not applicable	Not applicable
PTP
PTP ports ¹⁶	44	44	44	44	44	44	1305
sFlow
sFlow ports	256	64	64	64	64	64	256
SPAN and ERSPAN
Configurable SPAN or ERSPAN sessions	32	32	32	32	32	32	32
Active SPAN or ERSPAN sessions¹⁷	4 to 32, based on the number of line cards and the session configuration	4	4	4	4	4	4 to 32, based on the number of line cards and the session configuration
Active localized SPAN or ERSPAN sessions per line card¹⁸	4	4	4	4	4	4	4
Source interfaces per SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)	48	48	48	48	48	48	48
Destination interfaces per SPAN session	1 (physical/PO interface)	1 (physical/PO interface)	1 (physical/PO interface)	1 (physical/PO interface)	1 (physical/PO interface)	1 (physical/PO interface)	1 (physical/PO interface)
Source VLANs per SPAN or ERSPAN session	32	32	32	32	32	32	32
Tap Aggregation
Redirect interfaces in the redirect port list	12	12	12	12	12	12	Not applicable
Redirect port lists (or fan outs) per system	100	100	50	50	50	50	Not applicable
NetFlow
Flow monitors	Not applicable	Not applicable	Not applicable	2 exporters and 2 flow monitors per type (2 IPv4 flow monitors and 2 IPv6 flow monitors)	2 exporters and 32 flow monitors per type (32 Layer 2 flow monitors, 32 IPv4 flow monitors, and 32 IPv6 flow monitors)	2 exporters and 32 flow monitors per type (32 Layer 2 flow monitors, 32 IPv4 flow monitors, and 32 IPv6 flow monitors)	Not applicable
Number of Layer 3 interfaces (Layer 3 ports, port channels, and SVIs) to which IPv4 flow monitors can be applied	Not applicable	Not applicable	Not applicable	1016 (with members on just one ASIC slice) or 508 (with members on both ASIC slices) You can use the show interface hardware-mappings command to check if the interface belongs to ASIC slice 0 or slice 1.	Not applicable	Not applicable	Not applicable
Number of Layer 3 interfaces (Layer 3 ports, port channels, and SVIs) to which IPv6 flow monitors can be applied	Not applicable	Not applicable	Not applicable	252 (with members on just one ASIC slice) or 126 (with members on both ASIC slices) You can use the show interface hardware-mappings command to check if the interface belongs to ASIC slice 0 or slice 1.	Not applicable	Not applicable	Not applicable

¹⁶ With PTP offload enabled.

¹⁷ A single forwarding engine instance supports four SPAN or ERSPAN sessions. For Cisco Nexus 9300 Series switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources. This limitation might also apply to Cisco Nexus 9500 Series switches, depending on the SPAN or ERSPAN source's forwarding engine instance mappings.

¹⁸ The number of SPAN or ERSPAN sessions per line card reduces to two if the same interface is configured as the bidirectional source in more than one session.

Note

PTP is supported for all Cisco Nexus 9000 Series hardware except for the 100G 9408PC line card and the 100G M4PC generic expansion module (GEM).

Table 13. Unicast Routing Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9364C Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X9700-EX /FX Verified Limit
Unicast Routing
BFD sessions (echo mode)	512	128	128	128	128	128	128	512¹⁹
BGP neighbors	2000	512	512 (IPv4), 512 (IPv6), or 256 (IPv4 + IPv6)	512	512	512	512	2000
EIGRP routes	20,000	20,000	20,000	20,000	20,000	20,000	20,000	20,000
EIGRP neighbors	512	256	256	256	256	256	256	512
HSRP groups	490	490	490	490	490	490	490	490
IPv4 ARP	48,000	48,000	32,000	32,000	48,000 / 32,000 (with out/with urpf enabled) (in default routing mode without URPF enabled, Hash Table: Shared between IPv6 ND, IPv4 ARP)	48,000 / 32,000 (with out/with urpf enabled) (in default routing mode without URPF enabled, Hash Table: Shared between IPv6 ND, IPv4 ARP)	64,000 / 32,000 (with out/with urpf enabled) (in default routing mode, Hash Table: Shared between IPv6 ND, IPv4 ARP) 32,000/24,000 (with out / with urpf enabled) (with system routing template - lpm - heavy mode without urpf enabled, Hash Table: Shared between IPv6 ND, IPv4 ARP)	48,000 (default), 32,000 (LPM heavy)
IPv4 host routes ²⁰	Default System Routing Mode: 208,000 (hash table and there will be more collisions after 80%) ALPM Routing Mode: 128,000 with host Routes Programmed in the LPM Table	Default System Routing Mode: 208,000 (hash table and there will be more collisions after 80%) ALPM Routing Mode: 128,000 with host Routes Programmed in the LPM Table	96,000 (hash table and there will be more collisions after 80%)	96,000	458,000 (default); 786,000/720,000 (with  system routing  template -lpm - heavy  mode)	471,000 / 419,000 (with out/with urpf enabled) (default); 786,000/734,000 (with  system routing  template -lpm -heavy  mode)	471,000 (default); 786,000/734,000 (with out/with urpf enabled) (with  system routing  template -lpm -heavy  mode)	589,000 (default); 786,000 (with  system routing  template -lpm -heavy  mode)
IPv6 host routes ²¹	Default System Routing Mode: 104,000 (hash table and there will be more collisions after 80%) ALPM Routing Mode: 16000 with host Routes Programmed in the LPM Table	Default System Routing Mode: 104,000 (hash table and there will be more collisions after 80%) ALPM Routing Mode: 16000 with host Routes Programmed in the LPM Table	48,000 (hash table and there will be more collisions after 80%)	48,000	24000 / 16,000 (with out/with urpf enabled)	265,000 / 235,000 (with out/with urpf enabled) (default) 442,000 / 412,000 (with out/with urpf enabled) (with system routing template -lpm - heavy mode)	265,000 (default) 442,000 / 412,000 (with out/with urpf enabled) (with system routing template -lpm - heavy mode)	32,000 (FM-E), 235000 (FM-E2)
IPv6 ND	48,000	48,000	32,000 (default), 16,000 (lpm heavy)	32,000 (default), 16,000 (lpm heavy)	32,000 (default), 16,000 (lpm heavy)	32,000 (in default routing mode, Hash Table: Shared between IPv6 ND, IPv4 ARP) 16,000 (with system routing template - lpm - heavy mode, Hash Table: Shared between IPv6 ND, IPv4 ARP	32,000 (default), 16,000 (lpm heavy)	32,000
IPv4 unicast routes (LPM)*	128,000 (default system routing mode) 16,000 (max-host routing mode) 128,000 with no IPv6 routes (64-bit ALPM routing mode)	12,000 (default system routing mode) 128,000 (ALPM routing mode)	Default values: 6000 (IPv4), 1900 (IPv6), and 2000 (multicast) With hardware profile multicast max-limit lpm-entries 0 configured: 8000 (IPv4), 1900 (IPv6), and 0 (multicast) With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast) With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 0 (IPv4), 4096 (IPv6), and 0 (multicast) When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used	Default values: 8000 (IPv4), 1900 (IPv6), and 2000 (multicast) With hardware profile multicast max-limit lpm-entries 0 configured: 10000 (IPv4), 1900 (IPv6), and 0 (multicast) With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast) With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 4000 (IPv4), 4096 (IPv6), and 0 (multicast) When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used	458,000 (default)	471,000 / 419,000 (with out / with urpf enabled) (default)	471,000 (default)	589,000 (default)
IPv6 unicast routes (LPM)*	20,000 (default system routing mode) 4000 (max-host routing mode) 80,000 with no IPv4 routes (64-bit ALPM routing mode)	7000 (6000 routes < /64, 1000 routes > /64) (default system routing mode) 20,000 (ALPM routing mode)			206,000 (/64 prefix length); 1900 (non /64 prefix length)	265,000 / 235,000 (with out/with urpf enabled) (default)	265,000 (default)	176,000 (/64 prefix length); 3900 (non /64 prefix length) ( FM-E ) 235,000 ( any prefix len) ( FM-E2)
IPv4 and IPv6 unicast routes (LPM) in 64-bit ALPM routing mode	128,000 (IPv4) 80,000 (IPv6)	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
IPv4 host routes (LPM heavy mode)	Not applicable	Not applicable	Cisco Nexus 9236C, 9272Q, and 92304QC switches: 262,000 Cisco Nexus 92160YC-X switches: 650,000	262,000	786,000 / 720,000 (with out/with urpf enabled)	786000 / 734000 (with out/with urpf enabled	786,000 / 734,000 (with out/with urpf enabled)	786,000
IPv6 host routes (LPM heavy mode)	Not applicable	Not applicable	16,000	131,000	24000 / 16,000 (with out/with urpf enabled) (protocol learned host)	442,000 / 412,000 (with out/with urpf enabled) (protocol learned host)	442,000 / 412,000 (with out/with urpf enabled) (protocol learned host)	32,000 (shared between IPv6 ND and protocol learned host) ( FM-E ) 235,000 ( FM-E2)
IPv4 LPM routes (LPM heavy mode)	Not applicable	Not applicable	Cisco Nexus 9236C, 9272Q, and 92304QC switches: 262,000 Cisco Nexus 92160YC-X switches: 650,000	262,000	786,000 / 720,000 (with out/with urpf enabled)	786000 / 734000 (with out/with urpf enabled)	786,000 / 734,000 (with out/with urpf enabled)	786,000
IPv6 LPM routes (LPM heavy mode)	Not applicable	Not applicable	Cisco Nexus 9236C, 9272Q, and 92304QC switches: 131,000 (/64 prefix length); 1900 (non /64 LPM scale) Cisco Nexus 92160YC-X switches: 294,000 (/64 prefix length); 1900 (non /64 LPM scale)	131,000	353,000 / 324,000 (with out/with urpf enabled) (/64 prefix length); 1900 (non /64 prefix length)	442,000 / 412,000 (with out/with urpf enabled)	442,000 / 412,000 (with out/with urpf enabled)	235,000 (/64 prefix length); 3900 (non /64 prefix length) ( FM-E) 235,000 ( any prefix len) ( FM-E2
IPv4 host routes (dual-host mode)	Not applicable	Not applicable	163,000	163,000	262,000	262,000	262,000	Not applicable
IPv6 host routes (dual-host mode)	Not applicable	Not applicable	81,000	81,000	131,000	131,000	131,000	Not applicable
IPv4 LPM routes (dual-host mode)	Not applicable	Not applicable	6000	8000	6000	7000	7000	Not applicable
IPv6 LPM routes (dual-host mode)	Not applicable	Not applicable	1900	1900	1900	1900	1900	Not applicable
IPv4 ARP (dual-host mode)	Not applicable	Not applicable	64,000	64,000	64,000	64,000	64,000	Not applicable
IPv6 ND (dual-host mode)	Not applicable	Not applicable	64,000	64,000	64,000	64,000	64,000	Not applicable
IPv4 host routes (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	1 Million (protocol learned host)	1 Million (protocol learned host)	1 Million (protocol learned host)	1 Million (protocol learned host)
IPv6 host routes (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)	5000,000	5000,000	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)
IPv4 LPM routes (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	1 Million	1 Million	1 Million	1 Million
IPv6 LPM routes (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	500,000 (Prefix length 0-83) 1900 (Prefix length /84-127)	500,000	500,000	176,947 (Prefix 0-47) 500,000 (Prefix length 48-83) 1900 (Prefix length /84-127)
IPv4 ARP (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)	32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)	32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)	32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)
IPv6 ND (internet peering mode)	Not applicable	Not applicable	Not applicable	Not applicable	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)	16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)
IS-ISv4 adjacencies (either L1, L2, or sum of L1 and L2 with default timers)	255	255	255	255	255	255	255	255
IS-ISv4 BFD sessions (with default timers)	255	255	Not applicable	255	255	255	255	255
IS-ISv4 routes	10,000	10,000	10,000	10,000	10,000	10,000	10,000	10,000
IS-ISv4 network type	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast	Point to point, broadcast
OSPFv2 neighbors	1000	256	256	256	256	256	256	1000
OSPFv3 neighbors	1000	256	256	256	256	256	256	1000
OSPF/OSPFv3 LSA/LSDB size	100,000	100,000	100,000	100,000	100,000	100,000	100,000	100,000
OSPF/OSPFv3 areas	100	100	100	100	100	100	100	100
VRFs	1000	1000	1000	1000	1000	1000	1000	1000
VRRP groups per interface or I/O module	250	250	250	250	250	250	250	250

Policy-based routing (PBR)
Configured sequences per policy	256	256	128	Not applicable	128	128	128	128
Next-hop addresses per policy	32	32	32	Not applicable	32	32	32	32
IPv4 ACEs (unidimensional)	3072 (per network forwarding engine)	3072 (per network forwarding engine)	3582 (per network forwarding engine)	Not applicable	3582 (per network forwarding engine)	3582 (per network forwarding engine)	3582 (per network forwarding engine)	3582 (per network forwarding engine)
IPv6 ACEs (unidimensional)	1536 (per network forwarding engine)	1536 (per network forwarding engine)	1792 (per network forwarding engine)	Not applicable	1792 (per network forwarding engine)	1792 (per network forwarding engine)	1792 (per network forwarding engine)	Not applicable
IPv4 and IPv6s ACEs	2048 IPv4 + 256 IPv6	2048 IPv4 + 256 IPv6	1024 IPv4 + 128 IPv6	Not applicable	1024 IPv4 + 128 IPv6	1024 IPv4 + 128 IPv6	1024 IPv4 + 128 IPv6	1024 IPv4, IPv6 not applicable
Interfaces with PBR policy	512	512	256	Not applicable	256	256	256	256

VRRPv3
VRRPv3 groups per interface	255	255	255	Not applicable	255	255	255	255
VRRPv3 groups with default timers (1 s)	490	490	490	Not applicable	490	490	490	490
VRRPv3 groups with relaxed timers (3 s)	490	490	490	Not applicable	490	490	490	490
Pathways with one VRRPv3 group with default timer (1 s)	489	489	489	Not applicable	489	489	489	489
VRRPv3 groups and pathways combined	490	490	490	Not applicable	490	490	490	490

¹⁹ The limit of supported BFD sessions for each EoR line card is 75.

²⁰ The hash table is subject to collisions. Depending on the host route pattern, collisions might occur.

²¹ The hash table is subject to collisions. Depending on the host route pattern, collisions might occur.

*For the Cisco Nexus 9200 Platform switches, the default value for LPM unicast routes is 6000 (IPv4) or 1900 (IPv6). You can use the hardware profile multicast max-limit lpm-entries 0 command to increase the number of IPv4 LPM unicast routes to 8000. The hardware profile ipv6 lpm-entries maximum 0 command reserves the entire LPM table for IPv4. With this configuration, the IPv4 LPM scale is 14,000 (with 2000 reserved for multicast by default). This value can be increased to 16,000 with the hardware profile multicast max-limit lpm-entries 0 command. The hardware profile ipv6 lpm-entries maximum 4096 command reserves the entire LPM table for IPv6. With this configuration, the IPv6 LPM scale is 3900. When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used.

Note

The IPv4/IPv6 host routes and the IPv4 multicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.
The IPv4 and IPv6 unicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.
High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Guidelines and Limitations for OSPF Verified Scalability Limits

To achieve the highest scale, we recommend that you use a single OSPF instance instead of multiple instances.
Each OSPFv2 and OSPFv3 scale value might vary when combined with other parameters.
The graceful restart timeout value might need to be increased in multi-dimensional scenarios.

Table 14. PVLAN VXLAN Verified Scalability Limits (Unidimensional)
Feature	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit
Primary VLANs	16	16	16
Secondary VLANs	20	20	20
Ports in community host mode	40	40	40
Port in Isolated host mode	40	40	40
Ports in isolated trunk mode	40	40	40
Ports in promiscuous mode	5	5	5
PVLANs allowed on a PVLAN port	16	16	16

Table 15. VXLAN Verified Scalability Limits (Unidimensional)
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9300-FX2 Platform Verified Limit	9500 with X97xx-EX/FX Verified Limit
IGMP snooping over VXLAN
VXLAN VLANs	Not applicable	1000	1000	1000	1000	1000	1000
VTEP Peers²²	Not applicable	256	256	256	256	256	256
Underlay multicast groups	Not applicable	128	128	128	128	128	128
VXLAN Flood and Learn
Virtual network identifiers (VNIs) or VXLAN-mapped VLANs	1000	2000	2000	2000	2000	2000	1000
Underlay multicast groups	128	128	128	128	128	128	128
Overlay MAC addresses	64,000	64,000	64,000	90,000	90,000	60,000	90,000
Remote VXLAN tunnel endpoints (VTEPs Multicast)	256	256	256	256	256	256	256
Ingress replication peers²³	256	256	256	256	256	256	256
Ingress replication Layer 2 VNIs	1000	1000	1000	1000	1000	1000	1000
MAC addresses for ingress replication	64,000	64,000	64,000	90,000	90,000	90,000	90,000
Port VLAN translations under an interface	100	100	Not applicable	4000	4000	500	100
Port VLAN translations in a switch	200	2000	Not applicable	24000	24000	6000	2000
Static MAC addresses pointing to a remote VTEP	1000	1000	1000	1000	1000	1000	1000
VXLAN VLAN logical port VP count	7000	7000	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
VXLAN VLANs per FEX port (host interface)	75	75	Not applicable	75²⁴	Not applicable	Not applicable	Not applicable
Layer 2 routed VNIs for vPC-centralized gateway	450	450	450	450	450	450	450
IGMP groups	8192	8192	8192	8192	8192	8192	8192
VXLAN BGP eVPN
Layer 2 VNIs	1000	2000	2000	2000, 4000 (with no Layer 3 VNIs)	2000, 4000 (with no Layer 3 VNIs)	2000, 4000 (with no Layer 3 VNIs)	1000
Xconnect VLANs	Not applicable	40	Not applicable	40	40	40	Not applicable
SVI with Distributed Anycast Gateway; Layer 2 VNI extended	1000	2000	2000²⁵	2000²⁶	2000	2000	1000
Layer 3 VNIs / VRFs²⁷	750	900	900	900	900	900	750
Underlay multicast groups	128	128	128	128	128	128	128
VTEPs	256	256	256	256	256	256	256
MAC addresses	64,000	64,000	64,000	90,000	90,000	90,000	90,000
IPv4 host routes	60,000	60,000	60,000	458,000	471,000	471,000	656,000
IPv6 host routes	7000	7000	7000	24,000	265,000	265,000	34,000
Overlay IPv4 LPM routes	12,000	12,000	8000	458,000	471,000	471,000	656,000
Overlay IPv6 LPM routes	7000	7000	2000	206,000²⁸	265,000	265,000	174,000²⁹
VXLAN VLAN logical port VP count	7000	10000	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
VXLAN VLANs per FEX port (host interface)	75	75	Not applicable	Not applicable ³⁰	Not applicable	Not applicable	Not applicable
IGMP groups	8192	8192	8192	8192	8192	8192	8192
VXLAN BGP eVPN Ingress Replication
Layer 2 VNIs	1000	2000	2000	2000	2000	2000	1000
Xconnect VLANs	Not applicable	40	Not applicable	40	40	40	Not applicable
SVI with Distributed Anycast Gateway; Layer 2 VNI extended	1000	2000	2000³¹	2000³²	2000	2000	1000
Layer 3 VNIs / VRFs³³	750	900	900	900	900	900	750
VTEPs	256	256	256	256	256	256	256
MAC addresses	64,000	64,000	64,000	90,000	90,000	90,000	90,000
IPv4 host routes	32,000	32,000	32,000	458,000	471,000	471,00	656,000
IPv6 host routes	7000	7000	7000	24,000	265,000	265,000	34,000
Overlay IPv4 LPM routes	12,000	12,000	8000	458,000	471,000	471,000	656,000
Overlay IPv6 LPM routes	7000	7000	2000	206,000³⁴	265,000	265,000	174,000³⁵
VXLAN VLAN logical port VP count	7000	7000	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
VXLAN VLANs per FEX port (host interface)	75	75	Not applicable	Not applicable	Not applicable	Not applicable	Not applicable
IGMP groups	8192	8192	8192	8192	8192	8192	8192

²² In case of IR, each VNI can have a max of 64 peers.

²³ In case of IR, each VNI can have a max number of 64 peers

²⁴ This is the limit for the Cisco Nexus 93180YC-EX and other fiber based switches. All copper based 9300-EX switches are not applicable.

²⁵ Only 1900 SVI are supported if dual stack is used/IPv6 is used.

²⁶ Only 1900 SVI are supported if dual stack is used/IPv6 is used.

²⁷ ECMP objects are not shared across multiple VRFs.

²⁸ All /64 routes + 4000 for non /64 routes.

²⁹ All /64 routes + 4000 for non /64 routes.

³⁰ This particular combination has not been validated but the feature is supported.

³¹ Only 1900 SVI are supported if dual stack is used/IPv6 is used.

³² Only 1900 SVI are supported if dual stack is used/IPv6 is used.

³³ ECMP objects are not shared across multiple VRFs.

³⁴ All /64 routes + 4000 for non /64 routes.

³⁵ All /64 routes + 4000 for non /64 routes.

Table 16. Tetration Verified Scalability Limits (Unidimensional)
	92160YC-X Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit
TCAM size	1024 entries	1024 entries	1024 entries
	IPv4 – 4 entries per rule (TCP, UDP, ICMP, and IP)	IPv4 – 2 entries per rule (ICMP and IP)	IPv4 – 2 entries per rule (ICMP and IP)
	IPv6 – 16 entries per rule (4 entries per TCP, UDP, ICMPv6, and IPv6 for a total of 16 entries)	IPv6 – 8 entries per rule (4 entries per ICMP and IPv6 for a total of 8 entries)	IPv6 – 8 entries per rule (4 entries per ICMP and IPv6 for a total of 8 entries)
	(24 entries out of 1000 is consumed for default)	(24 entries out of 1000 is consumed for default)	(24 entries out of 1000 is consumed for default)
TCAM scale	250 (IPv4) or 62 (IPv6)	500 (IPv4) or 125 (IPv6)	500 (IPv4) or 125 (IPv6)
VRF match	Not applicable	Not applicable	Not applicable

The entire Cisco Tetration Analytics documentation set is available at the following URL: https://www.cisco.com/c/en/us/support/data-center-analytics/tetration-analytics/tsd-products-support-series-home.html

Deployment Case Studies

This section provides sample topologies for some common deployments. For each topology, the scalability numbers are the limits with all of the listed features enabled at the same time.

Attention

These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the "Verified Scalability Limits" section.

Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9636PQ line cards. The N9K-X9636PQ line cards are used in normal mode and breakout mode. Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale.

The Cisco Nexus 9508 switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes. The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), and Layer 2 scalability and other routing, switching, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. The scale numbers listed here exceed those used by most customers in their topologies. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 17. Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)
Feature	9508 Verified Limit (Max-Host Routing Mode)
Fully loaded chassis	1 N9K-X9636PQ, 1 N9K-X9564TX, 2 N9K-X9564PX, 1 N9K-X9432PQ, 1 N9K-X9536PQ
Physical interfaces enabled	276
Multicast S,G routes	653
Multicast *,G routes	500
IPv4 unicast routes (LPM)	5000
IPv6 unicast routes (LPM)	850
IPv4 ARP	65,000
IPv6 ND	40,000
MAC addresses	90,000
VLANs	490
vPCs*	200
OSPFv2 neighbors	20
OSPFv3 neighbors	4
BGP (IPv4) neighbors	65
BGP (IPv6) neighbors	65
SVIs	490
STP logical ports	2800 (RPVST)
HSRP VLANs (IPv4/IPv6)	490
Virtual ports	700
Port channel links	8

* The number of VLANs per vPC supported should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

Layer 2/Layer 3 Aggregation Topology (Default Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9516 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9536PQ line cards. The chassis is fully loaded with five line cards configured for breakout mode. The Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale. The Cisco Nexus 9516 nodes are running in default routing mode. The Cisco Nexus 3164Q switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes.

The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), Layer 2 scalability, IPv4 and IPv6 LPM routing, Layer 2 and Layer 3 multicast routing for IPv4, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 18. Layer 2/Layer 3 Aggregation Topology (Default Routing Mode)
Feature	9516 Switch Verified Limit (Default Routing Mode)	9300 Platform Verified Limit (Default Routing Mode)
Chassis configuration	5 N9K-X9432PQ line cards 4 N9K-X9464PX line cards 3 N9K-X9536PQ line cards 3 N9K-X9464TX line cards 1 N9K-X9564TX line card	9372
Physical ports	1335	50
vPCs	303	24
SVIs	450	450
VRFs	100	100
IPv4 ARP	40,000	40,000
IPv6 ND	10,000	10,000
STP logical ports	10,000	6000
BGP neighbors (IPv4 + IPv6)	502 + 502	502 + 502
IPv4 LPM routes	50,000	6000
IPv6 LPM routes	10,000	1000
BFD (IPv4 + IPv6)	300	102
IGP OSPFv2 neighbors	502	502
IGP OSPFv3 neighbors	502	502
HSRP (IPv4 + IPv6)	450 + 450	450 + 450
IGMP groups	2000	2000
Multicast *,G routes	2000	2000
Multicast S,G routes	8000	6000
Tracking objects	450	450
VLANs	500	500
PIM neighbors	502	502
MAC addresses	60,000	60,000
Network address translation (NAT)	Not applicable	756
sFlow	256	32

FEX System Topology

The FEX 9500 multi-dimensional scale topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) pairs. Each switch has multiple X9564PX line cards. Each switch has 32 FEX uplinks connected to them. The FEX 9300 multi-dimensional scale topology consists of two Cisco Nexus 9396PX switches used in vPC mode along with 16 FEX uplinks connected to each switch. Multiple FEXs of type Nexus 2248TP-E, 2232PP, 2248PQ, and 2348UPQ are used.

The switches are used at the Layer 2 and Layer 3 boundary and are also configured as VXLAN VTEPs. The FEX host ports are operating as Layer 2 ports. The switches are configured as gateways with the use of SVI interfaces.

Table 19. FEX System Topology
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit
Fabric Extenders	32	16
Up interfaces	1100	560
Port channels	426	256
vPC members	390	360
VLANs	744	416
PVLAN VLANs	56	56
Secondary VLANs per primary VLAN	25	25
MAC addresses	45,000	25,000
HSRP	365	365
ARP	12,000	10,000
Neighbor discovery (ND)	5000	5000
Multicast (*,G)	4000	4000
Multicast (S,G)	4000	4000

Multicast System Topology

Two Cisco Nexus 9508 switches are configured as vPC peers in one domain, and two Cisco Nexus 9372PX switches are configured as vPC peers in the other domain. The chassis are fully loaded with N9K-X9432PQ, N9K-X9464PX, N9K-X9536PQ, N9K-X9564PX, N9K-X9564TX, and N9K-X9636PQ line cards. eBGP routing is used to connect these two PIM domains. OSPF is used as IGP in one domain, and EIGRP is configured in the other domain. This setup is configured with multiple rendezvous points (RPs) to serve different multicast group ranges. BSR is used to advertise RP information in both of these PIM domains. PIM anycast is used in one domain, and MSDP anycast is used in the other domain for redundancy and load balancing. Static RP configuration is also used for a range of multicast groups.

The Cisco Nexus 9516 and Cisco Nexus 7000 Series switches are used as Layer 3 core routers in one domain. The Cisco Nexus 3164Q switches are used as Layer 3 core routers in the other domain. This topology also includes the Cisco Nexus 9396PX, Cisco Nexus 9372PX, and Cisco Nexus 3016/3064T switches in the access layer.

In addition to including Layer 2/Layer 3 IPv4 multicast routing, this topology also covers IPv4 and IPv6 host and LPM routing and Layer 2 unicast forwarding. All interfaces are configured for dual stack.

Table 20. Multicast System Topology
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit
Chassis configuration	N9K-X9636PQ, N9K-X9536PQ, N9K-X9564PX, N9K-X9564TX, N9K-X9432PQ, N9K-X9464PX, N9K-X9432PQ, C3164PQ	C9372PX, C9396PX, C3164PQ
Multicast S,G routes	17,500	5000
Multicast *,G routes	2500 (IGMP) 12500 (snooping)	500 (IGMP) 2500 (snooping)
Sources	2000, 200, 100, 40, 10, 3, 2, 1	2000, 200, 100, 40, 10, 3, 2, 1
Replications	40	20
ECMPs	16	8
SVIs	200	200
HSRP/VRRP	200 HSRP	100 VRRP
MAC addresses	40,000	10,000
ARP	20,000	4000
Unicast LPM IPv4 routes	20,000	4000
Unicast LPM IPv6 routes	10,000	1000
IPv4 ARP	18,000	4000
IPv6 ND	4000	2000
MSDP peers (fully mesh)	4	4
Anycast RPs (MSDP and PIM anycast) ³⁶	2 MSDP	2 PIM anycast
IPv4 multicast routes with PIM bidirectional groups	8000	8000

³⁶ This multicast system topology consists of two multicast PIM domains. The Multicast Source Discovery Protocol (MSDP) is used to exchange multicast source information between these two domains.

VXLAN BGP/eVPN iBGP Centric Topology

This VXLAN BGP/eVPN iBGP centric topology consists of Cisco Nexus 9300 and 9500 Platform switches acting as VXLAN vPC tunnel endpoints (VTEPs) and VXLAN non-vPC VTEPs. VXLAN VTEPs establish iBGP sessions to a Cisco Nexus 9508 switch (route reflector) acting as a spine node. VXLAN-distributed anycast gateway SVIs are configured for dual stack, and the traffic is dual stack.

The focus of this topology is to test VXLAN overlay network scale and underlay Layer 2 switching and other routing, multicast, and Layer 4 through Layer 7 features for management and operations. Underlay PIM neighbors and IS-IS adjacency were tested with the default timer and Bidirectional Forwarding Detection (BFD) enabled on all links.

Table 21. VXLAN BGP/eVPN iBGP Centric Topology
Feature	9500 Platform Verified Limit	9300 Platform Verified Limit	9200 Platform Verified Limit	9300-EX Platform Verified Limit	9300-FX Platform Verified Limit	9364C Platform Verified Limit	9500 with X9700-EX Verified Limit
System Routing Template	default	default	default	default³⁷	default³⁸	Not applicable	default³⁹
VXLAN VTEPs	128	128	128	128	128	128	128
VXLAN Layer 2 VNIs	1000	1000	1000	1000	1000	1000	1000
VXLAN Layer 3 VNIs/VRFs	500	500	500	500	500	500	500
VXLAN multicast groups	128	128	128	128	128	128	128
VXLAN overlay MAC addresses	60,000	60,000	60,000	60,000	60,000	60,000	60,000
VXLAN overlay IPv4 host routes	60,000	60,000	60,000	60,000	60,000	60,000	60,000
VXLAN overlay IPv6 host routes	4000	4000	4000	4000	4000	Not applicable	4000
VXLAN overlay IGMP Snooping groups	2000⁴⁰	2000	2000	2000	2000	2000	2000
VXLAN IPv4 LPM routes	10000	10000	5120	5120	5120	5120	5120
VXLAN IPv6 LPM routes	2000	2000	1500	1500	1500	Not applicable	1500
VXLAN VLAN logical port VP count	5200	5200	5200	5200	5200	Not applicable	5200
VLANs on VTEP node	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)	1700 (total VLANs) 1500 (VXLAN VLANs) 200 (non-VXLAN VLANs)
MST instances	40	40	40	40	40	Not applicable	40
STP logical ports	3500	3500	3500	3500	3500	Not applicable	3500
vPC port channels	50	20	20	20	20	Not applicable	20
Underlay IS-IS neighbors	64	32	32	32	32	Not applicable	32
Underlay PIM neighbors	200	200	200	200	200	Not applicable	200
Underlay HSRP groups for regular VLANs	200	200	200	200	200	Not applicable	200
Underlay vPC SVIs	200	200	200	200	200	Not applicable	200

³⁷ The vxlan-routing-template needs to be configured on 7.0(3)I5(1).

³⁸ The vxlan-routing-template needs to be configured on 7.0(3)I5(1).

³⁹ The vxlan-routing-template needs to be configured on 7.0(3)I5(1).

⁴⁰ IGMP Snooping on vxlan vlan on 9500 series switch supported from 7.0(3)I5(2) release onwards.

Verified Scalability Limits for Cisco Nexus 9508 and Cisco Nexus 9504 Switches

Verified Scalability Limits (Unidimensional)

The tables in this section list the verified scalability limits for the Cisco Nexus 9508 switch with an X9636C-R, X9636C-RX, or X9636Q-R line card or a C9508-FM-R fabric module and Cisco Nexus 9504 with -R line cards for Cisco NX-OS Release 9.2(1). These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Table 22. Interfaces Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
Port channel member links	32
SVIs	3,967
vPCs	255

Table 23. Label Switching Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
Forwarding Equivalence Classes (FECs)	1,000
Equal-cost multipaths (ECMPs)	8-way
FECs ECMPs	8,000
IAS option B labels	450,000
Layer 3 VPN routes	450,000

Table 24. Layer 2 Switching Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
MAC addresses	192,000
MST instances	64
MST virtual ports	236,000
RPVST virtual ports	13,750
VLANs	3,967
VLANs in RPVST mode	250

Table 25. Layer 3 Multicast Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
IPv4 multicast routes	32,000 (Layer 3)
Outgoing interfaces (OIFs)	16 OIFs for 32K mroutes or 287 OIFs for 1000 mroutes
PIM neighbors	500

Table 26. Security Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
IPv4 ingress access control entries (ACEs)	RACL on LC N9K-X9636C-RX: 100,000 PACL on LC N9K-X9636C-RX: 12,000 RACL-2048, PACL-1024 (without TCAM Carving) IPv4 52640 ACEs per system PACL IPv4: 1024 TCAM entries in internal TCAM PACL MAC: 2048 TCAM entries in internal TCAM RACL IPv4: 2048 TCAM entries in internal TCAM
IPv6 ingress access control entries (ACEs)	RACL-1024, PACL-1024 (without TCAM Carving) IPv6 25200 ACEs per system PACL IPv6: 1024 TCAM entries in internal TCAM RACL IPv6: 1024 TCAM entries in internal TCAM
System ACL	12000 TCAM entries in internal TCAM . 64000 TCAM entries in external TCAM .

Table 27. System Management Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

SPAN and ERSPAN

Configurable SPAN or ERSPAN sessions

Active SPAN or ERSPAN sessions

Active localized SPAN or ERSPAN session per line card

32 sessions across ports on single line card

Active localized SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)

32 sessions, 128 sources and 1 destination

Destination interfaces per SPAN session

Source VLANs per SPAN or ERSPAN

PTP secondary ports

64 for 9508-R line cards

Note

PTP Offload is supported on 9508-R line cards.

Table 28. Layer 3 Unicast Routing Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
BFD sessions (echo mode)	288
BGP neighbors	272
HSRP groups	490
IPv4 ARP	48,000
IPv4 host routes	750,000 (X9636C-R) 1,000,000 (X9636C-RX)
IPv6 host routes	62,000 256,000 (X9636C-RX)
IPv6 ND	32,000
IPv4 unicast routes (LPM)	192,000 (X9636C-R) 1,000,000 (X9636C-RX)
IPv6 unicast routes (LPM)	62,000 256,000 (X9636C-RX)
OSPFv2 neighbors	1,000
OSPFv3 neighbors	1,000
OSPF/OSPFv3 LSA/LSDB size	250,000
OSPF/OSPFv3 areas	15
VRFs	3,967
VRRP groups per interface or I/O module	16

Table 29. Tunnels Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
VxLAN BGP eVPN
Layer 3 VNIs / VRFs	100
VTEPs	256
IPv4 host routes	128,000
IPv6 host routes	32,000

Table 30. HSRP Verified Scalability Limits (Unidimensional)
Feature	Verified Limit
Groups with default timers (3s/10s) and multiple group optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]	7,928
Groups with aggressive timers (1s/3s) and multiple groups optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]⁴¹	7,928
Number of groups per interface or I/) module	Maximum 16 (Because 16 is the unique virtual MAC address limit)

⁴¹ If the user has Multi-protocol configuration, user should configure appropriate CoPP policies so as to avoid any control plane traffic drops.

Verified Scalability Limits (Multidimensional)

The tables in this section list the verified scalability limits for the Cisco Nexus 9508 switch with an X9636C-R, X9636C-RX, or X9636Q-R line card or a C9508-FM-R fabric module and Cisco Nexus 9504 with -R line cards for Cisco NX-OS Release 9.2(1). These limits are validated with a multidimensional configuration. The values provided in these tables focus on the scalability of all listed features at the same time.

Attention

These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the "Verified Scalability Limits" section.

Table 31. eBGP/IS-IS Profile Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
Number of 100G ports	288
ECMP	16-way (Upstream)
BGP neighbors	960
BGP IPv4 /32 unicast routes	30,000
BGP IPv4 VLSM unicast routes	18,000
BGP IPv6 /128 unicast routes	16,000
BGP IPv6 VLSM unicast routes	1,000
IS-IS v2 neighbors	255
IS-IS v3 neighbors	255
IS-IS L2 adjacency	16
IS-IS IPv4 /32 unicast routes	20,000
IS-IS IPv4 VLSM unicast routes	1,000
IS-IS IPv6 /128 unicast routes	20,000
IS-IS IPv6 VLSM unicast routes	1,000
BFD sessions	272
PIM neighbors	256
ACL ACEs	15,000 500
Sub-interfaces	712
SPAN sessions	1 local SPAN session
Multicast SSM	20,000

Table 32. iBGP/OSPF Profile Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
Number of 100G ports	180
Number of 40G ports	108
ECMP	8-way (Upstream)
BGP neighbors	8
BGP IPv4 VLSM unicast routes	40,000
BGP IPv6 VLSM unicast routes	10,000
OSPFv2 neighbors	108
OSPFv3 neighbors	30
OSPF IPv4 /32 unicast routes	100,000
OSPF IPv4 VLSM unicast routes	155,000
OSPFv3 IPv6 /128 unicast routes	1,000
OSPFv3 IPv6 VLSM unicast routes	9,000
BFD sessions	108
VRF	250
PIM neighbors	108
IPv4 (*,G) multicast routes	2,000
IPv4 (S,G) multicast routes	10,000
ACL ACEs	500 (IPv4) 500 (IPv6)
SPAN sessions	1 local SPAN session

Table 33. iBGP/EIGRP Profile Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
Number of 100G ports	180
Number of 40G ports	108
ECMP	16-way (Upstream)
BGP neighbors	8
BGP IPv4 VLSM unicast routes	40,000
BGP IPv6 VLSM unicast routes	10,000
EIGRP v4 neighbors	276
EIGRP v6 neighbors	276
EIGRP IPv4 /32 unicast routes	30,000
EIGRP IPv4 VLSM unicast routes	1,000
EIGRP IPv6 /128 unicast routes	30,000
EIGRP IPv6 VLSM unicast routes	1,000
BFD sessions	276
VRF	250
PIM neighbors	276
IPv4 (*,G) multicast routes	6,000
IPv4 (S,G) multicast routes	16,000
ACL ACEs	500 (IPv4) 500 (IPv6)
SPAN sessions	1 local SPAN session

Table 34. MPLS Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
MPLS L3VPN	3967
VPE	3967
PE nodes	3
PE routes	20,000
X9636C-RX line card: ACL - IPv4	95,000
X9636C-RX line card: ACL - IPv6	20,000
HSRP, HSRP VIP	3967 each for v4 and v6
vPC uRPF	3967
Strict uRPF	Yes
VRF	3967
SVI	3967
Layer 3 VPN routes IP ECMP	2,000
MPLS LSR ECMP	2,000
VPNv4 routes	400,000
VPNv6 routes	90,000
EBGP neighbors	750

Table 35. Layer 2/Layer 3 Boundary Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
Number of 100G ports	72
Number of 40G ports	36
vPC Port channels	31
ECMP	8-way (Upstream)
OSPFv2 neighbors	47
OSPFv3 neighbors	47
OSPF IPv4 /32 unicast routes	45,000
OSPF IPv4 VLSM unicast routes	1,000
OSPF IPv6 /128 unicast routes	25,000
OSPF IPv6 VLSM unicast routes	1,000
BFD sessions	49
VRF	250
VLAN	3,750
SVI	3,750
VRRP v4 groups	1,996 VRRS / 4 VRRPv3
VRRP v6 groups	1,996 VRRS / 4 VRRPv3
HSRP IPv4	1,743 Secondary groups / 7 Primary groups
HSRP IPv6	1,743 Secondary groups / 7 Primary groups
PIM neighbors	396
IPv4 (*,G) multicast routes	3,080
IPv4 (S,G) multicast routes	26,600
IGMP snooping database entries	6,400
sFlow enabled interfaces	65
UDLD enabled interfaces	70
SPAN sessions	1 local SPAN session

Table 36. Segment Routing Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
VLAN	100
SVI	100
MAC entries	10,000
ARP entries	70
HSRPv4 VIPs	100
HSRP v6 VIPs	100
LACP	11
LACP members	4
eBGP IPv6 neighbors	9
eBGP IPv4 LU neighbors	9
IPv4 (LU) routes	6888
IPv4 (LU) paths	17580
IPv6 routes	6,663
IPv4 (LU) routes	17,338
SR ECMP	18 (dual-homed)
MPLS HW entries	11,957

Table 37. VXLAN Profile Verified Scalability Limits (Multidimensional)
Feature	Verified Limit
Ports	16
ECMP	8-way (Upstream)
BGP neighbors	200
BGP EVPN Layer 2 VPN host routes	64,000
BGP IPv4 VLSM unicast routes or ospf	10,000
BGP IPv6 VLSM unicast routes or ospf	6,000
BFD sessions	20
PIM neighbors	20
IPv4 (*,G) multicast routes (co-existing)	4,000
IPv4 (S,G) multicast routes (co-existing)	2,000
Layer 3 VNI	100
Layer 2 VNI	400
Local VTEP	1
Remote VTEPs	205
VLAN	400
SVI	100
MAC	80,000

Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Release 9.2(1)

Available Languages

Download Options

Bias-Free Language

Verified Scalability Limits

Introduction

Verified Scalability Limits

Deployment Case Studies

Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

Layer 2/Layer 3 Aggregation Topology (Default Routing Mode)

FEX System Topology

Multicast System Topology

VXLAN BGP/eVPN iBGP Centric Topology

Verified Scalability Limits (Unidimensional)

Verified Scalability Limits (Multidimensional)

Was this Document Helpful?

Contact Cisco

This Document Applies to These Products