Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Releases 6.1(2)I3(4) and 6.1(2)I3(4a)

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Book Contents

Verified Scalability Limits

Find Matches in This Book

Available Languages

Download Options

Book Title

Cisco Nexus 9000 Series NX-OS Verified Scalability Guide, Releases 6.1(2)I3(4) and 6.1(2)I3(4a)

Chapter Title

Verified Scalability Limits

PDF - Complete Book (1.9 MB) PDF - This Chapter (1.12 MB)
View with Adobe Reader on a variety of devices
ePub - Complete Book (107.0 KB)
View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi - Complete Book (164.0 KB)
View on Kindle device or Kindle app on multiple devices

Results

Updated:: March 13, 2015

Chapter: Verified Scalability Limits

Introduction
Verified Scalability Limits
Deployment Case Studies
- Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

Verified Scalability Limits

This chapter describes the Cisco NX-OS configuration limits for the Cisco Nexus 9000 Series switches.

Introduction
Verified Scalability Limits
Deployment Case Studies

Introduction

The values provided in this guide should not be interpreted as theoretical system limits for Cisco Nexus 9000 Series hardware or Cisco NX-OS software. These limits refer to values that have been validated by Cisco. They can increase over time as more testing and validation is done.

Verified Scalability Limits

The tables in this section list the verified scalability limits for Cisco NX-OS Releases 6.1(2)I3(4) and 6.1(2)I3(4a). These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.

Table 1 Cisco Nexus 2000 Series Fabric Extenders (FEX) Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
Fabric Extenders and Fabric Extender server interfaces	Not applicable	16 and 768
VLANs per Fabric Extender	Not applicable	2000 (across all the Fabric Extenders)
VLANs per Fabric Extender server interface	Not applicable	75
Port channels	Not applicable	500

Note

The Cisco Nexus 2000 Series Fabric Extender is supported only with the Cisco Nexus 9396PX and Cisco Nexus 9372PX chassis.

Table 2 Interfaces Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
Generic routing encapsulation (GRE) tunnels	8	8
Port channel links	32	8
SVIs	490	250
vPCs	275	100 (280 with Fabric Extenders)

Table 3 Layer 2 Switching Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
MST instances	64	64
MST virtual ports	85,000	48,000
RPVST virtual ports	22,000	12,000
VLANs	4000	3900
VLANs in RPVST mode	500	500

Private VLANs (PVLANs)
Primary VLANs	16	16
Secondary VLANs	20	20
Ports in community host mode	40	40
Ports in isolated host mode	20	40
Ports in isolated trunk host mode	22	40
Ports in promiscuous mode	48	5
Ports in promiscuous trunk mode	80	5
PVLANs allowed on a PVLAN trunk port	16	16

Note

The number of supported VLANs per vPC should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

Note

For the Cisco Nexus 9500 Series switches, traffic is validated for 16 PVLAN groups, though there is no limit for configuring the number of primary or secondary VLANs. Also, there is no limit on the number of PVLANs that can be created as it becomes a function of the TCAM hardware table size limitation for PVLANs with SVIs and virtual MAC addresses enabled. The table can hold up to 507 entries and is used by regular VLANs and PVLANs. When SVIs are enabled on PVLANs, the number of PVLANs is limited by this resource availability. A message appears when the value exceeds this hardware resource.

Note

For the Cisco Nexus 9300 Series switches, a maximum of 45 PVLAN ports at a given time is validated, with 40 in host mode and 5 in promiscuous mode. The 40 in host mode can be all isolated, isolated trunk, or community host ports.

Table 4 Multicast Routing Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
IPv4 multicast routes	32,000	8000
Outgoing interfaces (OIFs)	40	40

Note

The IPv4 multicast routes and the IPv4/IPv6 host routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

Note

High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Table 5 Security Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
IPv4 ingress ACLs	3072 (per network forwarding engine)	3072 (per network forwarding engine)
IPv4 egress ACLs	768 (per network forwarding engine)	768 (per network forwarding engine)
IPv6 ingress ACLs	1536 (per network forwarding engine)	1536 (per network forwarding engine)
IPv6 egress ACLs	256 (per network forwarding engine)	256 (per network forwarding engine)

Note

The ACL scalability limits also apply to policy-based ACLs (PBACLs).

Table 6 System Management Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
Configurable SPAN or ERSPAN sessions	32	4
Active SPAN or ERSPAN sessions¹	4 to 32, based on the number of line cards and the session configuration	4
Active localized SPAN or ERSPAN sessions per line card²	4	4
Source interfaces per SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)	48	48
Destination interfaces per SPAN session	1 (physical interface)	1 (physical interface)
VLAN sources per SPAN or ERSPAN session	32	32

¹ A single forwarding engine instance supports four SPAN or ERSPAN sessions. For Cisco Nexus 9300 Series switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources. This limitation might also apply to Cisco Nexus 9500 Series switches, depending on the SPAN or ERSPAN source's forwarding engine instance mappings.

² The number of SPAN or ERSPAN sessions per line card reduces to two if the same interface is configured as the bidirectional source in more than one session.

Table 7 Unicast Routing Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
BGP neighbors	1000	150
HSRP groups per interface or I/O module	490	250
IPv4 ARP	60,000	45,000
IPv4 host routes	88,000 60,000 (max host mode)	90,000
IPv6 host routes	20,000 30,000 (max host mode)	40,000
IPv6 ND	30,000	20,000
IPv4 unicast routes (LPM)	128,000 16,000 (max host mode) 128,000 with no IPv6 routes (64-bit ALPM mode)	12,000
IPv6 unicast routes (LPM)	20,000 4000 (max host mode) 80,000 with no IPv4 routes (64-bit ALPM mode)	6000 (5000 routes < /64, 1000 routes > /64)
IPv4 and IPv6 unicast routes (LPM) in 64-bit ALPM mode	x IPv6 routes and y IPv4 routes, where 2x + y <= 128,000	Not applicable
MAC addresses	90,000	90,000
OSPFv2 neighbors	1000	200
OSPFv3 neighbors	300	200
VRFs	1000	500
VRRP groups per interface or I/O module	250	250
Policy-based routing (PBR)
Configured sequences per policy	256	256
Next-hop addresses per policy	32	32
IPv4 ACEs (unidimensional)	3072 (per network forwarding engine)	3072 (per network forwarding engine)
IPv6 ACEs (unidimensional)	1536 (per network forwarding engine)	1536 (per network forwarding engine)
IPv4 and IPv6 ACEs	2048 IPv4 + 256 IPv6	2048 IPv4 + 256 IPv6
Interfaces with PBR policy	512	512

Note

The IPv4/IPv6 host routes and the IPv4 multicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

Note

The IPv4 and IPv6 unicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

Note

High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Guidelines and Limitations for OSPF Verified Scalability Limits

To achieve the highest scale, we recommend that you use a single OSPF instance instead of multiple instances.
Each OSPFv2 and OSPFv3 scale value might vary when combined with other parameters.
The graceful restart timeout value might need to be increased in multi-dimensional scenarios.

Table 8 VXLAN Verified Scalability Limits (Unidimensional)
Feature	9500 Series Verified Limit	9300 Series Verified Limit
Virtual network identifiers (VNIs) or VXLAN-mapped VLANs	Not applicable	1000
Overlay multicast groups	Not applicable	128
Overlay MAC addresses	Not applicable	64,000
Remote VXLAN tunnel endpoints (VTEPs)	Not applicable	256

Deployment Case Studies

This section provides sample topologies for some common deployments. For each topology, the scalability numbers are the limits with all of the listed features enabled at the same time.

Attention:

These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see Verified Scalability Limits.

Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9636PQ line cards. The N9K-X9636PQ line cards are used in normal mode and breakout mode. Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale.

The Cisco Nexus 9508 switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes. The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), and Layer 2 scalability and other routing, switching, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. The scale numbers listed here exceed those used by most customers in their topologies. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 9 Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)
Feature	9500 Series Verified Limit (Max-Host Routing Mode)	9300 Series Verified Limit
Fully loaded chassis	6 N9636PQ line cards + 1 N9564TX line card + 1 N9564PX line card + 6 fabric modules + 2 system controllers + 2 supervisors	Not applicable
Physical interfaces enabled	300	Not applicable
Multicast S,G routes	500	Not applicable
Multicast *,G routes	500	Not applicable
IPv4 unicast routes (LPM)	2400	2200
IPv6 unicast routes (LPM)	2200	2200
IPv4 ARP	60,000	5000
IPv6 ND	30,000	5000
MAC addresses	90,000	72,000
VLANs	500 (RPVST)	3900 (MST)
vPCs*	275	100
OSPFv2 neighbors	16	200
OSPFv3 neighbors	16	200
BGP (IPv4) neighbors	64 (eBGP)	150 (iBGP)
BGP (IPv6) neighbors	64 (eBGP)	150 (iBGP)
SVIs	490	250
MST instances	Not applicable	64
HSRP VLANs (IPv4/IPv6)	490	250
Virtual ports	3000 (RPVST)	24,000 (MST)
Port channel links	32	8

* The number of VLANs per vPC supported should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

Was this Document Helpful?

Feedback

Contact Cisco

Open a Support Case
(Requires a Cisco Service Contract)