New and Changed Information
The following table provides an overview of the significant changes to the organization and features in this guide from the time the guide was first published to the latest update.
Date |
Changes |
---|---|
December 22, 2023 |
First release of this document. |
Overview
This guide contains the maximum verified scalability limits for Cisco Multi-Site.
These values are based on a profile where each feature was scaled to the numbers specified in the tables. These numbers do not represent the theoretically possible scale.
Note |
The total number of objects within each site must not exceed the maximum verified scalability limit for that fabric version. For more information on site-specific scalability limits, see the Cisco ACI Verified Scalability Guide, Cisco Cloud Network Controller Verified Scalability Guide, or Cisco NDFC Verified Scalability Guide for your fabric type and release. |
ACI Fabrics Scalability Limits
This release supports managing only NDFC fabrics or only ACI fabrics by the same Nexus Dashboard Orchestrator. The following scale limits apply when managing ACI fabrics.
Object |
Scale |
||
---|---|---|---|
Sites |
Up to 100 sites total on-boarded in Nexus Dashboard and managed by Nexus Dashboard Orchestrator. Up to 14 of those sites can be Multi-Site-enabled with EVPN sessions between them. For specific details about template object scale, which depends on the type of the templates you deploy (Multi-Site vs Autonomous), see the Table 5 and Table 6 tables below. |
||
Pods per site |
12 or 25, depending on the Cisco APIC release managing the site. For more information, see the Cisco APIC Verified Scalability Guide for your release. |
||
Leaf switches per site |
400 in a single pod 500 across all pods in Multi-Pod fabrics Number of leaf switches supported within each fabric depends on the Cisco APIC release managing that fabric. For more information, see the Cisco APIC Verified Scalability Guide for your release. |
||
Total leaf switches across all sites |
(max number of sites) * (max number of leaf switches per site) , for example:
Note that specific objects' scale (such as VRFs, BDs, EPGs, etc.) still applies, as described in the template-specific sections below. |
||
Endpoints per site |
The NDO endpoint scale for each site is the same as the scale supported by the site's APIC. For detailed information, see the Cisco APIC Verified Scalability Guide for the APIC release version managing each site.
|
Note |
If a specific object's scale (such as contracts, filters, or VRFs) is not included in the following table, that object does not have a unique scale limit and the general "Policy Objects per Schema" and "Policy Objects per Template" limits apply. If any such objects were explicitly listed in previous releases, those limitations have been lifted and removed from the list. |
Object |
Scale |
---|---|
Schemas |
1000 |
Templates per Schema |
30 |
Service Graphs per Schema |
500 |
Service Graph nodes per Service Graph |
5 for Autonomous templates 2 for Multi-Site templates |
Policy Objects per Schema |
2000 |
Policy Objects per Template |
2000 |
Contract Preferred Group (BD/EPG combinations) |
5000 |
PBR destinations per site (including all local and remote*) *Note that if you configure some of the new PBR use cases, such as vzAny with PBR or L3Out-to-L3Out with PBR, you may be required to implement hair-pinning of traffic across sites to ensure traffic can be always steered via both service devices deployed in the source and destination sites. As a result, the leaf nodes in a given site must be programmed with PBR information about the service device(s) in remote sites as well, and those remote PBR nodes are counted toward the maximum number listed here. |
1500 |
Note |
The following table applies to the Tenant Policies, Fabric Policies, Fabric Resource Policies, and Monitoring Policies templates introduced in Release 4.0(1). For Application template scalability information, see the previous tables in this document. |
Object |
Scale |
---|---|
Policy Objects per Template |
500 |
Monitoring Policy Scale |
|
ERSPAN Sessions |
20 per site |
Fabric SPAN Sessions |
30 per site |
Note |
The following table applies to the L3Out templates introduced in Release 4.1(1). For Application template scalability information, see the previous tables in this document. |
Object |
Scale |
---|---|
IP L3Outs per Template |
100 |
SR-MPLS L3Outs per Template |
100 |
All other objects scale |
The scale for other L3Out template objects that are not explicitly listed in this table is the same as the scale supported by the site's APIC. For detailed information, see the Cisco APIC Verified Scalability Guide for the APIC release version managing each site. |
NDO-Deployed Objects Scale
To better understand the scalability values captured in the following table for traditional Multi-Site deployments, it is important to clarify that there are three kind of NDO-deployed objects:
-
Site local objects—these are the objects defined in templates associated to a single site, which get deployed by NDO only in that specific site.
-
Shadow objects—these are the objects deployed by NDO in a site as a result of a contract established between site local and remote objects, they are the representation ("shadow)" of the remote object in the local site.
-
Stretched objects—these are the objects defined in templates that are associated to multiple sites, which get deployed by NDO concurrently on all those sites.
The table below captures the maximum number of objects that NDO can deploy in a given site and includes the sum of all three kinds of objects described above.
For example, if you have two sites and you define three templates on NDO—template-1
associated to site-1
, template-2
associated to site-2
, and template-stretched
associated to both site-1
and site-2
—then:
-
If you configure and deploy
EPG-1
intemplate-1
, this will count as one EPG towards maximum allowed forsite-1
. -
If you configure and deploy
EPG-2
intemplate-2
, this will count as one EPG towards maximum allowed forsite-2
. -
If you apply a contract between
EPG-1
andEPG-2
or add both EPGs to the Preferred Group), a shadowEPG-2
will be created insite-1
and a shadowEPG-1
insite-2
. As a result, two EPGs will now be counted towards maximum allowed in each site. -
Finally, if you configure and deploy
EPG-3
intemplate-stretched
, it will count as another EPG in each site, bringing the total to 3 EPGs towards maximum allowed scale.
It is worth adding that the maximum number of objects supported in a given fabric (and captured in the Verified Scalability Guide for Cisco APIC) must not exceed the sum of objects locally defined on APIC plus the objects pushed from NDO to that site (NDO-deployed objects).
Note |
For maximum scale Nexus Dashboard Orchestrator configurations with many features enabled simultaneously, we recommend that those configurations be tested in a lab before deployment. |
Object |
Maximum number of objects per site for up to 4 sites |
Maximum number of objects per site for 5-14 sites |
---|---|---|
Tenants |
1000 |
400 |
VRFs |
2000 |
1000 |
BDs |
6000 |
4000 |
Contracts |
6000 |
4000 |
EPGs |
6000 |
4000 |
Isolated EPGs |
500 |
500 |
Microsegment EPGs |
2000 |
500 |
L3Out external EPGs |
500 |
500 |
Subnets |
8000 |
8000 |
L4-L7 logical devices |
400 |
400 |
Graph instances |
250 |
250 |
Device clusters per tenant |
10 |
10 |
Number of graph instances per device cluster |
125 |
125 |
Beginning with Release 4.0(1), Nexus Dashboard Orchestrator adds support for autonomous sites. When creating Multi-Site application templates, you can now choose to designate the template as Autonomous. This allows you to associate the template to one or more sites that are operated independently and are not connected through an Inter-Site Network (no intersite VXLAN communication).
Because autonomous sites are by definition isolated and do not have any intersite connectivity, there is no shadow object configuration across sites and no cross-programming of pctags or VNIDs in the spine switches for intersite traffic flow.
The autonomous templates allow for significantly higher deployment scale as shown in the following table. Since there are no stretched objects or shadow objects, the scale values shown in the table below reflect the specific site-local objects that the Orchestrator deploys in each site. This is in contrast to the scale for Multi-Site templates listed above, where the numbers represent the maximum number of objects across all sites.
Object |
Maximum number of objects per site |
---|---|
Tenants |
1000 |
VRFs |
2000 |
BDs |
6000 |
Contracts |
6000 |
EPGs |
6000 |
Isolated EPGs |
500 |
Microsegment EPGs |
2000 |
L3Out external EPGs |
500 |
Subnets |
8000 |
Number of L4-L7 logical devices |
400 |
Number of graph instances |
250 |
Number of device clusters per tenant |
10 |
Number of graph instances per device cluster |
125 |
SD-Access and Cisco ACI Integration Scale
Starting with Release 3.7(1), you can onboard a Cisco DNA Center (DNAC) to your Nexus Dashboard Orchestrator for SD-Access and ACI integration.
Note |
Cisco Nexus Dashboard and Cisco DNAC integration allows for automation of a subset of network connectivity and macro segmentation scenarios across Nexus and campus SDA fabric deployments. This integration is under limited availability. Please contact your Cisco representative for additional information |
The following scale limits apply for this use case:
-
Only a single DNAC can be onboarded to your Nexus Dashboard Orchestrator for SD-Access and ACI integration.
-
Up to 2 Cisco ACI sites are supported for peering with SD-Access.
Each ACI site can be a single Pod or a Multi-Pod fabric.
-
Multiple SD-Access (campus) sites are supported if managed by a single DNAC.
-
A virtual network (VN) can be mapped to a maximum of 10 ACI VRFs.
-
Up to 32 virtual networks from the SD-Access domain can be extended into the ACI domain.
VRF/BD VNID Translation Scale
Object |
Scale |
---|---|
Fixed spines |
21,000 |
Modular spines |
42,000 |
NDFC Fabrics Scalability Limits
This release of Nexus Dashboard Orchestrator supports managing only NDFC fabrics or only ACI fabrics by the same Nexus Dashboard Orchestrator. The following scale limits apply when managing NDFC fabrics.
Object |
Scale |
---|---|
Sites A "site" in NDO context is equivalent to an NDFC "fabric". |
13 |
Fabrics per NDFC instance |
5 |
Border Gateways per site |
4 |
Object |
Scale |
---|---|
Schemas |
80 |
Templates per Schema |
10 |
Policy Objects per Schema |
1000 |
Note |
When NDO manages NDFC fabrics, there is no concept of "shadow" objects. Hence, the scalability values captured in the table below only refer to the sum of site-local and stretched objects deployed by NDO in a given fabric. |
Object |
Scale (Stretched) |
---|---|
VRFs |
500 |
Networks |
1000 (L3) 1500 (L2) |