Available Languages

Download Options

PDF (220.0 KB)
View with Adobe Reader on a variety of devices
ePub (59.2 KB)
View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi (Kindle) (87.5 KB)
View on Kindle device or Kindle app on multiple devices

Updated:August 10, 2017

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Cisco ACI Multi-Site Release Notes,
Release 1.0(1)

This document describes the features, caveats, and limitations for the Cisco Application Centric Infrastructure Multi-Site software.

The Cisco Application Centric Infrastructure (ACI) Multi-Site is an architecture that allows the application to define the networking requirements in a programmatic way. This architecture simplifies, optimizes, and accelerates the entire application deployment life cycle.

The Cisco ACI Multi-Site Fundamentals guide provides complete details about the Cisco ACI Multi-Site, including a glossary of terms that are used in the Cisco ACI Multi-Site.

Additional product documentation is listed in the "Related Documentation" section.

Release notes are sometimes updated with new information about restrictions and caveats. See the following website for the most recent version of this document:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

You can watch videos that demonstrate how to perform specific tasks in the Cisco ACI Multi-Site on the Cisco ACI YouTube channel:

https://www.youtube.com/c/CiscoACIchannel

Table 1 shows the online change history for this document.

Table 1 Online History Change

Date	Description
August 10, 2017	Release 1.0(1i) became available.
August 31, 2017	Added the Cisco ACI Multi-Site Hardware Requirements Guide to the following sections: · Compatibility Information and Hardware Requirements · New Hardware Features · Table 13 Hardware Documentation · New Documentation
October 05, 2017	Added associating a contract with the external EPG to the Usage Guidelines section.
January 17, 2018	Added the Cisco ACI Multi-Site and Cisco APIC Compatibility Matrix section.
March 8, 2019	Updated Usage Guidelines, DHCP relay for bridge domains that are stretched across sites is not supported.

This document includes the following sections:

■ Compatibility Information and Hardware Requirements

■ Cisco ACI Multi-Site and Cisco APIC Compatibility Matrix

■ Usage Guidelines

■ Verified Scalability Limits

■ New and Changed Information

■ Caveats

■ Related Documentation

Compatibility Information and Hardware Requirements

■ This release supports the hardware listed in the Cisco ACI Multi-Site Hardware Requirements Guide.

Cisco ACI Multi-Site and Cisco APIC Compatibility Matrix

Cisco ACI Multi-Site	Cisco APIC	Supported
1.0(1)	3.0(1)	Yes

Usage Guidelines

This section lists usage guidelines for the Cisco ACI Multi-Site software.

■ Up to Five Interconnected Sites are Supported

■ Multi-POD is not Supported

ACI cannot be deployed with multipod and Multi-Site configurations at the same time. If you are using multipod, you must disable multipod to use the APIC fabric with Multi-Site.

■ Site Local Bridge Domain with a Shared Service Relation to an L3Out Cannot be Stretched

A bridge domain that is associated with an L3Out in a shared service configuration cannot be stretched to multiple sites.

■ Shared Service Configuration is not Supported for L3Outs

It is not supported to have an L3out that is either providing or consuming a contract from a VRF that is different from its own VRF. Multi-Site does not create l3extInstP mappings. Therefore, intersite L3Out communication across VRFs is not supported.

■ Proxy ARP Glean and Unknown Unicast Flooding are not Supported Together

Unknown Unicast Flooding and ARP Glean are not supported together in Multi-Site across sites.

■ STP and "Flood in Encapsulation" Options are not Supported with Multi-Site

In Multi-Site topologies, regardless of whether EPGs are stretched between sites or localized, STP packets do not reach remote sites. Similarly, the "Flood in Encapsulation" option is not supported across sites. In both cases, packets are encapsulated using an FD VNID (fab-encap) of the access VLAN on the ingress TOR. It is a known issue that there is no capability to translate these IDs on the remote sites.

■ GOLF and Tenant L3Outs Must be Dedicated (Not Shared)

Each site must deploy a local L3Out connection. When configuring L3Outs for various sites, each L3Out must be dedicated (not shared). If you plan to configure GOLF in addition to Multi-Site, they both need separate L3Out policies to the IPN, but they can share a physical interface, as long as the same interface IP address is used.

■ VMM and Physical Domains Must be Configured in the APIC GUI at the Site

Although domains (VMM and physical) must be configured in APIC, Domain associations can be configured in APIC or Multi-Site.

■ Some VMM Domain Options Must be Configured in the APIC GUI

The following VMM domain options must be configured in the APIC GUI at the site:

· NetFlow/EPG CoS marking in a VMM domain association

· Encapsulation mode for an AVS VMM domain

■ L3Outs Must be Configured in the APIC GUI, For Each Site

Although tenant L3Outs and L3extInstPs must be created in the APIC GUI for each site, network mappings between them are configured in the Multi-Site GUI.

NOTE: The subnet in the L3extInstP must be the same for all inter-related sites (and variable length network masks are not supported).

■ Some uSeg EPG Attribute Options Must be Configured in the APIC GUI

The following uSeg EPG attribute options must be configured in the APIC GUI at the site:

· Sub-criteria under uSeg attributes

· match-all and match-any criteria under uSeg attributes

■ Site IDs Must Be Unique

In Multi-Site, site IDs must be unique.

■ To Change an APIC Fabric ID, You Must Erase and Reconfigure the Fabric

APIC fabric IDs cannot be changed. To change an APIC fabric ID, you must erase the fabric configuration and reconfigure it.

■ Caution: When Removing a Spine Switch Port from Multi-Site Infrastructure Configuration

When removing a spine switch port from the Multi-Site infrastructure, perform the following steps:

1. Click Sites.

2. Click Configure Infra.

3. Click the site where the spine switch is located.

4. Click the spine switch.

5. Click the x on the port details.

6. Click Apply.

■ Shared Services Use Case: Order of Importing Tenant Policies

When deploying a provider site group and a consumer site group for shared services by importing tenant policies, deploy the provider tenant policies before deploying the consumer tenant policies. This enables the relation of the consumer tenant to the provider tenant to be properly formed.

■ Caution for Shared Services Use Case When Importing a Tenant and Stretching it to Other Sites

When you import the policies for a consumer tenant and deploy them to multiple sites, including the site where they originated, a new contract is deployed with the same name (different because it is modified by the inter-site relation). To avoid confusion, delete the original contract with the same name on the local site. In the APIC GUI, the original contract can be distinguished from the contract that is managed by Multi-Site, because it is not marked with a cloud icon.

■ Shadow EPGs and BDs in Shared Services Use Case With Contract Relation Between Different VRFs, When EPGs or BDs are Site Local

When the EPGs in the Shared Services use case provider site group and consumer site group are in different VRFs and communicate through global contracts, the EPGs and bridge domains (BDs) deployed to one group of sites are mirrored in the other group of sites, so that in all these APIC sites they appear to be deployed, when they were actually deployed in only one of the site groups. These mirrored objects are known as "shadow EPGs or BDs".

For example, if the provider site group tenant and VRF are stretched across Site 1 and Site 2, and the consumer site group tenant and VRF are stretched across Site 3 and Site 4, in the APIC GUI at Site 1, Site 2, Site 3, and Site 4, you can see both tenants and their policies. They appear with the same names as the ones that were deployed directly to each site. This is expected behavior and the shadow objects should not be removed.

For more information, see the Schema Managemnet chapter in the Cisco ACI Multi-Site Configuration Guide.

■ DHCP relay for bridge domains that are stretched across sites is not supported.

■ Inter-Site Traffic Cannot Transit Sites

Site traffic cannot transit sites on the way to another site. For example, when Site 1 routes traffic to Site 3, it cannot be forwarded through Site 2.

■ The Cisco ACI Multi-Site GUI includes video demonstrations on the Cisco ACI YouTube channel:

https://www.youtube.com/c/CiscoACIchannel

■ The ? icon in Cisco ACI Multi-Site opens the menu for Show Me How modules, which provide step-by-step help through specific configurations.

· If you deviate while in progress of a Show Me How module, you will no longer be able to continue.

· You must have IPv4 enabled to use the Show Me How modules.

■ User passwords must meet the following criteria:

— Minimum length is 8 characters

— Maximum length is 64 characters

— Fewer than three consecutive repeated characters

— At least three of the following character types: lowercase, uppercase, digit, symbol

— Cannot be easily guessed

— Cannot be the username or the reverse of the username

— Cannot be any variation of "cisco", "isco", or any permutation of these characters or variants obtained by changing the capitalization of letters therein

■ If you are associating a contract with the external EPG, as provider, choose contracts only from the tenant associated with the external EPG. Do not choose contracts from other tenants. If you are associating the contract to the external EPG, as consumer, you can choose any available contract.

Verified Scalability Limits

For the verified scalability limits (except the CLI limits), see the Verified Scalability Guide for this release.

You can access these documents from the following website:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

New and Changed Information

This section lists the new and changed features in this release and includes the following topics:

■ New Software Features

■ New Hardware Features

■ Changes in Behavior

New Software Features

Table 2 lists the new software features in this release:

Table 2 New Software Features, Guidelines, and Restrictions

Feature	Description	Guidelines and Restrictions
Deploying Schemas and Templates (Tenants and Tenant-Policies) to Selective Sites	Cisco ACI Multi-Site includes the ability to deploy the use cases to one or multiple sites.	None.
Importing Tenant Policies from Local Sites and Deploying them to Other Sites	Using the Multi-Site GUI, you can import local APIC tenant policies and deploy them to other sites. Multi-Site also enables you to deploy tenant policies to a single site for testing, and then deploy them to multiple sites when the configuration has been verified.	None.
Migration of Cisco ACI Fabric to Cisco ACI Multi-Site	This is a common Cisco ACI Multi-Site use case, where a tenant is migrated or imported from Cisco ACI fabric to Cisco ACI Multi-Site. For Brownfield configurations, two scenarios are considered for the deployments: · A single pod ACI fabric is in place already. You can add another site in a multi-site configuration. · Two ACI fabrics are in place already (Each fabric is configured as a single pod), the objects (tenants, VRFs, and EPGs) across sites are initially defined with identical names and policies, and they are connected leveraging a traditional L2/L3 DCI solution. With this use case, you can convert an existing configuration to a multi-site configuration by importing the domains or other object types from Cisco ACI Fabric into Cisco ACI Multi-Site.	None.
Multi-Site Cross Launch into Cisco APIC	Multi-Site currently provides support for basic parameters that a user will choose when creating a Tenant and setting up a site. Most of the Tenant policies are supported by Multi-Site, but in addition to that users may want to configure some advanced parameters. The Multi-Site GUI is used to manage the basic properties that a user wants to configure. If there are additional advanced properties that a user wants to configure, the capability to cross launch into Cisco APIC GUI directly from the Multi-Site GUI is provided. If desired by the user, this enables them to configure additional properties directly in Cisco APIC. There are three different access points in Multi-Site GUI from where a user can cross launch into APIC.From these access points in Multi-Site, the user can open a new browser tab with access into Cisco APIC. The user will log in to Cisco APIC at that point for the first time, and the associated screen will be displayed in the Cisco APIC GUI.	None.
Operations Features	With Cisco ACI Multi-Site, you can display the faults for the individual sites, the individual site status, and the overall schema health using the GUI. You can also generate the troubleshooting report and the infrastructure log file for all the schemas, sites, tenants, and users that are managed by Cisco ACI Multi-Site. You can display all tenants in an aggregated view, calculate and display the resource utilization or free resources, and view the aggregated health score of the entire multi-site fabric and all APICs. You can display the schema health by hovering on the individual cells, clicking in the cells, viewing the health score slider, or using the search functionality to find a schema based on the keywords that are typed in the search window.	None.
Register APIC to Multi-Site	The Configuring Infra Using the Multi-Site GUI section provides how to register sites. See the Cisco ACI Multi-Site Configuration Guide.	None.
Register IPN Infra Configuration	The Configuring Infra Using the Multi-Site GUI section provides how to configure fabric connectivity infra. See the Cisco ACI Multi-Site Configuration Guide.	None.
Stretched BD with Layer 2 Broadcast Extension	This is the most basic Cisco ACI Multi-Site use case, in which a tenant and VRF are stretched between sites. The EPGs in the VRF (with their bridge domains (BDs) and subnets), as well as their provider and consumer contracts are also stretched between sites. In this use case, Layer 2 broadcast flooding is enabled across fabrics. Layer 2 broadcast, multicast, and unknown unicast traffic is forwarded across sites leveraging the Head-End Replication (HER) capabilities of the spine nodes that replicate and send the frames to each remote fabric where the Layer 2 BD has been stretched.	None.
Stretched Bridge Domain with No Layer 2 Broadcast Extension	This Cisco ACI Multi-Site use case is similar to the first use case where a tenant, VRF, and their EPGs (with their bridge domains and subnets) are stretched between sites. However, in this use case Layer 2 BUM flooding is disabled between sites. Layer 2 broadcast, multicast and unknown unicast traffic are not forwarded across fabrics over replicated VXLAN tunnels.	None.
Stretched EPG Across Sites	This Cisco ACI Multi-Site use case provides endpoint groups (EPGs) stretched across multiple sites. Stretched EPG is defined as an endpoint group that expands across multiple sites where the underlying networking, site local, and bridge domain can be distinct.	None.
Shared Services with Stretched Provider EPG	In this use case, the Provider EPGs in one group of sites offer shared services and the EPGs in another group of sites consume the services. All sites have local EPGs and bridge domains. In the this use case, at the VRF boundary routes are leaked between VRFs for routing connectivity and by importing contracts to remote sites.	None.
Stretched VRF with Inter-Site Contracts	This Multi-Site use case provides inter-site communication between endpoints connected to different Bridge Domains (BDs) that are part of the same stretched VRF. VRF Stretching is a convenient way to manage EPGs across sites (and the contracts between them). The tenant and VRF are stretched across sites, but EPGs and their policies (including subnets) are locally defined. Because the VRF is stretched between sites, contracts govern cross-site communication between the EPGs. Contracts can be conveniently provided (and consumed) within a site or across sites.	None.
Tenant Management in Cisco ACI Multi-Site	In Multi-Site, after you create a tenant, there are two ways to add tenant policies: · Import a fully configured tenant from an APIC site. · Configure the tenant policies in the Multi-Site GUI. The following tenant policies and their associations can be configured in the Multi-Site GUI: · VRFs · Bridge Domains with subnets and stretched or localized settings · Filters and Contracts · Application Network Profiles with EPGs · Associate EPGs with Physical and VMM domains · Intra-EPG Isolation · Microsegmented EPGs · EPGs deployed on a port, PC, or VPC · Links between L3Outs of different tenants/sites (Network Mappings) Other tenant policies, including L3Outs must be configured in the APIC GUI.	None.
User and Roles	The Cisco ACI Multi-Site provides access according to a user’s role through role-based access control (RBAC). The following user roles are available in Cisco ACI Multi-Site. · Power User—A power user can perform all the operations as an admin user. · Site and Tenant Manager—A site and tenant manager can manage sites, tenants, and associations. · Schema Manager—A schema manager can manage all schemas regardless of tenant associations. · Schema Manager - Restricted —A restricted schema manager can manage schemas that contain at least one tenant to which the user is explicitly associated. · User and Role Manager—A user and role manager can manage all the users, their roles, and passwords.	None.

New Hardware Features

■ The new hardware is listed in the Cisco ACI Multi-Site Hardware Requirements Guide, Release 1.0(1).

Changes in Behavior

This section lists changes in behavior in this release.

Caveats

This section contains lists of open and resolved caveats and known behaviors.

■ Open Caveats

■ Resolved Caveats

■ Known Behaviors

Open Caveats

This section lists the open caveats. Click the bug ID to access the Bug Search tool and see additional information about the bug. If a caveat is fixed in a patch of this release, the "Fixed In" column of the tables specifies the release..

Open Caveats in the 1.0(1) Release

The following table lists the open caveats in the 1.0 release.

Table 3 Open Caveats in the 1.0 Release

Bug ID	Description
CSCvf26024	When Linked L3Outs are Deleted or Modified in APIC, Re-Configure the Network Mappings If you have linked the L3Outs for multiple sites in Multi-Site, and you subsequently delete or modify an L3Out in APIC, you must redeploy the Network Mappings to your sites in the Multi-Site GUI Tenants panel. However, if the L3extInstP subnets are 0/0 on the remote sites, this step is not required.
CSCvf17113	Local Traffic Lost After Enabling Bridge Domain L2STRETCH, INTERSITEBUMTRAFFICALLOW, or OPTIMIZEWANBANDWIDTH Options When a bridge domain (managed by Multi-Site and deployed to a local site) has the L2STRETCH, INTERSITEBUMTRAFFICALLOW, or OPTIMIZEWANBANDWIDTH settings changed in Multi-Site, there may be traffic lost on the local site.
CSCvd59276	For Multi-Site, the Querier IP address Must be Unique on Each Site The APIC fabric querier functions have a distributed architecture, where each leaf switch acts as a querier, and packets are flooded. A copy is also replicated to the fabric port. There is an Access Control List (ACL) configured on each TOR to drop this query packet coming from the fabric port. In the Multi-Site Stretched BD use case with Layer 2 Broadcast Extension, the query packets from each TOR get to the other sites and should be dropped. If the querier IP address is configured the same on all the sites, it is not possible to drop them. To avoid this, configure a unique querier IP address on each site.

Resolved Caveats

There are no resolved caveats in the 1.0(1) release.

Known Behaviors

This section lists caveats that describe known behaviors. Click the Bug ID to access the Bug Search Tool and see additional information about the bug.

Known Behaviors in the 1.0(1) Release

The following table lists caveats that describe known behaviors in the 1.0(1) release.

Table 5 Known Behaviors in the 1.0(1) Release

Bug ID	Description
CSCve40295	Proxy ARP is not supported in Multi-Site Stretched BD without Flooding use case Unknown Unicast Flooding and ARP Glean are not supported together in Multi-Site across sites.
CSCvd61787	STP and "Flood in Encapsulation" Option are not Supported with Multi-Site In Multi-Site topologies, regardless of whether EPGs are stretched between sites or localized, STP packets do not reach remote sites. Similarly, the "Flood in Encapsulation" option is not supported across sites. In both cases, packets are encapsulated using an FD VNID (fab-encap) of the access VLAN on the ingress TOR. It is a known issue that there is no capability to translate these IDs on the remote sites.
CSCvd59276	For Multi-Site, Fabric IDs Must be the Same for All Sites, or the Querier IP address Must be Higher on One Site The APIC fabric querier functions have a distributed architecture, where each leaf switch acts as a querier, and packets are flooded. A copy is also replicated to the fabric port. There is an Access Control List (ACL) configured on each TOR to drop this query packet coming from the fabric port. If the source MAC address is the fabric MAC address, unique per fabric, then the MAC address is derived from the fabric-id. The fabric ID is configured by users during initial bring up of a pod site. In the Multi-Site Stretched BD with Layer 2 Broadcast Extension use case, the query packets from each TOR get to the other sites and should be dropped. If the fabric-id is configured differently on the sites, it is not possible to drop them. To avoid this, configure the fabric IDs the same on each site, or the querier IP address on one of the sites should be higher than on the other sites.

Cisco ACI Multi-Site Release Notes, Release 1.0(1)