Remote leaf switches extend ACI services and APIC management to remote data centers. These switches operate as part of an existing fabric pod without requiring local spine switches or APIC controllers.
A remote leaf switch is a Cisco ACI leaf switch deployed in a remote data center that connects to an existing fabric pod and extends ACI services and APIC management.
-
Operates without a local spine switch or APIC controller.
-
Functions like a local leaf switch in the fabric pod.
-
Uses Virtual Extensible LAN (VXLAN) over Layer 3 for unicast traffic and Head End Replication (HER) for broadcast, unknown-unicast, and multicast (BUM) traffic.
Operational characteristics and requirements
APIC automatically discovers remote leaf switches when they connect, enabling centralized management. Remote leaf switch behavior and support vary by release:
-
Before Release 4.0(1), inter-VRF traffic is forwarded to the spine switch before the fabric forwards the traffic.
-
Before Release 4.1(2), you must delete the vPC before you decommission a remote leaf switch.
-
Release 4.0(1) reduces WAN bandwidth usage for PBR, ERSPAN, and shared services by decoupling these services from the spine proxy. This release also improves Layer 3 endpoint resolution.
-
Release 4.1(2) supports direct traffic forwarding between remote leaf switches in different locations and improves redundancy during spine switch failures.
-
Release 4.2(4) supports a minimum inter-pod network (IPN) bandwidth of 10 Mbps for management-only traffic. This release also supports 802.1Q tunnels that map multiple VLANs to a single tunnel.
Remote leaf switch behavior in Release 4.0(1)
Beginning with Release 4.0(1), remote leaf switches have the following characteristics:
-
WAN bandwidth usage is reduced by decoupling the following services from the spine proxy:
-
PBR: For local PBR devices or PBR devices behind a vPC, local switching is used without forwarding traffic to the spine proxy. For PBR devices on orphan ports on a peer remote leaf switch, an RL-vPC tunnel is used. This behavior applies whether the spine link to the main data center is available or unavailable.
-
ERSPAN: For peer destination EPGs, an RL-vPC tunnel is used. EPGs on local orphan or vPC ports use local switching to reach the destination EPG. This behavior applies whether the spine link to the main data center is available or unavailable.
-
Shared services: Packets bypass the spine proxy path, which reduces WAN bandwidth usage.
-
Inter-VRF traffic is forwarded through an upstream router instead of the spine switch.
-
This enhancement applies only to a remote leaf vPC pair. Communication across remote leaf pairs continues to use a spine proxy.
-
-
Unknown Layer 3 endpoints are resolved through the top-of-rack (ToR) glean process at a remote leaf location when the spine proxy is unreachable.
Remote leaf switch behavior in Release 4.1(2)
Before Release 4.1(2), all local traffic within a remote leaf vPC pair is switched directly between physical or virtual endpoints at the remote leaf location, as shown in the following figure.
Before Release 4.1(2), traffic between remote leaf switch vPC pairs within the same remote location or across remote locations is forwarded to the spine switches in the ACI main data center pod, as shown in the following figure.
Beginning with Release 4.1(2), remote leaf switches support direct traffic forwarding between different remote locations. Direct traffic forwarding provides redundancy and availability for connections between remote locations, as shown in the following figure.
Beginning with Release 4.1(2), remote leaf switches also have the following characteristics:
-
When direct traffic forwarding is enabled and a spine switch fails in a single-pod configuration, the following behavior occurs:
-
Local switching continues for existing and new endpoint traffic between remote leaf switch vPC peers, as shown in the "Local switching traffic before Release 4.1(2)" figure.
-
For traffic between remote leaf switches in different remote locations, the following behavior occurs:
-
New endpoint traffic fails because the tunnel between the remote leaf switch and spine switch is unavailable. New endpoint information cannot be synchronized from the remote leaf switch to the spine switch. Therefore, other remote leaf switch pairs in the same or different locations cannot download the new endpoint information from COOP.
-
For unidirectional traffic, existing remote endpoints age out after 300 seconds, after which traffic fails. Bidirectional traffic within a remote leaf site, including traffic between remote leaf vPC pairs in the same pod, is refreshed and continues to function. Bidirectional traffic to other remote leaf locations is affected when COOP expires the remote endpoints after 900 seconds.
-
For shared services, or inter-VRF traffic, bidirectional traffic between endpoints on remote leaf switches in two remote locations in the same pod fails after the remote leaf switch COOP endpoint age-out time of 900 seconds. This failure occurs because the COOP session between the remote leaf switch and spine switch is unavailable. Shared-services traffic between endpoints on remote leaf switches in two different pods fails after the COOP fast-aging time of 30 seconds.
-
L3Out-to-L3Out communication stops because the BGP session to the spine switches is unavailable.
-
-
-
For unidirectional remote leaf direct traffic that originates on one remote leaf switch and is destined for another remote leaf switch that is not the vPC peer of the source, traffic is interrupted for a millisecond whenever the 300-second remote endpoint (XR EP) timeout occurs.
-
In a remote leaf switch deployment with ACI Multi-Site, traffic from a remote leaf switch continues to other pods and remote locations after a spine switch failure because the traffic uses an alternate available pod.
10-Mbps bandwidth support in the IPN for remote leaf switches
In some deployments, most remote leaf switch data traffic remains local, and the inter-pod network (IPN) is used only for management. These deployments might not require a 100-Mbps IPN. Beginning with Release 4.2(4), the IPN supports a minimum bandwidth of 10 Mbps for these deployments.
The following requirements apply to 10-Mbps IPN deployments:
-
Use the IPN path only to manage remote leaf switches. Management functions include upgrades, downgrades, discovery, COOP, and policy deployment.
-
Configure quality of service (QoS) on the IPN to prioritize control-plane and management-plane traffic between the Cisco ACI data center and remote leaf switch pairs. For configuration information, see the "Creating DSCP Translation Policy Using Cisco APIC GUI" section.
-
All traffic from the Cisco ACI data center and remote leaf switches uses the local L3Out.
-
Do not stretch an EPG or bridge domain between a remote leaf switch and the ACI main data center.
-
Pre-download software images to the remote leaf switches to reduce upgrade time.
The following figure shows the management traffic for a remote leaf switch through the IPN.
802.1Q tunnel support on remote leaf switches
A colocation provider might host multiple customers that each use thousands of VLANs per remote leaf switch pair. Beginning with Release 4.2(4), you can create an 802.1Q tunnel between a remote leaf switch and the ACI main data center. The tunnel maps multiple VLANs to a single 802.1Q tunnel, which reduces EPG scale requirements.
The following figure shows 802.1Q tunnel support on remote leaf switches.
To create an 802.1Q tunnel between a remote leaf switch and the ACI main data center, follow the instructions in the "802.1Q Tunnels" chapter of the Cisco APIC Layer 2 Networking Configuration Guide, available from the Cisco APIC documentation landing page.
You can configure remote leaf switches by using the APIC GUI, with or without a wizard, the REST API, or the NX-OS-style CLI.