Configuring vPC from a Host to Fabric Extenders (FEX) in ACI
Available Languages
Contents
Introduction
This document describes the configuration steps you use in order to configure Fabric Externders (FEX) as well as Virtual Port Channel (vPC) from a Host to those FEX's in an Application Centric Infrastructure (ACI) Environment.
Topology
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
The information in this document is based on these hardware and software versions:
- An ACI fabric that consists of two spine switches and two leaf switches
- 2 Supported Cisco FEX's, each plugged into their own leaf
- An Application Policy Infrastructure Controller (APIC)
- A host with connections to each FEX
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Configure
Setup vPC Between the 2 Leaf Switches
This configuration will walk you through setting up a vPC domain between 2 Leaf switches in ACI. The Leaf switches used in this section are labeled "101" and "102"
Navigate to Fabric > Access Policies > Switch Policies > Policies > Virtual Port-Channel default and click on the "+" next to "Explicit VPC Protection Groups". Enter the name, the 2 leaf switches you would like to vPC between, and the "Logical Pair ID" ("Domain ID" in traditional Nexus OS).
We can now Verify on the CLI that the vPC domain has been configured. SSH (Secure Shell) to one of the swithes and run "show vpc brief"
fab1-leaf1# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 10 Peer status : peer adjacency formed ok vPC keep-alive status : Disabled Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 4 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 up -
In ACI, the keepalive status is disabled since we use the Spine - Leaf connections as the "peer-link" internally.
Bring the FEX's Online
We can now register the FEX's with each Leaf switch. We do not support vPC connections between a FEX and 2 leaf switches, so each FEX must only be connected to 1 leaf switch.
Navigate to Fabric > Access Policies > Quick Start and click on "Configure an interface, PC, and VPC". CLick the Green "+" to add switches. Select the "Advanced" Radio Button. Enter the switch ID for the leaf that connects to the first FEX. Enter a name for the Switch Profile and add the FEX type and number. Finally, enter the port on the leaf that the FEX connects to.
Update, all fileds, save once and submit. At this time, you should see the FEX come online from the switch CLI. If the images are different, it will download the correct image from the leaf.
fab1-leaf1# show fex detail
FEX: 101 Description: FEX0101 state: Online
FEX version: 11.1(1o) [Switch version: 11.1(1o)]
FEX Interim version: 11.1(1o)
Switch Interim version: 11.1(1o)
Extender Model: N2K-C2232PP-10GE, Extender Serial: SSI1350063T
Part No: 68-3547-03
Card Id: 82, Mac Addr: 00:0d:ec:fa:4b, Num Macs: 75
Module Sw Gen: 22 [Switch Sw Gen: 21]
pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/36
Fabric interface state:
Eth1/36 - Interface Up. State: Active
Po5 - Interface Up. State: Active
Repeat the same process for the second FEX and Leaf switch.
NOTE: When looking at the Port-Channel or Physical Interfaces on the Leaf, that connect to the FEX, you will see "input discards" incrementing. This is normal and should be ignored. Input discards are raised due to a limitation of the Broadcom ASIC (Generation 1 Nexus 9000 switches) with Link Layer Control Packets. Under Normal operation, you will see input discards so monitoring may want to be disabled for these interfaces.
Configuring Host Interfaces on the FEX
There are 2 Ways to configure the host interfaces on each FEX to be in a vPC.
1) Add the interface to each individual FEX profile, and map them to the same vPC policy group.
2) Assign Both FEX's the same FEX profile and add the access port selectors.
After completing the wizard, you will see a "FEX" Profile and a "Interface Selector" for each FEX.
The "FEX Profile" is a profile that the FEX will be associated too. The FEX profile is similar to the "Switch Profile" where it links the interface to the FEX.
The Interface Selector is the policy that is created for the leaf interfaces that connect to the FEX.
Option 1 : Add the interface to each individual FEX profile, and map them to the same vPC policy group.
Navigate to Fabric > Access Policies > Interface Policies > Profiles and click on the FEX profile for FEX 101. Right Click and select "Create Access Port Selector". In this example, the host is plugged into port 10 on both FEX's.
Name the selector, enter the interface ID and the vPC policy group you want to map the interface to:
Complete the same for the Leaf 102 interface selector and make sure to use the Interface Policy Group so both get bundled into a vPC.
You can then verify that the interfaces are bundled on both switches by running the command "show port-channel summary" and "show vpc brief" from the Leaf CLI. Each switch may have it's own PC number but they should both map back to the same vPC ID :
fab1-leaf1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/16(P)
2 Po2(SU) Eth LACP Eth1/17(P)
3 Po3(SD) Eth NONE Eth1/48(D)
5 Po5(SU) Eth LACP Eth1/4(P)
6 Po6(SU) Eth NONE Eth1/36(P)
7 Po7(SU) Eth NONE Eth101/1/10(P)
fab1-leaf3# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/4(P)
2 Po2(SU) Eth LACP Eth1/17(P)
3 Po3(SU) Eth LACP Eth1/16(P)
4 Po4(SD) Eth NONE Eth1/48(D)
6 Po6(SU) Eth NONE Eth1/36(P)
7 Po7(SU) Eth NONE Eth102/1/10(P)
fab1-leaf1# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 13 Peer status : peer adjacency formed ok vPC keep-alive status : Disabled Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 5 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 up - vPC status ---------------------------------------------------------------------- id Port Status Consistency Reason Active vlans -- ---- ------ ----------- ------ ------------ 2 Po7 up success success -
fab1-leaf3# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 13 Peer status : peer adjacency formed ok vPC keep-alive status : Disabled Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 5 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 up - vPC status ---------------------------------------------------------------------- id Port Status Consistency Reason Active vlans -- ---- ------ ----------- ------ ------------ 2 Po7 up success success -
Option 2 : Assign Both FEX's the same FEX profile and add the access port selectors.
If Multiple vPC's will be configured, the confguration can get tedious to go into each individual FEX profile and add the interface(s). In order to simplify the configuration, you can assign mulitple FEX's the same FEX Profile that will then go and deploy the access ports on each FEX using that profile.
In order to do this, Navigate to Fabric > Access Policies > Interface Policies > Profiles, Right click and select "Ceate FEX Profile". Name it and click submit:
Now that we have the FEX Profile defined, we will need to map FEX 101 and 102 to this unique profile. If you expand out the FEX interface selector, you will see where you can choose the FEX profile from the dropdown, make sure to choose the profile created (101-102 in this example):
Repeat the same thing for the FEX 102 Interface Selector.
We can then Add Access Port Blocks to the "101-102" FEX Profile to deploy those interfaces on both FEX's simultaneously.
Right Click on the "101-102" FEX Profile and Choose "Create Access Port Selector". Name it and choose the interface and policy group like the section above:
You can then verify that the interfaces are bundled on both switches by running the command "show port-channel summary" and "show vpc brief" from the Leaf CLI. Each switch may have it's own PC number but they should both map back to the same vPC ID :
fab1-leaf1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/16(P)
2 Po2(SU) Eth LACP Eth1/17(P)
3 Po3(SD) Eth NONE Eth1/48(D)
5 Po5(SU) Eth LACP Eth1/4(P)
6 Po6(SU) Eth NONE Eth1/36(P)
7 Po7(SU) Eth NONE Eth101/1/10(P)
fab1-leaf3# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/4(P)
2 Po2(SU) Eth LACP Eth1/17(P)
3 Po3(SU) Eth LACP Eth1/16(P)
4 Po4(SD) Eth NONE Eth1/48(D)
6 Po6(SU) Eth NONE Eth1/36(P)
7 Po7(SU) Eth NONE Eth102/1/10(P)
fab1-leaf1# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 13 Peer status : peer adjacency formed ok vPC keep-alive status : Disabled Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 5 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 up - vPC status ---------------------------------------------------------------------- id Port Status Consistency Reason Active vlans -- ---- ------ ----------- ------ ------------ 2 Po7 up success success -
fab1-leaf3# show vpc brief Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 13 Peer status : peer adjacency formed ok vPC keep-alive status : Disabled Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : secondary Number of vPCs configured : 5 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Enabled (timeout = 240 seconds) Operational Layer3 Peer : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ -------------------------------------------------- 1 up - vPC status ---------------------------------------------------------------------- id Port Status Consistency Reason Active vlans -- ---- ------ ----------- ------ ------------ 2 Po7 up success success -
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)