Contents

Introduction

Topology

Prerequisites

Requirements

Components Used

Configure

Setup vPC Between the 2 Leaf Switches

Bring the FEX's Online

Configuring Host Interfaces on the FEX

Option 1 : Add the interface to each individual FEX profile, and map them to the same vPC policy group.

Option 2 : Assign Both FEX's the same FEX profile and add the access port selectors.

Introduction

This document describes the configuration steps you use in order to configure Fabric Externders (FEX) as well as Virtual Port Channel (vPC) from a Host to those FEX's in an Application Centric Infrastructure (ACI) Environment.

Topology

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these hardware and software versions:

- An ACI fabric that consists of two spine switches and two leaf switches

- 2 Supported Cisco FEX's, each plugged into their own leaf

- An Application Policy Infrastructure Controller (APIC)

- A host with connections to each FEX

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Configure

Setup vPC Between the 2 Leaf Switches

This configuration will walk you through setting up a vPC domain between 2 Leaf switches in ACI.  The Leaf switches used in this section are labeled "101" and "102"

Navigate to Fabric > Access Policies > Switch Policies > Policies > Virtual Port-Channel default and click on the "+" next to "Explicit VPC Protection Groups".  Enter the name, the 2 leaf switches you would like to vPC between, and the "Logical Pair ID" ("Domain ID" in traditional Nexus OS).

We can now Verify on the CLI that the vPC domain has been configured.  SSH (Secure Shell) to one of the swithes and run "show vpc brief"

fab1-leaf1# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 10

Peer status                       : peer adjacency formed ok

vPC keep-alive status             : Disabled

Configuration consistency status  : success

Per-vlan consistency status       : success

Type-2 consistency status         : success

vPC role                          : primary

Number of vPCs configured         : 4

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Enabled (timeout = 240 seconds)

Operational Layer3 Peer           : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans

--   ----   ------ --------------------------------------------------

1           up     -

In ACI, the keepalive status is disabled since we use the Spine - Leaf connections as the "peer-link" internally.

Bring the FEX's Online

We can now register the FEX's with each Leaf switch.  We do not support vPC connections between a FEX and 2 leaf switches, so each FEX must only be connected to 1 leaf switch.

Navigate to Fabric > Access Policies > Quick Start and click on "Configure an interface, PC, and VPC".  CLick the Green "+" to add switches.  Select the "Advanced" Radio Button.  Enter the switch ID for the leaf that connects to the first FEX.  Enter a name for the Switch Profile and add the FEX type and number.  Finally, enter the port on the leaf that the FEX connects to.

Update, all fileds, save once and submit.  At this time, you should see the FEX come online from the switch CLI.  If the images are different, it will download the correct image from the leaf.

fab1-leaf1# show fex detail

FEX: 101  Description: FEX0101   state: Online

FEX version: 11.1(1o) [Switch version: 11.1(1o)]

FEX Interim version: 11.1(1o)

Switch Interim version: 11.1(1o)

Extender Model: N2K-C2232PP-10GE,  Extender Serial: SSI1350063T

Part No: 68-3547-03

Card Id: 82, Mac Addr: 00:0d:ec:fa:4b, Num Macs: 75

Module Sw Gen: 22  [Switch Sw Gen: 21]

pinning-mode: static    Max-links: 1

Fabric port for control traffic: Eth1/36

Fabric interface state:

Eth1/36 - Interface Up. State: Active

Po5 - Interface Up. State: Active

Repeat the same process for the second FEX and Leaf switch.

NOTE:  When looking at the Port-Channel or Physical Interfaces on the Leaf, that connect to the FEX, you will see "input discards" incrementing.  This is normal and should be ignored.  Input discards are raised due to a limitation of the Broadcom ASIC (Generation 1 Nexus 9000 switches) with Link Layer Control Packets.  Under Normal operation, you will see input discards so monitoring may want to be disabled for these interfaces.

Configuring Host Interfaces on the FEX

There are 2 Ways to configure the host interfaces on each FEX to be in a vPC.

1) Add the interface to each individual FEX profile, and map them to the same vPC policy group.

2) Assign Both FEX's the same FEX profile and add the access port selectors.

After completing the wizard, you will see a "FEX" Profile and a "Interface Selector" for each FEX.

The "FEX Profile" is a profile that the FEX will be associated too.  The FEX profile is similar to the "Switch Profile" where it links the interface to the FEX.

The Interface Selector is the policy that is created for the leaf interfaces that connect to the FEX.

Option 1 : Add the interface to each individual FEX profile, and map them to the same vPC policy group.

Navigate to Fabric > Access Policies > Interface Policies > Profiles and click on the FEX profile for FEX 101.  Right Click and select "Create Access Port Selector".  In this example, the host is plugged into port 10 on both FEX's.

Name the selector, enter the interface ID and the vPC policy group you want to map the interface to:

Complete the same for the Leaf 102 interface selector and make sure to use the Interface Policy Group so both get bundled into a vPC.

You can then verify that the interfaces are bundled on both switches by running the command "show port-channel summary" and "show vpc brief" from the Leaf CLI.  Each switch may have it's own PC number but they should both map back to the same vPC ID :

fab1-leaf1# show port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

I - Individual  H - Hot-standby (LACP only)

s - Suspended   r - Module-removed

S - Switched    R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

F - Configuration failed

-------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

Channel

-------------------------------------------------------------------------------

1     Po1(SU)     Eth      LACP      Eth1/16(P)

2     Po2(SU)     Eth      LACP      Eth1/17(P)

3     Po3(SD)     Eth      NONE      Eth1/48(D)

5     Po5(SU)     Eth      LACP      Eth1/4(P)

6     Po6(SU)     Eth      NONE      Eth1/36(P)

7     Po7(SU)     Eth      NONE      Eth101/1/10(P)

fab1-leaf3# show port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

I - Individual  H - Hot-standby (LACP only)

s - Suspended   r - Module-removed

S - Switched    R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

F - Configuration failed

-------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

Channel

-------------------------------------------------------------------------------

1     Po1(SU)     Eth      LACP      Eth1/4(P)

2     Po2(SU)     Eth      LACP      Eth1/17(P)

3     Po3(SU)     Eth      LACP      Eth1/16(P)

4     Po4(SD)     Eth      NONE      Eth1/48(D)

6     Po6(SU)     Eth      NONE      Eth1/36(P)

7     Po7(SU)     Eth      NONE      Eth102/1/10(P)

fab1-leaf1# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 13

Peer status                       : peer adjacency formed ok

vPC keep-alive status             : Disabled

Configuration consistency status  : success

Per-vlan consistency status       : success

Type-2 consistency status         : success

vPC role                          : primary

Number of vPCs configured         : 5

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Enabled (timeout = 240 seconds)

Operational Layer3 Peer           : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans

--   ----   ------ --------------------------------------------------

1           up     -

vPC status

----------------------------------------------------------------------

id   Port   Status Consistency Reason                     Active vlans

--   ----   ------ ----------- ------                     ------------

2    Po7    up     success     success                    -

fab1-leaf3# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 13

Peer status                       : peer adjacency formed ok

vPC keep-alive status             : Disabled

Configuration consistency status  : success

Per-vlan consistency status       : success

Type-2 consistency status         : success

vPC role                          : secondary

Number of vPCs configured         : 5

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Enabled (timeout = 240 seconds)

Operational Layer3 Peer           : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans

--   ----   ------ --------------------------------------------------

1           up     -

vPC status

----------------------------------------------------------------------

id   Port   Status Consistency Reason                     Active vlans

--   ----   ------ ----------- ------                     ------------

2    Po7    up     success     success                    -

Option 2 : Assign Both FEX's the same FEX profile and add the access port selectors.

If Multiple vPC's will be configured, the confguration can get tedious to go into each individual FEX profile and add the interface(s).  In order to simplify the configuration, you can assign mulitple FEX's the same FEX Profile that will then go and deploy the access ports on each FEX using that profile.

In order to do this, Navigate to Fabric > Access Policies > Interface Policies > Profiles, Right click and select "Ceate FEX Profile".  Name it and click submit:

Now that we have the FEX Profile defined, we will need to map FEX 101 and 102 to this unique profile.  If you expand out the FEX interface selector, you will see where you can choose the FEX profile from the dropdown, make sure to choose the profile created (101-102 in this example):

Repeat the same thing for the FEX 102 Interface Selector.

We can then Add Access Port Blocks to the "101-102" FEX Profile to deploy those interfaces on both FEX's simultaneously.

Right Click on the "101-102" FEX Profile and Choose "Create Access Port Selector".  Name it and choose the interface and policy group like the section above:

You can then verify that the interfaces are bundled on both switches by running the command "show port-channel summary" and "show vpc brief" from the Leaf CLI.  Each switch may have it's own PC number but they should both map back to the same vPC ID :

fab1-leaf1# show port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

I - Individual  H - Hot-standby (LACP only)

s - Suspended   r - Module-removed

S - Switched    R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

F - Configuration failed

-------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

Channel

-------------------------------------------------------------------------------

1     Po1(SU)     Eth      LACP      Eth1/16(P)

2     Po2(SU)     Eth      LACP      Eth1/17(P)

3     Po3(SD)     Eth      NONE      Eth1/48(D)

5     Po5(SU)     Eth      LACP      Eth1/4(P)

6     Po6(SU)     Eth      NONE      Eth1/36(P)

7     Po7(SU)     Eth      NONE      Eth101/1/10(P)

fab1-leaf3# show port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

I - Individual  H - Hot-standby (LACP only)

s - Suspended   r - Module-removed

S - Switched    R - Routed

U - Up (port-channel)

M - Not in use. Min-links not met

F - Configuration failed

-------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

Channel

-------------------------------------------------------------------------------

1     Po1(SU)     Eth      LACP      Eth1/4(P)

2     Po2(SU)     Eth      LACP      Eth1/17(P)

3     Po3(SU)     Eth      LACP      Eth1/16(P)

4     Po4(SD)     Eth      NONE      Eth1/48(D)

6     Po6(SU)     Eth      NONE      Eth1/36(P)

7     Po7(SU)     Eth      NONE      Eth102/1/10(P)

fab1-leaf1# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 13

Peer status                       : peer adjacency formed ok

vPC keep-alive status             : Disabled

Configuration consistency status  : success

Per-vlan consistency status       : success

Type-2 consistency status         : success

vPC role                          : primary

Number of vPCs configured         : 5

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Enabled (timeout = 240 seconds)

Operational Layer3 Peer           : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans

--   ----   ------ --------------------------------------------------

1           up     -

vPC status

----------------------------------------------------------------------

id   Port   Status Consistency Reason                     Active vlans

--   ----   ------ ----------- ------                     ------------

2    Po7    up     success     success                    -

fab1-leaf3# show vpc brief

Legend:

(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 13

Peer status                       : peer adjacency formed ok

vPC keep-alive status             : Disabled

Configuration consistency status  : success

Per-vlan consistency status       : success

Type-2 consistency status         : success

vPC role                          : secondary

Number of vPCs configured         : 5

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Enabled

Auto-recovery status              : Enabled (timeout = 240 seconds)

Operational Layer3 Peer           : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans

--   ----   ------ --------------------------------------------------

1           up     -

vPC status

----------------------------------------------------------------------

id   Port   Status Consistency Reason                     Active vlans

--   ----   ------ ----------- ------                     ------------

2    Po7    up     success     success                    -