Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.1(3)N2(1)
Configuring Enhanced Virtual Port Channels
Download this chapter
Configuring Enhanced Virtual Port ChannelsConfiguring Enhanced Virtual Port Channels
Download the complete book
Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.1(3)N2(1)Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.1(3)N2(1) (PDF - 4 MB)
 Feedback

Configuring Enhanced Virtual Port Channels

This chapter contains the following sections:

Information About Enhanced vPCs

Enhanced Virtual Port Channels Overview

The virtual port channel (vPC) feature allows the dual homed connection of a host to two fabric extenders (FEXs) or a dual homed connection of a FEX to two switches. The enhanced vPC feature, or two-layer vPC, allows both dual homing topologies to be combined simultaneously, as shown in the following figure:

With enhanced vPC, all available paths from the hosts to the FEXs and from the FEXs to the switches are active and carry Ethernet traffic, maximizing the available bandwidth and providing redundancy at both levels.

For information about vPCs, see Configuring Virtual Port Channels.

Supported Platforms and Topologies

Supported Platforms

Enhanced vPC is supported on the Cisco Nexus 5500 Platform running NX-OS Release 5.1(3)N1(1) or a later release.

Any Cisco Nexus 2000 Series Fabric Extender can be used with Enhanced vPC.

Enhanced vPC is compatible with Layer 3 features on the switch.

Supported and Unsupported Topologies

Enhanced vPC supports the following topologies:

  • A single homed server connected to a single FEX
  • A dual homed server connected by a port channel to a single FEX
  • A dual homed server connected by a port channel to a pair of FEXs This topology allows connection to any two FEXs that are connected to the same pair of switches in a vPC domain. Static port channel and LACP-based port channel are supported.
  • A dual homed server connected by FCoE and port channel to a pair of FEXs
  • A dual homed server connected by active/standby NIC teaming to a pair of FEXs

Enhanced vPC does not support the following topologies:

  • A dual homed server connected to a pair of FEXs that connect to a single switch Although this topology becomes a functioning system when one switch has failed, it is not recommended in normal operation.
  • A multi-homed server connected by a port channel to more than two FEXs This topology results in increased complexity with little benefit.

Enhanced vPC Scalability

The scalability of enhanced vPC is similar to that of the dual homed FEX topology.

Each Cisco Nexus 5500 Platform switch supports up to 24 FEXs with no Layer 3 configuration, or 16 FEXs with Layer 3 configuration. In a dual homed FEX topology, such as that in enhanced vPC, each FEX is managed by two switches, so the pair together can support 24 or 16 FEXs.

Enhanced vPC Failure Response

The enhanced vPC topology provides a high level of resilience to the failure of system components and links as described in the following scenarios:

  • Failure of One or More Port Channel Member Links When one member link of a port channel fails, the traffic flow is moved to the remaining port channel member links. If all member links of a port channel fail, the traffic flow is redirected to the remaining port channel of the vPC.
  • Failure of One FEX When one FEX fails, the traffic flow from all dual homed hosts is moved to the remaining FEX.
  • Failure of One Switch When one switch fails, the traffic flow from all dual homed FEXs is moved to the remaining switch. Traffic from the hosts is unaffected.
  • Failure of Both Uplinks from Single FEX When both uplinks from one FEX fails, the FEX shuts down its host ports, and the traffic flow from all dual homed hosts is moved to the other FEX.
  • Failure of the vPC Peer Link When the vPC secondary switch detects the failure of the peer link, it checks the status of the primary switch by the peer-keepalive link. If the primary switch is unresponsive, the secondary switch maintains all traffic flows as before. If the primary switch is active, the secondary switch shuts down its interfaces to the FEXs, and the traffic flow from all dual homed FEXs is moved to the primary switch. Ethernet traffic from the hosts is unaffected in either case. If the secondary switch carries FCoE traffic and shuts down its interfaces to the FEXs, it also shuts down all virtual Fibre Channel (vFC) interfaces that are bound to the FEX host ports. In this case, the hosts must use multipathing to move SAN traffic to the remaining vFC interface.
  • Failure of the vPC Peer-Keepalive Link A failure of the vPC peer-keepalive link by itself does not affect the traffic flow.

Licensing Requirements for Enhanced vPC

The following table shows the licensing requirements for this feature:

Product License Requirement

Cisco NX-OS

This feature does not require a license. Any feature not included in a license package is bundled with the Cisco NX-OS system images and is provided at no extra charge to you. For a complete explanation of the Cisco NX-OS licensing scheme, see the Cisco NX-OS Licensing Guide.

Configuring Enhanced vPCs

Overview of Configuration Steps for Enhanced vPC

An enhanced vPC configuration consists of a combination of two standard vPC configurations: the dual homed connection of a host to two fabric extenders and the dual homed connection of a fabric extender to two switches. The required configuration tasks are listed here, but the detailed procedures for those two standard configurations are presented in the "Configuring Virtual Port Channels" chapter of this document.

To configure enhanced vPC, perform the following steps. Unless noted otherwise, the procedures for each step are given in Configuring Virtual Port Channels.


Note

In procedures where the configuration must be repeated on both switches, the configuration synchronization (config-sync) feature allows you to configure one switch and have the configuration automatically synchronized to the peer switch. See the Cisco Nexus 5000 Series NX-OS Operations Guide for information about configuration synchronization.

Procedure
    Step 1   Enable the vPC and LACP features on each switch.
    Step 2   Create required VLANs on each switch.
    Step 3   Assign a vPC domain ID and onfigure the vPC peer-keepalive link on each switch.
    Step 4   Configure the vPC peer link on each switch.
    Step 5   Configure port channels from the first FEX to each switch.
    Step 6   Configure port channels from the second FEX to each switch.
    Step 7   If the enhanced vPC must accommodate FCoE traffic, associate the first FEX to one switch, then associate the second FEX to the other switch.

    See “Configuring FCoE over Enhanced vPC” in the Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide.

    Step 8   Configure a host port channel on each FEX.

    Verifying Enhanced vPCs

    Verifying the Enhanced vPC Configuration

    Before bringing up a vPC, the two peer switches in the same vPC domain exchange configuration information to verify that both switches have compatible configurations for a vPC topology. Depending on the severity of the impact of possible mismatched configurations, some configuration parameters are considered as Type 1 consistency check parameters while others are considered as Type 2.

    When a mismatch in Type 1 parameters is found, both peer switches suspend VLANs on the vPC ports. When a mismatch in Type 2 parameters is found, a warning syslog message is generated, but the vPC remains up and running.


    Note

    Enhanced vPC does not support the graceful consistency check.

    For enhanced vPC, the consistency verification for global configuration parameters is the same as for a dual homed FEX topology, and is described in the documentation for dual homed FEX. In addition to the global consistency verification, enhanced vPC requires interface level verification using tasks described in this section.

    Use the following commands to verify the enhanced vPC configuration and consistency:

    Command Purpose

    switch# show feature

    Displays whether vPC is enabled.

    switch# show running-config vpc

    Displays running configuration information for vPCs.

    switch# show vpc brief

    Displays brief information on the vPCs.

    switch(config)# show vpc consistency-parameters global

    Displays the status of global vPC parameters that must be consistent across all vPC interfaces.

    switch(config)# show vpc consistency-parameters interface port-channel channel-number

    Displays the status of specific port channels that must be consistent across vPC devices.

    For detailed information about the fields in the output of these commands, see the Cisco Nexus 5000 Series Command Reference.

    Verifying the Consistency of Port Channel Numbers

    For enhanced vPC, both switches must use the same port channel number for the dual homed connection to a FEX. If different port channel numbers are used, the port channel and its member ports are suspended on both switches.

    This procedure verifies the consistency of the port channel numbering:

    Procedure
       Command or ActionPurpose
      Step 1show running-config interface type/slot[, type/slot[, ...]]


      Example: 
      switch-1# show running-config interface Ethernet110/1/1, Ethernet111/1/1
       

      Displays the configuration of the specified list of port channel member ports.

      Execute this command on both peer switches and compare the reported channel-group numbers to verify that they match between switches.

       
      Step 2show interface type/slot


      Example:
      switch-1# show interface Ethernet110/1/1
       

      Displays the status and configuration of the specified port channel member port.

      Execute this command on both peer switches and verify the status of the ports.

       

      This example shows how to verify the consistency of the port channel numbering between the two switches. In this example, the port channel numbering is inconsistent and the member ports are suspended:

      switch-1# show running-config interface Ethernet110/1/1, Ethernet111/1/1

!Command: show running-config interface Ethernet110/1/1, Ethernet111/1/1
!Time: Sun Aug 28 03:38:23 2011

version 5.1(3)N1(1)

interface Ethernet110/1/1
channel-group 102

interface Ethernet111/1/1
channel-group 102

switch-2# show running-config interface Ethernet110/1/1, Ethernet111/1/1

!Command: show running-config interface Ethernet110/1/1, Ethernet111/1/1
!Time: Sun Aug 28 03:38:23 2011

version 5.1(3)N1(1)

interface Ethernet110/1/1
channel-group 101

interface Ethernet111/1/1
channel-group 101

switch-1# show interface Ethernet110/1/1
Ethernet110/1/1 is down (suspended by vpc)
  Hardware: 100/1000 Ethernet, address: 7081.0500.2402 (bia 7081.0500.2402)
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
  [...]

switch-2# show interface Ethernet110/1/1
Ethernet110/1/1 is down (suspended by vpc)
  Hardware: 100/1000 Ethernet, address: 7081.0500.2402 (bia 7081.0500.2402)
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
  [...]

      Verifying Common Port Channel Members

      The port channel from a FEX to the switch pair will be up and operational when there is at least one common port channel member between the two switches. Any FEX interfaces that are assigned to the port channel only on one switch will be suspended.

      Procedure
         Command or ActionPurpose
        Step 1 show port-channel summary


        Example: 
        switch-1# show port-channel summary
         

        Displays a summary of the port channel interfaces.

         
        Step 2 show interface type/slot


        Example: 
        switch-1# show interface ethernet 111/1/3
                  		(Optional)

        Displays the status and configuration of the specified interface.

         

        This example shows how to verify the common member ports of the vPC. In this example, the vPC is configured with one port channel member that is not common to both switches. That member port is shown as shut down, and further investigation shows that the member is suspended by the vPC. In this part of the session, the port channel is configured on each switch, with an extra port on the first switch:

        switch-1(config)# interface ethernet 110/1/3, ethernet 111/1/3
switch-1(config-if)# channel-group 101
switch-1(config-if)# interface port-channel 101
switch-1(config-if)# switchport access vlan 20

switch-2(config)# interface ethernet 110/1/3
switch-2(config-if)# channel-group 101
switch-2(config-if)# interface port-channel 101
switch-2(config-if)# switchport access vlan 20

        In this part of the session, the extra port is shown to be in the down state, and a display of the port details shows that the port is suspended by the vPC:

        switch-1# show port-channel summary 
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        S - Switched    R - Routed
        U - Up (port-channel)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
1     Po1(SU)     Eth      LACP      Eth1/1(P)    Eth1/2(P)    
[...]
101   Po101(SU)   Eth      NONE      Eth110/1/3(P)  Eth111/1/3(D)  

switch-1# show interface ethernet 111/1/3
Ethernet111/1/3 is down (suspended by vpc)
  Hardware: 100/1000 Ethernet, address: 7081.0500.2582 (bia 7081.0500.2582)
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255

        Verifying Interface Level Consistency for Enhanced vPC

        For enhanced vPC, you must ensure consistency of the port mode and the shared VLAN in the port channel interface configuration.

        This procedure verifies that the configurations are consistent on a vPC interface.

        Procedure
           Command or ActionPurpose
          Step 1 show vpc consistency-parameters port-channel channel-number


          Example: 
          switch# show vpc consistency-parameters interface port-channel 101
switch(config)#
           

          For the specified port channel, displays the status information that must be consistent across vPC devices.

           

          This example shows how to display a comparison of the interface configuration across two peers for a vPC. In this case, VLAN 10 is allowed on both peers, but the port mode is mismatched, causing the VLAN to be suspended.

          NX-5000-1# show vpc consistency-parameters interface port-channel 101

    Legend:
        Type 1 : vPC will be suspended in case of mismatch

Name                        Type  Local Value            Peer Value             
-------------               ----  ---------------------- -----------------------
mode                        1     on                     on                    
Speed                       1     1000 Mb/s              1000 Mb/s             
Duplex                      1     full                   full                  
Port Mode                   1     access                 trunk                 
MTU                         1     1500                   1500                  
Admin port mode             1                                                  
Shut Lan                    1     No                     No                    
vPC+ Switch-id              1     3000                   3000                  
Allowed VLANs               -     10                     1-57,61-3967,4048-4093
Local suspended VLANs       -     10                     -

          Enhanced vPC Example Configuration

          The following example shows the complete configuration procedure using the topology of the enhanced vPC figure in this chapter. In the topology figure, the number pairs beside each port channel link represent the interface port numbers. For example, the switch link labeled with the numbers "3,4" represents interfaces eth1/3 and eth1/4 on the switch.


          Note

          In procedures where the configuration must be repeated on both switches, the configuration synchronization (config-sync) feature allows you to configure one switch and have the configuration automatically synchronized to the peer switch. See the Cisco Nexus 5000 Series NX-OS Operations Guide for information about configuration synchronization.

          Before You Begin

          Ensure that the Cisco Nexus 2000 Series Fabric Extenders FEX101 and FEX102 are attached and online.

          Procedure
            Step 1   Enable the vPC and LACP features on each switch

            Example: 
            NX-5000-1(config)# feature vpc
NX-5000-1(config)# feature lacp

NX-5000-2(config)# feature vpc
NX-5000-2(config)# feature lacp
            Step 2   Create required VLANs on each switch

            Example: 
            NX-5000-1(config)# vlan 10-20

NX-5000-2(config)# vlan 10-20
            Step 3   Assign a vPC domain ID and onfigure the vPC peer-keepalive link on each switch

            Example: 
            NX-5000-1(config)# vpc domain 123
NX-5000-1(config-vpc)# peer-keepalive destination 172.25.182.100

NX-5000-2(config)# vpc domain 123
NX-5000-2(config-vpc)# peer-keepalive destination 172.25.182.99
            Note   

            In configuring each switch, use the IP address of the peer switch as the peer-keepalive destination.

            Step 4   Configure the vPC peer link on each switch

            Example: 
            NX-5000-1(config)# interface eth1/1-2
NX-5000-1(config-if)# channel-group 1 mode active
NX-5000-1(config-if)# interface Po1
NX-5000-1(config-if)# switchport mode trunk
NX-5000-1(config-if)# switchport trunk allowed vlan 1, 10-20
NX-5000-1(config-if)# vpc peer-link

NX-5000-2(config)# interface eth1/1-2
NX-5000-2(config-if)# channel-group 1 mode active
NX-5000-2(config-if)# interface Po1
NX-5000-2(config-if)# switchport mode trunk
NX-5000-2(config-if)# switchport trunk allowed vlan 1, 10-20
NX-5000-2(config-if)# vpc peer-link
            Step 5   Configure port channels from the first FEX to each switch

            Example: 
            NX-5000-1(config)# fex 101
NX-5000-1(config-fex)# interface eth1/3-4
NX-5000-1(config-if)# channel-group 101
NX-5000-1(config-if)# interface po101
NX-5000-1(config-if)# switchport mode fex-fabric
NX-5000-1(config-if)# vpc 101
NX-5000-1(config-if)# fex associate 101

NX-5000-2(config)# fex 101
NX-5000-2(config-fex)# interface eth1/3-4
NX-5000-2(config-if)# channel-group 101
NX-5000-2(config-if)# interface po101
NX-5000-2(config-if)# switchport mode fex-fabric
NX-5000-2(config-if)# vpc 101
NX-5000-2(config-if)# fex associate 101
            Step 6   Configure port channels from the second FEX to each switch

            Example: 
            NX-5000-1(config)# fex 102
NX-5000-1(config-fex)# interface eth1/5-6
NX-5000-1(config-if)# channel-group 102
NX-5000-1(config-if)# interface po102
NX-5000-1(config-if)# switchport mode fex-fabric
NX-5000-1(config-if)# vpc 102
NX-5000-1(config-if)# fex associate 102

NX-5000-2(config)# fex 102
NX-5000-2(config-fex)# interface eth1/5-6
NX-5000-2(config-if)# channel-group 102
NX-5000-2(config-if)# interface po102
NX-5000-2(config-if)# switchport mode fex-fabric
NX-5000-2(config-if)# vpc 102
NX-5000-2(config-if)# fex associate 102
            Step 7   Configure a host port channel on each FEX

            Example: 
            NX-5000-1(config)# interface eth101/1/1, eth101/1/2
NX-5000-1(config-if)# channel-group 2 mode active
NX-5000-1(config-if)# interface eth102/1/1, eth102/1/2
NX-5000-1(config-if)# channel-group 2 mode active
NX-5000-1(config-if)# int po2
NX-5000-1(config-if)# switchport access vlan 10

NX-5000-2(config)# interface eth101/1/1, eth101/1/2
NX-5000-2(config-if)# channel-group 2 mode active
NX-5000-2(config-if)# interface eth102/1/1, eth102/1/2
NX-5000-2(config-if)# channel-group 2 mode active
NX-5000-2(config-if)# int po2
NX-5000-2(config-if)# switchport access vlan 10