Troubleshoot Active-Active NIC Teaming on ACI VMM Integration

Introduction

This document describes how to identify an issue with the Active-Active configuration on VMM integration with ACI use LACP Load Based Teaming.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Link Agregation Control Protocol (LACP)
• Virtual Machine Monitor (VMM)
• Network Interface Control (NIC)
• Application Centric Infrastructure (ACI)

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Configure

The network has configured an Active-Active LACP Load Based Teaming NIC on a VM configuration allowed on two servers and is connected to ACI on two different Leaves switches on a VMM integration.

Network Diagram

The next image is a high-level reference in order to illustrate the design.

vPC 5 members are Interface Eth 1/1 on Leaf 1 and Eth 1/2 on Leaf 2.

vPC 6 members are Interface Eth 1/3 on Leaf 1 and Eth 1/4 on Leaf 2.

VM IP: 10.10.10.1

VM MAC: AA.AA.AA.AA.AA.AA

Connections:

SERVER 1 NIC 1 <-> LEAF 1 Eth 1/1

SERVER 1 NIC 2 <-> LEAF 2 Eth 1/2

SERVER 2 NIC 1 <-> LEAF 1 Eth 1/3

SERVER 2 NIC 2 <-> LEAF 2 Eth 1/4

Verify

There is currently no verification procedure available for this configuration.

The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.

Troubleshoot

This section provides the information you can use to troubleshoot your configuration.

Navigate to the Endpoint Tracker (EP) tracker on the APIC GUI to track the IP for the server and review the attach/detach logs to identify the flap.

Step 1. Log in Cisco APIC GUI

Step 2. Navigate to Operations -> EP Tracker

Step 3. In the End Point Search area, input the IP address

Step 4. Click Search

Once a flap has been identified, the next step is to run the commands on each leaf switch.

LEAF1# show endpoint ip 10.10.10.1
Legend:
S - static           s - arp              L - local            O - peer-attached 
 V - vpc-attached     a - local-aged       p - peer-aged        M - span          
 B - bounce           H - vtep             R - peer-attached-rl D - bounce-to-proxy
E - shared-service   m - svc-mgr       
+-----------------------------------+---------------+-----------------+--------------+-------------+
      VLAN/                           Encap           MAC Address       MAC Info/       Interface
      Domain                          VLAN            IP Address        IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
2                                          vlan-100    aaaa.aaaa.aaaa LV                        po6
common:common-VRF                          vlan-100      10.10.10.1   LV                        po6

LEAF2# show endpoint ip 10.10.10.1
Legend:
S - static           s - arp              L - local            O - peer-attached 
 V - vpc-attached     a - local-aged       p - peer-aged        M - span          
 B - bounce           H - vtep             R - peer-attached-rl D - bounce-to-proxy
E - shared-service   m - svc-mgr       
+-----------------------------------+---------------+-----------------+--------------+-------------+
      VLAN/                           Encap           MAC Address       MAC Info/       Interface
      Domain                          VLAN            IP Address        IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
1                                          vlan-100    aaaa.aaaa.aaaa LV                        po6
common:common-VRF                          vlan-100      10.10.10.1   LV                        po6

LEAF1# show port-channel summary
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        b - BFD Session Wait
        S - Switched    R - Routed
        U - Up (port-channel)
        M - Not in use. Min-links not met
        F - Configuration failed
-------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
-------------------------------------------------------------------------------
5     Po5(SU)     Eth      LACP      Eth1/1(P) 
6     Po6(SU)     Eth      LACP      Eth1/3(P) 

LEAF2# show port-channel summary
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        b - BFD Session Wait
        S - Switched    R - Routed
        U - Up (port-channel)
        M - Not in use. Min-links not met
        F - Configuration failed
-------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
-------------------------------------------------------------------------------
5     Po5(SU)     Eth      LACP      Eth1/2(P)  
6     Po6(SU)     Eth      LACP      Eth1/4(P)     

LEAF1# show system internal epm endpoint ip 10.10.10.1
MAC : aaaa.aaaa.aaaa ::: Num IPs : 1
IP# 0 : 10.10.10.1 ::: IP# 0 flags : 
 
Output omitted
...
 
Interface : port-channel6                          <<<<<<<<<<<<<<<< learned on this interface
 
Output omitted
...
 
EP Flags : local|vPC|IP|MAC|sclass|timer|mac-ckt|
:::: 

LEAF1# show system internal epm endpoint mac aaaa.aaaa.aaaa
MAC : aaaa.aaaa.aaaa ::: Num IPs : 1
IP# 0 : 10.10.10.1 ::: IP# 0 flags :  ::: l3-sw-hit: No
 
Output omitted
...
 
Interface : port-channel6                         <<<<<<<<<<<<<<<< learned on this interface
 
Output omitted
...
 
EP Flags : local|vPC|IP|MAC|sclass|timer|mac-ckt|
:::: 

LEAF2# show system internal epm endpoint ip 10.10.10.1
MAC : aaaa.aaaa.aaaa ::: Num IPs : 1
IP# 0 : 10.10.10.1 ::: IP# 0 flags :  ::: l3-sw-hit: No

Output omitted
...

Interface : port-channel5                         <<<<<<<<<<<<<<<< learned on this interface

Output omitted
...

EP Flags : local|vPC|IP|MAC|sclass|timer|mac-ckt|
::::

LEAF2# show system internal epm endpoint mac aaaa.aaaa.aaaa
MAC : aaaa.aaaa.aaaa ::: Num IPs : 1
IP# 0 : 10.10.10.1 ::: IP# 0 flags :  ::: l3-sw-hit: No

Output omitted
...

Interface : port-channel6                         <<<<<<<<<<<<<<<< learned on this interface

Output omitted
...

EP Flags : local|vPC|IP|MAC|sclass|timer|mac-ckt|
:::: 

From the previous outputs, you can identify that ACI receives the same information on both port channels at the same time due to the active-active configuration on VMware.

All VMs use each pair of uplinks as logically one uplink, and the topology is configured with two different pairs of port channels, and each pair uses the same information to arrive on ACI.

Note: Cisco does not recommend this kind of deployment because this type of configuration works similar to MAC Pinning and re-pin every 30 seconds based on the link utilization, derived on MAC/IP flaps between nodes or ports.

If you require a similar topology, the recommendation is use Link Aggregation Group (LAG) such as LACP or static port channel.

The Cisco CLI Analyzer (registered customers only) supports certain show commands. Use the Cisco CLI Analyzer in order to view an analysis of show command output.

Refer to Important Information on Debug Commands before you use debug commands.

Related Information

• Technical Support & Documentation - Cisco Systems