Tracing of MAC Addresses in UCS with Nexus 1000V

Introduction

This document describes how to trace MAC addresses of a virtual machine (VM) and VMkernel (VMK) interface at these network levels:

• Cisco Nexus 5000 Series Switches
• Cisco Unified Computing System (UCS) 6248 Fabric Interconnect (FI)
• VMware ESXi host
• Cisco Nexus 1000V Switch

It is important to understand which uplink a VM or VMK interface uses for communication for both troubleshooting and design aspects.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• vPC feature in Cisco NX-OS
• Cisco Unified Computing System
• VMware ESXi
• Cisco Nexus 1000V Switch

Components Used

The information in this document is based on these software and hardware versions:

• Cisco Nexus 5020 Switch Version 5.0(3)N2(2a)
• Cisco Unified Computing System Version 2.1(1d)
• Cisco Unified Computing System B200 M3 Blade Server with Cisco Virtual Interface Card (VIC) 1240 (Palo) CNAvSphere 5.1 (ESXi and vCenter)
• Cisco Nexus 1000V Switch Version 4.2(1)SV2(1.1a)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Configure

Network Topology

In this example setup, the VM and VMK interfaces are on same host (IP address 172.16.18.236) and the same VLAN 18 (subnet 172.16.18.0/24).

In the Nexus 1000V, the host is represented as Virtual Ethernet Module (VEM) # 5.

In the UCS, the host is installed on blade 1 in chassis 1.

Tracing of MAC Addresses at Different Network Segments

This procedure describes how to trace MAC addresses at various network levels.

1. In the vCenter, find the MAC address of the VM you wish to trace. In this example, the MAC address of the VM (ciscolive-vm) is 0050:568f:63cc:
   
   
   
   
2. Enter the esxcfg-vmknic -l command on the ESXi shell in order to find the MAC address of VMK interface from the host. In this example, the VMK (vmk0) is the management interface and has a MAC address of 0050:56:67:8e:b9:
   
   
   
   
3. Confirm that the MAC addresses of the VM (ciscolive-vm) and the VMK interface (vmk0) are being learned on the ESXi host (VEM) and the Nexus 1000V.
   
   
   1. At the VEM level, enter the vemcmd show l2 18 command in order to confirm that both MAC addresses are learned:
      
      
      
      
   2. At the Nexus 1000V level, enter a show mac address-table command in order to confirm that both MAC addresses are learned on VLAN 18 on VEM # 5:
      
      
      
      
   3. Enter the show port-channel summary command for VEM # 5 in order to see the port-channel and member ports:
      
      
      
      
4. Gather additional details from Nexus 1000V.
   
   
   1. Enter the show interface vethernet 56 command in order to see that Veth56 corresponds to the VM (ciscolive-vm):
      
      
      
      
   2. Enter the show interface vethernet 19 command in order to see that Veth19 corresponds to the VMK interface (vmk0) of the host:
      
      
      
      
5. Check the pinning of the traffic from the VM (ciscolive-vm) and the VMK interface (vmk0) to the upstream interfaces from the host.
   
   
   
   
   This output shows the subscriber group ID (SGID) mapping for the VM (ciscolive-vm) and the VMK interface (vmk0) to their corresponding VM network interface controllers (VMNICs). The mapping reveals which VMNICs are used for communication:
   
   
• SGID 0 of the VM (ciscolive-vm) matches SGID 0 of vmnic0.
• SGID 1 of the VMK interface (vmk0) matches SGID 1 of vmnic1.
6. Get the MAC addresses of the VMNICs from either the vCenter or the ESXi command-line interface (CLI).
   
   
   1. In the vCenter, navigate to the Configuration tag:
      
      
      
      
   2. On the ESXi CLI, enter the esxcfg-nics -1 command:
      
      
      
      
7. In the UCS Manager (UCSM), find the virtual network interface controllers (vNICs) of the UCS that correspond to the VMNICs:
   
   
   
   The primary FI for vNIC-0 is FI-A, while the primary FI for vNIC-1 is FI-B. You can now infer that traffic from the VM (ciscolive-vm) traverses through FI-A and that traffic from the VMK interface (vmk0) traverses through FI-B.
   
   
8. Confirm that the MAC address of the VM (ciscolive-vm) is learned on FI-A:
   
   
   
   
9. Confirm that the MAC address of the VMK interface (vmk0) is learned on FI-B:
   
   
   
   
10. Check the pinning of these Veths to their uplinks with the show circuit detail command:
    
    
    
    

    Note: Other commands that output similar information are show pinning server-interfaces, show pinning border-interfaces, and show pinning interface vethernet x.

    You can also check the pinning in the UCSM:
    
    
    
    
11. Gather additional details about the port-channels. In this configuration, there are three port-channels in use for each FI. For example, FI-B has three associated port-channels:
    
    
• Port-channel 89 is the Link Aggregation Congrol Protocol (LACP) port-channel between FI-B and the upstream Nexus 5020.
• Port-channel 1153 is automatically created and is between FI-B and input/output module (IOM)-B.
• Port-channel 1287 is automatically created and is between IOM-B and Cisco VIC 1240 (blade).
1. Enter the show port-channel summary command in order to see the port-channel configuration of FI-B:
   
   
   
   
2. Enter the show cdp neighbors command in order to discover and view additional information on FI-B:
   
   
   
   
3. Enter the show port-channel summary command in order to see the port-channel configuration of FI-A:
   
   
   
   
4. Enter the show cdp neighbors command in order to discover and view additional information on FI-A:
   
   
   
   
12. Determine the specific pinning of the member interface from the port-channel.
    
    
    1. Enter a show port-channel command in order to see that the FI-B - VMK interface (vmk0) MAC address is pinned to Ethernet1/6 of port-channel 89:
       
       
       
       
    2. Enter a show port-channel command in order to see that the FI-A - VM (ciscolive-vm) MAC address is pinned to Ethernet1/5 of port-channel 88:
       
       
       
       
13. Check that the MAC addresses are learned on the upstream Nexus 5020.
    
    
    1. Enter a show mac address-table command in order to see that the VMK interface (vmk0) MAC address is learned on the Nexus 5020-1:
       
       
       
       
    2. Enter a show mac address-table command in order to see that the VM (ciscolive-vm) MAC address is learned on the Nexus 5020-2:
       
       

When you troubleshoot network issues, this example helps you quickly isolate and identify how and where a MAC address is learned and what the expected path is for network traffic.

Verify

Verification procedures are included in the configuration example.

Troubleshoot

This configuration example is intended to help with network troubleshooting.