This document describes the configuration steps you use in order to integrate a Cisco Unified Computing System (UCS) B Series into an Application Centric Infrastructure (ACI) fabric that leverages Virtual Machine Manager (VMM) domain integration.
There are no specific requirements for this document.
The information in this document is based on these hardware and software versions:
An ACI fabric that consists of two spine switches and two leaf switches
A UCS B Series chassis with two fabric interconnects
UCS B Series blades with VMware ESXi
An Application Policy Infrastructure Controller (APIC)
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Create the VMM Domain
Most of this configuration is similar to the deployment of a VMM domain on any server hardware. There are certain limitations for which the workaround is to configure the APIC a certain way. These workaround configurations are called out specifically in this procedure.
Create a dynamic VLAN pool. From the APIC user interface, choose Fabric > Access Policies > Pools > VLAN > Create VLAN Pool.
When the Create VLAN Pool window opens, enter this information:
Enter the name of the pool in the Name field.
Click the Dynamic Allocation radio button.
Click the Encap Blocks (+) plus symbol and enter the Encap Block Range in the Range fields of the Create Ranges dialog box. Click OK when you are finished.
From the APIC user interface, choose VM Networking > VM Provider VMware > Create vCenter Domain.
When the Create vCenter Domain window appears, enter this information:
Enter the domain name in the Name field.
Click the VMWare vSphere Distributed Switch radio button.
Choose Demo-AEP from the Associated Attachable Entity Profile drop-down list.
Choose Demo-Pool (dynamic) from the VLAN Pool drop-down list.
Click the vCenter Credentials(+) plus symbol and enter your vCenter Credential information in the Create vCenter Credential dialog box.
Click OK when you are finished.
Click the (+) plus symbol from the vCenter/vShield window. Enter this information when the Create vCenter/vShield Controller window appears:
Click the vCenter radio button.
Enter the host name or IP address in the Host Name (or IP Address) field.
Choose vCenter Default from the DVS Version drop-down list.
Enter the name of the datacenter in the Datacenter field.
Choose Demo-VMM-Creds from the Associated Credential drop-down list.
Click OK when you are finished.
Verify the DVS is Created in vCenter
You should see a few new tasks in the Recent Tasks window and the addition of a Distributed Virtual Switch (DVS) in the vCenter Server:
Create/Verify that CDP or LLDP is Enabled on the UCS vNICs
When you deploy UCS B in ACI, you can choose the discovery protocol you would like to use to discover the hosts. This section walks you through how to configure each type in the UCS Manager.
By default, Cisco Discovery Protocol (CDP) is disabled on the UCS virtual Network Interface Card (vNIC) because the default Network Control Policy has CDP disabled. In order to enable CDP, you can either modify the default Network Control Policy, or create a new one with CDP enabled. Then apply that policy to each vNIC in each Service Profile. In this example, the default Network Control Policy is modified since all of the Service Profiles use that by default:
If you use a different policy, ensure you add that policy to the vNICs in each Service Profile:
In Version 2.2(4b) and later, the UCS supports Link Layer Discovery Protocol (LLDP) from the Fabric Interconnects down to the blades. This means that you can also use LLDP in order to discover the hosts in vCenter and the fabric if you are run this version or later. The configuration is the exact same as above, but you would enable LLDP in both directions:
Configure the vSwitch Policies on APIC for UCS B
By default on the DVS, the Discovery Protocol used is LLDP. This is fine for any servers that support LLDP, but the UCS B series blades only support LLDP on UCSM version 2.2(4b) and later. Because of this, ESXi cannot report LLDP information to the APIC, unless you are on the correct code.
As an alternative to LLDP, use CDP in order to discover the hosts. In order to get the DVS to use CDP, configure a vSwitch policy on the Attachable Entity Profile (AEP) that has CDP enabled and LLDP disabled.
Along with this, the only supported load balancing mechanism when UCS B series is used is Route Based on Originating Virtual Port. If you configure a mac-pinning policy, it programs the port groups to use this mechanism. This is very important in order to prevent packet loss.
From the APIC user interface, choose Fabric > Access Policies > Global Policies > Attachable Access Entity Profiles > AEP > Config vSwitch Policies.
Enter this information when the Config vSwitch Policies window appears:
Choose enableCDP from the CDP Policy drop-down list.
Choose mac-pinning from the LACP Policy drop-down list.
Choose disLLDP from the LLDP Policy drop-down list and click Submit.
Note: If you are on UCSM 2.2(4b) or later, and you want to use LLDP, you can turn on LLDP in this vSwitch policy since the UCS supports it. This example is only for UCSM versions that do not support LLDP, or if CDP is desired.
After you click Submit, you can see that the DVS is reconfigured in the vCenter:
You can also see that the vmnics see CDP information from the Fabric Interconnect:
Verify that Route Based on Originating Virtual Port is programmed on the port groups. Right-click a port group in the Networking tab, and edit the setting in order to verify this:
Use this section to confirm that your configuration works properly.
After these changes are made, the APIC should be notified by the vCenter about the CDP information. In order to verify this, check the inventory of the VMM domain.
From the APIC user interface, choose VM Networking > Inventory > VMware > Domain > vCenter > Hypervisors > Hypervisor > General in order to view the Properties window.
At this point, you can change your VM Network settings to add the adapter to the proper port group and test connectivity. Pings should be successful. If pings are not successful, verify all settings in vCenter and in the APIC are correct for CDP neighbor discovery.
There is currently no specific troubleshooting information available for this configuration.