Chapter 1. Cisco ACI Virtual Edge Overview

This chapter contains the following sections:

- About Cisco ACI Virtual Edge

- About the Cisco ACI Virtual Edge and the VMware vCenter

- Cisco ACI Virtual Edge in a Multipod Environment

- Required Software

About Cisco ACI Virtual Edge

Beginning with the Cisco APIC Release 3.1(1), the Cisco Application Centric Infrastructure (ACI) supports the Cisco ACI Virtual Edge. Cisco ACI Virtual Edge is the next generation of the Application Virtual Switch (AVS) for Cisco ACI environments. Cisco ACI Virtual Edge is a hypervisor-independent distributed service VM that leverages the native distributed virtual switch that belongs to the hypervisor. Cisco ACI Virtual Edge runs in the user space, operates as a virtual leaf, and is managed by the Cisco Application Policy Infrastructure Controller (APIC).

If you use Cisco AVS, you can migrate to Cisco ACI Virtual Edge; if you use VMware VDS, you can run Cisco ACI Virtual Edge on top of it. Decoupling the Cisco ACI Virtual Edge from the kernel space makes the solution adaptable to different hypervisors. It also facilitates simple upgrades as Cisco ACI Virtual Edge is not tied to hypervisor upgrades. Cisco ACI Virtual Edge implements the OpFlex protocol for control plane communication. It supports two modes of traffic forwarding: local switching and no local switching.

Cisco ACI Virtual Edge Release 1.1(1a) supports only the VMware hypervisor. It leverages the vSphere Distributed Switch (VDS), which is configured in private VLAN (PVLAN) mode.

When network administrators create a Cisco ACI Virtual Edge VMM domain on Cisco APIC, they must associate the domain with a range of VLANs to be used for PVLAN pair association of port groups on the DVS. Server administrators do not need to associate PVLANs to port groups on vCenter because Cisco APIC automatically associates PVLAN pairs with the endpoint groups (EPGs).

Note

EPGs in Cisco APIC are equivalent to port groups in vCenter.

Local Switching Mode

In Local Switching mode, the Cisco ACI Virtual Edge locally forwards all intra-EPG traffic without involving the leaf. All inter-EPG traffic is forwarded through the leaf. In this mode, the Cisco ACI Virtual Edge can use either VLAN or VXLAN encapsulation—or both—for forwarding traffic to the leaf and back. You choose the encapsulation type during Cisco ACI Virtual Edge VMM domain creation.

You can configure a single VMM domain in Local Switching mode to use VLAN and VXLAN encapsulation.

If you choose VLAN encapsulation, a range of VLANs must be available for use by the Cisco ACI Virtual Edge. These VLANs have local scope in that they have significance only within the Layer 2 network between the Cisco ACI Virtual Edge and the leaf. If you choose VXLAN encapsulation, only the infra-VLAN must be available between the Cisco ACI Virtual Edge and the leaf. This results in a simplified configuration. It is the recommended encapsulation type if there are one or more switches between the Cisco ACI Virtual Edge and the physical leaf.

The Cisco ACI Virtual Edge in Local Switching Mode

No Local Switching Mode

In No Local Switching mode, the leaf forwards all traffic. In this mode, VXLAN is the only allowed encapsulation type.

The Cisco ACI Virtual Edge in No Local Switching Mode

Statistics Collection

Statistics collection is enabled on Cisco ACI Virtual Edge by default. You may see Cisco ACI Virtual Edge faults within the APIC GUI relating to VM resource use.

Troubleshoot those faults in the VMware vCenter because the Cisco ACI only generates these faults based on information it receives from VMware vCenter.

About the Cisco ACI Virtual Edge and the VMware vCenter

The Cisco ACI Virtual Edge is a distributed virtual switch that extends across many virtualized hosts. It manages a data center defined by the vCenter Server.

The Cisco ACI Virtual Edge is compatible with any upstream physical access layer switch that complies with the Ethernet standard, including Cisco Nexus switches. The Cisco ACI Virtual Edge is compatible with any server hardware listed in the VMware Hardware Compatibility List (HCL).

The Cisco ACI Virtual Edge is a distributed virtual switch solution that is fully integrated within the VMware virtual infrastructure, including VMware vCenter for the virtualization administrator. This solution allows the network administrator to configure virtual switch and port groups in order to establish a consistent data center network policy.

The following figure shows a topology that includes the Cisco ACI Virtual Edge with the Cisco APIC and VMware vCenter.

Sample Cisco ACI Virtual Edge Topology

Note

If there are multiple vCenters connected to a single Cisco ACI fabric, you should ensure that there are no overlapping MAC address allocation schema across the multiple vCenters while deploying the vCenters instead of the default OUI allocation. Overlaps can cause duplicate MAC address generation. For more information, see VMware documentation.

Cisco ACI Virtual Edge in a Multipod Environment

The Cisco ACI Virtual Edge can be part of a multipod environment. Multipod environments use a single Cisco APIC cluster for all the pods; all the pods act as a single fabric.

Multipod environments enable a more fault tolerant fabric comprising multiple pods with isolated control plane protocols. They also provide greater flexibility in full mesh cabling between leaf and spine switches.

Cisco ACI Virtual Edge does not require any additional configuration to operate in a multipod environment.

For detailed information about multipod environments, see the following documents on Cisco.com:

- Cisco Application Centric Infrastructure Fundamentals

- Cisco APIC Getting Started Guide

- Cisco APIC NX-OS Style Command-Line Interface Configuration Guide

The following features are not supported for Cisco ACI Virtual Edge with multipod in the Cisco APIC 3.1(1) release:

- L3 Multicast

- Storage vMotion with two separate NFS in two separate PODs

- ERSPAN destination in different PODs

- Distributed Firewall syslog server in different PODs

Required Software

The following table shows the versions of software required for Cisco ACI Virtual Edge to work with the Cisco APIC, VMware vCenter, and VMware ESXi hypervisor:

Component

Description

Cisco ACI Virtual Edge software

Cisco ACI Virtual Edge is supported beginning with Release 1.1(1).

Cisco APIC

Cisco ACI Virtual Edge is supported in Cisco APIC beginning with Release 3.1(1).

VMware vCenter

Cisco ACI Virtual Edge is compatible with release 6.0 and later versions of VMware vCenter Server.

VMware vSphere bare metal

Cisco ACI Virtual Edge is supported as a vLeaf for the Cisco APIC with release 6.0 and later releases of the VMware ESXi hypervisor.

Chapter 2. Cisco ACI Virtual Edge Installation

This chapter describes installation for Cisco ACI Virtual Edge, including prerequisites and installation methods.

- About Cisco ACI Virtual Edge Installation

- Cisco ACI Virtual Edge Installation Workflow

- Prerequisites for Installing Cisco ACI Virtual Edge

- Cisco ACI Virtual Edge Installation Using the vCenter

- Cisco ACI Virtual Edge Installation Using the VMware PowerCLI

- Cisco ACI Virtual Edge Installation Using Python

- Post-Installation Configuration

About Cisco ACI Virtual Edge Installation

Cisco ACI Virtual Edge installation consists of a series of tasks on the Cisco APIC, and VMware vCenter. You can then use one of three methods to deploy Cisco ACI Virtual Edge on ESXi hosts:

- Cisco ACI vCenter plug-in

- VMware PowerCLI (for Windows platforms)

- Python script

Note

Do not use the vSphere (thick) Client to install Cisco ACI Virtual Edge or modify its vApp properties. Use only the Cisco ACI vCenter plug-in, the VMware Power CLI, or a Python script to install Cisco ACI Virtual Edge. Use only the vSphere Web Client to modify Cisco ACI Virtual Edge vApp properties.

Note

When you deploy the Cisco ACI Virtual Edge VM on the ESXi hosts, OpFlex automatically comes online. Do not attach VMkernel ports to the Infra port group, as was done for OpFlex for Cisco AVS.

The following sections provide information about prerequisites and installation methods. For information about migrating from Cisco AVS to Cisco ACI Virtual Edge, see the chapter Migration from Cisco AVS to Cisco ACI Virtual Edge in this guide. For information about migrating from VMware VDS to Cisco ACI Virtual Edge, see the chapter Migration from VMware VDS to Cisco ACI Virtual Edge in this guide.

Note

Although you can install multiple Cisco ACI Virtual Edge VMs on the same host (one for each Cisco ACI Virtual Edge VMM domain), we recommend that you install only one Cisco ACI Virtual Edge VM per host.

Cisco ACI Virtual Edge Installation Workflow

This section provides a high-level description of the tasks required to install the Cisco ACI Virtual Edge.

1. Fulfill all the prerequisites, which include tasks in the Cisco Application Policy Infrastructure Controller (APIC), and vCenter. See the section Prerequisites for Installing Cisco ACI Virtual Edge.

2. Download the Cisco ACI Virtual Edge Open Virtualization Format (OVF) file from Cisco.com and then upload it to the vCenter content library. You can use the vCenter plug-in, the vCenter power CLI, or a Python script. See one of the following sections for instructions:

- Upload the Cisco ACI Virtual Edge VM OVF File to the vCenter

- Install Cisco ACI Virtual Edge Cisco ACI Virtual Edge Using the VMware PowerCLI

- Install Cisco ACI Virtual Edge Using Python

3. Deploy Cisco ACI Virtual Edge on the ESXi hosts. You can use one of four methods. See the following sections for instructions:

- Deploy Cisco ACI Virtual Edge on the ESXi Hosts Using the Cisco ACI vCenter Plug-in

- Install Cisco ACI Virtual Edge Cisco ACI Virtual Edge Using the VMware PowerCLI

- Install Cisco ACI Virtual Edge Using Python

Note

We recommend that you deploy Cisco ACI Virtual Edge on a local datastore.

Prerequisites for Installing Cisco ACI Virtual Edge

Perform the following tasks before you install Cisco ACI Virtual Edge:

- Make sure that Cisco APIC is set up correctly. See the Cisco APIC Getting Started Guide and Cisco APIC Basic Configuration Guide, on Cisco.com for instructions on how to configure Cisco APIC for the first time.

- Make sure that all switches are registered and that the Cisco ACI fabric is up-to-date. See Cisco Application Centric Infrastructure Fundamentals and the Cisco APIC Getting Started Guide on Cisco.com for instructions.

- Make sure that the ACI fabric is registered inside the vCenter plug-in. See "Connecting vCenter Plug-in to your ACI Fabric" in the chapter "Cisco ACI vCenter Plug-in" in the Cisco ACI Virtualization Guide.

- Create a new vCenter VMM domain and interface and switch profiles for Cisco ACI Virtual Edge.

We recommend that you use the unified configuration wizard to perform these tasks. See the procedure Create vCenter Domain, Interface, and Switch Profiles Using the GUI in this guide. However, you may need to configure separate, more detailed policies. If so, see the appendix Alternate Procedures for Creating vCenter Domain, Interface, and Switch Profiles in this guide.

- In order to use the Cisco ACI Virtual Edge management tools (the ACI vCenter plug-in, the VMware PowerCLI, and Python scripts), we recommend that you use vCenter 6.0 Update 3 or later.

- Add one or more ESXi hosts and their PNICs to the new Cisco ACI Virtual Edge distributed virtual switch (DVS) in using vSphere Web Client on vCenter.

- If the host belongs to a Distributed Resource Scheduler (DRS) cluster that already has VMs running on Cisco ACI Virtual Edge put the host in maintenance mode before you add the Cisco ACI Virtual Edge DVS to it. Starting the installation with the host in maintenance mode prevents the DRS from migrating VMs to the other hosts before the Cisco ACI Virtual Edge VM is fully ready.

- If the host belongs to a DRS cluster, make sure that the Enhanced VMotion Compatibility (EVC) mode for the DRS cluster is set to Nehalem or higher.

- When connecting the Cisco ACI Virtual Edge using VXLAN encapsulation, set the maximum transmission unit (MTU) value equal to or greater than 1600 on all intermediate devices on the path between the Cisco ACI fabric and the Cisco ACI Virtual Edge. These include FI switches and UCS-B. However, to optimize performance, the MTU should be set to the maximum supported size that all intermediate devices on the path between the Cisco ACI fabric and the Cisco ACI Virtual Edge support.

- If you plan to install Cisco ACI Virtual Edge using the VMware PowerCLI, synchronize the clocks for the vCenter Server, any Active Directory domain controllers, and the host making single-sign-on connection requests. If the clocks are not synchronized, you may encounter problems when deploying Cisco ACI Virtual Edge using the VMware PowerCLI tool.

For details, see the knowledge base article "Calling the SSOConnection SDK reports the exception: Client received SOAP Fault from server: The time now <timestamp> does not fall in the request lifetime interval extended with clock tolerance of 600000 ms (2125193)" on the VMware web site.

- Cisco APIC Settings Configuration

- Add ESXi Hosts and PNICs to the Cisco ACI Virtual Edge DVS

Cisco APIC Settings Configuration

The following sections describe how to configure the Cisco ACI Virtual Edge and the VMware ESXi hypervisor with the Cisco APIC:

1. vCenter Domain, Interface, and Switch Profile Creation

2. Interface and Switch Profile Guidelines and Prerequisites

3. vCenter Domain Profile Guidelines and Prerequisites

4. Create vCenter Domain, Interface, and Switch Profiles Using the GUI

- vCenter Domain, Interface, and Switch Profile Creation

- Interface and Switch Profile Guidelines and Prerequisites

- vCenter Domain Profile Guidelines and Prerequisites

- Create vCenter Domain, Interface, and Switch Profiles Using the GUI

vCenter Domain, Interface, and Switch Profile Creation

Before you can install the Cisco ACI Virtual Edge, you must create vCenter domain, interface, and switch profiles. We recommend that you perform these tasks in the united configuration wizard in the Cisco APIC. See the procedure Create vCenter Domain, Interface, and Switch Profiles Using the GUI in this guide.

Understand and follow the guidelines in this section before proceeding with the tasks.

Firewall Considerations

If you use the recommended united configuration wizard, the Cisco APIC automatically creates a firewall policy, which can be modified later. If you instead use the alternate procedures to create interface, switch, or vCenter domain profiles, you will need to create a firewall policy manually. Follow the instructions in the Distributed Firewall section of the Cisco ACI Virtual Edge Configuration Guide.

Interface and Switch Profile Guidelines and Prerequisites

Follow these guidelines and fulfill the prerequisites when creating interface and switch profiles for your Cisco ACI Virtual Edge.

Guidelines for Creating Interface and Switch Profiles

The Cisco ACI Virtual Edge supports port channel (PC), virtual port channel (VPC), MAC Pinning, and FEX interface policies.

- If there is a Layer 2 network between the leaf switch and the Cisco ACI Virtual Edge vSphere host, configure the interface policy on the interfaces that are connected to the Layer 2 network.

- The number of links and leafs that you use determine whether you configure a PC or a VPC policy for the Cisco ACI Virtual Edge:

- If you are using multiple links between one leaf and an ESXi host, you must configure a PC policy.

- If you are using multiple links between multiple leafs and an ESXi host, you must configure a VPC policy.

- Follow these guidelines for choosing a LACP policy:

- Choose LACP (Active or Passive) if the uplinks from the Cisco ACI Virtual Edge (vSphere host) are directly connected to the leaf switches and you want to use or turn on the LACP channeling protocol.

- Choose Static Channel - Mode On if the uplinks form the Cisco ACI Virtual Edge are directly connected to the leaf switches but you do not want to use the LACP channeling protocol.

- Choose MAC Pinning if the uplinks from the Cisco ACI Virtual Edge will not be channeled together and will operate as separate links.

- Follow these guidelines for choosing a vSwitch port group for the management interface:

Ensure that the vSwitch port group that you choose for the Cisco ACI Virtual Edge management interface can provide at least IPv4 addresses through DHCP or the vCenter IP pool. You can configure an additional IPv6 address for the vSwitch port group for the management interface; however, you cannot configure it only with an IPv6 address.

Prerequisites for Creating Interface and Switch Profiles

Verify that the leaf switch interfaces are physically connected to the ESXi hypervisor. Or, if you are using a Layer 2 device, verify that the leaf is physically connected to the Layer 2 device.

vCenter Domain Profile Guidelines and Prerequisites

You must create a new vCenter domain profile before you can install Cisco ACI Virtual Edge. You cannot convert an existing vCenter domain profile.

Guidelines for Creating a VMware vCenter Domain Profile

You can create multiple data centers and DVS entries under a single domain. However, you can have only one Cisco ACI Virtual Edge assigned to each data center.

You can use IPv6 when creating a VMM domain if the vCenter and ESXi host management are IPv6-enabled.

Prerequisites for Creating a VMware vCenter Domain Profile

Ensure that the multicast IP address pool has enough multicast IP addresses. You must accommodate the number of EPGs to be published to the VMware vCenter domain. You can add more IP addresses to a multicast address pool that is already associated with a VMware vCenter domain at any time.

Ensure that you have enough VLAN IDs. If you do not, ports on endpoint groups (EPGs) might report that no encapsulation is available.

vCenter must be installed, configured, and reachable through the in-band/out-of-band management network.

You must have the administrator/root credentials to the vCenter.

Create vCenter Domain, Interface, and Switch Profiles Using the GUI

Note

If you want to choose a delimiter for the VMware PortGroup name when you create a vCenter domain, you cannot do so in this procedure. This procedure uses a configuration wizard that enables you to configure a vCenter domain, interface, and switch profiles. Instead, you must create the vCenter domain separately; the delimiter option appears in the Create vCenter Domain dialog box. See the procedure Create a VMM Domain Profile for Cisco ACI Virtual Edge in this guide.

Before you begin

Before you create a vCenter domain profile, you must establish connectivity to an external network using in-band management network on the Cisco APIC.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

On the menu bar, click Fabric > Access Policies.

Step 3

In the Policies Navigation pane, right-click Switch Policies, and then click Configure Interfaces, PC, and VPC.

Step 4

In the Configure Interfaces, PC, and VPC dialog box, expand Configured Switch Interfaces, click the green + icon, and then perform the following steps:

1. In the Select Switches to Configure Interfaces area, make sure that the Quick radio button is selected.

2. From the Switches drop-down list, choose the appropriate leaf ID.

In the Switch Profile Name field, the switch profile name automatically appears.

3. Click the green + icon again.

The Configure Interfaces, PC, and VPC dialog box displays a wizard that enables you to configure vCenter domain, interface, and switch profiles.

Step 5

In the wizard, perform the following actions:

1. In the Interface Type area, choose the appropriate radio button.

PC and VPC are the only valid options for Cisco ACI Virtual Edge deployment. See the section Interface and Switch Profile Guidelines and Prerequisites in this guide.

2. In the Interfaces field, enter the interface or interface range for your vSphere hosts.

Once you enter the interface or interface range, the wizard enters a name in the Interface Selector Name field.

3. In the Interface Policy Group area, choose the Create One radio button.

Note

This procedure assumes that you are creating interface and switch policies and creating a vCenter domain rather than using existing ones. If you choose the Choose One radio button, you will not be able to create policies in the wizard.

4. From the CDP Policy or the LLDP Policy drop-down list, create a policy.

Note

If you use a Cisco Unified Computing System (UCS) server, create two policies. Create one policy to enable a Cisco Discovery Protocol (CDP) policy and a second policy to disable Link Layer Discovery Protocol (LLDP).

Note

CDP and LLDP policies are disabled by default. You can enable them in the configuration wizard. Enable CDP or LLDP policies in the Interface Policy Group area to enable them on Cisco ACI Virtual Edge and other switches in the fabric. If you want to enable CDP or LLDP only on Cisco ACI Virtual Edge, enable them in the vSwitch Policy area of the configuration wizard.

5. From the Link Level Policy drop-down list, choose a link level policy or create one.

The link level policy specifies the speed of the physical interface. If you do not choose a link level policy, the speed defaults to 10 Gbps.

6. In the Port Channel Policy drop-down list, choose Create Port Channel Policy.

7. In the Create Port Channel Policy dialog box, enter a name for the policy, choose a mode, and then click Submit.

Choose the same policy mode that is on the ESXi server. For example, if the server does not support LACP, you can choose Static Channel - Mode On or MAC Pinning. Other fields in the dialog box are optional.

8. In the Attached Device Type area, choose AVE VLAN Hosts or AVE VXLAN Hosts.

Note

If the hypervisors are directly connected to leaf switches, you can use either VLAN or VXLAN. (Cisco UCS blade servers, where Fabric Interconnects are connected to the fabric, are considered to be directly connected.) However, if the hypervisors are not directly connected to leaf switches, you must use VXLAN. For more information, see the Cisco ACI Virtual Edge Overview section.

9. In the Domain area, make sure that the Create One radio button is chosen.

The Create One option is used when creating a new VMM domain for an interface or switch profile, as you do in this procedure. The Choose One button is used when creating an interface or switch profile for a new host that you want to make part of an existing VMM domain.

10. In the Domain Name field, enter the domain name.

Note

When you create the VMM domain, you choose VLAN or VXLAN encapsulation, depending on the attached device type you chose in Step 5 h. However, you can configure a single VMM domain to use VLAN and VXLAN encapsulation. After you finish installing the Cisco ACI Virtual Edge, you can enable mixed encapsulation mode. See the section "Mixed-Mode Encapsulation Configuration" in the Cisco ACI Virtual Edge Configuration Guide.

11. Complete one of the following series of steps:

Mandatory: If you deploy Cisco ACI Virtual Edge in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If you deploy Cisco ACI Virtual Edge in VXLAN mode, only a private VLAN pool is necessary.

If in Step 5 h you chose...

Then...

AVE VLAN Hosts

1. In the VLAN area, make sure that the Create One radio button is chosen.

2. In the VLAN Range field, enter the VLAN range as appropriate.

Note

Do not define a range that includes the reserved VLAN ID for the infrastructure network because that VLAN is for internal use.

The VLAN range is for external or on-the-wire encapsulations. It is used for allocating VLANs for each EPG assigned to the domain. The VLANs are used when packets are sent to or from leafs.

3. In the Internal VLAN Range field, enter a range.

The internal VLAN range is used for private VLAN allocations in the internal vSwitch by the Cisco ACI Virtual Edge. The VLANs are not seen outside the ESX host or on the wire.

Note

If you use Cisco ACI Virtual Edge, and you deploy it in mixed-mode or VLAN mode, create two VLAN pools. Create one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If you deploy Cisco ACI Virtual Edge in VXLAN mode, only a private VLAN pool is necessary.

AVE VXLAN Hosts

1. In the VLAN area, make sure that the Create One radio button is chosen.

2. In the Internal VLAN Range field, enter a range.

3. In the Fabric Multicast Address field, enter a multicast address, such as 225.1.1.1.

4. In the Pool of Multicast Address Ranges field, create a new multicast pool or choose an existing one.

Note

The multicast address configured in Step 3 must not overlap with the ranges configured in Step 4.

5. In the Local Switching area, choose True or False.

With local switching, traffic within an EPG does not go to the leaf. So if you choose local switching, you may not see some traffic counters. If you want to see all intra-EPG traffic, choose False. See the section About Cisco ACI Virtual Edge for additional information about Local Switching and No Local Switching modes.

12. (Optional) From the Security Domains drop-down list, choose or create a security domain.

13. In the vCenter Login Name field, enter the vCenter Administrator/root username.

14. In the Password field, enter the vCenter Administrator/root password.

15. In the Confirm Password field, reenter the password.

Step 6

Click the + icon to expand vCenter, and in the Create vCenter Controller dialog box, perform the following actions:

1. In the Name field, enter a name to refer to the vCenter domain.

The name does not need to be the same as the vCenter domain name; you can use the vCenter host name.

2. In the Host Name (or IP Address) field, enter the host name or IP address.

If you use the host name, you must already have configured a DNS policy on Cisco APIC. If you do not have a DNS policy configured, enter the IP address of the vCenter server.

3. From the DVS Version drop-down list, choose a DVS version.

The DVS version that you choose represents the minimum ESXi version of the host that can be added to the virtual switch. So if you choose DVS version 6.0, you can add or manage hosts of ESXI version 6.0 and later.

Note

Cisco ACI Virtual Edge supports DVS and ESXi versions 6.0 and later.

4. In the Datacenter field, enter the data center name.

Note

The name that you enter for Datacenter must match exactly the name in vCenter. The name is case-sensitive.

5. Click OK.

Note

For the following three steps, if you do not specify port channel, vSwitch, or interface control policies, the same interface policy that you configured earlier in this procedure will take effect for the vSwitch.

6. From the Port Channel Mode drop-down list, choose a mode.

Choose MAC Pinning if you have a Unified Computing System (UCS) Fabric Interconnect (FI) between the top-of-rack switch and the Cisco ACI Virtual Edge.

7. In the vSwitch Policy area, choose a policy.

8. In the Interface Controls area, choose BPDU Guard, BPDU Filter, or both.

See the section "BPDU Features" in the Cisco ACI Virtual Edge Configuration Guide for information about BPDU Guard and BPDU Filter.

9. From the Firewall drop-down list, choose Learning, Enabled or Disabled mode.

Learning mode, the default, should be used only when upgrading to Cisco ACI Virtual Edge from a version of Cisco AVS that does not support Distributed Firewall. Otherwise, Distributed Firewall should be in Enabled mode. You can change the Distributed Firewall mode later. See the chapter "Distributed Firewall" in the Cisco ACI Virtual Edge Configuration Guide.

10. Disregard the NetFlow Exporter Policy option.

Step 7

In the Configure Interface, PC, And VPC dialog box, click Save, click Save again, and then click Submit.

Step 8

Verify the new domain and profiles, by performing the following actions:

1. On the menu bar, choose Virtual Networking > Inventory.

2. In the navigation pane, expand VMM Domains > VMware > Domain_name > Controllers, and then choose the vCenter.

In the work pane, under Properties, view the virtual machine manager (VMM) domain name to verify that the controller is online. In the work pane, the vCenter properties are displayed including the operational status. The displayed information confirms that connection from the Cisco APIC to the vCenter server is established, and the inventory is available.

Add ESXi Hosts and PNICs to the Cisco ACI Virtual Edge DVS

Before you can install Cisco ACI Virtual Edge, you must add one or more ESXi hosts and their respective PNICs to the new Cisco ACI Virtual Edge DVS.

Before you begin

- Create a VMM domain for Cisco ACI Virtual Edge. See the procedure Create a VMM Domain Profile for Cisco ACI Virtual Edge in this guide.

- Have at least one available PNIC on the host.

Procedure

Step 1

Log in to the VMware vCenter Web Client.

Step 2

Go to Networking.

Step 3

In the left navigation pane, expand the Cisco ACI Virtual Edge folder and the folder for the newly created Cisco ACI Virtual Edge VMM domain.

Step 4

Right click the Cisco ACI Virtual Edge domain and choose Add and Manage Hosts.

Step 5

In the Add and Manage Hosts dialog box, in the Select task pane, click the Add hosts radio button and then click Next.

Step 6

In the Select hosts pane, click New hosts.

Step 7

In the Select new hosts dialog box, choose all the hosts that you want to add to the Cisco ACI Virtual Edge DVS, and then click OK.

Step 8

In the Add and Manage Hosts dialog box, click Next.

Step 9

Check the Manage physical adapters check box and then click Next.

Step 10

In the Manage physical network adapters pane, choose a PNIC, and click Assign uplink.

Step 11

In the Select an Uplink dialog box, choose an uplink for the adapter, and then click OK.

Step 12

Repeat Step 10 and Step 11 for each additional PNIC you want to add.

Step 13

Click Next, click Next again, and then click Finish.

Each host that you chose in Step 6 appears in the Cisco ACI Virtual Edge domain work pane.

What to do next

Upload the OVF file of the Cisco ACI Virtual Edge VM to the vCenter.

Cisco ACI Virtual Edge Installation Using the vCenter

After you fulfill the installation prerequisites, you can use the vCenter to install Cisco ACI Virtual Edge. You use the Cisco ACI vCenter plug-in, which automates the process.

You first upload the Cisco ACI Virtual Edge VM Open Virtualization Format (OVF) file to the vCenter content library. You can then deploy Cisco ACI Virtual Edge on the ESXi hosts.

Note

If you use a local datastore for content library storage, re-create the content library after you remove a host and then reattach it to vCenter. That is because the datastore ID changes after the host is reattached, breaking the association between the content library and the datastore.

Note

After you deploy Cisco ACI Virtual Edge, do not remove it from the vCenter inventory and add it back. Doing so removes all the configurations you made during deployment. Deploy a new Cisco ACI Virtual Edge instead of adding an existing one back to the inventory.

- Upload the Cisco ACI Virtual Edge VM OVF File to the vCenter

- Deploy Cisco ACI Virtual Edge on the ESXi Hosts Using the Cisco ACI vCenter Plug-in

Upload the Cisco ACI Virtual Edge VM OVF File to the vCenter

You upload the Cisco ACI Virtual Edge VM OVF file to the vCenter before you deploy Cisco ACI Virtual Edge on the ESXi hosts.

Before you begin

You must have done the following:

- Created a VMM domain for the Cisco ACI Virtual Edge on Cisco APIC.

- Added one or more ESXi hosts and PNICs to the new Cisco ACI Virtual Edge DVS in the vCenter.

- Downloaded the folder with the OVF file to your computer.

- Made sure that the OVF file is compatible with the version of Cisco APIC.

- Have already registered the Cisco ACI fabric inside the Cisco ACI vCenter plug-in.

Procedure

Step 1

Log in to the vSphere Web Client.

Step 2

Choose Content Libraries.

You can use an existing content library or create one to receive the upload of the Cisco ACI Virtual Edge VM OVF. See VMware documentation for instructions.

Step 3

Choose the library and then click Import item.

Step 4

In the Import library item dialog box, click the Browse button.

Step 5

In the pop-up dialog box, choose the OVF file and click Open.

Once the OVF file is uploaded to the content library, it appears in the work pane under the Templates tab.

What to do next

Deploy Cisco ACI Virtual Edge on the ESXi hosts.

Deploy Cisco ACI Virtual Edge on the ESXi Hosts Using the Cisco ACI vCenter Plug-in

Before you begin

You must have done the following:

- Created a VMM domain for the Cisco ACI Virtual Edge on Cisco APIC.

- Added one or more ESXi hosts and PNICs to the new Cisco ACI Virtual Edge DVS in vCenter.

- Uploaded the Cisco ACI Virtual Edge VM OVF file to vCenter.

Note

If you use VMware vCenter 6.0 Web Client, the pop-up window for browsing to the OVF file may not appear. In that case, upload the OVF and virtual machine disk file (VMDK) to the HTTP server. Then use the OVF file URL from the server to download the OVF file to the content library.

Procedure

Step 1

Log in to the vSphere Web Client.

Step 2

In the Home work pane, click the Cisco ACI Fabric icon.

Step 3

In the Cisco ACI Fabric navigation pane, click ACI Virtual Edge.

Step 4

In the ACI Virtual Edge work pane, if there are multiple virtual domains, choose the domain from the Select an ACI Virtual Edge Domain drop-down list; if there is only one virtual domain, skip to the next step.

Step 5

Choose the host or hosts on which you want to deploy Cisco ACI Virtual Edge.

Step 6

From the ACI Virtual Edge version drop-down list, choose the version to be deployed.

Step 7

From the Management PortGroup drop-down list, choose the management port group.

Step 8

From the Datastore drop-down list, choose Custom, click Edit.

Step 9

In the Custom AVE Datastore selection dialog box, ensure that the Use local datastore only check box is checked, and then choose the local data store for each Cisco ACI Virtual Edge.

Note

Cisco ACI Virtual Edge installation is supported only on local data stores in the current release. If you try to choose a remote host, you see a warning message. Installation on remote data stores (SAN) will be supported in a future release.

Note

You may not see all types of local storage in vCenter. However, if you uncheck the Use local datastore only check box, vCenter shows all local data stores. For details, see the document "When installing ESX/ESXi 4.x or 5.x to a physical server, the local SAS drive appears as a remote storage (1027819)" on the VMware website for details.

Step 10

In the VM Admin Password fields, enter a new password for the Cisco ACI Virtual Edge VMs.

Step 11

Click Install/Upgrade ACI Virtual Edge.

Step 12

In the Install dialog box, click Yes.

In the work pane, the installed hosts display OpFlex status, the Cisco ACI Virtual Edge VM, and management IP. It could take a little while for OpFlex to come up.

What to do next

- Attach the correct EPGs to the VMM domain on the Cisco APIC controller or through vCenter using the Cisco ACI vCenter plug-in.

- Put the VMs into the correct port groups in vCenter.

Cisco ACI Virtual Edge Installation Using the VMware PowerCLI

After you fulfill the preinstallation prerequisites, you can use the VMware PowerCLI to install Cisco ACI Virtual Edge.

You first download the .zip file containing the VMware PowerCLI file, import the Cisco ACI Virtual Edge module, then deploy the new Cisco ACI Virtual Edge VM from the vCenter content library.

- Install Cisco ACI Virtual Edge Cisco ACI Virtual Edge Using the VMware PowerCLI

Install Cisco ACI Virtual Edge Cisco ACI Virtual Edge Using the VMware PowerCLI

If you have a Windows platform, you can use the VMware PowerCLI to install Cisco ACI Virtual Edge.

After you download the script module, you use the VMware PowerCLI to connect to vCenter, import the CiscoAVE module, and then deploy a new Cisco ACI Virtual Edge VM from the vCenter content library.

Note

You can use 'Get-Help' on any command to get help for any of the parameters. For example: Get-Help New-LocalContentLibrary

Procedure

Step 1

Step 2

Download the following .zip file from Cisco.com: cisco-ave-tools-version.zip.

The folder contains the following:

cisco-ave-tools-<version>

* powercli/

. CiscoAve.psm1

Step 3

Import the CiscoAVE module.

Example:

C:\> Import-Module .\Users\user name\Desktop\CiscoAve.psm1

Step 4

Connect to the vCenter.

You must connect to both the VIServer and the cisServer, as shown in the following example.

Example:

PowerCLI C:\> Connect-VIServer -Server vcenter_domain.com

PowerCLI C:\> Connect-cisServer -Server vcenter_domain.com

Step 5

Create a new content library on vCenter.

Example:

PowerCLI C:\> New-LocalContentLibrary -Name content_library_Name -Datastore data_store_name

Step 6

Upload the Cisco ACI Virtual Edge OVF file to content library.

Example:

PowerCLI C:\> New-AveContentLibraryItem -Name name_of_OVF_file -ContentLibrary content_library_name -Ovf path on your machine to OVF file

Step 7

(Optional) Remove an Item from the content library.

Example:

PowerCLI C:\> Remove-LocalContentLibraryItem -Name name_of_OVF_file -ContentLibrary content_library_name

Step 8

(Optional) Get the details of content library items.

Example:

PowerCLI C:\> Get-ContentLibraryItem -Name name_of_OVF_file

Step 9

List the Cisco ACI Virtual Edge VMs already deployed on the hosts.

Example:

C:\> Get-AveVM

The console displays a message similar to the following:

Virtual Machine             Host            Domain      Management IP
cisco-ave_192.0.2.101_ave-vm-1    192.0.2.101     ave-vm-1    192.0.2.141

Step 10

Deploy the Cisco ACI Virtual Edge VM from the vCenter content library using one of the following methods:

- Method 1—Use this method to deploy a new Cisco ACI Virtual Edge VM. This will statically assign an IP address to the Cisco ACI Virtual Edge management.

- Method 2—Use this method to deploy a new Cisco ACI Virtual Edge VM. This will require that the DHCP Server on management network to assign an IP address to the Cisco ACI Virtual Edge management.

Option

Description

If you want to use...

Then...

Method 1

Use the following command:

PowerCLI C:\> New-AveVM -HostName host where you want to deploy Cisco ACI Virtual Edge -DomainName VM_domain_name -MgmtPortgroupName
management_port_group_name -InfraVlan infraVLAN -OvfItem OVF_file_name -Datastore data_store_name -Ip Management IP address for Cisco ACI Virtual Edge -Netmask subnet_mask -Gateway gateway_ IP_address -Nameserver DNS_IP_address

Note

Choose a local data store for Cisco ACI Virtual Edge deployment. Installation of the Cisco ACI Virtual Edge VM on a remote host is not supported.

The following parameters are optional: (If you do not use them, the DHCP server on the management network must assign an IP address for Cisco ACI Virtual Edge management during deployment.)

- -iP

- -Netmask

- -Gateway

- -Nameserver

Method 2

Use the following command:

PowerCLI C:\> New-AveVM -HostName host where you want to deploy Cisco ACI Virtual Edge -DomainName VM_domain_name -MgmtPortgroupName
management_port_group_name -InfraVlan infraVLAN -OvfItem OVF_file_name -Datastore data_store_name

Note

Choose a local data store for Cisco ACIO Virtual Edge deployment. Installation of the Cisco ACI Virtual Edge VM on a remote host is not supported.

Wait for the installation to complete before proceeding to the next step. Until the deployment is finished, a progress bar and a status message display on the console:

Deploying AVE VM on Host <hostname> for domain <domainname>

Step 11

When you are prompted, enter the adminPassword, which is required to connect by SSH to the Cisco ACI Virtual Edge.

Example:

PowerCLI C:\> $password = Read-Host -AsSecureString

PowerCLI C:\> New-AveVM -HostName host where you want to deploy Cisco ACI Virtual Edge -DomainName VM VM_domain_name -MgmtPortgroupNamemanagement_port_group_name -AdminPassword $password -InfraVlan infraVLAN -OvfItem
OVF_file_name -Datastore data_store_name

Step 12

Verify the installation by again listing the Cisco ACI Virtual Edge VMs deployed on the hosts.

Example:

C:\> Get-AveVM

The console displays a list similar to the following:

Virtual Machine                            Host            Domain        Management IP
cisco-ave_192.0.2.101_ave-vm-2        192.0.2.103        ave-vm-2    192.0.2.143
cisco-ave_192.0.2.101_ave-vm-1        192.0.2.101        ave-vm-1    192.0.2.141

Step 13

(Optional) Remove any unneeded Cisco ACI Virtual Edge VMs from vCenter.

Example:

C:\> Remove-AveVM -HostName 192.0.2.101 -DomainName ave-vm-1

Example:

Wait for the uninstallation to complete before proceeding to the next step. Until the uninstallation is finished, a progress bar and status message display on the console.

Step 14

Verify the removal by again listing the Cisco ACI Virtual Edge VMs deployed on the hosts.

Example:

C:\> Get-AveVM

The console displays a list similar to the following:

Virtual Machine                            Host            Domain        Management IP
cisco-ave_192.0.2.101_ave-vm-2        192.0.2.103        ave-vm-2    192.0.2.143

Cisco ACI Virtual Edge Installation Using Python

After you fulfill the preinstallation prerequisites, you can use Python to install Cisco ACI Virtual Edge.

You first download the zip file containing the Python files, set up the environment to run Python, and then use Python commands to create a content library on vCenter, upload the Cisco ACI Virtual Edge VM OVF file to the vCenter content library, and then deploy the new VM from the content library.

- Set Up the Python Environment for Installing Cisco ACI Virtual Edge

- Install Cisco ACI Virtual Edge Using Python

Set Up the Python Environment for Installing Cisco ACI Virtual Edge

Set up the Python environment so you can use Python to install Cisco ACI Virtual Edge.

Before you begin

- Ensure that you have Python 2.7.13 or a later version.

- Ensure that you have VMware vCenter 6.0 GA U3 or later, which is required for Python installation for Cisco ACI Virtual Edge.

- We strongly recommend that you use a virtual environment for an easy and trouble-free installation process.

Procedure

Step 1

Download the following .zip file from Cisco.com: cisco-ave-tools-version.zip.

The folder contains the following:

cisco-ave-tools-<version>

* python/

. requirements.txt
. content-library.py
. get-avevm.py
. new-avevm.py
. remove-avevm.py
. utilities.py

Step 2

Complete one of the following series of steps:

Option

Description

If you...

Then...

Want to use a virtual environment

Note

If you plan to use a proxy to access the Internet, make sure to configure the proxy as follows:

export http_proxy=http://your_proxy_ip:your_proxy_port
export https_proxy=http://your_proxy_ip:your_proxy_port

1. Enter the following command:

# pip install virtualenv

2. Enter the following command:

# virtualenv venv

3. Activate your virtual environment by entering one of the following commands:

- For MAC OS and Linux systems: . venv/bin/activate

- For Windows systems: ven\Scripts\activate

Install the Python package references by entering the following command:

# pip install -r requirements.txt

Note

The file requirements.txt contains all the dependencies that the script relies on. This is a one-time task. If you have permission issues, use the following command: # sudo -H pip install -r requirements.txt

Do not want to use a virtual environment

Install the Python package references by entering the following command:

# pip install -r requirements.txt

Note

The file requirements.txt contains all the dependencies that the script relies on. This is a one-time task. If you have permission issues, use the following command: # sudo -H pip install -r requirements.txt

Install Cisco ACI Virtual Edge Using Python

You run a series of Python scripts to perform several tasks. They include creating a new content library on vCenter, uploading the Cisco ACI Virtual Edge VM OVF file to the content library, and deploying a new Cisco ACI Virtual Edge VM from the content library.

Note

You can enter -h on any script to get help for any of the parameters. Example:

# python new-avevm.py -h

Before you begin

- Ensure that you have set up the Python environment. See the procedure Set Up the Python Environment for Installing Cisco ACI Virtual Edge in this guide.

- If you used a proxy to access the Internet when setting up the Python environment, unset it before running Python scripts:

unset http_proxy
unset https_proxy

Procedure

Step 1

Create a new content library on vCenter by running the following script:

# python content-library.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password' Create --name content library name --datastore datastore name --datacenter datacenter name

Step 2

Upload an OVF file to the content library by running the following script:

# python content-library.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password' Upload --library content library name --item AVE OVF filename --path path on your machine to OVF file

The upload from your machine to the content library may take some time to complete. Wait until the upload is complete before proceeding to the next step.

A message showing the upload status displays; wait until the upload is complete.

Step 3

(Optional) Delete an item from the content library by running the following script:

# python content-library.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password' Remove --library content library name --item item to be deleted

Step 4

List all the Cisco ACI Virtual Edge VMs currently deployed by running the following script:

# python get-avevm.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password'

A list similar to the following example displays:

Virtual Machine             Host            Domain      Management IP
cisco-ave_192.0.2.101_ave-vm-1    192.0.2.101     ave-vm-1    192.0.2.141

Step 5

Deploy a new Cisco ACI Virtual Edge VM from the content library by using one of the following methods:

- Method 1—Use this method to deploy a new Cisco ACI Virtual Edge VM. This will statically assign an IP address to the Cisco ACI Virtual Edge management.

- Method 2—Use this method to deploy a new Cisco ACI Virtual Edge VM. This will require that the DHCP server on management network to assign an IP address to the Cisco ACI Virtual Edge management.

Option

Description

If you want to use...

Then...

Method 1

Use the following command:

# python new-avevm.py --vcHost vCenter host name --vcUser vCenter user name --vcPwd
vCenter password --hostName host where you want to deploy Cisco ACI Virtual Edge --domainName VM domain name --mgmtPortgroupName name of management portgroup  --infraVlan infraVLAN --ovfItem name of OVF file --datastore data store name --ip management IP address for Cisco ACI Virtual Edge --netmask subnet mask --gateway gateway IP address --nameserver DNS IP address

Note

Choose a local data store for Cisco ACI Virtual Edge deployment. Installation of the Cisco ACI Virtual Edge VM on a remote host is not supported.

The following parameters are optional: (If you do not use them, the DHCP server on the management network must assign an IP address for Cisco ACI Virtual Edge management during deployment.)

- -ip

- -subnet

- -gateway

- -nameserver

Method 2

Use the following command:

# python new-avevm.py --vcHost vCenter host name --vcUser vCenter user name --vcPwd
vCenter password --hostName host where you want to deploy Cisco ACI Virtual Edge --domainName VM domain name --mgmtPortgroupName name of management portgroup  --infraVlan infraVLAN --ovfItem name of OVF file --datastore data store name

Note

Choose a local data store for Cisco ACI Virtual Edge deployment. Installation of the Cisco ACI Virtual Edge VM on a remote host is not supported.

Step 6

When you are prompted, enter the adminPassword, which is required to connect by SSH to the Cisco ACI Virtual Edge.

Alternatively, you can read the adminPassword into a variable and use it while deploying the Cisco ACI Virtual Edge VM:

# read -s pass

[type in adminPassword]

# python new-avevm.py --vcHost vCenter host name --vcUser vCenter user name --vcPwd
vCenter password --hostName host where you want to deploy Cisco ACI Virtual Edge --domainName VM domain name --mgmtPortgroupName name of management portgroup  --infraVlan infraVLAN --ovfItem name of OVF file --adminPassword $pass --datastore data store name

Step 7

Wait for installation to complete before proceeding to the next step.

The screen displays the installation status:

Connecting to vCenter...
Getting objects from inventory...
Initializing tags...
Checking for existing AVE VM...
Getting OVF From content library...
Mapping networks...
Deploying OVF (this might take several minutes)...
Configuring VM...
Powering On VM...

Deployment successful!

Step 8

Verify the installation by again listing the Cisco ACI Virtual Edge VMs deployed by running the following script:

# python get-avevm.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password'

Example:

A list similar to the following displays:

Virtual Machine                            Host            Domain        Management IP
cisco-ave_192.0.2.103_ave-vm-2        192.0.2.103        ave-vm-2    192.0.2.143
cisco-ave_192.0.2.101_ave-vm-1        192.0.2.101        ave-vm-1    192.0.2.141

Step 9

(Optional) Remove any unneeded Cisco ACI Virtual Edge VM from vCenter by running the following script:

# python remove-avevm.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password' --hostName name of host where AVE VM resides --domainName name of VM domain

The following message displays:

Connecting to vCenter...
Getting objects from inventory...
Initializing tags...
Getting the AVE SVM...
Removing Tags...
Deleting VM...

Removal successful!

Step 10

Verify the removal by again listing the Cisco ACI Virtual Edge VMs deployed by running the following script:

# python get-avevm.py --vcHost vCenter host name --vcUser 'vCenter user name' --vcPwd 'vCenter password'

A list similar to the following displays:

Virtual Machine                            Host            Domain        Management IP
cisco-ave_192.0.2.101_ave-vm-2        192.0.2.103        ave-vm-2    192.0.2.143

Post-Installation Configuration

After you install the Cisco ACI Virtual Edge, perform key configuration tasks:

- Deploy an application profile, which includes creating a tenant, application profile, EPGs, filters, and contracts, and assigning port groups to VMs. Then verify the application profile.

See the Cisco APIC Basic Configuration Guide for instructions.

- If you want to use Distributed Firewall, Enable it after installation. See the chapter "Distributed Firewall" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

- In order for Cisco ACI Virtual Edge to forward multi-destination traffic—especially when traffic goes through a blade switch—configure an IGMP querier under the infra BD subnet. This enables devices to build their Layer 2 multicast tree.

See the section "Configuring IGMP Querier and Snooping" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

You can find instructions for other configuration tasks—including microsegmentation, SPAN, intra-EPG isolation enforcement, mixed-mode encapsulation, and BPDU features—in the Cisco ACI Virtual Edge Configuration Guide.

Chapter 3. Migration from Cisco AVS to Cisco ACI Virtual Edge

This chapter describes migration from Cisco AVS to Cisco ACI Virtual Edge, including different methods.

- About Migration from Cisco AVS to Cisco ACI Virtual Edge

- Methods of Migrating from Cisco AVS to Cisco ACI Virtual Edge

- Prerequisites for Migrating from Cisco AVS to Cisco ACI Virtual Edge

- Migrate from Cisco AVS to Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in

About Migration from Cisco AVS to Cisco ACI Virtual Edge

If you use Cisco AVS, you can migrate hosts and their VMs from that switch to Cisco ACI Virtual Edge.

After fulfilling several prerequisites, you use vCenter plug-in to use one of three methods to migrate from Cisco AVS to Cisco ACI Virtual Edge.

Methods of Migrating from Cisco AVS to Cisco ACI Virtual Edge

You can use one of three methods in the vCenter plug-in to migrate from Cisco AVS to Cisco ACI Virtual Edge. Each method automates the migration through the same vCenter GUI screens. The method you choose depends on your setup and network topology:

- DRS—All the hosts to be migrated are in the same Distributed Resource Scheduler (DRS) cluster. When the vCenter plug-in puts a host into maintenance mode, DRS automatically migrates the host's VMs to another host in the cluster.

With this method, any active VMs present on the host are automatically moved out of the host, resulting in minimal traffic loss.

- Reserve Host—You choose a reserve host, and the vCenter plug-in migrates all active VMs to this host before proceeding with the migration.

With this method, any active VMs present on the host are automatically moved out of the host, resulting in minimal traffic loss.

- In-Place Migration—The host is migrated to Cisco ACI Virtual Edge with active VMs running on the host. You may want to use this method if don't want to move VMs to another host and want only to deploy Cisco ACI Virtual Edge.

Note

This method causes significant outage to the VMs' network connectivity.

DRS Migration Workflow

The following is for migrating when there are no hosts in the cluster with Cisco ACI Virtual Edge running with OpFlex online.

1. The host is put into maintenance mode.

2. The Cisco ACI Virtual Edge DVS is added to the host.

Make sure that the Cisco ACI Virtual Edge DVS has inside/outside and intra port groups and the same port groups that are on the Cisco AVS.

3. The PNICs and vmknic are moved from the Cisco AVS DVS to the Cisco ACI Virtual Edge DVS.

4. The port group of powered-off VMs on the host is updated from Cisco AVS to Cisco ACI Virtual Edge.

5. The Cisco AVS DVS is removed from the host.

6. The Cisco AVS module is uninstalled from the host (if you chose to uninstall it).

7. An affinity rule is placed on the DRS cluster.

The affinity rule prevents DRS from moving any VMs running on the Cisco ACI Virtual Edge DVS to the host being upgraded until the Cisco ACI Virtual Edge VM is fully deployed.

8. The host is taken out of maintenance mode.

9. The Cisco ACI Virtual Edge VM is deployed on the host.

10. The OpFlex agent comes online.

11. The affinity rule on the DRS cluster is removed.

The host has now finished migrating from Cisco AVS to Cisco ACI Virtual Edge.

When you migrate another host in the cluster, the DRS cluster checks if any hosts in the cluster are running Cisco ACI Virtual Edge. When it finds one, it moves VMs from the second host to the host running Cisco ACI Virtual Edge through a cross-DVS VMotion. VMs are left on their original host if the new host does not have sufficient resources. When multiple hosts run Cisco ACI Virtual Edge, the DRS cluster balances the load among them during migration.

Reserve Host Migration Workflow

If the reserve host is running Cisco AVS, then the active VMs are moved to the reserve host using standard VMotion. If the reserve host is running Cisco ACI Virtual Edge, the active VMs are moved to the reserve host using cross-DVS VMotion.

Note

If you are using the reserve host migration method, and the reserve host is running Cisco ACI Virtual Edge, all hosts must run ESXi version 6.0. Otherwise, cross-DVS VMotion does not work.

1. The host is put into maintenance mode.

2. The Cisco ACI Virtual Edge DVS is added to the host.

3. The PNICs and vmknic are moved from the Cisco AVS DVS to the Cisco ACI Virtual Edge DVS.

4. The port group of powered-off VMs on the host is updated from Cisco AVS to Cisco ACI Virtual Edge.

5. The Cisco AVS DVS is removed from the host.

6. The Cisco AVS module is uninstalled from the host.

7. An affinity rule is placed on the DRS cluster. This prevents any VMs running on the Cisco ACI Virtual Edge DVS to move to the host being upgraded.

8. The host is taken out of maintenance mode.

9. The Cisco ACI Virtual Edge VM is deployed on the host.

10. The OpFlex Agent comes online.

11. The VMs that were automatically moved by VMotion to the reserve host in Step 1 are moved by VMotion back to the host that you just migrated to Cisco ACI Virtual Edge.

If VMs were moved to the reserve host using standard VMotion, then cross-DVS VMotion is used to move them back. If VMs were moved to the reserve host using cross-DVS VMotion, then standard VMotion is used to move them back.

In-Place Migration Workflow

1. The Cisco ACI Virtual Edge DVS is added to the host.

2. The Cisco ACI Virtual Edge VM is deployed on the host.

No uplinks are connected, and OpFlex is down.

3. The PNICs and vmknic are moved from the Cisco AVS DVS to the Cisco ACI Virtual Edge DVS.

The VMs running on Cisco AVS lose network connectivity at this point.

4. The OpFlex Agent comes online.

5. The VM port groups are updated from Cisco AVS to Cisco ACI Virtual Edge.

6. The Cisco AVS DVS is removed from the host.

Note

The Cisco ACI Virtual Edge module remains installed on the host using this method; uninstalling it requires putting the host into maintenance mode.

Prerequisites for Migrating from Cisco AVS to Cisco ACI Virtual Edge

Before you can migrate hosts and VMs from Cisco AVS to Cisco ACI Virtual Edge, perform the following tasks:

- Create a VMM domain for Cisco ACI Virtual Edge in Cisco APIC.

The associated DVS with all port groups already must be created in vCenter.

- Make sure that OpFlex is online so the hosts can migrate.

- Upload the Cisco ACI Virtual Edge OVF file to the vCenter content library.

See the procedure Upload the Cisco ACI Virtual Edge VM OVF File to the vCenter in this guide for instructions.

- Make sure that all VMs on the hosts that you plan to upgrade are using shared storage.

- Make sure that any EPGs used by vmknics use native mode switching.

Note

Unless they are backed by EPGs, vmknics will not be migrated.

- In order to use the Cisco ACI Virtual Edge management tools (including the ACI vCenter plug-in), we recommend that you use vCenter 6.0 Update 3 or later.

- Make sure that all hosts run ESXi 6.0 or later. Cross-DVS VMotion fails unless all hosts being migrated—and the reserve host when using reserve-host migration—run earlier versions of ESXi.

- Make sure that the Enhanced VMotion Compatibility (EVC) mode for the DRS cluster is set to Nehalem or higher.

- Plan VMotion migration to not exceed the limit on Cisco ACI Virtual Edge installed hosts; the limit is 300 VMs per ESXi host.

- If Cisco ACI Virtual Edge will be deployed in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation.

The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

Note

Alternatively, if you migrate from Cisco AVS to Cisco ACI Virtual Edge using the Cisco APIC GUI, you can create the private VLAN pools during migration.

- Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using REST API

- Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using the GUI

Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using REST API

A VMM domain must be ready for the hosts and VMs before you migrate from Cisco AVS to Cisco ACI Virtual Edge. You can use REST API to migrate the VMM domain. This is easier than creating a new VMM domain, which requires that you manually reproduce most of the configuration of the Cisco AVS domain.

Note

You can migrate only one Cisco AVS VMM domain at a time.

Procedure

Migrate the Cisco AVS VMM domain, providing its name, the name of the Cisco ACI Virtual Edge domain that you want to create, and user account information.

You must provide the user account information—the username and password—for the previous Cisco AVS. It is optional to specify a new VLAN pool.

Example:

The following example migrates a Cisco AVS domain named prod-avs to a domain named ave. It also creates a new VLAN pool for the Cisco ACI Virtual Edge domain.

{{ifc}}/mqapi2/migrateVMwareDomain.xml?name=prod-avs
<vmmDomP dn="uni/vmmp-VMware/dom-ave" name="ave" enableAVE="true" >
<vmmUsrAccP name="adminAcc" usr="administrator@vsphere.local" pwd="In$1eme1" />
<infraRsVlanNs tDn="uni/infra/vlanns-[inst_pvlan]-dynamic"/>
</vmmDomP>

By default, all EPGs from the Cisco AVS domain are automatically associated with the new domain. If you do not want to associate the EPGs, add the parameter migrateEPGs=False.

Associating the EPGs from the Cisco AVS domain to the new Cisco ACI Virtual Edge domain does not remove the old association.

What to do next

Follow instructions in the section Migrate from Cisco AVS to Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in in this guide.

Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using the GUI

You can use the Cisco APIC GUI to migrate an existing Cisco AVS VMM domain to a new Cisco ACI Virtual Edge domain. This method is easier than creating a new VMM domain, which requires that you manually reproduce most of the configuration of the Cisco AVS domain.

All the properties of the original domain remain; however, you must re-enter your vCenter credentials. The original VMM domain remains.

Note

You can migrate only one Cisco AVS VMM domain at a time.

Note

All EPGs associated to the original VMM domain are copied to the new Cisco ACI Virtual Edge VMM domain with Cisco ACI Virtual Edge switching mode.

Before you begin

Create a Cisco AVS domain. See the procedure Create vCenter Domain, Interface, and Switch Profiles Using the GUI in this guide.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Go to Virtual Networking > Inventory.

Step 3

In Inventory navigation pane, expand the VMM Domains and the VMware folders, right-click the Cisco AVS domain that you want to migrate, and then choose Migrate to Cisco AVE.

Step 4

In the Migrate To Cisco AVE dialog box, complete the following actions:

1. In the Virtual Switch Name field, enter a name for the new Cisco ACI Virtual Edge VMM domain.

2. From the VLAN Pool drop-down list, choose or create a VLAN pool.

If Cisco ACI Virtual Edge will be deployed in mixed mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

3. In the vCenter Credentials area, double-click Re-enter required and enter and confirm your vCenter password.

4. Click Update and then click OK.

What to do next

Follow instructions in the section Migrate from Cisco AVS to Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in in this guide.

Migrate from Cisco AVS to Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in

You use the Cisco ACI vCenter plug-in to migrate hosts and their VMs from Cisco AVS to Cisco ACI Virtual Edge, choosing one of three migration methods. The procedure is virtually the same for each method.

Before you begin

You must have fulfilled the tasks in the section Prerequisites for Migrating from Cisco AVS to Cisco ACI Virtual Edge in this guide. You also must have migrated the VMM domain; see the section Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using REST API or the section Migrate the Cisco AVS VMM Domain to Cisco ACI Virtual Edge Using the GUI in this guide.

Procedure

Step 1

Log in to the vSphere Web Client.

Step 2

In the Home work pane, click the Cisco ACI Fabric icon.

Step 3

In the Cisco ACI Fabric navigation pane, click Cisco AVS.

Step 4

In the Cisco AVS work pane, from the Select an AVS Domain drop-down list, choose a kernel Cisco AVS domain.

Step 5

On the right side of the Cisco AVS work pane, check the appropriate check box to choose each host that you want to migrate.

Step 6

Click Migrate to ACI Virtual Edge..

The AVS Migration dialog box appears.

Step 7

From the Mode drop-down list, choose a migration method.

You can choose DRS, Reserve Host, or In-Place. See the section Methods of Migrating from Cisco AVS to Cisco ACI Virtual Edge in this guide for details.

You can choose to retain the Cisco AVS kernel module by checking the retention check box.

Step 8

If you chose Reserve Host in the previous step, choose the host from the Reserve Host drop-down list that appears.

Step 9

From the Target VDS area, choose the Cisco ACI Virtual Edge domain that you want to migrate the host or hosts to.

If you have more than one Cisco ACI Virtual Edge domain, you choose the domain from a drop-down list.

Step 10

From the ACI Virtual Edge drop-down list, choose the Cisco ACI Virtual Edge installation configuration.

Step 11

From the Management drop-down list, choose the management port group for the management interface of the Cisco ACI Virtual Edge VM.

Step 12

From the Datastore drop-down list, choose Custom, click Edit.

Step 13

In the Custom AVE Datastore selection dialog box, ensure that the Use local datastore only check box is checked, and then choose the local datastore for each Cisco ACI Virtual Edge.

Installation of the Cisco ACI Virtual Edge VM on a remote host is not supported. If you try to choose a remote host, you see a warning message.

Note

You may not see all types of local storage in vCenter. However, if you uncheck the Use local datastore only check box, vCenter shows all local data stores. For details, see the document "When installing ESX/ESXi 4.x or 5.x to a physical server, the local SAS drive appears as a remote storage (1027819)" on the VMware website for details.

Step 14

In the VM Password fields, enter the admin password for the Cisco ACI Virtual Edge VM.

Step 15

Make sure that in the Migration Validity area, you see the message "The configuration is valid," and then click OK.

If the configuration is not valid, you see an error message indicating which preflight check failed and giving the reason that the migration cannot proceed. Fix the issue that caused the validation and try the migration again.

Chapter 4. Migration from VMware VDS to Cisco ACI Virtual Edge

This chapter describes migration from VMware VDS to Cisco ACI Virtual Edge, including different methods.

- About Migrating a VDS Domain to Cisco ACI Virtual Edge

- Migrate a VDS Domain to Cisco ACI Virtual Edge Using the GUI

- Migrate a VDS Domain to Cisco ACI Virtual Edge Using the NX-OS Style CLI

- Migrate a VDS Domain to Cisco ACI Virtual Edge Using REST API

About Migrating a VDS Domain to Cisco ACI Virtual Edge

If you have VMware VDS domains configured, you can migrate that domain to Cisco ACI Virtual Edge. The migration enables you to take advantage of Cisco ACI Virtual Edge features. These include the ability to use VXLAN encapsulation and Distributed Firewall.

When you migrate a VDS domain, Cisco APIC creates inside and outside and port groups on the DVS in vCenter. The domain appears as a Cisco ACI Virtual Edge domain. However, you can choose to have some endpoints operate in native VDS mode and other operate in AVE (Cisco ACI Virtual Edge) mode. That is, endpoints are switched through the VDS or the Cisco ACI Virtual Edge.

You can migrate VMware VDS domains to Cisco ACI Virtual Edge by using the Cisco APIC GUI, the NX-OS style CLI, or REST API.

Note

Changing the switching mode from native to AVE (Cisco ACI Virtual Edge) on an EPG requires changing the underlying switching platform from regular VMware DVS to Cisco ACI Virtual Edge. It also requires moving all the associated ports from DVS to Cisco ACI Virtual Edge.

This operation requires reprogramming of the port group associated with that EPG. That in turn requires a vCenter operation. It make take a few seconds for this operation to complete and for ports to show up in forwarding state on the Cisco ACI Virtual Edge switching platform. It length of time depends on the vCenter load as well as the number of endpoints that reside on the EPG that is being moved from native to AVE mode.

Migrate a VDS Domain to Cisco ACI Virtual Edge Using the GUI

This procedure migrates the existing VMM domain to a new Cisco ACI Virtual Edge VMM domain. The properties of the original VMM domain are preserved. However, no copy of the original VMM domain remains.

Before you begin

- You have created a VDS domain. See the procedure "Creating a VMM Domain Profile" in the Cisco ACI Virtualization Guide.

- If Cisco ACI Virtual Edge will be deployed in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation.

The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

Note

Alternatively, you can create the private VLAN pool when you migrate from VDS to Cisco ACI Virtual Edge.

- You have disabled First-Hop Security.

Take the following steps:

1. Under the Tenants tab, select your tenant, navigate to the bridge domain on which First Hop Security is enabled. Click the Advanced/Troubleshooting tab, and in the work pane, delete the policy.

2. Under the Tenants tab, navigate to the EPG on which the trust control policy is applied, click the General tab, and in the work pane, remove the FHS trust control policy.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Go to Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand the VMM Domains folder and the VMware folder, right-click the VDS domain that you want to migrate, and then choose Migrate to Cisco AVE.

Step 4

In the Migrate To Cisco AVE dialog box, complete the following actions:

1. In the AVE Fabric-Wide Multicast Address field, enter a multicast address.

2. From the Pool of Multicast Addresses (one per-EPG) drop-down list, choose or create a pool.

3. From the VLAN Pool drop-down list, choose or create a VLAN pool.

If Cisco ACI Virtual Edge will be deployed in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

4. Click OK.

The domain that you migrated retains its original name. However, if you click the domain in the navigation pane, you can see that the value for Virtual Switch changed from Distributed Switch to Cisco AVE.

Migrate a VDS Domain to Cisco ACI Virtual Edge Using the NX-OS Style CLI

During migration, you enable the domain to use Cisco ACI Virtual Edge.

Before you begin

You have created a VDS domain. See the procedure "Creating a VMM Domain Profile" in the Cisco ACI Virtualization Guide.

Procedure

Migrate the VDS VMM domain.

Example:

apic1(config)# vmware-domain mininet
apic1(config-vmware)# configure-ave
apic1(config-vmware-ave)# vxlan multicast-pool 225.2.1.1-225.2.1.100
apic1(config-vmware-ave)# exit
apic1(config-vmware)# exit
apic1(config)# exit
apic1#

Migrate a VDS Domain to Cisco ACI Virtual Edge Using REST API

Before you begin

You have created a VDS domain. See the procedure "Creating a VMM Domain Profile" in the Cisco ACI Virtualization Guide.

Procedure

Step 1

Create a multicast address pool if you have not done so.

Example:

<polUni>
<infraInfra>
<fvnsMcastAddrInstP name="mcast1">
<fvnsMcastAddrBlk name="mcastrange" from="225.2.1.1" to="225.2.1.100"/>
</fvnsMcastAddrInstP>
</infraInfra>
</polUni>

Step 2

Set enableAVE=true on the VMM domain and associate the multicast address pool with it.

Example:

<polUni>
<vmmProvP vendor="VMware">
<vmmDomP name="mininet" enableAVE="true">
<vmmRsDomMcastAddrNs tDn="uni/infra/maddrns-mcast1"/>
</vmmDomP>
</vmmProvP>
</polUni>

Chapter 5. Cisco ACI Virtual Edge Uninstallation

This chapter describes uninstallation of Cisco ACI Virtual Edge, including prerequisites and uninstallation methods.

- About Cisco ACI Virtual Edge Uninstallation

- Workflow for Uninstalling Cisco ACI Virtual Edge

- Uninstall Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in

- Uninstall Cisco ACI Virtual Edge Using the VMware PowerCLI

- Uninstall Cisco ACI Virtual Edge Using Python

About Cisco ACI Virtual Edge Uninstallation

You might need to remove Cisco ACI Virtual Edge for testing or if you need to remove all configuration from the Cisco ACI fabric, resetting the fabric to its initial state.

You can uninstall Cisco ACI Virtual Edge by using the Cisco ACI vCenter Plug-in, the VMware PowerCLI, or a Python script.

Note

In order to use the Cisco ACI Virtual Edge management tools (the ACI vCenter plug-in, the VMware PowerCLI, and Python scripts), we recommend that you use vCenter 6.0 Update 3 or later.

Workflow for Uninstalling Cisco ACI Virtual Edge

The order of tasks that you perform to uninstall Cisco ACI Virtual Edge depends on whether you want to keep using the VMM domain after uninstallation.

- To remove Cisco ACI Virtual Edge and keep using the VMM domain (in native mode):

1. Remove all VMs from Cisco ACI Virtual Edge port groups.

Alternatively, change the switching mode of the Cisco ACI Virtual Edge EPGs from AVE to native.

2. Uninstall the Cisco ACI Virtual Edge VM, using the Cisco ACI vCenter plug-in, VMware PowerCLI, or a Python script. Follow the instructions in this guide.

- To remove Cisco ACI Virtual Edge and delete the VMM domain:

1. Remove all VMs from EPG port groups.

2. Uninstall the Cisco ACI Virtual Edge VM, using the Cisco ACI vCenter plug-in, VMware PowerCLI, or a Python script. Follow the instructions in this guide.

3. Remove all hosts from the Cisco ACI Virtual Edge VDS.

4. Remove all VMM domain associations to EPGs to delete port groups.

5. Remove the Cisco ACI Virtual Edge VMM domain.

Uninstall Cisco ACI Virtual Edge Using the Cisco ACI vCenter Plug-in

Note

We recommend putting the host in maintenance mode before you uninstall Cisco ACI Virtual Edge if you have a DRS cluster where other hosts have VMs using Cisco ACI Virtual Edge. Going into maintenance mode prevents other VMs from migrating to the host after Cisco ACI Virtual Edge is uninstalled. If there are no other hosts in the cluster where VMs use Cisco ACI Virtual Edge, then you can proceed with uninstalling Cisco ACI Virtual Edge without putting the host in maintenance mode.

Procedure

Step 1

Log in to VMware vSphere Web Client.

Step 2

Choose Cisco ACI Fabric > ACI Virtual Edge.

Step 3

At the top of the central work pane, from the Select an ACI Virtual Edge Domain drop-down list, choose a domain.

When you choose a domain, the work pane displays the host or hosts in the vCenter related to the VMM domain.

Step 4

Choose one or more hosts by clicking the appropriate check box or check boxes.

Step 5

Below the Uninstall ACI Virtual Edge button, if you chose multiple hosts in Step 4, choose how many hosts on which to uninstall Cisco ACI Virtual Edge at the same time using the + and - buttons.

Step 6

Click Uninstall ACI Virtual Edge.

Step 7

In the Uninstall AVE dialog box, click Yes to confirm you wish to proceed with uninstalling Cisco ACI Virtual Edge.

If any of the selected hosts are part of a DRS cluster, a warning popup appears, asking if you agree to have the hosts put in maintenance mode as part of the uninstallation.

Step 8

Complete one of the following sets of actions:

If you...

Then...

Want to proceed with the uninstallation with the hosts put in maintenance mode

Click Yes.

Want to proceed with the uninstallation but not put the hosts in maintenance mode

Click No.
An additional warning displays, asking you to confirm:

- Click Yes to proceed with uninstallation without putting the hosts in maintenance mode.

- Click No to cancel the uninstallation.

Do not want to proceed with the uninstallation

Click No to cancel the uninstallation.

In the central work pane, the Status column for the host displays the uninstallation progress. You also can view progress of the individual uninstallation tasks in the Recent Tasks area. When uninstallation is complete, Not installed appears in the Status column for the host.

What to do next

If the host is part of a DRS cluster, do not move it out of maintenance mode unless it has been removed from the Cisco ACI Virtual Edge DVS.

Uninstall Cisco ACI Virtual Edge Using the VMware PowerCLI

If you have a Windows platform, you can use the VMware PowerCLI to uninstall Cisco ACI Virtual Edge.

Procedure

Uninstall Cisco ACI Virtual Edge.

Example:

Remove-AveVM –Hostname ESX IP/FQDN –DomainName VMMDomain/DVS name

Uninstall Cisco ACI Virtual Edge Using Python

You can uninstall Cisco ACI Virtual Edge using a Python script.

Before you begin

Set up the Python environment. If you have not already done so, follow the instructions in the section Set Up the Python Environment for Installing Cisco ACI Virtual Edge in this guide.

Procedure

Uninstall Cisco ACI Virtual Edge.

Example:

python remove-avevm.py –vcHost  vCenter IP/FQDN --vcUser vCenter Username --vcPwd vCenter password –hostName ESX IP/FQDN –domainName VMMDomain/DVS name

Supported Topologies

This appendix provides information about the topologies supported for the Cisco ACI Virtual Edge.

Important

Topologies not included in this appendix have not been tested and are not supported.

Note

For all topologies, we recommend using LACP wherever possible and supported by your hardware. We recommend using MAC pinning only when using LACP is not possible.

- Direct Connection

- Cisco Fabric Extender

- VPC with Cisco UCS Fabric Interconnects

- Dual-Side VPC with Cisco Nexus 5000 and MAC Pinning

- Dual-Side VPC with Cisco Nexus 5000 and VPC

- Single-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects

- Dual-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects

Direct Connection

This topology connects the ESXi hypervisor to the Cisco APIC directly.

Direct Connection

Cisco Fabric Extender

This topology connects the ESXi hypervisor to the Cisco APIC through a Fabric Extender (FEX). You can connect the ESXi to the following:

- Multiple leaf switches using a virtual port channel (VPC)

- A single leaf switch using a port channel (MAC pinning or LACP bundle)

In the following illustration, VPC is used as an example. You can use port channel instead.

Cisco Fabric Extender Topology

VPC with Cisco UCS Fabric Interconnects

This topology connects the ESXi hypervisor to the Cisco APIC using Cisco UCS Fabric Interconnects, VPCs, LACP, and MAC pinning.

VPC with Cisco UCS Fabric Interconnects Topology

In this topology, the Cisco ACI Virtual Edge can be configured only with MAC pinning. That is because Cisco UCS Fabric Interconnects don't support LACP or vLACP on the southbound ports towards the blade server. Therefore, the illustration shows MAC pinning only on the Cisco ACI Virtual Edge side.

Dual-Side VPC with Cisco Nexus 5000 and MAC Pinning

This topology connects the ESXi hypervisor to a Cisco APIC through the Cisco Nexus 5000 switch, virtual port channels, and MAC pinning.

Dual-Side VPC with Cisco Nexus 5000 and MAC Pinning Topology

Note

You also can have leaf switches and Cisco Nexus 5000 switches connected in a single-side VPC topology.

Dual-Side VPC with Cisco Nexus 5000 and VPC

This topology connects the ESXi hypervisor to a Cisco APIC through the Cisco Nexus 5000 switch and virtual port channels.

Dual-Side VPC with Cisco Nexus 5000 and VPC Topology

Note

You also can have leaf switches and Cisco Nexus 5000 switches connected in a single-side VPC topology.

Single-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects

This topology connects the ESXi hypervisor to the leaf switches using MAC pinning, directly or through Cisco Nexus 5000 switches and Cisco UCS Fabric Interconnects.

Single-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects Topology

In this topology, the Cisco ACI Virtual Edge can be configured only with MAC pinning. That is because Cisco UCS Fabric Interconnects do not support LACP on the southbound ports toward the blade server. Therefore, the illustration shows MAC pinning only on the Cisco ACI Virtual Edge side.

Dual-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects

This topology connects the ESX hypervisor to the leaf switches using MAC pinning, directly or through Cisco Nexus 5000 switches and Cisco UCS Fabric Interconnects.

Dual-Side VPC with Cisco Nexus 5000 and Cisco UCS Fabric Interconnects Topology

In this topology, the Cisco ACI Virtual Edge can be configured only with MAC pinning. That is because Cisco UCS Fabric Interconnects do not support LACP on the southbound ports toward the blade server. Therefore, the illustration shows MAC pinning only on the Cisco ACI Virtual Edge side.

Alternate Procedures for Creating vCenter Domain, Interface, and Switch Profiles

We recommend using the unified configuration wizard for performing configuration tasks before installing Cisco ACI Virtual Edge. However, you may need to configure separate, more detailed policies.

This appendix includes individual procedures for creating a vCenter domain profile and different kinds of interface and switch profiles:

- Create Port Channel Switch and Interface Profiles

- Create VPC Interface and Switch Profiles Using the GUI

- Create FEX Node Interface and Switch Profiles Using the GUI

- Modify the Interface Policy Group to Override vSwitch-Side Policies

- Create a VMM Domain Profile for Cisco ACI Virtual Edge

Create Port Channel Switch and Interface Profiles

Before you can install Cisco ACI Virtual Edge, create switch and interface profiles.

Before you begin

In Step 4 d of this procedure, you choose a leaf switch node ID from the drop-down list. It must match the node ID of the leaf switch connected to the ESXi or Layer 2 cloud host. Check the leaf switch node ID in the Fabric Membership window by going to Fabric > Inventory > Fabric Membership.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, open the Switch Policies folder, right-click the Profiles folder, and then choose Create Leaf Profile.

Step 4

In the Create Switch Profile (STEP 1 > Profile) dialog box, complete the following steps:

1. In the Name field, enter a name.

2. In the Switch Selectors field, click the + icon to create a new switch selector.

3. In the Name field, enter a name.

4. In the Blocks field, choose a leaf switch node ID from the drop-down list.

5. Click Update.

6. Click Next.

Step 5

In the Create Switch Profile (STEP 2 > Associations) dialog box, in the Interface Selectors Profiles area, click the + icon to create a new interface selector profile.

Step 6

In the Create Interface Profile dialog box, complete the following steps:

1. In the Name field, enter the vLeaf name.

2. In the Interface Selectors area, click the + icon to create a new interface selector.

Step 7

In the Create Access Port Selector dialog box, complete the following steps:

1. Enter a name for the selector in the Name field.

2. In the Interface IDs field, enter the access port interface IDs for the physical interfaces connected to the ESXi host.

3. In the Interface Policy Group drop-down list, choose Create PC Interface Policy Group.

Step 8

In the Create PC Interface Policy Group dialog box, enter the policy group name in the Name field.

Step 9

In the Port Channel Policy field, choose Create Port Channel Policy from the drop-down list.

Step 10

In the Create Port Channel Policy dialog box, complete the following steps:

1. Enter the policy name in the Name field.

2. In the Mode field, choose one of the following values:

- Static Channel - Mode On

- LACP Active

- LACP Passive

- MAC Pinning

- MAC Pinning-Physical-NIC-load

3. Click Submit.

Step 11

In the Create PC Interface Policy Group dialog box, complete the following steps:

1. In the Attached Entity Profile field, choose a profile you created earlier or create one from the drop-down list.

2. Click Submit.

Step 12

In the Create Access Port Selector dialog box, click OK.

Step 13

In the Create Leaf Interface Profile dialog box, click Submit.

Step 14

In the Create Leaf Profile dialog box, choose the new interface profile and then click Finish.

Create VPC Interface and Switch Profiles Using the GUI

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, expand the Switch Policies and Policies folder.

Step 4

Right-click the VPC Domain folder and choose Create VPC Domain Policy.

Step 5

In the Create VPC Domain Policy dialog box, complete the following steps:

1. In the Name field, enter a name for the policy.

2. In the Peer Dead Interval field, enter a value.

The range is from 3 seconds to 300 seconds.

3. Click Submit to save the policy.

Step 6

In the Policies navigation pane, under the Switch Policies folder, right-click the Profiles folder and choose Create Leaf Profile.

Step 7

In the Create Leaf Profile dialog box, complete the following steps:

1. In the Name field, enter a name for the profile.

2. In the Leaf Selectors area, click the + icon.

3. In the Name field, enter a switch selector name.

4. From the Blocks drop-down list, choose a leaf to be associated with a policy group.

5. Click Update.

6. Click Next.

Step 8

In the Create Leaf Profile dialog box, in the Interface Selector Profiles area, click the + icon.

Step 9

In the Create Leaf Interface Profile dialog box, complete the following steps:

1. In the Name field, enter a name for the profile.

2. In the Interface Selectors area, click the + icon.

Step 10

In the Create Access Port Selector dialog box, complete the following actions:

1. In the Name field, enter a selector name.

2. In the Interface IDs field, enter a range value.

3. From the Interface Policy Group drop-down list, choose Create VPC Interface Policy Group from the drop-down list.

Step 11

In the Create VPC Interface Policy Group dialog box, complete the following steps:

1. In the Name field, enter a name for the policy group.

2. From the Port Channel Policy drop-down list, choose Create Port Channel Policy from the drop-down list.

Step 12

In the Create Port Channel Policy dialog box, complete the following actions:

1. In the Name field, enter a name for the policy.

2. In the Mode field, choose one of the following options appropriate to your setup:

- Static Channel - Mode On

- LACP Active

- LACP Passive

- MAC Pinning

- MAC Pinning-Physical-load

3. Click Submit.

Step 13

In the Create VPC Interface Policy Group dialog box, complete the following actions:

1. In the Attached Entity Profile field, choose default from the drop-down list.

You can create a new attachable entity profile to override policy after you create the node policy. You may need to do so if you have intermediate Layer 2 devices between the leaf and ESXi hosts that are running the Cisco ACI Virtual Edge and if you want to use LACP with the top of rack (TOR) switch/leafs on the fabric side but use another policy, such as MAC pinning, on the Cisco ACI Virtual Edge side.

2. Click Submit.

Step 14

In the Create Access Port Selector dialog box, click OK.

Step 15

In the Create Leaf Interface Profile dialog box, click Submit.

Step 16

In the Create Leaf Profile dialog box, complete the following steps:

1. In the Interface Selector Profiles area, check the check box for the interface selector profile that you created in Step 9 a.

2. Click Finish.

Step 17

To add a second leaf to the VPC, complete the following steps:

1. Repeat Step 1 through Step 10 b; however at Step 7 b, enter the node ID of the other leaf.

2. In the Create Access Port Selector dialog box, choose the name of the policy group that you created in Step 11 a.

3. Click OK.

4. Repeat Step 15 and Step 16.

Step 18

In the Policies navigation pane, expand the Switch Policies folder and then the Policies folder.

Step 19

Right-click Virtual Port Channel default, and then choose Create VPC Explicit Protection Group.

Step 20

In the Create VPC Explicit Protection Group dialog box, enter a name, ID, and switch values for the protection group. Click Submit to save the protection group.

Note

Each pair of leaf switches has one VPC Explicit Protection Group with a unique ID.

Note

The same virtual port channel policy can contain multiple VPC explicit protection groups.

Create FEX Node Interface and Switch Profiles Using the GUI

Note

If you have a FEX directly connected to a leaf, see the section "Cisco Fabric Extender" in the Topology appendix of this guide for information about limitations.

Before you begin

In Step 4 d of this procedure, you choose a leaf switch node ID that is connected with the FEX from the drop-down list. It must match the node ID of the leaf switch connected to the ESXi or Layer 2 cloud host. Check the leaf switch node ID in the Fabric Membership window by going to Fabric > Inventory > Fabric Membership.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, open the Switch Policies folder, right-click the Profiles folder, and then choose Create Leaf Profile.

Step 4

In the Create Leaf Profile STEP 1 > Profile dialog box, complete the following steps:

1. In the Name field, enter a profile name.

2. In the Leaf Selectors field, click the + icon.

3. In the Name field, enter a name.

4. In the Blocks field, choose a leaf switch node ID that is connected with the FEX from the drop-down list.

5. Click on the Blocks drop-down arrow or anywhere on the Create Switch Profile dialog box so you can see the Update button.

6. Click Update.

7. Click Next.

Step 5

In the Create Leaf Profile STEP 2 > Associations dialog box, in the Interface Selectors Profiles area, click the + icon to create a new interface selector profile.

Step 6

In the Create Leaf Interface Profile dialog box, complete the following steps:

1. In the Name field, enter the vLeaf name.

2. In the Interface Selectors area, click the + icon to create a new interface selector.

Step 7

In the Create Access Port Selector dialog box, complete the following steps:

1. Enter a name for the selector in the Name field .

2. In the Interface IDs field, enter the access port interface IDs on the leaf that is connected to the FEX.

3. Check the Connected To Fex check box.

4. From the FEX Profile drop-down list, choose Create FEX profile.

Step 8

In the Create FEX Profile dialog box, complete the following steps:

1. In the Name field, enter an FEX profile name.

2. In the FEX Access Interface Selectors area, click the + icon to specify the FEX access ports.

Step 9

In the Create Access Port Selector dialog box, complete the following steps:

1. In the Name field, enter a selector name.

2. In the Interface IDs area, specify the access ports on the FEX that are connected to the ESXi server hosting the Cisco ACI Virtual Edge.

3. In the Interface Policy Group area, choose an option from the drop-down list.

You can choose Create PC Interface Policy Group, Create VPC Interface Policy Group, or Create Access Port Policy Group.

Step 10

In the dialog box for the option that you chose in Step 9c, complete the following steps:

1. In the Name field, enter an access policy group name.

2. In the Attached Entity Profile area, choose the appropriate attached entity profile.

3. Click Submit.

Step 11

In the Create Access Port Selector dialog box, verify that the newly created access port policy group appears in the Interface Policy Group area, and then click OK.

Step 12

In the Create FEX Profile dialog box, verify that the newly created FEX access interface selector profile appears in the FEX Access Interface Selectors area, and then click Submit.

Step 13

In the Create Access Port Selector dialog box, complete the following steps:

1. Verify that the newly created FEX profile appears in the FEX Profile area.

2. Enter an ID in theFEX ID field.

3. Click OK.

Step 14

In the Create Leaf Interface Profile dialog box, verify that the leaf-side interface port selector profile is present, and then click Submit.

Step 15

In the Create Switch Profile STEP 2 > Associations dialog box, in the Interface Selector Profiles area, check the check box for the interface selector profile that you created for the FEX, and then click FINISH.

What to do next

You should verify that the FEX node policy configuration was successful. However, you need to wait about 10 minutes to give the Cisco APIC time to complete the configuration.

To verify the FEX node policy configuration, complete the following steps in the Cisco APIC GUI:

1. Choose FABRIC > INVENTORY.

2. In the Inventory navigation pane, expand the folder for the pod containing the leaf node for which the FEX node profile was created.

3. Expand the folder for the leaf node.

4. Choose the Fabric Extenders folder.

5. In the Fabric Extenders work pane, ensure that the FEX is present.

Modify the Interface Policy Group to Override vSwitch-Side Policies

After you create a node policy, you may need to create your own attachable entity profile. This may be necessary if you have intermediate Layer 2 devices between the leaf and ESXi hosts that are running the Cisco ACI Virtual Edge. Such devices include the Cisco Nexus 5000/7000 series switches or blade servers (Unified Computing System [UCS].

The override allows a separate link policy to be configured for the intermediate devices and for the Cisco ACI Virtual Edge host uplinks. For example, if you have UCS Fabric Interconnects connected to ACI, and your Cisco ACI Virtual Edge hosts are running on UCS blades, you may want the UCS Fabric Interconnect uplinks for each FI channeled using Port Channel policy, but the host vNICs for the UCS blades are configured separately using MAC pinning.

Note

You may need to choose a vSwitch policy if both of the following are true:

- The ESXi servers hosting vSwitches are connected to the leaf through a Layer 2 switch or blade server.

- The network requires that the interface group policies between the Layer 2 device and the vSwitch hosted by the ESXi server be different from the interface group policies between the Layer 2 switch and leafs. The policies include Port Channel, Cisco Discovery Protocol (CDP), Link Layer Discovery Protocol (LLDP), Spanning Tree Protocol (STP), and Firewall.

Before you begin

- Before you create a custom attachable entity profile, you must create a VMware vCenter domain. See the section Create a VMM Domain Profile for Cisco ACI Virtual Edge in this guide for more information.

Note

When you create a vCenter domain, you must select an attachable entity profile. However, because you do not have one yet, leave the Attachable Entity Profile field blank, or choose default. After you create the custom profile, you can associate it with the vCenter domain.

- Ensure that under Fabric > Access Policies > Interface Policies > Profiles, the desired interface profiles with port selectors have been created. You associate the ports with the override polices later.

- You must have a vSwitch policy configured for your vCenter domain.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, expand the Global Policies folder, right-click the Attachable Access Entity Profiles folder, and then choose Create Attachable Access Entity Profile.

Step 4

In the Create Attachable Access Entity Profile, Step 1 > Profiles dialog box, complete the following actions:

1. In the Name field, enter a name for the profile.

2. Check the Enable Infrastructure VLAN check box.

3. Click the + icon to expand Domains, and add the VMM domain to be associated with the attachable entity profile.

4. Click Update.

5. Click Next.

Step 5

In the Create Attachable Access Entity Profile, Step 2> Association to Interfaces dialog box, choose the interface policy groups that you want to associate with the attachable entity profile.

Note

For each interface policy group, you can choose either the All or the Specific radio button. The All radio button associates all the interfaces from that interface policy group with the attachable entity profile. The Specific radio button associates specific interfaces from a specific node. If you choose a Specific radio button for an interface policy group, you will be asked to specify the switch IDs and Interfaces and then click an Update button.

Step 6

Click Finish.

Step 7

Go to Virtual Networking > Inventory

Step 8

In the left navigation pane, expand the VMM Domains and VMware folders, and then choose the relevant VMM domain.

Step 9

In the work pane, click the VSwitch Policy tab.

Step 10

From the appropriate vSwitch policy drop-down list, choose the policy that you want to apply as an override policy.

Step 11

Click Submit.

Create a VMM Domain Profile for Cisco ACI Virtual Edge

Before you can install Cisco ACI Virtual Edge, you must create a VMM domain for it in Cisco APIC.

Before you begin

- Ensure that the multicast IP address pool has enough multicast IP addresses to accommodate the number of EPGs that will be published to the VMware vCenter domain. You can add more IP addresses to a multicast address pool that is already associated with a VMware vCenter domain at any time.

- Ensure that you have a sufficient number of VLAN IDs. If you do not, ports EPGs might report that no encapsulation is available.

- Ensure that vCenter is installed, configured, and reachable through the in-band/out-of-band management network.

- Ensure that you have the administrator/root credentials to the vCenter.

- Create interface and switch profiles. See the section "Creating Port Channel Switch and Interface Profiles" in this guide for instructions.

- (Optional) Create an attachable entity profile (AEP).

During the procedure to create a vCenter domain profile, you are asked to choose or create an AEP. If you want to create one ahead of time, follow the procedure "Configuring an Attachable Entity Profile Using the GUI" in the Cisco ACI Virtual Edge Configuration Guide.

Note

You must enable the infrastructure VLAN within the AEP assigned to the Cisco ACI Virtual Edge VMM domain, regardless of whether you create the AEP before or during vCenter domain profile creation. In the Create Attachable Access Entity Profile dialog box, check the Enable Infrastructure VLAN check box.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Go to Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand VMM Domains, right-click VMware, and then choose Create vCenter Domain.

Step 4

In the Create vCenter Domain dialog box, complete the following steps:

1. In the Virtual Switch Name field, enter a name.

2. In the Virtual Switch Area, choose Cisco AVE.

Choosing Cisco AVE creates the VMM domain for Cisco ACI Virtual Edge.

3. In the Switching Preference area, choose No Local Switching or Local Switching.

For information about switching preferences, see the section About Cisco ACI Virtual Edge in the Overview chapter of this guide.

Note

If you choose No Local Switching, you can use only VXLAN encapsulation.

4. If you chose Local Switching in Step 4c, in the Default Encap Mode area, choose a mode.

You can choose VLAN mode or VXLAN mode. You can use both encapsulation methods within the same VMM domain. See the section "Mixed-Mode Encapsulation Configuration" in the Cisco ACI Virtual Edge Configuration Guide.

5. From the Associated Attachable Entity Profile drop-down list, create or choose a profile that you created earlier.

See "Configuring an Attachable Entity Profile Using the GUI" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

6. From the VLAN Pool drop-down list, choose or create a VLAN pool.

If Cisco ACI Virtual Edge will be deployed in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

7. In the AVS Fabric-Wide Multicast Address field, enter an address.

8. From the Pool of Multicast Addresses (one per-EPG) drop-down list, choose or create a pool.

9. In vCenter Credentials area, click the + (plus) icon, and in the Create vCenter credential dialog box, do the following: Enter the vCenter account profile name in the Name field, the vCenter username in the Username field, enter and confirm the vCenter password, and then click OK.

10. In the vCenter area, click the + (plus) icon, and in the Create vCenter Controller dialog box, do the following: Enter the vCenter controller name, the vCenter host name or IP address, the DVS version, data center name (which must match the data center name configured in vCenter), select the credentials created in the previous step, and then click OK.

You can choose a DVS version 5.5 or later.

11. In the Create vCenter Domain dialog box, click Submit.

In the VMware work pane, you should see the newly created VMM domain, which will be pushed to the vCenter.

Disregard the options for choosing a port channel mode, vSwitch policy, interface controls and firewall mode. You can configure Distributed Firewall later; see the instructions in the Cisco ACI Virtual Edge Configuration Guide.

What to do next

Add one or more ESXi hosts and their PNICs to the newly created Cisco ACI Virtual Edge DVS using the vSphere Web Client on the VMware vCenter.

Full Cisco Trademarks with Software License

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)