Cisco ACI Virtual Edge Release Notes, Release 2.2(1a)
This document describes the features, bugs, and limitations for the Cisco Application Centric Infrastructure (ACI) Virtual Edge software.
Note: Use this document in combination with the Cisco Application Policy Infrastructure Controller (APIC) Release Notes, which you can view at the following location:
Release notes are sometimes updated with new information about restrictions and bugs. See this website for the most recent version of this document.
Table 1 shows the online change history for this document.
Table 1 Online History Change
Cisco ACI Virtual Edge Release 2.2(1a) became available.
Added support for Cisco ACI Virtual Edge for Cisco ACI Multi-Site environments.
This document includes the following sections:
About Cisco ACI Virtual Edge
Cisco ACI Virtual Edge Software Compatibility
Limitations and Restrictions
Cisco Application Centric Infrastructure (ACI) Virtual Edge is a hypervisor-independent distributed service virtual machine (VM) that is specifically designed for Cisco ACI. It leverages the native distributed virtual switch that belongs to the hypervisor. Cisco ACI Virtual Edge runs in the user space, operates as a virtual leaf, and is managed by Cisco APIC. If you use Cisco AVS, you can migrate to Cisco ACI Virtual Edge; if you use VMware VDS, you can run Cisco ACI Virtual Edge on top of it.
Cisco ACI Virtual Edge supports two modes of traffic forwarding: local switching and no local switching. The forwarding mode is selected during Cisco ACI Virtual Edge installation.
Cisco ACI Virtual Edge is supported as a vLeaf for Cisco APIC with the VMware ESXi hypervisor. It manages a data center defined by the VMware vCenter Server.
Cisco ACI Virtual Edge is compatible with any upstream physical access layer switch that complies with the Ethernet standard, including Cisco Nexus switches. Cisco ACI Virtual Edge is compatible with any server hardware listed in the VMware Hardware Compatibility Guide on the VMware website.
Cisco ACI Virtual Edge Release 2.2(1a) is supported as a vLeaf for Cisco APIC with releases 6.0 and later versions of the VMware ESXi hypervisor.
Cisco ACI Virtual Edge and Cisco APIC Compatibility
Cisco ACI Virtual Edge Release 2.2(1a) is compatible with Cisco APIC 4.2(1i).
See the Cisco APIC and ACI Virtual Edge Support Matrix for details.
Virtualization Compatibility Information
The Cisco ACI Virtualization Compatibility Matrix provides interoperability information for Cisco ACI components and configurations that have been tested and validated by Cisco, by Cisco partners, or both.
Compatibility and Upgrade/Downgrade Considerations
The Cisco APIC and ACI Virtual Edge Support Matrix is an interactive tool that enables you to choose an APIC version and view the compatible Cisco ACI Virtual Edge, vSphere, and compatible Cisco APIC versions.
ACI Virtual Edge Upgrade Method
You must use the Cisco ACI vCenter plug-in to upgrade Cisco ACI Virtual Edge. See the chapter “Cisco ACI Virtual Edge Upgrade” in the Cisco ACI Virtual Edge Installation Guide on Cisco.com for upgrade instructions.
Number of IP Addresses when Upgrading Cisco ACI Virtual Edge
If you use static or DHCP IP pools and want to upgrade Cisco ACI Virtual Edge, ensure that you have enough IP addresses. There must be more IP addresses in the static or DHCP pools than there are in the Cisco ACI Virtual Edge service VMs in the data center in VMware vCenter. Otherwise, the upgrade of the new Cisco ACI Virtual Edge will fail.
Cisco ACI Virtual Edge provides most of the functionality of the Cisco AVS solution, including support VLAN and VXLAN encapsulation, Microsegmentation with Cisco ACI, and Distributed Firewall. Cisco ACI Virtual Edge also provides near-seamless migration from Cisco AVS and VMware VDS.
Note: When you install or configure Cisco ACI Virtual Edge, you may see Cisco ACI Virtual Edge options labeled Cisco AVE or AVE.
This section describes new of changed features in the Cisco ACI Virtual Edge Release 2.2(1a).
DHCP Relay Support
Beginning in Cisco APIC Release 4.2(1i), you can configure a DHCP relay policy for virtual machines behind a Cisco ACI Virtual Edge when the Cisco ACI Virtual Edge is part of Cisco ACI vPod.
For information and configuration procedures see the section “Configuring a DHCP Relay Policy” in the chapter “Provisioning Core ACI Fabric Services” of the Cisco APIC Basic Configuration Guide.
Layer 4 to Layer 7 Support
Beginning in Cisco APIC Release 4.2(1i), Layer 4 to Layer 7 service graphs are supported for Cisco ACI Virtual Edge when it is part of Cisco ACI vPod.
For general Layer 4 to Layer 7 information and procedures, see the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide. For information specific to Cisco ACI Virtual Edge and Cisco ACI Virtual Edge when used with Cisco ACI vPod, see the “Layer 4 to Layer 7 Services” chapter of the Cisco ACI Virtual Edge Configuration Guide.
Microsegmentation Support for Cisco ACI Virtual Edge with Cisco ACI Virtual Pod
Beginning in Cisco APIC Release 4.2(1i), Microsegmentation with Cisco ACI is supported for Cisco ACI Virtual Edge endpoints when Cisco ACI Virtual Edge is part of Cisco ACI vPod (vPod).
For information, see the chapter “Microsegmentation with Cisco ACI” in the Cisco ACI Virtualization Guide.
Beginning in Cisco APIC Release 4.2(1i), you can configure up to 32 uplinks for each Cisco ACI Virtual Edge distributed virtual switch (DVS) and rename them. You can also configure failover for uplinks within endpoint groups (EPGs) associated with the DVS.
Cisco ACI Multi-Site Support
Cisco ACI Virtual Edge is supported for Cisco ACI Multi-Site environments in Cisco APIC 4.2(1) and Cisco ACI Multi-Site Orchestrator (MSO) 2.2(1). Cisco ACI Virtual Edge is supported for Cisco ACI Multi-Site only when it is not part of Cisco ACI vPod.
Limitations and Restrictions
For Cisco ACI Virtual Edge scalability information, see the Verified Scalability Guide for Cisco ACI for the relevant Cisco APIC release.
L3 Multicast is not supported on bridge domains with endpoints after AVE.
Cisco ACI Virtual Edge is available only on the VMware hypervisor.
VMware vSphere Support
Cisco ACI Virtual Edge is supported only on VMware vSphere 6.0 and later versions.
Cisco ACI Virtual Edge Deployment
The server where you install Cisco ACI Virtual Edge must have an Intel Nehalem CPU or later. You also must set the cluster Enhanced vMotion Compatibility (EVC) to a Nehalem CPU or later. See the knowledge base article Enhanced vMotion Compatibility (EVC) processor support (1003212) on the VMware web site.
We recommend that you install only one Cisco ACI Virtual Edge virtual machine (VM) on each host.
Removing Cisco ACI Virtual Edge or the ESXi host from the VMware vCenter and then adding it back in is not supported. If you do that, Cisco ACI Virtual Edge loses password, infra VLAN, IP address, and other key configurations. You should instead delete the original Cisco ACI Virtual Edge and deploy a new one.
After you deploy Cisco ACI Virtual Edge, if the Cisco ACI Virtual Edge VM is moved across VMware vCenter, all the configurations that you made during deployment are lost.
Management Interface IP Address
The Cisco ACI Virtual Edge management interface must have an IPv4 address. It can have an additional IPv6 address, but you cannot configure it with only an IPv6 address.
VMware vSphere vMotion Support
VMware vSphere vMotion is supported for endpoints but not supported for Cisco ACI Virtual Edge itself.
Note: After you migrate VMs using cross-data center VMware vMotion in the same VMware vCenter, you may find a stale VM entry under the source DVS. This stale entry can cause problems, such as host removal failure. The workaround for this problem is to enable "Start monitoring port state" on the vNetwork DVS. See the KB topic "Refreshing port state information for a vNetwork Distributed Virtual Switch" on the VMware Web site for instructions.
Features Not Supported for Cisco ACI Virtual Edge with Multipod
The following features are not supported for Cisco ACI Virtual Edge with multipod:
■ Storage vMotion with two separate NFS in two separate PODs
■ ERSPAN destination in different PODs
■ Distributed Firewall syslog server in different PODs
Features Not Supported for Cisco ACI Virtual Edge when It Is Part of Cisco ACI vPod
Cisco ACI Virtual Edge is not supported for the following features when it is part of Cisco ACI vPod:
■ VMware vSphere Proactive HA
■ SPAN and ERSPAN
■ Subnets configured under endpoint group EPGs
Pre-provisioning Not Supported for EPG Resolution Immediacy
When you set EPG resolution immediacy, Cisco ACI Virtual Edge does not support pre-provisioning, which downloads a policy to a switch before the switch is installed.
VMware Fault Tolerance and Distributed Firewall
When VMware Fault Tolerance failover occurs, Cisco ACI Virtual Edge cannot transfer pods because no VMware vMotion is involved in the failover. When that occurs, you need to reestablish all TCP sessions. This issue occurs regardless of whether Cisco ACI Virtual Edge is part of a Cisco ACI Virtual Pod.
Unsupported Operations in VMware vCenter
Any manual modification and override of Cisco ACI Virtual Edge portgroups in VMWare vCenter or modification of the interface connection states in Cisco ACI Virtual Edge are not supported. Making these modifications might render the Cisco ACI Virtual Edge unstable.
Using the Bug Search Tool
Use the Bug Search tool to search for a specific bug or to search for all bugs in a release.
1. Go to http://tools.cisco.com/bugsearch.
2. At the Log In screen, enter your registered Cisco.com username and password; then, click Log In. The Bug Search page opens.
Note: If you do not have a Cisco.com username and password, you can register for them at http://tools.cisco.com/RPF/register/register.do.
3. To search for a specific bug, enter the bug ID in the Search For field and press Return.
4. To search for bugs in the current release:
a. In the Search For field, enter a problem, feature, or a product name and press Return. (Leave the other fields empty.)
b. When the search results are displayed, use the filter tools to find the types of bugs you are looking for. You can search for bugs by modified date, status, severity, and so forth.
5. To export the results to a spreadsheet, click the Export Results to Excel link.
Table 3 lists the open bugs in Cisco ACI Virtual Edge Release 2.2(1a):
Table 3 – Open Cisco ACI Virtual Edge bugs
Bulk vMotion of 50 or more ports per host incurs delays for ports to come to FORWARD state
Stale endpoint: VM moves in and out of Cisco ACI Virtual Edge when OpFlex is down
Cisco AVS/Cisco ACI Virtual Edge/VMware VDS: vMotion across TOR/PG can cause multicast traffic loss up to querier interval configured
Disconnecting Cisco ACI Virtual Edge VM NICs from VMware vCenter results in complete traffic loss for VMs pinned to it
Cisco ACI Virtual Edge services not started after deployment when NFS bandwidth is insufficient
Port removed on source ACI Virtual Edge on vMotion failure event causing traffic loss
The compatible Cisco APIC version contains bug fixes; see the Cisco APIC Release Notes.
Table 4 lists the resolved bugs in the Cisco ACI Virtual Edge Release 2.2(1a):
Table 4 – Resolved Cisco ACI Virtual Edge bugs
N9K-C9372PX: high memory utilization of svc_ifc_opflexelem
Cisco ACI Virtual Edge // serverports stuck in WAIT_ATTACH_ACK after vMmotion
Process vemfwd Always Runs at 100% of CPU
The Cisco ACI Virtual Edge process vemfwd always runs at 100% of CPU. This is by design; Cisco ACI Virtual Edge always runs at a high CPU on one core to accommodate the Data Plane Development Kit (DPDK).
False Memory Fault in VMware vCenter
The Cisco ACI Virtual Edge VM can trigger a false memory fault in VMware vCenter. This is a cosmetic issue. For performance reasons, the latency sensitivity of the Cisco ACI Virtual Edge VM is set to high. This setting is known to trigger false positives in VMware vCenter. Please see the following article on the VMware website: Memory usage alarm triggers for certain types of Virtual Machines in ESXi 6.x (2149787).
Permission Denied for some Files on vem-support with Admin Login
When you log in as an administrator, you may be denied some files when you enter a vem-support command. However, you can use one of two methods to get the files:
■ Generate a log bundle by entering a command in the Cisco APIC GUI:
o Go to Admin > Import/Export > Export Policies > AVE/AVS Tech Support.
o Right-click AVE/AVS Tech Support and choose Create AVS Tech Support.
o In the Create AVE/AVS Tech Support dialog box, create a tech support export policy, which creates an archive file containing configuration information, logs, and diagnostic data that will help Cisco Technical Assistance Center (TAC) with troubleshooting and resolving a technical issue.
■ Enter sudo mode and run the vem-support all command, as shown in the following example:
cisco-ave_192.0.2.162_AVE-Lag:admin# vem-support all
Brief Delay Possible When Switching EPG from Native to AVE Mode
Changing the switching mode from Native to AVE (Cisco ACI Virtual Edge) on an EPG requires changing the underlying switching platform from regular VMware DVS to Cisco ACI Virtual Edge. It also requires moving all the associated ports from DVS to Cisco ACI Virtual Edge.
This operation requires reprogramming of the port group associated with that EPG. That in turn requires a VMware vCenter operation. This operation may take a few seconds to complete and for ports to show up in forwarding state on the Cisco ACI Virtual Edge switching platform. The length of time depends on the VMware vCenter load as well as the number of endpoints that reside on the EPG that is being moved from Native to AVE mode.
LACP Policy Not Applied on Port Channel
When a Link Aggregation Control Protocol (LACP) policy is applied as a vSwitch policy for the VMM domain, the LACP policy is applied only to the VMware vSphere Distributed Switch (VDS) uplinks. However, it is not applied to the Cisco ACI Virtual Edge port channel.
This is expected behavior. Cisco ACI Virtual Edge does not support LACP on its uplinks because VDS does not support it for its virtual Ethernet (vEth) interfaces. So, the VMM port channel policy is applied only for the VDS uplinks.
Fault Raised After Process Crash Regardless of When Crash Occurred
If any process crashes on Cisco ACI Virtual Edge, it creates a core file the /var/crash directory. Cisco ACI Virtual Edge raises a fault on Cisco APIC if any such core files are in that directory, regardless of whether the crash was recent or old.
Fault Raised on Downgrade to Cisco ACI Virtual Edge Releases Before 2.0(1a)
A fault for an invalid port group is raised on Cisco ACI Virtual Edge downgrades earlier than 2.0(1a). In Cisco ACI Virtual Edge Release 2.0(1a), a new port group, outside-cloud, is created for the Cisco ACI Virtual Edge VMM. When Cisco ACI Virtual Edge is downgraded to pre-2.0(1a) releases, the port group is removed from Cisco APIC but not from VMware vCenter.
To avoid this fault, manually delete the port group outside-cloud from the VMware vCenter if you have an existing Cisco ACI Virtual Edge domain and you are downgrading from Cisco APIC 4.0(1h) or later.
Related Documentation for Cisco ACI Virtual Edge
Cisco ACI Virtual Edge documentation is available at the following URL:
Related Documentation for Cisco APIC
Cisco APIC documentation is available at the following URL:
Cisco APIC documentation includes the Cisco ACI Virtualization Guide, which provides detailed information about Distributed Firewall and Microsegmentation with Cisco AVS.
To provide technical feedback on this document or report an error or omission, please send your comments to email@example.com. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation at: http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html.
Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation, as an RSS feed and deliver content directly to your desktop using a reader application. The RSS feeds are a free service.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
© 2019 Cisco Systems, Inc. All rights reserved.