This document describes the features, caveats, and limitations for the Cisco Application Policy Infrastructure Controller (APIC) software. For more information on specific hardware features, see the Cisco NX-OS Release 11.1(4) Release Notes for Cisco Nexus 9000 Series ACI-Mode Switches. Additional product documentation is listed in the “Related Documentation” section.
Release notes are sometimes updated with new information about restrictions and caveats. See the following website for the most recent version of this document:
Table 1 shows the online change history for this document.
Table 1 Online History Change
Date | Description |
November 6, 2015 | 1.1(4e): Created the release notes for release 1.1(4e). |
November 12, 2015 | 1.1(4e): In the Resolved Caveats section, added the workaround for bug CSCuw40668. |
November 13, 2015 | 1.1(4e): In the Known Behaviors section, added bug CSCuw81638. |
December 3, 2015 | In the Installation Notes section, fixed the .egg file URLs. |
December 9, 2015 | Fixed incorrect URLs to the documentation on cisco.com. 1.1(4e): In the Open Caveats section, added bug CSCux40954. |
January 12, 2016 | 1.1(4f): Added the content for release 1.1(4f) |
February 10, 2016 | 1.1(4g): Added the content release 1.1(4g) |
February 11, 2016 | In the Compatibility Information section, corrected the supported AVS version. |
February 29, 2016 | In the Compatibility Information section, added a link to the AVS Release Notes. |
March 6, 2016 | 1.1(4i): Added the content for release 1.1(4i). |
March 16, 2016 | In the Installation Notes section, added mention that ACI with SCVMM or Windows Azure Pack only supports ASCII characters. |
April 20, 2016 | In the Verified Scalability Limits, clarified that you should see the 1.1(3f) release document. |
April 26, 2016 | Merged the all of the 1.1(4) Release Notes into a single document. 1.1(4l): Added the content for release 1.1(4l). |
August 8, 2016 | 1.1(4m): Added the content for release 1.1(4m). |
February 28, 2017 | In the Usage Guidelines section, added: If the communication between the APIC and vCenter is impaired, some functionality is adversely affected. The APIC relies on the pulling of inventory information, updating vDS configuration, and receiving event notifications from the vCenter for performing certain operations. |
This document includes the following sections:
■ Upgrading the APIC Controller
■ Downgrading the APIC Controller
■ Caveats
The Cisco Application Centric Infrastructure (ACI) is an architecture that allows the application to define the networking requirements in a programmatic way. This architecture simplifies, optimizes, and accelerates the entire application deployment life cycle.
The Cisco Application Centric Infrastructure Fundamentals guide provides complete details about the ACI, including a glossary of terms that are used in the ACI.
■ For installation instructions, see the Cisco ACI Fabric Hardware Installation Guide.
■ For instructions on how to access the APIC for the first time, see the Cisco APIC Getting Started Guide.
■ For the Cisco APIC Python SDK documentation, including installation instructions, see the Cisco APIC Python SDK Documentation.
■ Two installation egg files are needed for installation. You can download these files from a running APIC from the URLs below.
The following file is the SDK:
o http[s]://<APIC address>/cobra/_downloads/acimodel-1.1_4X-py2.7.egg
The following file includes the Python packages that model the Cisco ACI Management Information Tree:
o http[s]://<APIC address>/cobra/_downloads/acicobra-1.1_4X-py2.7.egg
“X” is the letter of the release. For example, “1.1_4e”.
Note: Installation of the SDK with SSL support on Unix/Linux and Mac OS X requires a compiler. For a Windows installation, you can install the compiled shared objects for the SDK dependencies using wheel packages.
Note: The model package depends on the SDK package; be sure to install the SDK package first.
■ Cisco ACI with Microsoft System Center Virtual Machine Manager (SCVMM) or Microsoft Windows Azure Pack only supports ASCII characters. Non-ASCII characters are not supported. Ensure that English is set in the System Locale settings for Windows, otherwise ACI with SCVMM and Windows Azure Pack will not install. In addition, if the System Locale is later modified to a non-English Locale after the installation, the integration components might fail when communicating with the APIC and the ACI fabric.
Table 2 lists the supported APIC and switch upgrades. If you are upgrading from a release that is prior to the earliest release that is listed on the table, you must upgrade to the minimum recommended release before upgrading to this release. The following document provides the minimum recommended release:
Note: APIC Image upgrades will be blocked by default if the target image is not in a supported upgrade path.
Table 2 Supported APIC and Switch Upgrades
From | To | Limitations | Recommended Procedure |
1.1.(3f) | 1.1(4) | None | 1. Upgrade APICs. 2. After APICs are upgraded successfully, upgrade the switches using two or more maintenance groups. |
1.1.(2h) | 1.1(4) | None | 1. Upgrade APICs. 2. After APICs are upgraded successfully, upgrade the switches using two or more maintenance groups. |
1.1.(1j), 1.1.(1o), 1.1.(1r) | 1.1(4) | None | 1. Upgrade APICs. 2. After APICs are upgraded successfully, upgrade the switches using two or more maintenance groups. |
1.0.(4o), 1.0.(4q) | 1.1(4) | None | 1. Upgrade APICs. 2. After APICs are upgraded successfully, upgrade the switches using two or more maintenance groups. |
Table 3 lists the supported APIC and switch downgrades.
Table 3 Supported APIC and Switch Downgrades
To | Limitations | Recommended Procedure | |
1.1(4) | 1.1(1o) and higher | None | 1. Downgrade APICs. 2. After APICs are downgraded successfully, downgrade the switches using two or more maintenance groups. |
1.1(4) | 1.0(4o) and higher | None | 1. Downgrade APICs. 2. After APICs are downgraded successfully, downgrade the switches using two or more maintenance groups. |
1.1(4) | Any other release | None | You must perform a stateless downgrade. See the procedure below. |
The following procedure performs a stateless downgrade:
Note: You must plan for a Fabric outage, as this procedure rebuilds the Fabric.
1 Export the Fabric configuration.
2 Run the “eraseconfig” command on the APIC controllers. This will reboot the controllers. Ensure that the controllers have been rebooted before moving on to step 3.
3 Run the “setup-clean-config.sh” script on the switch nodes and reload all of the switches. Steps 2 and 3 clear the configuration on the Fabric, making this is a stateless downgrade.
4 Rediscover the Fabric.
5 Downgrade the Fabric to the desired release.
6 Run the “eraseconfig setup” command on the APIC controllers. This step is required so that the script can run additional commands that might be required for the version that is being used. The “eraseconfig setup” command will reload the APICs.
7 Run the “setup-clean-config.sh” script on the switch nodes and reload them.
8 Complete the initial setup script on the APIC controllers.
9 Import the Fabric configuration using the import “merge” mode.
■ This release supports the hardware and software listed on the ACI Ecosystem Compatibility List and the software listed as follows:
— Cisco NX-OS Release 11.1(4)
— Cisco AVS
§ 1.1(4e), 1.1(4f): Release 5.2(1)SV3(1.5b)
§ 1.1(4g), 1.1(4i), 1.1(4l): Release 5.2(1)SV3(1.7)
For more information about the supported AVS releases, see the AVS software compatibility information in the Cisco Application Virtual Switch Release Notes at the following URL:
— Cisco UCS Manager software Release 2.2(1c) or later is required for the Cisco UCS Fabric Interconnect and other components, including the BIOS, CIMC, and the adapter
■ The breakout of 40G ports to 4x10G on the N9332PQ switch is not supported in ACI-Mode.
■ To connect the N2348UPQ to ACI leaf switches, the following options are available:
— Directly connect the 40G FEX ports on the N2348UPQ to the 40G switch ports on the N9332PQ switch
— Break out the 40G FEX ports on the N2348UPQ to 4x10G ports and connect to the N9396PX or N9372PX switches
■ Connecting the APIC (the controller cluster) to the ACI fabric requires a 10G interface on the ACI leaf. You cannot connect the APIC directly to the N9332PQ ACI Leaf.
■ This release supports the following firmware:
— 1.5(4e) CIMC HUU iso
— 2.0(3i) CIMC HUU iso (recommended)
■ The Cisco Application Virtual Switch (AVS) in either VLAN or VXLAN mode is not supported with Layer 4 to Layer 7 service insertion or service chaining. VMware vSphere Distributed Switch (VDS) is the only supported configuration.
■ This release supports the partner packages specified here: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-734587.html
■ This release supports Adaptive Security Appliance (ASA) device package version 1.2.3.4.
■ For information about APIC compatibility with UCS Director, see the appropriate Cisco UCS Director Compatibility Matrix document at the following URL:
This section lists usage guidelines for the APIC software.
■ The APIC GUI supports the following browsers:
— Chrome version 35 (at minimum) on Mac and Windows
— Firefox version 26 (at minimum) on Mac, Linux, and Windows
— Internet Explorer version 11 (at minimum)
— Safari 7.0.3 (at minimum)
Note: Restart your browser after upgrading to release 1.1(4).
Caution: A known issue exists with the Safari browser and unsigned certificates. Read the information presented here before accepting an unsigned certificate for use with WebSockets.
When you access the HTTPS site, the following message appears:
“Safari can’t verify the identity of the website APIC. The certificate for this website is invalid. You might be connecting to a website that is pretending to be an APIC, which could put your confidential information at risk. Would you like to connect to the website anyway?”
To ensure that WebSockets can connect, you must do the following:
1. Click Show Certificate.
2. Select Always Trust in the three drop-down lists that appear.
If you do not follow the steps above, WebSockets will not be able to connect.
■ The APIC GUI includes an online version of the Quick Start guide that includes video demonstrations.
■ The infrastructure IP address range must not overlap with other IP addresses used in the fabric for in-band and out-of-band networks.
■ The APIC does not provide IPAM services for tenant workloads.
■ To reach the APIC CLI from the GUI: select System > Controllers, highlight a controller, right-click and select "launch SSH". To get the list of commands, press the escape key twice.
■ In some of the 5-minute statistics data, the count of ten-second samples is 29 instead of 30.
■ For the following services, use a DNS-based host name with out-of-band management connectivity. IP addresses can be used with both in-band and out-of-band management connectivity.
— Syslog server
— Call Home SMTP server
— Tech support export server
— Configuration export server
— Statistics export server
■ In-band management connectivity to the spine switches is possible from any host that is connected to the leaf switches of the Fabric, and leaf switches can be managed from any host that has IP connectivity to the fabric.
■ When configuring an atomic counter policy between two endpoints, and an IP is learned on one of the two endpoints, it is recommended to use an IP-based policy and not a client endpoint-based policy.
■ When configuring two Layer 3 external networks on the same node, the loopbacks need to be configured separately for both Layer 3 networks.
■ All endpoint groups (EPGs), including application EPGs and Layer 3 external EPGs, require a domain. Interface policy groups must also be associated with an Attach Entity Profile (AEP), and the AEP must be associated with domains. Based on the association of EPGs to domains and of the interface policy groups to domains, the ports and VLANs that the EPG uses are validated. This applies to all EPGs including bridged Layer 2 outside and routed Layer 3 outside EPGs. For more information, see the Cisco Fundamentals Guide and the KB: Creating Domains, Attach Entity Profiles, and VLANs to Deploy an EPG on a Specific Port article.
Note: In the 1.0(3x) release and earlier releases, when creating static paths for application EPGs or Layer 2/Layer 3 outside EPGs, the physical domain was not required. In this release, it is required. Upgrading without the physical domain will raise a fault on the EPG stating “invalid path configuration.”
■ An EPG can only associate with a contract interface in its own tenant.
■ User passwords must meet the following criteria:
— Minimum length is 8 characters
— Maximum length is 64 characters
— Fewer than three consecutive repeated characters
— At least three of the following character types: lowercase, uppercase, digit, symbol
— Cannot be easily guessed
— Cannot be the username or the reverse of the username
— Cannot be any variation of “cisco”, “isco”, or any permutation of these characters or variants obtained by changing the capitalization of letters therein
■ The power consumption statistics are not shown on leaf node slot 1.
■ If the communication between the APIC and vCenter is impaired, some functionality is adversely affected. The APIC relies on the pulling of inventory information, updating vDS configuration, and receiving event notifications from the vCenter for performing certain operations.
For the verified scalability limits, see the Verified Scalability Guide for the 1.1(3f) release:
This section lists the new and changed features in this release and includes the following topics:
Table 4 lists the new software features in this release:
Table 4 New Software Features, Guidelines, and Restrictions
Feature | Description | Guidelines and Restrictions |
OpenStack VMM Support | This ACI release introduces support for Openstack VMM based on the OpFlex protocol (IETF draft). | None. |
This section contains lists of open and resolved caveats and known behaviors.
This section lists the open caveats. Click the bug ID to access the Bug Search tool and see additional information about the bug. If a caveat is fixed in a patch of this release, the “Fixed In” column of the tables specifies the release.
Table 5 lists the open caveats in the 1.1(4e) release.
Table 5 Open Caveats in the 1.1(4e) Release
Bug ID | Description | Fixed In |
A fault for prefix-entry-already-in-use is present when the fault is not expected. |
| |
A deployment query for an in-band endpoint group is not showing all in-band zones that are associated with the endpoint group. |
| |
An invalid path fault occurs when the same domain is attached to the selector domain and the domain is present on the override. |
| |
In the "Show Usage" table of the GUI, spine nodes are shown for an endpoint group to IP atomic counter policies. |
| |
The deployment query for dhcpRelayP sometimes returns nodes where the policy was previously deployed. |
| |
In the "Show Usage" table of the GUI, diagnostics policies applicable to leaf nodes are shown to be deployed on spine nodes as well, and vice versa. |
| |
If there are two or more primary IP addresses configured and if one of the primary IP addresses that is in use is deleted, then the deleted IP address is still used as the primary IP address. None of the remaining primary addresses are used. |
| |
The APIC GUI is stateless and there is no indication that a switch is in the rebooting state, as opposed to being disconnected for another reason. The switch disappears from fabric topology, firmware, and maintenance policies while being upgraded. The data displayed on those screens comes from the switch; however, since the switch is rebooting, no communication is possible. |
| |
In Microsoft SCVMM, if a virtual machine network is already attached and used by virtual machines, and if an admin changes the VLAN number of this virtual machine network on SCVMM, the virtual machine VLAN information is not automatically updated on Hyper-V host virtual machines. |
| |
When running a Troubleshooting Wizard session, if a VMKernel endpoint that is attached to a virtual distributed switch is used for the source or destination, the Troubleshooting Wizard fails and the following error message returns: "Error processing data returned from server: TypeError: Cannot read property 'findParentRecord' of null". |
| |
The Cisco APIC firmware process using the Upload button from the GUI does not work. The upload appears to complete successfully, but the firmware is not updated in the repository. |
|
There are no new open caveats in the 1.1(4f) release.
There are no new open caveats in the 1.1(4g) release.
There are no new open caveats in the 1.1(4i) release.
There are no new open caveats in the 1.1(4l) release.
There are no new open caveats in the 1.1(4m) release.
This section lists the resolved caveats. Click the bug ID to access the Bug Search tool and see additional information about the bug.
Table 6 lists the resolved caveats in the 1.1(4e) release.
Table 6 Resolved Caveats in the 1.1(4e) Release
Bug ID | Description |
In the APIC GUI, a server error pops up saying "Backend returned an unparsable response". | |
When expanding the subnet of an External Network Instance Profile in the GUI by double-clicking on the subnet entry, the "Aggregate Export' option will show as unset, even if it has been previously configured. | |
When APICs boots up without a keyring configuration, the default keyring is configured for HTTPS access. However, the localhost certification is applied instead of the default certification. | |
The syslog server shows many messages with "CDP interface is DOWN" on the server interfaces when the CDP policy is not configured (that is, the policy is DISABLED). | |
Endpoints with both the "learned" and "vmm" attributes do not appear for the "learned" filter nor the "vmm" filter when using the LEARNING SOURCE filter on TENANTS > Application Profiles > EPG > OPERATIONAL tab > CLIENT END-POINTS. To apply the display filter on ACI systems that are upgraded from a previous release to this release, perform the following procedure: 1. Enable the test API on the APIC while logged in as root: # enable_test_api 600 The following confirmation message displays: “Success : testapi functionality is now enabled for 600 seconds” 2. Push the following configuration: a. Log in to the APIC: {"aaaUser":{"attributes":{"name":"admin","pwd":"YOURPASSWORD"}}} b. Specify the uisettings: <polUni> <uiSettingsCont > <uiSettings epPathLcC="vmm,learned,static,dynamic" > </uiSettings> </uiSettingsCont> </polUni> | |
Traffic destined to an endpoint is incorrectly classified with an external EPG sclass. As a result, the traffic does not match the expected filters and might be policy dropped. | |
No Error is received when creating an AEP and attaching it to a USED Interface Policy Group through the Physical Domain Creation wizard. If you do not use the wizard, you are prompted with the following error message: Server Error:400 - Cannot create Attachable Access Entity Profile (infraAttEntityP); object uni/infra/attentp-asdf already exists. | |
The GUI does not accept the same value for the “from” and “to” field. Also, if a configuration is POSTed, cosmetic errors are seen in the dashboard. | |
With an EPG using multiple VLAN/VXLAN encapsulations and an Invalid Path Configuration fault, some encapsulations with known good configurations also fail to deploy. In this case, one encapsulation has failed validation, but others attached to the EPG are still not deployed. | |
When multiple Virtual Network Adapters are attached to the apicVswitch, the Hyper-V agent will sometimes choose the wrong adapter to communicate with the leaf. This will prevent the Hyper-V host from being discovered by ACI. | |
The ASA syslog shows periodic configuration input regarding an existing configuration. | |
If you use Browser Upload to upload an image to the APIC from your local machine and cancel the upload before it completes, a stale image file remains in /firmware/fwrepos/fwrepo.Uploads/. | |
No fault is created when the unicast packet counter exceeds the rising threshold (critical) for class l2.IngrPktsPart5min. | |
If you have two backup jobs with 2 different schedulers that are set to take backup at the same exact time, the first backup job succeeds, but the second backup job is sometimes becomes stuck in the "Backup job in progress" state and does not retry. | |
An alert is triggered for /var/log high usage even though usage remains below trigger threshold. | |
In ADMIN > AAA > AES Encryption > AES Encryption Passphrase, when typing in the "Passphrase" textbox, the page will freeze or the browser will crash. | |
Deleting an EPG or removing the association between an EPG and bridge domain does not remove pervasive SVI and route advertisements for this SVI. | |
When using the Visibility & Troubleshooting tool in the APIC, the following error is displayed: External ip address X.X.X.X in VRF EXAMPLE_VRF is not reachable. |
Table 7 lists the resolved caveats in the 1.1(4f) release.
Table 7 Resolved Caveats in the 1.1(4f) Release
Bug ID | Description |
If an EP moves to a different PathEP, the learned path is not updated. |
Table 8 lists the resolved caveats in the 1.1(4g) release.
Table 8 Resolved Caveats in the 1.1(4g) Release
Bug ID | Description |
An EPG that is deployed through OpFlex might get undeployed from the leaf after AEP reconfiguration. This happens when the host is directly connected to the leaf without a virtual port-channel. | |
A dynamic endpoint gets deleted from the network object group and does not get added back. |
Table 9 lists the resolved caveats in the 1.1(4i) release.
Table 9 Resolved Caveats in the 1.1(4i) Release
Bug ID | Description |
VRF route leaking must be compatible with L3Outs. | |
Dynamic object group entries are not deleted from nor added to ASA upon deleting a service graph. |
Table 10 lists the resolved caveats in the 1.1(4l) release.
Table 10 Resolved Caveats in the 1.1(4l) Release
Bug ID | Description |
Faults indicate DVS was deleted and port groups were deleted from vCenter on the APIC. During normal operations, some of the API calls fail from APIC to vCenter. As a result, the inventory fetched from vCenter is not successful and EPGs experience a connectivity issue. |
Table 10 lists the resolved caveats in the 1.1(4m) release.
Table 10 Resolved Caveats in the 1.1(4m) Release
Bug ID | Description |
A contract export to the common tenant fails with an error message similar to the following example: Server Error:400 – child (Rn) of class vzConsDef is already attached. dn[(Dn0)] Dn0=, Rn=cons-[uni/tn-dns_000test001/ap-prod_000test001/epg-auth_000test001]-any-yes, | |
Not all policies get pushed to a Layer 4 to Layer 7 service device due to the message size being exceeded, which results in the message getting dropped. |
This section lists caveats that describe known behaviors. Click the Bug ID to access the Bug Search Tool and see additional information about the bug.
Table 11 lists caveats that describe known behaviors in the 1.1(4e) release.
Table 11 Known Behaviors in the 1.1(4e) Release
Bug ID | Description |
The APIC does not validate duplicate IPs assigned to two device clusters. The communication to devices or the configuration of service devices might be affected. | |
In some of the 5-minute statistics data, the count of ten-second samples is 29 instead of 30. | |
The node ID policy can be replicated from an old appliance that is decommissioned when it joins a cluster. | |
The DSCP value specified on an external endpoint group does not take effect on filter rules on the leaf switch. | |
The hostname resolution of the syslog server fails on leaf and spine switches over in-band connectivity. | |
After importing an exported configuration, graph instances are not created and L4-L7 packages are missing in the system. | |
Following a FEX or switch reload, configured interface tags are no longer configured correctly. | |
Switches could get downgraded to a 1.0(1x) version if the imported configuration consists of a firmware policy with a desired version set to 1.0(1x). | |
Some reported client endpoints are not present on the APIC during an upgrade. | |
The APIC is rebooted using the CIMC power reboot. On reboot, the system enters into fsck due to a corrupted disk. | |
The Cisco APIC Service (ApicVMMService) shows as stopped in the Microsoft Service Manager (services.msc in control panel > admin tools > services) after valid domain credentials are entered during installation or configuration of the service. | |
The traffic destined to a shared service provider endpoint group picks an incorrect class Id (PcTag) and gets dropped. | |
Traffic from an external layer 3 network is allowed when configured as part of a vzAny (a collection of endpoint groups within a context) consumer. | |
The microsegment endpoint group is in the incorrect state after downgrading. | |
Downgrading the fabric starting with the leaf will cause faults such as policy-deployment-failed with fault code F1371. | |
The OpenStack metadata feature cannot be used with ACI integration with the Juno release (or earlier) of OpenStack due to limitations with both OpenStack and Cisco’s ML2 driver. |
There are no new known behaviors in the 1.1(4f) release.
There are no new known behaviors in the 1.1(4g) release.
There are no new known behaviors in the 1.1(4i) release.
There are no new known behaviors in the 1.1(4l) release.
There are no new known behaviors in the 1.1(4m) release.
The Cisco Application Policy Infrastructure Controller (APIC) documentation can be accessed from the following website:
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
© 2015-2017 Cisco Systems, Inc. All rights reserved.