Cisco Application Policy Infrastructure Controller Release Notes, Release 3.0(1)

This document describes the features, bugs, and limitations for the Cisco Application Policy Infrastructure Controller (APIC) software.

Note: Use this document in combination with the Cisco Nexus 9000 ACI-Mode Switches Release Notes, Release 13.0(1), which you can view at the following location:

https://www.cisco.com/c/en/us/support/switches/nexus-9000-series-switches/products-release-notes-list.html

Additional product documentation is listed in the “Related Documentation” section.

Release notes are sometimes updated with new information about restrictions and bugs. See the following website for the most recent version of this document:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

You can watch videos that demonstrate how to perform specific tasks in the Cisco APIC on the Cisco ACI YouTube channel:

https://www.youtube.com/c/CiscoACIchannel

Note: The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

Table 1 shows the online change history for this document.

Table 1 Online History Change

Date

Description

December 9, 2022

In the Open Bugs section, added bug CSCvw33061.

February 3, 2021

In the Miscellaneous Compatibility Information section, for CIMC HUU ISO, deleted the bullets that mentioned APIC-M3 and APIC-M4. These servers are not supported in this release.

October 4, 2019

In the Miscellaneous Guidelines section, added the following bullet:

■        When you create an access port selector in a leaf interface rofile, the fexId property is configured with a default value of 101 even though a FEX is not connected and the interface is not a FEX interface. The fexId property is only used when the port selector is associated with an infraFexBndlGrp managed object.

October 3, 2019

In the Miscellaneous Guidelines section, added the bullet that begins as follows:

■        Fabric connectivity ports can operate at 10G or 25G speeds (depending on the model of the APIC server) when connected to leaf switch host interfaces.

September 17, 2019

3.0(1k): In the Open Bugs section, added bug CSCuu17314 and CSCve84297.

November 21, 2018

3.0(1k): In the Open Bugs section, added bug CSCvn15374.

June 7, 2018

3.0(1k): In the Open Bugs section, added bug CSCvj75897.

February 9, 2018

In the New Software Features section, added the following item:

Cisco Tetration Analytics support on the Cisco N9K-C9348GC-FXP switch

February 2, 2018

In the Compatibility Information section, removed the following bullet:

■        Cisco Nexus 9348GC-FXP switch (N9K-C9348GC-FXP) does not support 1 gigabit or 10 gigabit on the fabric ports (53-54).

This information is now in the Cisco Nexus 9000 ACI-Mode Switches Release Notes, Release 13.0(1).

December 21, 2017

In the Changes in Behavior section, added the following bullet:

■        The source address type of a NetFlow exporter is now configurable. With this change you can configure the source address as the out-of-band management IP address.

December 12, 2017

In the New Software Features section, added the following sentence to the Forwarding Scale Profile Policy description:

The IPv4 option also increases the LPM scale up to 38k.

November 20, 2017

In the New Software Features section, changed the Forwarding Scale Profile Policy description from:

The IPv4 Scale option enables systems with no IPv6 configurations to increase scalability with up to 24K IPv4 endpoints.

To:

The IPv4 Scale option enables systems with no IPv6 configurations to increase scalability with up to 24K IPv4 local endpoints.

In the Usage Guidelines section, changed a mention of “Virtual Private Cloud (VPC)” to “virtual port channel (vPC).”

November 6, 2017

3.0(1k): In the Open Bugs section, removed bug CSCvf32908. This bug was erroneously included.

October 31, 2017

3.0(1k): In the Open Bugs section, added bug CSCvg38918.

October 24, 2017

Added the Changes in Behavior section, which has the following change:

Starting in this release, HMAC-SHA1 is no longer supported for SSH connections to the Cisco APIC and switches. HMAC-SHA1 has been replaced by HMAC-SHA2-256.

October 18, 2017

In the Known Behaviors section, added the following behavior:

Some Cisco APIC GUI screens were modified and relocated in this release, and they are not accurately reflected in the corresponding GUI Online Help. The GUI Online Help pages will be fixed in the next major release.

August 23, 2017

Added the resolved bugs.

August 10, 2017

3.0(1k): Release 3.0(1k) became available.

Contents

This document includes the following sections:

■        Introduction

■        Compatibility Information

■        Usage Guidelines

■        Verified Scalability Limits

■        New and Changed Information

■        Bugs

■        Related Documentation

Introduction

The Cisco Application Centric Infrastructure (ACI) is an architecture that allows the application to define the networking requirements in a programmatic way. This architecture simplifies, optimizes, and accelerates the entire application deployment life cycle.

The Cisco Application Centric Infrastructure Fundamentals guide provides complete details about the Cisco ACI, including a glossary of terms that are used in the Cisco ACI.

Compatibility Information

This release supports the following Cisco APIC servers:

Product ID

Description

APIC-L1

Cisco APIC with large CPU, hard drive, and memory configurations (more than 1000 edge ports)

APIC-L2

Cisco APIC with large CPU, hard drive, and memory configurations (more than 1000 edge ports)

APIC-M1

Cisco APIC with medium-size CPU, hard drive, and memory configurations (up to 1000 edge ports)

APIC-M2

Cisco APIC with medium-size CPU, hard drive, and memory configurations (up to 1000 edge ports)

The following list includes general compatibility information:

■        For the supported hardware, see the Cisco NX-OS Release Notes for Cisco Nexus 9000 Series ACI-Mode Switches, Release 13.0(1) at the following location:

https://www.cisco.com/c/en/us/support/switches/nexus-9000-series-switches/products-release-notes-list.html

■        This release supports the following software:

—      Cisco NX-OS Release 13.0(1)

—      Cisco AVS, Release 5.2(1)SV3(3.10)

For more information about the supported AVS releases, see the AVS software compatibility information in the Cisco Application Virtual Switch Release Notes at the following URL:

https://www.cisco.com/c/en/us/support/switches/application-virtual-switch/products-release-notes-list.html

—      Cisco UCS Manager software release 2.2(1c) or later is required for the Cisco UCS Fabric Interconnect and other components, including the BIOS, CIMC, and the adapter

■        To connect the N2348UPQ to Cisco ACI leaf switches, the following options are available:

—      Directly connect the 40G FEX ports on the N2348UPQ to the 40G switch ports on the Cisco ACI leaf switches

—      Break out the 40G FEX ports on the N2348UPQ to 4x10G ports and connect to the 10G ports on all other Cisco ACI leaf switches

Note: A fabric uplink port cannot be used as a FEX fabric port.

■        Connecting the Cisco APIC (the controller cluster) to the Cisco ACI fabric requires a 10G interface on the Cisco ACI leaf. You cannot connect the Cisco APIC directly to the N9332PQ Cisco ACI leaf switch, unless you use a 40G to 10G converter (part number CVR-QSFP-SFP10G), in which case the port on the N9332PQ switch will auto-negotiate to 10G without requiring any manual configuration.

■        This release supports the following firmware:

—      2.0(3i) CIMC HUU iso

—      2.0(9c) CIMC HUU iso

—      2.0(13i) CIMC HUU iso

■        This release supports VMM Integration and VMware Distributed Virtual Switch (DVS) 6.5.x. Review the Cisco ACI Virtualization Guide respective release for a full list of compatible DVS versions. For more information about guidelines for upgrading VMware DVS from 5.x to 6.x and VMM integration, see the Cisco ACI Virtualization Guide, Release 2.3(1) at the following URL:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

■        This release supports the Microsoft System Center Virtual Machine Manager (SCVMM) Update Rollup 9, 10, and 11 releases, and the Microsoft Windows Azure Pack Update Rollup 9, 10, and 11 releases.

■        This release supports SCVMM 2016 and Microsoft Hyper-V 2016.

■        This release supports the partner packages specified in the L4-L7 Compatibility List Solution Overview document at the following URL:

https://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/solution-overview-listing.html

■        This release supports Adaptive Security Appliance (ASA) device package version 1.2.5.5 or later.

■        If you are running a Cisco Adaptive Security Virtual Appliance (ASAv) version that is prior to version 9.3(2), you must configure SSL encryption as follows:

(config)# ssl encryption aes128-sha1

■        A known issue exists with the Safari browser and unsigned certificates, which applies when connecting to the Cisco APIC GUI. For more information, see the Cisco APIC Getting Started Guide.

■        For information about Cisco APIC compatibility with UCS Director, see the appropriate Cisco UCS Director Compatibility Matrix document at the following URL:

https://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-director/products-device-support-tables-list.html

■        Beginning with this release, contracts using matchDscp filters are only supported on switches with “EX” on the end of the switch name. For example, N9K-93108TC-EX.

■        N9K-X9736C-FX (ports 29 – 36) and N9K-C9364C-FX (ports 49-64) do not support 1G SFPs with QSA.

■        Cisco N9K-C9508-FM-E2 fabric modules must be physically removed before downgrading to releases earlier than Cisco APIC 3.0(1).

■        The fifth Cisco N9K-C9508-FM-E2 (also defined as FM-25) is not supported.

■        Cisco N9K-C9508-FM-E2 and N9K-X9736C-FX Locator LED Enable/Disable Feature is supported in the GUI and not supported in Cisco ACI NX-OS Switch CLIs.

■        N9K-C9508-FM-E2 and N9K-C9508-FM-E Fabric Modules in the mixed mode configuration are not supported on the same spine switch.

■        When the fabric node switch (spine or leaf) is out-of-Fabric, the Environmental Sensor values (example: Current Temperature, Power Draw, Power Consumption, and so on) might be reported as NA. A status might be reported as Normal even when the Current Temperature is NA.

■        N9K-C9348GC-FXP does not read SPROM information if the PSU is in a shut state. You might see an empty string in Cisco APIC output.

■        If you use Microsoft vSwitch and want to downgrade to Cisco APIC Release 2.3(1) from a later release, you first need to delete any microsegment EPGs configured with the Match All filter. The Match All filter is supported for Microsoft beginning with Cisco APIC Release 3.0(1).

■        For compatibility with Openstack and Kubernetes distributions, see the Cisco Application Policy Infrastructure Controller OpenStack and Container Plugins, Release 3.0(1), Release Notes.

Usage Guidelines

This section lists usage guidelines for the Cisco APIC software.

■        The Cisco APIC GUI includes an online version of the Quick Start guide that includes video demonstrations.

■        The infrastructure IP address range must not overlap with other IP addresses used in the fabric for in-band and out-of-band networks.

■        The Cisco APIC does not provide IPAM services for tenant workloads.

■        To reach the Cisco APIC CLI from the GUI: select System > Controllers, highlight a controller, right-click and select "launch SSH". To get the list of commands, press the escape key twice.

■        In some of the 5-minute statistics data, the count of ten-second samples is 29 instead of 30.

■        For the following services, use a DNS-based host name with out-of-band management connectivity. IP addresses can be used with both in-band and out-of-band management connectivity.

—      Syslog server

—      Call Home SMTP server

—      Tech support export server

—      Configuration export server

—      Statistics export server

■        If an IP address is learned on one of two endpoints for which you are configuring an atomic counter policy, you should use an IP-based policy and not a client endpoint-based policy.

■        When configuring two Layer 3 external networks on the same node, the loopbacks need to be configured separately for both Layer 3 networks.

■        All endpoint groups (EPGs), including application EPGs and Layer 3 external EPGs, require a domain. Interface policy groups must also be associated with an Attach Entity Profile (AEP), and the AEP must be associated with domains. Based on the association of EPGs to domains and of the interface policy groups to domains, the ports and VLANs that the EPG uses are validated. This applies to all EPGs including bridged Layer 2 outside and routed Layer 3 outside EPGs. For more information, see the Cisco Fundamentals Guide and the KB: Creating Domains, Attach Entity Profiles, and VLANs to Deploy an EPG on a Specific Port article.

Note: When creating static paths for application EPGs or Layer 2/Layer 3 outside EPGs, the physical domain is not required. Upgrading without the physical domain will raise a fault on the EPG stating “invalid path configuration.”

■        User passwords must meet the following criteria:

—      Minimum length is 8 characters

—      Maximum length is 64 characters

—      Fewer than three consecutive repeated characters

—      At least three of the following character types: lowercase, uppercase, digit, symbol

—      Cannot be easily guessed

—      Cannot be the username or the reverse of the username

—      Cannot be any variation of “cisco”, “isco”, or any permutation of these characters or variants obtained by changing the capitalization of letters therein

■       The power consumption statistics are not shown on leaf node slot 1.

■        For Layer 3 external networks created through the API or Advanced GUI and updated through the CLI, protocols need to be enabled globally on the external network through the API or Advanced GUI, and the node profile for all the participating nodes needs to be added through the API or Advanced GUI before doing any further updates through the CLI.

■        For Layer 3 external networks created through the Basic GUI or CLI, you should not to update them through the API. These external networks are identified by names starting with “__ui_”.

■        The output from "show" commands issued in the NX-OS-style CLI are subject to change in future software releases. Cisco does not recommend using the output from the show commands for automation.

■        The CLI is supported only for users with administrative login privileges.

■        Do not separate virtual port channel (vPC) member nodes into different configuration zones. If the nodes are in different configuration zones, then the vPCs’ modes become mismatched if the interface policies are modified and deployed to only one of the vPC member nodes.

■        If you defined multiple login domains, you can choose the login domain that you want to use when logging in to a Cisco APIC. By default, the domain drop-down list is empty, and if you do not choose a domain, the DefaultAuth domain is used for authentication. This can result in login failure if the username is not in the DefaultAuth login domain. As such, you must enter the credentials based on the chosen login domain.

■        A firmware maintenance group should contain a maximum of 80 nodes.

■        When contracts are not associated with an endpoint group, DSCP marking is not supported for a VRF with a vzAny contract. DSCP is sent to a leaf along with the actrl rule, but a vzAny contract does not have an actrl rule. Therefore, the DSCP value cannot be sent.

■        When creating a vPC domain between two leaf switches, both switches must be in the same switch generation. Switches not in the same generation are not compatible vPC peers. The generations are as follows:

—    Generation 1—Cisco Nexus N9000K switches without “EX” on the end of the switch name; for example, N9K-9312TX

—    Generation 2—Cisco Nexus N9K switches with “EX” on the end of the switch model name; for example, N9K-93108TC-EX

■        Cisco ACI does not support a class E address as a VTEP address.

■        In a multipod fabric, if a spine in POD1 uses the infra tenant L3extOut-1, the TORs of the other pods (POD2, POD3) cannot use the same infra L3extOut (L3extOut-1) for Layer 3 EVPN control plane connectivity. Each POD must use its own spine switch and infra L3extOut.

■        A multipod deployment requires the 239.255.255.240 system Global IP Outside (GIPo) to be configured on the inter-pod network (IPN) as a PIM BIDIR range. This 239.255.255.240 PIM BIDIR range configuration on the IPN devices can be avoided by using the Infra GIPo as System GIPo feature. The Infra GIPo as System GIPo feature must be enabled only after upgrading all of the switches in the Cisco ACI fabric, including the leaf switches and spine switches, to the latest Cisco APIC release.

■        The Cisco APICs must have 1 SSD and 2 HDDs, and both RAID volumes must be healthy before upgrading to this release. The Cisco APIC will not boot if the SSD is not installed.

■        You do not need to create a customized monitoring policy for each tenant. By default, a tenant shares the common policy under tenant common. The Cisco APIC automatically creates a default monitoring policy and enables common observables. You can modify the default policy under tenant common based on the requirements of your fabric.

■        If the communication between the Cisco APIC and vCenter is impaired, some functionality is adversely affected. The Cisco APIC relies on the pulling of inventory information, updating vDS configuration, and receiving event notifications from the vCenter for performing certain operations.

■        If you are upgrading VMware vCenter 6.0 to vCenter 6.5, you should first delete the following folder on the VMware vCenter:

C:\ProgramData\cisco_aci_plugin

If you do not delete the folder and you try to register a fabric again after the upgrade, you will see the following error message:

Error while saving setting in C:\ProgramData\cisco_aci_plugin\<user>_<domain>.properties

user is the user that is currently logged in to the vSphere Web Client, and domain is the domain to which the user belongs. Although you can still register a fabric, you do not have permissions to override settings that were created in the old VMware vCenter. You must enter any changes in the Cisco APIC configuration again after restarting VMware vCenter.

■        In a multipod fabric setup, if a new spine switch is added to a pod, it must first be connected to at least one leaf switch in the pod. Then the spine switch will be able to discover and join the fabric.

■        Caution: If you install 1 Gigabit Ethernet (GE) or 10GE links between the leaf and spine switches in the fabric, there is risk of packets being dropped instead of forwarded, because of inadequate bandwidth. To avoid the risk, use 40GE or 100GE links between the leaf and spine switches.

■        If FIPS is enabled in the Cisco ACI setups, then SHA256 support is mandatory on the SSH Client. Additionally, to have the SHA256 support, the openssh-client must be running version 6.6.1 or higher.

■        Basic mode will be deprecated after Cisco APIC Release 3.0(1). Cisco does not recommend using Cisco APIC Basic mode for configuration. However, if you want to use Cisco APIC Basic mode, use the following URL: APIC URL/indexSimple.html.

■        For a Cisco APIC REST API query of event records, the Cisco APIC system limits the response to a maximum of 500,000 event records. If the response is more than 500,000 events, it returns an error. Use filters to refine your queries. For more information, see Cisco APIC REST API Configuration Guide.

■        Subject Alternative Names (SANs) contain one or more alternate names and uses any variety of name forms for the entity that is bound by the Certificate Authority (CA) to the certified public key. These alternate names are called “Subject Alternative Names" (SANs). Possible names include:

—    DNS name

—    IP address

■        Fabric connectivity ports can operate at 10G or 25G speeds (depending on the model of the APIC server) when connected to leaf switch host interfaces.  We recommend connecting two fabric uplinks, each to a separate leaf switch or vPC leaf switch pair.

For APIC-M3/L3, virtual interface card (VIC) 1445 has four ports (port-1, port-2, port-3, and port-4 from left to right). Port-1 and port-2 make a single pair corresponding to eth2-1 on the APIC server; port-3 and port-4 make another pair corresponding to eth2-2 on the APIC server. Only a single connection is allowed for each pair. For example, you can connect one cable to either port-1 or port-2 and another cable to either port-3 or port-4, but not 2 cables to both ports on the same pair. Connecting 2 cables to both ports on the same pair creates instability in the APIC server. All ports must be configured for the same speed: either 10G or 25G.

■        When you create an access port selector in a leaf interface rofile, the fexId property is configured with a default value of 101 even though a FEX is not connected and the interface is not a FEX interface. The fexId property is only used when the port selector is associated with an infraFexBndlGrp managed object.

Verified Scalability Limits

For the verified scalability limits (except the CLI limits), see the Verified Scalability Guide for this release.

For the CLI verified scalability limits, see the Cisco NX-OS Style Command-Line Interface Configuration Guide for this release.

You can access these documents from the following website:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

New and Changed Information

This section lists the new and changed features in this release and includes the following topics:

■        New Software Features

■        New Hardware Features

■        Changes in Behavior

New Software Features

Table 2 lists the new software features in this release:

Table 2 New Software Features, Guidelines, and Restrictions

Feature

Description

Guidelines and Restrictions

Cisco APIC with Cisco ACI Multi-Site

As the newest advance on the Cisco ACI methods to interconnect networks, Cisco ACI Multi-Site is an architectural approach for interconnecting and managing multiple sites, each serving as a single fabric.

The Cisco ACI Multi-Site architecture has three main functional components:

■        Two or more Cisco ACI fabrics built with Nexus 9000 switches deployed as leaf and spine nodes.

■        One Cisco APIC cluster domain in each fabric.

■        An inter-site Policy Manager, named Cisco ACI Multi-Site, which is used to manage the different fabrics and to define inter-site policies.

Cisco ACI Multi-Site has the following benefits:

■        Complementary with Cisco APIC, in Multi-Site each site is an availability zone (Cisco APIC cluster domain), which can be configured to be a shared or isolated change-control zone.

■        MP-BGP EVPN is used as the control plane between sites, with data-plane VXLAN encapsulation across sites.

■        The Cisco ACI Multi-Site solution enables extending the policy domain end-to-end across fabrics.. You can create policies in the Cisco ACI Multi-Site GUI and push them to all sites or selected sites. Alternatively, you can import tenants and their policies from a single site and deploy them on other sites.

■        Multi-Site enables a global view of site health.

■        From the GUI of the Multi-Site policy manager, you can launch the site Cisco APICs.

■        Cross-site namespace normalization is performed by the connecting spine switches. This function requires Cisco Nexus 9000 Series switches with "EX" on the end of the name (and later).

■        Disaster recovery scenarios offering IP mobility across sites is one of the typical Cisco ACI Multi-Site use cases.

None.

Graceful Insertion and Removal (GIR) Mode

The Graceful Insertion and Removal (GIR) mode or maintenance mode allows you to isolate a switch from the network with minimum service disruption. In the GIR mode you can perform real-time debugging without affecting traffic.

None.

Ingress QoS Policing - per EPG per interface policing

Allows you to police all the traffic entering the traffic from all the members of an Endpoint Group. You can control the amount of traffic entering the fabric from a group of endpoints. It does so by sharing access links at the cost of other endpoints.

None.

802.1x support

IEEE 802.1x is a port-based authentication mechanism to prevent unauthorized devices from gaining access to the network.

None.

Enforced Bridge Domain

An end point in a subject endpoint group (EPG) can only ping subnet gateways within the associated bridge domain. You can then create a global exception list of IP addresses which can ping any subnet gateway.

None.

Application Quorum

Application Quorum ensures that a certain number of nodes must be online for the APP cluster to continue running and helps in preventing the split-brain scenario.

None.

Q-in-Q Encapsulation Mapping for EPGs

Using Cisco APIC, you can map double-tagged VLAN traffic ingressing on a regular interface, PC, or VPC to an EPG. When this feature is enabled and double-tagged traffic enters the network for an EPG, both tags are processed individually in the fabric and restored to double-tags when egressing the Cisco ACI switch. Ingressing single-tagged and untagged traffic is dropped. EPGs can simultaneously be associated with other interfaces on a leaf switch, that are configured for single-tagged VLANs. This feature is only supported on Nexus 9300-FX platform switches.

For configuration procedures and limitations, see Q-in-Q Encapsulation Mapping for EPGs in Cisco APIC Layer 2 Configuration Guide.

BGP Maximum Path Support

Enables you to configure the maximum number of paths that BGP adds to the route table to invoke equal-cost multipath load balancing.

None.

AS Path Prepend

Allows for the change to the length of the autonomous system path in a BGP route to invoke best-path selection by a remote peer.

None.

Kubernetes for baremetal servers

Kubernetes is an open source system that automates the deployment, scaling, and managing containers in a network. You can integrate Kubernetes on baremetal servers into the Cisco Application Centric Infrastructure (ACI).

None.

Intra-EPG contracts

Intra-EPG contracts allow some communication and forbid other communication between endpoints in the same EPG. Otherwise, intra-EPG communication is  unrestricted by default or barred completely. Intra-EPG contracts can be configured for application EPGs and uSeg EPG) on VMware VDS, Open vSwitch (OVS), and baremetal servers. For information, see the Cisco APIC Basic Configuration Guide.

Intra-EPG contracts require that the leaf switch support proxy Address Resolution Protocol (ARP). They are

supported on Cisco Nexus 9000 Series switches with EX or FX at the end of their model name or later models.

Endpoint Retention

You can now delay the amount of time between when you detach an endpoint form a host and the time it is actually detached. Doing so can prevent drops in traffic when you use VMotion on VMware VDS or Cisco AVS.  For information, see the Cisco ACI Virtualization Guide.

None.

Intra-EPG isolation support for Microsoft vSwitch

Intra-EPG Isolation is now supported for Microsoft vSwitch. By default, endpoint devices included in the same EPG are allowed to communicate with one another. However, Intra-EPG isolation enables you bar physical or virtual endpoint devices in the same base EPG or uSeg EPG from communicating with each other.

None.

NetFlow support for Cisco AVS

NetFlow technology is now supported for Cisco AVS. NetFlow  provides the metering base for a key set of applications, including network traffic accounting, usage-based network billing, denial of services monitoring, network monitoring, and data mining. For information, see the Cisco ACI Virtualization Guide.

None.

First Hop Security

Enables a better IPv4 and IPv6 link security and management over the layer 2 links. In a service provider environment, these features closely control address assignment and derived operations, such as Duplicate Address Detection (DAD) and Address Resolution (AR). Supported FHS features secure the protocols and help build a secure endpoint database on the fabric leaf switches, that are used to mitigate security threats such as MIM attacks and IP thefts.

None.

Latency and PTP

Latency is measured between endpoint groups, which requires all nodes in the fabric to be synchronized using the PTP protocol.

None.

SAML Management and 2 Factor Authentication

SAML is an XML-based open standard data format that  uses security tokens containing assertions that pass information between an SAML identity provider and a SAML service provider.

None.

Local User Authentication using OTP

OTP is a one-time password that is valid for only one session. Once OTP is enabled, Cisco APIC generates a random human readable 16 binary octets that are base32 OTP Key.

None.

Password Strength

Allows configuration of user password parameters for security management.

None.

vRealize Automation Event Broker

vRealize Automation Event Broker is a workflow subscription service for vRealize Automation to call workflows from the vRealize Orchestrator under predefined conditions. A deployment of a single/multitier application is automatically subscribed to the Event Broker. Machine operations configured by the vRA trigger the Event Broker, which invokes the preconfigured operations to the Cisco APIC.

None.

CoS Marking for Cisco AVS

Class of service (COS) marking is supported for Cisco AVS. CoS marking enables you to mark priority for traffic based on endpoint groups. For information, see the section “Class of Service and Cisco AVS” in the Cisco Application Virtual Switch Configuration Guide.

For Cisco Nexus 9000 Series switches with model names ending in EX or FX, be aware of the following:

If an egress data plane policer is already applied on a downlink from Cisco ACI, then Cisco AVS CoS cannot be preserved.

If the downlink interface is a Cisco Fabric Extender (FEX) port, then CoS in general cannot be preserved.

Forwarding Scale Profile Policy

The forwarding scale profile policy feature enables you to choose between dual stack (the default profile) and IPv4 scale. A forwarding scale profile policy that is set to dual stack provides scalability of up to 12k endpoints for IPv6 configurations and up to 24K endpoints for IPv4 configurations. The IPv4 scale option enables systems with no IPv6 configurations to increase scalability with up to 48k IPv4 endpoints. The IPv4 option also increases the LPM scale up to 38k.

For more information, see the Cisco APIC Forwarding Scale Profile Policy document.

The IPv4 scale option is supported only on LSE platforms.

Switches that support the IPv4 scale profile policy will reload after the IPv4 scale profile policy is applied. Switches that do not support the IPv4 scale profile policy will be ignored.

For a list of supported switches, see the Cisco APIC Forwarding Scale Profile Policy document.

Cisco Tetration Analytics support on the Cisco N9K-C9348GC-FXP switch

Cisco Tetration Analytics telemetry is now supported on the Cisco N9K-C9348GC-FXP switch.

None.

New Hardware Features

For new hardware features, see the Cisco Nexus 9000 ACI-Mode Switches Release Notes, Release 13.0(1) at the following location:

https://www.cisco.com/c/en/us/support/switches/nexus-9000-series-switches/products-release-notes-list.html

Changes in Behavior

This section lists changes in behavior in this release.

■        Starting in this release, HMAC-SHA1 is no longer supported for SSH connections to the Cisco APIC and switches. HMAC-SHA1 has been replaced by HMAC-SHA2-256.

■        The source address type of a NetFlow exporter is now configurable. With this change you can configure the source address as the out-of-band management IP address.

Bugs

This section contains lists of open and resolved bugs and known behaviors.

■        Open Bugs

■        Resolved Bugs

■        Known Behaviors

Open Bugs

This section lists the open bugs. Click the bug ID to access the Bug Search tool and see additional information about the bug. The "Exists In" column of the table specifies the 3.0(1) releases in which the bug exists. A bug might also exist in releases other than the 3.0(1) releases.

Table 3 Open Bugs in This Release

Bug ID

Description

Exists in

CSCuu17314

CDP is not enabled on the management interfaces for the leaf switches and spine switches.

3.0(1k) and later

CSCvd43548

The stats for a given leaf switch rule cannot be viewed if a rule is double-clicked.

3.0(1k) and later

CSCvd66359

The Port ID LLDP Neighbors panel displays the port ID when the interface does not have a description. Example:  Ethernet 1/5, but if the interface has description, the Port ID property shows the Interface description instead of the port ID.

3.0(1k) and later

CSCve84297

A service cannot be reached by using the APIC out-of-band management that exists within the 172.17.0.0/16 subnet.

3.0(1k) and later

CSCvf70362

This enhancement is to change the name of "Limit IP Learning To Subnet" under the bridge domains to be more self-explanatory.

Original :

Limit IP Learning To Subnet: [check box]

Suggestion :

Limit Local IP Learning To BD/EPG Subnet(s): [check box]

3.0(1k) and later

CSCvf70411

A route will be advertised, but will not contain the tag value that is set from the VRF route tag policy.

3.0(1k) and later

CSCvg00627

A tenant's flows/packets information cannot be exported.

3.0(1k) and later

CSCvg35344

Requesting an enhancement to allow exporting a contract by right clicking the contract itself and choosing "Export Contract" from the right click context menu. The current implementation of needing to right click the Contract folder hierarchy to export a contract is not intuitive.

3.0(1k) and later

CSCvg38918

The DHCP process crashes after a certain period of time.

3.0(1k) and later

CSCvh52046

This is an enhancement to allow for text-based banners for the Cisco APIC GUI login screen.

3.0(1k) and later

CSCvh59843

Enabling Multicast under the VRF on one or more bridge domains is difficult due to how the drop-down menu is designed. This is an enhancement request to make the drop-down menu searchable.

3.0(1k) and later

CSCvi41092

The APIC log files are extremely large, which takes a considerable amount of time to upload, especially for users with slow internet connectivity.

3.0(1k) and later

CSCvi82903

When authenticating with the Cisco APIC using ISE (TACACS), all logins over 31 characters fail.

3.0(1k) and later

CSCvj75897

A fault is raised that specifies problem that occurred while retrieving tagging information for a VMM controller.

Inventory pull from the VMware vCenter takes a long time (>10 minutes) and it continuously completes with a partial inventory result.

The processing of events from VMware vCenter is delayed, which may result in delays for the downloading of policies to the leaf switches when EPGs are deployed on-demand at the VMM domain. This would affect connectivity for newly deployed VMs or VMs which have been vMotioned.

3.0(1k) and later

CSCvn00576

An SHA2 CSR for the ACI HTTPS certificate cannot be configured in the APIC GUI.

3.0(1k) and later

CSCvn15374

When upgrading Cisco APICs, constant heartbeat loss is seen, which causes the Cisco APICs to lose connectivity between one another. In the Cisco APIC appliance_director logs, the following message is seen several hundred times during the upgrade:

appliance_director||DBG4||...||Lost heartbeat from appliance id= ...

appliance_director||DBG4||...||Appliance has become unavailable id= ...

On the switches, each process (such as policy-element) see rapidly changing leader elections and minority states:

adrs_rv||DBG4||||Updated leader election on replica=(6,26,1)

3.0(1k) and later

CSCvp64280

A vulnerability in the fabric infrastructure VLAN connection establishment of the Cisco Nexus 9000 Series Application Centric Infrastructure (ACI) Mode Switch Software could allow an unauthenticated, adjacent attacker to bypass security validations and connect an unauthorized server to the infrastructure VLAN.

The vulnerability is due to insufficient security requirements during the Link Layer Discovery Protocol (LLDP) setup phase of the infrastructure VLAN. An attacker could exploit this vulnerability by sending a malicious LLDP packet on the adjacent subnet to the Cisco Nexus 9000 Series Switch in ACI mode. A successful exploit could allow the attacker to connect an unauthorized server to the infrastructure VLAN, which is highly privileged. With a connection to the infrastructure VLAN, the attacker can make unauthorized connections to Cisco Application Policy Infrastructure Controller (APIC) services or join other host endpoints.

Cisco has released software updates that address this vulnerability. There are workarounds that address this vulnerability.

This advisory is available at the following link:

https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20190703-n9kaci-bypass

3.0(1k) and later

CSCvp72283

An APIC running the 3.0(1k) release sometimes enters the "Data Layer Partially Diverged" state. The acidiag rvread command shows the following output for the service 10 (observer):

Non optimal leader for shards :10:1,10:3,10:4,10:6,10:7,10:9,10:10,10:12,10:13,10:15,10:16,10:18,10:19,10:21,10:22,10:24,10:25,

10:27,10:28,10:30,10:31

3.0(1k) and later

CSCvq43101

When opening an external subnet, a user cannot see Aggregate Export/Import check boxes set in GUI even though they were already configured.

3.0(1k) and later

CSCvq86573

Under a corner case, the Cisco APIC cluster DB may become partially diverged after upgrading to a release that introduces new services. A new release that introduces a new DME service (such as the domainmgr in the 2.3 release) could fail to receive the full size shard vector update in first two-minute window, which causes the new service flag file to be removed before all local leader shards are able to boot into the green field mode. This results in the Cisco APIC cluster DB becoming partially diverged.

3.0(1k) and later

CSCvr65035

The last APIC in the cluster gets rebooted when APIC-1 is decommissioned due to some issue seen on APIC-1 while upgrading. In addition, after decommissioning APIC-1, the other APICs still wait for APIC-1 to get upgraded.

3.0(1k) and later

CSCvr94614

There is a minor memory leak in svc_ifc_policydist when performing various tenant configuration removals and additions.

3.0(1k) and later

CSCvu62465

For an EPG containing a static leaf node configuration, the Cisco APIC GUI returns the following error when clicking the health of Fabric Location:

Invalid DN topology/pod-X/node-Y/local/svc-policyelem-id-0/ObservedEthIf, wrong rn prefix ObservedEthIf at position 63

3.0(1k) and later

CSCvw33061

Traffic loss is observed from multiple endpoints deployed on two different vPC leaf switches.

3.0(1k) and later

Resolved Bugs

This section lists the resolved bugs. Click the bug ID to access the Bug Search tool and see additional information about the bug. The "Fixed In" column of the table specifies whether the bug was resolved in the base release or a patch release.

Table 4 Resolved Bugs in This Release

Bug ID

Description

Fixed in

CSCut11349

Gateway and netmask information might not be visible in administrator login through both the GUI or CLI console in Cisco APIC.

3.0(1k)

CSCuw36025

The following error might appear when attempting to SSH to a Cisco APIC:

“$ ssh admin@10.10.10.141

Unable to negotiate with 10.10.10.141: no matching host key type found. Their offer: ssh-dss$”

3.0(1k)

CSCux46025

The fault "invalid-vlan" might be raised by a policy element process running on the leaf switch.

3.0(1k)

CSCuz28088

Fabric discovery might fail if OSPF authentication configured on the spine switch is connected to pod-1 through Inter-Pod Network (IPN).

3.0(1k)

CSCuz80347

Clone function might not exist for Interface Policy Groups in Cisco APIC.

3.0(1k)

CSCuz80389

For Fabric wide, per vPC domain instance, or as a Switch Policy Group - there is no method to change the LACP load balancing hashing.

3.0(1k)

CSCvb49324

Cisco APIC enforces local user password complexity as follows: local users must have at least three of the following characters types: lowercase, uppercase, digit, symbol, and eight characters.

3.0(1k)

CSCvb49334

Cisco APIC allows both fallback and AAA authentication simultaneously.

3.0(1k)

CSCvd29750

After upgrade to Cisco 2.2(2k) error code F0971 might appear.

3.0(1k)

CSCvd44106

After Cisco APIC downgrade, node might show different TEP IP.

3.0(1k)

CSCvd86437

Unable to generate SAN CSR from Cisco APIC.

3.0(1k)

CSCve18041

When an AD (active directory) generated user certificate exists on Windows 2016, APIC-SCVMM agent 2.2.2e cannot be installed.

3.0(1k)

CSCve19946

When multiple Microsoft domains are present on Cisco APIC and SCVMM, powershell operations might take longer than expected and agent offline faults are seen on Cisco APIC.

3.0(1k)

CSCve21264

The topology view for an Application Profile (AP) in the Cisco APIC GUI might show EPG-Contract relations for other APs.

3.0(1k)

CSCve23475

When configuring Ingress/Egress Data Plan Policing Policy for a logical interface profile, the  error message:

"Error:400 - The interface [interface] is already being used in a different port profile with conflicting settings for l3extRsEgressQosDppPol" might appear.

3.0(1k)

CSCve36886

Deleting the association from interface policy group to Attach Entity Profile on Cisco APIC controller does not propagate to nodes. As a consequence, no fault in raised on the Endpoint Group.

3.0(1k)

CSCve52459

The Cisco APIC sends new entry during ID-Recovery and causes ACL QoS to crash.

3.0(1k)

CSCve52720

Memory leak on leaf switches might cause EPG PCTAG reallocation.

3.0(1k)

CSCve53713

When deploying a multi-pod set up with two VPC static bindings to fabric interconnects (for each pod), the configured access polices under the EPG Operational tab might only show access polices for pod 1.

3.0(1k)

CSCve61504

Cisco APIC GUI shows a different Name and Router MAC for Eth1-1 port and points to a Veth- Port.

3.0(1k)

CSCve67986

Virtual machines are not placed into micro segment EPGs when their attributes match.

3.0(1k)

CSCve70916

When combining the Cisco APIC GUI and CLI configurations, the CLI might display the GUI syslog configuration.

3.0(1k)

CSCve71413

Cisco APIC endpoint groups might become stuck in the “not-applied” state for an extended period time after a tenant is added or deleted.

3.0(1k)

CSCve75272

Deleting Fabric Extenders (FEX) in the Cisco APIC basic mode might not eliminate some created managed objects.

3.0(1k)

CSCve75781

Some virtual machine attributes might not be aplied when performing an import of configurations for uSEG endpoints.

3.0(1k)

CSCve75969

If the MAC address is configured and the burned-in MAC address is disabled, the Hot Standby Router Protocol (HSRP) uses the default MAC address instead of the configured MAC address.

3.0(1k)

CSCve78578

Automatic snapshot "remote location" does not sync with defaultAuto "remote location" in Cisco APIC.

3.0(1k)

CSCve79989

“Ctrl-c” does not reset the Cisco APIC setup script.

3.0(1k)

CSCvf16910

If the DNS extension for ML2 is enabled, floating IP allocation will fail.

3.0(1k)

CSCvf28341

Capacity Dashboard shows a limit of 200 VRFs instead of supported 400 VRFs.

3.0(1k)

Known Behaviors

This section lists bugs that describe known behaviors. Click the Bug ID to access the Bug Search Tool and see additional information about the bug. The "Exists In" column of the table specifies the 3.0(1) releases in which the known behavior exists. A bug might also exist in releases other than the 3.0(1) releases.

Table 5 Known Behaviors in This Release

Bug ID

Description

Exists in

CSCuo52668

The Cisco APIC does not validate duplicate IP addresses that are assigned to two device clusters. The communication to devices or the configuration of service devices might be affected.

3.0(1k) and later

CSCuo79243

In some of the 5-minute statistics data, the count of ten-second samples is 29 instead of 30.

3.0(1k) and later

CSCuo79250

The node ID policy can be replicated from an old appliance that is decommissioned when it joins a cluster.

3.0(1k) and later

CSCup47703

The DSCP value specified on an external endpoint group does not take effect on the filter rules on the leaf switch.

3.0(1k) and later

CSCup79002

The hostname resolution of the syslog server fails on leaf and spine switches over in-band connectivity.

3.0(1k) and later

CSCuq21360

Following a FEX or switch reload, configured interface tags are no longer configured correctly.

3.0(1k) and later

CSCur39124

Switches can be downgraded to a 1.0(1x) version if the imported configuration consists of a firmware policy with a desired version set to 1.0(1x).

3.0(1k) and later

CSCur71082

If the Cisco APIC is rebooted using the CIMC power reboot, the system enters into fsck due to a corrupted disk.

3.0(1k) and later

CSCus15627

The Cisco APIC Service (ApicVMMService) shows as stopped in the Microsoft Service Manager (services.msc in control panel > admin tools > services). This happens when a domain account does not have the correct privilege in the domain to restart the service automatically.

3.0(1k) and later

CSCut51929

The traffic destined to a shared service provider endpoint group picks an incorrect class ID (PcTag) and gets dropped.

3.0(1k) and later

CSCuu09236

Traffic from an external Layer 3 network is allowed when configured as part of a vzAny (a collection of endpoint groups within a context) consumer.

3.0(1k) and later

CSCuu61998

Newly added microsegment EPG configurations must be removed before downgrading to a software release that does not support it.

3.0(1k) and later

CSCuu64219

Downgrading the fabric starting with the leaf switch will cause faults such as policy-deployment-failed with fault code F1371.

3.0(1k) and later

CSCuw81638

The OpenStack metadata feature cannot be used with Cisco ACI integration with the Juno release (or earlier) of OpenStack due to limitations with both OpenStack and Cisco’s ML2 driver.

3.0(1k) and later

CSCva32534

Creating or deleting a fabricSetupP policy results in an inconsistent state.

3.0(1k) and later

CSCva60439

After a pod is created and nodes are added in the pod, deleting the pod results in stale entries from the pod that are active in the fabric. This occurs because the Cisco APIC uses open source DHCP, which creates some resources that the Cisco APIC cannot delete when a pod is deleted.

3.0(1k) and later

CSCva86794

When a Cisco APIC cluster is upgrading, the Cisco APIC cluster might enter the minority status if there are any connectivity issues. In this case, user logins can fail until the majority of the Cisco APICs finish the upgrade and the cluster comes out of minority.

3.0(1k) and later

CSCva97082

When downgrading to a 2.0(1) release, the spines and its interfaces must be moved from infra L3out2 to infra L3out1. After infra L3out1 comes up, delete L3out2 and its related configuration, and then downgrade to a 2.0(1) release.

3.0(1k) and later

CSCvb39702

No fault gets raised upon using the same encapsulation VLAN in a copy device in tenant common, even though a fault should get raised.

3.0(1k) and later

■        In a multipod configuration, before you make any changes to a spine switch, ensure that there is at least one operationally “up” external link that is participating in the multipod topology. Failure to do so could bring down the multipod connectivity. For more information about multipod, see the Cisco Application Centric Infrastructure Fundamentals document and the Cisco APIC Getting Started Guide.

■        Some Cisco APIC GUI screens were modified and relocated in this release, and they are not accurately reflected in the corresponding GUI Online Help. The GUI Online Help pages will be fixed in the next major release.

Related Documentation

The Cisco Application Policy Infrastructure Controller (APIC) documentation can be accessed from the following website:

https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

The documentation includes installation, upgrade, configuration, programming, and troubleshooting guides, technical references, release notes, and knowledge base (KB) articles, as well as other documentation. KB articles provide information about a specific use case or a specific topic.

By using the “Choose a topic” and “Choose a document type” fields of the Cisco APIC documentation website, you can narrow down the displayed documentation list to make it easier to find the desired document.

The following links provide release notes and verified scalability documentation:

■        Verified Scalability

■        Cisco ACI Simulator Release Notes

■        Cisco NX-OS Release Notes for Cisco Nexus 9000 Series ACI-Mode Switches

■        Cisco Application Policy Infrastructure Controller OpenStack and Container Plugins Release Notes

■        Cisco Application Virtual Switch Release Notes

New Documentation

This section lists the new Cisco APIC product documents for this release.

■        Verified Scalability Guide for Cisco ACI

■        Cisco ACI Virtualization Guide

■        Cisco AVS Installation Guide

■        Cisco AVS Configuration Guide

■        Cisco APIC and Cisco ACI Multi-Site

■        Cisco APIC Basic Configuration Guide

■        Cisco Nexus 9364C ACI-Mode Switch Hardware Installation Guide

■        Cisco Nexus 9348GC-FXP ACI-Mode Switch Hardware Installation Guide

■        KB article: Cisco ACI Latency and Precision Time Protocol

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

© 2017-2022 Cisco Systems, Inc. All rights reserved.