Table of Contents
This document describes the features, caveats, and limitations for Cisco NX-OS software for use on the Cisco Nexus 7000 Series switches. Use this document in combination with documents listed in the “Related Documentation” section.
Note Release notes are sometimes updated with new information about restrictions and caveats. See the following website for the most recent version of the Cisco Nexus 7000 Series NX-OS Release Notes, Release 5.x Release Notes:
Table 1 shows the online change history for this document.
Added Cisco NX-OS Release 4.2(8) to Table 4 .
- Added Cisco NX-OS Release 5.1(5) to Table 4 .
- Added open caveat CSCtr79772.
Added a Note to the “General Upgrade/Downgrade Caveats” section.
Added SFP-10G-ER to N7K-F132XP-15 in Table 3 .
Removed NTP update-calendar and NTP clock-period from the “NTP Enhancements” section.
Moved the ISSU limitation to the “Upgrade/Downgrade Caveats” section and expanded the description.
Updated the transceiver information for the 8-port 10-Gigabit Ethernet I/O module XL (N7K-M108X2-12L) in Table 3 .
Added caveat CSCts11774 to the “Resolved Caveats—Cisco NX-OS Release 5.2(3a)”section.
Modified the description of a caveat for QoS MIB and MPLS QoS defaults in the “Specific Upgrade/Downgrade Caveats for Cisco NX-OS Release 5.2(x)” section.
Corrected the bug ID of CSCua48852 in the “Open Caveats—Cisco NX-OS Release 5.2” section.
Added the “Slow SNMP Responses” limitation.
Added a caveat about removing IP ARP synchronization prior to an ISSU to the “Upgrade/Downgrade Caveats” section.
Added a footnote to Table 4 related to an IPFIB Errors caveat in the “Specific Upgrade/Downgrade Caveats for Cisco NX-OS Release 5.2(x)” section.
Corrected the bug ID of CSCus42812 to CSCua42812 in the “Resolved Caveats—Cisco NX-OS Release 5.2(7)” section.
- Added caveat CSCud84750 to the “Resolved Caveats—Cisco NX-OS Release 5.2(5)” section.
- Added a caveat to the “Specific Upgrade/Downgrade Caveats for Cisco NX-OS Release 5.2(x)” section related to upgrading from Cisco NX-OS Release 4.2(6) to Release 5.2(4).
Updated Table 4 .
Updated the Aggressive Failure Detection Timers caveat in the “Upgrade/Downgrade Caveats” section.
Added the Increased TCAM Usage for Handling Fragmented Packets in QoS ACL Entries caveat to the “Upgrade/Downgrade Caveats” section.
Added LISP caveat to the “Upgrade/Downgrade Caveats” section.
The Cisco NX-OS software for the Cisco Nexus 7000 Series switches fulfills the routing, switching, and storage networking requirements of data centers and provides an Extensible Markup Language (XML) interface and a command-line interface (CLI) similar to Cisco IOS software.
The Cisco NX-OS software supports the Cisco Nexus 7000 Series chassis. You can find detailed information about supported hardware in the Cisco Nexus 7000 Series Hardware Installation and Reference Guide .
Note The information in this section applies only if you have a Cisco Nexus 7000 Series system with a Supervisor 1 module with 4 GB of memory. If your system has a Supervisor 1 with 8 GB of memory, you do not need the information in this section because a memory upgrade is not needed.
An 8 GB supervisor memory upgrade kit, N7K-SUP1-8GBUPG=, allows for growth in the features and capabilities that can be delivered in existing Cisco Nexus 7000 Series supervisor modules. The memory upgrade kit is supported on Cisco Nexus 7000 Series systems running Cisco NX-OS Release 5.1 or later releases. Instructions for upgrading to the new memory are available in the “Upgrading Memory for Supervisor Modules” section of the Cisco Nexus 7000 Series Hardware Installation and Reference Guide.
- When the system memory usage exceeds 3 GB (75 percent of total memory), we recommend that you upgrade the memory to 8 GB. Use the show system resources command from any VDC context to check the system memory usage:
- If you create more than one VDC with XL mode enabled, or if you have more than two VDCs, 8 GB of memory is required.
For additional guidance about whether or not to upgrade a supervisor module to 8 GB of memory, see Figure 1.
When you insert a supervisor module into a Cisco Nexus 7000 Series switch running Cisco NX-OS Release 5.1(x) or a later release, be aware that one of the following syslog messages will display, depending on the software version and the amount of memory for the supervisor module:
- If you are running Cisco NX-OS Release 5.1(1) or a later release and you have an 8-GB supervisor as the active supervisor and you insert a 4-GB supervisor module as the standby, it will be powered down. A severity 2 syslog message indicates that the memory amounts should be equivalent between the active and the standby supervisor:2010 Dec 3 00:05:37 switch %$ VDC-1 %$ %SYSMGR-2-SUP_POWERDOWN: Supervisor in slot 10 is running with less memory than active supervisor in slot 9In this situation, you have the option to upgrade the memory in the 4-GB supervisor or shut down the system and remove the extra memory from the 8-GB supervisor.
- If you are running Cisco NX-OS Release 5.1(2) or a later release and you insert a 8-GB supervisor module as the standby, a severity 4 syslog message appears.2010 Dec 1 23:32:08 switch %SYSMGR-4-ACTIVE_LOWER_MEM_THAN_STANDBY: Active supervisor in slot 5 is running with less memory than standby supervisor in slot 6.
Table 2 shows the hardware supported by Cisco NX-OS Release 5.x and Cisco NX-OS Release 4.x software.
Table 3 shows the transceiver devices supported by each release.
For a list of minimum recommended Cisco NX-OS software releases for use with Cisco Nexus 7000 Series switches, see the document Minimum Recommended Cisco NX-OS Releases for Cisco Nexus 7000 Series Switches.
8-port 10-Gigabit Ethernet I/O module XL1
32-port 10-Gigabit Ethernet SFP+ I/O module XL 1
48-port 1-Gigabit Ethernet I/O module XL 1
48-port 10/100/1000 Ethernet I/O module XL 1
Cisco Nexus 2248TP Fabric Extender2
Cisco Nexus 2224TP Fabric Extender 2
Cisco Nexus 2232PP Fabric Extender 2
1.Requires the Cisco Nexus 7010 Scalable Feature Package license (N7K-C7010-XL) or the Cisco Nexus 7018 Scalable Feature Package license (N7K-C7018-XL), depending on the chassis, to enable all XL-capable I/O modules to operate in XL mode.
Note Before you upgrade or downgrade your Cisco NX-OS software, we recommend that you read the complete list of caveats in this section to understand how an upgrade or downgrade might affect your network, depending on the features that you have configured.
See Table 4 for the nondisruptive upgrade (ISSU) path to and nondisruptive downgrade (ISSD) path from Cisco NX-OS Release 5.2(9a). Releases that are not listed for a particular release train do not support a direct ISSU or ISSD to the current release.
5.2(1), 5.2(3a), 5.2(4), 5.2(5)6, 5.2(7)
5.2(1), 5.2(3a), 5.2(4), 5.2(5) 1
6.Before performing an ISSU to Cisco NX-OS Release 5.2(7) or a later release, see the IPFIB Errors caveat in this section.
2. Perform a second nondisruptive upgrade (or downgrade) to the current release.
For example, to upgrade from Release 4.2(3) to Release 5.2(x), you can perform an ISSU from Release 4.2(3) to Release 4.2(6), and then perform and ISSU from Release 4.2(6) to Release 5.2(x).
If you have LISP configured on a Cisco Nexus 7000 Series device, you must remove the configuration before an ISSU. Enter the no lisp feature command to individually unconfigure the LISP commands. Then enter the no feature lisp command. After the ISSU completes, enter the feature lisp command to reenable LISP and then reconfigure it
When you upgrade Cisco NX-OS software by changing boot variables and reloading the device, make sure to save the FEX HIF configuration to the startup configuration, as well as another location (such as bootflash or an external server). Once the upgrade to a new release is complete, and the FEX is fully online and associated, reapply the FEX HIF configuration.
- Cisco NX-OS Release 5.2(1) includes new mandatory configuration parameters for OTV. An ISSU to Release 5.2(1) will result in interruptions of the OTV service. In addition, be aware of the following points related to ISSU:
– If any overlay interface is in the no-shutdown state (up), the ISSU pre-upgrade stage cannot complete. All overlay interfaces must be in the shutdown state before the ISSU can successfully complete.
Recommendations on the best procedure to minimize the impact of ISSU on the OTV service can be found in the Cisco Nexus 7000 Series NX-OS OTV Configuration Guide . Closely follow this procedure when upgrading an existing OTV deployment.
- When you downgrade from Cisco NX-OS Release 5.2(x) to an earlier release such as Cisco NX-OS Release 4.2(1), you might see messages like the following:
- Before you attempt a downgrade from Cisco NX-OS Release 5.2(x) to any release prior to Release 5.2(1), you should clear the QoS MIB and MPLS QoS defaults using the clear qos mpls-snmp command. Enter these commands after the switch configuration has been erased and it has been reloaded. The downgrade might result in a continuous failure if the defaults are not cleared.
- Before you downgrade from Cisco NX-OS Release 5.2(x) or 5.1(x) to Cisco NX-OS Release 5.0(x) or an earlier release, remove all system QoS and QoS policies configured on F1-series modules. Use the clear qos policies command to remove the defaults for F1-series modules. An internal process failure can result if the QoS policies are not removed prior to the downgrade.
- ISSU, stateful switchover (SSO), and graceful restart are not supported when aggressive failure detection timers are used for any Layer 3 protocols. Starting in Cisco NX-OS Release 5.2(3a), the First Hop Redundancy Protocol (FHRP) with aggressive timers has been validated for SSO or ISSU using the extended hold timer feature. Other protocols such as OSPF have been validated with aggressive timers without SSO or ISSU support starting in Cisco NX-OS Release 5.2(1). For additional information on aggressive timer support and extended hold timers for FHRP, see the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide and the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide.
- Cisco NX-OS Release 5.2(1) extends the reserved VLAN range from 3968 to 4095 and makes it configurable. Previously, in releases prior to Cisco NX-OS Release 5.2(1), the reserved VLAN range was 3968 to 4048, and 4094, and it was not configurable. See the “Configurable Reserved VLAN Range” section for more information about this new feature.
Once you upgrade to Cisco NX-OS Release 5.2(1), user-defined VLANs might fall within the new reserved range. If that occurs, then the new reserved range will not take effect and the features that need the additional reserved VLANs will be impacted.
To address this situation, you can either migrate the affected user-defined VLAN before or after the upgrade, or you can modify the new VLAN range after the upgrade. See the Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide.
Caution Once you modify the VLAN range, an ISSD to a lower release will overwrite your configuration. Because of this, we recommend that you save a copy of your switch configuration to a separate file before you start an ISSU to Cisco NX-OS Release 5.2(1) so that you restore the configuration if necessary.
If you perform an ISSU to Cisco NX-OS Release 5.2(1) and you modify the new configurable reserved VLAN range, an ISSD to a lower version requires a reboot to restore the previous reserved VLAN range of 3968 to 4048, and 4094.
If you perform an ISSU to Cisco NX-OS Release 5.2(1) and you do not modify the new configurable reserved VLAN range of 3968 to 4095, then you can perform an ISSD to a lower version and your configuration is preserved.
- BFD for static routes does not support a stateful switchover (SSO) or an ISSU. When you perform an ISSU or an SSO, a small amount of packet loss can result in flows that follow static routes that are protected by BFD.
- The ACL resource allocation scheme was changed in Cisco NX-OS Release 5.1(x) to provide BFD improved interoperability with other features that use ACLs. Because of this change, you should disable BFD prior to a software upgrade from any Cisco NX-OS Release 5.0(x) to any Cisco NX-OS Release 5.1(x) or Release 5.2(x). Likewise, you should disable BFD before a downgrade from any Cisco NX-OS Release 5.2(x) or Release 5.1(x) to any Cisco NX-OS Release 5.0(x).
- Before you perform an ISSU from a Cisco NX-OS Release 5.2(x) earlier than Release 5.2(7) to Release 6.x or perform an ISSU or ISSD between any two Cisco NX-OS 6.x releases, you must first remove QoS policies and ACLs from interfaces that are in the down state. If this action is not performed, the installer process will abort the upgrade or downgrade process, and a message similar to the following will be displayed:Service "ipqosmgr" : Please remove inactive policies using the command "clear inactive-config qos” Pre-upgrade check failed. Return code 0x415E0055 (Need to clear inactive-if-config from qos manager using the command "conf;clear inactive-config qos" or can manually clear the config shown by the command: "show running-config ipqos inactive-if-config").
Guidelines for manual policy removal: during a manual removal, when the interface is part of a port channel, remove the policy map or access list from the port channel or remove the interface from the port channel before performing the ISSU or ISSD. For all other interface types, remove the policy map or access list from the interface.
- If you downgrade a Cisco Nexus 7000 Series device from Cisco NX-OS Release 5.2(x) or Release 5.1(x) to Cisco NX-OS Release 5.0(x) or Release 4.2(x), AAA configuration commands might fail. The workaround is to write-erase the startup configuration and reboot the device.
- A nondisruptive software upgrade or downgrade is not supported when vPC peers are on a single physical switch, but they run across VDCs.
- If you have IP ARP synchronization configured in a vPC, you should remove the configuration prior to a nondisruptive software upgrade from Cisco NX-OS Release 4.2(6) or Release 4.2(8) to Cisco NX-OS Release 5.2(x). You can reapply the configuration after the ISSU completes. Follow these steps:
Note The Transport Services Package license is required to enable LISP. If you do not have this license, you can enable the grace period for it. If you cannot enable the grace period, perform an ISSU and reload the affected modules.
- When you perform an ISSU from Cisco NX-OS Release 4.2(x) to Release 5.2(4) or an earlier 5.2(x) release, you might see the symptom described in CSCud84750, which is listed in the “Resolved Caveats—Cisco NX-OS Release 5.2(5)” section section.
- Due to an optimization in handling of fragmented packets in QoS ACL entries in Cisco NX-OS Release 5.2(9), Release 6.1(3), and later releases, TCAM usage might increase once the system is reloaded with the new software release. Once the new version boots, any ACL entry that references Layer 4 information will use an extra TCAM entry so that it can match on fragmented packets and that will cause TCAM usage to increase. This increase is not seen during an ISSU upgrade, until the system or module is reloaded at some point after the ISSU upgrade is complete.
When a Cisco Nexus 7000 Series switch is reloaded and the version of the image on the switch changes, the binary configuration is always removed and the ASCII configuration is applied. When this occurs, VLAN Trunking Protocol (VTP) restores the default configuration to the switch.
Cisco NX-OS Release 5.2(1) includes a new image for the connectivity management processor (CMP). The CMP is upgraded to Release 5.2(1) on successful ISSU of Cisco NX-OS to Release 5.2(1). When the ISSU completes, you should reload the CMP image on the active and standby supervisor modules. For additional information, see the Cisco Nexus 7000 Series NX-OS Software Upgrade and Downgrade Guide, Release 5.x .
For additional information about the CMP, see the Cisco Nexus 7000 Series Connectivity Management Processor Configuration Guide .
In conjunction with Cisco NX-OS Release 5.2(1), a new EPLD package is introduced. Certain features in Cisco NX-OS Release 5.2(1) may require an upgrade to the new EPLD images. LISP, for example, requires a specific EPLD version on the 32-port 10-Gigabit Ethernet SFP+ I/O module (N7K-M132XP-12) and the 32-port 10-Gigabit Ethernet SFP+ I/O module XL (N7K-M132XP-12L). MPLS does not require an EPLD upgrade.
To determine if you need to upgrade the EPLD images on your Cisco Nexus 7000 Series switch, see the Cisco Nexus 7000 Series FPGA/EPLD Upgrade Release Notes, Release 5.2.
Cisco Data Center Network Manager (DCNM) Release 5.2(1) supports Cisco NX-OS 5 Release 5.2(1) and Release 5.2(3a). See the Cisco DCNM Release Compatibility Matrix for specific information about the Cisco Nexus platforms and software release versions that Cisco DCNM supports.
Cisco NX-OS Release 5.2 supports the new Cisco Nexus 7009 chassis (N7K-7009) and new fabric module (N7K-7009-FAB-2) for the Cisco Nexus 7009 system. The Cisco Nexus 7009 chassis has 9 slots that allow for two supervisor modules and up to seven I/O modules. The chassis also holds up to five fabric modules, one fan tray, up to two power supply units, and a cable management system. For additional information about the Cisco Nexus 7009 system, see the Cisco Nexus 7000 Series Hardware Installation and Reference Guide.
This section briefly describes the new features introduced in Cisco NX-OS Release 5.2 for the Cisco Nexus 7000 Series switches. For detailed information about the features listed, see the documents listed in the “Related Documentation” section. The “New and Changed Information” section in each of these books provides a detailed list of all new features and includes links to the feature description or new command.
Some new features require a new license. See the “Licensing” section for additional information. For complete information about the licenses required for Cisco NX-OS features, see the Cisco NX-OS Licensing Guide.
- Cisco NX-OS Release 5.2(9)
- Cisco NX-OS Release 5.2(7)
- Cisco NX-OS Release 5.2(5)
- Cisco NX-OS Release 5.2(4)
- Cisco NX-OS Release 5.2(3a)
- Cisco NX-OS Release 5.2(1)
- FCoE (Fiber Channel over Ethernet)
- OTV Features
- FEX Features
- IEEE 1588v2 PTP Support
- ACL Capture
- ACLs Enhancements
- BFD SHA-1 Authentication
- BFD Support for VRRP
- BGP Local-AS
- BGP Prefix Independent Convergence Core
- CFS Enhancement
- Cisco TrustSec Enhancement
- Configurable Reserved VLAN Range
- CoPP Enhancements
- EEM Correlation
- EIGRP Wide Metrics
- Graceful vPC Type-1 Check Handling
- HTTP Proxy Server for Smart Call Home
- Multicast over GRE
- NetFlow Enhancement
- NTP Enhancements
- Parallel Upgrade of EPLD Images
- Parallel Upgrade of I/O Modules
- Password Encryption
- Smart DHCP Relay
- SPAN and ERSPAN Enhancements
- Static Multicast MAC
- System Message Logging
- Subnet Broadcast Support for the DHCP Relay Agent
- Unique MAC Address per VDC
- vPC Autorecovery
- XML Infrastructure Enhancements
The Locator/ID Separation Protocol (LISP) is a new routing architecture designed for Internet scale and global reach across organizations. Cisco NX-OS Release 5.2(1) introduces LISP VM mobility which is designed to enable global IP endpoint mobility across private networks and the Internet.
LISP functionality requires the use of the 32-port 10-Gigabit Ethernet SFP+ I/O module (N7K-M132XP-12) or the 32-port 10-Gigabit Ethernet SFP+ I/O module XL (N7K-M132XP-12L). These modules can be used independently or combined with F1 series modules in proxy mode to deliver LISP functionality in a Cisco Nexus 7000 Series switch. Traffic received on other M-series modules will not be processed by LISP because they cannot operate in proxy mode.
For additional information about LISP, see the Cisco Nexus 7000 Series NX-OS LISP Configuration Guide .
MPLS requires a new license as described in the “Licensing” section.
For additional information about MPLS, see the Cisco Nexus 7000 Series MPLS Configuration Guide .
MPLS forwarding is based on label switching. Labels are allocated based on per-prefix or per-VRF. LDP enables the exchange of labels and IGP prefix bindings. Per-Prefix and Per-VRF bindings are supported.
MPLS traffic engineering allows you to create paths in the network to efficiently use the network fabric and bandwidth. MPLS TE FRR supports restoration of a TE path in 50 ms or less. Link, node, path and bandwidth protection mechanisms are supported. Cisco Nexus 7000 Series XL linecards are required to achieve 50 ms convergence for MPLS TE FRR.
QoS mechanisms such as policing, marking and matching are available for MPLS labeled packets. Differentiated services models such as pipe, short-pipe, and uniform modes allow control of classification and remarking of traffic, which can be applied to applications that require tight service-level agreement (SLA) controls.
A multicast VPN is an IP VPN service that supports the transmission of IP multicast packets between sites. Cisco NX-OS Release 5.2(1) implements the Internet Draft, draft-rosen-vpn-mcast-10.txt, “Multicast in MPLS/BGP IP VPNs.” This multicast VPN service is an overlay to BGP or MPLS IP VPNs. The signaling specified is Protocol Independent Multicast (PIM) and the traffic encapsulation is Generic Routing Encapsulation (GRE).
The ability to export or import routes between VPNs, based on VPN route target communities as part of BGP extended communities, is available in Cisco NX-OS Release 5.2(1) for VRF-lite and MPLS Layer 3 VPNs. Both AS and IP address route targets are supported. An MPLS license is not required to export or import routes between VPNs with VRF-lite.
FCoE support is added for the 32-port 1/10 Gigabit Ethernet module (F1-Series) module (N7K-F132XP-15) in the Cisco Nexus 7000 Series chassis. FCoE can now be deployed in director class, highly available, modular platforms for the access and core of converged networks. To support FCoE hosts and targets, VE port support allows for FCoE ISLs, which help create scalable, multihop FCoE topologies. FCoE traffic within a Cisco Nexus 7000 Series switch can be segmented using a dedicated storage VDC.
To run FCoE on a Cisco Nexus 7000 Series device, you must create a separate storage VDC. Only one of the VDCs can be a storage VDC, and the default VDC cannot be configured as a storage VDC. The storage VDC enables isolation, security, and ease of management of FCoE traffic. An FCoE license (N7K-FCOEF132XP) is required to create the storage VDC. See the “Licensing” section.t
For additional information about the storage VDC, see the Cisco Nexus 7000 Series NX-OS Virtual Device Context Configuration Guide.
You can configure shared interfaces that carry both Ethernet and Fibre Channel traffic. In this specific case, the same interface belongs to more than one VDC. The shared interface is allocated to both an Ethernet VDC and a storage VDC. For additional information about FCoE and shared interfaces, see the Cisco NX-OS FCoE Configuration Guide .
Cisco Nexus 7000 Series FCoE converged networks can be seamlessly bridged to Cisco MDS 9500 switches with the introduction of the Cisco MDS 9000 8-port 10-Gbps Fibre Channel over Ethernet (FCoE) Module (DS-X9708-K9). For additional information about the FCoE module, see the Cisco MDS 9500 Series Hardware Installation Guide.
FCoE requires a new license as described in the “Licensing” section.
There are several new OTV features in Cisco NX-OS Release 5.2(1) which are briefly described in this section. For additional information, see the Cisco Nexus 7000 Series NX-OS OTV Configuration Guide.
The OTV adjacency server feature enables unicast based OTV deployment in environments in which the IP core does not support IP multicast. In an OTV environment, the edge devices build a relationship with each other from a control-plane perspective. The neighbor relationship can be built over both multicast-enabled and unicast-only transport infrastructure.
Additional checks have been added to OTV to prevent accidental misconfiguration that might lead to problems. This functionality introduces a new mandatory command in OTV. Since this command is introduced in NX-OS Release 5.2(1), an ISSU from previous versions of NX-OS will result in a disruption of the OTV service. Refer to the“Upgrade/Downgrade Caveats” section for more information.
The host vPC with FEX feature provides the ability to have a vPC from a host connected to two independent Cisco Nexus 2000 Series Fabric Extenders with a Cisco Nexus 7000 Series switch that acts as a parent switch to the FEX. The two Cisco Nexus 7000 Series switches that act as the parent switch form the vPC peers. The connectivity between the FEX and Cisco Nexus 7000 Series switch cannot be a vPC. It can be a link or a port channel.
Precision Time Protocol (PTP) is based on IEEE 1588v2, and it is implemented on F1-series modules. The implementation supports Boundary Clock for network synchronization, and includes support for multiple slaves. The precision provided by the implementation is approximately less than 50 ns.
PONG is the ability to do a traceroute based on the MAC addresses of the destination endpoint, and to provide a latency and connectivity check, using IEEE1588v2 for latency measurement. PONG can be enable with the Enhanced Layer 2 Package (N7K-EL21K9) license.
ACL capture provides a mechanism to selectively monitor traffic on all types of interfaces per VLAN. It allows the user to enable capture for a specific ACL rule. Packets that match an ACL rule with a capture option, are either forwarded or dropped based on a permit or deny action and also copied to an alternate destination port for further analysis.
This feature provides the capability to add to or change the values prepended onto the AS_PATH attribute on routes to or from the configured eBGP neighbor. Having this capability simplifies the process of AS migration by not disrupting existing peering arrangements by allowing the router to appear to external peers as a member of another autonomous system.
Cisco Release NX-OS 5.2(1) introduces BGP Prefix Independent Convergence (PIC) Core. This feature allows for faster convergence for traffic destined to BGP prefixes that share the same remote next hop in case of a failure in the core of the network. Both MPLS and pure IP traffic can benefit from BGP PIC Core. It is enabled by default and can not be disabled.
Added support for pause frame encryption and decryption on interfaces. Pause frames are MAC control frames used for Ethernet flow control. The ports on some line cards encrypt and decrypt pause frames while the ports on other line cards do not have this ability. This disparity causes interoperability issues and causes the ports to discard or ignore the pause frames. Beginning with Cisco NX-OS Release 5.2, you can configure if the pause frames are to be encrypted or clear on individual interfaces. If two ports are connected to form a CTS link and one is clear pause capable and the other is secure (encryption/decryption) pause capable, the pause frames must be sent in the clear across the link in order for them to be correctly sent and received.
On Cisco Nexus 7000 Series switches, certain VLANs are reserved for internal use. These VLAN numbers occasionally conflict with the network VLANs that customers assign. In Cisco NX-OS Release 5.2(1), the new system vlan start-vlan range command allows you to reassign the internal VLANs to a different value. In addition, the range of reserved VLANs is extended to 128.
Note Before upgrading to Cisco NX-OS 5.2(1), review the“Upgrade/Downgrade Caveats” section to understand the impact of the configurable reserved VLAN feature on a non-disruptive downgrade.
- Added the ability to change or reapply the default CoPP policy without rerunning the setup utility.
- Changed the CoPP best practice policy to read-only and added the ability to copy the policy in order to modify it.
- Added the show copp profile and show copp diff profile commands to display the details of the CoPP best practice policy and the differences between policies, respectively.
- Changed the show copp status command to display which flavor of the CoPP best practice policy is attached to the control plane.
- Changed the name of the none option for the best practices CoPP profile in the setup utility to skip.
- Updated the default class maps with support for MPLS LDP, MPLS OAM, MPLS RSVP, DHCP relay, and OTV-AS.
EIGRP wide metrics can accommodate interfaces faster than 1 Gigabit Ethernet, while computing the metric to be installed in the RIB or FIB. This feature allows EIGRP to perform meaningful path selection when high-speed links are involved.
Changing a type-1 parameter such as STP mode or MTU on one of the vPC port channels can cause a consistency check failure. As a result, the vPC is set to a down state, as is the associated vPC on the other peer device, and traffic for this particular vPC is blackholed. The graceful vPC type-1 check can avert a failure and preserve the network redundancy by keeping up the vPC member ports on a primary peer device. The graceful vPC type-1 check is applicable for the global type-1 parameter and the vPC level type-1 parameter.
This features allows you to upgrade Cisco NX-OS on I/O modules in parallel, instead of sequentially, which is the current model. Parallel upgrades allows control of how many modules can be upgraded at one time. This feature can greatly reduce the time to upgrade the I/O modules and help reduce the maintenance window at customer sites.
The Advanced Encryption Standard (AES) password encryption feature stores all existing and newly created clear-text passwords for supported applications (currently RADIUS and TACACS+) in the strong and reversible type-6 encrypted format. A master encryption key is used to encrypt and decrypt the passwords. You can also use this feature to convert all existing weakly encrypted passwords to type-6 encrypted passwords.
As of today when DHCP relay agent receives broadcast DHCP request packet from a host, it fills the primary address of the inbound interface and forwards to the server, which allocates IP addresses from the subnet pool until the pool is exhausted and ignores further requests. This may not work if the number of hosts is more than the number of IP addresses in the pool or if there are multiple subnets configured on an interface using secondary addresses. The relay functionality is enhanced so that the relay agent fills relay agent address of DHCP request packet with one of the secondary address and forward to the server in case IP addresses are exhausted in primary address subnet pool. The server allocates IP address in the secondary IP address subnet pool.
- Added SPAN and ERSPAN source support for Cisco Nexus 2000 Series Fabric Extender interfaces.
- MTU Truncation (Applies only to SPAN, not to ERSPAN) – To reduce the SPAN traffic bandwidth, you can configure the maximum bytes allowed for each replicated packet in a SPAN session.
- Source Rate Limit (Applies only to SPAN, not to ERSPAN) - When a SPAN session is configured with multiple interfaces or VLANs as the sources in a high-traffic environment, the destination port can be overloaded, causing the normal data traffic to be disrupted at the source port. You can alleviate this problem as well as traffic overload on the source forwarding instance by configuring a source rate limit for each SPAN session.
- Multicast Best Effort Mode - You can configure the multicast best effort mode for any SPAN or ERSPAN session. By default, SPAN/ERSPAN replication occurs on both the ingress and egress line card. When you enable the multicast best effort mode, SPAN/ERSPAN replication occurs only on the ingress line card for multicast traffic or on the egress line card for packets egressing out of Layer 3 interfaces (that is, on the egress line card, packets egressing out of Layer 2 interfaces are not replicated for SPAN/ERSPAN).
Currently on the Cisco Nexus 7000 Series platform, Layer 2 multicast table lookup is performed on the destination IP address instead of the destination MAC address. This type of lookup does not work for all network applications. Some applications share a single unicast cluster IP address and multicast cluster MAC address. Traffic destined for the unicast cluster IP address is forwarded by the last-hop router with the shared multicast MAC address. Forwarding is accomplished by assigning a static Multicast MAC address for the destination IP address of the end host or cluster.
You can configure the device to support the relaying of DHCP packets from clients to a subnet broadcast IP address. When this feature is enabled, the VLAN ACLs (VACLs) accept IP broadcast packets and all subnet broadcast (primary subnet broadcast as well as secondary subnet broadcast) packets.
VDCs currently point to a common MAC address that is shared as the source from a management perspective. With the new unique MAC address per VDC feature, customers can now manage or view a VDC as a unique device because each VDC will have a unique MAC address as an identifier.
Currently when a vPC peer-link goes down, a secondary switch takes down all its vPCs if it finds a peer-keep alive is working. If the peer-link does not recover, and the primary switch goes down and is unable to forward any traffic, then the access switches are disconnected.
For additional information about the licenses mentioned is this section, see the Cisco NX-OS Licensing Guide.
FCoE on the Cisco Nexus 7000 Series is licensed per module. One Cisco Nexus 7000 F1 FCoE License (N7K-FCOEF132XP) is required for each Cisco Nexus 32-port 1/10 Gigabit Ethernet module (N7K-F132XP-15) that runs the FCoE features.
- BFD MIB
- LSR MIB
- TE MIB
- PIM MIB
- MIB for TCP (RFC 4022)
- IP-MIB (RFC2011)
- Etherlike MIB (RFC1650)
- CISCO-PKI-PARTICIPATION MIB Enhancements
- Role-Based Access Control
- EIGRP Routes
- Standby Supervisor Can Reset With Feature-Set Operation
- NTP Servers Created with Cisco DCNM-SAN Are Not Listed for the Storage VDC
- GOLD Snake Loopback Test Disabled on F1 Series Modules
- Slow SNMP Responses
- Beginning with Cisco NX-OS Release 5.2, you can configure role-based access control (RBAC) in the Cisco Nexus 7000 storage VDC using Cisco NX-OS CLI commands. You cannot configure RBAC in the Cisco Nexus 7000 storage VDC using Cisco DCNM. Note that RBAC in the storage VDC is RBAC for the Cisco Nexus 7000 Series switches, which is different from that for the Cisco MDS 9500 Series switches.
- RBAC CLI scripts used in Cisco MDS 9500 Series switches cannot be applied to the storage VDC configured for a Cisco Nexus 7000 Series switch.
- You cannot distribute the RBAC configuration between a Cisco MDS 9500 Series switch and the storage VDC configured for a Cisco Nexus 7000 Series switch. To prevent this distribution, make sure to assign RBAC in Cisco MDS and the Cisco Nexus 7000 storage VDC to different CFS regions.
Due to a semantic difference between Cisco NX-OS and Cisco IOS software, EIGRP routes that are installed in the routing information base (RIB) are marked with the incorrect process number. When the EIGRP process tag is a number and an AS number is defined under that EIGRP process, the routes in RIB are installed with the process tag and not the AS number.
The standby supervisor might reload when a feature-set operation (install, uninstall, enable, or disable) is performed, if the HA state of the standby supervisor is not “HA standby” at the time of the feature-set operation. To prevent the reload, ensure that the state of the standby supervisor is “HA standby.” To check the HA state for the specific VDC where the feature-set operation is performed, enter the show system redundancy ha status command on the active supervisor.
In addition, if you perform a feature-set operation while modules are in the process of coming up, then those modules will be power cycled. Modules that are up and in the “ok” state are not power cycled when you perform a feature set operation.
If you use Cisco DCNM-SAN to create NTP servers for the Storage VDC, they are not listed for the Storage VDC. The reason is that the Storage VDC is not configured to control the clock and the clock manager cannot provide that information through SNMP.
- Open Caveats—Cisco NX-OS Release 5.2
- Resolved Caveats—Cisco NX-OS Release 5.2(9a)
- Resolved Caveats—Cisco NX-OS Release 5.2(9)
- Resolved Caveats—Cisco NX-OS Release 5.2(7)
- Resolved Caveats—Cisco NX-OS Release 5.2(5)
- Resolved Caveats—Cisco NX-OS Release 5.2(4)
- Resolved Caveats—Cisco NX-OS Release 5.2(3a)
- Resolved Caveats—Cisco NX-OS Release 5.2(1)
Note Release note information is sometimes updated after the product Release Notes document is published. Use the Cisco Bug Toolkit to see the most up-to-date release note information for any caveat listed in this document.
Conditions : This symptom might be seen when Bank Chaining (Hardware Resource Pooling) is enabled and a WCCP configuration is applied after a RACL configuration. This issue might result in a SBADDFAIL syslog that indicates an unsupported feature combination. The WCCP configuration on the interface is not removed when the error occurs and the WCCP redirect is not programmed in the TCAM.
Workaround : Remove the WCCP redirect from the interface. When this operation is done, the SBDELFAIL syslog will appear. Ignore the syslog message and remove the RACL configuration from the interface and reapply the WCCP redirect on the interface. TCAM programming should go through.
Conditions : This symptom might be seen if a failure is encountered with one of the crossbar ASICs. GOLD can incorrectly report the failed module or might be unable to isolate the exact module. For example, the Cisco Nexus 7000 Series switch active supervisor engine might report RewriteEngineLoopback or PortLoopback (or some other) test failed for all (or several) ports in all (or several) modules present in the switch.
Symptom : The subswitch ID for a vPC on the secondary switch is incorrectly programmed in the hardware as 1 (reserved) even though it has the correct SSID, as can be seen in the output of the show vpc brief command.
– Configure a vPC port channel on a secondary switch (for example, vPC 1 and port channel 1) and make sure that from the access switch's perspective (that is, port channel 1), only the links going to the secondary switch are up. (If the port channel 1 links from the access switch to primary switch are also up, then this problem will not occur.)
Conditions : This symptom might be seen when there is a heavy BFD and ACL Manager interaction, with many sessions going up or down, and the ACL manager process on the supervisor module can get busy processing BFD-related ACL requests. At the same time, if one or more port-channel members are trying to come up, they fail to be part of that port channel and potentially leave them in a suspended state on the local and remote end.
Conditions : This symptom might be seen on the 32-port 1/10 Gigabit Ethernet module when an atomic update is configured and policies which need slightly less than 512 TCAM entries are rejected with an atomic failure.
Conditions : This symptom might be seen when OSPFv3 forms an IPv6 neighbor, even though the local address is a duplicate in the network. This can result in a black hole of traffic to the local IPv6 address.
Conditions : This symptom might be seen in an MPLS environment when an alternative IGP path is available and traffic might switch from the old path to a new path if the newer path’s cost is better. During this switchover, there can be a traffic outage of a few 100 ms.
Symptom : When a dynamic Endpoint Identifier (EID) moves away and is discovered by a remote XTR, the old XTR will receive an SMR that indicates that the dynamic EID has moved away. In response, the old XTR installs a /32 (host) Null0 route for the dynamic EID. Installing /32 (host) Null0 makes sense in case of asm, but it should not be installed in the esm.
Conditions : This symptom might be seen every time the dynamic EID moves from one XTR to the other XTR. The only negative side is that the old XTR cannot reach the dynamic EID even though it is on the same (extended) subnet. All other hosts on the subnet are able to reach the dynamic EID, and the XTR will rarely need to reach the dynamic EID.
Conditions : This symptom might be seen when an EPLD upgrade is performed on the standby supervisor. As part of the EPLD upgrade, the standby supervisor is reloaded. The syslog message from the pixm service is a side-effect of the standby supervisor reload.
Workaround : Before performing an ISSD from Cisco NX-OS Release 5.2(1) to Cisco NX-OS Release 5.1(x), remove “match protocol mpls router-alert” from the referring class map and add it back to the same class map after the ISSD completes.
Symptom : When bundled CTS links into a Layer 3 port channel with a Catalyst 3000 switch, the interface(s) are reauthenticated every 30 seconds which causes the port channel to go up and down and eventually go into a suspended state. If the port channel is removed from the configuration, the CTS links stay up.
Symptom : Some conditional features such as OSPF and BGP register a MIB with SNMP and receive an error message due to a timeout issue. Because of the timeout, the response received later might be treated as an unknown MTS message by such conditional features.
Symptom : When you change an MTU on a main interface, the subinterface inherits this MTU as per the show interface ethernet command even though internally, the MTU for the subinterface is still set to default.
Workaround : Enter the shut command on the FabricPath member port and ensure that it is not a member of an outgoing flood list before adding it to a port channel. Enter the show l2 mroute flood vlan vlan-id command to verify that the member port is not a part of the flood outgoing interface list.%KERN-2-SYSTEM_MSG: mts_is_q_space_available_new():1416:Total mtsbuf size 10077904 for sap 3356, exceeds limit 15 perc%KERN-2-SYSTEM_MSG: mts_do_msg_input() failing since no space available in 28 (src_sap = 28, opc = 1355) - kernel%SYSMGR-3-SERVICE_TERMINATED: Service \snmpd\" (PID 23156) has finished with error code SYSMGR_EXITCODE_SYSERR (1)."
Conditions : This symptom might be seen when a route-map on a WCCP interface is removed and another WCCP service is enabled. TCAM programming only has the entry for the newly enabled WCCP service. The existing WCCP service does not have the corresponding TCAM entry.
Conditions : This symptom might be seen in very rare circumstances if the system encounters multiple stress conditions including prolonged high CPU utilization and a down vPC peer link. If at that point a supervisor switchover is initiated, VLANs would incorrectly be suspended on the vPC peer link.
Condition : This symptom might be seen when an OTV VDC is back-to-back connected to a FabricPath VDC by VPC+ channels. Both VDCs must reside on the same device. This only happens in case of VPC+ channels. This issue affects all releases prior Cisco NX-OS Release 6.1(4a).
Workaround : Once the CTS process shows low CPU utilization after the programming, issue the appropriate CLI commands for static policies. Enter the cts refresh role-based-policy command for dynamic policies.
– Currently ACLs are evaluated in order of peer, server, serve-only, query-only and all others are denied. If a packet does not match one category, such as peer, it is not forwarded further to see if it may match server, serve-only, or query-only mode and it is dropped.
Conditions : This symptom might be seen in a Cisco Nexus 7000 Series switch with both M1 and F1 Series modules, and a FabricPath VLAN with atomic update enabled. If SVI with DHCP is configured on the system and if the SVI is shut down and then brought back up, a DHCP relay issue can result.
Conditions : This symptom might be seen when there are continuous new learning or MAC address moves occurring in the system at the time of the system switchover. The following MTM debug messages appear on the module: mtm_mts_send(92): MTS: send failed for MTS_OPC_L2FM_NL_MV_UPD_RD_MSG_V2 size 50 on q 8 errno 110 [Connection timed out]
Conditions : This symptom might be seen when the redirect-list is attached to WCCP groups, when a policy is attached to a port-channel interface, or when a VLAN has a WCCP policy attached to a port-channel interface.
Conditions : This symptom might be seen when the logging server is configured for messages that are logged continually. When logging is stopped, the Cisco Nexus 7000 Series switch sends the syslog messages to the server.
Symptom : MAC addresses on F1 Series modules (F1 only VDC) are synchronized across all ASICs (even those that have no active ports) for VLANs that are active only on one of them when the destination MAC address is multicast.
Condition: In a FabricPath network, the source MAC address is always learned on egress from multicast traffic. For FabricPath VLANs in optimal mode, the MAC address is learned on egress on all FE ports, without considering if VLANs are active on that FE port or not.
Symptom : On two Cisco Nexus 7000 Series switches in a vPC with two FEXes in a FEX Straight-Through topology, a vPC host or server that connects to two FEXes might lose its MAC address entry on one of the switches. The output from the show mac address-table address command shows the correct entry on one switch, but it will be missing on the other.
Symptom : Following an ISSU from Cisco NX-OS Release 4.x to Release 5.x, if there is an ISSU failure in any module, the L2FM process can have the disable flush filter flag set to TRUE in the global configuration. As a result, MAC addresses go out of sync between vPC peers because the flush can happen in a peer-link interface.
Symptom : ARP broadcast requests that are sourced from downstream access switches with the FTAG 2 might get dropped when they reach either Cisco Nexus 7000 Series switch in a vPC+ pair when the fabricpath multicast load-balance command is enabled.
In a VPC+ environment when the fabricpath multicast load-balance command is enabled, one Cisco Nexus 7000 Series switch should be active (have affinity) for a FabricPath tree (FTAG1) for multicast or broadcast traffic and the other Cisco Nexus 7000 Series switch should be active for the other FabricPath tree (FTAG 2). In this case however, both Cisco Nexus 7000 Series switches show that they have affinity to FTAG 1 and neither of them shows as active for FTAG 2. As a result, incoming FTAG 2 traffic from downstream devices is silently dropped when it reaches either Cisco Nexus 7000 Series switch.
In a VPC+ environment when the fabricpath multicast load-balance command is enabled, the show system internal m2rib ftag 1 command and the show system internal m2rib ftag 2 command can be used to detect the problem. In this configuration, the problem is present if the command outputs on both Cisco Nexus 7000 Series switches in the vPC+ pair show ACTIVE for the FTAG 1 only.
A similar issue can occur without having the fabricpath multicast load-balance command configured. In this scenario, traffic from the CE ports is sent by one Cisco Nexus 7000 Series switch using the wrong FTAG, which results in the next FabricPath dropping the packet.
2. If applicable, remove the fabricpath multicast load-balance command from the configuration. Removing the command causes one of the two Cisco Nexus 7000 Series switches in the vPC+ to be active for both FTAG 1 and FTAG 2.
Conditions : Occurs when the user triggers this vulnerability via specific use of environmental variables while logging into the switch via SSH. The condition requires the user to log in successfully and authenticate via SSH to trigger this vulnerability
Symptom : When DHCP is configured on the switch and there are too many DHCP requests, DHCP sends a get MAC address request to Layer 2 FM as part of the response. If there are too many requests, MTM/L2FM will run out of MTS buffer space, which can result in a supervisor reset or line card reset.
Conditions : This symptom might be seen when the WCCP policy is removed on an interface by making it as a switchport and then configuring the same interface as an access port with the WCCP policy attached to it.
Symptom : The Vsh process on a Cisco Nexus 7000 Series switch might unexpectedly restart when you enter the commit command. This issue occurs even when a partial command such as “comm” is entered and it is interpreted as the commit command.
Symptom : When you enter the install all command and you are running the same Cisco NX-OS release on your switch the modules, the upgrade starts to occur, and a supervisor switchover occurs, but then the installation stops. This situation can leave some components without the proper signals that indicate that the upgrade has completed.
Conditions : This symptom might seen on the Cisco MDS 9000 32-port 8-Gbps Advanced Fibre Channel switching module (DS-X9232-256K9) and the Cisco MDS 9000 48-port 8-Gbps Advanced Fibre Channel Switching Module (DS-X9248-256K9).
Symptom : A Cisco Nexus 7000 Series switch that is running Cisco NX-OS Release 6.0(4) might experience a connected FEX going offline when one of two redundant links are shut down. In some instances, a FEX might fail to come online when it is connected to an F2 Series module.
Conditions : This symptom might be seen when a Cisco Nexus 2000 Series N2K-C2232PP-10GE FEX that has redundant links is connected to two separate F2 Series modules. The following debug messages are logged when you enter the debug sex error command:2012 Sep 17 21:28:15 N7K-FEX %$ VDC-2 %$ %FEX-2-NOHMS_ENV_FEX_OFFLINE: FEX-199 Off-line (Serial Number JAF1518BGJD)2012 Sep 17 21:28:15.057191 fex: satmgr_n7k_veobc_delete Unable to delete Pi for Veobc from ifndex 0x1a1140002012 Sep 17 21:28:15.057217 fex: satmgr_n7k_act_fport_failover: no control port for slot 131 of port 0x1a1140002012 Sep 17 21:28:16.044918 fex: satmgr_dequeue: (Error) SYSERR_FU_xx: 0x10, err_num (16) in fu_priority_select
Symptom : Cisco Nexus 7000 Series switches configured for a vPC send bridge protocol data units (BPDUs) with a source MAC address of a non-Cisco Organizational Unit Identifier (OUI) on vPC interfaces.
Symptom : When the no lacp suspend-individual command is configured on a port-channel vPC+ and the port channel is down on both vPC+ peers and one interface is up as an individual on one peer, the MAC address might flap between the port channel and the vPC peer link.
Conditions : This symptom might be seen when a port channel is configured with the no lacp suspend-individual command on a vPC+. FabricPath is configured on the peer link, the port channel is down on both peers, and one interface is up as an individual.
Conditions : This symptom might be seen on a Cisco Nexus 7000 Series switch with the TACACS+ authentication polling server enabled by the tacacs-server test command. The consistent memory leak can be viewed in the output of the show system internal tacacs+ mem-stat details command in the Grand total section.
Conditions : This symptom might be seen when the other side is sending LACP PDUs, but the Cisco Nexus 7000 Series switch is not seeing them in the output of the show lacp counters command, as shown in the following example:
Symptom : When the redistribute static route-map command is used for the routing information protocol (RIP) to redistribute specific prefixes, it leaks the default route in RIP. The command should only redistribute prefixes matched in the prefix-list RIP; however, it does redistribute the default route which should not occur.%MODULE-2-MOD_DIAG_FAIL: Module 1 (serial: xxxxx) reported failure due to Service on linecard had a hap-reset in device 1
Conditions : This symptom might be seen when new ports for a VLAN or a port channel are added to the existing running WCCP policy on the interface. New VLANs or ports do not get the WCCP policy applied for new members that are added. As a result, partial traffic does not get redirected to the WCCP client.
Symptom : On a Cisco Nexus 7000 Series switch with dual supervisor modules, if one supervisor exhibits a fatal error due to an inband driver link failure, the supervisor can take up to 60 seconds to fail over and might cause an interruption to service and a disruption to the network or links to fail. After the supervisor recovers, the following message appears in the onboard logs:
Condition : This symptom might be seen when the ACL with the object group is applied to an interface, or an already applied ACL is modified with the object group, the memory allocated for internal processing of the ACL is not freed. This situation leads to a memory leak of the size of the object group for each operation. If the modify operation for the ACL that has the object group is done periodically, then the memory leak is accumulated with the size of the object group for each operation.
Conditions : This symptom might be seen when BGP neighbors are configured with peer templates that have a common peer session. When the peer session is deleted, all BGP adjacencies that use peer templates with the common peer session go to a shutdown (Admin) state. Once a peer template is modified to remove the peer session, the BGP adjacency remains in an idle state.
Conditions : This symptom might be seen because of timing issues and might occur in supervisor OIR situations where OSPF recovers using a graceful restart. The problem should not be seen in a normal stateful supervisor switchover.
Conditions : This symptom might be seen with TACACS+ authentication when internal DNS requests are done. The default limit for these processes is 13. If the child process hangs, no more child processes can be created and TACACS+ authentication failures occur.
Symptom : An IP address is seen in multiple VRFs or in a different VRF than the one in which it is configured. This symptom occurs only on dual-supervisor systems. It cannot occur on a single-supervisor system.
Conditions : This symptom might be seen when a race condition occurs between the Netstack/IP and L3VM components. As part of a PSS synchronization between the active and the standby supervisor, the address synchronization occurs, and the VRF information on the standby supervisor is retrieved by querying L3VM from Netstack. It is possible for the L3VM database on the standby supervisor not to be in sync at the time query occurs, which can result in an interface on which the IP address is configured to be part of different VRF. As a result, the IP address that is configured on that interface will be seen in a different VRF other than the one on which it is configured.
Symptom : On a Cisco Nexus 7000 Series switch, if an interface index is queried that is higher than the number of ports on the specific line card, there is a chance that MTS memory can be held indefinitely by SNMPD and eventually exhaust MTS resources. In a dual supervisor environment, SNMPD cores and a HAP reset occurs. In a single supervisor environment, a core should be saved and the system fails or reboots.
Conditions : This symptom might be seen if a high-density line card is replaced in the same slot with a lower-density line card, and the management station continues to try and poll the nonexistent higher ports.
Symptom : The ipqosmgr process might fail and cause a supervisor switchover. In a switch with a single supervisor, the switch might reload if the network QoS template is applied and the Link Layer Discovery Protocol (LLDP) service is used.
Conditions : This symptom might be seen on a switch that is running Cisco NX-OS Release 6.1(1) or Release 6.1(2), if the user template includes "match protocol iscsi" in the no-drop class and it is used with the LLDP service and at least one interface is up. This problem activates the LLDP service.
Conditions : This symptom might be seen for the following reason. A Cisco Nexus 7000 Series switch has an OSPF Intra-Area for prefix X/24 and receives an Inter-Area prefix for X/16. When the switch loses the Intra-Area for subnet X/24, it returns to service, but it does not send an LSA update for the X/24 prefix. As a result, the rest of the network never reinstalls the X/24 prefix.
Symptom : A Cisco Nexus 7000 Series switch might incorrectly increment its DBD sequence number by 2 instead of 1 when it receives duplicate DBD packets. This behavior causes the neighboring device to detect a bad sequence number and reset the neighbor relationship to an exstart state.
Conditions : This symptom might be seen when a Cisco Nexus 7000 Series switch is a master in the neighbor relationship. The Cisco Nexus 7000 Series switch sends a DBD with a relative sequence number of 1:SWITCH SYSMGR-STANDBY-2-SERVICE_CRASHED Service "syslogd" (PID XXXX) hasn't caught signal 11 (core will be saved).
Symptom : AAA accounting does not send a stop record and the external AAA server does not reflect a stop record when a TELNET or SSH session times out due to inactivity. If the session is manually closed by the user, the stop record is correctly displayed.
Symptom : When a malformed Link Layer Discovery Protocol (LLDP) packet (such as an invalid chassis ID of port ID, or wrong type, length, or value (TLV)) is inserted by a Fuzzer tool, the LLDP process fails.
– The chassis ID or port ID TLVs have a length greater than 32 bytes (which results from the server syslog buffers being 32 characters in length, and the TLV values are copied to these buffers without length checks).
Condition : This symptom might be seen when a Cisco Nexus 7000 Series switch is running in source specific mode and receiving IGMPv3 join messages. In addition, the last octet of the source address must be higher than 222. The issue is not seen if the multicast source is 172.27.135.[1....223], but the issue does occur if the source is 172.27.135.[224....255]. This issue applies to a Layer 3 VLAN interface and a port that faces towards receivers as a Layer 2 switchport.SYSMGR-SLOT8-2-SERVICE_CRASHED: Service "lamira_usd" (PID 1944) hasn't caught signal 6 (core will be saved).
Conditions : This issue might be seen when a port-channel member port goes from individual mode back to being a member port, and the programming of the SWID and SSWID by the Ethernet port manager (EthPM) process does not occur.%VSHD-2-VSHD_SYSLOG_EOL_ERR: EOL function cli_enable_priv_level from library libcli_internal.so exited due to Signal 11
Symptom : When a switchport is configured for FabricPath but also has a legacy switchport trunk allowed vlan none command in the configuration, the port does not forward FabricPath traffic as expected. The following example shows the configuration:%NETSTACK-2-MPULLUP: netstack [PID #] p_ip_output: m_pullup failed for IP, error Resource temporarily unavailable%NETSTACK-2-MPULLUP: netstack [PID #] p_ip_output: m_pullup failed for IP, error Operation not permitted
Conditions : This symptom might be seen when a RIP neighbor advertises a new prefix through RIP. The Cisco Nexus 7000 Series switch receives the RIP updates, but it does not install the prefix in its routing table right away.
Symptom : A Cisco Nexus 7000 Series switch in a vPC with devices connected through orphan ports (devices that are single homed to one Cisco Nexus 7000 switch) might experience unicast flooding. MAC address entries can get out of sync between the MAC address table and the hardware MAC address table in M1 Series modules after a MAC address move and STP TCN.
Symptom : After an import map with a deny clause is added to a configuration with mutual imports between two VRFs, the CPU usage spikes 50 to 60 percent. If the import map is removed, the CPU usage goes back to normal.%KERN-2-SYSTEM_MSG: mts_is_q_space_available_new():1416:Total mtsbuf size 10070872 for sap 28, exceeds limit 15 perc of 67108864 - kernel%KERN-2-SYSTEM_MSG: mts_acquire_q_space() failing - no space in sap 28, uuid 26 send_opc 3176, pid 3616, proc_name sctpt_rx_thr - kernel
Conditions : This symptom might be seen when a monitoring device is using snmp-bulk-get requests on the entity-MIB for multiple FEXes at one time. In addition, this symptom might be seen if there is continuous polling from multiple polling stations on slow MIBs.
Symptom : An ISSU in a vPC+ with an F1 Series module causes LDB misallocation. This issue can occur during an ISSU from Cisco NX-OS Release 5.1(x) and all later releases where vPC+ support was introduced.
Once the ISSU completes, the LDB is not allocated properly. If there are subsequent interface flaps, it is possible for the interfaces to take LIF in the LDB range that is associated with the Replicator Interface that is used in mixed chassis setups. This situation can cause incorrect index programming in hardware.
Symptom : FabricPath does not come up across the vPC peer link on a Cisco Nexus 7000 Series switch. The vPC FabricPath status displayed with the show vpc command is “peer is not reachable through fabricpath.”
The problem occurs because the vPC peer link is incorrectly programmed as medium type broadcast, instead of P2P. The FabricPath ISIS adjacency cannot form over a port channel that is medium type broadcast. The show run all | section interface port-channel command displays the following:
Conditions : This symptom might be seen on a Cisco Nexus 7000 Series switch running Cisco NX-OS Release 5.2(3a). The servers can be pinged, but the switch is unable to authenticate with the AAA servers. This is not a connectivity issue with the AAA servers.
Symptom : A switch might stop responding to the session when a block of VLANs is added. The output of the show process cpu command indicates that the Ethpm process varies from 19% to 42% and is stuck in that range.
Conditions : This symptom might be seen when an ACL policy is applied on an XL I/O module (which use SPANSlogic TCAMs). The SPANSlogic TCAM segment usage is high, that is, the free segment count is low:
Symptom : Following a switch reload, discovery of new hosts no longer works correctly for LISP extended-subnet-mode. A null0 route is not present for the dynamic-eid configured for extended-subnet-mode after the reload.
Conditions : This symptom might be seen when an interface configured for extended-subnet-mode is also attached (that is, the interface configured for extended-subnet-mode has the same subnet prefix as the dynamic-eid).
Conditions : This symptom might be seen on a Cisco Nexus 7000 switch that is running NX-OS Release 5.2(3a). An ISSU from Release 5.2(3a) to Release 5.2(5) was attempted soon after the switch was reloaded with Release 5.2(3a).%SYSMGR-2-SERVICE_CRASHED: Service "netstack" (PID 4234) hasn't caught signal 11 (core will be saved).%ARP-3-IP_INTERNAL_ERROR: arp  -Traceback: libip.so+0x12a09 0x8085985 0x8086e35 librsw.so+0xd59e8 librsw.so+0xd5e26 librsw.so+0xd428a librsw.so+0xd5676 librsw.so+0xa6aff libpthread.so.0+0x6140 libc.so.6+0xca8ce
Conditions : This symptom might be seen when a Cisco Nexus 2000 Series FEX is connected to a Cisco Nexus 7000 Series switch. It is not seen when a Cisco Nexus 2000 Series FEX is connected to a Cisco Nexus 5000 Series switch.
Conditions : This symptom might be seen in the following situation. A MAC address was local to site A. Now the MAC address has been moved to site B. The OTV VDC at site B correctly learns the MAC address on a local port channel or local interface; however, it again points to the overlay interface. Site A never learns this MAC address on the overlay interface.
Conditions : The symptom might be seen when the SNMPwalk of the cEigrpPeerTable (220.127.116.11.18.104.22.168.422.214.171.124) does not return the correct cEigrpPeerIfIndex (126.96.36.199.188.8.131.52.4184.108.40.206.4). The ifIndex device that is returned does not correspond to any interface on the device.%ETHPORT-2-IF_CRITICAL_FAILURE: (Debug syslog)Critical failure: qosmgr_dce_gldb_get_all_vl_params returned error: , no such pss key
– There is an empty port channel in a random sequence of configurations that include adding or removing members of the port channel, and various commands such as the software monito r command or software mode access command are entered.
Conditions : This symptom might be seen when there are two Cisco Nexus 7000 Series switches in a vPC, and MSTP is the spanning tree protocol in use. All VLANs are assigned to one instance, but only several are created in the network. The root is a Catalyst 6500 switch or some other upstream switch. The symptom occurs only if the switch is in a vPC.
Symptom : When using VRF other than the management VRF to send SNMP traps, if the management port is down but not administratively down, all trap packets will be queued forever if the alarm for turning the management port on failed to run.
Symptom : Cisco NX-OS Release 5.1, Release 5.2, and Release 6.0 run a version of the Linux Kernel that has a known Linux Kernel caveat, which is discussed in the public forum at http://serverfault.com/questions/403732/anyone-else-experiencing-high-rates-of-linux-server-crashes-today?answertab=active#tab-top
– When the NTP server pushes the update to the Cisco Nexus 7000 Series switch NTPd client, which in turn schedules the update to the Kernel. This push should have happened 24 hours before June 30th, by most NTP servers.
Now that June 30th 23:59:60 UTC has passed, if your Cisco Nexus 7000 Series supervisor modules have not reset or switched over, you are not affected by this caveat until the next leap second update mentioned previously.
Symptoms : To confirm that you have experienced this issue, you should see all of the following symptoms. Seeing symptoms 1 and 2 is not sufficient to confirm that you have this issue. Seeing symptoms 3 and 4 provides the best confirmation.
2. Onboard Failure Logging (OBFL) has a message stating: system reset sw reason unknown, hw reason reset by platform or hw watchdog. For example, you might see output like the following from the show logging onboard mod reset-sup-slot-number internal reset-reason command:
Note If the switch has been rebooted or power cycled, there is no way to confirm if you have experienced this issue, because symptoms 3 and 4 might not be in the log. You might have to assume based on symptoms 1 and 2 and circumstantial evidence that the reset happened right around or before the UTC leap second update on June 30th.
Symptom : On a Cisco Nexus 7000 Series switch (PE), a VRF route that points to a next hop is on a remote PE under VRF blue, loopback 10. When it is pinged from a Cisco Nexus 7000 Series switch, it works, but when the traffic goes through the Cisco Nexus 7000 Series switch, it fails. On the packet capture, the Cisco Nexus 7000 Series switch puts two labels, 3 and 18 (VPN), for the failing one. But when pinged from a Cisco Nexus 7000 Series switch, 18 (VPN) is the only label that is correct because both PEs are directly connected.%ELTM-2-INTERFACE_INTERNAL_ERROR: Internal error: VlanX:SVI up before VLAN is created , collect output of show tech-support eltm2008 Mar 30 14:25:59.963 enakmt-agg4-sw 30 14:25:59 KERN-2-SYSTEM_MSG [901255.509710] mts_print_longest_queue_state: opcode counts for first and last 50 messages in recv_q of sap 27: - kernel2008 Mar 30 14:25:59.963 enakmt-agg4-sw 30 14:25:59 KERN-2-SYSTEM_MSG [901255.509728] mts_print_msg_opcode_in_queue: opcode 7679 - 100<tel:7679%20-%20100> messages - kernel
Symptom : During an ISSU or ISSD, due to potential differences in the SAPs used by services in either release of Cisco NX-OS, the System Manager might fail in rare circumstances due to a broken pipe. The behavior should be to ignore any SAPs on the active supervisor that are not valid in the release of Cisco NX-OS running on the standby supervisor.%ARP-3-REQ_IP: arp  Sending ARP request for invalid IP address 0.0.2.0 on port-channel22.4, request from pid: 6905
Symptom : This SA message with encapsulated data is sent with a wrong checksum, which causes the receiver MSDP peer to drop it. This packet will never be processed (decapsulated) and sent across to the downstream neighbors by the receiving MSDP peer.
Conditions : This symptom might be seen when SPAN is configured on a VDC on a F1-Series module. Moving a port to the VDC and making it a SPAN destination while it is administratively down can trigger this issue.
– If the user configures another pinned static route to the same x.x.x.x/y via another pinned interface and next hop a.a.a.a, but with a tag value of 100, and if the pinned interface in this case is down, then the route is not installed in URIB as is the expected behavior.
Symptom : After an ISSU or a supervisor switchover, a Cisco Nexus 7000 Series switch might send back a VTP packet on the same vPC from which it ingressed. In a Data Center Interconnect (DCI) topology, this packet return can cause a storm of VTP packets between the Cisco Nexus 7000 Series switches.
Symptom : When a virtual port-channel plus (vPC+) is configured with the no lacp suspend-individual command, if the port is down on both vPC+ peers and one interface is up as individual on one peer, the MAC address might flap between the port and the vPC peer link.
Conditions : This symptom might be seen when a port is configured with the no lacp suspend-individual command, there is a vPC+ (FabricPath is configured on the peer link), the port is down on both peers, and one interface is up as an individual interface.
Symptom : Remote SPAN traffic might not be forwarded correctly to the destination port on a Cisco Nexus 7000 Series switch (with a trunk port that allows RSPAN VLAN) when traffic ingresses on an F-Series module and egresses on an M1-Series module.
Conditions : This symptom might be seen when you create RSPAN and put the M1 port as the destination and the source as the F1 port. In this case, RSPAN learns the MAC address on the M1 port. RSPAN capture on the M1-Series module does not work for the F1-Series module ports.
Symptom : In a rare situation, a BGP best-path run might stall due to an issue in the BGP-ULIB flow control logic. To confirm the problem, examine the output of the show tech bgp command or the show tech l3vpn command for the following information:
Conditions : This issue might be seen because of an integer wraparound issue in the BGP-ULIB flow control logic. If during the wraparound period, ULIB is busy and slow to respond, the BGP best-path run is blocked indefinitely. This problem is very time sensitive and rare.
Conditions : This symptom might be seen when the QoS policy has MPLS related attributes in either matching or as part of the action for a class map. For example match/set mpls experimental as part of a QoS policy would lead to an ACLQOS failure on an F1-Series module.
Symptom : On a Cisco Nexus 7000 Series switch, bridge assurance can block or unblock some VLANs on some ports when the spanning-tree internal event-history all brief command completes. The tac-pac and show tech stp commands can have the same effect. STP is also seen to core on a few occasions.
Symptom : Policies such ACL, QoS, and PBR for FEX interfaces are not cleaned from connecting modules when the FEX fabric ports are moved to another VDC. If those ports are moved back later to the same VDC and configured as a fabric port, or some other ports in same module are configured to be fabric ports, the FEX might not come online (using those ports), or the relevant policies might not be enforced.
Conditions : This symptom might be seen when applying an ASCII configuration file in which every VLAN has a unique attribute, such as “name,” and one VLAN at a time is created. The sudden load on the system can cause a timeout.
Conditions : This symptom might be seen when the physical node has a large number of VDCs or a large configuration. In such a case, it takes time during the switchover for the OTV-IS-IS process to get its configuration. During that time, neighbors can time out the node that is undergoing the switchover.%VSHD-2-VSHD_SYSLOG_EOL_ERR: EOL function uri_copy from library liburi_copy.so exited due to Signal 11
Symptom : A RADIUS configuration is missing after a supervisor switchover. The output of the show running-configuration radius command shows that the RADIUS configuration is missing following the supervisor switchover.
Symptom : When a peer link is brought up, VLANs 2047-4094 are suspended because they are not allowed in the vPC peer, even those VLANs are allowed and correctly configured on the vPC peer device. As a result, 6 to10 second packet drops can occur in VLANs 2047 to 4094.
Conditions : This symptom might be seen if there are more than 2049 VLANs created and allowed on the vPC peer link. It is not necessary to have those VLANs in one range or started from number one. This symptom can occur when the total count of VLANs is more than 2049.
Symptom : OSPF does not automatically recalculate redistributed routes for database selection when route changes occur manually (such as removing static routes), or when routes are removed on neighboring devices into dynamic routing protocols (such as EIGRP). As a result, an outage could occur due to lack of a route.
OSPF requires unique link state IDs when inserting routes into the OSPF database. When OSPF chooses between two routes with different masks (such as 192.168.1.0/24 and 192.168.1.0/32) with identical link state IDs (that is,192.168.1.0) before inserting the routes into the database with identical parameters (such as. Advertising Router), the NX-OS software selects the route with the longest match (/32). In this scenario when the /32 route is removed, OSPF will not automatically recalculate the routes and insert the /24 into the OSPF database and advertise it to neighboring routers.
Symptom : A Cisco Nexus 7000 Series switch that is running NX-OS Release 4.2(6) with an access-list deny setting with the log option might report the egress interface in the log entry instead of the ingress interface.ERROR: Unable to perform the action due to incompatiblity: Module 2 returned status "Number of Mutation maps limit reached in the hardware"
Symptom : After BGP best path runs, some BGP IPv4 or Unicast learned routes in the default VRF might remain in an invalid state and are downloaded into the URIB or advertised to peers. The show ip bgp command on the route shows that the path is invalid. Correspondingly, a show bgp ipv4 unicast nexthop-database command on the route’s next hop shows that the RNH is resolved and reachable.
Conditions : This symptom might be seen when a router is configured as the stub router, and the partner router is told that the router is now the stub, and should therefore not send queries for failed routes to the router. However, even with the stub configured, the EIGRP neighbor still sends the query.
Conditions : This symptom might be seen if MRIB does not get an acknowledgement back from MFDM. This symptom has been seen during high churn in high scale Mroute tables (that is, there is a high frequency of adding or deleting Mroutes or OIFs).
Conditions : This symptom might be seen when memory is low in the vPC process. This symptom might also be triggered by a show running-configuration command or a similar command in which the vPC process needs to write its configuration but cannot because of the inability to allocate enough memory (which is denoted by the MALLOCFAIL errors).%SYSMGR-2-SERVICE_CRASHED: Service "netstack" (PID 4143) hasn't caught signal 11 (core will be saved).
Conditions : This symptom might be seen when pings destined to FF02::1 and sourced from a global address are received on an interface on the Cisco Nexus 7000 Series switch that has no global address assigned to it.
Symptom : On a Cisco Nexus 7000 Series switch that is running NX-OS Release 5.2(3a), a FEX port might stop learning MAC addresses after port-security with static secure MAC address configurations is removed.
Symptom : When a Cisco Nexus 7000 Series switch is a rendezvous point (RP) and a Cisco IOS device such as a Catalyst 4900M is a first-hop and last-hop router, the Cisco Nexus 7000 Series device does not return a registration stop when it receives a multicast source registration and PIM (S,G,R) prune message back-to-back. As a result, the S,G route gets stuck in registration mode on the IOS router and it has to software switch the multicast packets, which causes high CPU utilization.
Conditions : This symptom might be seen in a topology where a Cisco Nexus 7000 Series switch is a rendezvous point (RP) and a Cisco IOS device such as a Catalyst 4900M is a first hop and last hop router.
Conditions : This symptom might be seen with Layer 2 forwarded frames that hit one of the IDS checks. For example, a Layer 2 forwarded frame with an IP address that is all zeroes is forwarded but is counted as if it was dropped by the IDS check.
Symptom : On a Cisco Nexus 7000 Series switch running NX-OS Release 6.0(2), where DNS servers are configured and name lookup is enabled, the following error appears if the NTP server is configured using a host name such as ntp server hostname :
Symptom : Following a switch reload with a 32-port 1/10 Gigabit Ethernet module (N7K-F132XP-15) in the chassis, the 48-port 10/100/1000 Ethernet I/O module XL (N7K-M148GT-11L) came online as OK, but all of the interfaces on the module are missing.
Conditions : This symptom might be seen when an MTS process is unable to keep up with the amount of messages required to sync between modules in the switch. The buffer queue fills up which depletes the memory.
Symptom : The ISSU from NX-OS Release 4.2(4) to NX-OS Release 4.2(8) failed. As a result, some modules ended up running Release 4.2(4) and others were running Release 4.2(8), which caused packets to be software switched.
Symptom : Packets are destined for the router MAC address of one node of two Cisco Nexus 7000 Series switches that are set up for vPC. The peer link is on a F1 module. M1 modules are in the system. The peer-gateway that arrives on the peer can be policed heavily by control-plane policing after it is received from the peer link. This situation might lead to random connectivity being issued to any number of hosts when an ARP refresh occurs, which causes some replies to be dropped and the ARP entry to be flushed.
Conditions : This symptom might be seen in the following scenario. There are two Cisco Nexus 7000 Series switches: switch1 and switch2 are configured for vPC and the peer link is on the F1 Series module and there are M1 Series modules present in both switches and peer-gateway configured.
When switch2 sends an ARP request for a host and the reply packet hashes to switch1 on a vPC port channel, the destination MAC address of switch2 on switch1 has a gateway bit set because of the peer-gateway. The gateway bit is sent to software for encapsulation and forwarded across the peer link to switch2. Because the encapsulated packet uses the same destination MAC address as the original destination, when the packet arrives at switch2, it is sent to an M1 Series module because the MAC address has the gateway bit set and is subject to CoPP. These packets are classified under the Layer 2 default class and might be dropped if there is other unwanted Layer 2 traffic in the network.
Symptom : If any routes are received with an AS4 path attribute and that path has a loop, all feasible updates are dropped until another update is received that has an AS4 path attribute without a loop.
Conditions : This symptom might be seen when there are two peers and one advertises a single route with AS4 path and the other peer advertises multiple routes without an AS4 path. Once the update for the first peer with the loop is received, all updates from the other peer are dropped. The first peer without the loop can then advertise its update which clears the condition. This causes the DUT to accept the updates from the other peer.
Conditions : This symptom might be seen when a lot of group entries are inserted in the MAC address table. There might be MAC address table collisions, at which point the insertion fails. In such a condition, a syslog message is expected to be recorded in the logfile, but it was not because the severity level of the syslog message was previously set at two.
Conditions : This symptom might be seen only when there is a square topology with two FabricPath Layer 2 GSTP switches on one side and two legacy STP switches and the blocking port is between the STP devices. This issue is not present in a triangular topology.
Conditions : This symptom might be seen after an upgrade and switchover on the switch. The redistribute direct route-map command for IPV4 or IPV6 AFs or both is added and removed. There are no match statements with match interface conditions.
Symptom : Static routes that are redistributed on the Cisco Nexus 7000 Series switch into OSPF might not appear in the routing tables of OSPF neighbors because the forwarding address is not updated after route changes have occurred within the network.
Conditions : This symptom might be seen if the source Cisco Nexus 7000 Series switch is redistributing static routes that have available paths through SVI interfaces and other Layer 3 interfaces. There is a timing issue where OSPF learns of the reachability through the Layer 3 interfaces, however the preferred path to the network destination is through an SVI interface. After a reload of the source Cisco Nexus 7000 Series device, OSPF installs the forwarding address of valid Layer 3 interfaces while the SVI is still initializing. After the SVI is fully operational, OSPF is not be updated of this change in state.
Workaround : This issue is resolved in Cisco NX-OS Release 5.2(5). In releases earlier than Release 5.2(5), enter the shut command followed by the no shut command on the member interface or the port channel to resolve the issue.
Symptom : Too many MAC address moves over a vPC peer link can cause the l2fm process to fail or the chassis to reload. The output of the show system reset-reason command indicates that the reload reason is caused by a l2fm hap reset.
Conditions : This symptom might be seen when a Layer 2 trunk port (on a Catalyst 6000 switch) with a native VLAN other than 1 is connected to a Layer 3 port (on a Cisco Nexus 7000 Series switch) that does not have a subinterface with VLAN 1. CDP neighbors are not seen. This problem does not happen if the Layer 2 trunk port is configured with native VLAN 1.
Symptom : A module might get reloaded more than once before it comes up. In rare cases, the ports in the module might be up before the module is reloaded once. When the module is reloaded slightly after the ports are brought up, an adjacent switch might see a port flop.
Conditions : This symptom might be seen once you configure the system reserved VLAN range. All the VLAN configurations for the new range get deleted from the running configuration and any checkpoint that has a VLAN configuration in the new range also become obsolete.
Conditions : This symptom might be seen because the value in the SNMP SET operation is set to a zero-length string. If you set the managementDomainName to a non-zero-length value, that works correctly.
Conditions : This symptom might be seen when a table (address family of a VRF) in BGP is deleted and while the deletion is still in progress, a new table with the same table ID is created. BGP then fails with a “Table not found” error.
Symptom : An internal index related to IGMP snooping is not updated correctly when IGMP snooping or OMF are disabled. Once this situation occurs, OMF-related information remains incorrect even after IGMP snooping or OMF are enabled, which results in multicast flooding.
Condition : This symptom might be seen following a module reload. You can verify the issue by entering the show system internal mtm info all | grep ack_pending command. You might see the following output:
If sup_ack_pending is set to 1, then you have encountered the issue. The pending ack causes future Nls not to be reported from the modules to the supervisor which causes MAC addresses to be out of sync between the modules and the supervisor.
Conditions : This symptom might be seen when there is a large access list configuration on a Cisco Nexus 7000 Series switch and the ACL manager fails to respond to an ASCII configuration request in time. As a result, an incomplete ASCII startup configuration is saved.
Symptom : In certain rare situations, a Layer 2 MAC address forwarding table might become inconsistent between modules on a Cisco Nexus 7000 Series switch. This inconsistency causes traffic that is destined to the affected MAC address to be blackholed.
Symptom : On a vPC+ setup with asymmetric traffic flows across two vPC+ pair switches, traffic might drop if it is directed towards a peer switch where the host is singly connected. This condition could happen for orphan hosts and east-west traffic that also has vPC+ enabled.
Condition : This symptom might be seen when communication between the supervisor and a linecard fails and the system has a critical error. The pfstat process does not handle the error condition gracefully; it exits and fails.
Conditions : This symptom might be seen on a 32-port 10-Gigabit Ethernet SFP+ I/O module (N7K-M132XP-12) with a FEX connected to it, and an ISSU from Cisco NX-OS Release 5.1(3) to Release 5.2(3a) is performed.
Symptom : A Cisco Nexus 7000 Series switch with a FEX connected to an 32-port 10-Gigabit Ethernet SFP+ I/O module (N7K-M132XP-12) in slot 1 can incorrectly experience LIF exhaustion. The switch log shows a failure to allocate LIF entries:%ELTMC-SLOT1-2-ELTMC_L2_LIF_ALLOC_FAIL_INTF: Failed to allocate L2 LIF entries in forwarding engine for interfac Ethernet<slot/port>
Symptom : When polling at a sustained rate on a Cisco Nexus 7000 Series switch, certain objects from the BRIDGE-MIB might cause a relatively high CPU usage for SNMPD for some time after polling and might cause new requests to time out. On releases earlier than Cisco NX-OS Release 5.2, this polling might cause internal messages for interprocess communications to be queued and might affect other services.
Conditions : This symptom might be seen on a switch with multiple VDCs where a vPC is configured on one VDC and a vPC+ is configured on another VDC. The peer link is not learned and is not set in the port ASIC of the M1 series modules. As a result, the hardware learns the MAC address of packets coming in from the peer link.
Symptom : If an SVI for a VLAN is up, and you configure the corresponding VLAN as private-vlan non-primary, the SVI manager is unable to respond to the PVLAN. The CLI configuration might hang and not complete. SVI resources might stay locked and a subsequent SVI configuration on the affected SVI might fail.
Conditions : This symptom might be seen when FCoE traffic is looped at line rate between two Cisco Nexus 7000 Series switches, a Cisco Nexus 5000 Series switch, and a Cisco Nexus 7000 Series switch in a double-sided vPC topology.
Conditions : This symptom might be seen when vPCs are single-homed to the vPC secondary switch. All STP BPDUs for the mcec are generated by the primary switch and tunneled over the peer-link. These packets are subjected to a different rate limiter that is more aggressive.
Conditions : This symptom might be seen under normal operating conditions. F1 Series modules will leak broadcast ARP and link-local multicast traffic to the in-band CPU, regardless of whether an SVI exists for the VLAN. This traffic is rate limited, however in aggregate can cause unnecessary traffic to be processed.
Conditions : The symptom might be seen when there is a Layer 3 interface configured with Netflow and policy-based routing, and one of the ACLs that is referenced in the policy does not have any access-control entries installed.
Symptom : Forwarding for VLANs stops in the system when there is a FEX Host Port-Channel (HIFPC) down or a CBL is blocking for some or all the VLANs in the allowed VLAN list for the FEX Host Port-Channel.
Conditions : This symptom might be seen when a peer-link port channel is deleted and the vPC is brought down through a laser cut. In some cases, especially when there are a lot of VACLs, the ACL manager might take some time to clean up the VACLs which delays any notification to UDLD to stop listening for packets. As a result, UDLD continues to run and then it error disables the port after it time outs.
Symptom : A Cisco Nexus 7000 Series switch might redirect traffic to the CPU so that the traffic might experience random delays or drops. ARP is learned and FIB adjacency is in the FIB adjacency table.
Conditions : This issue might be seen because of race conditions. Some hosts do not respond to the ARP refresh sent by the Cisco Nexus 7000 Series switch which in turn triggers a deletion of the ARP entry due to expiry. Because of this, the route delete notification is sent to URIB from the process. However, traffic still arrives at the given IP address. As a result, the next packet triggers ARP and ARP is learned from the host.
Symptom : The otv extend-vlan command (and possibly other commands) might not be saved from the running configuration to the startup configuration. As a result, the commands do not appear after a reload. Other affected commands include the following:
Symptom : Following an ISSU, certain traffic for a VLAN that was flowing correctly before the upgrade starts to drop. This situation can be caused by incorrect hardware ACL identifiers being programmed on the affected VLANs, even though there might not be any ACLs present.
Conditions : This symptom might be seen if the same PBR policy is applied on multiple interfaces before the next hop adjacencies are resolved. It does not redirect the traffic correctly on some interfaces.
Conditions : This symptom might be seen on a Cisco Nexus 7000 Series switch that runs Cisco NX-OS Release 5.x software and the switchport command is executed on an Layer 3 port containing subinterfaces.
Conditions : This issue is seen only when the access switch reloads and the port-channel interfaces are split across the two vPC switches. This issue also requires a significant amount of STP traffic that originates from one of the vPC switches that goes to the access switch.
Symptom : When a module reloads or the weighted random early detection (WRED) configuration changes on a Cisco Nexus 7000 Series switch, continuous partial traffic loss that is independent of the traffic rate and WRED thresholds can occur.
Symptom : ERSPAN destination ports do not receive the copied traffic from ERSPAN sources. ERSPAN GRE encapsulated traffic is sent to the destination VDC or switch but it is not mapped to the ERSPAN destination port.
Symptom : A nondisruptive software upgrade (ISSU) from NX-OS Release 5.1(1) or Release 5.1(2) to Release 5.1(3) causes spanning tree bridge protocol data unit (BPDU) timeouts, Unidirectional Link Detection (UDLD) timeouts, and Enhanced Interior Gateway Routing Protocol (EIGRP) timeouts on adjacent devices which results in network disruptions.
Conditions : This symptom might be seen when the WRR configuration on an interface is modified. The existing priority queue configuration is not considered which results in bandwidth being taken from the existing queue to be allocated to the priority queue.
Symptom : A gratuitous ARP (GARP) storm can cause the MTS buffers to lock up which can cause connectivity issues on the network and eventually lead to a supervisor failover. The following syslog messages might be seen:
Conditions : This symptom is specific to a storm of GARPs from multiple hosts that claim the same IP address. This symptom causes the Cisco Nexus 7000 series switch to constantly update its ARP and adjacency tables which might result in an MTS buffer lockup.
Conditions : This symptom might be seen when more than 4000 VLANs are created on the 32-port 1/10 Gigabit Ethernet module. Internally the failure occurs because of the corresponding SVI creation for those VLANs. The failure happens when the module is supporting more than 1 VDC and the total VLAN count across all VDCs is greater than 4000. Such VLAN scale numbers are not currently supported taking into account the total Layer 2 group features supported on Cisco Nexus 7000 Series switches.
Symptom : In a dual-sided vPC setup, when one member link of each vPC pair is down or shut, there can be a software loop of IGMP Global Leave packets if there is a topology change. If this happens, it will lead to high CPU usage.
Condition : This symptom might be seen in a FabricPath topology with M1 series modules on the edge for ingress flows and two or more non-port-channel parallel links between the FabricPath core switches.
Symptom : In some scalability setups, where there are a lot of FEXes and lot of HIF vPCs, a reload of all the fabric modules (which in turn causes a reload of all the FEXes), can cause some satellite interfaces (FEX ports) to become error-disabled after the reload. Syslog messages are also generated with more details on specific ports that are error-disabled.
Conditions : This symptom might be seen if you enter the clock summer-time command and attempt to make changes to the summer-time configuration. Even though the output of the show clock detail command will show the correct summer-time settings, the changes are not updated in the RPM which can affect other components, such as key chains, that rely on timing.
Symptom : Cisco Nexus OS contains a vulnerability that could allow an authenticated, local attacker to execute arbitrary commands on a targeted device. The vulnerability is due to improper sanitization of user-supplied values to command line interface commands.
An authenticated, local attacker could exploit the vulnerability by issuing commands that contain malicious options on the device command line interface. If successful, the attacker could gain elevated privileges on the targeted device.
Symptom : The running configuration contains lines of a configuration that is no longer valid because they pertain to a feature that was active at some point but has since been disabled. If you try to execute the configuration, you receive syntax errors for those lines. The lines of the configuration in question are these:
Symptom : Two protocols add the same route: OSPF and RIP. The admin distance of RIP is configured to be the same as OSPF. If the metric for the RIP route is better than the OSPF route, the RIP route is selected (which is incorrect behavior).
Conditions : This symptom might be seen when two protocols are configured to have the same admin distance. If RIP and OSPF are configured to have the same admin distance, the software chooses the route with the lower metric. Because metrics do not have any meaning across protocols and only within a protocol, this selection does not make sense. The route found by the protocol with the lower default admin distance should be selected.
Conditions : This symptom might be seen when the mask value is 64 or greater or when there are many service groups (roughly greater than 20). The output is not displayed completely because the TLVs used to send the information to the frontend are not big enough to store all the necessary information.
Conditions : This issue might be seen if VRRP groups form peers with devices other than Cisco NX-OS 7000 Series switches, authentication is enabled, and the password configured is less than eight characters.
Symptom : Reloading an OTV VDC causes an OTV adjacency to immediately come up, but the show otv isis adjacency command shows that the neighbor name is not resolved and no IS-IS LSP is received from the neighbor until 8 to10 minutes later.
Symptom : Following a reload of a Cisco Nexus 7000 Series switch that has a core VDC and an OTV VDC, the other site ED cannot establish an OTV adjacency with the VDC on the reloaded switch. The other site ED has *,G for the OTV core multicast group and s,g for the other ED, but no s,g for the reloaded ED.
This issue can be triggered by an interface flap of OSPF neighbors, a module reload, or the clear ip ospf neighbor command. The probability of this issue occurring is higher if many neighbors flap at the same time, but it does not occur at each flap.
Symptom : When an ISSU from Cisco NX-OS Release 5.1(3) to Release 5.2(1) is performed on a Cisco Nexus 7000 Series switch, the MTU on the Layer 3 port channel interfaces that have a jumbo MTU configured will be misprogrammed in hardware which will result in traffic being switched incorrectly in software and will cause poor performance.
Conditions : This symptom might be seen if you enter the show running command or the s how startup command many times. A memory leak occurs in the service eth_port_channel when handling this operation.
Symptom : An ISSU from Cisco NX-OS Release 4.2(4) to Release 5.1(3) can cause an internal process to fail. In addition, the ISSU might be incomplete which can cause a few modules to remain on Release 4.2(4).
Conditions : This symptom might be seen when a user tries to manually disable autonegotiation by configuring a non-auto speed on a Cisco Nexus 7000 Series switch that is running Cisco NX-OS Release 5.2(1).
Conditions : This symptom might be seen when either dot1x or CTS is enabled, or both are enabled along with the FEX configuration in the same VDC. Dot1x or CTS do not be enabled on the FEX ports for this symptom to occur.
Symptom : In previous Cisco NX-OS releases, the NTP authentication key limit was 8 characters. As a result, following a downgrade, the ASCII replay might fail for the authentication key configuration. Also following a downgrade, deleting a longer key might fail.
Symptom : A DCHP relay on a Cisco Nexus 7000 Series switch does not flood the DHCP offer received from the server where the client set the broadcast bit. The destination MAC address is ffff.ffff.ffff, but the CPU sends the packet out the interface where the corresponding DHCP discover packet was received from the client.
Conditions : This symptom might be seen when the broadcast bit is set to client. The result should be flood to VLAN. In this case, the DHCP offer is not flooded, and if the client is now known through a different interface, or circumstances prevent that broadcast packet from reaching the client through the original path, DHCP times out.
Conditions : This symptom might be seen when a Cisco Nexus 7000 Series switch is a mixed chassis, with both M-1 and F1- Series modules, and there is a TX SPAN session configured with the destination port as a trunk port. The SPAN destination port can be in either the M-1 or F1- Series module. The switch is running Cisco NX-OS Release 5.2(1).
Conditions : This symptom might be seen if there are any topology changes during an ISSU, such as multicast join or leave, or link flaps of the FEX ports. The issue can cause some resource leaks and an MTS buffer leak in the vntag_mgr process. The issue might appear a long time after the ISSU.
If the first message is not received by one particular linecard, after two supervisor switchovers, the supervisor bound credited traffic will not go through because LTL entries that correspond to supervisor DIs are zeroed.
Symptom : Broadcast traffic fails to pass on a VLAN that is converted from a private VLAN to normal VLAN on the 32-port 10-Gigabit Ethernet SFP+ I/O module in a Cisco Nexus 7000 Series switch that is running Cisco NX-OS Release 5.2(1).
Conditions : This issue might be seen with multicast traffic when two or more source multicast ports are removed from the SPAN source or when the source of the VLAN is removed that has multicast traffic running.
Symptom : The mpls ldp sync command is removed from the OSPF configuration after a reload. The command is present in the startup configuration, but does not appear in running configuration. The feature is also not active after the reload.
Symptom : After a Layer 2 multicast lookup MAC address is configured, the Cisco Nexus 7000 Series switch still floods unicast traffic with the destination MAC address as a multicast address, if the Cisco Nexus 7000 Series switch routes the traffic. The switch should forward the traffic to ports in the mac address-table multicast 01xx.xxxx.xxxx vlan vlan-id interface interface-name command.
Conditions : This symptom might be seen when a switch has multiple VDCs. If a vPC is configured on one VDC and a vPC+ (emulated vPC) is configured on another VDC, then do not learn on peer_link is not set in the port ASIC for the M1 modules. This configuration causes packets coming in from the peer link to be learned by the hardware.
Symptom : When upgrading a Cisco Nexus 7000 Series switch to Cisco NX-OS Release 5.2(3), the vlan_mgr process might fail once the upgrade is complete if the show vlan command is executed manually or using a script.
Conditions : This symptom might be seen when there is a long-lived TCP connection from NMS to the Cisco Nexus 7000 Series switch. The netstack TCP buffer gets full and the following send() call gets stuck if it is a BLOCKING call. As a result, SNMP fails due to a missing heartbeat.
Conditions : This symptom might be seen when a MAC address move is initiated due to a topology change by STP. The MAC addresses that are missing in the output of the show mac address-table command do not have active traffic coming from them.
Conditions : This message is seen when the port-channel interface comes online or goes offline with a Web Cache Control Protocol (WCCP) policy applied to it. The message is seen only in Cisco NX-OS Release 5.1(1) and Cisco NX-OS Release 5.1(2).
Conditions : This symptom might be seen when ND packets are routed on a remote vPC peer switch and as a result, the TTL/hop-limit in the IPv6 header is decremented. When the packet reaches the vPC switch to which the ND packet is destined, the TTL will not be 255 and will be dropped in the software.
Symptom : A Cisco Nexus 7000 Series switch might be unreachable (through ping, HSRP, or Telnet), and stop routing all ingress traffic on an impacted module for a specific VLAN. Further analysis shows the RMAC of the impacted VLAN is not programmed in the hardware on the impacted module.
Symptom : When RBAC is disabled on a Cisco Nexus 7000 Series switch, all commands are forwarded for authorization to the TACACS server. For example, when you create a new user by entering the username test5 password cisco role network-operator command, the TACACS server passes this command, but the switch rejects the command with the message: “cannot make changes for other user.”
Conditions : This symptom might be seen when two Cisco Nexus 7000 Series switches that are running Cisco NX-OS Release 5.1(2) are connected through a vPC to a secondary switch. Root guard is enabled on the vPC and Spanning Tree Protocol priority is lower on the secondary switch, which disables the vPC. Root primary should be enabled on the Cisco Nexus 7000 switches to clear the root guard condition, however, the port does not recover.
Conditions : This symptom might be seen if the same PBR policy is applied on multiple interfaces before the next hop adjacencies are resolved. It does not redirect the traffic correctly on some interfaces.
Symptom : If you move a vPC peer link from one port-channel interface to another port-channel interface, and the peer link is composed of members that are on an F1-series module, then broadcast packets can loop from one vPC member, across the peer link, and out of the other vPC member.
Symptom : A faulty fabric module could cause a reset of the supervisor module and possibly other modules due to an ASIC fatal error. In the output of the show module internal exception log mod module-number command, the error description shows the following:
Conditions : This symptom might be seen when two members of the same FE (x and y) belong to the same FabricPath port channel (that contains any number of port-channel members) and one of the members (x or y) is brought down. This symptom occurs only on switches where FabricPath is enabled.
Symptom : On a Cisco Nexus 7000 Series switch that is not performing DHCP relay functionality or DHCP snooping, any DHCP discover or offer packet, or boot packet that has a source IP address of 0.0.0.0 and destination IP address of 255.255.255.255, and that is sourced or destined for UDP port 68 or 67, the forwarding engine will classify this packet and count it toward the control-plane policing statistics in the class where DHCP is defined.
Conditions : This symptom might be seen because by default, control-plane policing counts DHCP packets in copp-system-class-normal, which is where ARP is also classified. If there is enough constant DHCP traffic flowing through the switch, this CoPP policer might also discard valid ARP packets, possibly causing intermittent packet loss.
Symptom : When GRE tunnel(s) are configured between a Cisco Nexus 7000 Series switch and another device, the switch fails when ping is initiated to the Cisco Nexus 7000 Series switch tunnel interface IP address from the remote side of the GRE tunnel.
Conditions : This symptom might be seen when the ping for the GRE tunnel is received on a F series module. The GRE tunnel should use a source and destination loopback interface. The issue can be triggered by traffic that is destined to in-band over the GRE tunnel and switched from an F series module; however, the issue can also be triggered from an M series module.
Symptom : After adding a default static mroute on the Cisco Nexus 7000 Series switch, the route shows as hidden in the output of the show ip route rpf command. This route is not used to do RPF checks.
Symptom : By default in Cisco NX-OS, IBGP routes are redistributed into the IGP when redistribution is configured. In Cisco IOS software, the bgp redistribute-internal router bgp command is needed to redistribute the routes.
Conditions: This symptom might be seen in an extremely rare situation where the underlying hardware (either a module, a supervisor module, or the switch transmitting packets) has malfunctioned and causes timeout drops. A race condition occurs that uses a freed section of memory.
Symptom : In specific situations during bringup, a member port of a port channel with a min-link configuration can get error-disabled with the reason “undefined.” The following syslog indicates this condition:
2011 Jun 8 21:21:34 n7k-1 %$ VDC-1 %$ %ETHPORT-2-IF_SEQ_ERROR: Error ("undefined") communicating with MTS_SAP_ETH_PORT_CHANNEL_MGR for opcode MTS_OPC_ETHPM_PORT_BRINGUP (RID_PORT: Ethernet9/13) Follow PM FAQ #6 at: http://zed.cisco.com/confluence/display/KGP/Port+Manager+FAQ
Conditions : This symptom might be seen when a port (such as port-1) is in a transitory bringup state and at the same instant another member port (such as port-2) goes down in the port channel. If port-2 going down triggers a min-link condition, port-1 will also be suspended. Without this fix, the port will be error-disabled with the reason “undefined” instead of being suspended.
Symptom : A Cisco Nexus 7000 Series switch might not forward multicast streams because of a hardware issue where multicast entries are not installed in the hardware. Lack of a hardware entry can be verified with the following commands:
This queue is not expected to always be non zero. It is normal for it to be non zero. However, an indication of an issue is if the queue continues to steadily increase without decreasing. If the multicast environment is very dynamic, there is greater fluctuation in the number of entries in the queue.
For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What’s New in Cisco Product Documentation , which also lists all new and revised Cisco technical documentation, at:
Subscribe to the What’s New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)