Cisco Nexus 9000 Series NX-OS Release Notes, Release 6.1(2)I2(1)
Publication Date: March 17, 2014 Part Number: OL-31713-01 C0 Current Release: Release 6.1(2)I2(1)
This document describes the features, caveats, and limitations for Cisco NX-OS Release 6.1(2)I2(1) software for use on the Cisco Nexus 9000 Series switches. Use this document in combination with documents listed in the Related Documentation.
Table 1 shows the online change history for this document.
Table 1 Online History Change
March 17, 2014
Created the release notes for Release 6.1(2)I2(1).
Cisco NX-OS software is a data center-class operating system designed for performance, resiliency, scalability, manageability, and programmability at its foundation. The Cisco NX-OS software provides a robust and comprehensive feature set that meets the requirements of virtualization and automation in mission-critical data center environments. The modular design of the Cisco NX-OS operating system makes zero-impact operations a reality and enables exceptional operational flexibility.
The Cisco Nexus 9000 Series uses an enhanced version of Cisco NX-OS software with a single binary image that supports every switch in the series, which simplifies image management.
1.For use with the Cisco Nexus 9396 switch (N9K-C9396PX).
2.For use with the Cisco Nexus 93128 switch (N9K-C93128TX).
For additional information about the supported hardware, see the Cisco Nexus 9508 Switch Site Preparation and Hardware Installation Guide, the Cisco Nexus 9508 Switch Site Preparation and Hardware Installation Guide, and the Cisco Nexus 9300 Series Switch Site Preparation and Hardware Installation Guide.
40GBASE-LR4 QSFP+ 40G transceiver module for single mode fiber, 4 CWDM lanes in 1310-nm window muxed inside module, duplex LC connector, 10-km, 40G Ethernet rate only
40GBASE-CR4 QSFP+ to four 10GBASE-CU SFP+ direct attach breakout cable assembly, 7 meter active
40GBASE-CR4 QSFP+ to four 10GBASE-CU SFP+ direct attach breakout cable assembly, 10 meter active
40GBASE-CR4 QSFP+ direct-attach copper cable, 5 meter passive
40GBASE-CR4 QSFP+ direct-attach copper cable, 3 meter passive
40GBASE-CR4 QSFP+ direct-attach copper cable, 1 meter passive
40GBASE-CR4 QSFP+ direct-attach copper cable, 7 meter active
40GBASE-CR4 QSFP+ direct-attach copper cable, 10 meter active
10GBASE-SR SFP+ module
10GBASE-LR SFP+ module
10GBASE-CU SFP+ cable 1 meter
10GBASE-CU SFP+ cable 3 meter
10GBASE-CU SFP+ cable 5 meter
Active Twinax cable assembly, 7 meter
Active Twinax cable assembly, 10 meter
GE SFP, LC connector SX transceiver
GE SFP, LC connector LX/LH transceiver
Note For the current release, if you are using the four 10G breakout cables with a Cisco Nexus 9000 Series switch, all ports on the I/O module must be set to breakout mode. A maximum of three l/O modules can be placed in breakout mode.
New and Changed Information
This section lists the new and changed features in Release 6.1(2)I2(1), and includes the following topics:
– AC power supplies (up to four 3-kW AC power supplies (N9K-PAC-3000W-B)
The Cisco Nexus 9396PX switch (N9K-C9396PX) is a two-rack-unit (RU) switch that supports 960 Gbps of bandwidth across 48 fixed 10-Gbps SFP+ ports and 12 40-Gbps QSFP+ ports. The 40-Gbps ports are provided on the uplink module.
The Cisco Nexus 93128TX switch (N9K-C93128TX) is a 3RU switch that supports 1.28 Tbps across 96 fixed 1-/10-Gbps BASE-T ports and eight 40-Gbps QSFP uplink ports. The 1-/10-Gigabit BASE-T ports also support a speed of 100-Megabits. The 40-Gbps ports are the leftmost eight ports provided on the uplink module.
The Cisco Nexus 9300 uplink module (N9K-M12PQ) can be serviced and replaced by the user and is the same for both Cisco Nexus 9300 switches.
The Cisco Nexus 9500 Series 48-port, 1-/10-Gbps SFP+ plus 4-port QSFP+ I/O module (N9K-X9564PX) is designed for use with Cisco Nexus 2000 Series Fabric Extenders and for mixed 10 and 40 Gigabit Ethernet aggregation. It supports both direct-attach 10 Gigabit Ethernet copper cabling and optical transceivers and offers 4 QFSP+ ports that provide 40 Gigabit Ethernet server access, uplink, and downlink capacity. This module can be used in application centric infrastructure (ACI) leaf configurations.
The Cisco Nexus 9500 Series 48-port 1-/10-Gbps BASE-T plus 4-port QSFP+ I/O module (N9K-X9564TX) is designed for end-of-row (EoR) and middle-of-row (MoR) environments. It supports 100 Megabit Ethernet, 1 Gigabit Ethernet, and 10-Gbps BASE-T copper cabling connectivity for server access, and offers 4 QFSP+ ports that provide 40 Gigabit Ethernet server access, uplink, and downlink capacity. This module can be used in ACI leaf configurations.
Hardware rate limiters for catch-all exception traffic, Layer 3 glean packets, and Layer 3 multicast data packets.
New Software Features in Cisco NX-OS Release 6.1(2)I2(1)
The Cisco NX-OS Release 6.1(2)I2(1) supports the software features listed in this section.
Switch Virtual Interface (SVI), interface VLANs
VLAN Trunking Protocol (VTP) transparent mode
Spanning Tree Protocol (STP) features:
– STP Rapid PVST+ (Per VLAN Spanning Tree +)
– STP Multiple Spanning Tree (MST)
– STP Extensions, including bridge protocol data unit (BPDU) guard, Bridge Assurance, loop guard, and root guard
Layer 2 features:
– L2 storm control
– L2 switch port interfaces
– L2 portchannels
vPCs, including the following vPC features:
– Peer switch
– Peer Gateway
– ARP Synchronization
– ND Synchronization
– Reload delay
– IGMP join sync
Hot Standby Router Protocol (HSRP)
Virtual Router Redundancy Protocol (VRRP)
Port access control lists (PACL), VLAN access control lists (VACL)
Virtual Extensible Local Area Network (VXLAN) with a virtual port channel (vPC)
VXLAN gateway functionality on Cisco Nexus 9300 Series switches.
VXLAN bridging functionality on Cisco Nexus 9300 Series switches.
Enhanced object tracking
Access ports and trunk ports
VLAN as a SPAN source (for Rx only)
Software maintenance upgrades (SMUs)
Cisco Data Center Network Manager (DCNM)
Only one software image (called nx-os) is required to load the Cisco NX-OS operating system. This image runs on all Cisco Nexus 9000 Series switches. For installation instructions, see the Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide.
This section lists steps that you might need to take to upgrade your software if you already have a Cisco NX-OS image running on a Cisco Nexus 9000 Series switch. Read this entire section to determine if these steps apply to you. If they do not, follow the installation instructions in the Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide.
Note A software downgrade from Cisco NX-OS Release 6.1(2)I2(1) to Cisco NX-OS Release 6.1(2)I1(1) is not supported.
A software upgrade from Cisco NX-OS Release 6.1(2)I1(1) to Cisco NX-OS Release 6.1(2)I2(1) will be disruptive.
If you have a Cisco Nexus 9000 Series switch that is running Cisco NX-OS Release 6.1(2)I2(1) build 6.1(2)I2(0.206) or any earlier build of 6.1(2)I2(0.xxx), follow the special upgrade steps presented here.
Note If you have a Cisco Nexus 9508 switch that is running the n9000-dk22.214.171.124.I1.1.bin image from November 18, 2013, you do not need to enter the write erase command in Step 5. We do recommend upgrading EPLD images, but it is not mandatory.
To determine the date of the software image on the switch, enter the dir bootflash: command or the show version command which shows the date the image was compiled.
Step 1 Copy the n9000-dk126.96.36.199.I2.1.bin image to bootflash.
Step 2 Change the boot variables to the NX-OS image by entering the following commands:
Step 3 Enter the copy running-config startup-config command to set the startup boot variables to the NX-OS image.
Step 4 Copy the running-configuration file to a backup-configuration file to ensure that you load the running configuration after you make the upgrade.
Step 5 Enter the write erase command. The boot variables remain set.
Step 6 Enter the reload command.
On a Cisco Nexus 9508 or 9504 switch with dual supervisors, an “Autocopy in progress” message appears when you enter the reload command. Enter No and wait for the auto copy operation to finish.
Step 7 Wait 2 minutes after the reload for all modules to come online before proceeding to the next step.
Step 8 Enter the install all nxos bootflash:n9000-dk188.8.131.52.I2.1.bin command to upgrade the BIOS. The chassis will reboot at the completion of this step. Do not attempt to reboot or power off the chassis during this operation. Wait 2 minutes after the reload for all the modules to come online before proceeding to the next step.
Step 9 Enter the install epld bootflash:n9000-epld.6.1.2.I2.1.gimg module all command to upgrade the EPLD. The chassis will reboot automatically.
Step 10 Enter the install epld bootflash:n9000-epld.6.1.2.I2.1.gimg module all golden command to upgrade the golden EPLD. The chassis will reboot automatically.
Step 11 Restore the configuration that you saved in Step 4 from the bootflash. If this is a fresh install or upgrade, you do not need to restore the previous running configuration.
Note If you perform a software maintenance upgrade (SMU) and later upgrade your device to a new Cisco NX-OS software release, the new image will overwrite both the previous Cisco NX-OS release and the SMU package file.
This section lists limitations related to Cisco NX-OS Release 6.1(2)I2(1).
The uplink module should not be removed from a Cisco 9300 switch that is running Cisco NX-OS Release 6.1(2)I2(1). The ports on the uplink module should be used only for uplinks.
The Cisco ALE port is not able to strip the dot1q tag for native VLANs, so packets will carry the dot1q tag for native VLANs.
The N9K-M12PQ GEM module front panel ports do not support Auto negotiation with copper cables.
GOLD Port loopback tests are not supported.
The ASIC Memory-NS test is not applicable for N9K-X9636PQ line card and will be removed in future releases for the N9K-X9636PW line card. The test is also shown incorrectly for the N9K-X9636PQ line card. The test is applicable only for the N9K-X9564PX and N9K-X9564TX line cards.
802.1Q user-priority tagged frames (VLAN 0 with a 802.1p CoS value in the VLAN tag) are dropped on Cisco ALE access port on TOR switches.
A Cisco ALE trunk port sends out tagged packets on the native VLAN of the port. Normally, untagged packets are sent on the native VLAN.
On the Cisco Nexus 9300 Series switches with the N9K-X9636PQ and N9K-X9636TX line cards, there is no support for PFC, and no WRED and ECN support.
The N9K-M12PQ GEM module front panel ports do not support Auto negotiation with copper cables.
Cisco NX-OS Release 6.1(2)I2(2b) supports flooding for Microsoft Network Load Balancing (NLB) unicast mode on Cisco Nexus 9500 Series switches but not on Cisco Nexus 9300 Series switches. NLB is not supported in max-host system routing mode. NLB multicast mode is not supported on Cisco Nexus 9500 or 9300 Series switches.
Note To work around the situation of Unicast NLB limitation, we can statically hard code the ARP and MAC address pointing to the correct interface. Please refer to bug ID CSCuq03168 in detail in the “Open Caveats—Cisco NX-OS Release 6.1” section.
When routed ACL is applied to multiple SVIs (switched virtual interfaces) in the egress direction, TCAM resources are not shared.
When VACL (VLAN ACL) is applied to multiple VLANs, TCAM resources are not shared.
N9K hardware does not support range checks (layer 4 operators) in egress TCAM. Because of this, ACL/QoS policies with layer 4 operations-based classification need to be expanded to multiple entries in the egress TCAM. Egress TCAM space planning should take this limitation into account.
If the same QOS policy and ACL is applied on multiple interfaces, the label will be shared only when the qos-policy is applied with the no-stats option.
Limitations for ALE uplink ports are listed at the following URL:
This section lists features that are not supported in the current release.
This section lists VXLAN features that are not supported.
VXLAN routing is not supported.
The default Layer 3 gateway for VXLAN VLANs should be provisioned on a different device.
Switch virtual interface (SVI) is not supported on VXLAN VLANs.
VXLAN Layer 3 uplinks are not supported on a nondefault virtual routing and forwarding (VRF) instance.
Switched Port Analyzer (SPAN) Tx for VXLAN traffic is not supported for the access to the network direction.
RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs cannot be used on decapsulated packets in the network-to-access direction on the inner payload. As a best practice, use PACLs/VACLs for the access-to-network direction.
QoS classification is not supported for VXLAN traffic in the network-to-access direction.
The QoS buffer-boost feature is not applicable for VXLAN traffic.
Access control list (ACL) and quality of service (QoS) for VXLAN traffic in the network-to-access direction is not supported.
There is no uplink SVI support. As a best practice, use the Layer 3 port-channel uplinks/equal cost multi path (ECMP) uplinks instead.
There is no native VLAN support for VXLAN. All traffic on VXLAN Layer 2 trunks need to be tagged.
Consistency checkers are not supported for VXLAN tables.
Just one network virtualization edge (NVE) interface is allowed on the switch.
Because the NVE (VXLAN) process is not restartable, patching support is not supported for VXLAN.
Per-VNI statistics and per-VTEP statistics are not supported. Only aggregate statistics are available.
vPC type-1 consistency checkers are not supported for VXLAN configurations.
Dynamic re-IP of an NVE tunnel is not supported. Tunnels must be shut down prior to live IP address changes.
VXLAN Topology Restrictions
A device cannot be a VXLAN gateway mode (vxlan-vlan flows) and a VXLAN bridging mode (vxlan-vxlan flows) for the same multicast groups, which are also called the bud-node topology. As a best practice, use the device as either a bridging device or a gateway device, but not both.
Due to bud node restrictions, a VXLAN tunnel endpoint (VTEP) cannot reach the rendezvous point (RP) through another VTEP. Because of this limitation, there can be no direct Layer 3 links between two VTEPs, unless one of the VTEPs is the RP.
VXLAN ACL Limitations
The following ACL related features are not supported:
Ingress router access control list (RACL) that is applied on an uplink Layer 3 interface that matches on the inner or outer payload in the network-to-access direction (decapsulated path)
Egress RACL that is applied on an uplink Layer 3 interface that matches on the inner or outer payload in the access-to-network direction (encapsulate path)
Egress VACL for decapsulated VXLAN traffic
We recommend that you use a port access control list (PACL)/VACL on the access side to filter out traffic entering the overlay network.
Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide
Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide
Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide
Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide
Cisco Nexus 9000 Series NX-OS Multicast Routing Configuration Guide
Cisco Nexus 9000 Series NX-OS Quality of Service Configuration Guide
Cisco Nexus 9000 Series NX-OS Security Configuration Guide
Cisco Nexus 9000 Series NX-OS System Management Configuration Guide
Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide
Cisco Nexus 9000 Series NX-OS Verified Scalability Guide
Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide
Other Software Documents
Cisco Nexus 7000 Series and 9000 Series NX-OS MIB Quick Reference
Cisco Nexus 9000 Series NX-OS Programmability Guide
Cisco Nexus 9000 Series Software Upgrade and Downgrade Guide
Cisco Nexus 9000 Series NX-OS System Messages Reference
Cisco Nexus 9000 Series NX-OS Troubleshooting Guide
Cisco NX-OS Licensing Guide
Cisco NX-OS XML Interface User Guide
Cisco Nexus 9396 Switch Site Preparation and Hardware Installation Guide
Cisco Nexus 93128 Switch Site Preparation and Hardware Installation Guide
Cisco Nexus 9504 Switch Site Preparation and Hardware Installation Guide
Cisco Nexus 9508 Switch Site Preparation and Hardware Installation Guide
Cisco Nexus 9000 Series FPGA/EPLD Upgrade Release Notes
Cisco Nexus 9000 Series NX-OS Release Note s
To provide technical feedback on this document, or to report an error or omission, please send your comments to firstname.lastname@example.org. We appreciate your feedback.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation and gathering additional information, see the monthly What’s New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:
Subscribe to the What’s New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.
This product includes cryptographic software written by Eric Young (email@example.com). This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/). This product includes software written by Tim Hudson (firstname.lastname@example.org).
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)
Cisco Nexus 9000 Series NX-OS Release Notes, Release 6.1(2)I2(1)