The Cisco NX-OS software is a data center-class operating system built with modularity, resiliency, and serviceability at its foundation. Based on the industry-proven Cisco NX-OS software, Cisco NX-OS helps ensure continuous availability and sets the standard for mission-critical data center environments. The highly modular design of Cisco NX-OS makes zero-effect operations a reality and enables exceptional operational flexibility.
Several new hardware and software features are introduced for the Cisco Nexus 6000 Series device and the Cisco Nexus 2000 Series Fabric Extender (FEX) to improve the performance, scalability, and management of the product line.
Cisco Nexus 6000 Series Devices
The Cisco Nexus 6000 Series includes 10- and 40-Gigabit Ethernet density in energy-efficient compact form factor switches. The Cisco Nexus 6000 Series Layer 2 and Layer 3 set allow for multiple scenarios such as direct-attach 10- and 40-Gigabit Ethernet access and high-density Cisco Fabric Extender (FEX) aggregation deployments, leaf and spine architectures, or compact aggregation to build scalable Cisco Unified Fabric in the data centers.
Cisco Nexus 6000 Series products use the same set of Cisco application-specific integrated circuits (ASICs) and a single software image across the products within the family, which offers feature consistency and operational simplicity. Cisco Nexus 6000 Series switches support robust Layer 2 and Layer 3 functions, industry-leading FEX architecture with Cisco Nexus 2000 and Cisco Nexus B22 Blade FEX, in-service software upgrades (ISSUs), and Cisco FabricPath. Operational efficiency and programmability are enhanced on the Cisco Nexus 6000 Series through advanced analytics, PowerOn Auto Provisioning (POAP), and Python/Tool Command Language (Tcl) scripting.
For information about the Cisco Nexus 6000 Series, see the Cisco Nexus 6000 Series Platform Hardware Installation Guide.
Cisco Nexus 2000 Series Fabric Extenders
The Cisco Nexus 2000 Series Fabric Extender (FEX) is a highly scalable and flexible server networking solution that works with the Cisco Nexus 6000 Series devices to provide high-density and low-cost connectivity for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, and 40-Gigabit Ethernet, unified fabric, rack, and blade server environments, the FEX is designed to simplify data center architecture and operations.
The FEX integrates with its parent Cisco Nexus device, which allows zero-touch provisioning and automatic configuration. The FEX provides a single point of management that supports a large numbers of servers and hosts that can be configured with the same feature set as the parent Cisco Nexus 6000 Series switch, including security and quality of service (QoS) configuration parameters. Spanning Tree Protocol (STP) is not required between the Fabric Extender and its parent switch, because the Fabric Extender and its parent switch allow you to enable a large multi-path, loop-free, active-active topology.
Software is not included with the Fabric Extender. Cisco NX-OS software is automatically downloaded and upgraded from its parent switch. For information about configuring the Cisco Nexus 2000 FEX, see the “Configuring the Fabric Extender” chapter in the Cisco Nexus 6000 Series Layer 2 Switching Configuration Guide.
The Bidirectional Forwarding Detection (BFD) provides fast forwarding-path failure detection times for media types, encapsulations, topologies, and routing protocols. You can use BFD to detect forwarding path failures at a uniform rate, rather than at variable rates for different protocol hello mechanisms. BFD makes network profiling and planning easier and reconvergence time consistent and predictable.
New Software Features in Cisco NX-OS Release 6.0(2)N2(3)
There are no new software features in this release.
New Software Features in Cisco NX-OS Release 6.0(2)N2(2)
There are no new software features in this release.
New Software Features in Cisco NX-OS Release 6.0(2)N2(1)
Cisco NX-OS Release 6.0(2)N2(1) is a maintenance release that includes bug fixes and the following software features and enhancements:
Bidirectional Forwarding Detection (BFD) is a detection protocol designed to provide fast forwarding-path failure detection times for media types, encapsulations, topologies, and routing protocols. Starting with Release 6.0(2)N2(1), the Cisco Nexus 6000 supports BFD for BGP, EIGRP, OSPF, PIM, HSRP, VRRP, and static routes.
Command Update: show lldp system -detail
The show lldp command now includes an optional keyword for displaying system details.
You can use the default interface command to clear the existing configuration of multiple interfaces and return them to default settings. The command can be used for interfaces such as Ethernet, loopback, VLAN network, port-channel, and tunnel interfaces. You can use the checkpoint keyword with the command to create a copy of the interface configuration before clearing it so that you can restore it later.
Embedded Event Manager Support
The Embedded Event Manager (EEM) monitors events that occur on your device and takes action to recover from or troubleshoot these events, based on your configuration. You can use EEM to create policies that consist of a set of actions to be taken in response to a specific event. EEM can be controlled through CLI commands or Vsh scripts.
Flexible ACL Carving/TCAM Carving
You can create and administer up to 16 templates to resize the regions in ternary content-addressable memory (TCAM).
Ingress policing allows you to monitor the data rates for a particular class of traffic. When the data rate for the specified class exceeds user-configured values, the switch drops packets immediately. You can also configure policing to allow for traffic bursts.
MAC/ARP Resource Carving Template
On a Cisco Nexus 6000 Series switch, the IPv4/IPv6 and unicast/multicast entries share the same MAC (Media Access Control) and ARP (Address Resolution Protocol) tables. The software transactional memory (STM) and the host route table (HRT) also share these tables. You can use the STM/HRT template profile feature to carve out STM and HRT table sizes as needed using four pre-defined templates.
Maximum Paths for Equal-Cost Multipath Routing
For BGP, EIGRP, and OSPF routing protocols, the number of maximum paths that can be load balanced to a destination in equal-cost multipath routing (ECMP) has increased from 16 to 64.
Multicast Routing Table Maximum Entries
The maximum number of entries in the multicast routing table has increased to 16,000. This is the largest value that can be specified for the hardware profile multicast max-limit command.
Network Time Protocol Server
A Cisco Nexus 6000 Series switch can use the Network Time Protocol (NTP) to synchronize the network. Other devices can be configured to use the switch as an NTP time server. In an isolated network, the switch can be configured as an authoritative NTP clock source.
An NTP server usually receives its time from an authoritative time source, such as a radio clock or an atomic clock attached to a time server, and then distributes this time across the network. NTP is extremely efficient; no more than one packet per minute is necessary to synchronize two devices to within a millisecond of each other.
Policy Based Routing
Starting with Release 6.0(2)N2(1), the Cisco Nexus 6000 Series supports policy-based routing (PBR). PBR allows you to configure a defined policy for IPv4 and IPv6 traffic flows, lessening reliance on routes derived from routing protocols.
Up to 32 DHCP server addresses can now be configured on an interface. Previously, the maximum number of configurable server addresses was 16.
FEX Host Interface Storm Control
HIF Storm control allows ingress traffic suppression for unknown multicast, broadcast, and unknown unicast traffic on Fabric extender (FEX) Host Interface (HIF) ports and port channels.
As documented in the release specific Verified Scalability for Cisco Nexus 6000 Series documents, configuration limits (verified scalability) for several Layer 2 switching functions has increased. Verified and maximum limits have changed for some features, including:
FEX Host Interface Storm Control
IGMP Snooping groups
Logical interfaces (PVs)
Maximum FEXs per switch
Number of FEX ports
SNMP Bridge-MIB and LLDP MIB
SNMP bridge and LLDP MIBs have been published.
vPC+ Routing Protocol Peering
Added support for routing unicast and multicast protocol over vPC+.
You can use the vPC shutdown command to isolate a switch from the vPC complex. The switch can then be debugged, reloaded, or removed physically, without affecting the vPC traffic going through the nonisolated switch.
New Software Features in Cisco NX-OS Release 6.0(2)N1(2)
Cisco NX-OS Release 6.0(2)N1(2) is a maintenance release that includes bug fixes and the following software features and enhancements:
Support added for the IEEE 802.1X, which provides a client-server-based access control and authentication protocol that restricts unauthorized devices from connecting to a LAN through publicly accessible ports. The authentication server authenticates each client connected to a switch port before making available any services offered by the switch or the LAN.
FEX NIF Storm Control
The NIF Storm Control feature allows configuration on a Satellite Fabric port to all the pinned FEX HIF ports regardless of whether it is a logical or a physical HIF. In addition, a new syslog message informs the user when a switch port that has a Storm Control configuration is starting to see a storm of broadcast, multicast, or unicast when it starts dropping packets. You see another syslog message when the storm stops.
You can use an IP-directed broadcast to send a broadcast from a device that is not directly connected to the destination IP subnet. An ACL name can be specified for the broadcast. (This resolves caveat CSCuh1963.)
Support for DELL FEX
Added support for the Cisco Nexus B22 Dell Fabric Extender for Cisco Nexus 6000 Series switches starting with the 6.0(2)N1(2) release.
Port-channel Minimum Links
Added support to configure a minimum number of links for the port channel so that when a certain number of port-channel member ports go down, the host-facing interfaces are suspended.
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(7)
No new hardware features have been introduced with this release.
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(6)
No new hardware features have been introduced with this release.
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(5a)
There are no new hardware features in this release.
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(5)
There are no new hardware features in this release.
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(4)
Cisco NX-OS Release 6.0(2)N2(4) supports the following new hardware:
40G BiDi Optics
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(1b)
Cisco NX-OS Release 6.0(2)N2(1b) supports the following new hardware:
N2K-B22IBM-P—Cisco Nexus B22 Fabric Extender for IBM
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(2)
Cisco NX-OS Release 6.0(2)N2(2) supports the following new hardware:
N6K-C6004: Nexus 6004 EF chassis no power supply, no fan (requires 2 LEMs to be added), 4 RU chassis. To bring all modules up, you need six power supplies with redundancy and a minimum of three power supplies in standalone mode.
N6004-B-24Q—-Nexus 6004 EF chassis 24 x 40GE Ports/FCoE Bundle; 6 power supplies, 4 fans
N6004-M12Q—Nexus 6004 EF chassis 12Q 40GE Ethernet/FCoE
New Hardware Features in Cisco NX-OS Release 6.0(2)N2(1)
Cisco NX-OS Release 6.0(2)N2(1) supports the following new hardware:
Cisco Nexus 6001-64T—1 RU switch, fixed 48p of 10G BASE-T and 4p QSFP+ (PID:N6K-C6001-64T) -
The following guidelines apply to Cisco NX-OS Release 6.0(2)N2(7) for the Cisco Nexus devices:
If host interface (HIF) port channels or EvPCs are configured in the system, and if the system was already upgraded to NX-OS Release 5.1(3)N1(1) or Release 5.1(3)N1(1a) from any release earlier than Release 5.1(3)N1(1), ensure that the system was reloaded at least once before you upgrade to Release 5.1(3)N2(1a) or Release 5.1(3)N2(1). If the switch was not previously reloaded, reload it and upgrade to Release 5.1(3)N2(1a) or Release 5.1(3)N2(1).
When a Layer 3 license is installed, the Cisco Nexus 5500 Platform does not support an ISSU. Hot swapping a Layer 3 module, for example, the Layer 3 GEM (N55-M160L3-V2) or Version 2 Layer 3 daughter card (N55-D160L3-V2), is not supported. You must power down the Cisco Nexus device before removing or inserting a Layer 3 expansion module.
When you are upgrading from a 6.0(x) version to 7.0(x), there is a possibility of STP disputes on upstream devices, which could lead to an outage. See CSCuo74024 for details.
Supported Upgrade and Downgrade Paths for Cisco NX-OS Release 6.0(2)N2(7)
Table 5 shows the upgrade and downgrade possibilities for Cisco NX-OS Release 6.0(2)N2(7). For more information, see the Cisco Nexus 6000 Series NX-OS Software Upgrade and Downgrade Guide, Release 6.0(2)N2(7).
For other 6.0 releases, see the Cisco Nexus 6000 Series NX-OS Software Upgrade and Downgrade Guide specific for that release at:
Note If a supported upgrade or downgrade path is not taken, then certain configurations, especially related to unified ports, Fibre Channel (FC) ports, breakout, and FEX may be lost.
Note Doing a disruptive upgrade between incompatible images can result in loss of configurations such as unified ports, Fibre Channel (FC) ports, breakout, and FEX configurations, and VLAN database (VTP mode client/server). See CSCul22703 for details.
Note If you want to upgrade from a release, that is not listed in the “Current Cisco NX-OS Release” column of Table 5 to the latest Cisco NX-OS release version, then you must first upgrade to a release that is listed in the “Current Cisco NX-OS Release” column and then to the latest release version.
This section describes the limitations for Cisco NX-OS Release 6.0(2)N2(7).
The Server Virtualization Switch (SVS) connection is not deleted during a rollback when NIV is enabled. To resolve this issue, delete the current SVS connection and reapply the original SVS connection. For details, see CSCts17033.
If you configure a Cisco Nexus 2248TP port to 100 Mbps instead of autonegotiation, then autonegotiation does not occur, which is the expected behavior. Both sides of the link should be configured to both hardwired speed or both autonegotiate.
no speed —Autonegotiates and advertises all speeds (only full duplex).
speed 1000 —Autonegotiates only for an 802.3x pause.
speed 100 —Does not autonegotiate; pause cannot be advertised. The peer must be set to not autonegotiate and fix at 100 Mbps (similar to the N2248TP). For details, see CSCte81998.
When a private VLAN port is configured as a TX (egress) SPAN source, the traffic seen at the SPAN destination port is marked with the VLAN of the ingressed frame. There is no workaround.
In large-scale configurations, some Cisco Nexus 2000 Series Fabric Extenders might take up to 3 minutes to appear online after entering the reload command. A configuration can be termed large scale when the maximum permissible Cisco Nexus 2000 Series Fabric Extenders are connected to a Cisco Nexus 6000 Series switch, all host-facing ports are connected, and each host-facing interface has a large configuration that supports the maximum permissible ACEs per interface.
The Cisco Nexus 2148 Fabric Extender does not support frames with the dot1q vlan 0 tag.
VACLs of more than one type on a single VLAN are unsupported. Cisco NX-OS software supports only a single type of VACL (either MAC, IPv4, or IPv6) applied on a VLAN. When a VACL is applied to a VLAN, it replaces the existing VACL if the new VACL is a different type. For instance, if a MAC VACL is configured on a VLAN and then an IPv6 VACL is configured on the same VLAN, the IPv6 VACL is applied, and the MAC VACL is removed.
A MAC ACL is applied only on non-IP packets. Even if there is a match eth type = ipv4 statement in the MAC ACL, it does not match an IP packet. To avoid this situation, use IP ACLs to apply access control to the IP traffic instead of using a MAC ACL that matches the EtherType to IPv4 or IPv6.
Multiple boot kickstart statements in the configuration are not supported.
If you configure Multiple Spanning Tree (MST) on a Cisco Nexus 6000 Series switch, avoid partitioning the network into a large number of regions.
By design, vEth interfaces do not share the underlying behavior of a vPC port. As a result, a VLAN is not suspended when the peer switch suspends it. For example, when you shut a VLAN on a primary switch, the VLAN continues to be up on the secondary switch when the vEth interface is on a FEX. When the VLAN on the primary switch goes down, the VLAN on the vEth interface on the primary is suspended, but the vEth on the secondary switch remains up because it is an active VLAN on the secondary switch.
The packet length in the IP GRE header of a packet exiting from the switch is not equal to the MTU value configured in the ERSPAN source session. This is true for SPAN or ERSPAN. The Cisco Nexus 6000 Series switch terminates in multiples of 16 bytes. If MTU is configured as 100 bytes, then the actual truncated packet is 96 bytes.
Unknown unicast packets in FabricPath ports are counted as multicast packets in interface counters. This issue occurs when unknown Unicast packets are sent and received with a reserved multicast address (that floods to a VLAN) in the outer FabricPath header, and the Cisco Nexus 6000 Series switch increments the interface counter based on the outer FabricPath header. As a result, Multicast counters are incremented. There is no workaround for this issue.
In an emulated switch setup, an inband keepalive does not work. The following steps are recommended for peer keepalive over SVI when a switch is in FabricPath mode:
– Use a dedicated front panel port as a vPC+ keepalive. The port should be in CE mode.
– Use a dedicated VLAN to carry the keepalive interface. The VLAN should be a CE VLAN.
– Enter the dual-active exclude interface-vlan keepalive-vlan command to prevent the SVI from going down on the secondary when a peer-link goes down.
The limit of the table that holds the Router MAC and Virtual MAC entries for determining packet routing or switching is 500 entries. The Virtual MAC entries, the MAC used for HSRP/VRRP that is also programmed in this table, can be shared across multiple Layer 3 interfaces. If SVIs 1–100 all have the same group number configured, just one entry needs to be programmed in this table. We recommend that you configure the same group ID across all or multiple Layer 3 interfaces/SVIs. If multiple group IDs are configured on an Layer 3 interface, we recommend that you configure the same set of group IDs across all or multiple Layer 3 interfaces. This configuration supports HSRP/VRRP on more interfaces.
The maximum IP MTU that can be set on Layer 3 interfaces running Layer 3 protocols is 9192 because of the internal header used inside the switch. The related network-qos policy must be set to 9216.
Limitations on the Cisco Nexus 6000
The limitations on the Cisco Nexus 6000 Series switch are as follows:
The SPAN limitations on Fabric Extender ports are as follows:
On a Cisco Nexus device, if the SPAN source is a FEX port, the frames will always be tagged when leaving the SPAN destination.
On a Cisco Nexus 6000 Series switch, if the SPAN source is an access port on a switch port or FEX port, the spanned frames at the SPAN destination will be tagged.
On a Cisco Nexus 6000 Series switch, if the SPAN source is on an access port on the switch port, the frames will not be tagged when leaving the SPAN destination.
Ports on a FEX can be configured as a tx-source in one session only.
If two ports on the same FEX are enabled to be tx-source, the ports need to be in the same session. If you configure a FEX port as a tx-source and another port belonging to the same FEX is already configured as a tx-source on a different SPAN session, an error is displayed on the CLI.
In the following example, Interface Ethernet100/1/1 on a FEX 100 is already configured as a tx-source on SPAN session-1:
swor28(config-monitor)# show running-config monitor
monitor session 1
source interface Ethernet100/1/1 tx
destination interface Ethernet1/37
If you add an interface Ethernet100/1/2 as a tx-source to a different SPAN session (session-2) the following error appears:
ERROR: Eth100/1/2: Ports on a fex can be tx source in one session only
When a FEX port is configured as a tx-source, the multicast traffic is spanned on all VLANs that the tx-source port is a member of. The FEX port sends out only multicast packets that are not filtered by IGMP snooping. For example, if FEX ports 100/1/1–12 are configured on VLAN 11 and the switch port 1/5 sends multicast traffic on VLAN 11 in a multicast group, and hosts connected to FEX ports 100/1/3–12 are interested in receiving that multicast traffic (through IGMP), then that multicast traffic goes out on FEX ports 100/1/3–12, but not on 100/1/1–2.
If you configure SPAN Tx on port 100/1/1, although the multicast traffic does not egress out of port 100/1/1, the SPAN destination does receive that multicast traffic, which is due to a design limitation.
When a FEX port is configured as both SPAN rx-source and tx-source, broadcast non-IGMP Layer-2 multicast frames as well as unknown unicast frames originating from that port might be seen twice on the SPAN destination: once on the ingress and once on the egress path. On the egress path, the frames are filtered by the FEX to prevent them from going out on the same port on which they were received. For example, if FEX port 100/1/1 is configured on VLAN 11 and is also configured as SPAN rx-source and tx-source and a broadcast frame is received on that port, the SPAN destination recognizes two copies of the frame, even though the frame is not sent back on port 100/1/1.
A FEX port cannot be configured as a SPAN destination. Only a switch port can be configured and used as a SPAN destination.
Layer 3 Limitations
In a vPC topology, two Cisco Nexus 6000 Series switches configured as vPC peer switches need to be configured symmetrically for Layer 3 configurations such as SVIs, a peer gateway, routing protocol and policies, and RACLs.
Note vPC consistency check does not include Layer 3 parameters.
This section includes the open and resolved caveats for this release. Each caveat has a link to the Bug Toolkit, where you can find details.
If eBGP adjacency is formed via the disable-connected-check command instead of the ebgp-multihop command for eBGP peers that are not directly connected, creating a new Layer 3 interface bounces the eBGP session.
In a Cisco Nexus 6000 Series switch, the IGMP Optimized Multicast Flood (OMF) feature is enabled by default, but the output of the show ip igmp snooping vlan command shows the feature as disabled for the VLAN.
Receive error messages that repeat every 10 minutes. If you see the message for any other time interval or just one time, then it is not this bug. These messages do not affect the operation of the switch. It just does not allow the switch to get the power/input/voltage characteristics of the SFP on a port to show in the “show interface x/y transceiver details” output and you will see an error of “unknown error” at the end of the transceiver details output.
In Cisco NX-OS Release 6.0(2)N1(1), the ethanalyzer detail output prints internal Cisco Nexus 6000 Series protocol headers. If ethanalyzer output is written as a.pcap file, Wireshark cannot open this.pcap file because it does not understand the headers.
The feature ACL on ip-directed broadcast needs to be documented.
The Cisco Management Information Base (MIB) list includes Cisco proprietary MIBs and many other Internet Engineering Task Force (IETF) standard MIBs. These standard MIBs are defined in Requests for Comments (RFCs). To find specific MIB information, you must examine the Cisco proprietary MIB structure and related IETF-standard MIBs supported by the Cisco Nexus 6000 Series switch.
The MIB Support List is available at the following FTP site:
Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)