Table of Contents
This document describes the features, caveats, and limitations for the Cisco Nexus 5500 devices and the Cisco Nexus 2000 Series Fabric Extenders. Use this document in combination with documents listed in the “Related Documentation” section.
Note Release notes are sometimes updated with new information about restrictions and caveats. See the following website for the most recent version of the Cisco Cisco Nexus 5500 and Cisco Nexus 2000 Series release notes: http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/release/notes/Nexus_5500_Release_Notes.html
Note Table 1 shows the online change history for this document.
The Cisco NX-OS software is a data center-class operating system built with modularity, resiliency, and serviceability at its foundation. Based on the industry-proven Cisco MDS 9000 SAN-OS software, Cisco NX-OS helps ensure continuous availability and sets the standard for mission-critical data center environments. The highly modular design of Cisco NX-OS makes zero-effect operations a reality and enables exceptional operational flexibility.
Several new hardware and software features are introduced for the Cisco Nexus 5500 Series device and the Cisco Nexus 2000 Series Fabric Extender (FEX) to improve the performance, scalability, and management of the product line.
The Cisco Nexus devices include a family of line-rate, low-latency, lossless 10-Gigabit Ethernet, Cisco Data Center Ethernet, Fibre Channel over Ethernet (FCoE), and native Fibre Channel devices for data center applications.
The Cisco Nexus 2000 Series Fabric Extender (FEX) is a highly scalable and flexible server networking solution that works with the Cisco Nexus 5500 Series devices to provide high-density and low-cost connectivity for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade server environments, the FEX is designed to simplify data center architecture and operations.
The FEX integrates with its parent Cisco Nexus device, which allows zero-touch provisioning and automatic configuration. The FEX provides a single point of management that supports a large number of servers and hosts that can be configured with the same feature set as the parent Cisco Nexus 5500 Series switch, including security and quality of service (QoS) configuration parameters. Spanning Tree Protocol (STP) is not required between the Fabric Extender and its parent switch, because the Fabric Extender and its parent switch allow you to enable a large multi-path, loop-free, active-active topology.
Software is not included with the Fabric Extender. Cisco NX-OS software is automatically downloaded and upgraded from its parent switch. For information about configuring the Cisco Nexus 2000 FEX, see the “Configuring the Fabric Extender” chapter in the Cisco Nexus 5500 Series NX-OS Layer 2 Switching Configuration Guide, Release 7.x .
The Cisco NX-OS software supports the Cisco Nexus devices. Starting with Cisco NX-OS Release 7.0(0)N1(1), the Cisco Nexus 5010 and 5020 switches are not supported. You can find detailed information about supported hardware in the Cisco Nexus 5500 Series Hardware Installation Guide .
Table 2 shows the hardware supported by Cisco NX-OS Release 7.x software.
Cisco Nexus 5596T switch1
Cisco Nexus B22DELL FEX2
Cisco Nexus 2248PQ FEX5
Cisco Nexus 2232TM-E FEX6
Cisco Nexus B22HP FEX7
12-port 10GBASE-T GEM11
Gigabit Ethernet SFP, LH transceiver12
Gigabit Ethernet SFP, EX transceiver13
10GBASE CU SFP+ cable14
10GBASE CU SFP+ cable15
10GBASE CU SFP+ cable16
Generation-1 (Pre-FIP) CNAs17
Table 3 shows the hardware and Cisco NX-OS Release 7.x software that supports online insertion and removal (OIR).
Layer 3 GEM 1
Version 2 Layer 3 daughter card 1
N5596 Layer 3 GEM 1
N5548 Layer 3 daughter card 1
- New Software Features in Cisco NX-OS Release 7.0(5)N1(1)
- New Software Features in Cisco NX-OS Release 7.0(4)N1(1)
- New Hardware Features in Cisco NX-OS Release 7.0(4)N1(1)
- New Software Features in Cisco NX-OS Release 7.0(3)N1(1)
- New Hardware Features in Cisco NX-OS Release 7.0(3)N1(1)
- New Software Features in Cisco NX-OS Release 7.0(2)N1(1)
- New Software Features in Cisco NX-OS Release 7.0(1)N1(1)
- New Software Features in Cisco NX-OS Release 7.0(0)N1(1)
The FEX-based ACL Classification feature uses TCAM resources on a FEX to perform ACL-based packet classification of incoming packets on the switch. When QoS policies are processed on a FEX, the policies are enforced on the switch and on the associated FEX or FEXes.
This software release is the first release to support Cisco's Evolutionary Data Center Fabric solution called Dynamic Fabric Automation (DFA). DFA is evolutionary and is based on the industry leading Unified Fabric solution.
DFA focuses on simplifying, optimizing and automating data center fabric environments by offering an architecture based on four major pillars namely Fabric Management, Workload Automation, Optimized Networking and Virtual Fabrics. Each of these pillars provide a set of modular functions which can be used together or independently for easiness of adoption of new technologies in the data center environment.
Complete details on the DFA architecture can be found at: http://www.cisco.com/go/dfa.
Dynamic FCoE extends the capability and reliability of storage networks by leveraging FabricPath technology to create logical separation of SAN A and SAN B. FCoE VFCs and Interswitch-Links (ISLs) are dynamically configured, simplifying multihop FCoE deployments in leaf-spine topologies.
- Anycast Routing
- Early Warning for Forwarding Information Base Exhaustion
- Explicit Congestion Notification with Weighted Random Early Detection
- FabricPatch Operations, Administration, and Management
- Fibre Channel and Fibre Channel Over Ethernet Slow Drain
- Multi-Destination Tree
- OpenFlow v1.0
- Overload Bit
- Port Channel Max Links
- Switch Port Analyzer with Access Control List Filtering
- TCAM Carving
A Multi-Destination Tree (MDT), also referred to as a forwarding tag or ftag, is a spanning-tree used for forwarding packets within a topology. By default, a topology has two MDTs/ ftags: topology 0 has ftag 1 and 2, topology 1 has ftag 3 and 4, topology 2 has ftag 5 and 6, up to a maximum supported 64 topologies.
The OpenFlow feature is a specification from the Open Networking Foundation (ONF) that defines a flow-based forwarding infrastructure (L2-L4 Ethernet switch model) and a standardized application programmatic interface (protocol definition) to learn capabilities, add and remove flow control entries and request statistics. OpenFlow allows a controller to direct the forwarding functions of a switch through a secure channel.
Support has been added for One Platform Kit (onePK) Turbo API. OnePK is a cross-platform API and software development kit that enables you to develop applications that interact directly with Cisco networking devices. onePK provides you access to networking services by using a set of controlled APIs that share the same programming model and style. For more information, see the following URL:
Intermediate System to Intermediate System (IS-IS) uses the overload bit to tell other routers not to use the local router to forward traffic but to continue routing traffic destined for that local router.
The Switch Port Analyzer (SPAN) with Access Control List (ACL) filtering feature allows you to filter SPAN traffic so that you can reduce bandwidth congestion. To configure SPAN with ACL filtering, you use ACL’s for the session to filter out traffic that you do not want to span. An ACL is a list of permissions associated to any entity in the system; in the context of a monitoring session, an ACL is a list of rules which results in spanning only the traffic that matches the ACL criteria, saving bandwidth for more meaningful data. The filter can apply to all sources in the session.
- When a Layer 3 license is installed, the Cisco Nexus 5500 Platform does not support an ISSU. Hot swapping a Layer 3 module, for example, the Layer 3 GEM (N55-M160L3-V2) or Version 2 Layer 3 daughter card (N55-D160L3-V2), is not supported. You must power down the Cisco Nexus device before removing or inserting a Layer 3 expansion module.
Table 4 shows the upgrade and downgrade possibilities for Cisco NX-OS Release 7.0(5)N1(1). For more information, see the Cisco Nexus 5500 Series NX-OS Software Upgrade and Downgrade Guide, Release 7.0.
- Ingress inter-VLAN-routed Layer3 multicast packets are treated as “unknown multicast” by the storm-control feature. This is due to the Layer 3 forwarding design in the Cisco Nexus 5500 Series switch. For details, see CSCuh34068.
- The Server Virtualization Switch (SVS) connection is not deleted during a rollback when NIV is enabled. To resolve this issue, delete the current SVS connection and reapply the original SVS connection. For details, see CSCts17033.
- If SPAN traffic is rate-limited by entering the switchport monitor rate-limit 1G command, then a maximum transmission unit (MTU) truncation size cannot be used to truncate SPAN packets. For details, see CSCua05799.
- When an FC SPAN destination port is changed from SD to F mode and back to SD mode on an NPV switch, the port goes into an error-disabled state. Perform a shut/no-shut after the mode change recovers the port. This issue occurs only in NPV mode. For details, see CSCtf87701.
- If you configure a Cisco Nexus 2248TP port to 100 Mbps instead of autonegotiation, then autonegotiation does not occur, which is the expected behavior. Both sides of the link should be configured to both hardwired speed or both autonegotiate.
- Given the implementation of a single CPU ISSU, the STP root on the PVST region with switches on an MST region is not supported. The PVST simulation on the boundary ports goes into a PVST SIM inconsistent blocked state that breaks the STP active path. To work around this issue, move all STP roots to the MST region. However, the workaround causes a nondisruptive ISSU to fail because nonedge designated forwarding ports are not allowed for an ISSU. For additional information, see CSCtf51577.
- IGMP queries sent in CSCtf94558 are group-specific queries that are sent with the destination IP/MAC address as the group's address.
Group-specific queries are not forwarded to ports other than the one that joined the group during ISSU. The reason to forward group-specific queries toward hosts is to avoid having them leave the group. However, if a port has not joined the group, then this is not an issue. If there is an interface that has joined the group, the queries are expected to make it to the host. While the behavior is different when ISSU is not occurring, it is sufficient and works as expected and there is no impact to the traffic. For details, see CSCtf94558.
- When a private VLAN port is configured as a TX (egress) SPAN source, the traffic seen at the SPAN destination port is marked with the VLAN of the ingressed frame. There is no workaround.
- In large-scale configurations, some Cisco Nexus 2000 Series Fabric Extenders might take up to 3 minutes to appear online after entering the reload command. A configuration can be termed large-scale when the maximum permissible Cisco Nexus 2000 Series Fabric Extenders are connected to a Cisco Nexus device, all host-facing ports are connected, and each host-facing interface has a large configuration (that supports the maximum permissible ACEs per interface).
- Egress scheduling is not supported across the drop/no-drop class. Each Fabric Extender host port does not support simultaneous drop and no drop traffic. Each Fabric Extender host port can support drop or no drop traffic.
- The Cisco Nexus 2148 Fabric Extender does not support frames with the dot1q vlan 0 tag.
- VACLs of more than one type on a single VLAN are unsupported. Cisco NX-OS software supports only a single type of VACL (either MAC, IPv4, or IPv6) applied on a VLAN. When a VACL is applied to a VLAN, it replaces the existing VACL if the new VACL is a different type. For instance, if a MAC VACL is configured on a VLAN and then an IPv6 VACL is configured on the same VLAN, the IPv6 VACL is applied and the MAC VACL is removed.
- A MAC ACL is applied only on non-IP packets. Even if there is a match eth type = ipv4 statement in the MAC ACL, it does not match an IP packet. To avoid this situation, use IP ACLs to apply access control to the IP traffic instead of using a MAC ACL that matches the EtherType to IPv4 or IPv6.
- Multiple boot kickstart statements in the configuration are not supported.
- If you remove an expansion module with Fibre Channel ports, and the cable is still attached, the following FCP_ERRFCP_PORT errors appear:2008 May 14 15:55:43 switch %KERN-3-SYSTEM_MSG: FCP_ERRFCP_PORT: gat_fcp_isr_ip_fcmac_sync_intr@424, jiffies = 0x7add9a:Unknown intr src_id 42 - kernel2008 May 14 15:55:43 switch %KERN-3-SYSTEM_MSG: FCP_ERRFCP_PORT: gat_fcp_isr_ip_fcmac_sync_intr@424, jiffies = 0x7add9a:Unknown intr src_id 41 - kernel
- If you configure Multiple Spanning Tree (MST) on a Cisco Nexus device, we recommend that you avoid partitioning the network into a large number of regions.
- By design, vEth interfaces do not share the underlying behavior of a vPC port. As a result, a VLAN does not get suspended when the peer switch suspends it. For example, when you shut a VLAN on a primary switch, the VLAN continues to be up on the secondary switch when the vEth interface is on a FEX. When the VLAN on the primary switch goes down, the VLAN on the vEth interface on the primary is suspended, but the vEth on the secondary switch is up because it is an active VLAN on the secondary switch.
- Role-based Access Control List (RBACL) policy enforcement is performed on VLANs on which Cisco Trusted Security (CTS) enforcement is not configured. This situation occurs when there is at least one VLAN in the switch where CTS is enforced. On a VLAN where CTS is not enforced, RBACL policy lookup occurs for ingress packets and the packet is denied or permitted according to the policies in the system. To work around this issue, make sure that all VLANs on which SGT tagged packets ingress enforce CTS.
- The packet length in the IP GRE header of a packet exiting from the switch is not equal to the MTU value configured in the ERSPAN source session. This is true for SPAN or ERSPAN. This situation can occur whenever the MTU value that is configured in an ERSPAN or SPAN session is smaller than the SPAN packet, such as when the packet is truncated. The IP GRE packet is truncated to a value that differs by –2 to 10 bytes from the expected MTU.
- When you configure a Layer 3 interface as an ERSPAN source, and configure the ERSPAN termination on a Catalyst 5500 switch or a Cisco Nexus 7000 Series switch, you cannot terminate the Layer 3 interface ERSPAN source on the Cisco Nexus 7000 Series switch or the Catalyst 5500 switch. To work around this issue, configure VLAN 1 to 512 on the Cisco Nexus 7000 Series switch or the Catalyst 6000 switch.
- Unknown unicast packets in FabricPath ports are counted as multicast packets in interface counters. This issue occurs when unknown unicast packets are sent and received with a reserved multicast address (that floods to a VLAN) in the outer FabricPath header, and the Cisco Nexus device increments the interface counter based on the outer FabricPath header. As a result, multicast counters are incremented. In the case of a Cisco Nexus 7000 Series switch, unicast counters are incremented as they are based on an inner Ethernet header. There is no workaround for this issue.
- In an emulated switch setup, inband keepalive does not work. The following steps are recommended for peer keepalive over switch virtual interface (SVI) when a switch is in FabricPath mode:
- FabricPath requires 802.1Q tagging of the inner Ethernet header of the packet. Native VLAN packets that are sent by a Cisco Nexus 7000 Series switch are not tagged. As a result, a Cisco Nexus device drops packets due to packet parsing errors. To work around this issue, enter the vlan dot1q tag native command on the Cisco Nexus 7000 Series switch to force 802.1Q tagging of native VLAN packets.
- A nondisruptive ISSU is not supported when ingress policing is configured.
- The maximum IP MTU that can be set on Layer 3 interfaces on which Layer 3 protocols are running is 9196, because of the internal header used inside the switch. The network-qos policy must be set to 9216.
- On a Cisco Nexus device, if the SPAN source is a FEX port, the frames will always be tagged when leaving the SPAN destination.
- On a Cisco Nexus 5500 Platform switch, if the SPAN source is on an access port on the switch port, the frames will not be tagged when leaving the SPAN destination.
- Ports on a FEX can be configured as a tx-source in one session only.
If two ports on the same FEX are enabled to be tx-source, the ports need to be in the same session. If you configure a FEX port as a tx-source and another port belonging to the same FEX is already configured as a tx-source on a different SPAN session, an error is displayed on the CLI.
- When a FEX port is configured as a tx-source, the multicast traffic on all VLANs for which the tx-source port is a member, is spanned. The FEX port sends out only multicast packets that are not filtered by IGMP snooping. For example, if FEX ports 100/1/1–12 are configured on VLAN 11 and the switch port 1/5 sends multicast traffic on VLAN 11 in a multicast group, and hosts connected to FEX ports 100/1/3-12 are interested in receiving that multicast traffic (through IGMP), that multicast traffic goes out on FEX ports 100/1/3–12, but not on 100/1/1–2.
If you configure SPAN Tx on port 100/1/1, although the multicast traffic does not egress out of port 100/1/1, the SPAN destination does receive that multicast traffic, which is due to a design limitation.
- When a FEX port is configured as both SPAN rx-source and tx-source, the broadcast, non-IGMP Layer-2 multicast, and unknown unicast frames originating from that port might be seen twice on the SPAN destination: once on the ingress and once on the egress path. On the egress path, the frames are filtered by the FEX to prevent them from going out on the same port on which they were received. For example, if FEX port 100/1/1 is configured on VLAN 11 and is also configured as SPAN rx-source and tx-source and a broadcast frame is received on that port, the SPAN destination recognizes two copies of the frame, even though the frame is not sent back on port 100/1/1.
- A FEX port cannot be configured as a SPAN destination. Only a switch port can be configured and used as a SPAN destination.
In a vPC topology, two Cisco Nexus devices configured as vPC peer switches need to be configured symmetrically for Layer 3 configurations such as SVIs, the peer gateway, routing protocol and policies, and RACLs.
When a Layer 3 module goes offline, all non-management SVIs are shut down. To maintain connectivity when a Layer 3 module fails, you can configure an SVI as a management SVI using the command management under i nterface vlan . This prevents traffic to the management SVI from passing through the failed Layer 3 module.
- Open Caveats
- Resolved Caveats in Cisco NX-OS Release 7.0(5)N1(1)
- Resolved Caveats in Cisco NX-OS Release 7.0(4)N1(1)
- Resolved Caveats in Cisco NX-OS Release 7.0(3)N1(1)
- Resolved Caveats in Cisco NX-OS Release 7.0(2)N1(1)
- Resolved Caveats in Cisco NX-OS Release 7.0(1)N1(1)
- Resolved Caveats in Cisco NX-OS Release 7.0(0)N1(1)
Table 5 lists descriptions of open caveats in Cisco NX-OS Release 7.0(2)N1(1).
The Cisco Management Information Base (MIB) list includes Cisco proprietary MIBs and many other Internet Engineering Task Force (IETF) standard MIBs. These standard MIBs are defined in Requests for Comments (RFCs). To find specific MIB information, you must examine the Cisco proprietary MIB structure and related IETF-standard MIBs supported by the Cisco Nexus 5500 Series switch.
- Licensing Information Guide
- Release Notes
- Installation and Upgrade Guides
- Configuration Guides
- Configuration Examples and TechNotes
- Programming Guides
- Operations Guides
- Error and System Message Guides
- Field Notices
- Security Advisories, Responses and Notices
- Troubleshooting Guide
- Command References
- MIB Reference Guide
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)