Table of Contents
This document describes the features, caveats, and limitations for the Cisco Nexus 5500 devices and the Cisco Nexus 2000 Series Fabric Extenders. Use this document in combination with documents listed in the “Related Documentation” section.
Note Release notes are sometimes updated with new information about restrictions and caveats. See the following website for the most recent version of the Cisco Cisco Nexus 5500 and Cisco Nexus 2000 Series release notes: http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/release/notes/Nexus_5500_Release_Notes.html
Note Table 1 shows the online change history for this document.
Added information on the maximum IP MTU in the Doing a disruptive upgrade between incompatible images will result in loss of certain configurations such as unified ports, breakout, and FEX configurations. See CSCul22703 for details. section.
Added CSCtx42727 to the Resolved Caveats in Cisco NX-OS Release 6.0(2)N1(1)section.
Updated support for additional software features in the New Software Features in Cisco NX-OS Release 6.0(2)N1(1) section.
Added link to the Cisco Nexus 6000 Series MIB Support List in the “MIB Support” section
Added CSCul27686 to Open Caveats.
- System Requirements
- New and Changed Features
- Upgrading or Downgrading to a New Release
- Doing a disruptive upgrade between incompatible images will result in loss of certain configurations such as unified ports, breakout, and FEX configurations. See CSCul22703 for details.
- MIB Support
- Obtaining Documentation and Submitting a Service Request
The Cisco NX-OS software is a data center-class operating system built with modularity, resiliency, and serviceability at its foundation. Based on the industry-proven Cisco MDS 9000 SAN-OS software, Cisco NX-OS helps ensure continuous availability and sets the standard for mission-critical data center environments. The highly modular design of Cisco NX-OS makes zero-effect operations a reality and enables exceptional operational flexibility.
Several new hardware and software features are introduced for the Cisco Nexus 5500 Series device and the Cisco Nexus 2000 Series Fabric Extender (FEX) to improve the performance, scalability, and management of the product line. Cisco NX-OS Release 6.0 also supports all hardware and software supported in Cisco NX-OS Release 5.1, Cisco NX-OS Release 5.0.
The Cisco Nexus devices include a family of line-rate, low-latency, lossless 10-Gigabit Ethernet, Cisco Data Center Ethernet, Fibre Channel over Ethernet (FCoE), and native Fibre Channel devices for data center applications.
The Cisco Nexus 2000 Series Fabric Extender (FEX) is a highly scalable and flexible server networking solution that works with the Cisco Nexus 5500 Series devices to provide high-density and low-cost connectivity for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade server environments, the FEX is designed to simplify data center architecture and operations.
The FEX integrates with its parent Cisco Nexus device, which allows zero-touch provisioning and automatic configuration. The FEX provides a single point of management that supports a large number of servers and hosts that can be configured with the same feature set as the parent Cisco Nexus 5500 Series switch, including security and quality of service (QoS) configuration parameters. Spanning Tree Protocol (STP) is not required between the Fabric Extender and its parent switch, because the Fabric Extender and its parent switch allow you to enable a large multi-path, loop-free, active-active topology.
Software is not included with the Fabric Extender. Cisco NX-OS software is automatically downloaded and upgraded from its parent switch. For information about configuring the Cisco Nexus 2000 FEX, see the “Configuring the Fabric Extender” chapter in the Cisco Nexus 5500 Series Layer 2 Switching Configuration Guide.
The Cisco NX-OS software supports the Cisco Nexus devices. Starting with Cisco NX-OS Release 6.0(2)N1(2), the Cisco Nexus 5010 and 5020 switches are not supported. You can find detailed information about supported hardware in the Cisco Nexus 5500 Series Hardware Installation Guide.
Table 2 shows the hardware supported by Cisco NX-OS Release 6.0(x) software.
Cisco Nexus 5596T switch1
Cisco Nexus B22DELL FEX2
Cisco Nexus 2248PQ FEX5
Cisco Nexus 2232TM-E FEX6
Cisco Nexus B22HP FEX7
12-port 10GBASE-T GEM10
Gigabit Ethernet SFP, LX transceiver11
Gigabit Ethernet SFP, EX transceiver12
10GBASE CU SFP+ cable13
10GBASE CU SFP+ cable14
10GBASE CU SFP+ cable15
Generation-1 (Pre-FIP) CNAs16
Cisco Nexus 5596T switch17
Cisco Nexus B22DELL FEX18
Cisco Nexus 2248PQ FEX19
Cisco Nexus 2232TM-E FEX20
Cisco Nexus B22HP FEX21
12-port 10GBASE-T GEM22
Gigabit Ethernet SFP, LX transceiver23
Gigabit Ethernet SFP, EX transceiver24
10GBASE CU SFP+ cable25
10GBASE CU SFP+ cable26
10GBASE CU SFP+ cable27
Generation-1 (Pre-FIP) CNAs28
Table 4 shows the hardware and Cisco NX-OS Release 6.x software that supports online insertion and removal (OIR).
Layer 3 GEM 1
Version 2 Layer 3 daughter card 1
N5596 Layer 3 GEM 1
N5548 Layer 3 daughter card 1
- New Software Features in Cisco NX-OS Release 6.0(2)N2(7)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(7)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(5a)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(5)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(4)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(3)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(2)
- New Software Features in Cisco NX-OS Release 6.0(2)N2(1)
- New Software Features in Cisco NX-OS Release 6.0(2)N1(2)
- New Software Features in Cisco NX-OS Release 6.0(2)N1(1)
- New Hardware Features in Cisco NX-OS Release 6.0(2)N2(7)
- New Hardware Features in Cisco NX-OS Release 6.0(2)N2(5a)
- New Hardware Features in Cisco NX-OS Release 6/0(2)N2(5)
- New Hardware Features in Cisco NX-OS Release 6.0(2)N2(1)
- New Hardware Features in Cisco NX-OS Release 6.0(2)N1(2)
- New Hardware Features in Cisco NX-OS Release 6.0(2)N1(1)
- Bidirectional Forwarding Detection
- Command Update: show lldp system -detail
- Default Interface
- Embedded Event Manager Support
- Network Time Protocol Server
- Policy Based Routing
- DHCPv4-relay and DHCPv6-relay
- FEX Host Interface Storm Control
- Scalability Enhancements
- SNMP Bridge-MIB and LLDP MIB
- vPC Shutdown
- vPC+ Routing Protocol Peering
Bidirectional Forwarding Detection (BFD) is a detection protocol designed to provide fast forwarding-path failure detection times for media types, encapsulations, topologies, and routing protocols. Starting with Release 6.0(2)N2(1), the Cisco Nexus 5500 supports BFD for BGP, EIGRP, OSPF, PIM, HSRP, VRRP, and static routes.
You can use the default interface command to clear the existing configuration of multiple interfaces and return them to default settings. The command can be used for interfaces such as Ethernet, loopback, VLAN network, port-channel, and tunnel interfaces. You can use the checkpoint keyword with the command to create a copy of the interface configuration before clearing it so that you can restore it later.
The Embedded Event Manager (EEM) monitors events that occur on your device and takes action to recover from or troubleshoot these events, based on your configuration. You can use EEM to create policies that consist of a set of actions to be taken in response to a specific event. EEM can be controlled through CLI commands or Vsh scripts.
A Cisco Nexus 5500 switch can use the Network Time Protocol (NTP) to synchronize the network. Other devices can be configured to use the switch as an NTP time server. In an isolated network, the switch can be configured as an authoritative NTP clock source.
An NTP server usually receives its time from an authoritative time source, such as a radio clock or an atomic clock attached to a time server, and then distributes this time across the network. NTP is extremely efficient; no more than one packet per minute is necessary to synchronize two devices to within a millisecond of each other.
The Cisco Nexus 5500 now supports policy-based routing (PBR). PBR allows you to configure a defined policy for IPv4 and IPv6 traffic flows, lessening reliance on routes derived from routing protocols.
As documented in the release specific Verified Scalability for Cisco Nexus 5500 Series documents, configuration limits (verified scalability) for several Layer 2 switching functions has increased. Verified and maximum limits have changed for some features, including:
You can use the vPC shutdown command to isolate a switch from the vPC complex. The switch can then be debugged, reloaded, or removed physically, without affecting the vPC traffic going through the nonisolated switch.
Support added for the IEEE 802.1X, which provides a client-server-based access control and authentication protocol that restricts unauthorized devices from connecting to a LAN through publicly accessible ports. The authentication server authenticates each client connected to a switch port before making available any services offered by the switch or the LAN.
The NIF Storm Control feature allows configuration on a Satellite Fabric port to all the pinned FEX HIF ports regardless of whether it is a logical or a physical HIF. In addition, a new syslog message informs the user when a switch port that has a Storm Control configuration is starting to see a storm of broadcast, multicast, or unicast when it starts dropping packets. You see another syslog message when the storm stops.
You can use an IP-directed broadcast to send a broadcast from a device that is not directly connected to the destination IP subnet. An ACL name can be specified for the broadcast. (This resolves caveat CSCuh1963.)
As documented in the Cisco Nexus 5500 Series NX-OS SAN Switching Configuration Guide, Release 6.x, NICs and converged network adapters connected to a Cisco Nexus 5500 Series switch can use iSCSI as a storage protocol and can be programmed to accept the configuration values sent by the switch leveraging data center bridging exchange protocol (DCBX). DCBX negotiates configuration and settings between the switch and the adapter through a variety of type-length-values (TLV) and sub-TLVs. This process allows the switch to distribute configuration values to all attached adapters from a centralized location instead of having to manually program CoS markings on each individual server and adapter.
Policing allows you to monitor the data rates for a particular class of traffic. When the data rate exceeds user-configured values, the switch drops packets immediately. Because policing does not buffer the traffic, transmission delays are not affected. When traffic exceeds the data rate, you instruct the system to drop the packets. You can define single-rate and two-color ingress policing.
When forwarding an incoming IP packet in a line card, if the Address Resolution Protocol (ARP) request for the next hop is not resolved, the line card forwards the packets to the supervisor, referred to as glean throttling. The supervisor resolves the MAC address for the next hop and programs the hardware.
The Cisco Nexus 5500 Series device hardware has glean rate limiters to protect the supervisor from the glean traffic. If the maximum number of entries is exceeded, the packets for which the ARP request is not resolved continues to be processed in the software instead of getting dropped in the hardware.
When an ARP request is sent, the software adds a /32 drop adjacency in the hardware to prevent the packets to the same next-hop IP address from being forwarded to the supervisor. When the ARP entry is resolved, the hardware entry is updated with the correct MAC address. If the ARP entry is not resolved before a timeout period, the entry is removed from the hardware.
The ACL logging feature allows the logging of packets that hit the IPv4 ACLs. The log messages are displayed on a flow basis. The flow is identified using a combination of the IP source address, destination address, L4 protocol, and the L4 source/destination ports on an interface. The log message is generated under the following conditions:
- INFO message—When a new flow is created
- WARNING message—When the packet threshold is reached for a flow
- Configurable INFO message—At the end of a periodic interval containing information on the number of packets to hit the flow. The interval default is 5 minutes.
The Open Management Infrastructure (OMI) agent is a web server that runs on the Cisco Nexus switch. It is based on the Common Infrastructure Model (CIM) standard, which is an open standard that defines a schema for representing managed resources (for example, CPUs, disks, networks, processes, and so on.). The OMI agent enables you to perform the following operations:
- 4-port QSFP+ Nexus N55-M4Q GEM
- New power supplies for Cisco Nexus 5596T and Cisco Nexus 5596UP switches:
The new Cisco Nexus 2248TP-E Fabric Extender is a 1-RU, general purpose 100-Mb/1-G FEX that is optimized for specialized data center workloads such as data, distributed storage, distributed computing, market data, and video editing. The Cisco Nexus 2248TP-E FEX has 48x1 Gigabit Ethernet host ports and 4x10 Gigabit Ethernet uplinks. It supports all of the existing features and topologies as the Cisco Nexus 2248 and the Cisco Nexus 2148 support. In addition, the Cisco Nexus 2248TP-E offers rich counters for troubleshooting and capacity monitoring. It has a user-configurable shared buffer, and it has a per-port ingress and egress queue limit.
For detailed information about the Cisco Nexus 2248TP-E FEX, see the Cisco Nexus 2000 Series Hardware Installation Guide.
The 10-Gigabit Ethernet, extended range SFP+ module (SFP-10G-ER) supports a link length of up to 40 kilometers on standard single-mode fiber (SMF, G.652). All Cisco Nexus 5500 switches and all Cisco FEX models support the new SFP-10G-ER optic.
Cisco FabricPath is a set of multipath Ethernet technologies that combine the reliability and scalability benefits of Layer 3 routing with the flexibility of Layer 2 networks, which enables it to build scalable data centers. Cisco FabricPath offers a topology-based Layer 2 routing mechanism that provides an equal-cost multipath (ECMP) forwarding model. Cisco NX-OS Release 5.1(3)N1(1) supports one FabricPath topology.
- Allows Layer 2 multipathing in the FabricPath network.
- Provides built-in loop prevention and mitigation with no need to use the Spanning Tree Protocol (STP).
- Provides a single control plane for unknown unicast, unicast, broadcast, and multicast traffic.
- Enhances mobility and virtualization in the FabricPath network.
The FabricPath network uses the Layer 2 Intermediate System-to-Intermediate System (IS-IS) protocol to forward traffic in the network using the FabricPath headers. Layer 2 IS-IS is different than Layer 3 IS-IS; the two protocols work independently. Layer 2 IS-IS requires no configuration and becomes operational when you enable FabricPath on the device. The frames carry the same FTag that is assigned at ingress throughout the FabricPath network, and Layer 2 IS-IS allows all devices to have the same view of all the trees built by the system. Known unicast traffic uses the Equal Cost Multipath Protocol (ECMP) to forward traffic throughout the network. The system automatically load balances traffic throughout the FabricPath network by using ECMP and the trees.
Cisco FabricPath is supported on all Cisco Nexus 5500 switches (N5K-C5596UP-FA, N5K-C5548UP-FA, and N5K-C5548P-FA). The switch must be running Cisco NX-OS Release 5.1(3)N1(1). In addition, Cisco FabricPath requires the Enhanced Layer 2 license. For licensing information, see the License and Copyright Information for Cisco NX-OS Software document.
For detailed information about Cisco FabricPath, see the Cisco Nexus 5000 Series NX-OS FabricPath Configuration Guide.
The Cisco TrustSec security architecture builds secure networks by establishing clouds of trusted network devices. Cisco TrustSec also uses the device information acquired during authentication for classifying, or coloring, the packets as they enter the network. This packet classification is maintained by tagging packets on ingress to the Cisco TrustSec network so that they can be properly identified for the purpose of applying security and other policy criteria along the data path. The tag, also called the security group tag (SGT), allows the network to enforce the access control policy by enabling the endpoint device to act upon the SGT to filter traffic.
For more information about Cisco TrustSec, see the Cisco Nexus 5000 Series NX-OS Security Configuration Guide.
IEEE 1588 or Precision Time Protocol (PTP) is a time synchronization protocol for nodes distributed across a network. Its hardware timestamp feature provides greater accuracy than other time synchronization protocols such as Network Time Protocol (NTP).
A PTP system can consist of a combination of PTP and non-PTP devices. PTP devices include ordinary clocks, boundary clocks, and transparent clocks. Non-PTP devices include ordinary network switches, routers, and other infrastructure devices.
PTP is a distributed protocol that specifies how real-time PTP clocks in the system synchronize with each other. These clocks are organized into a master-member synchronization hierarchy with the grandmaster clock, the clock at the top of the hierarchy, determining the reference time for the entire system. Synchronization is achieved by exchanging PTP timing messages, with the members using the timing information to adjust their clocks to the time of their master in the hierarchy. PTP operates within a logical scope called a PTP domain.
Cisco is introducing Adapter-FEX support on the Cisco Nexus 5500 platform and on Cisco Nexus 2200 FEXes that are connected to a Cisco Nexus 5500 parent switch. The Cisco NX-OS Adapter-FEX feature provides the advantages of the FEX Link architecture with that of server I/O virtualization to create multiple virtual interfaces over a single Ethernet interface. This allows the deployment of a dual port NIC on the server and the ability to configure more than two virtual interfaces that the server sees as a regular Ethernet interface. The advantage of this approach is a reduction of power and cooling requirements and a reduction of the number of network ports.
The Adapter-FEX implementation is designed to work on a variety of FEX-capable adapters including the Cisco adapter for Cisco UCS C-Series Platform (UCS P81E VIC) and third-party adapters that implement VNTag technology. For additional, see Cisco UCS C-Series documentation.
Adapter-FEX at the access layer needs a FEX-enabled adapter in a server that connects to a parent switch that supports Adapter-FEX functionality. There are two adapters that support Adapter-FEX functionality:
For detailed information about Adapter-FEX, see the Cisco Nexus 5000 Series NX-OS Adapter-FEX Configuration Guide.
The VM-FEX is an extension of the FEX that extends to the VIC virtual interface card (VIC) in the server. It simulates ports and enables a high-speed link between the switch and the server. The VM-FEX consolidates the virtual and physical network. Each VM gets a dedicated port on the switch. In addition, the VM-FEX provides for vCenter management of Adapter-FEX interfaces.
- Policy-based VM connectivity
- Mobility of network and security properties
- A nondisruptive operation model
For more information about VM-FEX, see the Cisco Nexus 5000 Series NX-OS Layer2 Switching Configuration Guide.
The Cisco Adapter FEX with FCoE feature allows you to create an FCoE connection to a Cisco Nexus 2000 Series Fabric Extender (FEX), which can then establish an FCoE connection to a server with a virtual interface card (VIC) adapter. The switch connects to the FEX through a virtual port channel (vPC) while the FEX connects to the server using a standard FCoE link between the FEX and the VIC adapter.
- The VIC must be configured in Network Interface Virtualization (NIV) mode, which makes the two unified ports appear to the system as virtual host bus adapters (vHBAs).
- The VIC cannot be connected to the FEX through a VNP port. If this type of connection is used, NIV mode cannot be enabled on the VIC.
- The NIC mode on the Cisco UCS C-Series Rack-Mount Server must be set to active-standby.
- If you deploy FCoE over Adapter FEX on a server with a Cisco UCS P81E Virtual Interface Card (VIC) and that is running Windows 2008, you must install new versions of software drivers.
For more information about support for FCoE on Dual Homed FEX, see the Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide.
Control Plane Policing (CoPP) provides QoS-based prioritization and protection of control plane traffic that arrives at the switch in the data plane, which ensures network stability, reachability, and packet delivery.
Cisco NX-OS Release 5.1(3)N1(1) provides several predefined CoPP policies that administrators can deploy for different environments. In these predefined CoPP policies, the classification of flows is predetermined and the policing rates for the flows is fixed. In addition, there is one flexible CoPP policy for cases where predefined policies do not address the needs of the deployment.
The CoPP policies can be changed at run time like any other QoS configuration. Classification of flows is predetermined and cannot be modified. Policing rates for the flows is fixed and cannot be modified.
For additional information about CoPP, see the Cisco Nexus 5000 Series NX-OS Security Configuration Guide.
Enhanced vPC (EvPC) provides a uniform access layer for any server to any FEX in hybrid deployments. In addition, EvPC provides data, control plane, and management plane redundancy. A new vPC option allows port channel connectivity to dual-homed FEXes.
For more information about EvPC, see the Cisco Nexus 5000 Layer 2 Switching Configuration Guide.
Cisco NX-OS Release 5.1(3)N1(1) introduces the ip arp synchronize command.When this command is enabled, faster convergence of address tables between the vPC peers is possible. This convergence is designed to overcome the delay involved in ARP table restoration when the peer-link port channel flaps or when a vPC peer comes back online.
Enabling ARP synchronization improves convergence times during the restart of a vPC peer when a Cisco Nexus 5000 Series switch acts as a default gateway. By default, ARP synchronization is not enabled.
For more information about IP ARP sync, see the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide.
A switch virtual interface (SVI) is a VLAN of switch ports represented by one interface to a routing or bridging system. The SVI can be configured for routing, in which case it supports Layer 3 protocols for processing packets from all switch ports associated with the VLAN, or for in-band management of the switch.
Starting with Release 5.1(3)N1(1), the NX-OS switch has specific support for management SVIs. Having different SVIs for routing and management separates data traffic from management traffic, which can reduce competition for routing resources. If you are using an SVI for management purposes, we recommend that you specifically configure your SVI for management using the management command so that you can take advantage of this added functionality.
- Although the CLI does not prevent you from configuring routing protocols on a management SVI, we recommend that you do not configure them on management SVIs.
- Routed SVIs that are being used for management (that is, routed SVIs that have not been specifically configured for management using the management command) can still be used for management as long as the Layer 3 license is not installed.
- Management SVIs do not support configuration synchronization mode (config-sync). Configuration synchronization is performed using the mgmt 0 interface.
- RACL is not supported. Use VACL to filter the management traffic.
For more information about management SVIs, see the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide.
Encapsulated Remote Switched Port Analyzer (ERSPAN) introduces an additional level of flexibility to the powerful network monitoring capabilities of SPAN and RSPAN. ERSPAN allows the analyzer to be placed on one location and multiple switches can send mirrored traffic to this analyzer. Traffic from any port on the network on any remote switch can be analyzed without physically moving the analyzer tool.
- ERSPAN encapsulates SPAN traffic to IP-GRE frame format and allows remote monitoring traffic over an IP network.
- All Cisco Nexus 5000 Series switches, including Cisco Nexus 5500 switches, support ERSPAN.
- Cisco NX-OS Release 5.1(3)N1(1) supports an ERSPAN source session only; there is no support for an ERSPAN destination session. The Cisco Nexus 5000 Series switch hardware cannot deencapsulate an ERSPAN frame.
- ERSPN does not require a Layer 3 module and Layer 3 license.
- The Cisco Nexus 5010 and Nexus 5020 switches support two active ERSPAN sessions. The Cisco Nexus 5548P, Nexus 5548UP, and Nexus 5596UP switch support four active ERSPAN sessions.
For more information about ERSPAN, see the Cisco Nexus 5000 Series NX-OS System Management Configuration Guide.
- It reduces the overhead of Layer 3 multicast replication on a multicast router.
- It reduces the bandwidth consumption for the link between the Layer 2 switch and the multicast router.
- It reduces the multicast forwarding table size on the Layer 2 switch.
For more information about MVR, see the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide.
Port security is a simple Ethernet MAC-based security feature that can restrict input to an interface by limiting and identifying MAC addresses of the end host that are allowed to access the port. Cisco NX-OS Release 5.1(3)N1(1) adds port security to the Cisco Nexus 5000 Series and Cisco Nexus 2000 Series, and it is available on both Cisco Nexus 5000 and Nexus 5500 switches. Port security supports the following features:
- It supports both physical ports and port channels.
- It supports vPC ports for the first time in Cisco NX-OS, and only in the Cisco Nexus 5000 Series. (Port security support for vPC ports is not available in the Cisco Nexus 7000 Series, although the port security feature itself is supported on that platform.)
- It supports EvPC ports.
- It does not support NIV ports.
For additional information about the port security feature, see the Cisco Nexus 5000 NX-OS Security Configuration Guide.
Beginning with the Cisco NXOS Release 5.1(3)N1(1) release, Cisco Nexus 5000 switches support FCoE on Enhanced vPC (eVPC). In a topology that uses FCoE with eVPC, the SAN fabrics must remain isolated. Therefore, each Cisco Nexus 2000 Fabric Extender in the system must be associated with one and only one Cisco Nexus 5000 Series switch. This guarantees that every time a Fabric Extender forwards FCoE traffic, it forwards it to the same Nexus 5000 switch.
For more information about FCoE over Enhanced vPC, see the Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide.
Cisco Nexus Series 5000 switches support SAN boot with vPC. A VFC interface must be bound to a vPC member physical interface (and not to the vPC port-channel interface itself) for a SAN boot to occur.
For more information about SAN boot with vPC, see the Cisco Nexus 5000 Series NX-OS Fibre Channel over Ethernet Configuration Guide.
Config-sync allows you to synchronize the configuration between a pair of vPC switches. It eliminates downtime due to vPC inconsistencies, simplifies vPC operations, and reduces administrative overhead.
The enhancements to config-sync in Cisco NX-OS Release 5.1(3)N1(1) remove the port channel configuration restriction that previously existed. All port channels and member interfaces should be configured inside a switch profile.
For more information about config-sync enhancements, see the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide.
For more information, see the Cisco Nexus 5000 Series NX-OS System Management Configuration Guide.
In Cisco NX-OS Release 5.1(3)N1(1), you can configure up to eight syslog servers. You use the Cisco Fabric Services (CFS) to distribute the syslog server configuration; however, CFS distribution of the syslog server configuration is limited to three servers.
For more information, see the Cisco Nexus 5000 Series NX-OS System Management Configuration Guide.
- If host interface (HIF) port channels or EvPCs are configured in the system, and if the system was already upgraded to NX-OS Release 5.1(3)N1(1) or Release 5.1(3)N1(1a) from any release earlier than Release 5.1(3)N1(1), ensure that the system was reloaded at least once before you upgrade to Release 5.1(3)N2(1a) or Release 5.1(3)N2(1). If the switch was not previously reloaded, reload it and upgrade to Release 5.1(3)N2(1a) or Release 5.1(3)N2(1).
- When a Layer 3 license is installed, the Cisco Nexus 5500 Platform does not support an ISSU. Hot swapping a Layer 3 module, for example, the Layer 3 GEM (N55-M160L3-V2) or Version 2 Layer 3 daughter card (N55-D160L3-V2), is not supported. You must power down the Cisco Nexus device before removing or inserting a Layer 3 expansion module.
Table 6 shows the upgrade and downgrade possibilities for Cisco NX-OS Release 6.0(2)N2(7). For more information, see the Cisco Nexus 5500 Series NX-OS Software Upgrade and Downgrade Guide, Release 6.0.
Note Doing a disruptive upgrade between incompatible images will result in loss of certain configurations such as unified ports, breakout, and FEX configurations. See CSCul22703 for details.
- Ingress inter-VLAN-routed Layer3 multicast packets are treated as “unknown multicast” by the storm-control feature. This is due to the Layer 3 forwarding design in the Cisco Nexus 5500 Series switch. For details, see CSCuh34068.
- When performing an ISSU from Cisco NX-OS Release 5.1(3)N1(1) or Cisco NX-OS Release 5.1(3)N2(1) to Cisco NX-OS Release 5.2(1)N1(1), a Forwarding Manager (FWM) core can occur, which causes the system to reset. This situation occurs when network interface virtualization (NIV) is enabled. To work around this issue, use the force option in the install command to perform a disruptive upgrade. For details, see CSCty92117.
- Starting from Cisco Release 6.0(2)N2(6), the ip igmp join-group command does not work any longer. The OIL will not be programmed in the hardware. You can now verify the null OIL using the show forwarding multicast route source x.x.x.x group y.y.y.y.
- The SAN admin user role (san-admin) is a new predefined user role in Cisco NX-OS Release 5.2(1)N1(1). If you have an existing user role with the name san-admin in Cisco NX-OS Release 5.1(3)N1(1) or Cisco NX-OS Release 5.1(3)N2(1), the new system-defined role is removed when you upgrade. To resolve this issue, downgrade to the previous release, rename the user role, and perform the upgrade. For details, see CSCua21425.
- Bridge and STP traps are displayed in the downgrade incompatibility list when you downgrade from Cisco NX-OS Release 5.2(1)N1(1) to Cisco NX-OS Release 5.0(3)N1(1c). To resolve this issue, reset the STP/Bridge trap configuration to the default settings by entering the no snmp-server enable traps bridge, the no snmp-server enable traps stpx command, and then the copy running-config startup-config command. For details, see CSCua75907.
- The Server Virtualization Switch (SVS) connection is not deleted during a rollback when NIV is enabled. To resolve this issue, delete the current SVS connection and reapply the original SVS connection. For details, see CSCts17033.
- If SPAN traffic is rate-limited by entering the switchport monitor rate-limit 1G command, then a maximum transmission unit (MTU) truncation size cannot be used to truncate SPAN packets. For details, see CSCua05799.
- When an FC SPAN destination port is changed from SD to F mode and back to SD mode on an NPV switch, the port goes into an error-disabled state. Perform a shut/no-shut after the mode change recovers the port. This issue occurs only in NPV mode. For details, see CSCtf87701.
- If you configure a Cisco Nexus 2248TP port to 100 Mbps instead of autonegotiation, then autonegotiation does not occur, which is the expected behavior. Both sides of the link should be configured to both hardwired speed or both autonegotiate.
- Given the implementation of a single CPU ISSU, the STP root on the PVST region with switches on an MST region is not supported. The PVST simulation on the boundary ports goes into a PVST SIM inconsistent blocked state that breaks the STP active path. To work around this issue, move all STP roots to the MST region. However, the workaround causes a nondisruptive ISSU to fail because nonedge designated forwarding ports are not allowed for an ISSU. For additional information, see CSCtf51577. For information about topologies that support a nondisruptive upgrade, see the Cisco Nexus 5500 Series NX-OS Upgrade and Downgrade Guide.
- IGMP queries sent in CSCtf94558 are group-specific queries that are sent with the destination IP/MAC address as the group's address.
Group-specific queries are not forwarded to ports other than the one that joined the group during ISSU. The reason to forward group-specific queries toward hosts is to avoid having them leave the group. However, if a port has not joined the group, then this is not an issue. If there is an interface that has joined the group, the queries are expected to make it to the host. While the behavior is different when ISSU is not occurring, it is sufficient and works as expected and there is no impact to the traffic. For details, see CSCtf94558.
- When a private VLAN port is configured as a TX (egress) SPAN source, the traffic seen at the SPAN destination port is marked with the VLAN of the ingressed frame. There is no workaround.
- In large-scale configurations, some Cisco Nexus 2000 Series Fabric Extenders might take up to 3 minutes to appear online after entering the reload command. A configuration can be termed large-scale when the maximum permissible Cisco Nexus 2000 Series Fabric Extenders are connected to a Cisco Nexus device, all host-facing ports are connected, and each host-facing interface has a large configuration (that supports the maximum permissible ACEs per interface).
- Egress scheduling is not supported across the drop/no-drop class. Each Fabric Extender host port does not support simultaneous drop and no drop traffic. Each Fabric Extender host port can support drop or no drop traffic.
- The Cisco Nexus 2148 Fabric Extender does not support frames with the dot1q vlan 0 tag.
- VACLs of more than one type on a single VLAN are unsupported. Cisco NX-OS software supports only a single type of VACL (either MAC, IPv4, or IPv6) applied on a VLAN. When a VACL is applied to a VLAN, it replaces the existing VACL if the new VACL is a different type. For instance, if a MAC VACL is configured on a VLAN and then an IPv6 VACL is configured on the same VLAN, the IPv6 VACL is applied and the MAC VACL is removed.
- A MAC ACL is applied only on non-IP packets. Even if there is a match eth type = ipv4 statement in the MAC ACL, it does not match an IP packet. To avoid this situation, use IP ACLs to apply access control to the IP traffic instead of using a MAC ACL that matches the EtherType to IPv4 or IPv6.
- Multiple boot kickstart statements in the configuration are not supported.
- If you remove an expansion module with Fibre Channel ports, and the cable is still attached, the following FCP_ERRFCP_PORT errors appear:2008 May 14 15:55:43 switch %KERN-3-SYSTEM_MSG: FCP_ERRFCP_PORT: gat_fcp_isr_ip_fcmac_sync_intr@424, jiffies = 0x7add9a:Unknown intr src_id 42 - kernel2008 May 14 15:55:43 switch %KERN-3-SYSTEM_MSG: FCP_ERRFCP_PORT: gat_fcp_isr_ip_fcmac_sync_intr@424, jiffies = 0x7add9a:Unknown intr src_id 41 - kernel
- If you configure Multiple Spanning Tree (MST) on a Cisco Nexus device, we recommend that you avoid partitioning the network into a large number of regions.
- A downgrade from Cisco NX-OS Release 5.1(3)N1(1) to any 5.0(3)N1(x) image can cause the Cisco Nexus device to fail. For details, see CSCty92945.
- If you upgrade a vPC peer switch from Cisco NX-OS Release 5.0(3)N2(1) to Cisco NX-OS Release 5.1(3)N2(1) or Cisco NX-OS Release 5.2(1)N1(1), and feature-set FabricPath is enabled on the upgraded switch, the vPC Peer-Link enters STP Bridge Assurance Inconsistency, which affects all VLANs except VLAN 1 and affects traffic forwarding for vPC ports.
To avoid this issue, upgrade the peer switch that is running Cisco NX-OS Release 5.0(3)N2(1) to Cisco NX-OS Release 5.1(3)N2(1) or later release and then enable feature-set FabricPath on the switch or switches. If you accidentally enable feature-set FabricPath in Cisco NX-OS Release 5.1(3)N2(1) when the peer vPC switch is running Cisco NX-OS Release 5.0(3)N2(1), disable the feature-set FabricPath and the vPC will resume the STP forwarding state for all VLANs.
- By design, vEth interfaces do not share the underlying behavior of a vPC port. As a result, a VLAN does not get suspended when the peer switch suspends it. For example, when you shut a VLAN on a primary switch, the VLAN continues to be up on the secondary switch when the vEth interface is on a FEX. When the VLAN on the primary switch goes down, the VLAN on the vEth interface on the primary is suspended, but the vEth on the secondary switch is up because it is an active VLAN on the secondary switch.
- Rol-based Access Control List (RBACL) policy enforcement is performed on VLANs on which Cisco Trusted Security (CTS) enforcement is not configured. This situation occurs when there is at least one VLAN in the switch where CTS is enforced. On a VLAN where CTS is not enforced, RBACL policy lookup occurs for ingress packets and the packet is denied or permitted according to the policies in the system. To work around this issue, make sure that all VLANs on which SGT tagged packets ingress enforce CTS.
- The packet length in the IP GRE header of a packet exiting from the switch is not equal to the MTU value configured in the ERSPAN source session. This is true for SPAN or ERSPAN. This situation can occur whenever the MTU value that is configured in an ERSPAN or SPAN session is smaller than the SPAN packet, such as when the packet is truncated. The IP GRE packet is truncated to a value that differs by –2 to 10 bytes from the expected MTU.
- When you configure a Layer 3 interface as an ERSPAN source, and configure the ERSPAN termination on a Catalyst 5500 switch or a Cisco Nexus 7000 Series switch, you cannot terminate the Layer 3 interface ERSPAN source on the Cisco Nexus 7000 Series switch or the Catalyst 5500 switch. To work around this issue, configure VLAN 1 to 512 on the Cisco Nexus 7000 Series switch or the Catalyst 6000 switch.
- Unknown unicast packets in FabricPath ports are counted as multicast packets in interface counters. This issue occurs when unknown unicast packets are sent and received with a reserved multicast address (that floods to a VLAN) in the outer FabricPath header, and the Cisco Nexus device increments the interface counter based on the outer FabricPath header. As a result, multicast counters are incremented. In the case of a Cisco Nexus 7000 Series switch, unicast counters are incremented as they are based on an inner Ethernet header. There is no workaround for this issue.
- If you configure a speed of 1 G on a base or GEM port and then check for compatibility with a Cisco NX-OS Release 5.0(2) image, no incompatibility is shown. However, because 1 G was not supported in the Cisco NX-OS Release 5.0(2), an incompatibility should be shown. To work around this issue, manually remove the 1 G configuration from the ports before downgrading to Cisco NX-OS Release 5.0(2) or an earlier release.
- In an emulated switch setup, inband keepalive does not work. The following steps are recommended for peer keepalive over switch virtual interface (SVI) when a switch is in FabricPath mode:
- FabricPath requires 802.1Q tagging of the inner Ethernet header of the packet. Native VLAN packets that are sent by a Cisco Nexus 7000 Series switch are not tagged. As a result, a Cisco Nexus device drops packets due to packet parsing errors. To work around this issue, enter the vlan dot1q tag native command on the Cisco Nexus 7000 Series switch to force 802.1Q tagging of native VLAN packets.
- SPAN traffic is rate limited on Cisco Nexus 5500 Series devices to prevent impact to production traffic:
- Cisco Nexus 5548UP and Cisco Nexus 5598UP devices with a fibre-channel connection to HP Virtual Connect modules experience link destabilization and packet loss when the speed is set to 8 GB. To work around this issue, leave the speed set to 4 GB. For details, see CSCtx52991.
- A nondisruptive ISSU is not supported when ingress policing is configured.
- The maximum IP MTU that can be set on Layer 3 interfaces on which Layer 3 protocols are running is 9196, because of the internal header used inside the switch. The network-qos policy must be set to 9216.
- On a Cisco Nexus device, if the SPAN source is a FEX port, the frames will always be tagged when leaving the SPAN destination.
- On a Cisco Nexus 5500 Platform switch, if the SPAN source is on an access port on the switch port, the frames will not be tagged when leaving the SPAN destination.
- Ports on a FEX can be configured as a tx-source in one session only.
If two ports on the same FEX are enabled to be tx-source, the ports need to be in the same session. If you configure a FEX port as a tx-source and another port belonging to the same FEX is already configured as a tx-source on a different SPAN session, an error is displayed on the CLI.
- When a FEX port is configured as a tx-source, the multicast traffic on all VLANs for which the tx-source port is a member, is spanned. The FEX port sends out only multicast packets that are not filtered by IGMP snooping. For example, if FEX ports 100/1/1–12 are configured on VLAN 11 and the switch port 1/5 sends multicast traffic on VLAN 11 in a multicast group, and hosts connected to FEX ports 100/1/3-12 are interested in receiving that multicast traffic (through IGMP), that multicast traffic goes out on FEX ports 100/1/3–12, but not on 100/1/1–2.
If you configure SPAN Tx on port 100/1/1, although the multicast traffic does not egress out of port 100/1/1, the SPAN destination does receive that multicast traffic, which is due to a design limitation.
- When a FEX port is configured as both SPAN rx-source and tx-source, the broadcast, non-IGMP Layer-2 multicast, and unknown unicast frames originating from that port might be seen twice on the SPAN destination: once on the ingress and once on the egress path. On the egress path, the frames are filtered by the FEX to prevent them from going out on the same port on which they were received. For example, if FEX port 100/1/1 is configured on VLAN 11 and is also configured as SPAN rx-source and tx-source and a broadcast frame is received on that port, the SPAN destination recognizes two copies of the frame, even though the frame is not sent back on port 100/1/1.
- A FEX port cannot be configured as a SPAN destination. Only a switch port can be configured and used as a SPAN destination.
- Cisco NX-OS Release 5.1(3)N2(1) does not support SPAN on a VM FEX.
If you previously used the switchport access vlan command, the switchport trunk allowed vlan command, or the switchport trunk native vlan command to configure the switch profile mode, the configurations you created are not visible.
Table 8 lists the situations where you might experience fex associate command issues and the workarounds.
In a vPC topology, two Cisco Nexus devices configured as vPC peer switches need to be configured symmetrically for Layer 3 configurations such as SVIs, the peer gateway, routing protocol and policies, and RACLs.
When a Layer 3 module goes offline, all non-management SVIs are shut down. An SVI can be configured as a management SVI by entering the interface vlan command and configuring management. This configuration allows traffic to the management SVIs to not go through the Layer 3 module which maintains connectivity in case of a Layer 3 module failure.
- Open Caveats
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(7)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(7)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(5a)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(5)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(4)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(3)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N2(2)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N1(2)
- Resolved Caveats in Cisco NX-OS Release 6.0(2)N1(2a)
Table 9 lists descriptions of open caveats in Cisco NX-OS Release 6.0(2)N2(5a).
Table 18 lists the caveats that are resolved in Cisco NX-OS Release 6.0(2)N1(2a). The caveats might be open in previous Cisco NX-OS releases.
Table 19 lists the caveats that are resolved in Cisco NX-OS Release 6.0(2)N1(2). The caveats might be open in previous Cisco NX-OS releases.
Table 20 lists the caveats that are resolved in Cisco NX-OS Release 6.0(2)N1(1). The caveats might be open in previous Cisco NX-OS releases.
The Cisco Management Information Base (MIB) list includes Cisco proprietary MIBs and many other Internet Engineering Task Force (IETF) standard MIBs. These standard MIBs are defined in Requests for Comments (RFCs). To find specific MIB information, you must examine the Cisco proprietary MIB structure and related IETF-standard MIBs supported by the Cisco Nexus 5500 Series switch.
- Licensing Information Guide
- Release Notes
- Installation and Upgrade Guides
- Configuration Guides
- Configuration Examples and TechNotes
- Programming Guides
- Operations Guides
- Error and System Message Guides
- Field Notices
- Security Advisories, Responses and Notices
- Troubleshooting Guide
- Command References
- MIB Reference Guide
For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation at: http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html.
Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)