10 Features of NX-OS That Everyone Should Consider

With the launch of NX-OS in 2008, an operating system that was designed from the ground up to meet the unique and specific requirements of the ever-changing Data Center was introduced to the networking community. NX-OS is exciting, innovative, and game changing to say the least. Being at the forefront of this solution set, I have seen firsthand the tangible benefits NX-OS and the Nexus family bring to our customers. Initial deployments were not without challenges as any new platform will have, but these challenges were tempered with very real expense reduction, fabric consolidation, enhanced visibility and insight to virtual networking and supportability.

As I discuss NX-OS with customers, I have had the opportunity to learn many things about NX-OS as an operating system and what my customers tell me are important features to them. With that in mind, I wanted to share a concise list of features available in NX-OS, where they are used, and why they are important. Where appropriate, I'll indicate if the feature is platform specific and if so, which platforms can utilize the feature. I am not ranking the features in any particular order.

1. Virtual Device Contexts (VDCs) on the Nexus 7000 series of switches is a feature that enables a single physical chassis to be virtualized into multiple logical contexts. The level of virtualization is quite comprehensive in that each context has its own configuration, routing and switching topology, VLAN allocation, interfaces, security policy and memory processes. This enables the ability to have a process, OSPF for example, suffer a process level failure and because of the virtualization of VDCs, OSPF in other contexts is not impacted. The segmentation with VDCs is so complete that to enable communication between VDCs on the same platform, a physical interconnect using front panel ports is required. This requirement provides the appropriate hardware resources for QoS, Netflow, ACLs and other functions that require hardware to operate correctly. My customers utilize VDCs to provide a network consolidation effect where they can vertically consolidate hierarchical layers of the network into a single unit while still maintaining the benefits hierarchy brings. For example, consolidation of the core and distribution/aggregation layers of the network reduces the number of physical chassis in a network from four to two, which can reduce the amount of space, power, and cooling that separate physical layers of the network might have consumed. Alternative uses for VDCs on Nexus 7000s leverage designs where the multiple layers of a tiered application are provided by a VDC or where the Internet Edge, DMZ and internal networks all reside in VDCs on a Nexus 7000. Firewalls and other security appliances are used to provide the correct policy where applicable. Additionally, being able to take advantage of In Service Software Upgrade (ISSU) with VDCs simplifies change or maintenance windows furthering the benefit of using Nexus 7000 series switches. In the topology below VDCs are being used to provide horizontal consolidation of core and 2 aggregation/distribution layers into 3 VDCs per Nexus 7000 chassis rather than 6 separate devices with a non-VDC implementation.

Figure 1

2. Overlay Transport Virtualization (OTV) is a new feature on the Nexus 7000 series of switches that provides the ability to extend Layer 2 domains between Nexus 7000 switches using an IP enabled network. The Nexus 7000 encapsulates the full L2 frame into an IP packet for transport. The transporting network then can carry the IP packet, with its encapsulated L2 frame, to the appropriate destination where the L2 frame is de-encapsulated and transparently placed into the appropriate VLAN. With built-in loop avoidance and the ability to interconnect more than one data center without merging Spanning Tree Protocol (STP) domains make OTV a very compelling technology. This capability is significant as more and more requirements are placed on inter-data center connectivity and promises a simpler alternative to the current proverbial acronym soup of options. Technologies such as Ethernet over MPLS (EoMPLS), Virtual Private LAN Services (VPLS) and others have provided the fundamental capabilities customers needed, but complex STP implementations were required to provide fault tolerance and mitigate merging STP domains. OTV eliminates this complexity by using a routing protocol, Intermediate System to Intermediate System (IS-IS), to provide the control plane, but hides the complexity from the user and facilitates a basic configuration in as few as 6 lines of commands. OTV enables customers to break down the barriers that keep them from implementing a realistic Business Continuance/Disaster Recovery (BC/DR) program by safely extending VLANs between multiple data centers and enables mobility of systems and applications while maintaining their current addressing.

3. Unified Fabrics, also called Fibre Channel over Ethernet (FCoE) is a capability we've been shipping on the Nexus 5000 series of switches for over a year and a half. The ability to combine native Fibre Channel protocol and transport it using an enhanced version of Ethernet, referred to as Data Center Bridging (DCB), has a significant impact on the face of the modern data center. FCoE brings the ability to reduce cable and network infrastructure while providing the same, if not better performance in some cases, to the critical data center access layer. This drives a real financial save where fewer lower-cost cables can be used to replace a myriad of copper and fiber cable that were used for the Ethernet and Fibre Channel networks. Additionally with the 2nd generation of Converged Network Adapter (CNAs), a combined Fibre Channel adapter and 10 Gigabit Ethernet adapter, richer QoS and performance benefits allow for block I/O access for even the most demanding environments. Using technologies such as N-Port ID Virtualization (NPIV) allowing multiple Fabric Logins (FLOGIs) per port and N-Port Virtualization (NPV) to make the Fibre Channel component of the switch use the fabric services of the upstream switch without requiring a domain ID allows for easy integration into any storage environment.

4. Virtual Port Channels (vPC) is a feature available on the Nexus 7000, 5000 and 1000V series of switches and provides the ability to dramatically simplify STP domains and reduce blocked ports. Using vPC, customers can build multi-chassis Etherchannel topologies to enhance high availability and available bandwidth by having active/active L2 connections to two Nexus switches rather than having STP block one of links. vPC supports industry standard 802.3ad channeling and can also utilize Link Aggregation Control Protocol (LACP). This provides the ability to extend vPC benefits to hosts, especially useful as more critical applications and services are placed in higher density platforms such as blade enclosures. The topology below illustrates a typical network topology before and after vPC.

Figure 2

5. Roles Based Access Control (RBAC) is a feature that spans the entire Nexus product line and provides the ability to assign authorization to commands and feature sets based on the concept of a role rather than the traditional privilege level commands from IOS. This capability provides granular access to functions in NX-OS and is particularly useful when discussing Unified Fabrics. Using RBAC it is possible to provide both the storage and network teams access to the Nexus switches and maintain maximum separation of job duties and command structures based on roles. This helps assuage concerns that the network team may inadvertently impact the storage team’s configuration and vice versa. RBAC is flexible to map roles and responsibilities into an organization that has multiple levels of access to the environment for auditors, operators, escalation teams and engineering resources.

6. Fabric Extenders (FEX) are a combination of hardware and software for the Nexus 5000 series switches today that provide a compelling solution to the ever-changing, but critical, access layer in data centers. The concept of Fabric Extenders is analogous to a remote line card in a traditional modular chassis based switch in that the Fabric Extender is centrally managed by the “mother ship” device where the configuration is stored but rather than facilitating the connection to the FEX via traces on a backplane, 10 Gigabit Ethernet ports are utilized. Additionally, these 10 Gigabit Ethernet ports do not run STP between the mother ship and the FEX but rather are treated as backplane extensions, providing further simplification of the network topology. One of the main challenges with top of rack access layer technologies is the proliferation of switches that can complicate the L2 topology and may be looked upon as an undesirable solution for this reason. FEX and the Nexus 5000 enable the benefits of top of rack switching without the complication of an increasing L2 topology.

7. VM-aware networking with the Nexus 1000V is a capability customers are eager to leverage. The Nexus 1000V fills the visibility void created when customers move from a physical network implementation to a virtualized environment with VMware. While the VMware switch is very effective at what it does, which is basic networking, there is a significant feature gap that needs to be addressed including Netflow, QoS, SPAN and ERSPAN and security policy enforcement. The Nexus 1000V provides these services to a VMware vSphere environment and gives the network team the access, visibility and control to fully measure, troubleshoot and operate the network. The Nexus 1000V uses the concept of port profiles to associate network policy to a virtual machine. This network policy could be as simple as VLAN membership or include comprehensive QoS, Netflow, and security policies that will follow the virtual machine as it moves around the environment. Additionally, network statistics are viewable through the familiar NX-OS CLI to show information per virtual machine, and these statistics follow the virtual machine wherever it moves enabling accurate troubleshooting and capacity planning to be performed. In essence, the Nexus 1000V provides the insight into the virtual access layer that customers lost when they virtualized.

8. In Service Software Upgrade (ISSU), a feature on the Nexus 7000 and soon to be available on the Nexus 5000, enables the ability to upgrade the operating system of the Nexus device without disruption to the network and traffic traversing the switch. This is facilitated by the architecture of Nexus switches where the control plane and data plane are separated. Couple this separation with a modular operating system that supports Stateful Switch Over (SSO) and you get ISSU. NX-OS has its roots in SAN-OS, the operating system that runs our Fibre Channel series of MDS switches. SAN-OS has been doing ISSU since its initial releases over six years ago and we’ve been able to take that know-how and technical acumen and apply it to NX-OS on Nexus switches to provide a key capability for our customers. With tighter change control/maintenance windows and increasing uptime considerations in the evolving data center, the need for ISSU is now more than ever a key requirement.

9. Configuration Checkpoint and Rollback is a capability in the Nexus 7000 and soon to be available on the Nexus 5000 that provides the ability to take a snapshot of the device’s configuration file and restore the configuration from this checkpoint if desired. Over the years, many customers have become very adept at various cut and paste, scripting, or other operations to affect change on their network and restore the original configuration or back the change out if needed. Some of these methods have been learned painfully after a mis-configuration caused an outage or other issue and may not be the most efficient way to restore the network configuration. With NX-OS, the capability to make multiple snapshots of the configuration and restore them easily with a single command is very, very compelling. Consider the example of a network engineer making changes to the network in the middle of the night and after completing and testing, feeling the network is stable and working as desired only to be awoken in the early hours of the morning by the network operations team to help address a problem that may be related to the recent change. With only a short amount of sleep, reversing a complex change can take long minutes if not hours, but with configuration checkpoint and rollback, a single command can revert the NX-OS device to its original configuration. Speaking from personal experience, I would have LOVED to have this capability a few years ago and now we do!

10. Last, but not least on the list is the NX-OS CLI in general. We’ve taken years of feedback and customer suggestions on how best to improve the CLI and incorporated many of these into NX-OS. For example, the ability to issue exec commands from config mode without needing to prepend “do” to the command is handy. NX-OS has a UNIX kernel at its heart, which provides the ability to grep and pipe output and can be a significant time saver. Interface naming is very consistent as the interface speed is not incorporated into the name the way it is on many IOS platforms. The speed is learned by the OS and the appropriate adjustments are made to routing metrics, STP, etc. This simplified naming provides the ability to configure and administrate the network without needing to remember the interface’s speed. Finally, the ability to parse the local log file and specify where to start and the number of lines to display makes troubleshooting a busy network with while reviewing logs much easier. For example, show log last 20 will display the 20 most recent entries in the log file.


As you can see, there are many interesting and compelling reasons to consider NX-OS in your network. If your goal is to simplify the network topology, vPC and FEX may well be suited to meet your needs. Enhancing your BC/DR posture and providing inter-data center mobility of workloads and virtual machines without changing IP addresses with OTV answers many of the questions and challenges my customers face daily. Consolidating fabrics with FCoE may provide a significant reduction in capital and operating expenditures and when coupled with RBAC enable the separation of job functions many organizations require. Visibility into a virtualized environment with the Nexus 1000V to see interface statistics, Netflow, QoS, and security policy and ensure it is maintained consistently as the virtual machines move around the network is compelling as customers are asked to do more with less. Finally, simplifying operations of the network via checkpoint and rollback as well as the enhancement to the CLI are powerful additions to the suite of capabilities within NX-OS.

There are dozens of additional features that are incorporated into NX-OS that equally fulfill needs within networks, but I wanted to highlight ten of the most compelling. I hope you found this information useful and would encourage you to speak with your Cisco account team to understand more about these capabilities and how they can benefit your organization. Additional details on these and the rest of the NX-OS feature sets can be found on http://www.cisco.com/go/datacenter

About the Author:

Ron Fuller, CCIE No. 5851 (Routing and Switching/Storage Networking) is a Technical Solutions Architect for Cisco specializing in data center architectures. He has 19 years of experience in the industry and has held certifications from Novell, HP, Microsoft, ISC2, SNIA and Cisco. His focus is working with Enterprise customers to address their challenges with comprehensive end-to-end data center architectures.

Ron Fuller

NX-OS and Cisco Nexus Switching:
Next-Generation Data Center ArchitecturesTrunking

By Kevin Corbin, Ron Fuller, David Jansen.
ISBN-10: 1-58705-892-8
ISBN-13: 978-1-58705-892-9
Available May 14, 2010
US SRP $54.00
Published by Cisco Press.