Cisco MDS 9500 Series Multilayer Directors

Cisco MDS 9000 Family SANs Increase Cisco Unified Computing System Benefits

  • Viewing Options

  • PDF (346.3 KB)
  • Feedback
IT departments are working with their business counterparts to identify ways to substantially decrease cost of ownership while increasing IT business value. The Cisco Unified Computing System™ helps address these challenges by streamlining data center resources, scaling service delivery, and dramatically reducing the number of devices requiring setup, management, power and cooling, and cabling.
The Cisco Unified Computing System delivers these benefits through:

• Reduced total cost of ownership (TCO) at the platform, site, and organizational levels

• Increased IT staff productivity and business agility through just-in-time provisioning and mobility support for both virtualized and non-virtualized environments

• Scalability through a design for up to 320 discrete servers and thousands of virtual machines in a single highly available management domain

• Default SAN enablement of every server and virtual machine through the use of converged network adapters (CNAs) and unified fabric interconnects

• Use of industry standards supported by a partner ecosystem of innovative, trusted industry leaders

In the highly mobile, highly virtualized, rapidly provisioned and dynamically scalable environment of unified computing, the traditional model of direct-attached storage (DAS) just can't keep up. When new virtual machines can be added or migrated to different physical servers simply by changing a service-level agreement (SLA), and with the blade server becoming a more popular deployment platform for virtual machines, even server hardware is becoming more mobile. To meet the challenges and fully exploit the economic opportunities of these environments, you need a SAN built from the beginning to support multiple protocols, and with virtualization woven into its very fabric.

Financially Justifying a SAN

Challenge: Justifying SAN Enablement

With earlier rack-mounted servers and blade servers, enterprises had to address the question of whether SAN-enabling each server made financial sense. Did the importance or performance requirements of the application justify at least two host bus adapters (HBAs) at several hundred U.S. dollars apiece, plus consumption of corresponding SAN switch ports, plus cables, plus installation and provisioning?

Solution: Build the Server Around Unified Fabric

The Cisco Unified Computing System was designed from the start for the new unified fabric environment, which allows LAN, SAN, and low-latency cluster traffic to coexist in a high-performance, next-generation Ethernet environment designated in the IEEE standards as Data Center Bridging (DCB). This design concept yields major savings in capital expenditures (CapEx) by reducing the number of controllers, cables, switch ports, and switches required. These in turn reduce operating expenses (OpEx) with fewer cables, adapters, and switches, less management, space, power, and cooling is needed.

Providing a Common Environment

Challenge: Building on Staff Knowledge as Networks Converge

While products are constantly becoming more powerful and less expensive, technical staff members, such as LAN and SAN administrators, are becoming more costly, and their time proportionally more valuable. With technology promising significant savings through the convergence of LAN, SAN, and inter-process communication (IPC) connections onto a few CNAs, employment of staff knowledge across disciplines becomes critical.

Solution: Common Software to Unify the LAN and SAN

Most networking professionals are familiar with the Cisco Catalyst ® Family of Ethernet switches and their Cisco IOS ® Software operating environment. Many members of the original team that created the Cisco Catalyst products also created the Cisco MDS 9000 Family of SAN switches and directors and the Cisco MDS 9000 SAN-OS Software operating system. The Cisco MDS 9000 SAN-OS was expanded into the Cisco NX-OS Software as Ethernet and DCB capabilities were added during the team's subsequent creation of the Cisco Nexus ® Family of Ethernet products. The fabric interconnects in the Cisco Unified Computing System are based on the Cisco Nexus 5000 Series hardware.
The result is a networking environment that will seem immediately familiar to staff who are used to Cisco IOS Software. Just as Ethernet switches can be partitioned and joined into VLANs, so Cisco MDS 9000 Family switches can be partitioned and joined into virtual SANs (VSANs). Indeed, this Cisco technology has been built into the Cisco MDS 9000 Family since the beginning, and it is the basis for the ANSI Virtual Fabric standard. Similarly, anyone who knows Cisco IOS Software will understand immediately the style of the Cisco NX-OS command-line interface (CLI).
This familiarity decreases training requirements, especially for the Cisco MDS 9000 and Nexus Family products, and increases the transferability of skills and the ease of cross-training LAN and SAN administrative staff.

Achieving Consolidation Across the Data Center

Challenge: Deploying a Scalable SAN Infrastructure to a Consolidated Unified Computing Server Farm

One of the reasons for the adoption of the Cisco Unified Computing System is the opportunity for savings and efficiency from consolidation of large numbers of physical servers into physical or virtualized servers within the unified system, and consolidation of the I/O for those servers as well at the SAN server edge and LAN access layer. This high density places new demands on SAN scalability and ease of management. However, if the SAN containing the storage that these servers need is split into numerous physical fabrics, much of the simplicity and efficiency of the Cisco Unified Computing System model is lost.

Solution: A Consolidated, Virtualized Cisco MDS 9000 Family SAN Infrastructure

The same benefits that accrue from consolidation and virtualization of a disjointed server infrastructure can be gained from consolidation and virtualization of multiple, disjointed SANs (Figure 1). While parallel fabrics (SAN A and SAN B) remain a best practice, each pair of SAN islands can be consolidated into a pair of VSANs in two enterprise fabrics. This design allows sharing of a highly available physical infrastructure while isolating each VSAN's:

• Traffic

• Management

• Configuration

• Errors, such as misconfigurations

Thus, VSANs expand the isolation of various applications in the Cisco Unified Computing System into the SAN fabric itself. By using VSANs in the Cisco Unified Computing System fabric interconnects and in the Cisco MDS 9000 Family SANs, you increase the capability of the Cisco Unified Computing System to consolidate diverse workloads without their interfering with each other. For situations that require exceptions to the rule of isolation, portals can be created between VSANs using built-in Inter-VSAN Routing (IVR) to connect VSANs in different locations or to share specialized equipment, such as backup hardware.

Figure 1. Consolidation: Servers to Cisco Unified Computing System, and SAN Islands to VSANs

Getting the Performance You Need

Challenge: Meeting Bandwidth and Performance Service Levels

The Cisco Unified Computing System is designed to support the greatest density of server virtualization in the industry, as well as to provide outstanding density of compute power at the physical server level. These capabilities make it an outstanding platform for consolidating a wide variety of applications, which may vary widely in their demands for I/O bandwidth. However, the SAN platform connecting the Cisco Unified Computing System to its storage must to be able to provide predictable, finely tunable performance to help ensure that each virtual machine and application in this consolidated mix gets the bandwidth and performance it needs.

Solution: Bandwidth Management and Quality of Service

The centrally arbitrated architecture of the Cisco MDS 9000 Family helps ensure fair treatment and consistent latency for every frame, regardless of whether it is destined for a physically adjacent port or one several line cards away. Within a group of adjacent ports, a given port can be allocated dedicated bandwidth, or it can be put in a pool of ports, with ports in the pool sharing the unused bandwidth. Bandwidth sharing can be further regulated using the Cisco MDS 9000 Family's robust quality-of-service (QoS) feature. Administrators can specify the maximum amount of bandwidth for each QoS level, and QoS levels can be assigned to an individual end-to-end traffic flow, to a zone, or to an entire VSAN. QoS can be especially valuable when applications of different priorities and performance characteristics are consolidated on a shared Cisco Unified Computing System server infrastructure.

Enabling Cloud-like Agility

Challenge: Managing Storage for Rapidly Changing Virtual Machine Deployments

The highly integrated management environments of the Cisco Unified Computing System and VMware make it easier than ever to provision and modify physical and virtual server environments simply and rapidly. To gain the full benefits of this agility on the server side, storage management must be equally responsive.

Solution: Integrated Management

The Cisco Fabric Manager is the GUI for managing not only the Cisco MDS 9000 Family, but all storage networking capabilities of the Cisco Nexus 5000 Series used to connect the Cisco Unified Computing System to Cisco MDS 9000 Family SANs. Furthermore, when synchronized changes are required to the SAN and LAN, the Cisco Fabric Manager and Data Center Network Manager (DCNM) can cross-launch each other, and the Cisco Fabric Manager can also cross-launch the Cisco UCS Manager. Even more integration can be expected as the pace of Cisco Unified Computing System and unified fabric deployment increases.

Monitoring and Managing the System

Challenge: Monitoring and Managing Each Virtual Machine's Storage Use

Aggregation of traffic from the large number of virtual machines supported in the Cisco Unified Computing System can make it hard to monitor bandwidth consumption and other use statistics on a per-virtual machine basis in the same way that SAN administrators are used to monitoring physical servers. In addition, the common technique of placing each virtual machine's storage in a container file can add overhead and reduce security, because this approach requires all virtual machines to have access to the same storage volume (the same logical unit number [LUN]) on which the pooled container files reside.

Solution: Give Each Virtual Machine Its Own Storage Identity

Cisco MDS 9000 Family fabrics allow each virtual machine to have its own identity in the SAN through the industry standard known as N-Port ID Virtualization (NPIV). When NPIV is used, each virtual machine logs into the SAN fabric separately and becomes a full member of the SAN fabric. It can be given its own access privileges (zones) and its own QoS, and its performance can be isolated, monitored, and analyzed.

Migrating Virtual Machines

Challenge: Adapting to Volatility

While SANs have always been adaptable, with most environments adjusting to the addition and removal of servers and storage devices, the capability of virtual servers to migrate from one host to another in only seconds, and while in use, poses an unprecedented challenge to a SAN's ability to respond. SANs use zoning for access control, to specify which servers can access which storage devices. But what happens to your zones when the server is a virtual machine, and the virtual machine moves rapidly, perhaps due to an automated policy, from one physical host to another? How can the SAN respond to the automated provisioning of eight new virtual machines when the existing virtual machines become too heavily loaded?

Solution: Predictable Performance with Flexible Administration

When important applications can move around the SAN, an architecture that delivers predictable performance is essential. The centrally arbitrated design of the Cisco MDS 9000 Family products helps ensure consistent latency for traffic passing through each switch no matter what path the traffic takes.
Cisco MDS 9000 Family services-oriented SAN fabrics can deploy commonly used storage applications, such as encryption, LUN migration, LUN erasure, and I/O acceleration to reduce the effects of metropolitan area network (MAN) or WAN latency in service engines integrated into Cisco MDS 9000 Family switches and directors. Unlike with host-based or storage-device-based deployment models, the service can be configured for any host or storage device or traffic flow in the SAN. Clusters of SAN service engines provide scalability, load balancing, and failover for high availability (Figure 2).

Figure 2. Intelligent Storage Applications Delivered as a Transparent Fabric Service

Through the use of the virtual worldwide port name (vWWPN) assigned to a virtual machine for purposes such as zone membership, zones automatically adapt to the migration of virtual servers. When the virtual machine logs back into the fabric from its new location, it will use the same vWWPN that was assigned to it originally, and it will be given the same zone memberships that it had at its prior location. Through the use of a virtual machine management utility, such as VMware vCenter or Microsoft System Center Virtual Machine Manager, vWWPNs for a group of virtual machines can be provisioned in advance and zoned appropriately, so that when the virtual machines are actually instantiated and run, they will log in immediately and have ready access to the storage they need.

Interconnecting UCS-Based Data Centers Over Metro Area Networks

Challenge: Cost-Effective Cross-Site Availability

More and more enterprises are establishing a second data center within metro distances to provide a hot backup site or active-active processing with their primary data center. However, there are numerous challenges to making a pair of distant data centers function as one, including data center interconnection costs, risks to application and data availability, security of data in transit, and virtual machine mobility limits.

Solution: MDS SANs are the Masters of MANs

The Cisco Unified Computing System is designed to provide a superior platform for data center virtualization, including servers, networks, and SANs. Pioneering work by Cisco and VMware have recently introduced Long-Distance Vmotion, which is documented in a white paper at
However, moving virtual machines is only one part of the multi-site metro solution. Several other MDS 9000 Family features provide significant advantages in cost, performance, and security in a MAN environment:

• Integrated DWDM and CWDM optics can plug directly into the SFP slots of any MDS 9000 Family switch, eliminating the need for transponder equipment, and yielding savings in cables, optics, transponders, rack space, power, cooling, and management effort.

• The 8 Gbps switching modules for the MDS 9000 family support Cisco TrustSec Fibre Channel Link Encryption, which can encrypt every link between a pair of switches at full line rate. This ensures the privacy of data that leaves the confines of the data center. Of course, it can be used within a data center as well.

• I/O Accelerator, a licensed software feature for the MDS 9000 Family, improves the performance of tape reads and writes and of disk writes through a combination of data compression and protocol spoofing.

• Another way to reduce the impact of distance-based latency is to configure additional buffers on the Fibre Channel switch ports. The MDS 9000 8 Gbps switching modules support up to 5,880 buffers on a single port.

• Physical links between Fibre Channel switches (Inter-Switch Links, or ISLs) are often bundled into logical links called PortChannels to provide load balancing and logical links that stay up when some (but not all) of their component ISLs fail. MDS 9000 PortChannels can include ISLs of widely differing lengths, enabling logical connections between servers and storage to be spread over multiple physical paths, such as the "north" and "south" routes of a DWDM ring. This allows links to stay up even if one path is completely severed.


The Cisco Unified Computing System offers outstanding scalability, performance, network integration, and cost effectiveness in a compute server environment. However, its integrated, heavily virtualized design center almost demands the use of SANs for storage connectivity. Cisco MDS 9000 Family SANs, with a common heritage, consistent feature sets, and many optimizations for the virtualized server environment, are the perfect complement to the Cisco Unified Computing System and allow enterprises that deploying the Cisco Unified Computing System to increase its performance, optimize its management, and increase its agility in their data centers.

For More Information