FlexPod Datacenter with VMware vSphere 5.1, Cisco Nexus 7000 Series Switches, and NetApp MetroCluster for Multisite Deployment

  • Viewing Options

  • PDF (484.7 KB)
  • Feedback


The IT industry today faces the challenge of transitioning to reliable, highly available virtualized and cloud-based data center infrastructure for improved business continuity, fault tolerance, and cost-effectiveness. To enable this migration with reduced risk and cost, Cisco, in partnership with NetApp, has developed the FlexPod Datacenter solution. FlexPod Datacenter is a predesigned unified computing, storage, and network infrastructure composed of Cisco Unified Computing System™ (Cisco UCS®) servers, Cisco UCS Manager, the Cisco Nexus® family of switches, and NetApp FAS storage arrays. FlexPod Datacenter provides a standard, flexible data center building block that can be used to build an optimized data center to support a variety of workloads and applications. FlexPod Datacenter, along with NetApp MetroCluster software, makes it easy to build data center clusters that span geographically distributed metro sites to provide high availability and undisrupted services with transparent disaster recovery capabilities.

Cisco, through its Cisco® Validated Designs, offers a comprehensive suite of pretested FlexPod Datacenter solutions, developed using various system components, configurations, and hypervisor and application software. Using a solution based on the Cisco Validated Design for FlexPod Datacenter as the baseline platform, IT customers can deploy a virtualized and cloud-enabled data center quickly, with low risk and cost. The FlexPod Datacenter] solution described in this document implements a distributed, multicluster data center solution that offers workload balance with near zero downtime and a highly reliable infrastructure for today’s businesses.

Benefits of the Cisco Validated Design for FlexPod with Cisco Nexus 7000 Series Switches and NetApp MetroCluster Software

High availability: Data center architecture spanning multiple geographical locations offers high-availability of resources through workload balance and a high level of reliability.
Resilience and reduced risk: Prevalidated configurations help ensure proper deployment of the current applications on the new infrastructure. MetroCluster software enables multisite data centers with near zero downtime and fault tolerance.
Transparent recovery: Infrastructure provides full end-to-end recovery from failures with no data loss.
Accelerated deployment: Unified and standard architecture reduces transition downtime and management overhead.
Flexibility: A wide variety of applications and workloads are supported through proven hardware and software combinations.
Scalability: Modular architecture supports future expansion through scale-up and scale-out strategies.
Lower total cost of ownership (TCO):

Workload balance and non-disruptive application mobility offer better utilization of resources.

Standardized architecture reduces management and training overhead.

Modular infrastructure enables IT to build a right-sized data center and avoid underutilization of resources.

Pretested solutions reduce deployment, procurement, and setup times.

FlexPod Datacenter Design Elements

FlexPod Datacenter is a unified data center platform composed of Cisco UCS servers, Cisco Nexus network switches, and NetApp storage array elements, as shown in Figure 1. The FlexPod Datacenter modules can be configured to support a variety of applications by mixing and matching the component families and versions to achieve optimal capacity, price, and performance targets. The solution can be scaled by augmenting the elements of a single FlexPod Datacenter instance. Adding multiple FlexPod Datacenter instances makes it possible to build numerous solutions for virtualized and nonvirtualized single-site and multisite data centers cost-effectively.

Figure 1. FlexPod Datacenter Design Elements

Cisco UCS Family

Cisco UCS is a next-generation data center platform that unites computing, networking, and storage access and virtualization into a cohesive server system for superior performance and cost-effectiveness. All resources participate in a unified management domain in an integrated multichassis platform. This platform is composed of Cisco UCS B-Series Blade Servers and C-Series Rack Servers, fabric interconnects, blade server chassis, and adapter connectivity.

Cisco UCS Manager device management software enables unified management of the entire computing environment. It provides role-based and policy-based management by using the service profiles of the resources, reducing maintenance overhead.

Cisco UCS fabric interconnects are the communication and management backbone for the Cisco UCS solution, providing LAN and SAN connectivity to the server blades. The fabric interconnects connect the Cisco UCS B-Series and C-Series servers (connected through Cisco Nexus 2200 platform fabric extenders) and the blade server chassis to form a unified fabric, which becomes a single management domain, providing high availability and simplicity of management.

The Cisco UCS B-Series Blade Servers are an enterprise-class computing platform designed to efficiently deliver a broad range of IT workloads, from web infrastructure to distributed databases.

Cisco UCS C-Series Rack Servers extend the unified computing model to further increase scalability and business agility.

Cisco UCS Manager offers a seamless management platform through multi-domain and self-service management capabilities. In addition, Cisco UCS Central Software provides multi-domain Cisco UCS management, and Cisco UCS Director provides converged infrastructure management across multisite data center clusters.

For more information about Cisco UCS servers, refer to

Cisco Nexus Switches

The Cisco Nexus family of switches forms the networking foundation for a FlexPod Datacenter deployment. All switches in the Cisco Nexus family use the Cisco NX-OS operating system.

The Cisco Nexus 7000 Series Switches offer a comprehensive switching and storage connectivity platform for building a unified, extended data center. They provide the high level of network availability needed to sustain nonstop operations. The key features of the Cisco Nexus 7000 Series, essential for a multisite data center deployment, include:

Cisco Overlay Transport Virtualization (OTV): Offers Layer 2 extension capability on a routed infrastructure

Multihop Fibre Channel over Ethernet (FCoE): Consolidates Ethernet and Fibre Channel (FC) for a unified fabric and provides transport flexibility

In-Service Software Upgrade (ISSU): Enables upgrades with zero downtime and provides operation continuity

Virtual device contexts (VDCs): Reduce software and hardware resource use

In addition, the Cisco Nexus 7000 Series provides critical high-availability features, including Network File System (NFS) awareness and stateful supervisor switchover.

For more information about the Cisco Nexus 7000 Series, refer to

NetApp FAS Storage and NetApp MetroCluster Software

The NetApp FAS storage system forms the scalable storage element in a Datacenter platform. It uses the NetApp Data ONTAP operating system in Data ONTAP 7-Mode, which supports SAN (FC and SCSI over IP [iSCSI]), network-attached storage (NAS), Common Internet File System (CIFS), and NFS. It enables primary and secondary storage in a single, unified platform so that all virtual desktop data components can be hosted on the same storage array.

The NetApp FAS2000 and FAS3000 Series support midsize enterprises, and the NetApp FAS6000 Series supports large enterprises. The NetApp FAS storage controllers provide hot-swappable, redundant components with fault tolerance through clustered storage controllers that offer scalability of up to several terabytes. For additional information about NetApp storage systems, refer to

NetApp MetroCluster software is a synchronous replication solution between two NetApp controllers. It enables high-availability storage and disaster recovery in multisite metropolitan data center clusters. MetroCluster manages storage domain failures without disruption to data availability and provides zero data loss and quick recovery. For more information on NetApp MetroCluster, refer to

Cisco Validated Design for FlexPod with Cisco Nexus 7000 Series Switches and NetApp MetroCluster Software: New Features

NetApp MetroCluster software provides high-availability failover capability between multiple data centers. It offers the ability to perform site maintenance with zero downtime. The multisite architecture enables businesses to balance workloads between two data centers using nondisruptive workload mobility, and to migrate services between sites without the need for an outage.
Cisco UCS Central software, along with Cisco UCS Manager offers a unified platform to manage multiple, globally distributed Cisco UCS domains.
Cisco UCS Director software provides a unified management solution to achieve converged infrastructure management across multiple sites.
Cisco Nexus 7000 Series Switches provide a high level of network availability for unified multisite data centers. Along with Cisco NX-OS, the Cisco Nexus 7000 Series provides critical high-availability features, including ISSU and OTV as well as Ethernet and Fibre Channel consolidation.
VMware vSphere Metro Storage Cluster (vMSC) enables efficient load balancing of resources across data centers to improve utilization and availability of resources for virtualized servers and supports nondisruptive VM mobility.

Cisco Validated Design for FlexPod Datacenter

The Cisco Validated Design for FlexPod Datacenter validates the cohesive operation of industry-standard hypervisors and applications running on various FlexPod Datacenter configurations for a multisite data center. Figure 2 provides a comprehensive list of supported hypervisors, management, and FlexPod Datacenter design elements and versions.

The highlighted components make up the Cisco Validated Design for FlexPod Datacenter described in this document. This Cisco Validated Design illustrates a typical FlexPod Datacenter architecture running the NetApp MetroCluster software to support a multisite data center. It offers functional validation of a single-site and a multisite FlexPod Datacenter architecture with NetApp MetroCluster software that can support VMware vSphere 5.1.

Figure 2. Components of the Cisco Validated Design for FlexPod Datacenter with Cisco Nexus 7000 Series Switches and NetApp MetroCluster Software

NetApp MetroCluster features used in this validation include:

Active-active controller, which provides high-availability failover capability between local and remote sites

SyncMirror feature that helps ensure the availability of up-to-date data after a failover

VMware vSphere 5.1 features used in this validation include VMware vMSC, VMware vCenter, and VMware DRS. This Cisco Validated Design aligns with the best practices offered in the VMware vMSC solution. The multisite architecture supports VMware ESXi for file- and block-level access to local and remote shared datastores.

The vMSC solution, through its high-availability and fault-tolerance features, provides uniform failover protection for a virtualized multisite IT infrastructure. VMware vCenter provides a centralized management tool for ESX clusters, and VMware DRS provides load balancing of host resources.

In this validation effort, the following design parameters are tested for a multisite FlexPoddata center:

Maximum network latency for VMware ESXi management is 10-ms round-trip time between multiple sites.

Maximum distance for the FCoE link between two Cisco Nexus 7000 switches is 80 km.

Maximum distance for fabric MetroCluster using the MDS 9148 is 160 km at 1 Gbps.

Note that this multisite FlexPod Datacenter solution is validated for a distance of 80 km between two data centers.

The FlexPod Datacenter configurations used in this validation effort are:

Single-site FlexPod Datacenter

Multisite FlexPod Datacenter

Single-Site FlexPod Datacenter Design

Figures 3 illustrates the FlexPod Datacenter configuration for a single-site configuration and shows its design components. This architecture uses Cisco Nexus 7000 Series Switches and Cisco UCS B-Series Blade Servers with the Cisco UCS Virtual Interface Card (VIC). The NetApp FAS family of storage controllers are connected in a high-availability topology using Cisco Virtual Port-Channels (vPC) technology for link aggregation across the two physical switches. The FlexPod Datacenter module uses the Cisco Nexus 1000V as a VMware virtual distributed switch, and it is hosted on the Cisco Nexus 1110 Cloud Services Platform. The Cisco Nexus 1000V, along with Cisco Nexus 1110, provides an active-standby high-availability switching platform.

Note: Although Cisco UCS B-Series servers are used in this validation effort, this infrastructure can support Cisco UCS C-Series servers and fabric extenders.

Figure 3. Single-Site FlexPod Datacenter Design

Multisite FlexPod Datacenter Design

Figure 4 illustrates the FlexPod Datacenter architecture that is deployed in each of the data centers in a multisite application. This configuration uses site-distributed computing, network, and storage resources to achieve high availability, workload balance, and fault tolerance.

This FlexPod Datacenter reference architecture includes the Cisco UCS B-Series servers with the Cisco UCS 6200 Series Fabric Interconnects and supports up to 16 Cisco UCS B-Series servers across two sites. It uses Cisco Nexus 7000 Series Switches and the NetApp FAS3250 storage controller operating in MetroCluster 7-Mode configuration in each site. This configuration uses ATTO FibreBridge 6500N for SAS-to-FC connectivity and Cisco MDS 9148 Multilayer Fabric Switches in each site to support the NetApp MetroCluster configuration. The Cisco Nexus 1110, Cisco Nexus 1000V, and NetApp FAS storage controllers are included at each of the two sites, and additional Cisco UCS servers and Cisco Nexus 7000 Series Switches are included at the second location. The Cisco Nexus 1000V, along with the Cisco Nexus 1110 Cloud Services Platform, establishes high-availability pairing between the two sites. This distributed device configuration allows access to the virtual supervisor module (VSM) even if one of the sites is completely down.

Figure 4. Multisite FlexPod Datacenter Design

Note: The reference design tested in this Cisco Validated Design uses Cisco UCS B-Series Blade Servers and Cisco Nexus 7000 Series Switches. However, multisite FlexPod Datacenter based on the Cisco Nexus 5000 Series Switches is also a valid configuration, and in such cases, a Cisco Nexus 7000 Series Switch should be used to support the OTV feature.

Table 1 lists the hardware and software versions used in the Cisco Validated Design VMware vSphere 5.1 on FlexPod Datacenter with NetApp MetroCluster and Cisco Nexus 7000 Series.

Table 1. Hardware and Software Versions of FlexPod Datacenter Components Used in the Cisco Validated Design for VMware vSphere 5.1 on FlexPod Datacenter with NetApp MetroCluster and Cisco Nexus 7000 Series



Version or Release



Cisco UCS 6100 Series Fabric Interconnect


Embedded management

Cisco UCS B200 M3 Blade Server


Software bundle release

Cisco UCS B22 M3 Blade Server


Software bundle release

Cisco eNIC

Ethernet driver for Cisco VIC

Cisco fNIC

FCoE driver for Cisco VIC

Cisco UCS Central Software


UCS Central Software


Cisco Nexus 7000 Series Switches


Operating system version


NetApp FAS3250-AE

Data ONTAP 8.2 7-Mode

Operating system version


Cisco UCS hosts

VMware vSphere ESXi 5.1

Operating system version

Microsoft.NET Framework


Feature enabled within Windowsoperating system

Microsoft SQL Server

Microsoft SQL Server 2008 R2 SP1

VM (1 each): SQL Server DB

VMware vCenter


VM (1 each): VMware vCenter

NetApp OnCommand


VM (1 each): OnCommand

NetApp Virtual Storage Console (VSC)


Plug-in within VMware vCenter

Cisco Nexus 1110-x Virtual Services Appliance


Virtual services appliance

Cisco Nexus 1000V Switch


Virtual services blade within the 1110-x

NetApp NFS plug-in for VMware vStorage APIs for Array Integration (VAAI)


Plug-in within VMware vCenter

NetApp FAS/V-Series vSphere Storage APIs for Storage Awareness (VASA) Provider


VM (1 each): NetApp VASA Provider

In addition to the configurations discussed so far, FlexPod Datacenter architecture supports a wide range of other hardware and software versions. To verify support for a specific implementation of FlexPod Datacenter, and to identify compatible hardware and software configurations, refer to the following compatibility guides:

NetApp Interoperability Matrix Tool:

Cisco UCS Hardware and Software Interoperability Tool:

VMware Compatibility Guide:

Note: The FlexPod Cooperative Support model applies to both the FlexPod Datacenter architecture component versions in this Cisco Validated Design and the individual component versions listed as supported in the interoperability matrix. Cooperative support is available for all FlexPod components and versions so long as customers have purchased the appropriate levels of support contracts.This approach allows customization of the FlexPod solution according to the organization’s specific requirements.


FlexPod Datacenter is an integrated, standardized, and prevalidated data center module that can be used to build a right-sized single-site or multisite data center for a variety of IT applications. Its flexible architecture is composed of Cisco UCS servers, Cisco Nexus network switches, and NetApp FAS storage arrays. By selecting the appropriate versions of these design elements, the customer can deploy a flexible and scalable data center optimized to meet the organization’s workload requirements.

The Cisco Validated Design discussed in this document focuses on the case in which VMware vSphere 5.1 is run on FlexPod Datacenter with the NetApp MetroCluster solution for multisite deployment. It provides a prevalidated data center solution with transparent, end-to-end disaster recovery capabilities with zero data loss and zero downtime. Using the FlexPod Datacenter architecture validated in this Cisco Validated Design as the foundation, the customer can cost-effectively deploy a geographically dispersed, virtual data center.

For More Information

FlexPod Datacenter with Cisco Nexus 7000 and NetApp MetroCluster for Multisite Deployment Design Guide: