Cisco Virtualization Solution for EMC VSPEX with VMware vSphere 5.1 for 50 Virtual Machines
Available Languages
Table Of Contents
About Cisco Validated Design (CVD) Program
Cisco Virtualization Solution for EMC VSPEX with VMware vSphere 5.1 for 50 Virtual Machines
Cisco Solution for EMC VSPEX with VMware Architectures
Cisco Unified Computing System
Cisco UCS C220 M3 Rack-Mount Servers
EMC Storage Technologies and Benefits
Solution architecture overview
Memory Configuration Guidelines
ESX/ESXi Memory Management Concepts
Virtual Machine Memory Concepts
Allocating Memory to Virtual Machines
VMware Memory Virtualization for VSPEX
Virtual Networking Best Practices
VMware Storage Layout for VSPEX
Architecture for 50 VMware virtual machines
Defining the Reference Workload
Applying the Reference Workload
VSPEX Configuration Guidelines
Prepare and Configure the Cisco Nexus 3048 Switch
Initial Setup of Nexus Switches
Global Port-Channel Configuration
Global Spanning-Tree Configuration
Virtual Port-Channel (vPC) Global Configuration
Configuring Storage Connections
Configuring Server Connections
Configure ports connected to infrastructure network
Verify VLAN and port-channel configuration
Prepare the Cisco UCS C220 M3 Servers
Configure Cisco Integrated Management Controller (CIMC)
Enabling Virtualization Technology in BIOS
Install ESXi 5.1 on Cisco UCS C220 M3 Servers
Connect and log into the Cisco UCS C-Series Standalone Server CIMC Interface
VMware vCenter Server Deployment
Adding ESXi hosts to vCenter or Configuring Hosts and vCenter Server
Configuring vSwitch for Management and VM traffic
Create and Configure vSwitch for vMotion
Create and Configure vSwitch for iSCSI
Configuring Software iSCSI Adapter (vmhba)
Prepare the EMC VNXe3150 Storage
Configure iSCSI Storage Servers
Add VMware Hosts to EMC VNXe Storage
Create Storage Pools for VMware
Configuring Discover Addresses for iSCSI Adapters
Configuring vSphere HA and DRS
Template-Based Deployments for Rapid Provisioning
Jumbo MTU validation and diagnostics
Validating Cisco Solution for EMC VSPEX with VMware Architectures
Verify the Redundancy of the Solution Components
Customer Configuration Data Sheet
Cisco Virtualization Solution for EMC VSPEX with VMware vSphere 5.1 for 50 Virtual MachinesLast Updated: June 26, 2013
Building Architectures to Solve Business Problems
About the Authors
Sanjeev Naldurgkar, Technical Marketing Engineer, Server Access Virtualization Business Unit, Cisco SystemsSanjeev Naldurgkar is a Technical Marketing Engineer at Cisco Systems with Server Access Virtualization Business Unit (SAVBU). With over 12 years of experience in information technology, his focus areas include UCS, Microsoft product technologies, server virtualization, and storage technologies. Prior to joining Cisco, Sanjeev was Support Engineer at Microsoft Global Technical Support Center. Sanjeev holds a Bachelor's Degree in Electronics and Communication Engineering and Industry certifications from Microsoft, and VMware.
Acknowledgements
For their support and contribution to the design, validation, and creation of the Cisco Validated Design, I would like to thank:
•
Vadiraja Bhatt-Cisco
•
•
•
•
•
•
•
About Cisco Validated Design (CVD) Program
The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information visit:
http://www.cisco.com/go/designzone
ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks can be found at http://www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1005R)
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
© 2013 Cisco Systems, Inc. All rights reserved.
Cisco Virtualization Solution for EMC VSPEX with VMware vSphere 5.1 for 50 Virtual Machines
Executive Summary
Cisco solution for EMC VSPEX proven and modular infrastructures are built with best of-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, compute, and networking layers. VSPEX eases server virtualization planning and configuration burdens. VSPEX accelerate your IT Transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. This Cisco Validated Design document focuses on the VMware architecture for 50 virtual machines with Cisco solution for EMC VSPEX.
Introduction
Virtualization is a key and a critical strategic deployment model for reducing the Total Cost of Ownership (TCO) and achieving better utilization of the platform components like hardware, software, network and storage. However, choosing the appropriate platform for virtualization can be challenging. Platform should be flexible, reliable and cost effective to facilitate the virtualization platform to deploy various enterprise applications. Also, the ability to slice and dice the underlying platform to size the application requirement is essential for a virtualization platform to utilize compute, network and storage resources effectively. In this regard, Cisco solution implementing EMC VPSEX provide a very simplistic yet fully integrated and validated infrastructure for you to deploy VMs in various sizes to suite your application needs.
Target Audience
The reader of this document is expected to have the necessary training and background to install and configure VMware vSphere, EMC VNXe3150, Cisco Nexus 3048 switch, and Cisco Unified Computing (UCS) C220 M3 rack servers. External references are provided wherever applicable and it is recommended that the reader be familiar with these documents.
Readers are also expected to be familiar with the infrastructure and database security policies of the customer installation.
Purpose of this Document
This document describes the steps required to deploy and configure the Cisco solution for EMC VSPEX for VMware architecture to a level that will allow for confirmation that the basic components and connections are working correctly. This CVD covers the VMware vSphere 5.1 for 50 Virtual Machines private cloud architecture. While readers of this CVD are expected to have sufficient knowledge to install and configure the products used, configuration details that are important for deploying this solution are specifically mentioned.
The 50 virtual machine environment discussed is based on a defined reference workload. While not every virtual machine has the same requirement, this document contains methods and guidance to adjust the system to be cost-effective when deployed.
A private cloud architecture is a complex system offering. This document facilitates its setup by providing up-front software and hardware material lists, step-by-step sizing guidance and worksheets, and verified deployment steps. When the last component has been installed, there are validation tests to ensure that your system is operating properly. Following the procedures defined in this document ensures an efficient and painless journey to the cloud.
Business Needs
VSPEX solutions are built with proven best-of-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, server, and networking layers. VSPEX infrastructures accelerate your IT transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk.
Business applications are moving into the consolidated compute, network, and storage environment. Cisco solution for EMC VSPEX for VMware helps to reduce complexity of configuring every component of a traditional deployment. The complexity of integration management is reduced while maintaining the application design and implementation options. Administration is unified, while process separation can be adequately controlled and monitored. Following are the business needs for the Cisco solution of EMC VSPEX with VMware architectures:
•
•
•
Solutions Overview
This section provides a list of components used for deploying the Cisco solution for EMC VSPEX for 50 VMs using VMware vSphere 5.1.
Cisco Solution for EMC VSPEX with VMware Architectures
This solution provides an end-to-end architecture with Cisco, EMC, VMware, and Microsoft technologies that demonstrate support for up to 50 generic virtual machines and provide high availability and server redundancy.
Following are the components used for the design and deployment:
•
•
•
•
•
•
•
•
•
The solution is designed to host scalable, mixed application workloads. The scope of this CVD is limited to the Cisco solution for EMC VSPEX with VMware solutions up to 50 virtual machines only.
Technology Overview
Cisco Unified Computing System
The Cisco Unified Computing System is a next-generation data center platform that unites computing, network, storage access, and virtualization into a single cohesive system.
The main components of the Cisco UCS are:
•
•
•
•
The Cisco Unified Computing System is designed to deliver:
•
•
•
•
Cisco UCS C220 M3 Rack-Mount Servers
Building on the success of the Cisco UCS C200 M2 Rack Servers, the enterprise-class Cisco UCS C220 M3 server further extends the capabilities of the Cisco Unified Computing System portfolio in a 1-rack-unit (1RU) form factor. And with the addition of the Intel® Xeon® processor E5-2600 product family, it delivers significant performance and efficiency gains.
Figure 1 Cisco UCS C220 M3 Rack Server
The Cisco UCS C220 M3 offers up to 256 GB of RAM, up to eight drives or SSDs, and two 1GE LAN interfaces built into the motherboard, delivering outstanding levels of density and performance in a compact package.
Cisco Nexus 3048 Switch
The Cisco Nexus® 3048 Switch is a line-rate Gigabit Ethernet top-of-rack (ToR) switch and is part of the Cisco Nexus 3000 Series Switches portfolio. The Cisco Nexus 3048, with its compact one-rack-unit (1RU) form factor and integrated Layer 2 and 3 switching, complements the existing Cisco Nexus family of switches. This switch runs the industry-leading Cisco® NX-OS Software operating system, providing customers with robust features and functions that are deployed in thousands of data centers worldwide.
Figure 2 Cisco Nexus 3048 Switch
EMC Storage Technologies and Benefits
The VNXe™ series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security.
The VNXe series is purpose-built for the IT manager in smaller environments. The EMC VNXe storage arrays are multi-protocol platform that can support the iSCSI, NFS, and CIFS protocols depending on the customer's specific needs. The solution was validated using iSCSI for data storage.
VNXe series storage arrays have following customer benefits:
•
•
•
•
•
Software Suites Available
•
•
•
Software Packs Available
Total Value Pack—Includes all protection software suites and the Security and Compliance Suite
This is the available EMC protection software pack.
The VNXe™ series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security.
The VNXe series is purpose-built for the IT manager in smaller environments. The EMC VNXe storage arrays are multi-protocol platforms that can support the iSCSI, NFS, and CIFS protocols depending on the customer's specific needs. The solution was validated using iSCSI for data storage.
EMC NetWorker and Data Domain
EMC's NetWorker coupled with Data Domain deduplication storage systems seamlessly integrate into virtual environments, providing rapid backup and restoration capabilities. Data Domain deduplication results in vastly less data traversing the network by leveraging the Data Domain Boost technology, which greatly reduces the amount of data being backed up and stored, translating into storage bandwidth and operational savings.
The following are two of the most common recovery requests made to backup administrators:
•
•
The NetWorker System State protection functionality adds backup and recovery capabilities in both of these scenarios.
EMC Avamar
EMC's Avamar® data deduplication technology seamlessly integrates into virtual environments, providing rapid backup and restoration capabilities. Avamar's deduplication results in vastly less data traversing the network, and greatly reduces the amount of data being backed up and stored - translating into storage, bandwidth and operational savings.
VMware vSphere 5.1
VMware vSphere 5.1 transforms a computer's physical resources by virtualizing the CPU, memory, storage and network functions. This transformation creates fully functional virtual machines that run isolated and encapsulated operating systems and applications just like physical computers.
The high availability features of VMware vSphere 5.1 such as vMotion and Storage vMotion enable seamless migration of virtual machines and stored files from one vSphere server to another with minimal or no performance impact. Coupled with vSphere DRS and Storage DRS, virtual machines have access to the appropriate resources at any point in time through load balancing of compute and storage resources.
VMware vCenter
VMware vCenter is a centralized management platform for the VMware Virtual Infrastructure. It provides administrators with a single interface for all aspects of monitoring, managing, and maintaining the virtual infrastructure that can be accessed from multiple devices.
VMware vCenter is also responsible for managing some of the more advanced features of the VMware virtual infrastructure like VMware vSphere High Availability (vSphere HA) and Distributed Resource Scheduling (DRS), along with the vMotion and Update Manager.
Architectural overview
This CVD focuses on VMware solution for up to 50 virtual machines.
For the VSPEX solution, the reference workload was defined as a single virtual machine. Characteristics of a virtual machine are defined in Table 1.
See "Sizing Guideline" section for more detailed information.
Solution architecture overview
Table 2 lists the mix of hardware components, their quantities and software components used in this architecture.
Table 3 lists the various hardware and software versions of the components which occupies different tiers of the Cisco solution for EMC VSPEX with VMware architectures under test.
Table 4 outlines the C220 M3 server configuration across all the VMware architectures. Table 4 shows the configuration on per server basis.
The reference architecture assumes that there is an existing infrastructure/ management network available where a virtual machine hosting vCenter server and Windows Active Directory/ DNS server are present. The below diagram illustrates high level solution architecture for 50 virtual machines.
Figure 3 Reference Architecture for 50 Virtual Machines
As it is evident in the above diagrams, following are the high level design points of VMware architectures:
•
•
•
This design does not dictate or require any specific layout of infrastructure network. The vCenter server and Microsoft Windows Active Directory are hosted on infrastructure network. However, design does require accessibility of certain VLANs from the infrastructure network to reach the servers.
ESXi 5.1 is used as hypervisor operating system on each server and is installed on local hard drives. Typical load is 25 virtual machines per server.
Memory Configuration Guidelines
This section provides guidelines for allocating memory to the virtual machines. The guidelines outlined here take into account vSphere memory overhead and the virtual machine memory settings.
ESX/ESXi Memory Management Concepts
vSphere virtualizes guest physical memory by adding an extra level of address translation. Shadow page tables make it possible to provide this additional translation with little or no overhead. Managing memory in the hypervisor enables the following:
•
•
•
For more information about vSphere memory management concepts, see the VMware vSphere Resource Management Guide at: http://www.vmware.com/files/pdf/perf-vsphere-memory_management.pdf
Virtual Machine Memory Concepts
Figure 4 shows the use of memory settings parameters in the virtual machine.
Figure 4 Virtual Machine Memory Settings
The vSphere memory settings for a virtual machine include the following parameters:
•
•
•
Allocating Memory to Virtual Machines
Memory sizing for a virtual machine in VSPEX architectures is based on many factors. With the number of application services and use cases available determining a suitable configuration for an environment requires creating a baseline configuration, testing, and making adjustments, as discussed later in this paper. Table 1 outlines the resources used by a single virtual machine:
Following are the recommended best practices:
•
•
•
–
–
–
Storage Guidelines
VSPEX architecture for VMware virtual machines at 50 VMs scale uses iSCSI to access storage arrays. This simplifies the design and implementation for the small to medium level businesses. vSphere provides many features that take advantage of EMC storage technologies such as VNX VAAI plug-in for NFS storage and storage replication. Features such as VMware vMotion, VMware HA, and VMware Distributed Resource Scheduler (DRS) use these storage technologies to provide high availability, resource balancing, and uninterrupted workload migration.
Storage Protocol Capabilities
VMware vSphere provides vSphere and storage administrators with the flexibility to use the storage protocol that meets the requirements of the business. This can be a single protocol datacenter wide, such as iSCSI, or multiple protocols for tiered scenarios such as using Fibre Channel for high-throughput storage pools and NFS for high-capacity storage pools.
For VSPEX solution on vSphere NFS is a recommended option because of its simplicity in deployment.
For more information, see the VMware white paper Comparison of Storage Protocol Performance in VMware vSphere 5.1: http://www.vmware.com/files/pdf/perf_vsphere_storage_protocols.pdf
Storage Best Practices
Following are the vSphere storage best practices:
•
•
•
•
•
VMware Memory Virtualization for VSPEX
VMware vSphere 5.1 has a number of advanced features that help to maximize performance and overall resources utilization. This section describes the performance benefits of some of these features for the VSPEX deployment.
Memory Compression
Memory over-commitment occurs when more memory is allocated to virtual machines than is physically present in a VMware ESXi host. Using sophisticated techniques, such as ballooning and transparent page sharing, ESXi is able to handle memory over-commitment without any performance degradation. However, if more memory than that is present on the server is being actively used, ESXi might resort to swapping out portions of a VM's memory.
For more details about Vsphere memory management concepts, see the VMware Vsphere Resource Management Guide at: http://www.VMware.com/files/pdf/mem_mgmt_perf_Vsphere5.pdf
Virtual Networking Best Practices
Following are the vSphere networking best practices:
•
•
•
•
VMware Storage Layout for VSPEX
This section explains the EMC storage layout used for this solution.
Storage Layout
The architecture diagram in this section shows the physical disk layout. Disk provisioning on the VNXe series is simplified through the use of wizards, so that administrators do not choose which disks belong to a given storage pool. The wizard may choose any available disk of the proper type, regardless of where the disk physically resides in the array.
Figure 5 shows storage architecture for 50 virtual machines on VNXe3150:
Figure 5 Storage Architecture for 50 VMs on EMC VNXe3150
Table 5 provides size of datastores for VMware 50 VMs architecture laid out in Figure 5.
Table 5 Datastore Details for V50 Architecture
Disk capacity and type
300GB SAS
Number of disks
45
RAID type
4 + 1 RAID 5 groups
Number of pools
1
Hot spare disks
2
The reference architecture uses the following configuration:
•
•
•
The VNX/VNXe family is designed for five 9s availability by using redundant components throughout the array. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures, and the available hot spare drives can be dynamically allocated to replace a failing disk.
Storage Virtualization
VMFS is a cluster file system that provides storage virtualization optimized for virtual machines. Each virtual machine is encapsulated in a small set of files and VMFS is the default storage system for these files on physical SCSI disks and partitions.
It is preferable to deploy virtual machine files on shared storage to take advantage of VMware VMotion, VMware High Availability™ (HA), and VMware Distributed Resource Scheduler™ (DRS). This is considered a best practice for mission-critical deployments, which are often installed on third-party, shared storage management solutions.
Architecture for 50 VMware virtual machines
Figure 6 demonstrates logical layout of 50 VMware virtual machines. Following are the key aspects of this solution:
•
•
•
•
•
•
Figure 6 Logical Layout Diagram for VMware 50 VMs
Sizing Guideline
In any discussion about virtual infrastructures, it is important to first define a reference workload. Not all servers perform the same tasks, and it is impractical to build a reference that takes into account every possible combination of workload characteristics.
Defining the Reference Workload
To simplify the discussion, we have defined a representative reference workload. By comparing your actual usage to this reference workload, you can extrapolate which reference architecture to choose.
For the VSPEX solutions, the reference workload was defined as a single virtual machine. This virtual machine characteristics is shown in Table 1. This specification for a virtual machine is not intended to represent any specific application. Rather, it represents a single common point of reference to measure other virtual machines.
Applying the Reference Workload
When considering an existing server that will move into a virtual infrastructure, you have the opportunity to gain efficiency by right-sizing the virtual hardware resources assigned to that system.
The reference architectures create a pool of resources sufficient to host a target number of reference virtual machines as described above. It is entirely possible that your virtual machines may not exactly match the specifications above. In that case, you can say that a single specific virtual machine is the equivalent of some number of reference virtual machines, and assume that number of virtual machines have been used in the pool. You can continue to provision virtual machines from the pool of resources until it is exhausted. Consider these examples:
Example 1 Custom Built Application
A small custom-built application server needs to move into this virtual infrastructure. The physical hardware supporting the application is not being fully utilized at present. A careful analysis of the existing application reveals that the application can use one processor and needs 3 GB of memory to run normally. The IO workload ranges between 4 IOPS at idle time to 15 IOPS when busy. The entire application is only using about 30 GB on local hard drive storage.
Based on these numbers, the following resources are needed from the resource pool:
•
•
•
•
In this example, a single virtual machine uses the resources of two of the reference VMs. If the original pool had the capability to provide 50 VMs worth of resources, the new capability is 48 VMs.
Example 2 Point of Sale System
The database server for a customer's point-of-sale system needs to move into this virtual infrastructure. It is currently running on a physical system with four CPUs and 16 GB of memory. It uses 200 GB storage and generates 200 IOPS during an average busy cycle.
The following are the requirements to virtualize this application:
•
•
•
•
In this case the one virtual machine uses the resources of eight reference virtual machines. If this was implemented on a resource pool for 50 virtual machines, there are 42 virtual machines of capability remaining in the pool.
Example 3 Web Server
The customer's web server needs to move into this virtual infrastructure. It is currently running on a physical system with two CPUs and 8GB of memory. It uses 25 GB of storage and generates 50 IOPS during an average busy cycle.
The following are the requirements to virtualize this application:
•
•
•
•
In this case the virtual machine would use the resources of four reference virtual machines. If this was implemented on a resource pool for 50 virtual machines, there are 46 virtual machines of capability remaining in the pool.
Example 4 Decision Support Database
The database server for a customer's decision support system needs to move into this virtual infrastructure. It is currently running on a physical system with ten CPUs and 48 GB of memory. It uses 5 TB of storage and generates 700 IOPS during an average busy cycle.
The following are the requirements to virtualize this application:
•
•
•
•
In this case the one virtual machine uses the resources of fifty-two reference virtual machines. If this was implemented on a resource pool for 100 virtual machines, there are 48 virtual machines of capability remaining in the pool.
Summary of Example
The three examples presented illustrate the flexibility of the resource pool model. In all three cases the workloads simply reduce the number of available resources in the pool. If all three examples were implemented on the same virtual infrastructure, with an initial capacity of 50 virtual machines they can all be implemented, leaving the capacity of thirty six reference virtual machines in the resource pool.
In more advanced cases, there may be tradeoffs between memory and I/O or other relationships where increasing the amount of one resource decreases the need for another. In these cases, the interactions between resource allocations become highly complex, and are outside the scope of this document. However, once the change in resource balance has been examined, and the new level of requirements is known; these virtual machines can be added to the infrastructure using the method described in the examples.
VSPEX Configuration Guidelines
The configuration for Cisco solution for EMC VSPEX with VMware architectures is divided into the following steps:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
Each of these steps are discussed in detail in the following sections.
Pre-deployment Tasks
Pre-deployment tasks include procedures that do not directly relate to environment installation and configuration, but whose results will be needed at the time of installation. Examples of pre-deployment tasks are collection of hostnames, IP addresses, VLAN IDs, license keys, installation media, and so on. These tasks should be performed before the customer visit to decrease the time required onsite.
•
•
•
Customer Configuration Data
To reduce the onsite time, information such as IP addresses and hostnames should be assembled as part of the planning process.
"Customer Configuration Data Sheet" section provides a set of tables to maintain a record of relevant information. This form can be expanded or contracted as required, and information may be added, modified, and recorded as deployment progresses.
Additionally, complete the VNXe Series Configuration Worksheet, available on the EMC online support website, to provide the most comprehensive array-specific information.
Cabling Information
The following information is provided as a reference for cabling the physical equipment in a VSPEX V50 environment. Figure 8 and Figure 9 in this section provide both local and remote device and port locations in order to simplify cabling requirements.
This document assumes that out-of-band management ports are plugged into an existing management infrastructure at the deployment site.
Be sure to follow the cable directions in this section. Failure to do so will result in necessary changes to the deployment procedures that follow because specific port locations are mentioned. Before starting, be sure that the configuration matches what is described in Figure 7, Figure 8, and Figure 9.
Figure 7 shows a VSPEX V50 cabling diagram. The labels indicate connections to end points rather than port numbers on the physical device. For example, connection A is a 1 Gb target port connected from EMC VNXe3150 SP B to Cisco Nexus 3048 A and connection R is a 1 Gb target port connected from Broadcom NIC 3 on Server 2 to Cisco Nexus 3048 B. Connections W and X are 10 Gb vPC peer-links connected from Cisco Nexus 3048 A to Cisco Nexus 3048 B.
Figure 7 VSPEX V50 Cabling Diagram
Figure 7, Figure 8, Figure 9 elaborates the detailed cable connectivity for the 50 virtual machines configuration.
Figure 8 Cisco Nexus 3048-A Ethernet Cabling Information
Figure 9 Cisco Nexus 3048-B Ethernet Cabling Information
Connect all the cables as outlined in the Figure 7, Figure 8, and Figure 9.
Prepare and Configure the Cisco Nexus 3048 Switch
This section provides a detailed procedure for configuring the Cisco Nexus 3048 switches for use in EMC VSPEX V50 solution.
See the Nexus 3048 configuration guide for detailed information about how to mount the switches on the rack. Following diagrams show connectivity details for the VMware architecture covered in this document.
As it is apparent from these figures, there are five major cabling sections in these architectures:
1.
2.
3.
4.
Figure 10 shows two switches configured for vPC. In vPC, a pair of switches acting as vPC peer endpoints looks like a single entity to port-channel-attached devices, although the two devices that act as logical port-channel endpoint are still two separate devices. This provides hardware redundancy with port-channel benefits. Both switches form a vPC Domain, in which one vPC switch is Primary while the other is secondary.
Note
Figure 10 Network Configuration for EMC VSPEX V50
Initial Setup of Nexus Switches
This section details the Cisco Nexus 3048 switch configuration for use in a VSPEX V50 environment.
This section explains switch configuration needed for the Cisco solution for EMC VSPEX with VMware architectures. Details about configuring password, management connectivity and strengthening the device are not covered here; please refer to the Nexus 3000 series configuration guide for that.
On initial boot and connection to the serial or console port of the switch, the NX-OS setup should automatically start. This initial configuration addresses basic settings such as the switch name, the mgmt0 interface configuration, and SSH setup and defines the control plane policing policy.
Initial Configuration of Cisco Nexus 3048 Switch A and B
Figure 11 Initial Configuration
Software Upgrade (Optional)
It is always recommended to perform any required software upgrades on the switch at this point in the configuration. Download and install the latest available NX-OS software for the Cisco Nexus 3048 switch from the Cisco software download site. There are various methods to transfer both the NX-OS kick-start and system images to the switch. The simplest method is to leverage the USB port on the Switch. Download the NX-OS kick-start and system files to a USB drive and plug the USB drive into the external USB port on the Cisco Nexus 3048 switch.
Copy the files to the local bootflash and update the switch by using the following procedure.
Figure 12 Procedure to update the Switch
Enable Features
Enable certain advanced features within NX-OS. This is required for configuring some additional options. Enter configuration mode using the (config t) command, and type the following commands to enable the appropriate features on each switch.
Enabling Features in Cisco Nexus 3048 Switch A and B
Figure 13 Command to Enable Features
Global Port-Channel Configuration
The default port-channel load-balancing hash uses the source and destination IP to determine the load-balancing algorithm across the interfaces in the port channel. Better distribution across the members of the port channels can be achieved by providing more inputs to the hash algorithm beyond the source and destination IP. For this reason, adding the source and destination TCP port to the hash algorithm is highly recommended.
From configuration mode (config t), type the following commands to configure the global port-channel load-balancing configuration on each switch.
Configuring Global Port-Channel Load-Balancing on Cisco Nexus Switch A and B
Figure 14 Commands to Configure Global Port-Channel and Load-Balancing
Global Spanning-Tree Configuration
The Cisco Nexus platform leverages a new protection feature called bridge assurance. Bridge assurance helps to protect against a unidirectional link or other software failure and a device that continues to forward data traffic when it is no longer running the spanning-tree algorithm. Ports can be placed in one of a few states depending on the platform, including network and edge.
The recommended setting for bridge assurance is to consider all ports as network ports by default. From configuration mode (config t), type the following commands to configure the default spanning-tree options, including the default port type and BPDU guard on each switch.
Configuring Global Spanning-Tree on Cisco Nexus Switch A and B
Figure 15 Configuring Spanning-Tree
Enable Jumbo Frames
Cisco solution for EMC VSPEX with VMware architectures require MTU set at 9000 (jumbo frames) for efficient storage and live migration traffic. MTU configuration on Nexus 5000 series switches fall under global QoS configuration. You may need to configure additional QoS parameters as needed by the applications.
From configuration mode (config t), type the following commands to enable jumbo frames on each switch.
Enabling Jumbo Frames on Cisco Nexus 3048 Switch A and B
Figure 16 Enabling Jumbo Frames
Configure VLANs
For VSPEX M50 configuration, create the layer 2 VLANs on both the Cisco Nexus 3048 Switches using the Table 7 as reference. Create your own VLAN definition table with the help of "Customer Configuration Data Sheet" section.
From configuration mode (config t), type the following commands to define and describe the L2 VLANs.
Defining L2 VLANs on Cisco Nexus 3048 Switch A and B
Figure 17 Commands to Define L2 VLANs
Virtual Port-Channel (vPC) Global Configuration
Virtual port-channel effectively enables two physical switches to behave like a single virtual switch, and port-channel can be formed across the two physical switches.
The vPC feature requires an initial setup between the two Cisco Nexus switches to function properly. From configuration mode (config t), type the following commands to configure the vPC global configuration for Switch A.
Configuring vPC Global on Cisco Nexus Switch A
Figure 18 Commands to Configure vPC Global Configuration on Switch A
From configuration mode (config t), type the following commands to configure the vPC global configuration for Switch B.
Configuring vPC Global on Cisco Nexus Switch B
Figure 19 Commands to Configure vPC Global Configuration on Switch B
Configuring Storage Connections
Switch interfaces connected to the VNXe storage ports are configured as access ports. Each controller will have two links to each switch.
From the configuration mode (config t), type the following commands on each switch to configure the individual interfaces.
Cisco Nexus 3048 Switch A with VNXe SPA configuration
Figure 20 Commands to Configure VNXe Interface on Switch A
Cisco Nexus 3048 Switch B with VNXe SPA configuration
Figure 21 Commands to Configure VNXe Interface on Switch B
Configuring Server Connections
Each server has six network adapters (two Intel and four Broadcom ports) connected to both switches for redundancy as shown in Figure 10. This section provides the steps to configure the interfaces on both the switches that are connected to the servers.
Cisco Nexus Switch A with Server 1 configuration
Figure 22 Commands to Configure Interface on Switch A for Server 1 Connectivity
Cisco Nexus Switch B with Server 1 configuration
Figure 23 Commands to Configure Interface on Switch B and Server 1 Connectivity
Cisco Nexus Switch A with Server 2 configuration
Figure 24 Commands to Configure Interface on Switch A and Server 2 Connectivity
Cisco Nexus Switch B with Server 2 configuration
Figure 25 Commands to Configure Interface on Switch B and Server 2 Connectivity
Cisco Nexus Switch A with Server 3 configuration
Figure 26 Commands to Configure Interface on Switch A and Server 3 Connectivity
Cisco Nexus Switch B with Server 3 configuration
Figure 27 Commands to Configure Interface on Switch B and Server 3 Connectivity
Configure ports connected to infrastructure network
Port connected to infrastructure network need to be in trunk mode, and they require at least infrastructure and management VLANs at the minimum. You may require enabling more VLANs as required by your application domain. For example, Windows virtual machines may need to access to active directory / DNS servers deployed in the infrastructure network. You may also want to enable port-channels and virtual port-channels for high availability of infrastructure network.
Verify VLAN and port-channel configuration
At this point of time, all ports and port-channels are configured with necessary VLANs, switchport mode and vPC configuration. Validate this configuration using the show vlan, show port-channel summary and show vpc commands as shown in the following figures.
Figure 28 Show vlan brief
Figure 29 Show Port-Channel Summary Output
In this example, port-channel 10 is the vPC peer-link port-channel, port-channels 3, 4 and 5 are connected to the Cisco 1GigE I350 LOM on the host and port-channels 14, 16 and 18 are connected to the Broadcom NICs on the host. Make sure that state of the member ports of each port-channel is "P" (Up in port-channel). Note that port may not come up if the peer ports are not properly configured. Common reasons for port-channel port being down are:
•
•
vPC status can be verified using show vpc brief command. Example output is shown in Figure 30:
Figure 30 Show vpc Brief Output
Make sure that vPC peer status is peer adjacency formed ok and all the port-channels, including the peer-link port-channel, have status up.
Prepare the Cisco UCS C220 M3 Servers
This section provides the detailed procedure for configuring a Cisco Unified Computing System C-Series standalone server for use in VSPEX M50 configurations. Perform all the steps mentioned in this section on all the hosts.
For information on physically mounting the servers, see:
http://www.cisco.com/en/US/docs/unified_computing/ucs/c/hw/C220/install/install.html
Configure Cisco Integrated Management Controller (CIMC)
These steps describe the setup of the initial Cisco UCS C-Series standalone server. Follow these steps on all servers:
1.
2.
3.
4.
5.
6.
7.
Figure 31 CIMC Configuration Utility
Once the CIMC IP is configured, the server can be managed using the https based Web GUI or CLI.
Note
Enabling Virtualization Technology in BIOS
Vmware requires an x64-based processor, hardware-assisted virtualization (Intel VT enabled), and hardware data execution protection (Execute Disable enabled). Follow these steps on all the servers to enable Intel ® VT and Execute Disable in BIOS:
1.
2.
3.
4.
Figure 32 Cisco UCS C220 M3 KVM Console
Configuring RAID
The RAID controller type is Cisco UCSC RAID SAS 2008 and supports 0, 1, 5 RAID levels. We need to configure RAID level 1 for this setup and set the virtual drive as boot drive.
To configure RAID controller, follow these steps on all the servers:
1.
2.
Figure 33 Cisco UCS C220 M3 CIMC GUI
3.
Figure 34 Cisco UCS C220 M3 KVM Console - Server Booting
4.
Figure 35 Adapter Selection for RAID Configuration
5.
Figure 36 MegaRAID BIOS Config Utility Configuration Wizard
6.
Figure 37 MegaRAID BIOS Config Utility Confirmation Page
7.
Figure 38 MegaRAID BIOS Config Utility Configuration Wizard - Select Configuration Method
8.
Figure 39 MegaRAID BIOS Config Utility Config Wizard - Preview
9.
Figure 40 MegaRAID BIOS Config Utility Confirmation Page
10.
Figure 41 MegaRAID BIOS Config Utility Virtual Drives
11.
Figure 42 MegaRAID BIOS Config Utility Virtual Configuration
Install ESXi 5.1 on Cisco UCS C220 M3 Servers
This section provides detailed procedures for installing ESXi 5.1 in a V50 VSPEX configuration. Multiple methods exist for installing ESXi in such an environment. This procedure highlights using the virtual KVM console and virtual media features within the Cisco UCS C-Series CIMC interface to map remote installation media to each individual server.
Connect and log into the Cisco UCS C-Series Standalone Server CIMC Interface
1.
2.
Figure 43 CIMC Manager Login Page
3.
4.
5.
a.
b.
c.
Figure 44 CIMC Manager Remote Presence - Virtual Media
6.
Figure 45 CIMC Manager Remote Presence - Virtual KVM
7.
8.
Figure 46 CIMC Manager Virtual Media - Add Image
9.
10.
11.
Figure 47 Cisco UCS C220 M3 BIOS Setup Utility
12.
Installing VMware ESXi
The section explains the steps to complete the installation of ESXi 5.1 on Cisco C220 M3 server and basic configuration of management network for remote access. Follow these steps on all the three Cisco C220 M3 servers:
1.
Figure 48 ESXi installation boot menu
2.
Figure 49 ESXi installation boot welcome screen
3.
4.
5.
6.
Figure 50 ESXi installation - Set Password
7.
Figure 51 ESXi installation - Confirm Install
8.
Figure 52 ESXi installation Complete - Reboot
9.
10.
11.
Figure 53 ESXi System Customization
12.
13.
Figure 54 ESXi Configure Management Network - Network Adapters
14.
15.
Figure 55 ESXi Configure Management Network - IP Configuration
16.
17.
Figure 56 ESXi Configure Management Network - VLAN
18.
Once the above steps are completed you can further configure and manage the ESXi remotely using the vSphere Client, the vSphere Web client and vCenter Server. To manage the host with the vSphere Client, the vSphere Web Client, and vCenter Server, the applications must be installed on a computer that serves as a management station with network access to the ESXi host. To download and install vSphere client, open a web browser and enter the IP address of the ESXi management network that you configured in the above step 15. Use the below URL for detailed information on installing the vSphere client:
VMware vCenter Server Deployment
This section describes the installation of VMware vCenter for VMware environment and to get the following configuration:
•
•
•
For more information on installing a vCenter Server, see:
Following steps provides high level configuration steps to configure vCenter server:
1.
If the VMware vCenter Server is to be deployed as a virtual machine on an ESXi server installed as part of this solution, connect directly to an Infrastructure ESXi server using the vSphere Client. Create a virtual machine on the ESXi server with the customer's guest OS configuration, using the Infrastructure server datastore presented from the storage array. The memory and processor requirements for the vCenter Server are dependent on the number of ESXi hosts and virtual machines being managed. The requirements are outlined in the vSphere Installation and Setup Guide.
2.
Install the guest OS on the vCenter host virtual machine. VMware recommends using Windows Server 2008 R2 SP1. To ensure that adequate space is available on the vCenter and vSphere Update Manager installation drive, see vSphere Installation and Setup Guide.
3.
Before installing vCenter Server and vCenter Update Manager, you must create the ODBC connections required for database communication. These ODBC connections will use SQL Server authentication for database authentication. Appendix A Customer Configuration Data provides SQL login information.
For instructions on how to create the necessary ODBC connections see, vSphere Installation and Setup and Installing and Administering VMware vSphere Update Manager.
4.
Install vCenter by using the VMware VIMSetup installation media. Use the customer-provided username, organization, and vCenter license key when installing vCenter.
5.
To perform license maintenance, log into the vCenter Server and select the Administration - Licensing menu from the vSphere client. Use the vCenter License console to enter the license keys for the ESXi hosts. After this, they can be applied to the ESXi hosts as they are imported into vCenter server.
Adding ESXi hosts to vCenter or Configuring Hosts and vCenter Server
This section describes on how to populate and organize your inventory by creating a virtual datacenter object in vCenter server and adding the ESXi hosts to it. A virtual datacenter is a container for all the inventory objects required to complete a fully functional environment for operating virtual machines.
1.
Figure 57 vSphere Client
2.
Figure 58 VMware vCenter - Host and Clusters
3.
Figure 59 VMware vCenter - Create Datacenter
4.
Figure 60 VMware vCenter - Add a Host
5.
Figure 61 Add Host Wizard -Connection Settings
6.
7.
Figure 62 Add Host Wizard - Host Summary
8.
Figure 63 Add Host Wizard -Assign License
9.
10.
Figure 64 Add Host Wizard -Ready to Complete
11.
Figure 65 VMware vCenter with hosts added
Configure ESXi Networking
This section instructs how to configure the networking for ESXi hosts using the vCenter server. In this document we will be using four virtual standard switches (vSS). vSwitches used for management and vMotion traffic would have two vmnics, one on each fabric for load balancing and high-availability. The other two vSwitches used for iSCSI storage will have each with a single vmnic uplink and a single vmk port, bound to the iSCSI adapter. The VSPEX solution recommends an MTU set at 9000 (jumbo frames) for efficient storage and migration traffic. The below table can be used as a reference for creating and configuring virtual switch on ESXi hosts.
Configuring vSwitch for Management and VM traffic
During the installation of VMware ESXi, a virtual standard switch (vSS) will be created. By default, ESXi chooses only one physical NIC as a virtual switch uplink. This section describes the steps to add a redundant link and configure the load balancing for the default vSwitch0. Policies set at the standard switch apply to all of the port groups on the standard switch. The exceptions are the configuration options that are overwritten at the standard port group. The vSwitch for management and VM port group uses the same policy settings configured on the standard vSwitch properties.
1.
2.
Figure 66 VMware vCenter - Networking
3.
Figure 67 Management vSwitch Properties
4.
Figure 68 Management vSwitch - Add Adapter Wizard
5.
6.
7.
Figure 69 Management vSwitch - Ports Edit
8.
Figure 70 Management vSwitch - NIC Teaming
Note
Since we are using Ether channel for link aggregation on the Cisco Nexus switches, the supported vSwitch NIC teaming mode is IP hash. For more details, see the VMware KB articles at:
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1004048
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&externalId=1001938
9.
Figure 71 Management vSwitch - VM Network VLAN Settings
Figure 72 Management VMkernel Port-Group
10.
Create and Configure vSwitch for vMotion
In this section we will create a new vSwitch for vMotion, add two uplink active adapters and set the load balancing policy to route based on IP hash. The MTU on this vSwitch will be set to 9000 (jumbo frames)
1.
2.
Figure 73 VMware vCenter - Add Networking
3.
Figure 74 Add Network Wizard - Connection Type
4.
Figure 75 Add Network Wizard - Network Access
5.
Figure 76 Add Network Wizard - Connection Settings
6.
Figure 77 Add Network Wizard - VMkernel IP Settings
7.
Figure 78 VMware vCenter - Network Configuration
8.
Figure 79 vSwitch Properties - MTU
9.
Figure 80 vSwitch Properties - NIC Teaming Policy
10.
Create and Configure vSwitch for iSCSI
The iSCSI adapter and physical NIC connect through a virtual VMkernel adapter, also called virtual network adapter or VMkernel port. You create a VMkernel adapter (vmk) on a vSphere switch (vSwitch) using 1:1 mapping between each virtual and physical network adapter.
One way to achieve the 1:1 mapping when you have multiple NICs, is to designate a separate vSphere switch for each virtual-to-physical adapter pair as shown in the below figure.
Figure 81 1:1 adapter mapping on separate vSphere standard switches
Note
In this section we will be creating two vSwitches, each with a single vmnic uplink and a single VMkernel port, bound to the iSCSI adapter. A MTU size of 9000 will also be set on the vSwitches and VMkernal ports to enable jumbo frames. Follow these steps to create vSwitch:
1.
2.
Figure 82 VMware vCenter - Add Networking
3.
Figure 83 Add Network Wizard - Connection Type
4.
Figure 84 Add Network Wizard - Network Access
5.
Figure 85 Add Network Wizard - Connection Settings
6.
Figure 86 Add Network Wizard - VMkernel IP Settings
7.
Figure 87 iSCSI vSwitch - Properties
8.
Figure 88 iSCSI vSwitch Properties - Edit
9.
Figure 89 iSCSI vSwitch Properties - MTU
10.
Figure 90 iSCSI vSwitches vCenter view
Configuring Software iSCSI Adapter (vmhba)
The software iSCSI adapter (vmbha) uses standard NICs to connect your host to a remote iSCSI target on the IP network. The software iSCSI adapter that is built into ESXi facilitates this connection by communicating with the physical NICs through the network stack. Before you can use the software iSCSI adapter, you must set up networking (this is completed in the previous section - Create and Configure vSwitch2 for iSCSI), activate the adapter, and configure parameters such as discovery addresses and CHAP. This section describes these steps to configure the software iSCSI storage adapter to access the remote storage configured on the VNXe3150 storage array.
1.
2.
3.
Figure 91 iSCSI Software Adapter
4.
Figure 92 iSCSI Software Adapter Properties
5. Make sure that the adapter is enabled and click OK. After enabling the adapter, the host assigns the default iSCSI name to it. If you change the default name, follow iSCSI naming conventions.
Figure 93 iSCSI Software Adapter - Enable
5.
Figure 94 iSCSI Software Adapter Properties - Network Configuration
6.
Figure 95 iSCSI Software Adapter Properties - Bind VMkernel
The network connection appears on the list of VMkernel port bindings for the iSCSI adapter as shown in Figure 96.
Figure 96 iSCSI Software Adapter Properties - Network Configuration
Prepare the EMC VNXe3150 Storage
Figure 97 shows the steps involved in deploying VMware environments by using VNXe storage.
Figure 97 VNXe VMware vSphere storage provisioning flowchart
Initial Setup of VNXe
1.
2.
http://www.emc.com/support-training/support/emc-powerlink.htm
3.
4.
Figure 98 EMC Unisphere - Dashboard Page
Note
iSCSI Server Configuration
The iSCSI Storage Server is the portal through which storage will be accessed by the hosts within the Fast Track configuration. The goal of the proposed iSCSI server configuration is to provide redundancy, multi-pathing and balanced access across all 1 GigE connections and both storage processors. Each 1 GigE module will have 2 ports, referred to as eth10 and eth11. Considering there is an I/O module for each storage processor, both SPA and SPB will have eth10 and eth11 connections.
iSCSI servers will run on either SPA or SPB. This means storage assigned to a given iSCSI server will only be available to one SP at a given time. To utilize both SPA and SPB concurrently, two iSCSI servers will be created.
With respect to iSCSI server high availability, the eth10 and eth11 connections are paired across the storage processors. If an iSCSI server running with an IP address dedicated to eth10 on SP A needs to move to SP B, for maintenance as an example, the IP address will move to the corresponding eth10 port on SPB. Therefore subnet connectivity will need to be the same for the associated eth10 and eth11 connections across the storage processors. Figure 99 shows a logical example of the connections.
Figure 99 VNXe Array Logical Network Connections
Configure iSCSI Storage Servers
1.
2.
Figure 100 EMC Unisphere - iSCSI Server SP-A eth10
3.
Figure 101 EMC Unisphere - iSCSI Server SP-B eth10
4.
Figure 102 EMC Unisphere - iSCSI Server Settings
5.
6.
Figure 103 EMC Unisphere - iSCSI Server SP-A eth11
7.
Figure 104 EMC Unisphere - iSCSI Server SP-B eth11
Note
http://www.emc.com/collateral/hardware/white-papers/h8178-vnxe-storage-systems-wp.pdf
Figure 105 EMC Unisphere - iSCSI Server Settings
Add VMware Hosts to EMC VNXe Storage
To integrate virtual infrastructure with VNXe for storage provisioning, follow these steps:
1.
Figure 106 EMC Unisphere - Hosts
2.
Figure 107 EMC Unisphere - Find ESX Hosts
3.
Figure 108 EMC Unisphere - Find ESX Hosts
4.
Figure 109 EMC Unisphere - vCenter Credentials
5.
Figure 110 EMC Unisphere - Find ESX Hosts Selection
6.
Create Storage Pools for VMware
A storage pool is an aggregation of storage disk resources that are configured with a particular storage profile. The storage profile defines the type of disks that are used to provide storage and the RAID configuration of the component disks. The storage pool configuration defines the number of disks and the quantity of storage associated with the pool.
Create a pool with the appropriate number of disks as suggested in the storage layout figure.
1.
2.
Figure 111 EMC Unisphere - Storage Pools Configure Disks
3.
4.
Figure 112 EMC Unisphere - Disk Configuration Wizard, Select Mode
5.
Figure 113 EMC Unisphere - Disk Configuration Wizard, Specify Pool Name
6.
Figure 114 EMC Unisphere - Disk Configuration Wizard, Select Storage Type
7.
Figure 115 EMC Unisphere - Disk Configuration Wizard, Select Amount of Storage
8. Review the summary and click Finish.
Note
Configure Jumbo Frames
The Cisco networking environment will have a Maximum Transmission Unit (MTU) size of 9000 for the iSCSI connections to the VNXe. To match the configured MTU size via EMC Unisphere, follow these steps:
1.
2.
3.
Figure 116 EMC Unisphere - Advanced Configuration Jumbo Frames
Create VMware Datastores
There are two options to create datastores in the EMC VNXe storage array. We have used generic iSCSI storage option to create the virtual disks and present them to the ESXi hosts. However, you can create VMware/VMFS datastores using VMware option in the storage section of EMC Unisphere.
Note
To create an iSCSI Virtual disk on iSCSI storage on a VNXe platform, follow these steps:
1.
Figure 117 EMC Unisphere - Generic iSCSI Storage
2.
Figure 118 EMC Unisphere - Generic iSCSI Storage, Create
3.
Figure 119 EMC Unisphere - Generic iSCSI Storage Wizard, Specify Name
4.
Figure 120 EMC Unisphere - Generic iSCSI Storage Wizard, Configure Storage
5.
Figure 121 EMC Unisphere - Generic iSCSI Storage Wizard, Configure Protection
6.
Figure 122 EMC Unisphere - Generic iSCSI Storage Wizard, Configure Host Access
7.
8.
Figure 123 EMC Unisphere - Generic iSCSI Storage Wizard, Results
Configuring Discover Addresses for iSCSI Adapters
We now have to setup target discovery addresses so that the iSCSI adapter can determine the shared storage resources configured and made available for access to the ESXi hosts on the VNXe3150 storage array in the above section. The ESXi system supports Dynamic and Static Discovery methods. With dynamic discovery, all targets associated with an IP address or host name and the iSCSI name are discovered. With static discovery, you must specify the IP address or host name and the iSCSI name of the target you want to access. The iSCSI HBA must be in the same VLAN as both ports of the iSCSI array.
1.
2.
3.
Figure 124 VMware vCenter - Storage Adapter Configuration
4.
5.
Figure 125 iSCSI Initiator Properties
6.
Figure 126 Add Send Target Server
7.
Figure 127 iSCSI Initiator Properties - Dynamic Discovery
8.
Figure 128 iSCSI Initiator Properties - Static Discovery
9.
10.
Figure 129 VMware vCenter - Add Storage
11.
Figure 130 Add Storage - Disk/LUN
12.
Figure 131 Add Storage - Select Disk/LUN
13.
Figure 132 Add Storage - File System Version
14.
Figure 133 Add Storage - Current Disk Layout
15.
Figure 134 Add Storage - Properties
16.
Figure 135 Add Storage - Formatting
17.
Figure 136 Add Storage - Ready to Complete
18.
Figure 137 VMware vCenter - Storage View
Configuring vSphere HA and DRS
vSphere High Availability (HA) and Distributed Resource Scheduler (DRS) provide protection for virtual machines (VMs) running on ESXi hosts, as well as optimize resources and performance and simplify VM management. The HA feature continuously monitors all ESX Server hosts in a cluster and detects failures, and will automatically restart VMs on other host servers in an ESX cluster in case of a host failure. DRS aggregates vSphere host resources into clusters and automatically distributes these resources to virtual machines by monitoring utilization and continuously optimizing virtual machine distribution across vSphere hosts. vCenter Server is required for the use of this feature.
The section explains steps to create cluster and configure HA and DRS for the VSPEX V50 solution:
1.
2.
Figure 138 VMware vCenter - New Cluster
3.
Figure 139 New Cluster Wizard - Cluster Features
4.
Figure 140 New Cluster Wizard - Ready to Complete
5.
Figure 141 VMware vCenter - Add Host to Cluster
6.
Figure 142 Add Host Wizard - Connection Settings
7.
8.
Figure 143 Add Host Wizard - Host Summary
9.
Figure 144 Add Host Wizard - Assign License
10.
Figure 145 Add Host Wizard - Lockdown Mode
11.
Figure 146 Add Host Wizard - Choose Resource Pool
12.
Figure 147 Add Host Wizard - Ready to Complete
13.
Figure 148 VMware vCenter - Cluster View
Template-Based Deployments for Rapid Provisioning
Figure 149 Rapid Provisioning Using VM Templates
In an environment with established procedures, deploying new application servers can be streamlined, but can still take many hours or days to complete. Not only must you complete an OS installation, but downloading and installing service packs and security updates can add a significant amount of time. Many applications require features that are not installed with Windows by default and must be installed prior to installing the applications. Inevitably, those features require more security updates and patches. By the time all deployment aspects are considered, more time is spent waiting for downloads and installs than is spent configuring the application.
Virtual machine templates can help speed up this process by eliminating most of these monotonous tasks. By completing the core installation requirements, typically to the point where the application is ready to be installed, you can create a golden image which can be sealed and used as a template for all of your virtual machines. Depending on how granular you want to make a specific template, the time to deployment can be as little as the time it takes to install, configure, and validate the application. You can use PowerShell tools and VMware vSphere Power CLI to bring the time and manual effort down dramatically.
Make sure to spread VMs across different VM datastores to properly load-balance the storage usage.
Jumbo MTU validation and diagnostics
To validate the jumbo MTU from end to end, SSH to the ESXi host. By default, SSH access is disabled to ESXi hosts. Enable SSH to ESXi host by editing hosts' security profile under the Configuration tab.
After connecting to the ESXi host through SSH, initiate ping to the iSCSI storage server and to vMotion VMkernel port of other ESXi hosts with large MTU size and set "Do Not Fragment" bit of IP packet to 1. Use the vmkping command as shown in Figure 150.
Figure 150 Validation of Jumbo Frames support
Ensure that the packet size is 8972 due to various L2/L3 overhead. Ping must be successful. If ping is not successful verify that 9000 MTU configured at each of the following steps:
1.
2.
3.
Validating Cisco Solution for EMC VSPEX with VMware Architectures
This section provides a list of items that should be reviewed once the solution has been configured. The goal of this section is to verify the configuration and functionality of specific aspects of the solution, and ensure that the configuration supports core availability requirements.
Post Install Checklist
The following configuration items are critical to functionality of the solution, and should be verified prior to deployment into production:
•
•
•
•
Verify the Redundancy of the Solution Components
Following redundancy checks were performed at the Cisco lab to verify solution robustness:
1.
2.
3.
4.
5.
Cisco validation test profile
"vdbench" testing tool was used with Windows Server 2012 to test scaling of the solution in Cisco labs. Figure 150 provides details on the test profile used.
Bill of Material
Table 11 gives details of the components used in the CVD for 50 virtual machines configuration.
For more information on the part numbers and options available for customization, see Cisco C220 M3 server specsheet at:
http://www.cisco.com/en/US/prod/collateral/ps10265/ps10493/C220M3_SFF_SpecSheet.pdf
Customer Configuration Data Sheet
Before you start the configuration, gather some customer-specific network and host configuration information. Table 12, Table 13, Table 14, Table 15, Table 16, Table 17 provide information on assembling the required network and host address, numbering, and naming information. This worksheet can also be used as a "leave behind" document for future reference.
Table 12 Common Server Information
Domain Controller
DNS Primary
DNS Secondary
DHCP
NTP
SMTP
SNMP
vCenter Console
SQL Server
Table 13 VMware Server Information
ESXi
Host 1
ESXi
Host 2
ESXi
Host 3
Table 14 Array Information
Admin account
Management IP
Storage pool name
Datastore name
iSCSI Server IP (SP-A and SP-B)
Table 15 Network Infrastructure Information
Cisco Nexus 3048 Switch A
Cisco Nexus 3048 Switch B
References
Cisco UCS:
http://www.cisco.com/en/US/solutions/ns340/ns517/ns224/ns944/unified_computing.html
Cisco Nexus:
http://www.cisco.com/en/US/products/ps9441/Products_Sub_Category_Home.html
EMC VNXe3xxx series resources:
http://www.emc.com/storage/vnx/vnxe-series.htm#!resource
VMware vSphere:
http://www.vmware.com/products/vsphere/overview.html
VMware vSphere 5.1 documentation:
http://pubs.vmware.com/vsphere-51/index.jsp
Microsoft SQL Server installation guide
http://msdn.microsoft.com/en-us/library/ms143219.aspx
Feedback