Microsoft Windows Server 2008 R2 SP1 Hyper-V Solution for 50 and 100 Virtual Machines
Available Languages
Table Of Contents
About Cisco Validated Design (CVD) Program
Cisco Solution for EMC VSPEX Microsoft Hyper-V Architectures
Cisco Unified Computing System
Cisco C220 M3 Rack Mount Servers
Microsoft Windows Server 2008 R2 SP1 Hyper-V
Microsoft System Center 2012 Virtual Machine Manager
EMC Storage Technologies and Benefits
Architecture for 50 Microsoft Hyper-V Virtual Machines
Architecture for 100 Microsoft Hyper-V Virtual Machines
Applying the Reference Workload
Networking Configuration Guidelines
VSPEX Configuration Guidelines
VSPEX M100 Configuration Details
Prepare and Configure the Cisco Nexus 5548UP Switch
Initial Setup of Cisco Nexus Switches
Enabling Features and Global Configuration
Configure Virtual Port Channels
Enabling Switch for Adapter-FEX
Configuring the Switch for Adapter-FEX
Prepare the Cisco UCS C220 M3 Servers
Configuring Cisco Integrated Management controller (CIMC)
Enable Virtualization Technology in BIOS
Installing the VMM Server and VMM Administrator Console
Install Microsoft Windows Server on Cisco UCS C220 M3 Servers
Installation of Microsoft Windows Server 2008 R2 SP1
NIC Teaming of Cisco 1 GigE LOM
Creating and Configuring vNICs on Cisco P81E VIC
Prepare the EMC VNXe3300 Storage
Configure Advanced Features—Link Aggregation and Jumbo Frames
Create Microsoft Hyper-V Datastores
Create Hosts and Provide Host Access
Microsoft Windows Failover Cluster Setup
Microsoft System Center-2012 VMM Configuration
Add Microsoft Hyper-V Hosts and Cluster
Create a Template for Virtual Machine Deployment
Create a Virtual Machine Template
Create a Highly Available Virtual Machine from a Template
VSPEX M50 Configuration Details
Prepare and Configure the Cisco Nexus 3048 Switch
Initial Setup of Cisco Nexus Switches
Enable Features, Jumbo Frames and Global Configuration
Add Port Channel Configurations
Configure Virtual Port Channels
Active Directory Domain Controller
Prepare the Cisco UCS C220 M3 Servers
Configure Cisco Integrated Management controller (CIMC)
Enable Virtualization Technology in BIOS
Installing Microsoft Windows Server OS on UCS C220 M3 Servers
NIC Teaming of Cisco 1 GigE LOM
NIC Teaming of Broadcom BCM5709C NetXtreme II GigE adapter
Prepare the EMC VNXe3150 Storage
Configure Advanced features—Link Aggregation and Jumbo Frames
Microsoft Windows Failover Cluster Setup
Microsoft System Center-2012 VMM Configuration
Validating Cisco Solution for EMC VSPEX Microsoft Hyper-V Architectures
Verify the Redundancy of the Solution Components
Customer Configuration Data Sheet
Cisco Solution for EMC VSPEX Microsoft Hyper-V
ArchitecturesDesign for 50 and 100 Virtual MachinesLast Updated: October 3, 2013
Building Architectures to Solve Business Problems
About the Authors
Sanjeev Naldurgkar, Technical Marketing Engineer, Server Access Virtualization Business Unit, Cisco SystemsSanjeev has over 12 years of experience in information technology, his focus areas include UCS, Microsoft product technologies, server virtualization, and storage technologies. Prior to joining Cisco, Sanjeev was Support Engineer at Microsoft Global Technical Support Center. Sanjeev holds a Bachelor's Degree in Electronics and Communication Engineering and Industry certifications from Microsoft, and VMware.
Tim Cerling, Technical Marketing Engineer, Datacenter Group, Cisco SystemsTim's focus is on delivering customer-driven solutions on Microsoft Hyper-V and System Center products. He has been in the IT business since 1979. He started working with Windows NT 3.5 on the DEC Alpha product line during his 19 year tenure with DEC, and he has continued working with Windows Server technologies since then with Compaq, Microsoft, and now Cisco. During his twelve years as a Windows Server specialist at Microsoft, he co-authored a book on Microsoft virtualization technologies - Mastering Microsoft Virtualization. Tim holds a BA in Computer Science from the University of Iowa.
Acknowledgements
For their support and contribution to the design, validation, and creation of the Cisco Validated Design, we would like to thank:
•
Vadiraja Bhatt-Cisco
•
•
•
•
•
•
•
•
•
About Cisco Validated Design (CVD) Program
The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information visit www.cisco.com/go/designzone.
ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.
CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, the Cisco logo, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, IronPort, the IronPort logo, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0809R)
© 2012 Cisco Systems, Inc. All rights reserved
Cisco Solution for EMC VSPEX Microsoft Hyper-V Architectures
Executive Summary
Cisco solution for the EMC VSPEX is a pre-validated and modular architecture built with proven best of-breed technologies to create and complete an end-to-end virtualization solution. The end-to-end solutions enable you to make an informed decision while choosing the hypervisor, compute, storage and networking layers. VSPEX eliminates the server virtualization planning and configuration burdens. The VSPEX infrastructures accelerate your IT Transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk. This Cisco Validated Design document focuses on the Microsoft Hyper-V architecture for 50 and 100 virtual machines with Cisco solution for the EMC VSPEX.
Introduction
As part of an effort to improve and enhance the performance and capabilities of its product line, Cisco and EMC from time to time release revisions of its hardware and software. Therefore, some functions described in this guide may not be supported by all revisions of the software or hardware currently in use. For the most up-to-date information on product features, refer to your product release notes.
Target Audience
The reader of this document is expected to have the necessary training and background to install and configure Microsoft Hyper-V, EMC VNXe series storage, Cisco Nexus 5548UP and 3048switches, and Cisco Unified Computing System (UCS) C220 M3 rack servers. External references are provided wherever applicable and it is recommended that the reader be familiar with these documents.
Readers are also expected to be familiar with the infrastructure and database security policies of the customer installation.
Purpose
This document describes the steps required to deploy and configure a Cisco solution for the EMC VSPEX for Microsoft Hyper-V architectures. The document covers two types of Microsoft Hyper-V architectures:
•
•
The readers of this document are expected to have sufficient knowledge to install and configure the products used, configuration details that are important to the deployment models mentioned above.
Business Needs
The VSPEX solutions are built with proven best-of-breed technologies to create complete virtualization solutions that enable you to make an informed decision in the hypervisor, server, and networking layers. The VSPEX infrastructures accelerate your IT transformation by enabling faster deployments, greater flexibility of choice, efficiency, and lower risk.
Business applications are moving into the consolidated compute, network, and storage environment. The Cisco solution for the EMC VSPEX using Microsoft Hyper-V helps to reduce every component of a traditional deployment. The complexity of integration management is reduced while maintaining the application design and implementation options. Administration is unified, while process separation can be adequately controlled and monitored. The following are the business needs for the Cisco solution for EMC VSPEX Microsoft Hyper-V architectures:
•
•
•
Solution Overview
The Cisco solution for EMC VSPEX using Microsoft Hyper-V provides an end-to-end architecture with Cisco, EMC, and Microsoft technologies that demonstrate support for up to 50 and 100 generic virtual machines and provides high availability and server redundancy.
The following are the components used for the design and deployment:
•
•
•
•
•
•
•
The solution is designed to host scalable, mixed application workloads. The scope of this CVD is limited to the Cisco solution for EMC VSPEX Microsoft Hyper-V solutions for 50 and 100 virtual machines only.
Technology Overview
This section describes the various technologies used in this solution and their benefits.
Cisco Unified Computing System
The Cisco Unified Computing System is a next-generation data center platform that unites computing, network, storage access, and virtualization into a single cohesive system. The platform, optimized for virtual environments, is designed using open industry-standard technologies and aims to reduce total cost of ownership (TCO) and increase business agility. The system integrates a low-latency; lossless 10 Gigabit Ethernet unified network fabric with enterprise-class, x86-architecture servers. It is an integrated, scalable, multi chassis platform in which all resources participate in a unified management domain.
The main components of the Cisco UCS are:
•
•
•
•
The Cisco Unified Computing System is designed to deliver:
•
•
•
•
Cisco C220 M3 Rack Mount Servers
Building on the success of the Cisco UCS C220 M3 Rack Servers, the enterprise-class Cisco UCS C220 M3 server further extends the capabilities of the Cisco Unified Computing System portfolio in a 1-rack-unit (1RU) form factor. And with the addition of the Intel® Xeon® processor E5-2600 product family, it delivers significant performance and efficiency gains. Figure 1 shows the Cisco UCS C220 M3 rack server.
Figure 1 Cisco UCS C220 M3 Rack Mount Server
The Cisco UCS C220 M3 also offers up to 256 GB of RAM, eight drives or SSDs, and two 1GE LAN interfaces built into the motherboard, delivering outstanding levels of density and performance in a compact package.
I/O Adapters
The Cisco UCS rack mount server has various Converged Network Adapters (CNA) options. The UCS P81E Virtual Interface Card (VIC) option is used in this Cisco Validated Design.
This Cisco UCS P81E VIC is unique to the Cisco UCS rack mount server system. This mezzanine card adapter is designed around a custom ASIC that is specifically intended for virtualized systems. As is the case with the other Cisco CNAs, the Cisco UCS P81E VIC encapsulates fibre channel traffic within the 10-GE packets for delivery to the Ethernet network.
UCS P81E VIC provides the capability to create multiple VNICs (up to 128) on the CNA. This allows complete I/O configurations to be provisioned in virtualized or non-virtualized environments using just-in-time provisioning, providing tremendous system flexibility and allowing consolidation of multiple physical adapters.
System security and manageability is improved by providing visibility and portability of network policies and security all the way to the virtual machines. Additional P81E features like VN-Link technology and pass-through switching, minimize implementation overhead and complexity. Figure 2 shows the Cisco UCS P81E VIC.
Figure 2 Cisco UCS P81e VIC
Cisco Adapter Fabric Extender
Cisco Adapter FEX extends the Cisco FEX technology into traditional rack servers. The Cisco Adapter FEX technology enables the server adapter to be logically partitioned into multiple virtual network interface cards (vNICs). Each vNIC behaves like a physical NIC port and meets the network connectivity needs for each application, so that security and quality of service (QoS) policies can be applied for each vNIC and each application.
Figure 3 Network Adapter vNICs as Physical Ports on the Cisco Nexus 5500 Series
Figure 3 shows that a server with the virtualized adapters (called vNICs) can offer an operating system a number of virtual adapters, and with the A-FEX technology, vNICs are presented as directly connected interfaces to the Cisco Nexus 5500 series switches. All the switching between vNICs occurs on the upstream Cisco Nexus 5500 series switches, as though they were interfaces of a remote linecard or fabric extenders. In addition to this, all the features right from access control lists (ACLs) to private VLANs, quality of service (QoS), and so on, are available on the remote interfaces.
The redundancy or teaming configuration is not required on the operating system anymore since it is implemented in hardware and controlled by the Cisco Nexus 5500 series switches.
The provisioning model allows the network administrator to define profiles with specific network definitions (mode access or trunk, VLAN, and so on). The server administrator has the choice of defining the number of vNICs and the profile to map them with.
Cisco Nexus 5548UP Switch
The Cisco Nexus 5548UP is a 1RU 1 Gigabit and 10 Gigabit Ethernet switch offering up to 960 gigabits per second throughput and scaling up to 48 ports. It offers 32 1/10 Gigabit Ethernet fixed enhanced Small Form-Factor Pluggable (SFP+) Ethernet/FCoE or 1/2/4/8-Gbps native FC unified ports and three expansion slots. These slots have a combination of Ethernet/FCoE and native FC ports. The Cisco Nexus 5548UP switch is shown in Figure 4.
Figure 4 Cisco Nexus 5548UP Switch
Cisco Nexus 3048 Switch
The Cisco Nexus® 3048 Switch is a line-rate Gigabit Ethernet top-of-rack (ToR) switch and is part of the Cisco Nexus 3000 Series Switches portfolio. The Cisco Nexus 3048, with its compact one-rack-unit (1RU) form factor and integrated Layer 2 and 3 switching, complements the existing Cisco Nexus family of switches. This switch runs the industry-leading Cisco® NX-OS Software operating system, providing customers with robust features and functions that are deployed in thousands of data centers worldwide. The Cisco Nexus 3048 switch is shown in Figure 5.
Figure 5 Cisco Nexus 3048 Switch
Microsoft Windows Server 2008 R2 SP1 Hyper-V
Microsoft Hyper-V is an integral part of Windows Server and provides a foundational virtualization platform that enables you to transition to the cloud. With Windows Server 2008 R2 you get a compelling solution for core virtualization scenarios - production server consolidation, dynamic datacenter, business continuity, VDI, and test & development.
Microsoft Hyper-V provides you better flexibility with features like live migration and cluster shared volumes for storage flexibility.
Microsoft Hyper-V also delivers greater scalability with support for up to 64 logical processors, 2 TB of RAM, NUMA awareness, and improved performance with support for dynamic memory and enhanced networking support.
Microsoft System Center 2012 Virtual Machine Manager
Microsoft System Center 2012 Virtual Machine Manager (VMM) is a management solution for the virtualized datacenter. This solution enables you to configure and manage your virtualization host, networking, and storage resources in order to create, deploy, and manage virtual machines and services to private clouds that you have created.
EMC Storage Technologies and Benefits
The EMC VNX™ family is optimized for virtual applications delivering industry-leading innovation and enterprise capabilities for file, block, and object storage in a scalable, easy-to-use solution. This next-generation storage platform combines powerful and flexible hardware with advanced efficiency, management, and protection software to meet the demanding needs of today's enterprises.
The EMC VNXe™ series is powered by Intel Xeon processor, for intelligent storage that automatically and efficiently scales in performance, while ensuring data integrity and security.
The EMC VNXe series is purpose-built for the IT manager in smaller environments and the VNX series is designed to meet the high-performance, high-scalability requirements of midsize and large enterprises. The EMC VNXe and VNX storage arrays are multi-protocol platform that can support the iSCSI, NFS, and CIFS protocols depending on the customer's specific needs. The solution was validated using NFS for data storage.
The EMC VNXe series storage arrays have the following customer benefits:
•
•
•
•
•
Software Suites
The following are the available EMC software suites:
•
•
•
Software Packs
Total Value Pack—Includes all protection software suites, and the Security and Compliance Suite.
This is the available EMC protection software pack.
EMC Avamar
EMC's Avamar® data deduplication technology seamlessly integrates into virtual environments, providing rapid backup and restoration capabilities. Avamar's deduplication results in vastly less data traversing the network, and greatly reduces the amount of data being backed up and stored; resulting in storage, bandwidth and operational savings.
The following are the two most common recovery requests used in backup and recovery:
•
•
The Avamar System State protection functionality adds backup and recovery capabilities in both of these scenarios.
Architectural Overview
This Cisco Validated Design discusses the deployment model for the following two Microsoft Hyper-V server virtualization solutions:
•
•
Table 1 lists the mix of hardware components, their quantities and software components used for different solutions:
Table 2 lists the various hardware and software components which occupies different tiers of the Cisco solution for EMC VSPEX using Microsoft Hyper-V architectures under test.
Table 3 outlines the C220 M3 server configuration details (per server basis) across all the Microsoft Hyper-V architectures.
Storage Guidelines
The architecture diagrams in this section show the physical disk layout. Disk provisioning on the EMC VNXe series is simplified through the use of wizards, so that administrators do not choose which disks belong to a given storage pool. The wizard may choose any available disk of the proper type, regardless of where the disk physically resides in the array
The reference architecture uses the following configuration:
•
–
–
•
The EMC VNX/VNXe family is designed for five 9s availability by using redundant components throughout the array. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures, and the available hot spare drives can be dynamically allocated to replace a failing disk.
Figure 6 Storage Architecture for 50 VMs on EMC VNXe3150
Figure 7 Storage Architecture for 100 VMs on EMC VNXe3300
Table 4 provides the datastores size details for the two types of architectures laid out in Figure 6 and Figure 7.
Both reference architectures assume that there is an existing infrastructure / management network available where a virtual machine or physical machine hosting SCVMM server, Database server, and Microsoft Windows Active Directory / DNS server are present. Figure 8 and Figure 9 show high level solution architecture for up to 50 and up to 100 virtual machines, respectively.
Figure 8 Reference Architecture for 50 Virtual Machines
Figure 9 Reference Architecture for 100 Virtual Machines
As it is evident in the above diagrams, following are the high level design points of Microsoft Hyper-V architectures:
•
•
•
This design does not dictate or require any specific layout of infrastructure network which hosts the SCVMM, Database, and Active Directory servers. However, design does require accessibility of certain VLANs from the infrastructure network to reach the servers.
Microsoft Windows Server 2008 R2 SP1 Hyper-V is used as hypervisor operating system on each server and is installed on local hard drives. Typical load is 25 virtual machines per server.
Architecture for 50 Microsoft Hyper-V Virtual Machines
Figure 10 shows the logical layout of 50 Microsoft Hyper-V virtual machines. Following are the key aspects of this solution:
•
•
•
•
•
Figure 10 Cisco Solution for 50 Virtual Machines Using Microsoft Hyper-V
Architecture for 100 Microsoft Hyper-V Virtual Machines
Figure 11 shows the logical layout of 100 Microsoft Hyper-V virtual machines. Following are the key aspects of this solution:
•
•
•
•
•
Figure 11 Cisco Solution for 100 Virtual Machines Using Microsoft Hyper-V
Sizing Guidelines
It is important to define a reference workload in virtual infrastructures. Not all servers perform the same tasks, and it is impractical to build a reference that takes into account every possible combination of workload characteristics.
Defining a Reference Workload
To simplify the discussion, we have defined a representative reference workload. By comparing your actual usage to this reference workload, you can extrapolate which reference architecture to choose.
For the VSPEX solutions, the reference workload was defined as a single virtual machine. Table 5 provides the characteristics of the virtual machine:
This specification for a virtual machine is not intended to represent any specific application. Rather, it represents a single common point of reference to measure other virtual machines.
Applying the Reference Workload
When considering an existing server that will move into a virtual infrastructure, you have the opportunity to gain efficiency by correctly sizing the virtual hardware resources assigned to that system.
The reference architectures create a pool of resources sufficient to host a target number of reference virtual machines. It is entirely possible that your virtual machines may not exactly match the specifications above. In that case, you can say that a single specific virtual machine is the equivalent of some number of reference virtual machines, and assume that the number of virtual machines have been used in the pool. You can continue to provision virtual machines from the pool of resources until it is exhausted. Consider these examples:
Example 1 Custom Build Application
A small custom-built application server needs to move into this virtual infrastructure. The physical hardware supporting the application is not being fully utilized at present. A careful analysis of the existing application reveals that the application can use one processor, and needs 3 GB of memory to run normally. The IO workload ranges between 4 IOPS at idle time to 15 IOPS when busy. The entire application is only using about 30 GB on local hard drive storage.Based on these numbers, following resources are needed from the resource pool:- CPU resources for 1 VM- Memory resources for 2 VMs- Storage capacity for 1 VM- IOPS for 1 VMIn this example, a single virtual machine uses the resources of two of the reference VMs. If the original pool had the capability to provide 100 VMs worth of resources, the new capability is 98 VMs.Example 2 Point of Sale System
The database server for a customer's point-of-sale system needs to move into this virtual infrastructure. It is currently running on a physical system with four CPUs and 16 GB of memory. It uses 200 GB storage and generates 200 IOPS during an average busy cycle.The following are the resources needed from the resource pool to virtualize this application:- CPUs of 4 reference VMs- Memory of 8 reference VMs- Storage of 2 reference VMs- IOPS of 8 reference VMsIn this case the one virtual machine uses the resources of eight reference virtual machines. If this was implemented on a resource pool for 50 virtual machines, there are 42 virtual machines of capability remaining in the pool.Example 3 Web Server
The customer's web server needs to move into this virtual infrastructure. It is currently running on a physical system with two CPUs and 8GB of memory. It uses 25 GB of storage and generates 50 IOPS during an average busy cycle.The following are the requirements to virtualize this application:- CPUs of 2 reference VMs- Memory of 4 reference VMs- Storage of 1 reference VMs- IOPS of 2 reference VMsIn this case the virtual machine would use the resources of four reference virtual machines. If this was implemented on a resource pool for 100 virtual machines, there are 96 virtual machines of capability remaining in the pool.Summary of Examples
The three examples presented show the flexibility of the resource pool model. In all the three cases the workloads simply reduce the number of available resources in the pool. If all the three examples were implemented on the same virtual infrastructure, with an initial capacity of 100 virtual machines they can all be implemented, leaving the capacity of eighty six reference virtual machines in the resource pool.
In more advanced cases, there may be trade-offs between memory and I/O or other relationships where in increasing the amount of one resource decreases the need for another. In these cases, the interactions between resource allocations become highly complex, which is out of the scope of this document. However, once the change in the resource balance has been examined, and the new level of requirements is known; these virtual machines can be added to the infrastructure using the method described in the examples. You can also use the Microsoft Assessment and Planning (MAP) toolkit to assist in the analysis of the current workload. You can download the toolkit from the following Microsoft link:
Networking Configuration Guidelines
This document provides details for setting up a redundant, highly-available configuration. As such, references are made as to which component is being configured with each step whether that be A or B. For example, SP A and SP B, are used to identify the two EMC VNXe storage controllers that are provisioned with this document while the Nexus A and Nexus B identify the pair of Cisco Nexus switches that are configured. Additionally, this document details steps for provisioning multiple UCS hosts and these are identified sequentially, M100N1 and M100N2, and so on. Finally, when indicating that the reader should include information pertinent to their environment in a given step, this is indicated with the inclusion of <italicized text> as part of the command structure. See the following example for the VLAN create command on the Cisco Nexus Switch:
switchA(config)# vlan {vlan-id | vlan-range}switchA(config)# vlan <storage VLAN ID>This document is intended to allow the reader to fully configure the customer environment. In order to do so, there are various steps which will require you to insert your own naming conventions, IP addresses, and VLAN schemes, as well as record appropriate iSCSI IQN or MAC addresses. Table 8 details the list of VLANs necessary for deployment as outlined in this guide.
VSPEX Configuration Guidelines
To configure the Cisco solution for EMC VSPEX Microsoft Hyper-V architectures, follow these steps:
3.
5.
6.
7.
8.
9.
The above steps are described in the following sections.
Pre-Deployment Tasks
Pre-deployment tasks include procedures that do not directly relate to environment installation and configuration, but whose results will be needed at the time of installation. Examples of pre-deployment tasks are collection of hostnames, IP addresses, VLAN IDs, license keys, installation media, and so on. These tasks should be performed before the customer visit to decrease the time required onsite.
•
•
•
Customer Configuration Data
To reduce the onsite time, information such as IP addresses and hostnames should be assembled as part of the planning process.
The Customer Configuration Data section provides a table to maintain a record of relevant information. This form can be expanded or contracted as required, and information may be added, modified, and recorded as deployment progresses.
Additionally, complete the EMC VNXe Series Configuration Worksheet, available on the EMC online support website, to provide the most comprehensive array-specific information.
VSPEX M100 Configuration Details
Cabling Information
The following information is provided as a reference for cabling the physical equipment in a VSPEX M100 environment. The tables in this section include both local and remote device and port locations in order to simplify cabling requirements.
This document assumes that out-of-band management ports are plugged into an existing management infrastructure at the deployment site.
Follow the cabling directions in this section. Failure to do so will result in necessary changes to the deployment procedures that follow because specific port locations are mentioned.
Before starting, ensure that the configurations match the cabling details provided in the tables and figures in this section.
Figure 12 shows the VSPEX M100 cabling diagram. The alphabets labeled indicate connections to the end points rather than port numbers on the physical device. For example, connection A is a 10 Gb target port connected from EMC VNXe3300 SP B to Cisco Nexus 5548 A and connection R is a 10 Gb target port connected from Cisco VIC P81E uplink port 1 on Server 2 to Cisco Nexus 5548 B. Connections U and V are 10 Gb vPC peer-links connected from Cisco Nexus 5548 A to Cisco Nexus 5548 B.
Figure 12 Cabling Details for VSPEX Microsoft Hyper-V 100 Virtual Machines
Table 6 and Table 7 show that there are five major cabling in these architectures:
•
•
•
•
•
Table 6 provides the Cisco Nexus 5548 A Ethernet Cabling Information Local Device Local Port Connection Remote.
Prepare and Configure the Cisco Nexus 5548UP Switch
The following section provides a detailed procedure for configuring the Cisco Nexus 5548 switches for use in EMC VSPEX Microsoft Hyper-V 100 VMs.
Figure 13 shows two switches configured for vPC. In vPC, a pair of switches acting as vPC peer endpoints looks like a single entity to port-channel-attached devices, although the two devices that act as logical port-channel endpoint are still two separate devices. This provides hardware redundancy with port-channel benefits. Both switches form a vPC Domain, in which one vPC switch is Primary while the other is secondary.
Note
Figure 13 Networking Configuration for Microsoft Hyper-V 100 Virtual Machines
Initial Setup of Cisco Nexus Switches
These steps provide details for the initial setup on both Cisco Nexus 5548 switches.
For Cisco Nexus A and Cisco Nexus B
After booting and connecting to the serial or console port of the switch, the NX-OS setup should automatically start.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
Enabling Features and Global Configuration
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
Set Global Configurations
These steps provide details for setting global configurations.
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
Configure VLANs
The steps in this section provide details for creating the VLANs as per the below given reference Table 8.
Note
For Nexus A and Nexus B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Configure Port-Channels
Create Port-Channels
For Cisco Nexus 5548 A and Cisco 5548 B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
Add Port Channel Configurations
These steps provide details for adding Port Channel configurations.
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
Configure Virtual Port Channels
These steps provide details for configuring virtual Port Channels (vPCs).
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
At this point of time, all ports and port-channels are configured with necessary VLANs, switchport mode and vPC configuration. Validate this configuration using the "show port-channel summary" and "show vpc" commands as shown in Figure 14.
Figure 14 Show VLAN Brief Output
Ensure that on both switches, all required VLANs are in "active" status and right set of ports and port-channels are part of the necessary VLANs.
Port-channel configuration can be verified using "show port-channel summary" command. Figure 15 shows the expected output of this command.
Figure 15 Show Port Channel Summary Output
In this example, port-channel 7 is the vPC peer-link port-channel, port-channels 2, 3, 4 and 5 are connected to the Cisco 1GigE I350 LOM on the host, port-channels 15 and 17 are connected to the infrastructure network, and port-channels 21 and 22 are connected to the storage array. Make sure that state of the member ports of each port-channel is "P" (Up in port-channel). Note that port may not come up if the peer ports are not properly configured. Common reasons for port-channel port being down are:
•
•
vPC status can be verified using "show vpc" command. Example output is shown in Figure 16.
Figure 16 Show vPC Brief Output
Ensure that the vPC peer status is "peer adjacency formed ok" and all the port-channels, including the peer-link port-channel, have their status as "up".
Configure Adapter FEX
The Cisco NX-OS Adapter-FEX feature combines the advantages of the FEX link architecture with server I/O virtualization to create multiple virtual interfaces over a single Ethernet interface. This allows you to deploy a dual port NIC on the server and to configure more than two virtual interfaces that the server sees as a regular Ethernet interface. The advantage of this approach is that it allows you to reduce power and cooling needs and to reduce the number of network ports.
Adapter-FEX can be thought of as a way to divide a single physical link into multiple virtual links or channels. Each channel is identified by a unique channel number and its scope is limited to the physical link.
The physical link connects a port on a server network adapter with an Ethernet port on the switch. This allows the channel to connect a vNIC on the server with a vEthernet interface on the switch.
Packets on each channel are tagged with a VNTag that has a specific source virtual interface identifier (VIF). The VIF allows the receiver to identify the channel that the source used to transmit the packet.
For more information on Adapter-FEX, check the below URLs:
Cisco Nexus 5000 Series NX-OS Adapter-FEX Software Configuration Guide, Release 5.1(3)N1(1)
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/adapter-fex/513_n1_1/b_Configuring_Cisco_Nexus_5000_Series_Adapter-FEX_rel_5_1_3_N1.pdf
Cisco Adapter Fabric Extender
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/data_sheet_c78-657397.html
This section provides information about how to enable and configure the Cisco Nexus 5500 series for Adapter-FEX. After completing the following steps in this section you will carve out vNICs on Cisco VIC P81E adapter (explained in the later section "Creating and configuring vNICs on Cisco P81E VIC") using the below created port-profiles.
Enabling Switch for Adapter-FEX
Following steps show enabling the virtualization feature on both the Cisco Nexus switches.
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
Configuring the Switch for Adapter-FEX
Following steps show creation of port-profiles on both the switches and put the FEX interfaces into vntag mode for NIC.
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Figure 17 Port-Profile Brief Output
Enable Jumbo Frames
Cisco solution for EMC VSPEX Microsoft Hyper-V architectures require MTU set at 9000 (jumbo frames) for efficient storage and live migration traffic. MTU configuration on Cisco Nexus 5000 series switches fall under global QoS configuration. You may need to configure additional QoS parameters as needed by the applications.
The following commands enable jumbo frames on the Cisco Nexus switches.
For Cisco Nexus A and Cisco Nexus B
switch(config)#policy-map type network-qos jumboswitch(config-pmap-nq)#class type network-qos class-defaultswitch(config-pmap-c-nq)#mtu 9216switch(config-pmap-c-nq)#exitswitch(config-pmap-nq)#exitswitch(config)#system qosswitch(config-sys-qos)#service-policy type network-qos jumboFigure 18 Validate Jumbo Frames Support in the Storage Processors
Infrastructure Servers
Most environments will already have an Active Directory in their infrastructure either running on a virtual machine or on a physical server. This section will not cover the installation of an Active Directory Domain Controller, however it will cover the brief installation of standalone SQL Server 2008 R2 SP1 and System Center VMM 2012 on Microsoft Windows Server 2008 R2 SP1. The following infrastructure servers were used to validate the VSPEX Microsoft Hyper-V architectures.
Note
Prepare the Cisco UCS C220 M3 Servers
Preparing the Cisco C220 M3 servers is a common step for all the Hyper-V architectures. Firstly, you need to install the C220 M3 server in a rack. For more information on mounting the Cisco C220 servers, see the installation guide on details about how to physically mount the server: http://www.cisco.com/en/US/docs/unified_computing/ucs/c/hw/C220/install/install.html
To prepare the servers, follow these steps:
1.
2.
These steps are discussed in detail in the following sections.
Configuring Cisco Integrated Management controller (CIMC)
This section describes procedures to prepare the Cisco UCS C220 M3 servers.
Connecting and Powering on the Server (Standalone Mode)
For connecting and powering on the server (Standalone Mode), follow these steps:
1.
2.
3.
4.
5.
6.
7.
8.
Figure 19 CIMC Configuration Utility
Once the CIMC IP is configured, the server can be managed using the https based Web GUI or CLI.
Note
Enable Virtualization Technology in BIOS
Microsoft Hyper-V requires an x64-based processor, hardware-assisted virtualization (Intel VT enabled), and hardware data execution protection (Execute Disable enabled).
To enable Intel ® VT and Execute Disable in BIOS, follow these steps:
1.
2.
3.
Figure 20 Cisco UCS C220 M2 KVM Console
4.
Configuring RAID
The RAID controller type is Cisco UCSC RAID SAS 2008 and supports 0, 1, 5 RAID levels. We need to configure RAID level 1 for this setup and set the virtual drive as boot drive.
To configure RAID controller, follow these steps:
1.
2.
Figure 21 Cisco UCS C220 M2 CIMC GUI
3.
Figure 22 KVM Console Showing Cisco UCS C220 M2 Server Booting
4.
Figure 23 Adapter Selection for RAID Configuration
5.
Figure 24 MegaRAID BIOS Config Utility Configuration
6.
Figure 25 Confirmation Window for Clearing the Configuration
7.
Figure 26 Selecting the Configuration Method
8.
Figure 27 Configuration Preview
9.
10.
Figure 28 Setting the Boot Option for Virtual Drives
11.
Figure 29 Virtual Drive Configuration in WebBIOS
SQL Server Installation
This document will not go into details and best practices for SQL server installation and configuration. Consult your DBA team to ensure your SQL deployment is configured for best practices according to your corporate standards. For more information on the system requirements for the VMM database, see the Microsoft TechNet link:
http://technet.microsoft.com/en-us/library/gg610574
1.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
http://technet.microsoft.com/en-us/library/cc764295.aspx
12.
http://support.microsoft.com/kb/909801
Microsoft System Center VMM
System Center 2012 - Virtual Machine Manager (VMM) is a management solution for the virtualized datacenter. It enables you to configure and manage your virtualization host, networking, and storage resources in order to create and deploy virtual machines and services to private clouds that you have created. For an overview of System Center 2012 - VMM, see the Microsoft TechNet link:
http://technet.microsoft.com/en-us/library/gg671827
This section deals with the installation of the System Center Virtual Machine Manager 2012 on a virtual machine running Microsoft Windows Server 2008 R2 SP1 OS. However, this section does not cover how to create and build a Microsoft Windows Server 2008 R2 with SP1 in a virtual environment.
Before installing a VMM management server, ensure that the computer meets the minimum hardware requirements and that all the prerequisite software is installed. For information about hardware and software requirements for VMM, see the link:
http://technet.microsoft.com/en-us/library/gg610592
Installing the VMM Server and VMM Administrator Console
To install the VMM Server and VMM administrator console, follow these steps:
1.
2.
3.
4.
Figure 30 Selecting Feature in Microsoft SCVMM 2012 Setup Wizard
5.
6.
Figure 31 Database Configuration in Microsoft SCVMM 2012 Setup Wizard
7.
Figure 32 Configuring Service Account in Microsoft SCVMM 2012 Setup Wizard
For more information about which type of account to use, under "Specifying a Service Account for VMM", see the link:
http://technet.microsoft.com/library/gg697600.aspx
8.
9.
10.
11.
For more information on installing the System Center 2012 - VMM, see the TechNet article:
http://technet.microsoft.com/en-us/library/gg610617.aspx
Install Microsoft Windows Server on Cisco UCS C220 M3 Servers
This section describes installation of Microsoft Windows Server 2008 R2 SP1 along with driver installation.
Installation of Microsoft Windows Server 2008 R2 SP1
To install Microsoft Windows Server 2008 R2 Sp1 on all the Cisco UCS C220 M3 bare metal server using the virtual media, follow these steps:
1.
2.
Figure 33 CIMC Manager Login Page
3.
a.
b.
c.
Figure 34 Enabling Virtual Media in CIMC
4.
Figure 35 Setting Virtual KVM Properties in CIMC
5.
6.
The ISO image is displayed in the "Client View" pane.
Figure 36 Adding an ISO Image in CIMC
7.
8.
9.
10.
Figure 37 Cisco UCS C220 M3 BIOS Setup Utility
11.
The Microsoft Windows installation begins when the image is booted.
12.
Figure 38 prompt to Boot from CD
13.
14.
To install the drivers, follow these steps:
Figure 39 Load Drivers for Installing Microsoft Windows
a.
You are prompted by a "Load Driver" dialog to choose the driver to be installed. In the next steps, you first define a virtual device with your driver ISO image.
Figure 40 Selecting Drivers for Installing Microsoft Windows
b.
c.
d.
e.
The ISO appears in the "Client View" pane.
f.
Figure 41 Adding an ISO Image in CIMC
g.
h.
i.
Windows loads the drivers and when finished, the driver is listed under the prompt "Select the driver to be installed".
Driver Path - CDROM Drive:\ \Windows\Storage\LSI\2008M\W2K8R2\x64
j.
Figure 42 Selecting Drivers for Microsoft Windows Installation
15.
To unmap the driver ISO and map the Windows installation image, follow these steps:
a.
b.
Wait for the mapping to complete. Observe the progress in the "Details" pane.
c.
Figure 43 Allocating Disk Space for installing Microsoft Windows
16.
17.
Device Driver Installation
The Cisco UCS C220 M3 in VSPEX M100 solution contains two Cisco 1GigE I350 LOM (LAN-on-motherboard) and a Cisco VIC P81E adapter for which you need to install the drivers. This section explains how to locate and install the chipset and adapter drivers for Microsoft Windows Server 2008 R2 SP1.
Figure 44 Device Manager
To locate and install the chipset and adapter drivers for Microsoft Windows Server 2008 R2 SP1, follow these steps:
1.
2.
3.
Figure 45 LOM Drive Installation
4.
5.
Network Configuration
This section provides steps to configure the NICs and assign IP addresses on all the Windows host servers.
As shown in Figure 46 (for complete diagram, see Figure 13, the two Cisco 1GigE I350 LOM are connected to different switches and teamed at the host side for redundancy and load balancing.
At the Cisco VIC P81E adapter with two 10 GigE uplink ports, four vNICs are created. Cisco Nexus 5500 series switches with Adapter-FEX feature and Cisco VIC P81E in NIV mode provides redundancy at the adapter level. In Figure 46, the vNIC in purple and the vNIC in green shown as solid lines connecting to uplink ports 0 and 1, respectively, indicate active links. The dotted line connection to uplink ports 2 and 1 for these vNICs indicate standby links. It failovers to standby link in the event of active link failing, thus providing redundancy at the adapter level.
Figure 46 Cisco UCS C220 M3 Network Adapters Configuration
NIC Teaming of Cisco 1 GigE LOM
Teaming of the Cisco 1 GigE LOMs provides redundancy and doubles the available bandwidth. This teamed adapter is used for Microsoft Hyper-V host management.
For NIC teaming of Cisco 1 GigE LOMs, follow these steps:
1.
2.
3.
Figure 47 Cisco 1GigE I350 LOM Properties
4.
Figure 48 Selecting Adapters for Teaming
5.
Figure 49 Selecting the Type of Adapter Teaming
A new team will be created as shown in Figure 50.
Figure 50 Cisco 1GigE I350 LOM Team Properties
6.
Figure 51 Network Connections page after NIC teaming
7.
Figure 52 Assigning IP Address to the Teamed Adapter
8.
After completion of the above steps successfully, the status of the port-channel summary output on both Cisco Nexus A and B looks like the one shown in Figure 53.
Figure 53 Show Port Channel Summary Output
This Cisco LOM teamed adapter will be used for the host management.
Note
Creating and Configuring vNICs on Cisco P81E VIC
Cisco P81E VIC supports the Adapter FEX feature and functionality that is enabled on both the Cisco Nexus 5500 series switches. Adapter-FEX can be thought of as a way to divide a single physical link into multiple virtual links or channels. Each channel is identified by a unique channel number and its scope is limited to the physical link. The physical link connects a port on a server network adapter with an Ethernet port on the switch. This allows the channel to connect a vNIC on the server with a Ethernet interface on the switch. Packets on each channel are tagged with a VNTag that has a specific source virtual interface identifier (VIF). The VIF allows the receiver to identify the channel that the source used to transmit the packet.
Referring to Table 10, the following steps explain how to modify the existing vNICs properties and create two additional vNICs on Cisco P81E VIC for the VSPEX M100 configuration.
To modify the existing vNICs properties and create the additional vNICs, follow these steps:
1.
2.
3.
4.
Figure 54 Enabling NIV Mode for Adapter Cards
5.
6.
7.
Figure 55 vNIC Properties
8.
9.
Figure 56 CIMC- vNIC Properties 1
10.
Figure 57 Adding vNIC
11.
Figure 58 Adding vNIC 1
12.
13.
The switches respond by creating a VevEthernet interface for each vNIC on the server network adapter and associate the port-profile and channel number to the VevEthernet interface. Figure 59 shows the Cisco Nexus switch's partial running-config output showing the veEthernet interfaces created for each vNICs on the four Cisco UCS C220 M3 servers.
Figure 59 Running-config Showing veEthernet Interfaces on Cisco Nexus Switch
14.
Figure 60 Renaming the NICs for Easy Identification
15.
Figure 61 Jumbo Frame Validation from Source Switch to Destination Storage
Host Rename and Domain Join
This section covers step-by-step instructions to rename and join the hosts to a domain.
To rename and join the hosts to a domain, follow these steps:
1.
2.
Figure 62 Changing System Properties in Server Manager Window
3.
Figure 63 System Properties Window
4.
5.
Figure 64 Specifying the Domain
6.
7.
8.
Install Roles and Features
To install roles and features, follow these steps:
1.
2.
3.
Figure 65 Adding Features to the Server
4.
5.
Figure 66 Enabling iSCSI Support in MPIO Properties Window
6.
Figure 67 MPIO Properties Window
7.
8.
Figure 68 Adding Roles in the Server Manager Window
9.
Figure 69 Selecting Roles for the Server
10.
11.
12.
13.
14.
Figure 70 Adding New Virtual Networks
15.
Figure 71 Add Virtual Networks in Virtual Network Manager 1
16.
Enable iSCSI Initiator
To enable iSCSI initiator, follow these steps:
1.
2.
Figure 72 Configuring iSCSI Initiator
Prepare the EMC VNXe3300 Storage
This section explains the following topics:
•
•
•
•
Initial Setup of EMC VNXe
To configure the initial setup of the EMC VNXe, follow these steps:
1.
2.
http://www.emc.com/support-training/support/emc-powerlink.htm
3.
Figure 73 EMC Unisphere GUI
Note
Create Storage Pools
Create a pool with the appropriate number of disks as shown in the EMC storage layout in Figure 74.
1.
2.
3.
Figure 74 Selecting the Disk Configuration Mode
4.
5.
Figure 75 Specifying Pool Name in the Disk Configuration Wizard
6.
Figure 76 Selecting Storage Type in the Disk Configuration Wizard
Figure 77 shows the details of the storage pool you have created.
Figure 77 Window Showing Storage Pools Details
Figure 78 Window Showing Storage Pools in EMC Unisphere
Note
Note
Configure Advanced Features—Link Aggregation and Jumbo Frames
To configure advanced features - link aggregation and jumbo frames, follow these steps:
1.
2.
3.
4.
Figure 79 Advanced Configuration Window in EMC Unisphere
Create iSCSI Servers
To create iSCSI servers, follow these steps:
1.
2.
Figure 80 Adding iSCSI Server for Storage Processor - A
3.
Figure 81 Adding iSCSI Server for Storage Processor - B
Note
Figure 82 iSCSI Server Settings Window in EMC Unisphere
Create Microsoft Hyper-V Datastores
To create Hyper-V datastores, follow these steps:
1.
2.
Figure 83 Specifying Hyper-V Datastore Name
3.
Note
Figure 84 Configuring Storage Pool and Size for the Datastore
4.
Figure 85 Setting Configure Protection
5.
6.
Note
Figure 86 Hyper-V Datastores for Cluster Shared Volume
Create Hosts and Provide Host Access
To create hosts and provide host access, follow these steps:
1.
The "Host Wizard" page appears.
2.
The "Operating System" page appears.
Figure 87 Specifying Name for Host Configuration
3.
The "Network Address" page appears.
Figure 88 Specifying Host Operating System
4.
Figure 89 Specifying the Host Network Address
5.
The "Summary" page appears.
Figure 90 Specifying iSCSI Unique Number (IQN)
6.
Figure 91 Host Configuration Summary
7.
Figure 92 Hosts Window Listing All the Hyper-V Hosts
8.
9.
10.
11.
Figure 93 Hyper-V Datastore Details Window
12.
Microsoft Windows Failover Cluster Setup
iSCSI Initiator Configuration
To connect iSCSI targets and configure advanced settings, follow these steps:
1.
2.
3.
Figure 94 Adding Target Portal in iSCSI Initiator Properties Window
4.
Figure 95 Changing the Default Settings of Target Portal
5.
a.
b.
c.
Figure 96 Advanced Settings Window
6.
7.
8.
The "Connect To Target dialog box" appears.
9.
a.
b.
Figure 97 Window Displaying Discovered Targets
10.
11.
12.
Figure 98 iSCSI Initiator Properties Displaying Session Identifiers
13.
The Advanced Settings dialog box appears.
14.
a.
b.
c.
Figure 99 Advanced Settings Window
15.
16.
Figure 100 Status of the Discovered Targets
17.
The Advanced Settings dialog box appears.
18.
a.
b.
c.
Figure 101 Advanced Settings
Figure 102 shows the partial output of command mpclaim.exe -b captured in a file.
Figure 102 mpclaim.exe Configuration Output
19.
20.
21.
Once all the disks are initialized, format with the NTFS file system and assign drive letters to them.
22.
Cluster Validation
For cluster validation, follow these steps:
1.
2.
3.
a.
b.
c.
d.
e.
Note
f.
Figure 103 Failover Cluster Validation Report
Failover Cluster Setup
To setup a failover cluster, follow these steps:
1.
2.
3.
Figure 104 Selecting Servers for Adding into the Cluster
4.
Figure 105 Cluster Details
5.
Figure 106 Confirmation Window for Creating Cluster
6.
Figure 107 Window Showing Summary of the Created Cluster
7.
Figure 108 Failover Cluster Manager
8.
Figure 109 Configure Quorum Configuration
9.
10.
Figure 110 Failover Cluster Manager Online Status
11.
12.
13.
Figure 111 Enabling Cluster Shared Volumes (CSV)
14.
Figure 112 Adding Storage to CSV
15.
Figure 113 Select Available Disks For CSV Storage
16.
Figure 114 Window Showing CSV Disk Status
17.
Figure 115 Live Migration Summary in Failover Cluster Manager
Microsoft System Center-2012 VMM Configuration
This section provides configuration details of Microsoft System Center Virtual machine Manager (VMM).
Add Microsoft Hyper-V Hosts and Cluster
To add Microsoft Hyper-V hosts and cluster, follow these steps:
1.
Figure 116 Create a Host Group in SCVMM
2.
Figure 117 Selecting Resource Location
3.
4.
Figure 118 Specifying Discovery Scope for VM Hosts
5.
Figure 119 Target Resources to Add as Hosts
6.
Figure 120 Specifying Host Group and VM Path for Hosts
Create a Template for Virtual Machine Deployment
This section covers how to create A Virtual Machine, A Virtual Machine Template, and Highly Available Virtual Machines from the Template.
To create a virtual machine from a blank .vhd file, follow these steps:
1.
2.
The "Create Virtual Machine" wizard appears.
Figure 121 Create Virtual Machine
3.
Figure 122 Selecting Source for New VM
4.
5.
a.
b.
6.
a.
b.
Under Machine Resources, click Virtual Hard Disk. You can accept the default values, or choose a different destination path on the host for the .vhd file. To change the .vhd file name, enter a new name in the File name box.
c.
d.
e.
7.
Create a Virtual Machine Template
This section covers on how-to create a virtual machine template from an existing virtual machine that is deployed on a host.
To create a virtual machine template, follow these steps:
1.
2.
Figure 123 Creating VM Template
A warning message is displayed "creating a template destroys the source virtual machine, and any user data on the source virtual machine may be lost".
3.
4.
5.
6.
If you have an existing guest operating system profile that you want to use, in the Guest OS profile list, click the desired guest operating system profile.
7.
8.
9.
Figure 124 Selecting Location for Saving the VM
10.
11.
Figure 125 Modifying Template Properties for VM High Availability
Create a Highly Available Virtual Machine from a Template
You can use the following procedure to create a virtual machine from a virtual machine template in System Center 2012 - Virtual Machine Manager (VMM).
To create a highly available virtual machine from a template, follow these steps:
1.
2.
The "Create Virtual Machine Wizard" opens.
3.
4.
Figure 126 Selecting Source for Virtual Machine
5.
6.
7.
If you have an existing hardware profile with settings that you want to use, in the Hardware profile list, click the desired hardware profile.
8.
Figure 127 Configuring Hardware for the VM
In the "Configure Operating System" page, configure the guest operating system settings. If you have an existing guest operating system profile that you want to use, in the Guest OS profile list, click the desired guest operating system profile.
9.
Figure 128 Configuring OS for the VM
10.
11.
12.
a.
b.
c.
Figure 129 Window Showing VM Settings
d.
You can either accept or change the computer name.
Figure 130 Window Showing Virtual Machine Settings
e.
f.
g.
h.
Figure 131 Virtual Machine Settings Confirmation Window
13.
Figure 132 100 Highly Available Virtual Machines in the SCVMM Console
14.
Figure 133 100 Highly Available Virtual Machines in the Failover Cluster Manager
VSPEX M50 Configuration Details
Cabling Information
This information is provided as a reference for cabling the physical equipment in a VSPEX M50 environment. The tables in this section include both the local and remote device and the port locations in order to simplify cabling requirements.
This document assumes that out-of-band management ports are plugged into an existing management infrastructure at the deployment site.
Ensure that you follow the cable directions in this section. Failure to do so results in the necessary changes to the deployment procedures that follow because specific port locations are mentioned. Before starting, ensure that the configuration matches what is described in the tables and diagrams in this section.
Figure 134 shows the VSPEX M50 cabling diagram. The labels indicate connections to end points rather than port numbers on the physical device.
For example, connection A is a 1 Gb target port connected from the EMC VNXe3150 SP B to Cisco Nexus 3048 A and connection R is a 1 Gb target port connected from Broadcom NIC 3 on Server 2 to Cisco Nexus 3048 B. Connections W and X are 10 Gb vPC peer-links connected from Cisco Nexus 3048 A to Cisco Nexus 3048 B
Figure 134 Cabling Details for VSPEX Microsoft Hyper-V 50 Virtual Machines
Table 12 and Table 13 lists five major cabling sections in these architectures:
1.
2.
3.
4.
5.
Connect all the cables as outlined in Figure 134 and in Table 12 and Table 13.
Prepare and Configure the Cisco Nexus 3048 Switch
The following section provides a detailed procedure for configuring the Cisco Nexus 3048 switches for use in EMC VSPEX M50 solution.
Figure 135 shows two Cisco Nexus switches configured for vPC. In vPC, a pair of switches acting as vPC peer endpoints looks like a single entity to port-channel-attached devices, although the two devices that act as logical port-channel endpoint are still two separate devices. This provides hardware redundancy with port-channel benefits. Both switches form a vPC Domain, in which one vPC switch is Primary while the other is secondary.
Note
Figure 135 Networking Configuration for EMC VSPEX M50
Initial Setup of Cisco Nexus Switches
See the corresponding section in the VSPEX M100 Configuration Details to complete the initial setup on both Cisco Nexus 3048 switches.
Enable Features, Jumbo Frames and Global Configuration
See the corresponding section in the VSPEX M100 Configuration Details to complete the global configuration on both Cisco Nexus 3048 switches.
Configure VLANs
In this VSPEX M50 configuration, create 3 VLANs on both the Cisco Nexus switches using the below table as reference. Storage VLAN is for iSCSI traffic between the host iSCSI NICs and the storage array. Cluster VLAN is for the cluster communication traffic (heartbeat, CSV, and live migration) between the failover cluster nodes. VM_traffic VLAN is for the virtual machines data traffic. Default VLAN is used by the host for management and infrastructure traffic.
For Cisco Nexus A and Cisco Nexus B
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Configure Port Channels
This section describes configuring port-channels on both the Cisco Nexus switches.
Create Port Channels
For Cisco Nexus A and Cisco Nexus B
From the global configuration mode, type interface Po10.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
Add Port Channel Configurations
These steps provide details for adding Port Channel configurations.
For Cisco Nexus A and Cisco Nexus B
From the global configuration mode, type interface Po10.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
Configure Virtual Port Channels
These steps provide details for configuring virtual Port Channels (vPCs).
For Cisco Nexus A and Cisco Nexus B
From the global configuration mode, type vpc domain <Nexus vPC domain ID>.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
At this point of time, all ports and port-channels are configured with necessary VLANs, switchport mode and vPC configuration. Validate this configuration using the "show port-channel summary" and "show vpc" commands as shown in Figure 137 and Figure 138.
Figure 136 Command for Showing VLAN Details
Ensure that on both switches, all required VLANs are in "active" status and right set of ports and port-channels are part of the necessary VLANs.
Port-channel configuration can be verified using "show port-channel summary" command. Figure 137 shows the expected output of this command.
Figure 137 Command for Showing Port Channel Summary
In this example, port-channel 10 is the vPC peer-link port channel, port channels 3, 4 and 5 are connected to the Cisco 1GigE I350 LOM on the host, port channels 14, 16 and 18 are connected to the Broadcom NICs on the host, and port channels 25 and 26 are connected to the storage array. Make sure that state of the member ports of each port channel is "P" (Up in port-channel).
Note
Common reasons for port channel port being down are:
•
•
vPC status can be verified using "show vpc" command. Figure 138 shows an example output.
Figure 138 Command for Showing vPC Details
Make sure that vPC peer status is "peer adjacency formed ok" and all the port-channels, including the peer-link port-channel, have status "up".
Infrastructure Servers
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Active Directory Domain Controller
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Microsoft SQL Server
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Microsoft System Center VMM
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Prepare the Cisco UCS C220 M3 Servers
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Configure Cisco Integrated Management controller (CIMC)
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Configure RAID
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Enable Virtualization Technology in BIOS
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Installing Microsoft Windows Server OS on UCS C220 M3 Servers
See the corresponding section in the VSPEX M100 Configuration Details to complete the task.
Device Driver Installation
VSPEX M50 solution contains Cisco GigE I350 LOM and quad-port Broadcom BCM5709C NetXtreme II GigE adapter.
To install device drivers follow these steps:
1.
2.
a.
http://www.broadcom.com/support/ethernet_nic/netxtremeii.php
b.
Network Configuration
This section provides steps to configure the NIC teaming of Cisco 1 GigE I350 LOM adapters and Broadcom BCM5709C NetXtreme II GigE adapters and assign IP addresses on all the Windows host servers.
NIC Teaming of Cisco 1 GigE LOM
See the corresponding section in the VSPEX M100 Configuration Details to complete the task. Assign an IP address to this teamed adapter from the management VLAN subnet.
NIC Teaming of Broadcom BCM5709C NetXtreme II GigE adapter
In this section, only the NICs connected to "cluster" VLAN is teamed.
Note
To team the Broadcom NICs connected to the "cluster" VLAN follow these steps:
1.
2.
Figure 139 Teaming Broadcom Adapters
3.
4.
5.
6.
Figure 140 Selecting Broadcom Adapters for Teaming
7.
8.
9.
10.
11.
Figure 141 Confirmation Window to Commit Changes
12.
Figure 142 Window Showing Teamed Broadcom Adapters
13.
Host Rename and Domain Join
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task
Install Roles and Features
See the steps in the corresponding section of VSPEX M100 Configuration Details and complete the steps from 1 to 13.
In step 14, choose the teamed Cisco 1GigE I350 LOM adapter for the External Connection type for creating a virtual switch. Choose "Allow management operating system to share the network adapter".
Figure 143 Microsoft Hyper-V Virtual Network Manager
The above step also creates a virtual NIC for the host machine and retains the static IP address assigned in the earlier step for host management. Note, this NIC in management VLAN allows host management traffic.
Figure 144 Control Panel Network Connections
Enable iSCSI initiator
See the corresponding section of VSPEX M100 Configuration Details to complete the task.
Prepare the EMC VNXe3150 Storage
The interface and configuration of the EMC VNXe3150 is very similar to the EMC VNXe3300, so see the section "Prepare the EMC VNXe3300 storage" in VSPEX M100 Configuration Details to complete the task.
Initial Setup of EMC VNXe
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Create Storage Pools
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task. Here you create a single storage pool using 45 disks.
Configure Advanced features—Link Aggregation and Jumbo Frames
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
Create iSCSI Servers
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task
Create Hosts
See the steps in the corresponding section of the VSPEX M100 Configuration Details to complete the task
Create Hyper-V Datastores
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task. For the 50 Virtual Machines configuration create six 750GB Hyper-V datastores for CSV and another small datastore for the cluster witness disk.
Microsoft Windows Failover Cluster Setup
See the steps in the corresponding section of VSPEX M100 Configuration Details to complete the task.
iSCSI Initiator Configuration
See the steps in the corresponding section of the VSPEX M100 Configuration Details to complete the task
Cluster Validation
See the corresponding section of VSPEX M100 Configuration Details to complete the task
Failover Cluster setup
See the corresponding section in the VSPEX M100 Configuration Details to complete the task.
Microsoft System Center-2012 VMM Configuration
See the corresponding section in the VSPEX M100 Configuration Details to complete the task. You need to ensure to connect the VM vNICs to the vm_traffic VLAN. This can be achieved by following the steps while creating the VM or editing the settings for the VM. This step is to allow traffic from management VLAN and vm_traffic VLAN to pass through the teamed Cisco 1GigE I350 LOM. On the other end of the Cisco Nexus switch port where the teamed adapters are connected are configured as trunk ports to allow multiple VLAN traffic.
In the settings for VM, choose "Enable Virtual LAN Identification" and enter 23 in the field for VLAN ID.
Figure 145 Window Showing Network Adapter Settings
Validating Cisco Solution for EMC VSPEX Microsoft Hyper-V Architectures
This section provides a list of items that should be reviewed once the solution has been configured. The goal of this section is to verify the configuration and functionality of specific aspects of the solution, and ensure that the configuration supports core availability requirements.
Post Install Checklist
The following configuration items are critical to functionality of the solution, and should be verified prior to deployment into production.
For post install checklist follow these steps:
1.
2.
3.
4.
Figure 146 Validating Jumbo Frames Support
Verify the Redundancy of the Solution Components
The following redundancy checks were performed at the Cisco lab to verify solution robustness:
1.
2.
3.
4.
5.
Note
Cisco Validation Test Profile
"vdbench" testing tool was used with the Microsoft Windows 2008 R2 SP1 server to test scaling of the solution in Cisco labs. The details on the test profile used is displayed in Table 15.
Bill of Material
Table 16 gives details of the components used in the CVD for 50/100 virtual machines configuration.
For more information on the part numbers and options available for customization, see Cisco C220 M3 server specsheet at:
http://www.cisco.com/en/US/prod/collateral/ps10265/ps10493/C220M3_SFF_SpecSheet.pdf
Customer Configuration Data Sheet
Before you start the configuration, gather some customer-specific network and host configuration information. Table 17, Table 18, Table 19, Table 20, Table 21, Table 22 provide information on assembling the required network and host address, numbering, and naming information. This worksheet can also be used as a "leave behind" document for future reference.
The EMC VNXe Series Configuration Worksheet should be cross-referenced to confirm customer information.
Table 17 Common Server Information
Domain Controller
DNS Primary
DNS Secondary
DHCP
NTP
SMTP
SNMP
vCenter Console
SQL Server
Table 19 Array Information
Admin account
Management IP
Storage pool name
Datastore name
iSCSI Server IP
Table 20 Network Infrastructure Information
Cisco Nexus 5548UP Switch A
Cisco Nexus 5548UP Switch B
Table 22 Service Accounts
Microsoft Windows Server administrator
Array administrator
SCVMM administrator
SQL Server administrator
References
Cisco Unified Computing System:
http://www.cisco.com/en/US/solutions/ns340/ns517/ns224/ns944/unified_computing.html
Cisco UCS C-Series Servers Documentation Roadmap
http://www.cisco.com/go/unifiedcomputing/c-series-doc
Cisco Nexus:
http://www.cisco.com/en/US/products/ps9441/Products_Sub_Category_Home.html
Cisco Nexus 5000 Series NX-OS Software Configuration Guide:
http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide.html
EMC VNXe3xxx series resources
http://www.emc.com/storage/vnx/vnxe-series.htm#!resources
EMC VNX5xxx series resources
http://www.emc.com/storage/vnx/vnx-series.htm#!resources
Network Adapter Virtualization Design (Adapter-FEX) with Cisco Nexus 5500 Switches
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/guide_c07-690080_ns1118_Networking_Solutions_White_Paper.html
Configuring Port Channels
http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/dcnm/interfaces/configuration/guide/if_portchannel.html
Configuring Port Profiles
http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/dcnm/interfaces/configuration/guide/if_portprofile.html
Configuring vPCs
http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/dcnm/interfaces/configuration/guide/if_vPC.html
System Center 2012 - Virtual Machine Manager
http://technet.microsoft.com/en-us/library/gg610610
Microsoft SQL Server installation guide
http://msdn.microsoft.com/en-us/library/ms143219.aspx
Feedback