FlashStack Virtual Server Infrastructure with FibreChannel Storage for VMware vSphere 7.0

FlashStack Virtual Server Infrastructure with FibreChannel Storage for VMware vSphere 7.0

Deployment Guide for FlashStack with Cisco UCS 6400 Fabric
Interconnects, Cisco UCS M5 Servers, and Pure Storage FlashArray//X R3 SeriesNOTE: Works with document’s Advanced Properties “First Published” property. Click File | Properties | Advanced Properties | Custom.

Published: January 2021

About the Cisco Validated Design Program

The Cisco Validated Design (CVD) program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information, go to:

http://www.cisco.com/go/designzone.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS-IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, the Cisco logo, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unified Computing System (Cisco UCS), Cisco UCS B-Series Blade Servers, Cisco UCS C-Series Rack Servers, Cisco UCS S-Series Storage Servers, Cisco UCS Manager, Cisco UCS Management Software, Cisco Unified Fabric, Cisco Application Centric Infrastructure, Cisco Nexus 9000 Series, Cisco Nexus 7000 Series. Cisco Prime Data Center Network Manager, Cisco NX-OS Software, Cisco MDS Series, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study, LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. LD1.

All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0809R)

© 2021 Cisco Systems, Inc. All rights reserved.

 

Contents

Executive Summary. 4

Solution Overview.. 5

Deployment Hardware and Software. 8

Network Switch Configuration. 19

Pure Storage Configuration. 25

MDS 9132T Configuration. 34

Cisco UCS Configuration. 37

MDS Fabric Zoning. 118

FlashArray Storage Deployment 123

vSphere Deployment 131

Appendix. 162

About the Author 172

Feedback. 173

Executive Summary

Cisco Validate Designs consist of systems and solutions that are designed, tested, and documented to facilitate and improve customer deployments. These designs incorporate a wide range of technologies and products into a portfolio of solutions that have been developed to address the business needs of our customers.

This document details the design in the FlashStack Virtual Server Infrastructure Design Guide for VMware vSphere 7.0, which describes a validated converged infrastructure jointly developed by Cisco and Pure Storage. This solution covers the deployment of a predesigned, best-practice data center architecture with VMware vSphere built on the Cisco Unified Computing System (Cisco UCS), the Cisco Nexus® 9000 family of switches, Cisco MDS 9000 family of Fibre Channel switches, and Pure Storage FlashArray//X all flash storage configured for Fibre Channel based storage access.

When deployed, the architecture presents a robust infrastructure viable for a wide range of application workloads implemented as a virtual server infrastructure.

Solution Overview

Introduction

In the current industry there is a trend for pre-engineered solutions which standardize the data center infrastructure, offering the business operational efficiencies, agility, and scale to address cloud, bimodal IT, and their business. Their challenge is complexity, diverse application support, efficiency, and risk; all these are met by FlashStack with:

●      Reduced complexity and automatable infrastructure and easily deployed resources

●      Robust components capable of supporting high performance and high bandwidth virtualized applications

●      Efficiency through optimization of network bandwidth and in-line storage compression with de-duplication

●      Risk reduction at each level of the design with resiliency built into each touch point throughout

Cisco and Pure Storage have partnered to deliver this Cisco Validated Design, which uses best of breed storage, server, and network components to serve as the foundation for virtualized workloads, enabling efficient architectural designs that can be quickly and confidently deployed.

This document describes a reference architecture detailing a Virtual Server Infrastructure composed of Cisco Nexus switches, Cisco UCS Compute, Cisco MDS Multilayer Fabric Switches, and a Pure Storage FlashArray//X delivering a VMware vSphere 7.0 hypervisor environment.

Audience

The audience for this document includes, but is not limited to; sales engineers, field consultants, professional services, IT managers, partner engineers, and customers who want to take advantage of an infrastructure built to deliver IT efficiency and enable IT innovation.

Purpose of this Document

This document details a step-by-step configuration and implementation guide for FlashStack, centered around the Cisco UCS 6454 Fabric Interconnect and the Pure Storage FlashArray//X50 R3. These components are supported by the 1/10/25/40/50/100G capable Cisco Nexus 93180YC-FX switch and 32G FC capable Cisco MDS 9132T to deliver a Virtual Server infrastructure on Cisco UCS C125 M5 Server nodes and Cisco UCS B200 M5 Blade Servers running VMware vSphere 7.0.

The design that will be implemented is discussed in the FlashStack Virtual Server Infrastructure for VMware vSphere 7.0 Design Guide.

Figure 1.                 FlashStack with Cisco UCS 6454 and Pure Storage FlashArray //50 R3

The reference hardware configuration includes:

●      Two Cisco Nexus 93180YC-FX Switches

●      Two Cisco MDS 9132T Switches

●      Two Cisco UCS 6454 Fabric Interconnects

●      Cisco UCS 5108 Chassis with two Cisco UCS 2408 Fabric Extenders

●      Four Cisco UCS B200 M5 Blade Servers

●      Four Cisco UCS C125 M5 Server Nodes

●      One Pure Storage FlashArray//X50 R3

The virtual environment this supports is within VMware vSphere 7.0 and includes virtual management and automation components from Cisco and Pure Storage built into the solution, or as optional add-ons.

This document will provide a low-level example of steps to deploy this base architecture that may need some adjustments depending on the customer environment. These steps include physical cabling, network, storage, compute, and virtual device configurations

What’s New in this Release?

This FlashStack Datacenter with VMware vSphere 7.0 validated design introduced new hardware and software into the portfolio, enabling 10/25/40/100GbE along with native 32Gb FC via the Cisco MDS Fibre Channel switch or the Cisco Nexus 93180YC-FX switch. This primary design has been updated to include the latest Cisco and Pure Storage hardware and software and has the following new features:

●      Support for the Cisco UCS 4.1(2) unified software release, Cisco UCS C125 servers with AMD EPYC 2^nd Generation Processors, Cisco UCS B200-M5 and C220-M5 servers with 2^nd Generation Intel Xeon Scalable Processors, and Cisco 1400 Series Virtual Interface Cards (VICs)

●      Support for the latest Cisco UCS 6454 and 64108 (supported but not validated) Fabric Interconnects

●      Support for the latest Cisco UCS 2408 Fabric Extender

●      Support for 32Gb FC SAN Switching in the Cisco Nexus 93180YC-FX switch (appendix)

●      Support for Cisco Intersight Software as a Service (SaaS) Management

●      Support for the Pure Storage FlashArray//X R3 array

●      Support for the latest release of Pure Storage Purity 6

●      Validation of VMware vSphere 7.0

●      Unified Extensible Firmware Interface (UEFI) Secure Boot of VMware ESXi 7.0

●      25 or 100 Gigabit per second Ethernet Connectivity

Deployment Hardware and Software

Software Revisions

Table 1 lists the software versions for hardware and virtual components used in this solution. Each of these versions have been used have been certified within interoperability matrixes supported by Cisco, Pure Storage, and VMware. For the current supported version information, consult the following sources:

●      Cisco UCS Hardware and Software Interoperability Tool: http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html

●      Pure Storage Interoperability(note, this interoperability list will require a support login form Pure):  https://support.purestorage.com/FlashArray/Getting_Started/Compatibility_Matrix

●      Pure Storage FlashStack Compatibility Matrix (note, this interoperability list will require a support login from Pure): https://support.purestorage.com/FlashStack/Product_Information/FlashStack_Compatibility_Matrix

●      VMware Compatibility Guide: http://www.vmware.com/resources/compatibility/search.php

●      Additionally, it is also strongly suggested to align FlashStack deployments with the recommended release for the Cisco Nexus 9000 switches used in the architecture:

◦    Cisco Nexus: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/recommended_release/b_Minimum_and_Recommended_Cisco_NX-OS_Releases_for_Cisco_Nexus_9000_Series_Switches.html

◦    MDS: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/b_MDS_NX-OS_Recommended_Releases.html

If versions differ from the validated versions listed below, it is highly recommended to read the release notes of the selected version to be aware of any changes to features or commands that may have occurred.

Table 1. Software Revisions

Configuration Guidelines

This document details the step-by-step configuration of a fully redundant and highly available Virtual Server Infrastructure built on Cisco and Pure Storage components. References are made to which component is being configured with each step, either 01 or 02 or A and B. For example, controller-1 and controller-2 are used to identify the two controllers within the Pure Storage FlashArray//X that are provisioned with this document, and Cisco Nexus A or Cisco Nexus B identifies the pair of Cisco Nexus switches that are configured. The Cisco UCS fabric interconnects are similarly configured. Additionally, this document details the steps for provisioning multiple Cisco UCS hosts, and these examples are identified as: VM-Host-FC-01, VM-Host-FC-02 to represent Fibre Channel booted infrastructure and production hosts deployed to the fabric interconnects in this document. Finally, to indicate that you should include information pertinent to your environment in a given step, <<text>> appears as part of the command structure. See the following example during a configuration step for both Nexus switches:

BB08-93180YC-FX-A (config)# ntp server <<var_oob_ntp>> use-vrf management

This document is intended to enable you to fully configure the customer environment. In this process, various steps require you to insert customer-specific naming conventions, IP addresses, and VLAN schemes, as well as to record appropriate MAC addresses. Table 2 lists the VLANs necessary for deployment as outlined in this guide, and Table 3 lists the external dependencies necessary for deployment as outlined in this guide.

Table 2. Required VLANs

Table 3. Infrastructure Servers

Table 4. Configuration Variables

Physical Topology

This section details a cabling example for a FlashStack environment. To make connectivity clear in this example, the tables include both the local and remote port locations.

This document assumes that out-of-band management ports are plugged into an existing management infrastructure at the deployment site. The upstream network from the Cisco Nexus 93180YC-FX switches is out of scope of this document, with only the assumption that these switches will connect to the upstream switch or switches with a vPC.

Figure 2.                 FlashStack Cabling in Validated Topology

Table 5. Cisco Nexus 93180YC-FX-A Cabling Information

Table 6. Cisco Nexus 93180YC-FX-B Cabling Information

Table 7. Cisco UCS-6545-A Cabling Information

Table 8. Cisco UCS-6545-B Cabling Information

Table 9. Cisco MDS-9132T-A Cabling Information

Table 10.                Cisco MDS-9132T-B Cabling Information

Table 11.                Pure Storage FlashArray//X50 R3 Controller 1 Cabling Information

Table 12.                Pure Storage FlashArray//X50 R3 Controller 2 Cabling Information

Network Switch Configuration

Network Configuration

The following procedures describe how to configure the Cisco Nexus switches for use in a base FlashStack environment. This procedure assumes you’re using Cisco Nexus 93180YC-FX switches running NX-OS 9.3(5). Configuration on a differing model of the Cisco Nexus 9000 series switch should be comparable but may differ slightly with the model and changes in NX-OS release. The Cisco Nexus 93180YC-FX switch and NX-OS 9.3(5) release were used to validate this FlashStack solution, so the steps will reflect this model and release.

Figure 3.                 Network Configuration Workflow

Physical Connectivity

Physical cabling should be completed by following the diagram and table references in the previous section referenced as FlashStack Cabling.      

Cisco Nexus Basic System Configuration Dialog

This section provides detailed instructions for the configuration of the Cisco Nexus 93180YC-FX switches used in this FlashStack solution. Some changes may be appropriate for a customer’s environment, but care should be taken when stepping outside of these instructions as it may lead to an improper configuration.

Cisco Nexus 93180YC-FX-A

To set up the initial configuration for the Cisco Nexus A switch on <nexus-A-hostname>, follow these steps:

1.     Review the configuration summary before enabling the configuration.

Cisco Nexus 93180YC-FX-B

To set up the initial configuration for the Cisco Nexus A switch on <nexus-A-hostname>, follow these steps:

1.     Review the configuration summary before enabling the configuration.

Cisco Nexus Switch Configuration

Enable Features and Settings

To enable the required features for this deployment, run the following commands on both Cisco Nexus Switches:

Configure Global Settings

To configure global settings for this deployment, run the following commands on both Cisco Nexus Switches:

Configure VLANs

To configure VLANs for this deployment, run the following commands on both Cisco Nexus Switches:

Continue adding VLANs as appropriate for the environment.

Add Interface Port Descriptions for Cisco Nexus 93180YC-FX-A

To configure port descriptions for this deployment, run the following commands on both Cisco Nexus Switches:

Add Interface Port Descriptions for Cisco Nexus 93180YC-FX-B

To configure port descriptions for this deployment, run the following commands on both Cisco Nexus Switches:

Port Channel Setup

Configure vPC Domain Settings for Cisco Nexus 93180YC-FX-A

The vPC domain will be assigned a unique number from 1-1000 and will handle the vPC settings specified within the switches. For this deployment, vPC domain 10 is used.

Configure vPC Domain Settings for Cisco Nexus 93180YC-FX-B

The vPC domain will be assigned a unique number from 1-1000 and will handle the vPC settings specified within the switches. For this deployment, vPC domain 10 is used.

Configure vpc Peer-Link

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC Peer Link and then configure the Peer Link.

Configure Port-Channel to Fabric Interconnect A

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to Fabric Interconnect A.

Configure Port-Channel to Fabric Interconnect B

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to Fabric Interconnect B.

Configure Port-Channel to Upstream network

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to the upstream network.

Pure Storage Configuration

Pure Storage FlashArray//X50 R3 Configuration

FlashArray Initial Configuration

The following information should be gathered to enable the installation and configuration of the FlashArray. An official representative of Pure Storage will help rack and configure the new installation of the FlashArray.

When the FlashArray has completed the initial configuration, it is important to configure the Cloud Assist phone-home connection to provide the best proactive support experience possible. Furthermore, this will enable the analytics functionalities provided by Pure1.

Add an Alert Recipient

The Alerts sub-view is used to manage the list of addresses to which Purity delivers alert notifications, and the attributes of alert message delivery. You can designate up to 19 alert recipients. The Alert Recipients section displays a list of email addresses that are designated to receive Purity alert messages. Up to 20 alert recipients can be designated. The list includes the built-in flasharray-alerts@purestorage.com address, which cannot be deleted.

The email address that Purity uses to send alert messages includes the sender domain name and is comprised of the following components:

<Array_Name>-<Controller_Name>@<Sender_Domain_Name>.com

To add an alert recipient, follow these steps:

1.     Click Settings.

2.     In the Alert Watchers section, enter the email address of the alert recipient and click the + icon.

The Relay Host section displays the hostname or IP address of an SMTP relay host if one is configured for the array. If you specify a relay host, Purity routes the email messages via the relay (mail forwarding) address rather than sending them directly to the alert recipient addresses.

In the Sender Domain section, the sender domain determines how Purity logs are parsed and treated by Pure Storage Support and Escalations. By default, the sender domain is set to the domain name please-configure.me.

Configure Pure1 Support

The Pure1 Support section manages settings for Phone Home, Remote Assist, and Support Logs.

The phone home facility provides a secure direct link between the array and the Pure Storage Technical Support web site. The link is used to transmit log contents and alert messages to the Pure Storage Support team so that when diagnosis or remedial action is required, complete recent history about array performance and significant events is available. By default, the phone home facility is enabled. If the phone home facility is enabled to send information automatically, Purity transmits log and alert information directly to Pure Storage Support via a secure network connection. Log contents are transmitted hourly and stored at the support web site, enabling detection of array performance and error rate trends. Alerts are reported immediately when they occur so that timely action can be taken.

Phone home logs can also be sent to Pure Storage Technical support on demand, with options including Today's Logs, Yesterday's Logs, or All Log History.

The Remote Assist section displays the remote assist status as "Connected" or "Disconnected". By default, remote assist is disconnected. A connected remote assist status means that a remote assist session has been opened, allowing Pure Storage Support to connect to the array. Disconnect the remote assist session to close the session.

The Support Logs section allows you to download the Purity log contents of the specified controller to the current administrative workstation. Purity continuously logs a variety of array activities, including performance summaries, hardware and operating status reports, and administrative actions.

Configure DNS Server IP Addresses

To configure the DNS server IP addresses, follow these steps:

1.     Click Settings > Network.

2.     In the DNS section, hover over the domain name and click the pencil icon. The Edit DNS dialog box appears.

3.     Complete the following fields:

a.     Domain: Specify the domain suffix to be appended by the array when doing DNS lookups.

b.     NS#: Specify up to three DNS server IP addresses for Purity to use to resolve hostnames to IP addresses. Enter one IP address in each DNS# field. Purity queries the DNS servers in the order that the IP addresses are listed.

4.     Click Save.

Directory Service

The Directory Service manages the integration of FlashArray with an existing directory service. When the Directory Service sub-view is configured and enabled, the FlashArray leverages a directory service to perform user account and permission level searches. Configuring directory services is OPTIONAL.

The FlashArray is delivered with a single local user, named pureuser, with array-wide (Array Admin) permissions.

To support multiple FlashArray users, integrate the array with a directory service, such as Microsoft Active Directory or OpenLDAP.

Role-based access control is achieved by configuring groups in the directory that correspond to the following permission groups (roles) on the array:

●      Read Only Group. Read Only users have read-only privilege to run commands that convey the state of the array. Read Only uses cannot alter the state of the array.

●      Storage Admin Group. Storage Admin users have all the privileges of Read Only users, plus the ability to run commands related to storage operations, such as administering volumes, hosts, and host groups. Storage Admin users cannot perform operations that deal with global and system configurations.

●      Array Admin Group. Array Admin users have all the privileges of Storage Admin users, plus the ability to perform array-wide changes. In other words, Array Admin users can perform all FlashArray operations.

To configure the Directory Service, follow these steps:

1.     Click Settings > Users.

2.     Click the icon in the Directory Services panel:

a.     Enabled: Click the check box to leverage the directory service to perform user account and permission level searches.

b.     URI: Enter the comma-separated list of up to 30 URIs of the directory servers. The URI must include a URL scheme (ldap, or ldaps for LDAP over SSL), the hostname, and the domain. You can optionally specify a port. For example, ldap://ad.company.com configures the directory service with the hostname "ad" in the domain "company.com" while specifying the unencrypted LDAP protocol.

c.     Base DN: Enter the base distinguished name (DN) of the directory service. The Base DN is built from the domain and should consist only of domain components (DCs). For example, for ldap://ad.storage.company.com, the Base DN would be: “DC=storage,DC=company,DC=com”.

d.     Bind User: Username used to bind to and query the directory. For Active Directory, enter the username - often referred to as sAMAccountName or User Logon Name - of the account that is used to perform directory lookups. The username cannot contain the characters " [ ] : ; | = + * ? < > / \ and cannot exceed 20 characters in length. For OpenLDAP, enter the full DN of the user. For example, "CN=John,OU=Users,DC=example,DC=com".

e.     Bind Password: Enter the password for the bind user account.

f.      Group Base: Enter the organizational unit (OU) to the configured groups in the directory tree. The Group Base consists of OUs that, when combined with the base DN attribute and the configured group CNs, complete the full Distinguished Name of each groups. The group base should specify "OU=" for each OU and multiple OUs should be separated by commas. The order of OUs should get larger in scope from left to right. In the following example, SANManagers contains the sub-organizational unit PureGroups: "OU=PureGroups,OU=SANManagers".

g.     Array Admin Group: Common Name (CN) of the directory service group containing administrators with full privileges to manage the FlashArray. Array Admin Group administrators have the same privileges as pureuser. The name should be the Common Name of the group without the "CN=" specifier. If the configured groups are not in the same OU, also specify the OU. For example, "pureadmins,OU=PureStorage", where pureadmins is the common name of the directory service group.

h.     Storage Admin Group: Common Name (CN) of the configured directory service group containing administrators with storage related privileges on the FlashArray. The name should be the Common Name of the group without the "CN=" specifier. If the configured groups are not in the same OU, also specify the OU. For example, "pureusers,OU=PureStorage", where pureusers is the common name of the directory service group.

i.       Read Only Group: Common Name (CN) of the configured directory service group containing users with read-only privileges on the FlashArray. The name should be the Common Name of the group without the "CN=" specifier. If the configured groups are not in the same OU, also specify the OU. For example, "purereadonly,OU=PureStorage", where purereadonly is the common name of the directory service group.

j.       Check Peer: Click the check box to validate the authenticity of the directory servers using the CA Certificate. If you enable Check Peer, you must provide a CA Certificate.

k.     CA Certificate: Enter the certificate of the issuing certificate authority. Only one certificate can be configured at a time, so the same certificate authority should be the issuer of all directory server certificates. The certificate must be PEM formatted (Base64 encoded) and include the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" lines. The certificate cannot exceed 3000 characters in total length.

3.     Click Save.

4.     Click Test to test the configuration settings. The LDAP Test Results pop-up window appears. Green squares represent successful checks. Red squares represent failed checks.

SSL Certificate

Self-Signed Certificate

Purity creates a self-signed certificate and private key when you start the system for the first time. The SSL Certificate sub-view allows you to view and change certificate attributes, create a new self-signed certificate, construct certificate signing requests, import certificates and private keys, and export certificates.

Creating a self-signed certificate replaces the current certificate. When you create a self-signed certificate, include any attribute changes, specify the validity period of the new certificate, and optionally generate a new private key.

When you create the self-signed certificate, you can generate a private key and specify a different key size. If you do not generate a private key, the new certificate uses the existing key.

You can change the validity period of the new self-signed certificate. By default, self-signed certificates are valid for 3650 days.

CA-Signed Certificate

Certificate Authorities (CA) are third party entities outside the organization that issue certificates. To obtain a CA certificate, you must first construct a certificate signing request (CSR) on the array.

The CSR represents a block of encrypted data specific to your organization. You can change the certificate attributes when you construct the CSR; otherwise, Purity will reuse the attributes of the current certificate (self-signed or imported) to construct the new one. Note that the certificate attribute changes will only be visible after you import the signed certificate from the CA.

Send the CSR to a certificate authority for signing. The certificate authority returns the SSL certificate for you to import. Verify that the signed certificate is PEM formatted (Base64 encoded), includes the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" lines, and does not exceed 3000 characters in total length. When you import the certificate, also import the intermediate certificate if it is not bundled with the CA certificate.

If the certificate is signed with the CSR that was constructed on the current array and you did not change the private key, you do not need to import the key. However, if the CSR was not constructed on the current array or if the private key has changed since you constructed the CSR, you must import the private key. If the private key is encrypted, also specify the passphrase.

Claim FlashArray//X in Intersight (optional)

Claiming a Pure Storage FlashArray or VMware vCenter in Cisco Intersight requires the use of an Intersight Assist virtual machine. Please refer to the following if link if there is not already an Intersight Assist system in your environment:  https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/cisco-intersight-assist-getting-started-guide/m-installing-cisco-intersight-assist.html

To claim FlashArray//X in Intersight, follow these steps:

1.     Open a browser to Cisco Intersight, https://intersight.com and log in to your Intersight account.

2.     Click Admin > Devices.

3.     Click Claim a New Device and click Claim Though Intersight Assist.

4.     Set Type to Pure Storage FlashArray.

5.     Enter FlashArray Hostname/ IP address and credentials.

6.     Click Claim.

MDS 9132T Configuration

The following procedures describe how to configure the Cisco MDS fibre channel switches for use in a base FlashStack environment. This procedure assumes the use of Nexus 9132T switches running NX-OS 8.4(1a)  Configuration on a differing model of Nexus 9000 series switch should be comparable but may differ slightly with model and changes in NX-OS release.

Physical Connectivity

Physical cabling should be completed by following the diagram and table references in the previous section referenced as FlashStack Cabling.      

Cisco MDS Basic System Configuration Dialog

This section provides detailed instructions for the configuration of the Cisco  9132T switches used in this FlashStack solution. Some changes may be appropriate for a customer’s environment, but care should be taken when stepping outside of these instructions as it may lead to an improper configuration.

Cisco MDS 9132T-A

To set up the initial configuration for the Cisco MDS A switch on <mds_A_hostname>, follow these steps:

1.     Configure the switch.

2.     Review the configuration summary before enabling the configuration.

Cisco MDS 9132T-B

To set up the initial configuration for the Cisco MDS B switch on <mds_B_hostname>, follow these steps:

1.     Configure the switch.

2.     Review the configuration summary before enabling the configuration.

Cisco MDS Switch Configuration

Enable Features and Settings

To enable the required features for this deployment, run the following commands on both Cisco Nexus Switches:

Configure Port Channel for MDS 9132T-A

To configure a port channel on MDS 9132T-A and add the interfaces connecting into the Cisco UCS Fabric Interconnect-A, run the following commands:

Configure VSAN for MDS 9132T-A

On MDS 9132T A create the VSAN that will be used for connectivity to the Cisco UCS Fabric Interconnect and the Pure Storage FlashArray. Assign this VSAN to the interfaces that will connect to the Pure Storage FlashArray, as well as the interfaces and the Port Channel they create that are connected to the Cisco UCS Fabric Interconnect, by running the following commands:

Configure Port Channel for MDS 9132T-B

On MDS 9132T-B configure a port channel and add the interfaces connecting into the Cisco UCS Fabric Interconnect-B, by running the following commands:

Configure VSAN for MDS 9132T-B

On MDS 9132T B create the VSAN that will be used for connectivity to the Cisco UCS Fabric Interconnect and the Pure Storage FlashArray. Assign this VSAN to the interfaces that will connect to the Pure Storage FlashArray, as well as the interfaces and the Port Channel they create that are connected to the Cisco UCS Fabric Interconnect, by running the following commands:

Cisco UCS Configuration

The following procedures describe how to configure the Cisco UCS domain for use in a base FlashStack environment. This procedure assumes you’re using Cisco UCS Fabric Interconnects running 4.1(2a). Configuration on a differing model of Cisco UCS Fabric Interconnects should be comparable but may differ slightly with the model and changes in the Cisco UCS Manager (Cisco UCSM) release. The Cisco USC 6454 Fabric Interconnects and Cisco UCS Manager 4.1(2a) release were used in validation of this FlashStack solution, so the steps will reflect this model and release.

Figure 4.                 Cisco UCS Configuration Workflow

Physical Connectivity

The physical cabling should be completed by following the diagram and table references in section Physical Topology.      

Cisco UCS Basic System Configuration Dialog

This section provides detailed instructions for the configuration of the Cisco UCS 6454 Fabric Interconnects used in this FlashStack solution. Some changes may be appropriate for a customer’s environment, but care should be taken when stepping outside of these instructions as it may lead to an improper configuration.

Cisco UCS Fabric Interconnect A

To set up the initial configuration for the Cisco Fabric Interconnect A, follow these steps:

1.     Configure the Fabric Interconnect:

2.     Review the configuration summary before enabling the configuration:

Cisco UCS Fabric Interconnect A

To set up the initial configuration for the Cisco Fabric Interconnect A, follow these steps:

1.     Configure the Fabric Interconnect:

2.     Review the configuration summary before enabling the configuration:

Cisco UCS Manager Configuration

To log into the Cisco Unified Computing System (Cisco UCS) environment, follow these steps:

1.     Open a web browser and navigate to the Cisco UCS fabric interconnect cluster address.

2.     Click the Launch UCS Manager link within the opening page.

3.     If prompted to accept security certificates, accept as necessary.

4.     When the UCS Manager login is prompted, enter admin for the user name and enter the administrative password.

5.     Click Login to log into Cisco UCS Manager.

Upgrade Cisco UCS Manager to Version 4.1(2a)

This document assumes you’re using Cisco UCS 4.1(2a). To upgrade the Cisco UCS Manager software and the Cisco UCS Fabric Interconnect software to version 4.1(2a), refer to Cisco UCS Manager Install and Upgrade Guides.

Enable Anonymous Reporting

During the first connection to the Cisco UCS Manager GUI, a pop-up window will appear to allow for the configuration of Anonymous Reporting to Cisco on use to help with future development. To create anonymous reporting, follow this step:

1.     In the Anonymous Reporting window, choose whether to send anonymous data to Cisco to improve future products, and provide the appropriate SMTP server gateway information if configuring:

If you want to enable or disable Anonymous Reporting at a later date, it can be found within Cisco UCS Manager under:  Admin -> Communication Management -> Call Home, which has a tab for Anonymous Reporting.

Configure Cisco UCS Call Home

During the first connection to the Cisco UCS Manager GUI, a pop-up window will appear to allow for the configuration of Anonymous Reporting to Cisco on use to help with future development. To create anonymous reporting, follow these steps:

1.     In Cisco UCS Manager, click the Admin tab in the navigation pane.

2.     Expand Communication Management and click Call Home.

3.     Change State to On.

4.     Fill in the fields according to your preferences and click Save Changes and then click OK.

Configure NTP

To synchronize the Cisco UCS environment to the NTP server, follow these steps:

1.     In Cisco UCS Manager, click the Admin tab in the navigation pane.

2.     Expand Timezone Management and click Timezone.

3.     In the Properties pane, select the appropriate time zone in the Timezone menu.

4.     Click Save Changes and then click OK.

5.     Click Add NTP Server.

6.     Enter <<var_oob_ntp>> and click OK.

7.     Click OK.

Configure Cisco UCS Servers

Edit Chassis Discovery Policy

Setting the discovery policy simplifies the addition of Cisco UCS B-Series chassis. To modify the chassis discovery policy, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane and select Policies from the drop-down list.

2.     Under Global Policies, set the Chassis/FEX Discovery Policy to match the number of uplink ports that are cabled between the chassis or fabric extenders (FEXes) and the fabric interconnects.

3.     Set the Link Grouping Preference to Port Channel.

4.     Leave other settings alone or change if appropriate to your environment.

5.     Click Save Changes.

6.     Click OK.

Enable Server and Uplink Ports

To enable server and uplink ports, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane.

2.     Click Equipment > Fabric Interconnects > Fabric Interconnect A (primary) > Fixed Module.

3.     Expand Ethernet Ports.

4.     Select the ports that are connected to the 2408, C4200 Servers, and/or C Series Servers , right-click them, and click “Configure as Server Port.”

5.     Click Yes to confirm server ports and click OK.

6.     Verify that the ports connected to the chassis are now configured as server ports.

7.     Select ports 49 and 50 that are connected to the Cisco Nexus switches, right-click them, and select Configure as Uplink Port.

8.     Click Yes to confirm uplink ports and click OK.

9.     Click Equipment > Fabric Interconnects > Fabric Interconnect B (subordinate) > Fixed Module.

10.  Expand Ethernet Ports.

11.  Select the ports that are connected to the chassis, right-click them and select Configure as Server Port.

12.  Click Yes to confirm server ports and click OK.

13.  Select ports 39 and 40 that are connected to the Cisco Nexus switches, right-click them, and select Configure as Uplink Port.

14.  Click Yes to confirm the uplink ports and click OK.

Acknowledge Cisco UCS Chassis

To acknowledge all Cisco UCS chassis, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane.

2.     Expand Chassis and select each chassis that is listed.

3.     Right-click each chassis and click Acknowledge Chassis.

4.     Click Yes and then click OK to complete acknowledging the chassis.

Create Pools

Create MAC Address Pools

To configure the necessary MAC address pools for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Click Pools > root.

3.     Right-click MAC Pools under the root organization.

4.     Click Create MAC Pool to create the MAC address pool.

5.     Enter MAC_Pool_A for the name of the MAC pool.

6.     Optional: Enter a description for the MAC pool.

7.     Select Sequential for the option for Assignment Order.

8.     Click Next.

9.     Click Add.

10.  Specify a starting MAC address.

11.  Specify a size for the MAC address pool that is enough to support the available blade or server resources.

12.  Click OK.

13.  Click Finish.

14.  In the confirmation message, click OK.

15.  Right-click MAC Pools under the root organization.

16.  Select Create MAC Pool to create the MAC address pool.

17.  Enter MAC_Pool_B for the name of the MAC pool.

18.  Optional: Enter a description for the MAC pool.

19.  Click Next.

20.  Click Add.

21.  Specify a starting MAC address.

22.  Specify a size for the MAC address pool that is sufficient to support the available blade or server resources.

23.  Click OK.

24.  Click Finish.

25.  In the confirmation message, click OK.

Create WWNN Pool

To configure the necessary WWNN pool for the Cisco UCS environment, follow these steps on Cisco UCS Manager:

1.     Click the SAN tab on the left.

2.     Click Pools > root.

3.     Right-click WWNN Pools under the root organization.

4.     Click Create WWNN Pool to create the WWNN pool.

5.     Enter WWNN_Pool for the name of the WWNN pool.

6.     Optional: Enter a description for the WWNN pool.

7.     Click Sequential for Assignment Order.

8.     Click Next.

9.     Click Add.

10.  Modify the From field as necessary for the UCS Environment.

11.  Specify a size of the WWNN block enough to support the available server resources.

12.  Click OK.

13.  Click Finish to create the WWNN Pool.

14.  Click OK.

Create WWPN Pools

To configure the necessary WWPN pools for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Click Pools > root.

3.     In this procedure, two WWPN pools are created, one for each switching fabric.

4.     Right-click WWPN Pools under the root organization.

5.     Click Create WWPN Pool to create the WWPN pool.

6.     Enter WWPN_Pool_A for the name of the WWPN pool.

7.     Optional: Enter a description for the WWPN pool.

8.     Select Sequential for Assignment Order.

9.     Click Next.

10.  Click Add.

11.   Specify a starting WWPN.

12.   Specify a size for the WWPN pool that is enough to support the available blade or server resources.

13.  Click OK.

14.  Click Finish.

15.  In the confirmation message, click OK.

16.  Right-click WWPN Pools under the root organization.

17.  Click Create WWPN Pool to create the WWPN pool.

18.  Enter WWPN_Pool_B for the name of the WWPN pool.

19.  Optional: Enter a description for the WWPN pool.

20.   Select Sequential for Assignment Order.

21.  Click Next.

22.  Click Add.

23.  Specify a starting WWPN.

24.  Specify a size for the WWPN address pool that is sufficient to support the available blade or server resources.

25.  Click OK.

26.  Click Finish.

27.  In the confirmation message, click OK.

Create UUID Suffix Pool

To configure the necessary universally unique identifier (UUID) suffix pool for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Click Pools > root.

3.     Right-click UUID Suffix Pools.

4.     Click Create UUID Suffix Pool.

5.     Enter UUID_Pool for the name of the UUID suffix pool.

6.     Optional: Enter a description for the UUID suffix pool.

7.     Keep the prefix at the derived option.

8.     Select Sequential for the Assignment Order.

9.     Click Next.

10.  Click Add to add a block of UUIDs.

11.  Keep the From field at the default setting.

12.  Specify a size for the UUID block that is enough to support the available blade or server resources.

13.  Click OK.

14.  Click Finish.

15.  Click OK.

Create Server Pool

To configure the necessary server pool for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Click Pools > root.

3.     Right-click Server Pools.

4.     Click Create Server Pool.

5.     Enter AMD_Pool for the name of the server pool.

6.     Optional: Enter a description for the server pool.

7.     Click Next.

8.     Select two (or more) servers to be used for the VMware management cluster and click >> to add them to AMD_Pool server pool.

9.     Click Finish.

10.  Click OK

Create IP Pool for KVM Access

To create a block of IP addresses for in band server Keyboard, Video, Mouse (KVM) access in the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Click Pools > root > IP Pools.

3.     Right-click IP Pool ext-mgmt and click Create Block of IPv4 Addresses.

4.     Enter the starting IP address of the block and the number of IP addresses required, and the subnet and gateway information.

5.     Click OK to create the block of IPs.

6.     Click OK.

Set Packages and Policies

Create Host Firmware Package

Firmware management policies allow the administrator to select the corresponding packages for a given server configuration. These policies often include packages for adapter, BIOS, board controller, FC adapters, host bus adapter (HBA) option ROM, and storage controller properties.

To create a firmware management policy for a given server configuration in the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Click Policies > root.

3.     Expand Host Firmware Packages.

4.     Click default.

5.     In the Actions pane, click Modify Package Versions.

6.     Select the version 4.1(2a)B for the Blade Package, and optionally set version 4.1(2a)C for the Rack Package.

7.     Leave Excluded Components with only Local Disk selected.

8.     Click OK to modify the host firmware package.

Create Server Pool Qualification Policy (Optional)

To create an optional server pool qualification policy for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Click Policies > root.

3.     Right-click Server Pool Policy Qualifications.

4.     Click Create Server Pool Policy Qualification.

5.     Name the policy UCS-C125-M5.

6.     Click Create Server PID Qualifications.

7.     Choose UCSC-125 from the PID drop-down list.

8.     Click OK.

9.     Optionally select additional qualifications to refine server selection parameters for the server pool.

10.  Click OK to create the policy then OK for the confirmation.

Create Server BIOS Policy

To create a server BIOS policy for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click Servers.

2.     Click Policies > root.

3.     Right-click BIOS Policies.

4.     Click Create BIOS Policy.

5.     Enter AMD-VM-Host for the BIOS policy name.

6.     Click the newly created BIOS Policy.

7.     Within the Main tab of the Policy:

a.     Change CDN Control to enabled.

b.     Change the Quiet Boot setting to disabled.

8.     Click the Advanced tab, leaving the Processor tab selected within the Advanced tab.

9.     Set the following within the Processor tab:

a.     Core Performance Boost -> Auto

b.     Global C-state Control -> Disabled

c.     L1 Stream HW Prefetcher -> Enabled

d.     L2 Stream HW Prefetcher -> Enabled

e.     Determinism Slider -> Power

f.      IOMMU -> Enabled

g.     AMD Memory Interleaving -> Auto

h.     AMD Memory Interleaving Size -> Auto

i.       SMEE -> Enabled

j.       SMT Mode -> Auto

k.     SVM Mode -> Enabled

10.  Click Save Changes.

11.  Click OK.

Update Default Maintenance Policies

To update the default Maintenance Policy, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab.

2.     Click Policies > root.

3.     Click Maintenance Policies > default.

4.     Change the Reboot Policy to User Ack.

5.     (Optional: Click “On Next Boot” to delegate maintenance windows to server owners).

6.     Click Save Changes.

7.      Click OK to accept the change.

Create Local Disk Configuration Policy (Optional)

A local disk configuration for the Cisco UCS environment is necessary if the servers in the environment do not have a local disk.

To create a local disk configuration policy, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab.

2.     Click Policies > root.

3.     Right-click Local Disk Config Policies.

4.     Click Create Local Disk Configuration Policy.

5.     Enter SAN-Boot for the local disk configuration policy name.

6.     Change the mode to No Local Storage.

7.     Click OK to create the local disk configuration policy.

8.     Click OK.

Create Power Control Policy

To create a power control policy for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab.

2.     Click Policies > root.

3.     Right-click Power Control Policies.

4.     Click Create Power Control Policy.

5.     Enter No-Power-Cap for the power control policy name.

6.     Change the power capping setting to No Cap.

7.     Click OK to create the power control policy.

8.     Click OK.

Create Network Control Policy for Cisco Discovery Protocol

To create a network control policy that enables Cisco Discovery Protocol (CDP) on virtual network ports, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Click Policies > root.

3.     Right-click Network Control Policies.

4.     Click Create Network Control Policy.

5.     Enter Enable_CDP for the policy name.

6.     For CDP, choose the Enabled option.

7.     Click OK to create the network control policy.

8.     Click OK.

Configure Cisco UCS LAN Connectivity

Create Uplink Port Channels

To configure the necessary port channels out of the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab.

2.     Under LAN > LAN Cloud, expand the Fabric A tree.

3.     Right-click Port Channels.

4.     Click Create Port Channel.

5.     Enter a unique ID for the port channel, (149 in our example to correspond with the upstream Nexus port channel).

6.     With 149 selected, enter PC-149-Nexus for the name of the port channel.

7.     Click Next.

8.     Select the following ports to be added to the port channel:

a.     Slot ID 1 and port 49

b.     Slot ID 1 and port 50

9.     Click >> to add the ports to the port channel.

10.  Click Finish to create the port channel.

11.  Click OK.

12.  In the navigation pane, under LAN > LAN Cloud, expand the fabric B tree.

13.  Right-click Port Channels.

14.  Click Create Port Channel.

15.  Enter a unique ID for the port channel, (150 in our example to correspond with the upstream Nexus port channel).

16.  With 150 selected, enter PC-150-Nexus for the name of the port channel.

17.  Click Next.

18.  Select the following ports to be added to the port channel:

a.     Slot ID 1 and port 49

b.     Slot ID 1 and port 50

19.  Click >> to add the ports to the port channel.

20.  Click Finish to create the port channel.

21.  Click OK.

Create VLANS

To configure the necessary virtual local area networks (VLANs) for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab.

2.     Click LAN > LAN Cloud.

3.     Right-click VLANs.

4.     Click Create VLANs.

5.     Enter Native-VLAN for the name of the VLAN to be used for the native VLAN.

6.     Keep the Common/Global option selected for the scope of the VLAN.

7.     Enter the native VLAN ID.

8.     Keep the Sharing Type as None.

9.     Click OK and then click OK again.

10.  Expand the list of VLANs in the navigation pane, right-click the newly created Native-VLAN and choose Set as Native VLAN.

11.  Click Yes and then click OK.

12.  Right-click VLANs.

13.  Click Create VLANs.

14.  Enter IB-Mgmt for the name of the VLAN to be used for management traffic.

15.  Keep the Common/Global option selected for the scope of the VLAN.

16.  Enter the In-Band management VLAN ID.

17.  Keep the Sharing Type as None.

18.  Click OK and then click OK again.

19.  Right-click VLANs.

20.  Click Create VLANs.

21.  Enter vMotion for the name of the VLAN to be used for vMotion.

22.  Keep the Common/Global option selected for the scope of the VLAN.

23.  Enter the vMotion VLAN ID.

24.  Keep the Sharing Type as None.

25.  Click OK and then click OK again.

26.  Right-click VLANs.

27.  Click Create VLANs.

28.  Enter VM-App- for the prefix of the VLANs to be used for VM Traffic.

29.  Keep the Common/Global option selected for the scope of the VLAN.

30.  Enter the VM-Traffic VLAN ID range.

31.  Click OK and then click OK again.

Create vNIC Templates

To create the multiple virtual network interface card (vNIC) templates for the Cisco UCS environment, follow these steps:

Create Management vNICs

To create the vNIC_Mgmt_A Template, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Click Policies > root.

3.     Right-click vNIC Templates.

4.     Click Create vNIC Template.

5.     Enter vNIC_Mgmt_A for the vNIC template name.

6.     Keep Fabric A selected.

7.     Click Primary Template for the Redundancy Type.

8.     Leave Peer Redundancy Template as <not set>

9.     Under Target, make sure that the VM checkbox is not selected.

10.  Select Updating Template for the Template Type.

11.  Under VLANs, check the boxes for IB-Mgmt and Native-VLAN VLANs.

12.  Set Native-VLAN for the native VLAN.

13.  Leave vNIC Name selected for the CDN Source.

14.  Leave 1500 for the MTU.

15.  In the MAC Pool list, choose MAC_Pool_A.

16.  In the Network Control Policy list, choose Enable_CDP.

17.  Click OK to create the vNIC template.

18.  Click OK.

For the vNIC_Mgmt_B Template, follow these steps:

1.     In the navigation pane, click the LAN tab.

2.     Click Policies > root.

3.     Right-click vNIC Templates.

4.     Click Create vNIC Template.

5.     Enter vNIC_Mgmt_B for the vNIC template name.

6.     Click Fabric B.

7.     Click Secondary Template for Redundancy Type.

8.     For the Peer Redundancy Template drop-down, click vNIC_Mgmt_A.

9.     Under Target, make sure the VM checkbox is not selected.

10.  In the MAC Pool list, click MAC_Pool_B.

11.  Click OK to create the vNIC template.

12.  Click OK.

Create Data vNICs

To create the vNIC_Data_A Template, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab.

2.     Click Policies > root.

3.      Right-click vNIC Templates.

4.     Click Create vNIC Template.

5.     Enter vNIC_Data_A for the vNIC template name.

6.     Keep Fabric A selected.

7.     Select Primary Template for the Redundancy Type.

8.     Leave Peer Redundancy Template as <not set>.

9.     Under Target, make sure that the VM checkbox is not selected.

10.  Select Updating Template for the Template Type.

11.  Under VLANs, check the boxes vMotion, Apps, and Native-VLAN.

12.  Set Native-VLAN for the native VLAN.

13.  For MTU, enter 9000.

14.  In the MAC Pool list, click MAC_Pool_A.

15.  In the Network Control Policy list, choose Enable_CDP.

16.  Click OK to create the vNIC template.

17.  Click OK.

For the vNIC_Data_B Template, follow these steps:

1.     In the navigation pane, click the LAN tab.

2.     Click Policies > root.

3.     Right-click vNIC Templates.

4.     Click Create vNIC Template.

5.     Enter vNIC_Data_B for the vNIC template name.

6.     Click Fabric B.

7.     Select Secondary Template for Redundancy Type.

8.     For the Peer Redundancy Template drop-down, click vNIC_Data_A.

9.     Under Target, make sure the VM checkbox is not selected.

10.  In the MAC Pool list, click MAC_Pool_B.

11.  Click OK to create the vNIC template.

12.  Click OK.

Create LAN Connectivity Policy

To configure the necessary FC Infrastructure LAN Connectivity Policy, follow these steps:

13.  In Cisco UCS Manager, click LAN.

14.  Click LAN > Policies > root.

15.  Right-click LAN Connectivity Policies.

16.  Click Create LAN Connectivity Policy.

17.  Enter FC-LAN-Policy for the name of the policy.

18.  Click the upper Add button to add a vNIC.

19.  In the Create vNIC dialog box, enter 00-Mgmt-A for the name of the vNIC.

20.  Check the box for Use vNIC Template.

21.  In the vNIC Template list, choose 00-Mgmt-A.

22.  In the Adapter Policy list, click VMWare.

23.  Click OK to add this vNIC to the policy.

24.  Click the upper Add button to add another vNIC to the policy.

25.  In the Create vNIC box, enter 01-Mgmt-B for the name of the vNIC.

26.  Check the box for Use vNIC Template.

27.  In the vNIC Template list, click 01-Mgmt-B.

28.  In the Adapter Policy list, click VMWare.

29.  Click OK to add the vNIC to the policy.

30.  Click the upper Add button to add a vNIC.

31.  In the Create vNIC dialog box, enter 02-Data-A for the name of the vNIC.

32.  Check the box for Use vNIC Template.

33.  In the vNIC Template list, click vNIC_Data_A.

34.  In the Adapter Policy list, click VMWare.

35.  Click OK to add this vNIC to the policy.

36.  Click Add to add a vNIC to the policy.

37.  In the Create vNIC dialog box, enter 03-Data-B for the name of the vNIC.

38.  Check the box for Use vNIC Template.

39.  In the vNIC Template list, click vNIC_Data_B.

40.  In the Adapter Policy list, click VMWare.

41.  Click OK to add this vNIC to the policy.

42.  Click OK again to create the LAN Connectivity Policy.

Configure FC SAN Connectivity

This section details the Fibre Channel configuration steps that will enable you to provision the volumes for FlashStack that are used as datastores by the FlashStack vSphere hosts, and the creation of UCS Service Profiles that will be configured to boot from Fibre Channel LUNs.

Configure Unified Ports

The Cisco UCS 6454 Fabric Interconnects will have a slider mechanism within the Cisco UCS Manager GUI interface that will control the first 8 ports starting from the first port and configured in increments of the first 4 or 8 of the unified ports. 

To enable the fibre channel ports, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab.

2.     Click Equipment > Fabric Interconnects > Fabric Interconnect A (primary).

3.     Click Configure Unified Ports.

4.     Click Yes on the pop-up window warning that changes to the fixed module will require a reboot of the fabric interconnect and changes to the expansion module will require a reboot of that module.

5.     Within the Configured Fixed Ports pop-up window move the gray slider bar from the left to the right to choose either 4 or 8 ports to be set as FC Uplinks.

6.     Click OK to continue.

7.     Click Equipment > Fabric Interconnects > Fabric Interconnect B (primary).

8.     Click Configure Unified Ports.

9.     Click Yes on the pop-up window warning that changes to the fixed module will require a reboot of the fabric interconnect and changes to the expansion module will require a reboot of that module.

10.  Within the Configured Fixed Ports pop-up window move the gray slider bar from the left to the right to choose either 4 or 8 ports to be set as FC Uplinks.

11.  Click OK to continue.

Create VSANs

To configure the necessary virtual storage area networks (VSANs) for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Click SAN > SAN Cloud.

3.     Right-click VSANs.

4.     Click Create VSAN.

5.     Enter VSAN_A for the name of the VSAN to be used for Fabric A.

6.     Leave Disabled selected for FC Zoning.

7.     Click Fabric A.

8.     Enter a unique VSAN ID and a corresponding FCoE VLAN ID. It is recommended use the same ID for both parameters and to use something other than 1.

9.     Click OK and then click OK again.

10.  Under SAN Cloud, right-click VSANs.

11.  Click Create VSAN.

12.  Enter VSAN_B for the name of the VSAN to be used for Fabric B.

13.  Leave Disabled selected for FC Zoning.

14.  Click Fabric B.

15.  Enter a unique VSAN ID and a corresponding FCoE VLAN ID. It is recommended use the same ID for both parameters and to use something other than 1.

16.  Click OK and then click OK again.

Create FC Port Channels

To configure the necessary port channels for the Cisco UCS environment, follow these steps:

Fabric A

1.     In the navigation pane under SAN > SAN Cloud expand the Fabric A tree.

2.     Right-click FC Port Channels.

3.     Click Create FC Port Channel.

4.     Enter 1 for the ID and SAN-Po1 for the Port Channel name.

5.     Click Next then select the appropriate ports and click >> to add the ports to the port channel.

6.     Click Finish.

7.     Click OK.

8.     Click the newly created Port-Channel.

9.     From the VSAN drop-down list for Port-Channel SAN-Po1, choose VSAN_A(100).

10.  Click Save Changes and then click OK.

Fabric B

1.     In the navigation pane under SAN > SAN Cloud expand the Fabric B tree.

2.     Right-click FC Port Channels.

3.     Click Create Port Channel.

4.     Enter 2 for the ID and SAN-Po2 for the Port Channel name.

5.     Click Next then select the appropriate ports and click >> to add the ports to the port channel.

6.     Click Finish.

7.     Click OK.

8.     Click the newly created Port-Channel.

9.     Under the VSAN drop-down for Port-Channel SAN-Po2, choose VSAN_B(200).

10.  Click Save Changes and then click OK.

Create vHBA Templates

To create the necessary virtual host bus adapter (vHBA) templates for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab.

2.     Click Policies > root.

3.     Right-click vHBA Templates.

4.     Click Create vHBA Template.

5.     Enter vHBA_Template_A for the vHBA template name.

6.     Keep Fabric A selected.

7.     Leave Redundancy Type as No Redundancy.

8.     Click VSAN_A.

9.     Leave Initial Template for the Template Type.

10.  Click WWPN_Pool_A for the WWPN Pool.

11.  Click OK to create the vHBA template.

12.  Click OK.

13.  Right-click vHBA Templates.

14.  Click Create vHBA Template.

15.  Enter vHBA_Template_B for the vHBA template name.

16.  Click Fabric B.

17.  Leave Redundancy Type as No Redundancy.

18.  Click VSAN_B.

19.  Leave Initial Template for the Template Type.

20.  Click WWPN_Pool_B for the WWPN Pool.

21.  Click OK to create the vHBA template.

22.  Click OK.

Create SAN Connectivity Policy

To configure the necessary Infrastructure SAN Connectivity Policy, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Click SAN > Policies > root.

3.     Right-click SAN Connectivity Policies.

4.     Click Create SAN Connectivity Policy.

5.     Enter Infra_SAN_Policy for the name of the policy.

6.     Select the previously created WWNN_Pool for the WWNN Assignment.

7.     Click Add to add a vHBA.

8.     In the Create vHBA dialog box, enter Fabric_A for the name of the vHBA.

9.     Check the box for Use vHBA Template.

10.  Leave Redundancy Pair unselected.

11.  In the vHBA Template list, choose vHBA_Template_A.

12.  In the Adapter Policy list, choose VMWare.

13.  Click OK.

14.  Click Add to add a second vHBA.

15.  In the Create vHBA dialog box, enter Fabric_B for the name of the vHBA.

16.  Check the box for Use vHBA Template.

17.  Leave Redundancy Pair unselected.

18.  In the vHBA Template list, choose vHBA_Template_B.

19.   In the Adapter Policy list, choose VMWare.

20.  Click OK.

21.  Click OK to create the SAN Connectivity Policy.

22.  Click OK to confirm creation.

Create Boot Policy

This procedure will define the Primary and Secondary Boot Targets for each Fabric side (A/B). These are the WWNs that need to be collected from the first adapter of each controller on the Pure Storage FlashArray that are visible from the Connections tab under the Health  section of the FlashArray Web GUI.

As an alternative to the GUI, connect to the FlashArray//X via ssh using the pureuser account and find the WWNs using the pureport list command:

Find the FC0 adapters for each controller from within the System view and record the values to be used for Primary and Secondary Targets. In the example lab environment listed in the table below, these appear as the first ports of each controller.

Within the same System view, find the FC1 adapters for each controller and record the values to be used for Primary and Secondary Targets. In the example lab environment, these appear as the second ports of each controller shown.

To create boot policies for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab.

2.     Click Policies > root.

3.     Right-click Boot Policies.

4.     Click Create Boot Policy.

5.     Enter Boot-FC-A for the name of the boot policy.

6.     Optional: Enter a description for the boot policy.

7.     Set boot Mode to Uefi.

8.     Expand the Local Devices drop-down list and choose Add Remote CD/DVD.

9.     Expand the vHBAs drop-down list and choose Add SAN Boot.

10.  In the Add SAN Boot dialog box, enter Fabric-A in the vHBA field.

11.  Confirm that Primary is selected for the Type option.

12.  Click OK to add the SAN boot initiator.

13.  From the vHBA drop-down list, choose Add SAN Boot Target.

14.  Enter 1 for the value for Boot Target LUN.

15.  Enter the WWPN for CT0.FC0. 

16.  Choose Primary for the SAN boot target type.

17.  Click OK to add the SAN boot target.

18.  From the vHBA drop-down list, choose Add SAN Boot Target.

19.  Enter 1 for the value for Boot Target LUN.

20.  Enter the WWPN for CT1.FC0.

21.  Click OK to add the SAN boot target.

22.  From the vHBA drop-down list, choose Add SAN Boot.

23.  In the Add SAN Boot dialog box, enter Fabric_B in the vHBA box.

24.  Click OK to add the SAN boot initiator.

25.  From the vHBA drop-down list, choose Add SAN Boot Target.

26.  Enter 1 for the value for Boot Target LUN.

27.  Enter the WWPN for CT0.FC1.

28.  Choose Primary for the SAN boot target type.

29.  Click OK to add the SAN boot target.

30.  From the vHBA drop-down list, select Add SAN Boot Target.

31.  Enter 1 for the value for Boot Target LUN.

32.  Enter the WWPN for CT1.FC1. 

33.  Click OK to add the SAN boot target.

34.  Expand CIMC Mounted Media and click Add CIMC Mounted CD/DVD.

35.  Click OK, then click OK again to create the boot policy.

Create Service Profile Templates

In this procedure, one service profile template for Infrastructure ESXi hosts is created for FC boot.

To create the service profile template, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab.

2.     Click Service Profile Templates > root.

3.     Right-click root.

4.     Click Create Service Profile Template to open the Create Service Profile Template wizard.

5.     Enter VM-AMD-Host-FC-A for the name of the service profile template. This service profile template is configured to boot from FlashArray//X50 R3 controller 1 on fabric A.

6.     Click Updating Template.

7.     Under UUID, select UUID_Pool for the UUID pool.

8.     Click Next.

Configure Storage Provisioning

To configure the storage provisioning, follow these steps:

1.     Click the Local Disk Configuration Policy tab.

2.     If you have servers with no physical disks, click the Local Disk Configuration Policy tab and select the SAN-Boot Local Storage Policy. Otherwise, select the default Local Storage Policy.

3.     Click Next.

Configure Networking Options

To configure the network options, follow these steps:

1.     Keep the default setting for Dynamic vNIC Connection Policy.

2.     Choose the “Use Connectivity Policy” option to configure the LAN connectivity.

3.     Click FC-LAN-Policy from the LAN Connectivity Policy drop-down list.

4.     Click Next.

Configure SAN Connectivity Options

To configure the SAN connectivity options, follow these steps:

1.     Choose the Use Connectivity Policy option for the “How would you like to configure SAN connectivity?” field.

2.     Choose the Infra_SAN_Policy option from the SAN Connectivity Policy drop-down list.

Configure Zoning Options

1.     Leave Zoning configuration unspecified and click Next.

Configure vNIC/HBA Placement

1.     From the Select Placement drop-down list, leave the placement policy as “Let System Perform Placement.”

2.     Click Next.

Configure vMedia Policy

1.     Do not select a vMedia Policy.

2.     Click Next.

Configure Server Boot Order

1.     Click Boot-FC-A for Boot Policy.

2.     Click Next to continue to the next section.

Configure Maintenance Policy

1.     Change the Maintenance Policy to default.

2.     Click Next.

Configure Server Assignment

To configure server assignment, follow these steps:

1.     In the Pool Assignment list, select AMD_Pool

2.     Optional: Select a Server Pool Qualification policy.

3.     Select Down for the power state to be applied when the profile is associated with the server.

4.     Optional: Select “UCS-C125-M5” for the Server Pool Qualification.

5.     Firmware Management at the bottom of the page can be left alone as it will use default from the Host Firmware list.

6.     Click Next.

Configure Operational Policies

To configure the operational policies, follow these steps:

1.     In the BIOS Policy list, select AMD-VM-Host.

2.     Expand Power Control Policy Configuration and select No-Power-Cap in the Power Control Policy list.

3.     Click Finish to create the service profile template.

4.     Click OK in the confirmation message.

Create Service Profiles

To create service profiles from the service profile template, follow these steps:

1.     Connect to the UCS 6454 Fabric Interconnect UCS Manager, click the Servers tab in the navigation pane.

2.     Click Service Profile Templates > root > Service Template VM-Host-FC-A.

3.     Right-click VM-AMD-Host-FC-A and select Create Service Profiles from Template.

4.     Enter VM-AMD-Host-FC- for the service profile prefix.

5.     Leave 1 as “Name Suffix Starting Number.”

6.     Leave 2 for the “Number of Instances.”

7.     Click OK to create the service profiles.

8.     Click OK in the confirmation message to provision two FlashStack Service Profiles.

Claim UCS Domain in Intersight

To claim a UCS domain in Intersight, follow these steps:

1.     Connect to the UCS 6454 Fabric Interconnect UCS Manager, click the Admin tab.

2.     Click Device Connector.

3.     Set Intersight Management to Enabled.

4.     Copy the Device ID and Claim Code.

5.     Open a browser to Cisco Intersight, https://intersight.com and log into your Intersight account.

6.     Click Admin > Devices.

7.     Click Claim a New Device and enter your Device ID and Claim Code under the Direct Claim option.

8.     Click Claim.

MDS Fabric Zoning

This section continues the configuration process of the Cisco MDS 9148S Multilayer Fabric Switches now that resources are attached, to provide zoning for supported devices.

Create Device Aliases

To create device aliases, follow these steps:

1.     Gather the WWPN of the FlashArray adapters using the show flogi database command on each switch and create a spreadsheet to reference when creating device aliases on each MDS. For MDS 9132T-A this will be:

2.     Match these values to their sources from the Purity command line output gained from a ssh connection to the FlashArray using the pureuser account:

3.     Match these values to the UCS Service Profile vHBA listing for each host found within Servers -> Service Profiles -> <Service Profile of Source Host> -> Storage -> vHBAs:

4.     Record the values to be used for zoning and host mapping:

5.    Create device alias database entries for each of the PWWNs mapping them to their human readable source names on MDS-9132T-A:

6.     Gather the WWPN of the FlashArray adapters using the show flogi database command on each switch and create a spreadsheet to reference when creating device aliases on each MDS. For MDS 9132T-B this will be:

7.     Record the values to be used for zoning and host mapping:

8.     Create device alias database entries for each of the PWWNs mapping them to their human readable source names on MDS-9132T-B:

MDS Zoning

Create zones for each host using the device aliases created in the previous step, specifying init and target roles to optimize zone traffic:

Zone for UCS VM-AMD-Host-FC-01 on MDS A

Zone for UCS VM-AMD-Host-FC-02 on MDS A

Zone for UCS VM-AMD-Host-FC-01 on MDS B

Zone for UCS VM-AMD-Host-FC-02 on MDS B

Create and Activate Zoneset

Add the zones to a zoneset on each MDS switch:

Zoneset for MDS A

Zoneset for MDS B

FlashArray Storage Deployment

The Pure Storage FlashArray//X is accessible to the FlashStack.

The storage to be deployed will include:

●      ESXi FC Boot LUNs

●      VMFS Datastores

●      vVol Data Stores

The FC Boot LUNs will need to be setup from the Pure Storage Web Portal, and the VMFS datastores will be directly provisioned from the vSphere Web Client after the Pure Storage vSphere Web Client Plugin has later been registered with the vCenter.

Host Port Identification

FC Boot LUNs will be mapped by the FlashArray//X using the assigned Initiator PWWN to the provisioned service profiles. This information can be found within the service profile.

Host Registration

To create a host, follow these steps:

1.     Selection Configure -> Orchestration.

2.     Click New Storage Host.

3.     Click Execute.

4.     Click the appropriate Organization (default by default).

5.     Click the appropriate Pure Storage device.

6.     Enter the name of the Host name and WWNs for host VM-AMD-Host-FC-01.

7.     Click Execute.

8.     Repeat steps 2-7 for all host.

Create Host Group

To create a host group, follow these steps:

1.     Selection Configure -> Orchestration.

2.     Click New Storage Host Group.

3.     Click Execute.

4.     Select the appropriate Organization (default by default).

5.     Select the appropriate Pure Storage device.

6.     Enter the name of the Host Group and of the Hosts created during Host Registration. VM-AMD-Host-FC-01 and VM-AMD-Host-FC-02 are the host used in this deployment.

7.     Click Execute.

Private Boot Volumes for each ESXi Host

To create private boot volumes for each ESXi Host, follow these steps in the Pure Storage Web Portal:

1.     Click Storage > Volumes.

2.     Click the + icon in the Volumes Panel.

A pop-up will appear to create a volume on the FlashArray.

3.     To create more than one volume, click the Create Multiple… option, filling in the Name, Provisioned Size, Staring Number, Count, and Number of Digits, with a “#” appearing in the name where an iterating number will appear.

4.     Click Create to provision the volumes to be used as FC boot LUNs.

5.     Go back to the Hosts section under the Storage tab. Click one of the hosts and select the gear icon drop-down list within the Connected Volumes tab within that host.

6.     From the gear icon drop-down list, select Connect Volumes.

7.     Select the volume that has been provisioned for the host, set the LUN ID for the volume, click the + next to the volume, and click Confirm to proceed. Repeat steps 1-7 to connecting volumes for each of the host/volume pairs configured.

vSphere Deployment

ESXi Installation

This section provides detailed instructions to install VMware ESXi 7.0 in a FlashStack environment. After the process is completed, the FC SAN booted ESXi hosts will be configured

Several methods exist for installing ESXi in a VMware environment. These procedures focus on how to use the built-in keyboard, video, mouse (KVM) console and virtual media features in Cisco UCS Manager to map remote installation media to individual servers and connect to their boot logical unit numbers (LUNs).

Download Cisco Custom Image for ESXi 7.0

The VMware Cisco Custom Image will be needed for use during installation by manual access to the Cisco UCS KVM vMedia, or through a vMedia. If the Cisco Custom Image was not downloaded earlier, download it by following these steps:

1.     Click the following link: Cisco Custom ISO for UCS 4.1.2a

2.     Download the .iso file.

Log into Cisco UCS 6454 Fabric Interconnect

The IP KVM enables the administrator to begin the installation of the operating system (OS) through remote media. It is necessary to log in to the Cisco UCS environment to run the IP KVM.

To log into the Cisco UCS environment, follow these steps:

1.     Open a web browser to https:// <<var_ucs_mgmt_vip>>

2.     Choose the Launch UCS Manager Section in the HTML section to pull up the UCSM HTML5 GUI.

3.     Enter admin for the Username, and provide the password used during setup.

4.     Within the UCSM click Servers -> Service Profiles and pick the first host provisioned as VM-Host-FC-01.

5.     Click the KVM Console option within Actions and accept the KVM server certificate in the new window or browser tab that is spawned for the KVM session.

6.     Click the link within the new window or browser tab to load the KVM client application.

Set Up VMware ESXi Installation

To prepare the server for the OS installation, follow these steps on each ESXi host:

1.     In the KVM window, click Virtual Media icon .

2.     Click Activate Virtual Devices.

3.     Click Virtual Media again and click Map CD/DVD.

4.     Browse to the ESXi installer ISO image file and click Open.

5.     Click Map Device.

6.     Click the KVM tab to monitor the server boot.

7.     Boot the server by selecting Boot Server and clicking OK, then click OK again.

Install ESXi

To install VMware ESXi to the FC bootable LUN of the hosts, follow these steps on each host:

1.     On reboot, the machine detects the presence of the ESXi installation media. Choose the ESXi installer from the boot menu that is displayed.

2.     After the installer is finished loading, press Enter to continue with the installation.

3.     Read and accept the end-user license agreement (EULA). Press F11 to accept and continue.

4.     Click the LUN that was previously set up as the installation disk for ESXi and press Enter to continue with the installation.

5.     Click the appropriate keyboard layout and press Enter.

6.     Enter and confirm the root password and press Enter.

7.     The installer issues a warning that the selected disk will be repartitioned. Press F11 to continue with the installation.

8.     From the KVM window, press Enter to reboot the server.

Set Up Management Networking for ESXi Hosts

Adding a management network to each VMware host is necessary for managing the host. To add a management network for the VMware hosts, follow these steps on each ESXi host:

1.     After the server has finished rebooting, press F2 to customize the system.

2.     Log in as root, enter the corresponding password, and press Enter.

3.     Click Troubleshooting Options.

4.     Enable ESXi shell.

5.     Enable SSH.

6.     Hit Esc to exit.

7.     Click the Configure the Management Network option and press Enter.

8.     Click the Network Adapters option and leave vmnic0 selected, then arrow down to vmnic1 and press space to select vmnic1 as well and press Enter.

9.     Click the VLAN (Optional) option and press Enter.

10.  Enter the <<var_ib_mgmt_vlan_id>> and press Enter.

11.  From the Configure Management Network menu, select IPv4 Configuration and press Enter.

12.  Choose the Set Static IP Address and Network Configuration option by using the space bar.

13.  Enter <<var_vm_host_FC_01_ip>> for the IPv4 Address for managing the first ESXi host.

14.  Enter <<var_ib_mgmt_vlan_netmask_length>> for the Subnet Mask for the first ESXi host.

15.  Enter <<var_ib_mgmt_gateway>> for the Default Gateway for the first ESXi host.

16.  Press Enter to accept the changes to the IPv4 configuration.

17.  Choose the DNS Configuration option and press Enter.

18.  Enter the IP address of <<var_nameserver_ip>> for the Primary DNS Server.

19.  Optional: Enter the IP address of the Secondary DNS Server.

20.  Enter the fully qualified domain name (FQDN) for the first ESXi host.

21.  Press Enter to accept the changes to the DNS configuration.

22.  Press Esc to exit the Configure Management Network submenu.

23.  Press Y to confirm the changes and return to the main menu.

24.  The ESXi host reboots. After reboot, press F2 and log back in as root.

25.  Choose Test Management Network to verify that the management network is set up correctly and press Enter.

26.  Press Enter to run the test.

27.  Press Enter to exit the window, and press Esc to log out of the VMware console.

28.  Repeat steps in Set Up VMware ESXi Installation, Install ESXi, and Set UP Management Networking for ESXi Host for additional hosts provisioned, using appropriate values.

Create FlashStack Datacenter

If a new datacenter is needed for FlashStack, follow these steps on the vCenter:

1.     Connect to the vSphere Web Client and click Hosts and Clusters or click the Hosts and Clusters icon from the Home center window.

2.     Right-click the vCenter icon and select New Datacenter… from the drop-down list.

3.     Enter a Datacenter name and click OK.

Create VMware vDS for vMotion and Application Traffic

The VMware vDS setup consists of one vDS that are vMotion and Application traffic. 

FlashStack vDS

To configure the VMware vDS, follow these steps:

1.     Connect to the vSphere Web Client and click Networking or click the Networking icon from the Home center window.

2.     Right-click the FlashStack-VSI datacenter and click Distributed Switch > New Distributed Switch…

3.     Provide the Distributed Switch a descriptive name and click Next.

4.     Make sure Distributed switch: 7.0.0 is selected and click Next.

5.     Change the number of uplinks to 2. If VMware Network I/O Control is to be used for Quality of Service, leave Network I/O Control Enabled. Otherwise, Disable Network I/O Control. Enter VM-Traffic for the name of the default Port group to be created. Click Next.

6.     Review the information and click Finish to complete creating the vDS.

7.     Right-click the newly created vDS and click Settings -> Edit Settings…

8.     Change the MTU to 9000. The Discovery Protocol can optionally be changed to Link Layer Discovery Protocol and the Operation to Both. Click OK.

9.     Expand the FlashStack VSI datacenter and the newly created vDS. 

10.  Right-click the VM-Traffic Distributed Port Group, and click Edit Settings…

11.  Click VLAN, change the VLAN type from None to VLAN, and enter in the appropriate VLAN number for the VM-Traffic network.

12.  Click OK to save the changes.

13.  Right-click and select Distributed Port Group -> New Distributed Port Group…

14.  Name the new Port Group vMotion and click Next.

15.  Change the VLAN type from None to VLAN, select the VLAN ID appropriate for your vMotion traffic, and select the Customize default policies configuration check box under the Advanced section.

16.  Click Next.

17.  Click Next through the Security and Traffic Shaping sections.

18.  Within the Teaming and failover section move Uplink 1 to the Standby uplinks section.

19.  Click Next.

20.  Click Next Past Monitoring, Miscellaneous, and Edit additional settings sections.

21.  Review the Ready to complete section.

22.  Click Finish to create the Distributed Port Group.

Add the VMware ESXi Hosts Using the VMware vSphere Web Client

To add the VMware ESXi Hosts using the VMware vSphere Web Client, follow these steps:

1.     From the Hosts and Clusters tab, right-click the new or existing Datacenter within the Navigation window and select New Cluster… from the drop-down list.

2.     Enter a name for the new cluster, enable vSphere DRS and vSphere HA, leaving all other options with defaults.

3.     Click OK to create the cluster.

4.     Right-click the newly created cluster and from the drop-down list select the Add Host.

5.     Enter the IP or FQDN, User Name, and password of the ESXi hosts and click Next.

6.     Click Yes in the Security Alert pop-up to confirm the host’s certificate.

7.     Click Next past the Host summary dialogue.

8.     Provide a license by clicking the green + icon under the License title, select an existing license, or skip past the Assign license dialogue by clicking Next.

9.     Leave lockdown mode Disabled within the Lockdown mode dialogue window and click Next.

10.  Skip past the Resource pool dialogue by clicking Next.

11.  Click Next to confirm the Summary dialogue and add the ESXi host to the cluster.

Claim vCenter in Intersight

To claim vCenter in Intersight, follow these steps:

1.     Open a browser and go to Cisco Intersight https://intersight.com and log into your Intersight account

2.     Click Admin > Devices.

3.     Click Claim a New Device and select Claim Though Intersight Assist.

4.     Set Type to VMware vCenter.

5.     Enter vCenter Hostname/ IP address and credentials.

6.     Click Claim.

Create VMFS Swap Datastore             

To create a vmfs 6 datastore to place swap using Intersight Orchestration, follow these steps:

1.     Selection Configure -> Orchestration.

2.     Click New VMFS Datastore.

3.     Click Execute.

4.     Select the appropriate Organization (default by default).

5.     Select the appropriate Pure Storage device.

6.     Enter the name of the Volume that will appear on the Pure Storage Array: ESXi-Swap.

7.     Enter Volume size and units.

8.     Choose Storage Host Group: VM-AMD-Host-Group.

9.     Choose an unused LUN Number.

10.  Choose the appropriate Hypervisor Manager and Datacenter.

11.  Enter the name of the Datastore that will appear in vSphere.

12.  Choose VMFS Version VMFS-6.

13.  Click Execute

Pure Storage vSphere Client Plugin

The Pure Storage vSphere Client Plugin will be accessible through the vSphere Client after registration through the Pure Storage Web Portal.

To access the Pure Storage vSphere Client Plugin, follow these steps:

1.     Go to Settings > Software.

2.     Click the edit icon in the vSphere Plugin panel.

3.     Enter the vCenter information and click Save.

4.     After the discovery completes. Click install.

5.     In vCenter, click Pure Storage.

6.     Click Authenticate with Pure1.

7.     Input your Pure1 JWT (link).

8.     Click Authenticate.

9.     Click Add.

10.  Click Import Arrays from Pure1 and input the Username and Password.

11.  Click Done

12.  Select the newly added array.

13.  Click Register Storage Provider.

14.  Enter Username and Password.

15.  Click Register.

Create vVol Datastore

To create a vVol datastore, follow these steps:

1.     In vCenter, click Host and Clusters.

2.     Right-click the FlashStack Cluster and click Pure Storage -> Create Datastore.

3.     Click vVol.

4.     Click Next.

5.     Enter a Datastore Name.

6.     Click Next.

7.     Choose the Cluster under Compute Resources.

8.     Click Next.

9.     Click the Registered FlashArray.

10.  Click Next.

11.  Review the information and click Finish.

Configure ESXi Settings

A couple of base settings are needed for stability of the vSphere environment, as well as an optional enablement of SSH connectivity to each host for the updating of drivers.

To configure ESXi settings, follow these steps:

1.     Select the first ESXi host to configure with standard settings.

2.     Click the Configure tab and select Time Configuration within the options and click Edit for Network Time Protocol.

3.     Check Enable, enter <<var_oob_ntp>> for the NTP Servers, select Start and stop with port usage for NTP Service Startup Policy, and click Start within NTP Service Status. Click OK to submit the changes.

4.     Click System Swap in the System section within the Configure tab and click Edit.

5.     Select Can use datastore specified by host for swap files.

6.     Repeat steps 1-5 on each ESXi host being added into the cluster.

Configure vSwitch0

To configure vSwitch0, follow these steps:

1.     From the Hosts and Clusters, choose the first host and click the Configure tab for that host.

2.     Click Virtual Switches under the Networking section

3.     Click Manage Physical Adapters for Standard Switch: vSwitch0

4.     Click the green + sign under Assigned adapters.

5.     Click vmnic1 and click OK.

6.     Confirmed that vnmic0 and vmnic1 are listed as Active adapters and click OK.

Add the ESXi hosts to the vDS

To Add the ESXi Hosts to each vDS, follow these steps:

1.     From the Networking tab of the Navigator window, right-click the FlashStack-vDS vDS and click Add and Manage Hosts…

2.     Leave Add hosts selected and click Next.

3.     Click the green + icon next to New hosts…

4.     In the Select new hosts pop-up that appears, select the hosts to be added, and click OK to begin joining them to the vDS. 

5.     Click Next.

6.     Choose vmnic2 on each host and click Assign uplink. Choose Uplink 1 and click OK. Choose vmnic3 on each host and click Assign uplink. Choose Uplink 2 and click OK. If more than one host is being connected to the vDS, use the Apply this uplink assignment to the rest of the hosts checkbox.

7.     Click Next.

8.     Do not migrate any VMkernel ports and click Next.

9.     Do not migrate any VM ports and click Next.

10.  Click Finish to complete adding the ESXi host(s) to the vDS

Create vMotion VMkernel adapters

A vMotion VMkernel adapter will be created for FlashStack infrastructure to keep vMotion traffic independent of management traffic. To create the vMotion VMkernel adapters, follow these steps:

1.     From the Hosts and Clusters, choose the first host and click the Configure tab for that host.

2.     Choose the VMkernel adapters option within the Networking section of Configure.

3.     Click the first icon under VMkernel adapters to Add host networking.

4.     Leave the connection type selected as VMkernel Network Adapter and click Next.

5.     Choose Browse with Select an existing network.

6.     Choose the vMotion network and click OK.

7.     Click Next.

8.     Choose the vMotion from the Available services and click Next.

9.     Provide an IP address and subnet mask within the vMotion network.

10.  Click Next.

11.  Review the settings and click Finish to create the VMkernel adapter.

12.  Optionally, create two more vMotion VMkernel NICs to increase the speed of multiple simultaneous vMotions on this solution’s 50GE vNICs.

13.  Repeat steps 1-12 to create vMotion VMkernel adapters for each additional ESXi host.

Appendix

FlashStack with Cisco Nexus 93180YC-FX SAN Switching Configuration

FlashStack Cisco Nexus Switch Base Configuration

The following procedures describe how to configure the Cisco Nexus switches for use in a base FlashStack environment using the Cisco Nexus 93180YC-FX switches for both LAN and SAN switching. This procedure assumes you’re using Cisco Nexus 9000 9.3(5), which is the Cisco suggested Nexus switch release at the time of this validation.

Set Up Initial Configuration in Cisco Nexus 93180YC-FX A

To set up the initial configuration for the Cisco Nexus A switch on <nexus-A-hostname>, follow these steps:

1.     Configure the switch.

2.     Review the configuration summary before enabling the configuration.

Set Up Initial Configuration in Cisco Nexus 93180YC-FX B

To set up the initial configuration for the Cisco Nexus B switch on <nexus-B-hostname>, follow these steps:

1.     Configure the switch.

2.     Review the configuration summary before enabling the configuration.

Enable Features in Cisco Nexus 93180YC-FX A and B

SAN switching requires both the SAN_ENTERPRISE_PKG and FC_PORT_ACTIVATION_PKG licenses. Please ensure these licenses are installed on each Cisco Nexus 93180YC-FX switch. To enable the appropriate features on the Cisco Nexus switches, follow these steps:

1.     Log in as admin.

2.     Because basic FC configurations were entered in the setup script, feature-set fcoe has been automatically installed and enabled. Run the following commands:

Perform TCAM Carving and Configure Unified Ports in Cisco Nexus 93180YC-FX A and B

SAN switching requires TCAM carving for lossless fibre channel no-drop support. Also, unified ports need to be converted to fc ports. To perform TCAM carving on the Cisco Nexus switches and to convert ports 1-16 to fc, follow these steps:

1.     Run the following commands:

2.     After the switch reboots, log back in as admin. Run the following commands:

Set System-Wide QoS Configurations in Cisco Nexus 93180YC-FX A and B

To set system-wide Qos configurations for FCoE for no-drop traffic support, follow this step on both switches:

1.     Run the following commands to set global configurations:

Configure VLANs

To configure VLANs for this deployment, run the following commands on both Cisco Nexus Switches:

Continue adding VLANs as appropriate for the environment.

Add Interface Port Descriptions for Cisco Nexus 93180YC-FX-A

To configure port descriptions for this deployment, run the following commands on both Cisco Nexus Switches:

Add Interface Port Descriptions for Cisco Nexus 93180YC-FX-B

To configure port descriptions for this deployment, run the following commands on both Cisco Nexus Switches:

Port Channel Setup

Configure vPC Domain Settings for Cisco Nexus 93180YC-FX-A

The vPC domain will be assigned a unique number from 1-1000 and will handle the vPC settings specified within the switches. For this deployment, vPC domain 10 is used.

Configure vPC Domain Settings for Cisco Nexus 93180YC-FX-B

The vPC domain will be assigned a unique number from 1-1000 and will handle the vPC settings specified within the switches. For this deployment, vPC domain 10 is used.

Configure vPC Peer-Link

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC Peer Link and then configure the Peer Link.

Configure Port-Channel to Fabric Interconnect A

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to Fabric Interconnect A.

Configure Port-Channel to Fabric Interconnect B

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to Fabric Interconnect B.

Configure Port-Channel to Upstream Network

On Cisco Nexus 93180YX-FX-A and Cisco Nexus 93180YC-FX-B switches, configure the Port Channel member interfaces that will be part of the vPC link to the upstream network.

Configure Fibre Channel Ports in Cisco Nexus 93180YC-FX A

To configure individual ports and port-channels for switch A, follow this step:

From the global configuration mode, run the following commands:

Configure Fibre Channel Ports in Cisco Nexus 93180YC-FX B

To configure individual ports and port-channels for switch B, follow this step:

From the global configuration mode, run the following commands:

Create VSANs in Cisco Nexus 93180YC-FX A

To create the necessary VSANs for fabric A and add ports to them, follow these steps:

From the global configuration mode, run the following commands:

Create VSANs in Cisco Nexus 93180YC-FX B

To create the necessary VSANs for fabric B and add ports to them, follow these steps:

From the global configuration mode, run the following commands:

Switch Testing Commands

The following commands can be used to check for correct switch configuration:

Cisco Nexus 93180YC-FX Zoning

This section explains how to configure Device Aliases, Zoning, and Zonesets in the Cisco Nexus 93180YC-FX switches for use in a FlashStack environment. Follow the steps precisely because failure to do so could result in an improper configuration.

Create Device Aliases in Cisco Nexus 93180YC-FX A

To create device aliases for Fabric A that will be used to create zones, follow these steps:

Log in as admin and from the global configuration mode, run the following commands:

Create Device Aliases in Cisco Nexus 93180YC-FX B

To create device aliases for Fabric B that will be used to create zones, follow these steps:

Log in as admin and from the global configuration mode, run the following commands:

Create Zones and Zoneset in Cisco Nexus 93180YC-FX A

To create the required zones and zoneset on Fabric A, run the following commands:

Create Zones and Zoneset in Cisco Nexus 93180YC-FX B

To create the required zones and zoneset on Fabric B, run the following commands:

About the Author

Allen Clark, Technical Marketing Engineer, Cisco Systems, Inc.

Allen Clark has over 15 years of experience working with enterprise storage and data center technologies. As a member of various organizations within Cisco, Allen has worked with hundreds of customers on implementation and support of compute and storage products. Allen holds a bachelor’s degree in Computer Science from North Carolina State University and is a dual Cisco Certified Internetwork Expert (CCIE 39519, Storage Networking and Data Center)

Acknowledgements

For their support and contribution to the design, validation, and creation of this Cisco Validated Design, the author would like to thank:

●      John George, Technical Marketing Engineer, Cisco Systems, Inc.

●      Craig Waters, Solutions Architecture and Product Management, Pure Storage, Inc.

Feedback

For comments and suggestions about this guide and related guides, join the discussion on Cisco Community at https://cs.co/en-cvds.

NOTE: Available paragraph styles are listed in the Quick Styles Gallery in the Styles group on the Home tab. Alternatively, they can be accessed via the Styles window (press Alt + Ctrl + Shift + S).