Explore Cisco
How to Buy

Have an account?

  •   Personalized content
  •   Your products and support

Need an account?

Create an account

Configure Cisco Intersight Managed Mode for FlexPod: Technical Preview Release

Available Languages

Download Options

  • PDF
    (18.4 MB)
    View with Adobe Reader on a variety of devices
Updated:February 2, 2021

Available Languages

Download Options

  • PDF
    (18.4 MB)
    View with Adobe Reader on a variety of devices
Updated:February 2, 2021

Table of Contents

 

Executive summary

The FlexPod solution delivers an integrated architecture that incorporates compute, storage, and network design best practices to reduce IT risks by validating the integrated architecture and helping ensure compatibility among the components. The solution addresses IT challenges by providing documented design and deployment guidance and solution support that can be used in various stages—in the planning, design, and implementation—of a deployment.

The Cisco Intersight™ platform is a management solution delivered as a service with embedded analytics for Cisco® and third-party IT infrastructures. The Cisco Intersight managed mode (also referred to as Cisco IMM or Intersight managed mode) is a new architecture that manages Cisco Unified Computing System™ (Cisco UCS®) fabric interconnect–attached systems through a Redfish-based standard model. Cisco Intersight managed mode combines the capabilities of Cisco UCS and the cloud-based flexibility of the Cisco Intersight platform, thereby unifying the management experience for both standalone and fabric interconnect–attached systems. Cisco Intersight managed mode standardizes both policy and operation management for fourth-generation fabric interconnects and Cisco UCS M5 servers. The modular nature of the Cisco Intersight platform provides an easy upgrade path to additional services such as workload optimization and Kubernetes.

This document helps Cisco customers and business partners position and deploy Cisco Intersight managed mode and Cisco UCS systems within FlexPod environments. The document also presents the new Cisco Intersight managed mode constructs, policies, and workflow involved in deploying Cisco UCS in a FlexPod Datacenter environment. The Cisco UCS functions previously configured in a FlexPod environment using Cisco UCS Manager will now be delivered through the Cisco Intersight portal, which provides global visibility into the infrastructure health and status along with advanced management and support capabilities.

Although the focus of this document is the Cisco UCS and Cisco Intersight managed mode platforms, customers interested in understanding FlexPod design and deployment details, including configuration of other elements of design and associated best practices, should refer to Cisco Validated Designs for FlexPod, at https://www.cisco.com/c/en/us/solutions/design-zone/data-center-design-guides/flexpod-design-guides.html.

Note:      Cisco Intersight managed mode is currently a technical preview release feature, and this initial technical preview release contains features and functions that are not meant for use in a production or commercial operating environment or with live data. Cisco recommends using this software on a test network or system. Not all the functions and features in Cisco UCS Manager are supported in the Cisco Intersight managed mode technical preview release.

Overview

This section provides an overview of the Cisco Intersight and FlexPod platforms.

Cisco Intersight overview

The Cisco Intersight platform is a software-as-a-service (SaaS) infrastructure lifecycle management platform that delivers simplified configuration, deployment, maintenance, and support. With the Cisco Intersight platform, customers get all the benefits of SaaS delivery and the full lifecycle management Cisco Intersight connected distributed servers and third-party storage systems such as NetApp across data centers, remote sites, branch offices, and edge environments (Figure 1).

The Cisco Intersight platform is designed to be modular, so customers can adopt services based on their individual requirements. The platform significantly simplifies IT operations by bridging applications with infrastructure, providing visibility and management from bare-metal servers and hypervisors to serverless applications, thereby reducing costs and mitigating risk. This unified SaaS platform uses a unified OpenAPI design that natively integrates with the third-party platforms and tools.

DiagramDescription automatically generated

Figure 1.       

Cisco Intersight overview

The main benefits of Cisco Intersight infrastructure services are summarized here:

     Simplify daily operations by automating many daily manual tasks.

     Combine the convenience of a SaaS platform with the capability to connect from anywhere and manage infrastructure through a browser or mobile app.

     Stay ahead of problems and accelerate trouble resolution through advanced support capabilities.

     Gain global visibility of infrastructure health and status along with advanced management and support capabilities.

     Upgrade to add workload optimization and Kubernetes services when needed.


Cisco Intersight Virtual Appliance and Private Virtual Appliance

In addition to the SaaS deployment model running on Intersight.com, on-premises options can be purchased separately. The Cisco Intersight Virtual Appliance and Cisco Intersight Private Virtual Appliance are available for organizations that have additional data locality or security requirements for managing systems. The Cisco Intersight Virtual Appliance delivers the management features of the Cisco Intersight platform in an easy-to-deploy VMware Open Virtualization Appliance (OVA) or Microsoft Hyper-V Server virtual machine that allows you to control the system details that leave your premises. The Cisco Intersight Private Virtual Appliance is provided in a form factor specifically designed for users who operate in disconnected (air gap) environments. The Private Virtual Appliance requires no connection to public networks or back to Cisco to operate. At this time, Cisco Intersight managed mode is a tech-preview feature, and the configuration is available only through the Cisco Intersight SaaS platform.

FlexPod Datacenter overview

Customers seeking to deploy applications using a shared data center infrastructure face several challenges. A recurring infrastructure challenge is achieving the required levels of IT agility and efficiency to effectively meet the organization’s business objectives. Addressing these challenges requires an optimal solution with the following main characteristics:

     Availability: Help ensure that applications and services are available at all times, with no single point of failure.

     Flexibility: Support new services without requiring changes to the underlying infrastructure.

     Efficiency: Facilitate efficient operation of the infrastructure through reusable policies.

     Manageability: Facilitate ease of deployment and ongoing management, reducing operating costs.

     Scalability: Support easy expansion and growth, providing significant investment protection.

     Compatibility: Reduce risk by helping ensure compatibility of integrated components.

Cisco and NetApp have partnered to deliver a series of FlexPod solutions that enable strategic data center platforms with these characteristics. FlexPod delivers an integrated architecture that incorporates computing, storage, and network design best practices, reducing IT risks by validating the integrated architecture to help ensure compatibility among the system components.

FlexPod components

FlexPod architecture includes the following core components (Figure 2):

     Cisco UCS

     Cisco Nexus® Family switches

     Cisco MDS Family switches

     NetApp All Flash FAS (AFF) systems

Macintosh HD:Users:sandygraul:Documents:ETMG:Cisco:221011_Cisco:1_cisco-imm-for-flexpod:art:fig02_FlexPod-Datacenter-components.jpg

Figure 2.       

FlexPod Datacenter components

All the FlexPod components have been integrated so that customers can deploy the solution quickly and economically while eliminating many of the risks associated with researching, designing, building, and deploying similar solutions from the foundation. One of the main benefits of FlexPod is its ability to maintain consistency at scale. Each of the component families shown in Figure 2 (Cisco UCS, Cisco Nexus, Cisco MDS, and NetApp controllers) offers platform and resource options to scale the infrastructure up or down while supporting the same features and functions that are required under the configuration and connectivity best practices of FlexPod.

Solution design

This section discusses the infrastructure setup, software and hardware requirements, and some of the design details of the Cisco Intersight managed mode deployment model. Cisco Intersight managed mode is a new feature, currently released for technical preview, and specific hardware and software requirements must be followed to configure Cisco UCS using Cisco Intersight managed mode. The selection of FlexPod infrastructure components presented here closely aligns with Cisco Intersight managed mode requirements. This section does not cover the design details of FlexPod components such as Cisco Nexus and Cisco MDS switches and NetApp storage systems because their design and configuration conform to various Cisco Validated Designs for FlexPod and are covered widely elsewhere. This document focuses on the design elements of the new Cisco Intersight managed mode configuration.

Cisco Intersight managed mode

During initial fabric interconnect setup for the fabric-attached Cisco UCS system, customers can choose to deploy fabric interconnect and Cisco UCS in the native Cisco UCS Manager managed mode or the new Cisco Intersight managed mode. This document discusses Cisco UCS deployment using Cisco Intersight managed mode, and all the configuration steps are performed using the Cisco Intersight SaaS platform.

Note:      This document does not cover the migration of policies from a Cisco UCS Manager managed system to a Cisco Intersight managed mode system. The configuration parameters and procedures for the two configuration modes are quite different and require manual translation of policies when you move from one mode to the other.

Before setting up Cisco Intersight managed mode, review the supported hardware, software, and licensing requirements that follow.

Cisco Intersight managed mode supported hardware

The hardware listed in Table 1 is required to deploy Cisco UCS using Cisco Intersight managed mode.

Table 1.        Cisco Intersight managed mode supported hardware

Component

Model number

Fabric interconnect

Fourth-generation fabric interconnect: UCS-FI-6454

Cisco UCS B-Series Blade Servers

Cisco UCS B-Series M5: UCSB-B200-M5 or UCSB-B480-M5

Cisco UCS C-Series Rack Servers

Cisco UCS C-Series M5:UCSC-C220-M5, UCSC-C240-M5, or UCSC-C480-M5

Chassis

N20-C6508 or UCSB-5108-AC2

I/O module (IOM)

UCS-IOM-2204XP, UCS-IOM-2208XP, or UCS-IOM-2408

Adapters

Cisco UCS B-Series: UCSB-MLOM-40G-04 or UCSB-VIC-M84-4P. Cisco UCS C-Series: UCSC-MLOM-C25Q-04 or UCSC-PCIE-C25Q-04

Validated software

Release 4.1(2a)*

* Make sure all the Cisco UCS components including servers and adapters have been upgraded to the correct version. Device discovery will fail if an unsupported version is installed on the Cisco UCS components. The items highlighted in bold were used during the validation process discussed in this document.

For the most up-to-date support information for Cisco Intersight managed mode, see https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/b_Intersight_Managed_Mode_Configuration_Guide/b_Intersight_Managed_Mode_Beta_Guide_chapter_01010.html.

Licensing requirements

The Cisco Intersight platform uses a subscription-based license with multiple tiers. You can purchase a subscription duration of one, three, or five years and choose the required Cisco UCS server volume tier for the selected subscription duration. Each Cisco endpoint automatically includes a Cisco Intersight Base license at no additional cost when you access the Cisco Intersight portal and claim a device. You can purchase any of the following higher-tier Cisco Intersight licenses using the Cisco ordering tool:

     Cisco Intersight Essentials: Essentials includes all the functions of the Base license plus additional features, including Cisco UCS Central Software and Cisco Integrated Management Controller (IMC) supervisor entitlement, policy-based configuration with server profiles, firmware management, and evaluation of compatibility with the Cisco Hardware Compatibility List (HCL).

     Cisco Intersight Advantage: Advantage offers all the features and functions of the Base and Essentials tiers. It includes storage widgets and cross-domain inventory correlation across compute, storage, and virtual environments (VMWare ESXi). It also includes OS installation for supported Cisco UCS platforms.

     Cisco Intersight Premier: In addition to the functions provided in the Advantage tier, Premier includes full subscription entitlement for Cisco UCS Director providing orchestration across Cisco UCS and third-party systems.

Servers in the Cisco Intersight managed mode require at least the Essentials license. The validation process for this document used a Premier license; however, all the functions covered in this document are supported with the Essentials license. For more information about the features provided in the various licensing tiers, see https://intersight.com/help/getting_started#licensing_requirements.

Cisco Intersight managed mode technical preview release

This initial technical preview release contains features and functions that are not meant for use in a production or commercial operating environment or with live data. Cisco recommends that you install this software on a test network or system. The Cisco Intersight managed mode technical preview release does not support all the features and functions of the Cisco Intersight platform. The appendix of this document presents detailed instructions about how to set up and configure Cisco Intersight managed mode for Cisco UCS fabric interconnects and covers the configuration of features and functions delivered as part of this initial technical preview release.

FlexPod setup for Cisco Intersight managed mode configuration

The FlexPod setup used to validate the Cisco Intersight managed mode configuration aligns with the FlexPod Datacenter for VMware vSphere design, described at https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3_design.html. Figure 3 shows the connectivity between the various elements of this setup.

Macintosh HD:Users:sandygraul:Documents:ETMG:Cisco:221011_Cisco:1_cisco-imm-for-flexpod:art:fig03_FlexPod-Cisco-Intersight-managed-mode.jpg

Figure 3.       

Topology to verify Cisco Intersight managed mode configuration in a FlexPod environment

To validate the Cisco Intersight managed mode configuration in a FlexPod environment, the components were set up as follows:

     Cisco UCS 6454 Fabric Interconnects provide the chassis and network connectivity.

     The Cisco UCS 5108 Blade Server Chassis connects to fabric interconnects using Cisco UCS 2208XP IOMs, where four 10 Gigabit Ethernet ports were used on each IOM to connect to the fabric interconnect.

     Cisco UCS B200 M5 servers contain fourth-generation Cisco virtual interface cards (VICs): UCSB-MLOM-40G-04.

     Cisco Nexus 9336C Switches running in Cisco NX-OS mode provide the switching fabric.

     Cisco UCS 6454 Fabric Interconnect 100 Gigabit Ethernet uplink ports connect to Cisco Nexus 9336C Switches in a virtual port channel (vPC).

     Cisco UCS 6454 Fabric Interconnects are connected to the Cisco MDS 9132T switches using 16-Gbps Fibre Channel connections configured as a port channel for SAN connectivity.

     The NetApp AFF controller connects to the Cisco Nexus 9336C Switches using 10 Gigabit Ethernet ports configured as a vPC.

     The NetApp AFF controller connects to the Cisco MDS 9132T switches using 16-Gbps Fibre Channel connections for SAN connectivity.

     VMware 6.7 Update 3 ESXi software is installed on Cisco UCS B200 M5 servers to validate the infrastructure.

Note:      At this time, Small Computer System Interface over IP (iSCSI) boot is not supported in the Cisco Intersight managed mode. NetApp storage controller connection to Cisco UCS fabric interconnects using Fibre Channel links, also known as direct-attached SAN connectivity, also is not supported.

Configuration constructs for Cisco Intersight managed mode

Cisco Intersight managed mode unites the capabilities of the Cisco UCS platform and the cloud-based flexibility of the Cisco Intersight platform, thus unifying the management experience for standalone and fabric interconnect–attached systems. Cisco Intersight managed mode standardizes policy and operation management for fourth-generation fabric interconnects and Cisco UCS M5 servers.

At a high level, configuring Cisco UCS using Cisco Intersight managed mode consists of the steps shown in Figure 4. The details of these steps are presented in the following sections.

Macintosh HD:Users:sandygraul:Documents:ETMG:Cisco:221011_Cisco:1_cisco-imm-for-flexpod:art:fig04_configuring-Cisco-UCS.jpg

Figure 4.       

Steps for configuring Cisco UCS using Cisco Intersight managed mode

Setting up Cisco UCS fabric interconnects for Cisco Intersight managed mode

The initial configuration for a fabric interconnect can be performed using the serial console when the fabric interconnect boots for the first time. This can happen either during factory installation or after the existing configuration has been erased. During the initial configuration, for the management mode the configuration wizard enables customers to choose whether they want to manage the fabric interconnect through Cisco UCS Manager or the Cisco Intersight platform. Customers can switch the management mode for the fabric interconnects between Cisco Intersight and Cisco UCS Manager at any time. However, this is a disruptive process because it causes all endpoint configurations to be reset and results in the loss of the current configuration. In the validation process described here, the existing configuration on the Cisco UCS fabric interconnects was cleared, and the system was set up for Cisco Intersight managed mode.

Figure 5 shows the output from the fabric interconnect console to enable Cisco Intersight managed mode.

TextDescription automatically generated

Figure 5.       

Fabric interconnects set up for Cisco Intersight managed mode

Claiming a Cisco UCS fabric interconnect in the Cisco Intersight platform

After you set up the Cisco UCS fabric interconnect for Cisco Intersight managed mode, you can add the fabric interconnects to a new or an existing Cisco Intersight account (Figure 6). The details of the device claim process are covered in the appendix. When a Cisco UCS fabric interconnect is successfully added to the Cisco Intersight platform, all future configuration steps are completed in the Cisco Intersight portal.

Related image, diagram or screenshot

Figure 6.       

Cisco Intersight platform: Adding fabric interconnects

You can verify whether a Cisco UCS fabric interconnect is in Cisco UCS Manager managed mode or Cisco Intersight managed mode by clicking the fabric interconnect name and looking at the detailed information screen for the fabric interconnect, as shown in Figure 7.

Graphical user interface, applicationDescription automatically generated

Figure 7.       

Fabric Interconnect in Cisco Intersight managed mode

Configuring a Cisco UCS domain profile

A Cisco UCS domain profile configures a fabric interconnect pair through reusable policies, allows configuration of the ports and port channels, and configures the VLANs and VSANs to be used in the network. It defines the characteristics of and configures the ports on the fabric interconnects. The domain-related policies can be attached to the profile either at the time of creation or later. One Cisco UCS domain profile can be assigned to one fabric interconnect domain, and the Cisco Intersight platform supports the attachment of one port policy per Cisco UCS domain profile. Policies that are attached to a Cisco UCS domain profile can be created either before or during the creation of the profile.

Some of the characteristics of the Cisco UCS domain profile set up for this validation are as follows:

     A single domain profile is created for the pair of Cisco UCS fabric interconnects.

     Separate port policies are defined for the two fabric interconnects because each fabric interconnect uses unique Fibre Channel and VSAN configurations. If boot from SAN were not required, the same port policy could have been reused across the two fabric interconnects.

     The VLAN configuration policy is common to the fabric interconnect pair because both fabric interconnects are configured for same set of VLANs.

     The VSAN configuration policies are unique for the two fabric interconnects because the VSANs are unique.

     The Network Time Protocol (NTP), network connectivity, and system quality-of-service (QoS) policies are common to the fabric interconnect pair.

After the Cisco UCS domain profile has been successfully created, the fabric interconnects in FlexPod environment can do the following:

     Form an Ethernet port channel with the Cisco Nexus switch.

     Form a Fibre Channel port channel with the Cisco MDS switch.

     Discover the Cisco UCS chassis and the blades.

Figure 8 shows a summary of the Cisco UCS fabric interconnect and the port configuration after the Cisco UCS domain profile was deployed.

A screenshot of a computerDescription automatically generated

Figure 8.       

Cisco UCS domain profile

Creating a server profile

A server profile enables resource management by simplifying policy alignment and server configuration. You can create server profiles using the server profile wizard to provision servers, create policies to help ensure smooth deployment of servers, and eliminate failures caused by inconsistent configurations. The server profile wizard groups the server policies into the following four categories to provide a quick summary view of the policies that are attached to a profile:

     Compute policies: BIOS, boot order, and virtual media policies

     Network policies: Adapter configuration, LAN connectivity, and SAN connectivity policies

    The LAN connectivity policy requires you to create Ethernet network policy, Ethernet adapter policy, and Ethernet QoS policy.

    he SAN connectivity policy requires you to create Fibre Channel network policy, Fibre Channel adapter policy, and Fibre Channel QoS policy.

     Storage policies: Secure Digital (SD) card and storage policies (not used in FlexPod)

     Management policies: Device connector, Intelligent Platform Management Interface (IPMI) over LAN, Lightweight Directory Access Protocol (LDAP), local user, network connectivity, Simple Mail Transfer Protocol (SMTP), Simple Network Management Protocol (SNMP), Secure Shell (SSH), Serial over LAN (SOL), syslog, and virtual keyboard, video, and mouse (KVM) policies
 

Some of the characteristics of the server profile set up for this validation are as follows:

     BIOS policy is created to specify various server parameters in accordance with FlexPod best practices.

     Boot order policy defines virtual media (KVM mapper DVD) and all four SAN paths for NetApp Fibre Channel logical interfaces (LIFs).

     IMC access policy defines the management IP address pool for KVM access.

     Local user policy is used to create KVM access.

     LAN connectivity policy is used to create four virtual network interface cards (vNICs)—two for management virtual switches (vSwitches) and two for application virtual dedicated servers (VDSs)—along with various policies and pools.

     SAN connectivity policy is used to create two virtual host bus adapters (vHBAs)—one for SAN A and one for SAN B—along with various policies and pools .

Figure 9 shows various policies associated with the server profile, and Figure 10 shows a successfully deployed server profile and associated blade.

Related image, diagram or screenshot

Figure 9.       

Server profile policies

Related image, diagram or screenshot

Figure 10.     

Server profile details

After a server profile has been successfully deployed, the server successfully boots from SAN storage hosted on the NetApp storage. Additional server profiles are created simply by cloning the first server profile and programming the Cisco MDS switches and NetApp controllers for various SAN parameters. For step-by-step deployment guidance for Cisco UCS and Cisco Intersight managed mode, refer to the appendix.

Conclusion

The Cisco Intersight platform is a SaaS infrastructure lifecycle management solution that delivers simplified configuration, deployment, maintenance, and support. The FlexPod solution delivers an integrated architecture that incorporates computing, storage, and network design best practices to reduce IT risk by validating the integrated architecture and helping ensure compatibility among the components.

Integrating the Cisco Intersight platform into a FlexPod environment provides global visibility of infrastructure health and status along with advanced management and support capabilities. The Cisco Intersight platform delivers a convenient SaaS solution with the capability to connect from anywhere and manage infrastructure through a browser or mobile app while allowing customers to stay ahead of problems and accelerate trouble resolution through advanced support capabilities.

For more information

Consult the following references for additional information about the topics discussed in this document.

Products and solutions

     Cisco Intersight platform:
https://www.intersight.com

     Cisco Intersight managed mode:
https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/b_Intersight_Managed_Mode_Configuration_Guide.html

     Cisco Unified Computing System:
http://www.cisco.com/en/US/products/ps10265/index.html

     Cisco UCS 6454 Fabric Interconnect:
https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/datasheet-c78-741116.html

     Cisco UCS 5100 Series Blade Server Chassis: http://www.cisco.com/en/US/products/ps10279/index.html

     Cisco UCS B-Series Blade Servers:
http://www.cisco.com/en/US/partner/products/ps10280/index.html

     Cisco UCS adapters:
http://www.cisco.com/en/US/products/ps10277/prod_module_series_home.html

     Cisco Nexus 9000 Series Switches:
http://www.cisco.com/c/en/us/products/switches/nexus-9000-series-switches/index.html

     NetApp ONTAP 9:
http://www.netapp.com/us/products/platform-os/ontap/index.aspx

Interoperability matrixes

     Cisco UCS Hardware Compatibility Matrix:
https://ucshcltool.cloudapps.cisco.com/public/

     NetApp Interoperability Matrix Tool:
http://support.netapp.com/matrix/

Configuration guides

     FlexPod Datacenter Design Guide:
https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3_design.html

     FlexPod Datacenter Deployment Guide
https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3.html

Appendix: Configuration details

This appendix describes how to set up a Cisco UCS fabric in Cisco Intersight managed mode and specify the FlexPod-related computing configuration using the Cisco Intersight platform. This appendix does not discuss how to set up the switching infrastructure or the storage. Refer to the relevant FlexPod deployments guides for details about these components: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3.html.

Set up Cisco Intersight managed mode on Cisco UCS fabric interconnects

The Cisco UCS fabric interconnects need to be set up to support Cisco Intersight managed mode. If you are converting an existing pair of Cisco UCS fabric interconnects, first erase the configuration and reboot your system. Converting fabric interconnects to Cisco Intersight managed mode is a disruptive process, and configuration information will be lost. Customers are encouraged to make a backup of their existing configuration if they plan only to test Cisco Intersight managed mode and then revert to Cisco UCS Manager managed mode.

1.    Erase the configuration on existing fabric interconnects. Connect to each of the fabric interconnect consoles, log in as admin, and enter the following commands:

Note:             This erasure process is not needed on brand-new fabric interconnects that have not been configured yet.

UCS-A# connect local-mgmt

UCS-A(local-mgmt)# erase configuration

All UCS configurations will be erased and system will reboot. Are you sure? (yes/no): yes

2.    Configure Fabric Interconnect A (FI-A). On the Basic System Configuration Dialog screen, set the management mode to Intersight. All the remaining settings are similar to those for the Cisco UCS Manager managed mode (UCSM-Managed). Note that there is no virtual IP address setting anymore when Cisco Intersight managed mode is selected.

TextDescription automatically generated

TextDescription automatically generated

After applying the settings, make sure you can ping the fabric interconnect management IP address. When Fabric Interconnect A is correctly set up and is available, Fabric Interconnect B will automatically discover Fabric Interconnect A during its setup process as shown in the next step.

3.    Configure Fabric Interconnect B (FI-B). For the configuration method, choose console. Fabric Interconnect B will detect the presence of Fabric Interconnect A and will prompt you to enter the admin password for Fabric Interconnect A. Provide the management IP address for Fabric Interconnect B and apply the configuration.

TextDescription automatically generated

Set up Cisco Intersight account

In this step, using the unique device information for the Cisco UCS, you set up a new Cisco Intersight account. Customers also can choose to add the Cisco UCS devices set up for Cisco Intersight managed mode to an existing Cisco Intersight account; however, that procedure is not covered in this document.

Claim a device

After completing the initial configuration for the fabric interconnects, log in to Fabric Interconnect A using your web browser to capture the Cisco Intersight connectivity information.

1.    Use the management IP address of Fabric Interconnect A to access the device from a web browser and the previously configured admin password to log in to the device.

Graphical user interface, applicationDescription automatically generated

2.    Under DEVICE CONNECTOR, you should see the current device status as ”Not claimed.” Note, or copy, the Device ID and Claim Code information to use to set up a new Cisco Intersight account.

Note:             The Device ID and Claim Code information can also be used to claim the Cisco UCS devices set up with Cisco Intersight managed mode in an existing Cisco Intersight account.

A screenshot of a computerDescription automatically generated

Create a new Cisco Intersight account

Next, create a new Cisco Intersight account.

1.    Visit https://www.intersight.com and click “Don't have an Intersight Account? Create an account.”

2.    Provide an account name and the device information captured in the preceding steps to create the account. This step will automatically add the Cisco UCS device to the new Cisco Intersight account.

A screen shot of a computerDescription automatically generated

3.    After the account has been created successfully, click Go To Intersight.

Related image, diagram or screenshot

4.    You should see a screen with your Cisco Intersight account.

A screenshot of a stereoDescription automatically generated

Verify addition of Cisco UCS fabric interconnects to Cisco Intersight

Now verify Cisco UCS fabric interconnects are added to your account in Cisco Intersight.

1.    Go back to the web GUI of the Cisco UCS fabric interconnect and click the Refresh button.

2.    The fabric interconnect status should now be set to Claimed.

Graphical user interface, application, website, TeamsDescription automatically generated

Set up licensing

When setting up a new Cisco Intersight account (as discussed in this document), the account needs to be enabled for Cisco Smart Software Licensing.

1.    Associate the Cisco Intersight account with Cisco Smart Licensing by following these steps:

    Log in to the Cisco Smart Licensing portal: https://software.cisco.com/software/csws/ws/platform/home?locale=en_US#module/SmartLicensing.

    Select the correct virtual account.

    Under Inventory > General, generate a new token for product registration.

    Copy this newly created token.

Graphical user interface, text, application, emailDescription automatically generated

2.    With the Cisco Intersight account associated with Cisco Smart Licensing, log in to the Cisco Intersight portal and click Settings (the gear icon) in the top-right corner. Choose Licensing.

Graphical user interface, application, TeamsDescription automatically generated

3.    Under Cisco Intersight > Licensing, click Register.

Related image, diagram or screenshot

4.    Enter the copied token from the Cisco Smart Licensing portal.

A screenshot of a computerDescription automatically generated

5.    Click Register and wait for registration to go through. When the registration is successful, the information about the associated Cisco Smart account is displayed.

A screenshot of a cell phoneDescription automatically generated

6.    For all new accounts, the default licensing tier is set to Base. For Cisco Intersight managed mode, the default tier needs to be changed to Essential or a higher tier. To make this change, click Set Default Tier.

Graphical user interface, applicationDescription automatically generated

7.    Select the tier supported by your Smart License.

Graphical user interface, applicationDescription automatically generated

8.    In this deployment, the default license tier is set to Premier.

Graphical user interface, applicationDescription automatically generated

Troubleshoot license issues

At this time, a known issue occurs: if Cisco UCS was claimed when the license default tier was set to Base (for all new account setups), customers may see error messages when creating certain policies (for example, domain profile). The error message looks similar to the following screen:

Graphical user interface, applicationDescription automatically generated

To work around this issue, you need to delete Cisco UCS from the account and then reclaim it after making sure that the correct license tier has been set up as the default. To delete an existing Cisco UCS device, follow these steps:

1.    Log in to the Cisco Intersight portal.

2.    Go to ADMIN > Devices in the left column and select the device.

3.    Click the Delete icon.

A picture containing monitor, displayDescription automatically generated

4.    Click Delete.

Graphical user interface, websiteDescription automatically generated

5.    Go back to the Cisco UCS device GUI and make sure that the Cisco UCS status has changed to Not Claimed.

 Graphical user interfaceDescription automatically generated

6.    In the Cisco Intersight portal, click Settings (the gear Icon) in the top-right corner and choose Licensing.

7.    Verify that the default licensing tier is set to Essentials (or higher).

8.    Go to ADMIN > Targets and click Claim a New Target in the top-right corner.

9.    Select Cisco UCS Domain (Intersight Managed).

10.  Click Start.

11.  Add the device ID and claim code copied from the Cisco UCS device GUI.

12.  Click Claim at the bottom right.

Note:      This issue is not observed when adding Cisco UCS Manager managed systems or Cisco HyperFlex systems to a new or existing Cisco Intersight account.

Set up Cisco Intersight organization

You need to define all Cisco Intersight managed mode configurations for Cisco UCS, including policies, under an organization. To define a new organization, follow these steps:

1.    Log in to the Cisco Intersight portal.

2.    Click Settings (the gear icon) and choose Settings.

Graphical user interface, applicationDescription automatically generated

3.    Click Organizations in the middle panel.

4.    Click Create Organization in the top-right corner.

A screenshot of a computer screenDescription automatically generated

5.    Provide a name for the organization (for example, FPV).

6.    Under Memberships, select Custom.

7.    Select the recently added Cisco UCS device for this organization.

8.    Click Create.

A screenshot of a stereoDescription automatically generated

Configure a Cisco UCS domain profile

A Cisco UCS domain profile configures a fabric interconnect pair through reusable policies, allows configuration of the ports and port channels, and configures the VLANs and VSANs in the network. It defines the characteristics of and configures ports on fabric interconnects. The domain-related policies can be attached to the profile either at the time of creation or later. One Cisco UCS domain profile can be assigned to one fabric interconnect domain.

To create a Cisco UCS domain profile, follow these steps:

1.    Log in to the Cisco Intersight portal

2.    Click to expand CONFIGURE in the left pane and select Profiles.

3.    In the main window, select UCS Domain Profiles and click Create UCS Domain Profile.

A screenshot of a video gameDescription automatically generated

4.    On the Create UCS Domain Profile screen, click Start.

Graphical user interface, websiteDescription automatically generated

Step 1: General

Follow these steps for the general configuration:

1.    Choose the organization from the drop-down menu (for example, FPV).

2.    Provide a name for the domain profile (for example, AA04-6454-DomProf).

Graphical user interface, application, TeamsDescription automatically generated

3.    Click Next.

Step 2: UCS Domain Assignment

1.    Follow these steps for Cisco UCS domain assignment:

2.    Assign the Cisco UCS domain to this new domain profile by clicking Assign Now and selecting the previously added Cisco UCS domain (AA04-6454).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

Step 3: Ports Configuration

Follow these steps to configure the ports:

1.    Click Select Policy for Fabric Interconnect A.

A screenshot of a cell phoneDescription automatically generated

2.    Click Create New in the top-right pane to define a new port configuration policy.

Note:               This document uses separate port policies for the two fabric interconnects because each fabric interconnect uses unique Fibre Channel and VSAN. If boot from SAN were not required, the same port policy could have been reused across the two fabric interconnects.

3.    Choose the organization from the drop-down menu.

4.    Provide a name for the policy (for example, AA04-6454-A-PortPol)

A screenshot of a cell phone screen with textDescription automatically generated

5.    Move the slider to set up unified ports. In this example, the first four ports were selected as Fibre Channel ports. Click Next.

Graphical user interface, applicationDescription automatically generated

6.    Verify that ports 1 to 4 are indeed configured as Fibre Channel ports.

A screenshot of a computerDescription automatically generated

7.    Select all the ports that need to be configured as server ports by clicking the ports in the graphics (or from the list below the graphic). When all ports are selected, click Configure.

A screenshot of a cell phone screen with textDescription automatically generated

8.    From the drop-down menu, choose Server as the role. Leave Forward Error Correction (FEC) set to Auto and click Save.

Graphical user interface, applicationDescription automatically generated

9.    Configure the Ethernet uplink port channel by selecting the port channel in the main pane and then clicking Create Port Channel.

A picture containing graphical user interfaceDescription automatically generated

10.  Choose Ethernet Uplink Port Channel as the role, provide a port-channel ID (for example, 15), and choose a value for Admin Speed (Auto is used in this example).

Graphical user interface, textDescription automatically generated

11.  Click Save.

12.  Configure a Fibre Channel Port Channel by selecting the port channel in the main pane again and clicking Create Port Channel.

13.  In the drop-down menu under Role, choose FC Uplink Port Channel.

14.  Provide a port-channel ID (for example, 111), choose a value for Admin Speed (16Gbps is used here), and provide a VSAN ID (for example, 111).

Graphical user interface, applicationDescription automatically generated

15.  Click Save.

16.  Verify the port-channel IDs and ports after both the Ethernet uplink port channel and the Fibre Channel uplink port channel have been created.

A screenshot of a computer screenDescription automatically generated

17.  Click Save to create the port policy for Fabric Interconnect A. Use the summary screen here to verify that the ports were selected and configured correctly.

A screenshot of a computer screenDescription automatically generated

18.  Now create policy for Fabric Interconnect B. Click Select Policy for Fabric Interconnect B, and in the pane at the right, click Create New.

19.  Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-B-PortPol).

Graphical user interface, applicationDescription automatically generated

20.  Repeat the steps you used for Fabric Interconnect A to configure Fibre Channel ports, server ports, and Ethernet and Fibre Channel port channels with appropriate IDs (for example, Ethernet port-channel ID 16 and Fibre Channel port-channel ID 112).

Graphical user interface, applicationDescription automatically generated

21.  Use the summary screen shown here to verify that the ports were selected and configured correctly for Fabric Interconnect B.

Graphical user interface, textDescription automatically generated

22.  When the port configuration for both fabric interconnects is complete and looks good, click Next.

Step 4: VLAN and VSAN Configuration

In this step, you create a single VLAN policy for both fabric interconnects, but you create individual policies for the VSANs because the VSAN IDs are unique for each fabric interconnect.

Create and apply VLAN policy

Follow these steps to create and apply the VLAN policy:

1.    Click Select Policy next to VLAN Configuration under Fabric Interconnect A. Then, in the pane on the right, click Create New.

A screenshot of a cell phoneDescription automatically generated

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-VLANPol).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

4.    Click Add VLANs.

A screenshot of a computer screenDescription automatically generated

5.    Provide a name and VLAN ID for the native VLAN (for example, Native-Vlan and 2).

Graphical user interface, applicationDescription automatically generated

6.    Click Add.

7.    Select Set Native VLAN ID and enter the VLAN number (for example, 2) under VLAN ID.

A screenshot of a cell phoneDescription automatically generated

8.    Add the remaining VLANs for FlexPod by clicking Add VLANs and entering the VLANs one by one. The VLANs used for this validation are shown in the screen image here.

A screenshot of a computerDescription automatically generated

Note:               The iSCSI VLANs shown in the screen image above are only needed if you are using iSCSI datastores (iSCSI boot from SAN is not supported at this time). For this document, iSCSI SAN access was not validated.

9.    Click Create at bottom right to create all the VLANs.

10.  Click Select Policy next to VLAN Configuration for Fabric Interconnect B and select the same VLAN policy that was created in the preceding step.

A screenshot of a cell phoneDescription automatically generated

Create and apply VSAN policy

Follow these steps to create and apply the VSAN policy:

1.    Click Select Policy next to VSAN Configuration under Fabric Interconnect A. Then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-A-VSANPol).

Graphical user interface, applicationDescription automatically generated

3.    Click Next.

4.    Click Add VSAN and provide a name (for example, VSAN-A), VSAN ID (for example, 111), and associated Fibre Channel over Ethernet (FCoE) VLAN ID (for example, 111) for SAN A.

5.    Click Add.

A screenshot of a cell phone screen with textDescription automatically generated

6.    Enable uplink trunking for this VSAN.

A screenshot of a cell phoneDescription automatically generated

7.    Click Create.

8.    Now repeat the same steps to create a new VSAN policy for SAN-B. Click Select Policy next to VSAN Configuration under Fabric Interconnect B. Then, in the pane on the right, click Create New.

9.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-B-VSANPol).

Graphical user interface, applicationDescription automatically generated

10.  Click Next.

11.  Click Add VSAN and provide a name (for example, VSAN-B), VSAN ID (for example, 112), and associated FCoE VLAN ID (for example, 112) for SAN-B.

12.  Click Add.

A screenshot of a cell phone screen with textDescription automatically generated

13.  Enable uplink trunking for this VSAN.

A picture containing monitor, screenshot, sitting, screenDescription automatically generated

14.  Click Create.

15.  Verify that a common VLAN policy and two unique VSAN policies are associated with the two fabric interconnects.

A screenshot of a cell phoneDescription automatically generated

16.  Click Next.


 

Step 5: UCS Domain Configuration

You need to define some additional policies such as NTP, network connectivity, and system QoS for the Cisco UCS domain configuration.

A screenshot of a cell phoneDescription automatically generated

Configure NTP policy

To define an NTP server for the Cisco UCS domain, configure NTP policy.

1.    Click Select Policy next to NTP and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-NTPPol).

Graphical user interface, applicationDescription automatically generated

3.    Click Next.

4.    Enable NTP, provide the NTP server IP addresses (for example, 192.168.160.254), and select the time zone from the drop-down menu (for example, America/New_York).

Graphical user interface, applicationDescription automatically generated

5.    Click Create.

Configure network connectivity policy

To define the Doman Name Service (DNS) servers for Cisco UCS, configure network connectivity policy.

1.    Click Select Policy next to Network Connectivity and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-NetConnPol).

Graphical user interface, applicationDescription automatically generated

3.    Provide DNS server IP addresses for Cisco UCS (for example, 192.168.160.53 and 192.168.160.54).

A screenshot of a cell phone screen with textDescription automatically generated

4.    Click Create.

Configure system QoS policy

To define the QoS settings for Cisco UCS, configure system QoS policy.

1.    Click Select Policy next to System QoS and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-SystemQoSPol).

Graphical user interface, text, applicationDescription automatically generated

3.    Keep the default selections or change the parameters if necessary.

A screenshot of a cell phone screen with textDescription automatically generated

4.    Click Create.

5.    Click Next.

Step 6: Summary

Verify all the settings (including the fabric interconnect settings, by expanding the settings) and make sure that the configuration is correct.

A picture containing monitor, screen, hand, sittingDescription automatically generated

Deploy the Cisco UCS domain profile

After verifying the configuration, deploy the Cisco UCS profile.

1.    Click Deploy.

Graphical user interface, applicationDescription automatically generated

2.    Acknowledge the warning and click Deploy again.

The system will take some time to validate and configure the settings on the fabric interconnects. You can log into the terminal or console servers to see when the Cisco UCS fabric interconnects have finished configuration and are successfully rebooted.

Verify Cisco UCS domain profile deployment

When the Cisco UCS domain profile has been successfully deployed, the Cisco UCS chassis and the blades should be successfully discovered.

1.    Log in to the Cisco Intersight portal. Under CONFIGURE > Profiles > UCS Domain Profiles, verify that the domain profile has been successfully deployed.

A screenshot of a video gameDescription automatically generated

2.    Verify that the chassis has been discovered and is visible under OPERATE > Chassis.

A screenshot of a cell phoneDescription automatically generated

3.    Verify that the servers have been successfully discovered and are visible under OPERATE > Servers.

A picture containing monitor, display, sitting, personDescription automatically generated

Configure the server profile

In the Cisco Intersight platform, a server profile enables resource management by simplifying policy alignment and server configuration. You can create server profiles using the server profile wizard to provision servers, create policies to help ensure smooth deployment of servers, and eliminate failures that are caused by inconsistent configuration. After creating server profiles, you can edit, clone, deploy, or unassign them as required.

To configure a server profile, follow these steps:

1.    Log in to the Cisco Intersight portal.

2.    Go to Configure > Profiles and in the main window select UCS Server Profile.

3.    Click Create UCS Server Profile.

4.    Click Start.

Graphical user interface, websiteDescription automatically generated

Step 1: General

Follow these steps for the general configuration:

1.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the server profile (for example, SP-FlexPod).

2.    Select UCS Server (FI-Attached).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

Step 2: Server Assignment

Follow these steps for server assignment:

1.    Make sure server assignment is set to Assign Now.

2.    Select a server (for example, AA06-6454-1) and click Next.

 

A screenshot of a cell phoneDescription automatically generated

Step 3: Compute Configuration

Next, configure the computing resources.

Configure BIOS policy

Follow these steps to configure BIOS policy:

1.    Click Select Policy next to BIOS Configuration and the, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-BiosPol).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

4.    On the Policy Details screen, select appropriate values.

A screenshot of a computer screenDescription automatically generated

The validation described in this document used the following values to align with the Cisco Validated Designs for FlexPod:

     LOM and PCIe Slot > CDN Support for LOM: Enabled

     Processor > DRAM Clock Throttling: Performance

     Processor > Freq Floor Override: Enabled

     Processor > CPU C State: Disabled

     Processor > Processor C1E: Disabled

     Processor > Processor C3 Report: Disabled

     Processor > Processor C6 Report: Disabled

     Processor > Power Technology: Custom

     Processor > Energy Performance: Performance

     Memory > NVM Performance Setting: Balanced Profile

     Memory > Memory RAS Configuration: Maximum Performance

Note:             Some BIOS settings (Enable Quiet Boot and Processor C7 Report) are not available at this time and cannot be configured.

5.    Click Create.

Configure boot-order policy

Follow these steps to configure boot-order policy:

1.    Click Select Policy next to BIOS Configuration and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-BootPol).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

4.    For Configured Boot Mode, select Unified Extensible Firmware Interface (UEFI).

5.    From the Add Boot Device drop-down menu, choose Virtual Media.

6.    Provide a device name (for example, ISO) and then, for the subtype, choose KVM Mapped DVD.

Graphical user interface, applicationDescription automatically generated

For this validation, all four NetApp controller LIFs will be added as boot options. The four LIFs are named as follows:

    NA-FCP-LIF01a: NetApp Controller 1, LIF for Fibre Channel SAN A

    NA-FCP-LIF02a: NetApp Controller 2, LIF for Fibre Channel SAN A

    NA-FCP-LIF01b: NetApp Controller 1, LIF for Fibre Channel SAN B

    NA-FCP-LIF02b: NetApp Controller 2, LIF for Fibre Channel SAN B

7.    From the Add Boot Device drop-down menu, choose SAN Boot.

8.    Provide the device name (for example, NA-fcp-lif01a) and the Logical Unit Number (LUN) value (for example, 0).

9.    Provide an interface name (for example, vHBA-A) and note this name for use later in the vHBA definition. This value is important and should match the vHBA name.

Note:             vHBA-A is used for NA-fcp-lif01a and NA-fcp-lif02a, and vHBA-B is used for NA-fcp-lif01b and NA-fcp-lif02b.

10.  Add the appropriate World Wide Port Name (WWPN) as the target WWPN. You can obtain this value from NetApp after logging in and entering the following command on the NetApp controller: network interface show -vserver <vserver name> .

 

A screenshot of a computer screenDescription automatically generated

11.  Click Create.

12.  Repeat these steps three more times to add all the NetApp LIFs. You can rearrange the policies using the arrow keys if needed.

13.  After you have added all the boot devices, they should be listed on the Policy Details screen.

A screenshot of a computerDescription automatically generated

14.  Click Next.

Step 4: Management Configuration

1.    Next, configure management policy.

A screenshot of a cell phoneDescription automatically generated

Configure Cisco IMC access policy

Follow these steps to configure Cisco IMC access policy:

1.    Click Select Policy next to IMC Access and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-IMCPol).

 

A screenshot of a computerDescription automatically generated

3.    Click Next.

4.    Provide the in-band (or out-of-band) management VLAN ID (for example, 511)

A screenshot of a computerDescription automatically generated

5.    Select “Configure IPv4 address configuration” and click Select IP Pool to define a KVM IP address assignment pool.

6.    Click Create New in the menu on the right.

7.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-Pool).

8.    Select Configure IPv4 Pool and provide the information to define a pool for KVM IP address assignment.

A screenshot of a cell phone screen with textDescription automatically generated

Note:                The management IP pool subnet should be accessible from the host that is trying to open the KVM connection. In the example shown here, the hosts trying to open a KVM connection would need to be able to route to 192.168.160.0 subnet.

9.    Click Next.

10.  Unselect Configure IPv6 Pool.

11.  Click Create to finish configuring the IP address pool.

12.  Click Create to finish configuring the IMC access policy.

Configure local user policy

Follow these steps to configure local user policy:

1.    Click Select Policy next to Local User and the, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-LocalUser-Pol).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Verify that UCS Server (FI-Attached) is selected.

4.    Verify that Enforce Strong Password is selected.

A screenshot of a cell phone screen with textDescription automatically generated

5.    Click Add New User.

6.    Provide the username (for example, flexpodadmin), choose a role (for example, admin), and provide a password.

Note:                The username and password combination defined here will be used to log in to KVMs. The typical Cisco UCS admin username and password combination cannot be used for KVM access.

A screenshot of a cell phone screen with textDescription automatically generated

7.    Click Create to finish configuring the user.

8.    Click Create to finish configuring local user policy.

9.    Click Next.

Step 5: Storage Configuration

Click Next on the Storage Configuration screen. You will not make any changes to this configuration.

Step 6a: Network Configuration > LAN Connectivity

LAN connectivity policy defines the connections and network communication resources between the server and the LAN on the network. This policy uses pools to assign MAC addresses to servers and to identify the vNICs that the servers use to communicate with the network. Follow these steps to define LAN connectivity:

1.    Click Select Policy next to LAN Connectivity and then, in the pane on the right, click Create New.

Graphical user interface, applicationDescription automatically generated

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-LanConn).

A screenshot of a cell phone screen with textDescription automatically generated

This deployment uses four vNICs, as follows:

     00-vSwitch0-A: Fabric Interconnect A vNIC; assigned to vSwitch for management and Network File System (NFS) VLANs

     01-vSwitch0-B: Fabric Interconnect A vNIC; assigned to vSwitch for management and NFS VLANs

     02-VDS0-A: Fabric Interconnect A vNIC; assigned to VDS for vMotion and traffic VLANs

     03-VDS0-B: Fabric Interconnect A vNIC; assigned to VDS for vMotion and traffic VLANs

3.    To keep the vNIC placement simple, select Auto vNIC Placement.

4.    Click Add vNIC.

A screenshot of a cell phone screen with textDescription automatically generated

5.    Provide the name of the vNIC (for example, 00-vSwitch0-A).

A screenshot of a computer screenDescription automatically generated

Create MAC address pool for Fabric A

The MAC address pool has not been defined yet, so a new MAC address pool will be created now for Fabric A. This pool will be reused for all current (02-VDS0-A) and future Fabric-A vNICs.

1.    Click Select Pool under MAC Address Pool and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, MAC-Pool-A).

Graphical user interface, applicationDescription automatically generated

3.    Click Next.

4.    Provide the starting MAC address. The recommended prefix for MAC addresses is 00:25:B5:xx:xx:xx. As a best practice, in FlexPod some additional information is always coded into the MAC address pool for ease of troubleshooting. For example, in the starting address 00:25:B5:A4:0A:00, A4 is the rack ID and 0A indicates Fabric A.

5.    Provide the size of the MAC address pool (for example, 64).

Graphical user interface, application, TeamsDescription automatically generated

6.    Click Create to finish creating the MAC address pool.

7.    Back in the Add vNIC window, from the drop-down menu, choose A as the switch ID.

8.    For Consistent Device Naming (CDN), from the drop-down menu, choose vNIC Name.

9.    Verify that Failover is disabled. Failover will be provided by attaching multiple NICs to the VMware vSwitch and VDS.

A screenshot of a cell phone screen with textDescription automatically generated

Create Ethernet network group policy

The Ethernet policies have not been created yet, so these policies will be created now. These policies will be reused when additional vNICs are defined.

Ethernet network group policy defines the VLANs allowed for a particular vNIC. Two network group policies will be defined for this deployment:

    vSwitch0 network group policy to define the VLANs for vSwitch0 (management and NFS traffic)

    VDS network group policy to define the VLANs for VDS (application and VMware vMotion traffic)

1.    Click Select Policy under Ethernet Network Group Policy and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, vSwitch0-NetGrp-Pol).

A screenshot of a cell phoneDescription automatically generated

3.    Click Next.

4.    Enter the allowed VLANs (for example, 2,11,511,3050) and the native VLAN ID (for example, 2).

A screenshot of a cell phoneDescription automatically generated

5.    Click Create to finish configuring the Ethernet network group policy.

Create Ethernet network control policy

Ethernet network control policy is used to enable Cisco Discovery Protocol (CDP) and Link Layer Discovery Protocol (LLDP) for the vNICs. A single policy will be created here and reused for all the vNICs.

1.    Click Select Policy under Ethernet Network Control Policy and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, Enable-CDP-LLDP).

Graphical user interface, applicationDescription automatically generated

3.    Click Next.

4.    Enable Cisco Discovery Protocol and both Transmit and Receive under LLDP.

A screenshot of a cell phoneDescription automatically generated

5.    Click Create to finish creating Ethernet network control policy.

Create Ethernet QoS policy

Ethernet QoS policy is used to enable jumbo maximum transmission units (MTUs) for all the vNICs. A single policy will be created and reused for all the vNICs.

1.    Click Select Policy under Ethernet QoS and in then, the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, Jumbo-MTU-QoS).

Graphical user interface, text, applicationDescription automatically generated

3.    Click Next.

4.    Change the MTU, Bytes value to 9000.

A screenshot of a cell phone screen with textDescription automatically generated

5.    Click Create to finish setting up the Ethernet QoS policy.


 

Create Ethernet adapter policy

Ethernet adapter policy is used to set the interrupts and the send and receive queues. The values are set according to the best-practices guidance for the operating system in use.

1.    Click Select Policy under Ethernet Adapter and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, VMware-HighTrf).

Graphical user interface, application, TeamsDescription automatically generated

3.    Change the Interrupts, Receive Queue Count, Transmit Queue Count, and Completion Queue Count values to 11, 8, 1, and 9, respectively, as shown here.

A screenshot of a computerDescription automatically generated

4.    Verify all the policies are assigned to vNIC 00-vSwitch0-A.

Graphical user interface, applicationDescription automatically generated

5.    Click Add to add the vNIC.

A screenshot of a computer screenDescription automatically generated

Note:                Repeat all the step under Step 6a: Network Configuration > LAN Connectivity to create additional vNICs. Most of the policies created for first vNIC will be reused for the remaining vNICs. MAC-Pool-B and VDS0-NetGrp-Policy used for subsequent vNICs are covered in the following steps.

6.    MAC-Pool-B is used by vNICs 01-vSwitch0-B and 03-VDS0-B. When adding vNIC 01-vSwitch0-B, click Select Pool under MAC Address Pool and click Create New in the pane on the right. Select the Organization (for example FPV) and provide the name (for example MAC-Pool-B).

A screenshot of a cell phone screen with textDescription automatically generated

7.    Note that the same prefix 00:25:B5 is used for MAC Pool B, but 0B in the second-to-the-last octet signifies the these MAC addresses are assigned to vNICs associated with Fabric B.

Graphical user interface, applicationDescription automatically generated

8.    Note the 01-vSwitch0-B policy mappings on the Create LAN Connectivity Policy screen.

Graphical user interface, applicationDescription automatically generated

Create network group policy for VDS0

Network group policy for vNICs 02-VDS0-A and 03-VDS0-B is different because the VLANs used for the VDS are different.

1.    Define the network group policy for VDS0 when you add the 02-VDS0-A vNIC. Instead of selecting the existing network group policy, select Create New in the pane on the right.

A screenshot of a cell phone screen with textDescription automatically generated

2.    Define the correct VLANs associated with the VDS (the application traffic and vMotion traffic VLANs).

Graphical user interface, applicationDescription automatically generated

3.    Note the vNIC 02-VDS0-A policy mappings on the Create LAN Connectivity Policy screen.

Graphical user interface, application, TeamsDescription automatically generated

4.    Note the vNIC 03-VDS0-B policy mappings on the Create LAN Connectivity Policy screen.

Graphical user interface, application, TeamsDescription automatically generated

5.    Verify that all the vNICs have been added successfully before moving on to create SAN connectivity policies.

A screenshot of a cell phoneDescription automatically generated

6.    Click Create.

Step 6b: Network Connectivity > SAN Connectivity

A SAN connectivity policy determines the network storage resources and the connections between the server and the storage device on the network. This policy enables you to configure the vHBAs that the servers use to communicate with the SAN.

1.    Click Select Policy next to SAN Connectivity and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, AA04-6454-SanConn).

A screenshot of a cell phone screen with textDescription automatically generated

This deployment uses two vHBAs, as follows:

    vHBA-A: Fabric Interconnect A vHBA for SAN A

    vHBA-B: Fabric Interconnect B vHBA for SAN B

3.    To keep the vHBA placement simple, select Auto vHBAs Placement.

A screenshot of a cell phoneDescription automatically generated

Create the WWNN address pool

The WWNN address pools have not been defined yet, so you will now create a new WWNN address pool.

1.    Click Select Pool under WWNN Address Pool and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, WWNN-Pool).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Click Next.

4.    Provide the starting WWNN block address. The recommended prefix for WWNN addresses is 20:00:00:25:B5:xx:xx:xx. As a best practice, in FlexPod some additional information is always coded into the WWNN address pool for ease of troubleshooting. For example, in the address 20:00:00:25:B5:A4:00:00, A4 is the rack ID.

Graphical user interface, application, TeamsDescription automatically generated

5.    Click Create to finish creating the WWNN address pool.

Create the vHBA for SAN A

Now create a vHBA for SAN A.

1.    Click Add vHBA.

2.    Provide the name of the vNIC (for example, vHBA-A).

3.    For vHBA Type, choose fc-initiator from the drop-down menu.

4.    Choose Switch ID A from the drop-down menu.

Create the WWPN pool for SAN A

The WWPN address pool has not been defined yet, so you will now create a WWPN address pool for Fabric A.

1.    Click Select Pool under WWPN Address Pool and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, WWPN-Pool-A).

A screenshot of a cell phone screen with textDescription automatically generated

3.    Provide the starting WWPN block address for SAN A. The recommended prefix for WWPN addresses is 20:00:00:25:B5:xx:xx:xx. As a best practice, in FlexPod some additional information is always coded into the WWPN address pool for ease of troubleshooting. For example, in the address 20:00:00:25:B5:A4:0A:00, A4 is the rack ID and 0A signifies SAN A.

4.    Provide the size of the pool (for example, 32).

Graphical user interface, application, TeamsDescription automatically generated

5.    Click Create.


 

Create Fibre Channel network policy for SAN A

A Fibre Channel network policy governs the VSAN configuration for the virtual interfaces. VSAN 111 will be used for vHBA-A, and VSAN 112 will be used for vHBA-B.

1.    Click Select Policy under Fibre Channel Network and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, SAN-A-Network).

A screenshot of a cell phone screen with textDescription automatically generated

3.    For the scope, select UCS Server (FI-Attached).

4.    Under Default VLAN, provide the VSAN information (for example, 111).

Note:             The current GUI shows Default VLAN instead of Default VSAN due to a GUI defect.

Graphical user interface, applicationDescription automatically generated

5.    Click Create to finish creating the Fibre Channel network policy.


 

Create Fibre Channel QoS policy

The Fibre Channel QoS policy assigns a system class to the outgoing traffic for a vHBA. This system class determines the quality of service for the outgoing traffic. The Fibre Channel QoS policy used in this deployment uses default values and will be shared by both vHBA-A and vHBA-B.

1.    Click Select Policy under Fibre Channel QoS and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, FC-QoS).

Graphical user interface, application, TeamsDescription automatically generated

3.    For the scope, select UCS Server (FI-Attached).

4.    Do not change the default values on the Policy Details screen.

A screenshot of a cell phone screen with textDescription automatically generated

5.    Click Create to finish creating the Fibre Channel QoS policy.


 

Create Fibre Channel adapter policy

A Fibre Channel adapter policy governs the host-side behavior of the adapter, including the way that the adapter handles traffic. This validation uses the default values for the adapter policy, and the policy will be shared by both vHBA-A and vHBA-B.

1.    Click Select Policy under Fibre Channel Adapter and then, in the pane on the right, click Create New.

2.    Choose the organization from the drop-down menu (for example, FPV) and provide a name for the policy (for example, FC-Adapter).

A screenshot of a cell phone screen with textDescription automatically generated

3.    For the scope, select UCS Server (FI-Attached).

4.    Do not change the default values on the Policy Details screen.

A screenshot of a computer screenDescription automatically generated

5.    Click Create to finish creating the Fibre Channel adapter policy.

6.    Click Add to create vHBA-A.

Create the vHBA for SAN B

Repeat the preceding steps to add vHBA-B for SAN B.

1.    Use switch ID B for this vHBA. The WWPN pool and Fibre Channel network policy (VSAN) for this vHBA are unique, but the Fibre Channel QoS and Fibre Channel adapter policies defined previously for vHBA-A will be reused.

2.    Note the WWPN-Pool-B information used for this validation.

Graphical user interface, applicationDescription automatically generated

The recommended prefix for WWPN addresses is 20:00:00:25:B5:xx:xx:xx. As a best practice, in FlexPod some additional information is always coded into the WWPN address pool for ease of troubleshooting. For example, in the address 20:00:00:25:B5:A4:0B:00, A4 is the rack ID and 0B signifies SAN B.

A screenshot of a cell phoneDescription automatically generated

3.    Note the Fibre Channel network policy for SAN B used in this validation.

Graphical user interface, text, applicationDescription automatically generated

4.    For the scope, select UCS Server (FI-Attached) and enter the VSAN information (for example, 112) under Default VLAN.

Note:             The current GUI shows Default VLAN instead of Default VSAN due to a GUI defect.

Graphical user interface, applicationDescription automatically generated

5.    After adding all the vNICs and vHBAs, verify their placement by expanding the vNICs & vHBAs Placement option:

A screenshot of a video gameDescription automatically generated

6.    Click Next.


 

Step 7: Summary

On the summary screen, verify which policies are mapped to various settings and the status of the server profile. The server profile has not been deployed yet, so the status will be Not Deployed.

A screenshot of a cell phoneDescription automatically generated

Deploy the server profile

After verifying the settings on the server profile Summary screen, click Deploy and then Deploy again to deploy the server profile. You would see a task in progress in the top-right corner. You can click the task icon to view the details of the task in progress.

A screenshot of a cell phoneDescription automatically generated

Verify SAN and LAN addresses

After the server profile has been deployed successfully, gather the information about the MAC addresses assigned to vNICs and the WWPN addresses assigned to vHBAs by following these steps:

1.    Log in to Cisco Intersight portal.

2.    Go to CONFIGURE > Profiles and select the server profile you just deployed.

3.    In the main window, click LAN Connectivity.

A screenshot of a computerDescription automatically generated

4.    In the pane on the right, each NIC is listed along with the assigned MAC address. This information is useful for identifying the management (vSwitch0) vNICs for installing ESXi on the server and setting up the initial management access.

Related image, diagram or screenshot

5.    Click SAN Connectivity to gather the information about the WWPN address assigned to vHBA-A and vHBA-B. This information is required to set up Cisco MDS zoning and to map boot LUNs on NetApp. You can add this information can be added to Table 2 for quick reference.

Table 2.        Server profile MAC and WWPN addresses

Server profile

Interface

Address

SP-FlexPod-01

00-vSwitch0-A

00:25:B5:A4:0A:01

 

01-vSwitch0-B

00:25:B5:A4:0B:07

 

02-VDS0-A

00:25:B5:A4:0A:06

 

03-VDS0-B

00:25:B5:A4:0B:06

 

vHBA-A

20:00:00:25:B5:A4:0A:00

 

vHBA-B

20:00:00:25:B5:A4:0B:00

Note:      At this time, most of the policy changes prompt the user to redeploy the server profile. During this testing, because most of the features are still in technical preview, rebooting a server after redeploying the server profile resulted in the most consistent deployments.

Configure Cisco MDS zoning

The Cisco MDS configuration for zoning is no different than the typical Cisco MDS configuration in FlexPod. Refer to the Cisco MDS configuration for zoning in the FlexPod deployment guide: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3.html#_Toc46246585.

Configure NetApp LUNs

The NetApp LUN configuration is no different than the typical NetApp storage configuration in FlexPod. Refer to the NetApp configuration for LUN creation and LUN and igroup mapping in the FlexPod deployment guide:

·       https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3.html#_Toc46246513

·       https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/fp_dc_ontap_97_ucs_4_vmw_vs_67_U3.html#_Toc46246596

Deploy additional server profiles

When a server profile deployment is complete, you can deploy additional server profiles simply by cloning an existing server profile. An existing server profile can also be renamed before cloning (if required) by editing the profile and changing the name on the first screen. During this validation, the server profile created in the previous steps, SP-FlexPod, was renamed to SP-FlexPod-01 before additional clones (02, 03, etc.) were created.

Note:      Customers should verify various settings of the profile before creating additional clones and preferably test the server profile by installing and configuring an operating system.

 

To create additional server profiles by cloning, follow these steps:

1.    Go to Profile and click the Options icon (…). Choose Clone.

A screenshot of a video gameDescription automatically generated

2.    Under Server Assignment, keep the Assign Server selection and select an available server.

A picture containing monitor, road, screen, sittingDescription automatically generated

3.    Click Next.

4.    Provide the name of the new server profile by updating the Clone Name Prefix and updating the Clone Name.

A screenshot of a cell phone screen with textDescription automatically generated

5.    Click Clone & Deploy. When the clone process has been completed successfully, the new server profile will be deployed.

6.    Update Cisco MDS and NetApp with the appropriate WWPN information for zoning and mapping boot LUNs to enable boot from SAN.

Install VMware ESXi on a server profile

After a server profile has been deployed successfully, install an operating system by following these steps:

1.    Go to Servers and click the Options icon (…) and choose Launch the vKVM.

A screenshot of a stereoDescription automatically generated

Note:             Make sure that virtual machines and hosts trying to access the KVM can route to the management IP address pool.

2.    Log in using the username (for example, flexpodadmin) and password previously defined in the local user policy.

3.    In the new KVM tab on the browser, click Virtual Media and choose Activate Virtual Devices.

A picture containing graphical user interfaceDescription automatically generated

4.    Click Virtual Media again and choose Map CD/DVD.

5.    Browse to ESXi ISO and click Map Drive.

6.    In the Cisco Intersight portal, power-cycle the server:

    Click the Options icon (…) next to the server and choose Power Cycle.

A screenshot of a video gameDescription automatically generated

    Select Set One Time Boot Device and choose ISO (the label previously created for the CD/DVD) from the Boot Device drop-down menu.

Graphical user interface, application, TeamsDescription automatically generated

    Click Power Cycle.

7.    Go to the KVM tab. In the KVM window, you should see the server being power cycled. If the zoning and boot LUNs were configured correctly, you will see that the server has successfully discovered the boot LUN over all four paths.

8.    After ESXi has been loaded, navigate through the ESXi installer instructions. The installer should discover the NetApp boot LUN as an installation location.

Graphical user interface, text, application, emailDescription automatically generated

9.    Proceed with the ESXi installation. When installation is complete, unmap the installer by choosing the Deactivate Virtual Devices menu option under Virtual Media and reboot the server.

Note:             You can use the Cisco Validated Design for FlexPod to set up VMware vCenter and several other management tools.

 

 

 

Learn more