Set Up Clusters with Mixed CPUs

This chapter describes how to add HX nodes with different Intel CPU versions on the same FI.

Overview

HyperFlex supports Intel v3 and Intel v4 CPUs on the same fabric interconnect. Enable VMware Enhanced vMotion Compatibility (EVC) to migrate virtual machines between Intel v3 and Intel v4 CPUs. After EVC is enabled, all hosts in the HyperFlex cluster are configured to give them a baseline with features of the lower model of CPU. Identical CPU features are exposed to virtual machines regardless of which host they are running on, so virtual machines can be migrated between hosts in a HyperFlex cluster. This ensures CPU compatibility for vMotion even if the hosts have different underlying hardware.

Prerequisites for Using Mixed CPUs

  • You can only use CPUs from a single vendor in an EVC-enabled cluster. You cannot add a host from a different vendor into an EVC-enabled cluster.

  • When enabling EVC for a cluster with Intel processors of the Xeon E3 or Xeon E5 family that have different revision numbers (v2, v3, or v4), an EVC baseline is required.

  • Enable advanced virtualization CPU features in the BIOS if they are available. Otherwise, it can cause problems in enabling EVC, as EVC compatibility checks may fail to detect the features that are expected to be present for a particular CPU.

  • Migration of virtual machines using vMotion may fail, even if they are within an EVC cluster under the following scenarios:

    • When a host is not connected to a vCenter Server system.

    • When a host is not configured for vMotion.

    • If the virtual machine does not reside on storage shared by the source and destination hosts.

CPU Compatibility with EVC Mode

To determine the Enhanced vMotion Compatibility (EVC) modes compatible with your CPU, search the VMware Compatibility Guide. Search for the server model or CPU family, and click the entry in the CPU Series column to display the compatible EVC modes.

Finding the Current EVC Mode for a Host

Each EVC mode corresponds closely to the features available in processors with the same name.

Using vSphere Web Client

  1. From the vSphere Web Client Navigator, select Hosts and Clusters > HX Cluster > Summary. The Summary tab indicates whether EVC is enabled, and displays the current EVC mode for the host.

  2. Click the blue icon next to the EVC mode to display a list of all the supported EVC modes for the host.

Using VMware Shared Utilities Tool

VMware provides a free CPU identification utility that displays the compatible EVC modes, in addition to other CPU features. You can download this utility and boot the host from the ISO image from Shared Utilities.

Enable Enhanced vMotion Compatibility (EVC) on an Existing Cluster

Enable EVC to ensure that migration with vMotion is possible between hosts in the cluster. EVC mode is required when mixing different CPU families within the same HyperFlex cluster. Once EVC mode is enabled, only hosts that meet the minimum requirements of the configured EVC mode may be added to the cluster. EVC mode can be enabled, even during cluster expansion without disruption.


Note

  • EVC is disabled by default. It can be enabled in the cluster settings under VMware EVC.

  • This is not an HX Data Platform constraint but rather a VMware limitation. For further details, refer to the VMware KB article EVC and CPU Compatibility FAQ (1005764).


Note

  • If you are enabling EVC mode, verify that EVC mode supports Advanced Encryption Standard New Instructions (AES-NI).

  • This is not an HX Data Platform constraint but rather a VMware limitation. For further details, refer to the VMware KB article EVC and CPU Compatibility FAQ (1005764).

There are two paths to consider, uniform clusters that you want to add newer generation servers to, and existing clusters with mixed generations of servers.

Adding Newer Generation Servers to Uniform Clusters

If the cluster is currently uniform and you want to add newer generation servers to the cluster, you can enable EVC online and without disruption by selecting the current generation EVC mode in VC. Then proceed normally with expansion (converged or compute only). It is imperative the EVC mode is set before any expansion is attempted.

To enable EVC mode on a uniform cluster prior to performing a cluster expansion, perform the following steps:

Procedure

Step 1

Enable Enhanced vMotion Compatibility (EVC) in the HX cluster.

  1. From the vSphere Web Client Navigator, select Hosts and Cluster > Datacenter > HX Cluster.

  2. Select the cluster for which you want to enable EVC. In the Work pane, click the Manage or Configure tab. Select VMware EVC.

  3. Click the Edit button and select the desired EVC mode. Click Ok.

Step 2

Proceed with compute only or converged node expansion using the HyperFlex installer.

Adding Mixed or Older Generation Servers to Existing Clusters

The cluster already has mixed generations of servers OR you wish to add older generation servers to an existing cluster (compute only nodes).


Note

These steps should be followed if EVC mode was not enabled during the cluster expansion workflow with newer nodes.

To add an older generation server to an existing cluster perform the following steps:

Procedure

Step 1

Ensure that the HyperFlex cluster is healthy, and all the nodes are online before you start.

  • Using vSphere Web Client

    From the vSphere Web Client Navigator, select Home > Global Inventory Lists > Cisco HyperFlex Systems > Cisco HX Data Platform > HX Cluster > Summary.

    Example response:

    Operational Status: Online
Resiliency Status: Healthy
  • Using the Controller VM

    In the controller VM, run the command # hxcli cluster info.

    Example response:

    healthstate: healthy
state: online

Step 2

Power off all non-storage controller virtual machines.

Step 3

Log into one storage controller VM and execute the command hxcli cluster shutdown. Wait until the execution is complete.

Step 4

Shut down all the storage controller VMs.

  1. From the vSphere Web Client Navigator, select VMs and Templates > vCenter server > Datacenter > Discovered virtual machine > Virtual Machines > controller_vm.

  2. Right-click the controller_vm or from the Actions menu, select Power > Shut Down Guest OS.

Step 5

Put each HX host in maintenance mode.

  1. From the vSphere Web Client Navigator, select Hosts and Cluster > Datacenter > HX Cluster > node.

  2. Right-click the node and select Maintenance Mode > Enter Maintenance Mode.

    Note

     

    Do not use the Cisco HX Maintenance Mode menu for this operation.

Step 6

Enable Enhanced vMotion Compatibility (EVC) in the HX cluster.

  1. From the vSphere Web Client Navigator, select Hosts and Cluster > Datacenter > HX Cluster.

  2. Select the cluster for which you want to enable EVC. In the Work pane, click the Manage or Configure tab. Select VMware EVC.

  3. Click the Edit button and select the desired EVC mode. Click Ok.

Step 7

Exit maintenance mode.

  1. From the vSphere Web Client Navigator, select Hosts and Cluster > Datacenter > HX Cluster > node.

  2. Right-click the node and select Maintenance Mode > Exit Maintenance Mode.

    Note

     

    Do not use the Cisco HX Maintenance Mode menu for this operation.

Step 8

The controller VMs should automatically power on after the host exists maintenance mode. If the controller VMs do not power on automatically, do the following:

  1. From the vSphere Web Client Navigator, select VMs and Templates > vCenter server > Datacenter > ESXi Agents > Virtual Machines > controller_vm.

  2. Right-click the controller_vm or from the Actions menu. Select Power > Power On or Power > Power ON.

Step 9

Make sure all controller VMs are fully booted. Then, log in to one of the controller VMs, and run the command hxcli cluster start.

Step 10

Make sure that all datastores are mounted from vCenter HX plug-in and ensure that the cluster is healthy.

Step 11

Start the user VMs.

Step 12

Proceed with compute only expansion using the HyperFlex installer.