vCenter on HyperFlex

Cisco HX Data Platform deployment, including installation and cluster configuration and management, requires a vCenter server for day-to-day operations. Typically, vCenter is hosted on a server that is outside of (external to) the HX storage and compute cluster.


Important

The recommended and preferred HyperFlex configuration is to have vCenter on a server external to the HX storage cluster. See the Cisco HyperFlex Systems Installation Guide for VMware ESXi for complete HyperFlex installation and configuration information.

For those environments where you cannot dedicate an external physical server to the vCenter server application, you can install a vCenter VM on the HX storage cluster.


This document describes how to deploy and run an HX Data Platform configuration that has a vCenter VM running on the HX storage cluster, rather than on a server external to the HX storage cluster. When a vCenter VM is hosted on a HyperFlex cluster, it is referred to as a nested vCenter. The vCenter VM can be either a VMware vCenter Server Appliance (VCSA) or a Microsoft Windows vCenter on a Windows Server VM.

There are two methods for installing a vCenter VM on an HX storage cluster:

  • Installing a vCenter VM as part of the HX Data Platform installation

    This method allows you to install a vCenter VM after the HX storage cluster is installed. It does not require any additional hardware as compared to the second option.

  • Using a vCenter VM stored on an external server

    This method uses the typical and recommended external vCenter to install and configure the HX Data Platform and storage cluster, and then migrates the vCenter server to reside on the HX storage cluster.

    In this context, external server means a server that is not included in the HX storage cluster. For example, it could be on an NFS mount.

Requirements

The nested vCenter method requires:

  • HX Data Platform Installer version 2.6(1a) or later. Prior HX Data Platform versions are not supported.

  • vCenter to be installed inside a VM.

  • Compute-only nodes may be added post installation only after the HX storage cluster is registered to a vCenter server.

  • Cluster expansion with additional HyperFlex nodes may be performed only after the HX storage cluster is registered to a vCenter server.

  • When installing vCenter, select the embedded Platform Services Controller option. An external Platform Services Controller is not supported.

Known Constraints

When vCenter is installed on a VM within the HX Data Platform storage cluster, there are some vCenter-related limitations:

  • For remote office / branch office (ROBO) deployments using Cisco HyperFlex Edge, you may use either deployment method.

  • Nested vCenter on a stretched cluster deployment is not supported.

  • Deploying Stretched Cluster without an instance of vCenter is not supported.

  • vCenter has limited auto-start capability.

    vCenter installed on a VM in the HX storage cluster does not always automatically restart. See Recovering an HX Storage Cluster after a vCenter Shutdown.

    • If the HX storage cluster is properly configured for high availability (HA), vCenter automatically restarts if the node that hosts it fails.

    • If the entire HX storage cluster suffers a power interruption, HA does not restart vCenter and manual intervention is needed.

    • If the HX storage cluster is gracefully brought down, manually initiate vCenter power-on from the last local ESXi host to which it was registered.

  • As each HyperFlex node is brought down during a rolling upgrade, the vCenter VM migrates automatically if DRS is enabled. If DRS is not enabled, the vCenter VM must be manually migrated during upgrades.

  • Only online upgrade is supported while hosting a nested vCenter. Do not use the offline upgrade option.

  • When the HX storage cluster is down or must shut down, then any support-related operations completed through vCenter, must be performed directly on ESXi hosts.

  • Do not take a snapshot of the vCenter VM due to certain limitations. For further details, refer to the VMware KB article, VMware VirtualCenter Server service fails due to a quiesced snapshot operation on the vCenter Server database virtual machine (2003674).

    This applies to scheduled snapshots and one-time-only snapshots. Do not schedule snapshots for a folder or resource pool that includes the vCenter VM.

  • Snapshots cannot be deleted when the HX storage cluster is in the ENOSPACE state.

    Deleting snapshots is frequently used to reclaim space for an HX storage cluster. When an HX storage cluster enters ENOSPACE, the VMs hosted on that HX storage cluster can no longer perform writes. This includes the vCenter VM. Therefore, the vCenter hosted on the HX storage cluster cannot perform actions. This includes deleting snapshots to help bring the storage cluster out of ENOSPACE. Use an ESXi host command-line option to delete snapshots.

  • HX snapshot of nested vCenter is not supported.

Install a vCenter VM as Part of the HX Data Platform Installation

To Install a vCetnter VM as part of the HX Data Platform Installation, complete the following:

  1. Meet the Requirements and understand the Known Constraints.

  2. Install and configure the HX Data Platform and HX Storage Cluster (Nested vCenter) using one of the following options:

  3. Install a vCenter VM on the HX Storage Cluster.

Install and Configure HX Data Platform and HX Storage Cluster (Nested vCenter) via HX Data Platform Installer

Perform the following steps to install a nested vCenter on an HX storage cluster using the Cisco HX Data Platform Installer.

Procedure


Step 1

Verify that your system meets the HX Data Platform system requirements.

Step 2

Log in to the HX Data Platform Installer. If needed, read and accept the End User License Agreement.

Step 3

On the Workflow page, select Cluster Creation with HyperFlex (FI) or Cluster Creation with HyperFlex Edge, and click Continue.

Step 4

On the Credentials page, enter the UCS Manager (only FI workflow) and HX Data Platform (hypervisor) credentials, and click Continue.

Leave all three vCenter fields blank. The vCenter server is registered as a post-installation task.

Step 5

When prompted to confirm that the installation is being started without a vCenter, click Continue.

Step 6

On the Server Selection page, select the servers under Unassociated to include in the HX storage cluster, and click Continue.

Step 7

(Only FI workflow) On the UCS Manager Configuration page, enter the UCS Manager configuration information, and click Continue.

UCS Manager configuration information includes the VLAN, MAC pool, subnet, gateway, iSCSI storage, FC storage, UCS firmware, HyperFlex cluster name, and org name. Fill out these fields normally as required for any HyperFlex storage cluster.

Step 8

(Only FI workflow) On the Hypervisor Configuration page, enter common hypervisor settings, such as subnet mask, gateway, DNS server, static IP addresses, and hostnames for the HyperFlex nodes, and click Continue.

Step 9

On the IP Addresses page, for each HyperFlex node, complete the listed fields for Hypervisor Management, Data IP addresses, subnet masks, and gateways. For the IP addresses, specify the network that the IP address should belong to (Data Network and Management Network).

Step 10

On the Cluster Configuration page, enter the HX storage cluster settings, such as HX storage cluster name, controller VM credentials, Data Replication Factor, DNS and NTP servers, and Auto Support (ASUP). Skip the vCenter configuration inputs and leave the fields blank.

See the Cisco HyperFlex Systems Installation Guide for VMware ESXi for complete deployment and cluster creation steps.

Step 11

Click Start.

A Progress page displays the progress of various configuration tasks.
Caution 

Do not skip validation warnings without a complete understanding of the triggered issue.

See the "Warnings" section of Cisco HyperFlex Systems Installation Guide for VMware ESXi for more details.

Step 12

When cluster creation completes and the summary page displays, click Launch HyperFlex Connect to launch the Cisco HyperFlex Connect user interface, and log in with credentials for local/root (username) and the controller VM password that you used on the Cluster Configuration page.


What to do next

Proceed to Install a vCenter VM on the HX Storage Cluster.

Install and Configure HX Data Platform and HX Storage Cluster (Nested vCenter) via Cisco Intersight

Perform the following steps to install a nested vCenter on an HX storage cluster using the Cisco HX Data Platform Installer.

Procedure


Step 1

Verify that your system meets the HX Data Platform system requirements.

Step 2

Log in to the Cisco Intersight with CEC Credentials or with SSO.

Step 3

In the Devices Details page, select Claim a New Device and claim the servers you will deploy in the cluster. For additional information, see See the Cisco HyperFlex Systems Installation Guide for Cisco Intersight, Chapter: Deploying HyperFlex Edge Cluster.

Step 4

In the left navigation panel, click Profiles.

Step 5

In the Profiles page, make sure that the HyperFlex Cluster Profiles tab is selected and click Create HyperFlex Cluster Profile to launch the Create HX Cluster Profile installation wizard. by selecting Create HyperFlex Cluster Profile.

  1. Complete the following fields: Organization, Name, HX Date Platform Version, Type radio button, select Cisco HyperFlex Edge, and optional Tags and Description.

Step 6

Click Next.

Step 7

In the Cluster Configuration Page, complete the following fields: Security, DNS, NTP, and Time Zone. Leave the vCenter Configrutaion field blank.

Note 

HyperFlex will be deployed without being registered in vCenter. The vCenter Server is registered as a post-installation task. Configure the following fields as needed: Storage Configuration, Auto Support, IP Host, Network Configuration, Proxy Setting and Hyperflex Storage Network Policies.

Step 8

In the Nodes Assignment page, you can assign nodes now or optionally, you can choose to assign the nodes later. To Assign nodes, click the Assign nodes check box and select the node you want to assign.

Step 9

Click Next.

Step 10

In the Nodes Configuration page, you can view the IP and Hostname settings that were automatically assigned. Additionally, you can enter the Cluster Management IP Address and the MAC Prefix Address. Enter the MAC address prefix range in the form of 00:25:B5:XX. The cluster management IP address must belong to the management subnet.

Step 11

In the Summary page, you can view the cluster configuration and node configuration details. Review and confirm that all information entered is correct. Ensure that there are no errors triggered under the Errors/Warnings tab.

Step 12

Click Validate and Deploy to begin the deployment. Optionally, click Validate, and click Save & Close to complete deployment later. The Results page displays the progress of the various configuration tasks.


What to do next

Proceed to Install a vCenter VM on the HX Storage Cluster.

Install a vCenter VM on the HX Storage Cluster

Install a vCenter VM on the HX storage cluster only after completing your install and configuration via HX Data Platform Installer or Cisco Intersight .

Before you begin

You must have the following to perform this task:

  • HX Data Platform fully installed with an HX storage cluster created

  • VMware vCenter installer files (VCSA ISO or Windows installer executable)

Procedure


Step 1

Log in to HX Connect.

Step 2

Click Dashboard > Datastores > Create Datastore, enter Datastore Name (for example, VC-DS01, Size, and Block Size, and click Create Datastore.

On the Datastores page, the new datastore must display as MOUNTED before continuing.

Step 3

Using the vCenter installer wizard, deploy a new vCenter to any of the HyperFlex servers in the cluster.

Use the new datastore for persistent storage and select an appropriate port-group for the network. This port-group must have network access to the HyperFlex cluster management IP address and all ESXi management IP addresses.

Step 4

After successfully deploying vCenter, log in to the vSphere Web Client using a supported browser and do the following:

  1. Create a new datacenter; for example, DC1.

  2. Create a new cluster in the new datacenter (for example, Cluster1) leaving HA and DRS disabled. They are enabled after running the post_install script.

  3. Depending on which VMware vSphere ESXi version you are running:

    If running VMware vSphere version ESXi 6.5 or earlier, manually add all HyperFlex servers to the newly created vCenter cluster.

    If running VMware vSphere version ESXi 6.7 or later, first add the HyperFlex node to the destination datacenter and then move into it to the newly created vCenter cluster.

  4. Log out of the vSphere Web Client using the option in the top-right menu.

Step 5

Register the HX storage cluster to the newly configured vCenter server:

  1. Log in to any controller VM using ssh, and run the following command:

    stcli cluster reregister --vcenter-datacenter <DC Name> --vcenter-cluster <Cluster Name>  --vcenter-url <URL or IP of vCenter> --vcenter-user <admin username>
    Note 

    Run this command on any controller VM only once. After successfully running the command, log in to the vSphere Web Client and verify that the HX Data Platform Plug-in displays in the extension list.

    The following sample command shows successful output:

    
    root@SpringpathControllerE1M9XZGRFM:~# stcli cluster reregister --vcenter-datacenter Edge --vcenter-cluster HX-01 --vcenter-url 10.1.1.70 --vcenter-user administrator@vsphere.local
    
    Reregister StorFS cluster with a new vCenter ...
    
    Enter NEW vCenter Administrator password:***********
    Cluster reregistration with new vCenter succeeded
    
    
  2. After successfully running the command, log in to vSphere Web Client and verify the HX Data Platform Plug-in displays in the extension list.

Step 6

Run the post_install script from the installer VM and follow the prompts. For more information, see Post Installation Tasks in the Cisco HyperFlex Systems Installation Guide for Cisco Intersight.

This script sets up certain additional cluster settings, including vMotion.

Step 7

Create DRS pin rules.

These steps place the vCenter VM on a known host making troubleshooting and manual restart easier. You may need to search for the vCenter VM on all hosts to perform any manual steps, such as, bringing up the vCenter VM after a full shutdown. See VMware documentation for additional information.

Note 

If you are using a license that does not support DRS, you do not need to perform this step.

  1. Click cluster > Manage > Settings > Configuration > VM/Host Groups.

  2. Click Add, and select Type: VM group.

  3. Click Add, select the vCenter VM, click OK, and click OK again.

  4. Click Add, select Type: Host group, add an ESXi host, click OK, and click OK again.

  5. Click VM/Host Rules, and select Type: Virtual Machines to Hosts.

  6. Select the VM group created earlier, select Should run on hosts in group, select the Host group created earlier, and click OK.

  7. Configure the vSphere HA Advanced Options parameter das.respectvmhostsoftaffinityrules.

    Note 

    This parameter determines if vSphere HA enforces VM-VM anti-affinity rules. The default value is "true" and rules are enforced even if vSphere DRS is not enabled. In this case, vSphere HA does not fail over a virtual machine if doing so violates a rule, but it issues an event reporting there are insufficient resources to perform the failover. This option can also be set to "false", whereby the rules are not enforced.


Deploying vCenter on HyperFlex Using an External Server

Complete the following tasks.

  1. Meet the Requirements and understand the Known Constraints.

  2. Installing and Configuring HX Data Platform and HX Storage Cluster (External vCenter).

    This is the standard HX Data Platform installation and deployment using an external vCenter that is on a VM.

  3. Migrating a vCenter VM to an HX Storage Cluster from an External Server.

Installing and Configuring HX Data Platform and HX Storage Cluster (External vCenter)

Installing and configuring HX Data Platform and deploying the HX storage cluster using a vCenter server that is external to the HX storage cluster is the standard deployment method.

This task follows the standard HyperFlex installation and configuration described in Cisco HyperFlex Systems Installation Guide for VMware ESXi.

Procedure


Step 1

Verify that your system meets the HX Data Platform system requirements.

Step 2

Log in to the HX Data Platform Installer. If needed, read and accept the End User License Agreement.

Step 3

On the Workflow page, select Cluster Creation, and click Continue.

Step 4

On the Credentials page, enter the UCS Manager, vCenter, and HX Data Platform (hypervisor) credentials, and click Continue.

Step 5

On the Server Selection page, select the servers under Unassociated to include in the HX storage cluster, and click Continue.

Step 6

On the UCS Manager Configuration page, enter the UCS Manager configuration information, and click Continue.

UCS Manager configuration information includes the VLAN, MAC pool, subnet, gateway, iSCSI storage, FC storage, UCS firmware, HyperFlex cluster name, and org name. Fill out these fields normally as required for any HyperFlex storage cluster.

Step 7

On the Hypervisor Configuration page, enter common hypervisor settings, such as subnet mask, gateway, DNS server, static IP addresses, and hostnames for the HyperFlex nodes, and click Continue.

Step 8

On the IP Addresses page, for each HyperFlex node, complete the listed fields for Hypervisor Management, Data IP addresses, subnet masks, and gateways. For the IP addresses, specify the network that the IP address should belong to (Data Network and Management Network).

Step 9

On the Cluster Configuration page, enter the HX storage cluster settings, such as HX storage cluster name, controller VM credentials, Data Replication Factor, vCenter information, DNS and NTP servers, and Auto Support (ASUP).

See the Cisco HyperFlex Systems Installation Guide for VMware ESXi for complete deployment and cluster creation steps.

During this process, point the HX Data Platform Installer to the vCenter VM installed on the external source. This vCenter continues to run during HX Data Platform installation.

Step 10

Click Start.

A Progress page displays the progress of various configuration tasks.
Caution 

Do not skip validation warnings without a complete understanding of the triggered issue.

See the "Warnings" section of Cisco HyperFlex Systems Installation Guide for VMware ESXi for more details.

Step 11

Run the post_install script from the installer VM and follow the prompts. For more information on running the post install script, see Run Post Installation Script in the Cisco HyperFlex Systems Installation Guide for VMware ESXi, Release 4.0.

This script sets up certain additional cluster settings, including vMotion.


What to do next

Proceed to Migrating a vCenter VM to an HX Storage Cluster from an External Server.

Migrating a vCenter VM to an HX Storage Cluster from an External Server

This task assumes that you completed the standard HX Data Platform deployment, which includes installing the HX Data Platform Installer and configuring the UCS Manager and ESXi host. You can find instructions in Installing and Configuring HX Data Platform and HX Storage Cluster (External vCenter) from this tech note or the Cisco HyperFlex Systems Installation Guide for VMware ESXi.

Before you begin

You must have the following to perform this task:

  • HX Data Platform fully installed with an HX storage cluster created

  • vCenter in a VM on an external server

Procedure


Step 1

From the HX Data Platform Plug-in, create a new HyperFlex datastore; for example, ds1.

Step 2

Perform storage vMotion from the current external location to a new HyperFlex datastore through vCenter.

If the CPU families between the source and destination hosts are compatible, perform a manual vMotion of the vCenter VM. This option does not require that the vCenter is powered off.

Otherwise, complete the following steps. See VMware KB article Moving a virtualized vCenter Server virtual machine between ESXi/ESX hosts with different processor types (2058684).

  1. Connect directly to the external ESXi server hosting the vCenter VM.

  2. Power off the vCenter VM.

  3. Click File > Export > Export OVF Template.

  4. Connect directly to any HyperFlex ESXi server.

  5. Click File > Deploy OVF Template.

  6. Power on the vCenter VM.

Step 3

If not already done, configure vMotion interfaces to allow DRS to move VMs.

  1. Log in to vSphere, and view Networking.

    From vSphere Home, click vCenter Inventory Lists Resources > Resources > Hosts > host > Manage > Networking > VMKernel adapters.

  2. Click the Add host networking icon.

    Through the Add Networking wizard:

    1. Select VMKernel Network Adapter, and click Next.

    2. Select an existing standard switch, click Browse, select the Switch, select vMotion, and click OK.

    3. Enter a Network label name, define the proper VLAN ID, accept TCP/IP stack as the default unless you require L3 for your vMotion network, check vMotion traffic, and click Next.

    4. Enter a static IPv4 address, click Next, and click Finish.

Step 4

Create DRS pin rules.

These steps place the vCenter VM on a known host making troubleshooting and manual restart easier. You may need to search for the vCenter VM on all hosts to perform any manual steps, such as, bringing up the vCenter VM after a full shutdown. See VMware documentation for additional information.

Note 

If you are using a license that does not support DRS, you do not need to perform this step.

  1. Click cluster > Manage > Settings > Configuration > VM/Host Groups.

  2. Click Add, and select Type: VM group.

  3. Click Add, select the vCenter VM, click OK, and click OK again.

  4. Click Add, select Type: Host group, add an ESXi host, click OK, and click OK again.

  5. Click VM/Host Rules, and select Type: Virtual Machines to Hosts.

  6. Select the VM group created earlier, select Should run on hosts in group, select the Host group created earlier, and click OK.

  7. From the VM/Host Rules > vSphere HA Rule Settings section, click Edit.

  8. Check vSphere HA should respect VM to Host affinity rules during failover.


Shut Down Nested vCenter

This section captures the procedure to shutdown a nested vCenter within a cluster.

Procedure


Step 1

Shutdown all User VMs on a cluster.

Step 2

Shutdown vCenter.

Step 3

Note down the name of host on which vCenter is running, as the vCenter VM must be manually started later.

Step 4

Stop Storfs on all controllers for HXDP version 4.5 and later.

priv stop storfs
For 4.0x HXDP versions and earlier
  stop storfs
Step 5

Manually power off stCTLVMs from all ESXi hosts.

Step 6

Put all individual hosts in maintenance mode, by running the following command:

esxcli system maintenanceMode set -e true
Step 7

Shutdown the ESX hosts.

Step 8

Boot up the ESX hosts.

Step 9

Exit all hosts from maintenance mode, by running the following command:

esxcli system maintenanceMode set -e false
Step 10

Manually start the stCTLVMs (as vCenter is down, they may not start automatically).

Step 11

Verify if Storfs is running on each controller, by running the following command:

# pidof storfs

If the pidof command does not return any output, start Storfs by running the following command:

For HXDP version 4.5 and later
priv start storfs
For HXDP versions 4.0x and earlier
start storfs
Step 12

From the controller, check for the cluster status by running the following command:

sysmtool --ns cluster --cmd info

Wait for cluster to be healthy.

Step 13

Power on vCenter from the host (use the host name note down in step 3).

Wait for vCenter to be up. Check if the cluster is healthy by running the following command:

stcli cluster info  | grep -A 1 vCluster

After vCenter is up and running, you will get a state of online from this command.

Note 

If the stcli cluster storage-summary command fails and the cluster is in the healthy state, start the cluster using the stcli cluster start command.

Step 14

Power on all User VMs.


Recovering an HX Storage Cluster after a vCenter Shutdown

Typically, if vCenter shuts down or is forced to shut down, it automatically restarts. However, when the vCenter VM is hosted on an HX Data Platform node, manual steps may be required to completely recover the HX storage cluster.

Recovering an HX Storage Cluster after a Power Loss

Procedure


Step 1

Power on the ESXi hosts.

ESXi hosts power on the VMs on the servers, including the controller VMs.

Step 2

Verify the HX storage cluster is online from the HX Data Platform perspective:

  • Verify that the HX datastores are available on the ESXi host.
  • From the ESXi host command line, run the following command.
    # sysmtool --ns cluster --cmd info
Step 3

Power on the vCenter VM and wait for it to be ready.

To verify that the vCenter VM is ready, log in to the vSphere Web Client in a browser.

Step 4

Log in to the controller VM, and run the following command to verify the HX storage cluster is online.

# stcli cluster info

Example response

vCluster:
state : online

Recovering an HX Storage Cluster after an Administrative Shutdown

In some shutdown and failure scenarios, the vCenter VM may remain powered on but in an unresponsive state due to an offline HX storage cluster. It is important to completely power off the vCenter VM and bring the HX storage cluster online before restarting the vCenter VM.

Procedure


Step 1

Locate the vmid of the vCenter VM on the ESXi host.

# vim-cmd vmsvc/getallvms
Step 2

Power off the vCenter VM using the vmid.

# vim-cmd vmsvc/power.off <vmid>
Step 3

Log in to the controller VM and run the following command to recover the HX storage cluster from the HX Data Platform perspective.

# stcli cluster start
Step 4

Verify the HX storage cluster is online from the HX Data Platform perspective:

  • Verify that the HX datastores are available on the ESXi host.
  • From the ESXi host command line, run the following command.
    # sysmtool --ns cluster --cmd info
Step 5

Power on the vCenter VM and wait for it to be ready.

To verify that the vCenter VM is ready, log in to the vSphere Web Client in a browser.

Step 6

Log in to the controller VM, and run the following command to verify the HX storage cluster is online.

# stcli cluster info

Example response

vCluster:
state : online

Communications, Services, Bias-free Language, and Additional Information

  • To receive timely, relevant information from Cisco, sign up at Cisco Profile Manager.

  • To get the business impact you’re looking for with the technologies that matter, visit Cisco Services.

  • To submit a service request, visit Cisco Support.

  • To discover and browse secure, validated enterprise-class apps, products, solutions and services, visit Cisco Marketplace.

  • To obtain general networking, training, and certification titles, visit Cisco Press.

  • To find warranty information for a specific product or product family, access Cisco Warranty Finder.

Documentation Feedback

To provide feedback about Cisco technical documentation, use the feedback form available in the right pane of every online document.

Cisco Bug Search Tool

Cisco Bug Search Tool (BST) is a web-based tool that acts as a gateway to the Cisco bug tracking system that maintains a comprehensive list of defects and vulnerabilities in Cisco products and software. BST provides you with detailed defect information about your products and software.

Bias-Free Language

The documentation set for this product strives to use bias-free language. For purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on standards documentation, or language that is used by a referenced third-party product.