Preinstallation Checklist

Network Requirements

10 Gbps dedicated, 5-ms Round-Trip Time (RTT) latency between the two active sites is required.
100 Mbps, 200 ms-RTT worst case latency for 16 kilobyte packet sizes between the active sites and witness site is required.
Existing fabric interconnects are supported, provided the fabric interconnects support M5 servers.
User VMs should be capable of vMotioning to any site, without impacting external network connectivity to these VMs.
Similar to regular HX Clusters, two separate IP subnets are required—Both over Stretch L2. One subnet for data traffic and one for management traffic, with the management subnet reachable from vCenter and the witness node.
Static IP address assigned to Witness VM, can be changed only with cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM.

Network Topology

Witness Requirements

Configure and enable NTP on all servers.
An independent third witness site is required.
A separate witness is required for each cluster.
Both the main sites must have connectivity to the third witness site with a minimum bandwidth of 100 MBPS, 200 ms-RTT worst case latency for 16 kilobyte packet sizes.
Site must have the capability to deploy and run Open Virtualization Format (OVF) image.

Network Latency Requirements for the Witness VM

The HyperFlex Stretched Cluster solution requires that the Witness VM be located in a third site to ensure that a storage site failure does not affect the Witness VM.
The solution can support a Witness bandwidth as low as 100 Mbps, 200 ms-RTT worst case latency for 16 kilobyte packet sizes.
Latency to the witness impacts site failure times, and it is recommended that for larger clusters with significant load and data, to have RTT times in the order of 10ms or lower.

Fabric Interconnect and Node Requirements

Symmetric configuration is required across both sites.
There must be a minimum of two converged nodes on each site.
A maximum of 16 converged nodes on each site is supported. Ensure that both sites have the same number of converged nodes.
There must be a redundant fabric interconnect configuration on each site.
Converged nodes have to be M5 nodes.
Ensure that the Fabric Interconnect pair is of the same model in the same domain.
Compute-only nodes are supported.

VMware Requirements

VMware Enterprise Plus edition with HyperFlex Stretched Cluster is highly recommended to ensure proper failover behavior and guarantee high performance during normal operations. While it is possible to run Stretched Cluster without VMware Enterprise Plus edition, the advanced DRS features such as site affinity will not be available, negating some of the intended operational functions of Stretched Cluster.
Use a single vCenter for both sites.
The vCenter can be a VM running at the same site as the witness.
Nested vCenter is not supported in Stretched Cluster.
The vCenter must be configured independently for High Availability, as required.

UCS Manager Requirements

Two separate, manually synchronized fabric interconnect domains are required.

VLAN Requirements

IP addresses for nodes on both sites are required.
Stretched VLANs across both sites are required.

Stretch Witness

Required only when deploying HyperFlex Stretched Cluster.


Port Number	Service/Protocol	Source	Port Destinations	Essential Information
2181 2888 3888	Zookeeper/TCP	Witness	Each CVM Node	Bidirectional, management addresses
8180	Exhibitor (Zookeeper lifecycle)/TCP	Witness	Each CVM Node	Bidirectional, management addresses
80	HTTP/TCP	Witness	Each CVM Node	Potential future requirement
443	HTTPS/TCP	Witness	Each CVM Node	Potential future requirement

Deploying the Witness Node

Important

A HyperFlex witness node is mandatory in a stretched cluster environment to achieve quorum in case of total failure in any of the sites or when the network link between the sites encounters a failure.
In case of accidental deletion or loss of Witness VM, to replace the Witness VM please contact Cisco TAC.

The Witness VM requires a Static IP address that cannot be changed without cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM.

Note

If the IP address is requested from a DHCP server please verify that the clustering service is running after deploying the Witness VM, perform the following steps:

Check for the clustering service by running the following command after every reboot of the Witness VM:

service exhibitor status
If the output does not show the service as running, then restart exhibitor using the following command after ensuring that the witness does have an IP address:

service exhibitor restart

The following procedure details the steps to follow, for deploying a witness virtual node on a physical ESXi host.

Before you begin

Attention

HyperFlex Witness node version 1.0.2 is supported from 3.5(1a) and later releases.

The Witness VM must be deployed on an ESXi server that has sufficient hosting capability. The Witness VM requires 4 vCPUs, 8GB of memory, and 40 GB of disk space.
Ensure that the virtual network on this ESXi host is reachable from both the stretched cluster sites.
Download the HyperFlex witness node on to your desktop or host that runs vSphere Web Client from Download Software.
```
Example:
HyperFlex-Witness-1.0.2.ova
```
High Availability is optional for the witness node.

Procedure

Step 1

Log in to vSphere Web Client. Choose the ESXi server where the witness should be deployed. Right-click the ESXi host and select Deploy OVF Template.

Step 2

Browse and select the HyperFlex-Witness.ova file. Click Next.

Step 3

Specify a unique name for the witness node in the Virtual Machine Name field. Select a location for the virtual machine from the drop-down list. Click Next.

Step 4

From the Select a compute resource drop-down list, choose the ESXi host where you want to deploy the witness node. Click Next.

Step 5

In the Review details pane, verify the template details. Click Next.

Step 6

In the Select Storage pane, do the following:


Field	Description
Select virtual disk format drop-down list	Thick Provision Lazy Zeroed Thick Provision Eager Zeroed Thin Provision
VM Storage Policy drop-down list	Datastore Default

Select the datastore where the virtual machine will reside. Ensure that this datastore has at least 40 GB of available free space. Click Next.

Step 7

In the Select Networks pane, select a Destination Network port group, where the witness VM has to connect. Click Next.

Step 8

On the Customize Template page, complete the fields that are relevant for your configuration. If no values are entered, the VM uses DHCP server provided network configuration parameters.


Field	Description
Static IP Address field	The IP address for Witness VM. The Static IP address can be changed only with cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM. Leave blank if DHCP is desired.
Netmask field	The netmask or prefix for this interface. Leave blank if DHCP is desired.
Default Gateway field	The default gateway address for this VM. Leave blank if DHCP is desired.
DNS field	The domain name servers for this VM (comma separated). Leave blank if DHCP is desired.
NTP field	NTP servers for this VM (comma separated) to sync time. Leave blank if DHCP is desired.

Click Next.

Step 9

On the Ready to complete page, verify all the details entered. Click Finish.

Step 10

Repeat this process for each cluster.

What to do next

After successfully deploying the witness node, you can proceed to installing your stretched cluster. When prompted enter the IP address of this witness node on the IP Address page when creating a HyperFlex stretched cluster. The witness node is automatically used during configuration of the stretched cluster.

Changing the Witness VM Password

After successful deployment of witness VM, you must change the default password.

Before you begin

Download and deploy the witness VM.

Procedure

Step 1	Log in to the witness VM using SSH. `$ ssh root@<IP address of witness VM>`
Step 2	Enter the default password.
Step 3	Enter the `passwd` command and change the password. `$ passwd Changing password for user admin.`
Step 4	Log out from the witness VM.

What to do next

Network IP Addressing

IP addresses for HyperFlex Stretched Cluster need to be allocated from the appropriate subnets and VLANs to be used.

Important

Ensure that the Data and Management Networks are on different subnets for a successful installation.
Ensure that the IP addressing is in the same subnet for the same components (Management Network IP addresses, Data Network IP addresses).

Table 1. Example: Stretched Cluster IP Addressing
Storage Cluster Management IP address	10.10.10.128	Storage Cluster Data IP address	192.168.10.160
Subnet mask IP address	255.255.255.0	Subnet mask IP address	255.255.255.0
Default gateway IP address	10.10.10.1	Default gateway IP address	192.168.10.1

Table 2. Example: Stretched Cluster IP Addressing for Site A
Management Network IP Addresses (must be routable)			Data Network IP Addresses (does not have to be routable)
ESXi Hostname*	Hypervisor Management Network	Storage Controller Management Network	Hypervisor Data Network (Not Required for Cisco Intersight)	Storage Controller Data Network (Not Required for Cisco Intersight)
Server 1:	10.10.10.2	10.10.10.32	192.168.10.2	192.168.10.32
Server 2:	10.10.10.3	10.10.10.33	192.168.10.3	192.168.10.33
Server 3:	10.10.10.4	10.10.10.34	192.168.10.4	192.168.10.3
Server 4:	10.10.10.5	10.10.10.35	192.168.10.5	192.168.10.35
Server 5:	10.10.10.6	10.10.10.36	192.168.10.6	192.168.10.36

Table 3. Example: Stretched Cluster IP Addressing for Site B
Management Network IP Addresses (must be routable)			Data Network IP Addresses (does not have to be routable)
ESXi Hostname*	Hypervisor Management Network	Storage Controller Management Network	Hypervisor Data Network (Not Required for Cisco Intersight)	Storage Controller Data Network (Not Required for Cisco Intersight)
Server 1:	10.10.10.64	10.10.10.96	192.168.10.64	192.168.10.96
Server 2:	10.10.10.65	10.10.10.97	192.168.10.65	192.168.10.97
Server 3:	10.10.10.66	10.10.10.98	192.168.10.66	192.168.10.98
Server 4:	10.10.10.67	10.10.10.99	192.168.10.67	192.168.10.99
Server 5:	10.10.10.68	10.10.10.100	192.168.10.68	192.168.10.100

Cisco HyperFlex Systems Stretched Cluster Guide, Release 3.0

Bias-Free Language

Book Title

Cisco HyperFlex Systems Stretched Cluster Guide, Release 3.0

Chapter Title