Appendix

Creating VLANs for HX Servers

Procedure


Step 1

Open a web browser and enter the IP address for Cisco UCS Manager. Enter the login credentials.

Step 2

Navigate to LAN tab > LAN > LAN Cloud > VLANS.

Step 3

Right-click and select Create VLANs as shown in the table below:

VLAN Name

Description

Multicast Policy Name

VLAN ID (by default)

hx-inband-mgmt

Used for:

  • ESX management

  • SSH to storage controller VM

  • HX Cluster management IP - using multicast traffic.

  • vCenter connectivity to the HyperFlex VM for the HX Data Platform plug-in

HyperFlex

3091

hx-storage-data

Used for:

  • ESX NFS client(IOvisor)

  • HyperFlex replication/cluster

  • Cluster data VIP

HyperFlex

3092

hx-vmotion

Used for:

  • VM and storage vmotion, FT, iSCSI

HyperFlex

3093

insert existing vlan name

Used for:

  • VM data traffic

HyperFlex

Any*

Note:

  • Configuration option is Common/Global. It applies to both fabrics and uses the same configuration parameters in both cases.

  • *There is no specific recommendation for VM data VLANs. You can create your own VLANs for the VM data traffic. By default, the HXDP installer will not create VLANs for the VM data traffic.

  • Installer sets the VLANs as non-native by default. Ensure to configure the upstream switches to accommodate the non-native VLANs.


Creating MAC Address Pools

You can change the default MAC address blocks to avoid duplicate MAC addresses. Each block contains 100 MAC addresses by default to allow for up to 100 HX server for deployment per UCS system. We recommend that you use one MAC pool per vNIC for easier troubleshooting.


Note


The 8th digit is set to A or B. "A" is set on vNICs pinned to Fabric Interconnect A. And "B" is set on vNICs pinned to Fabric Interconnect B.

Procedure


Step 1

Open a web browser and enter the IP address for Cisco UCS Manager. Enter the login credentials.

Step 2

In Cisco UCS Manager, navigate to LAN tab > Pools > root > Sub-org > hx-cluster > MAC Pools.

Step 3

Right-click MAC Pools and select Create MAC Pool.

Step 4

In the Define Name and Description page of the Create MAC Pool wizard, complete the required fields as shown below:

MAC Pool Name

Description

Assignment Order

MAC Address block

hv-mgmt-a

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:01:01-64

hv-mgmt-b

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:02:01-64

storage-data-a

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:03:01-64

storage-data-b

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:04:01-64

vm-network-a

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:05:01-64

vm-network-b

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:06:01-64

hv-vmotion-a

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:07:01-64

hv-vmotion-b

MAC pool for HyperFlex System

Sequential

00:25:B5:XX:08:01-64

Step 5

Click Next.

Step 6

In the Add MAC Addresses page of the Create MAC Pool wizard, click Add.

Step 7

In the Create a Block of MAC Addresses dialog box, complete the following fields:

Name Description

First MAC Address field

Enter the first MAC address for the block. A MAC address is 6 bytes (48 bits) long, typically displayed as 12 hexadecimal digits separated by colons. It is recommended to use the prefix 00:25:b5 to ensure unique MAC addresses in the LAN fabric. Example, 00:25:b5:xx:xx:xx.

Size field

Specify the number of MAC addresses in the block. This determines how many consecutive addresses will be generated starting from the first MAC address. The size can range from 1 to 1000 per block.

Step 8

Click OK.

Step 9

Click Finish.

After the MAC address change, ESXi will be reconfigured as it was configured earlier. If management IP was DHCP assigned, then the IP will change.

Impact of Manufacturing process on MAC address change

  • The MAC address will change between the manufacturing process and the customer site, especially if the customer orders HyperFlex serves without UCS Fabric Interconnects.

  • A MAC address is configured during Service Profile association. It is un-configured during Service Profile disassociation.

  • At the end of manufacturing process, the service profiles are disassociated, hence the MAC addresses are un-configured.

  • When a HyperFlex server is deployed, configure the MAC address pools as described above.

  • VMWare supports Consistent Device Naming, but issues have been reported since 5.5.SR has been opened.


Configure the vSwitches

In both VMware ESX and ESXi host, you can configure vSwitches from either the GUI or the command line.

The CLI configurations are very helpful when you are installing multiple ESX servers and planning to script the vSwitch configuration.

After the ESX installation, configure your vSwitches on the ESX host with the following steps:

Procedure


Step 1

Log in to the command line of each ESX Server.

Step 2

Create three vSwitches on each ESX server using the listed names.

  • vswitch-hx-storage-data

    Set the MTU to 9000 on this switch.

  • vmotion

    Set the MTU to 9000 on this switch.

  • vswitch-hx-vm-network

Step 3

Use the following CLI commands to create the three new vSwitches:

# esxcli network vswitch standard add -v vswitch-hx-storage-data

# esxcli network vswitch standard set -v vswitch-hx-storage-data -mtu= 9000

# esxcli network vswitch standard add -v vswitch-vmotion

# esxcli network vswitch standard set -v vswitch-vmotion -mtu=9000

# esxcli network vswitch standard add -v vswitch-hx-vm-network

Step 4

The default vSwitch vSwitch0 created during installation of ESXi needs to be renamed to vswitch-hx-inband-mgmt for the Hx Data Platform node set up scripts to work properly. Use the following command to rename the switch and then reboot the host so that the vmkernel re-reads its configuration file to use the new name.

# sed -i 's/vSwitch0/vswitch-hx-inband-mgmt/g' /etc/vmware/esx.conf

# reboot

Step 5

You can verify the creation and renaming of the vSwitches after a host reboot with the following command:

# esxcli network vswitch standard list

Confirm that you see the four previously listed vSwitches in the command output. Only the switch-hx-inband-mgmt vSwitch will have Uplinks and Port groups listed. The HX Data Platform installer scripts perform the rest of the network configuration.


Migrating vMotion Networks to Virtual Distributed Switches (VDS) or Cisco Nexus 1000v (N1Kv)


Note


  • The HX Data Platform can be configured with VMware DVS or Cisco Nexus 1000v for specific non-HX dependent networks:

    vMotion networks

    and virtual machine networks

  • For further details, see Cisco Nexus 1000v documentation.


To migrate non-HX dependent vSwitches and associated port groups to DVS or N1Kv networks, follow the listed steps:

Procedure


Step 1

From vCenter, create DVS Switch and port groups.

  1. Select vCenter Inventory Lists > Datacenters > datacenter > Related Objects > Distributed Switches . Click Add Distributed Switch icon.

  2. Complete the New Distributed Switch wizard. Create each DVS switch with two uplinks.

    For example: VM network and vmotion pg

    • DVSwitch-VMNetwork: DVPortGroup-VMNetwork

    • DVSwitch-Vmotion: DVPortGroup-Vmotion

Step 2

Migrate the vSwitch, VMNetwork. Perform the following steps to migrate VMNetwork from legacy vSwitch to DVS.

  1. Select vCenter Inventory Lists > Datacenters > datacenter > Related Objects > Distributed Switches.

  2. Select the DVSwitch-VMNetwork vSwitch. Click the Add and Manage Hosts icon. This starts the Add and Manage Hosts wizard.

  3. On the Select task page, select Add Hosts. Click Next.

  4. On the Select hosts page, click Add New Hosts. Select all hosts in the cluster. Click Next.

  5. On the Select network adapter tasks page, select Manage physical adapters and Migrate virtual machine networking. Click Next.

  6. On the Manage physical network adapters page, the physical adapters part of vswitch-hx-vm-network:VM Network are assigned to the DVSwitch-VMNetwork.

  7. Under the On other switches/unclaimed list, select the vmnic corresponding to the In Use by Switch, vswitch-hx-vm-network.

  8. Click Assign uplink.

  9. Select Auto-assign.

  10. Click OK. The page refreshes with the newly assigned vmnic listed under On this switch.

  11. The Analyze impact page shows the impact of this migration. Verify the impact is all green. Click Next.

  12. On the Migrate VM networking page, select the VMs to migrate to the new network, DVPortGroup-VMNetwork.

    Next

    Select all the VMs, except the controller VMs, stCtlVM, from all the hosts. Select the DVPortGroup-VMNetwork. Click Next.

    Note

     

    The list of VMs for each host includes all the VMs, including the controller VMs. DO NOT select any controller VMs. Migrating the controller VMs will break your storage cluster.

  13. On the Ready to complete page, confirm the summary of the migration. Click Finish.

    Note

     

    Post migration system generates several network related alarms. Verify and clear the alarms.

Step 3

Migrate the vSwitch to vmotion pg. Perform the following steps to migrate vmotion pg from legacy vSwitch to DVS.

  1. Select vCenter Inventory Lists > Datacenters > datacenter > Related Objects > Distributed Switches.

  2. Select the DVSwitch-Vmotion vSwitch. Click the Add and Manage Hosts icon. This starts the Add and Manage Hosts wizard.

  3. On the Select task page, select Add Hosts. Click Next.

  4. On the Select hosts page, click Add New Hosts. Select all hosts in the cluster. Click Next.

  5. On the Select network adapter tasks page, select the tasks Manage physical adapters and Manage VMkernel adapters. Click Next.

  6. On the Manage physical network adapters page, the physical adapters part of vmotion:vmotion pg are assigned to the DVSwitch-Vmotion.

    Under the On other switches/unclaimed list, select the vmnic corresponding to the In Use by Switch, vmotion. Click Assign uplink, select Auto-assign, and click OK. The page refreshes with the newly assigned vmnic listed under On this switch. Click Next.

  7. On the Manage VMkernel network adapters page, migrate the VMkernel adapter to the port group, DVPortGroup-Vmotion.

    For each host, under the On other switches list, select the VMKernel adapter corresponding to the In Use by Switch, vmotion. Click Assign port group. Select the destination port group, DVPortGroup-Vmotion. Click OK. The page refreshes with the Reassigned VMkernel network adapters, listing the Source Port Group and Destination Port Group.

  8. Select the hosts to migrate to the new network, DVPortGroup-Vmotion. Click Next.

  9. On the Ready to complete page, confirm the summary of the migration, click Finish.

Step 4

Post migration step. Verify there is no impact on the VMs with respect to IO, Network connectivity and VM Migration.