Pre-Upgrade Validation Checks

Test Upgrade Eligibility

Beginning with Cisco HyperFlex Release 4.0(2a), the Upgrade page displays the last cluster upgrade eligibility test result and last tested version of UCS server, HX data platform, and/or ESXi.

Before upgrading UCS server, HyperFlex data platform, and/or ESXi, perform upgrade eligibility test in the Upgrade page to validate and check the cluster readiness and the infrastructure compatibility for an upgrade.

To perform upgrade eligibility test:

  1. Select Upgrade > Test Upgrade Eligibility.

  2. Select the UCS Server Firmware check box to test upgrade eligibility of UCS server firmware.

    Enter the Cisco UCS Manager FQDN or IP address, username, and password. In the Current Version field, click Discover to choose the UCS firmware package version that need to be validated before upgrade.

  3. Slect the HX Data Platform check box to test upgrade eligibility of HyperFlex Data Platform.

    Enter the vCenter username and password. Upload the Cisco HyperFlex Data Platform Upgrade Bundle that need to be validated before upgrade.

  4. Select the ESXi check box to test upgrade eligibility of ESXi.

    Enter the vCenter username and password. Upload the Cisco HyperFlex Custom Image Offline Bundle that need to be validated before upgrade.

  5. Click Validate.

    The progress of the upgrade eligibility test is displayed.

HyperFlex Node Upgrade Validations

Perform the following validations on each HyperFlex node before moving on to upgrade the next node in the cluster.

  • Verify that the HyperFlex cluster is healthy and online. Verify all HyperFlex cluster nodes are connected to the vCenter and are online.

  • Verify that DRS is enabled and set to fully automated.

  • Verify that vSphere services are running and ESXi Agent Manager (EAM) health is normal.

  • Verify the health of the cluster in Cisco UCS Manager.

Viewing HyperFlex Cluster Health

Using CLI

Log in to any controller VM in the storage cluster. Run the command stcli cluster storage-summary –-detail. 


address: 192.168.100.82
name: HX-Cluster01
state: online
uptime: 0 days 12 hours 16 minutes 44 seconds
activeNodes: 5 of 5
compressionSavings: 78.1228617455
deduplicationSavings: 0.0
freeCapacity: 38.1T
healingInfo:
    inProgress: False
resiliencyDetails:
        current ensemble size:5
        # of ssd failures before cluster shuts down:3
        minimum cache copies remaining:3
        minimum data copies available for some user data:3
        minimum metadata copies available for cluster metadata:3
        # of unavailable nodes:0
        # of nodes failure tolerable for cluster to be available:2
        health state reason:storage cluster is healthy.
        # of node failures before cluster shuts down:3
        # of node failures before cluster goes into readonly:3
        # of hdd failures tolerable for cluster to be available:2
        # of node failures before cluster goes to enospace warn trying to move the existing data:na
        # of hdd failures before cluster shuts down:3
        # of hdd failures before cluster goes into readonly:3
        # of ssd failures before cluster goes into readonly:na
        # of ssd failures tolerable for cluster to be available:2
resiliencyInfo:
    messages:
     Storage cluster is healthy.
     state: healthy
     hddFailuresTolerable: 2
     nodeFailuresTolerable: 1
     ssdFailuresTolerable: 2
spaceStatus: normal
totalCapacity: 38.5T
totalSavings: 78.1228617455
usedCapacity: 373.3G
clusterAccessPolicy: lenient
dataReplicationCompliance: compliant
dataReplicationFactor: 3

Sample response that indicates the HyperFlex storage cluster is online and healthy.

Checking Cluster Storage Capacity

We recommend that you check the cluster storage capacity before starting the upgrade of an existing installation of Cisco HX Data Platform. If the storage cluster capacity is above 70%, it is highly recommended to either reduce the amount of storage capacity used or increase the storage capacity by adding new nodes or disks. This confirmation of cluster storage capacity is important because if a node goes down in such a situation, the cluster will not be able to rebalance and will stay unhealthy (online).

Refer to the HX Storage Cluster Overview chapter in the Cisco HyperFlex Data Platform Administration Guide for background details about checking cluster storage capacity.

Verifying If DRS Is Enabled

Procedure

Click the vSphere DRS tab.

Check if Migration Automation Level is set to Fully Automated.

Verifying and Configuring the Net.TeamPolicyUpDelay Default Value

To avoid loss of storage access during Fabric Interconnect reboot for firmware update, perform this check prior to UCSM Infrastructure upgrade.

Upgrades to 3.0(1), 3.5(1), and 3.5(2) require that the default value of the ESXi host Net.TeamPolicyUpDelay be set to 30000. Complete the following steps to verify, and if needed, modify the default value of the ESXi host Net.TeamPolicyUpDelay to 30000.


Note
This change does not require a reboot of the ESXi host, and is persistent across a reboot.

SUMMARY STEPS

  1. From the vSphere Web Client Navigator, click on each ESXi Host > Configure > System > Advanced System Settings.
  2. In Advanced System Settings, scroll down to Net.TeamPolicyUpDelay.
  3. If needed, change the value to 30000. The default value is 100.

DETAILED STEPS

Step 1

From the vSphere Web Client Navigator, click on each ESXi Host > Configure > System > Advanced System Settings.

Step 2

In Advanced System Settings, scroll down to Net.TeamPolicyUpDelay.

Step 3

If needed, change the value to 30000. The default value is 100.

Viewing ESX Agent Manager

Procedure

From the vSphere Web Client Navigator, select Administration > vCenter Server Extensions > vSphere ESX Agent Manager > Summary.

Verify the Health of a HyperFlex Cluster In Cisco UCS Manager

Procedure

Step 1

Verify if the high availability status of the fabric interconnects shows that both the fabric interconnects are up and running. See the Cisco UCS Manager System Monitoring Guide for more information.

Step 2

Verify that the data path is up and running. See the Cisco UCS Manager Firmware Management Guide for more information.

Step 3

Verify that the HyperFlex servers have no faults.

Step 4

Verify that vNIC faults are cleared to ensure VMware ESXi vSwitch uplinks are up and operational.
Step 5

Verify if all servers have been discovered.

Verify UCS Server Firmware (C-Bundle) Version

Using UCS Manager

  1. Log in to UCS Manager.

  2. Select the Server tab.

  3. Select the Host Firmware Package policy by navigating to, Policies > Root > Sub-Organizations > <hx-cluster> > Host Firmware Packages > HyperFlex.


    Note

    Ensure that you select the desired cluster under the sub-org list.

  4. Under properties, note the current Rack Package version. It is listed as X.Y(Z)C. For example, 3.1(2g)C.

Configuring vMotion Interfaces

Complete the following steps to add the VMkernel interface necessary for vMotion to work:

Before you begin

Only default TCP/IP stack is supported for vMotion vmkernel adapters.

Procedure

Step 1

Click Add Host Networking.

Step 2

Select VMkernel Network Adapter.

Step 3

Select the existing vmotion vSwitch by selecting browse.

Step 4

Provide a name, and refer to table below to enter the appropriate VLAN ID.

Cluster Installation Version

VLAN ID

1.7.x

0 (default)

1.8.x and later

same as vMotion network

Step 5

Provide a Static IP Address and complete the wizard.

Step 6

(Optional) To use jumbo frames, edit the vmk2 and set the MTU to 9000. Your upstream switch must be configured to pass jumbo frames on the vMotion VLAN.

Step 7

Repeat steps 1 to 6 for all hosts in the cluster.

Configure Lenient Mode

Cluster access policy is set by default to lenient mode. To manually set the cluster access policy to lenient, use the following procedure.

SUMMARY STEPS

  1. SSH to any one of the controller VMs and login as root.
  2. Check if lenient mode is already configured.
  3. If set to strict, change to lenient. If already set to lenient, no further action is required.
  4. Confirm the change.

DETAILED STEPS

  Command or Action Purpose
Step 1

SSH to any one of the controller VMs and login as root.

Step 2

Check if lenient mode is already configured.

#stcli cluster get-cluster-access-policy
Step 3

If set to strict, change to lenient. If already set to lenient, no further action is required.

~/#stcli cluster set-cluster-access-policy --name lenient
Step 4

Confirm the change.

stcli cluster info | grep -i policy

Example

~/#stcli cluster get-cluster-access-policy strict
~/#stcli cluster set-cluster-access-policy --name lenient
stcli cluster info | grep -i policy