Troubleshooting
Verifying and Recreating the HX User Account
-
Check for missing hxuser account:
-
Secure Shell (SSH) to all esxi hosts.
-
Type esxcli system account list.
-
Verify that the hxuser account is present on all esxi hosts. If the hxuser account is missing, proceed to step 2.
-
-
Secure Shell (SSH) to any controller:
-
Type cd /opt/springpath/storfs-mgmt/hxtoolbox-1.0/bin.
-
Type ./hxtoolbox –u.
Note
If you see an error, open a TAC case.
-
Verify that the huxser is created on the esxi hosts (Repeat step 1).
-
-
Rerun the upgrade.
Verifying and Recreating the Springpath User Account
-
Check for missing Springpath user account:
-
Secure Shell (SSH) to all esxi hosts.
-
Type esxcli system account list.
-
Verify that the Springpath user account is present on all esxi hosts. If the Springpath user account is missing, proceed to step 2.
-
-
On the ESXi hosts where springpath user is missing:
-
Type esxcli system account add -i springpath -p spr!n9p@th -c spr!n9p@th.
-
Type esxcli system permission set -i springpath -r Admin.
-
Verify that the Springpath User is present on missing nodes (Repeat step 1).
-
-
Rerun the upgrade.
After upgrade vcluster shows offline from CLI/nodes show offline in vCenter HXDP plugin summary section
Recreate Springpath_security.properties link:
-
Confirm alias to springapth properties exists on all controllers:
-
Secure Shell (SSH) to all controller VMs.
-
Type ls /usr/share/springpath/storfs-misc/ and confirm thespringpath_security.properties link exists on all controllers.
-
Type ls /etc/springpath/secure to confirm that the springpath_security.properties file exists on all nodes.
-
-
If the file is missing, copy the file from another node to the missing node:
-
Example (from controller with the file): Type scp /etc/springapth/secure/springapth_security.properties root@10.10.10.5:/etc/springpath/secure/springpath_security.properties.
-
-
If the alias link is missing, you must recreate the link on the missing node: Example: /etc/springpath/secure/springpath_security.properties
-
Type ln -s /usr/share/springpath/storfs-misc/springpath_security.properties /etc/springpath/secure/springpath_security.properties.
-
Type restart stMgr.
-
-
Verify the fix:
-
Type stcli cluster info and confirm that the vCluster state is online.
-
In the HXDP vCenter plugin, confirm that the nodes all show as online in the summary section.
-
Update Net.Team Policy
ON ALL ESXi HOSTS:
-
Confirm current value is set to 100
[root@JR-NODE-1:~] esxcli system settings advanced list -o /Net/TeamPolicyUpDelay
-
Change /Net/TeamPolicyUpDelay --int-value from 100 (Default value) to 30000.
[root@JR-NODE-1:~] esxcli system settings advanced set -o /Net/TeamPolicyUpDelay --int-value 30000
-
Confirm the new value of /Net/TeamPolicyUpDelay --int-value is 30000.
[root@JR-NODE-1:~] esxcli system settings advanced list -o /Net/TeamPolicyUpDelay Path: /Net/TeamPolicyUpDelay Type: integer Int Value: 30000 <— Value set in Step-1 above from 100(Default value) to 30000 Default Int Value: 100 Min Value: 0 Max Value: 600000 String Value: Default String Value: Valid Characters: Description: Delay (ms) before considering an 'uplink up' event relevant
Precheck Validation Failure Due to an Algorithm Change in 6.0 U3
While upgrading HX, with version 6.0 U3 and later, we see following error: Failed upgrade validations : Checking vCenter configuration. Reason: Upgrade validations failed. Failed to query ESX version on host X.X.X.X
root@SpringpathControllerXXXX:~# stcli cluster upgrade --components hxdp --location
/tmp/storfs-packages-3.5.2d-31738.tgz --vcenter-user administrator@XXX.com
…Waiting for upgrade validations to finish... ['Checking vCenter configuration']
Failed upgrade validations : Checking vCenter configuration. Reason: Upgrade validations
failed. Failed to query ESX version on host: X.X.X.X
From ESXi /var/run/log/auth.log
2019-07-03T19:56:18Z sshd[757807]: Connection from X.X.X.Xport 42416
2019-07-03T19:56:18Z sshd[757807]: Unable to negotiate with X.X.X.Xport 42416:
no matching key exchange method found. Their offer: diffie-hellman-group1-sha1,
diffie-hellman-group-exchange-sha1 [preauth]
Workaround:
-
On each ESXI host in the cluster, edit the /etc/ssh/sshd_config file. Add
KexAlgorithms diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1
to the end of the file. -
Rerun the upgrade command or resume from UI.
Pycrypto Minimum Version
Verify the minimum package version is 2.6.
EXAMPLE:
$ pip show pycrypto
---
Name: Jinja2
Version: 2.6.1
Upgrading a Version before HyperFlex Release 1.8(1a)
Contact TAC to up upgrade the package.
Extra or Duplicate stNode is Present in stcli after Upgrade
Contact TAC.
Duplicate pnode is Present in stcli cluster info after Upgrade
Contact TAC.