简介

本文描述进程运行Hypercheck健康&升级前工具。此工具是执行在hyperflex系统的积极的自检的工具保证其稳定性和弹性。在hyperflex升级和维护操作期间，它帮助自动化健康列表和在hyperflex系统的升级前检查节省时间。

NOTE: 总是工具的下载新版本，在您使用它前。因为工具频繁地被提高，使用早版本可以导致错过重要检查。

支持的HX系统 

• Hyperflex版本1.8， 2.0， 2.1， 2.5， 2.6， 3.0， 3.5， 4.0
• Hyperflex标准簇
• Hyperflex边缘簇(2节点， 3节点& 4节点)
• 只支持在VMWare ESXi的Hyperflex簇

什么时候使用

• 在Hyperflex升级前。
• Hyperflex在维护Windows前后的健康检查
• 识别发生故障的驱动器/磁盘。
• 当与Cisco TAC一起使用时
• 任何时候积极的健康检查。

如何使用

步骤1.从Cisco github devnet的下载Hyperflex-Hypercheck.zip认为得这里。 请获得将有最新的改进和更新的新副本。 

注意：请使用从Cisco github devnet帐户下载的仅脚本。

步骤2.加载它到存贮控制器VM (SCVM)与集群管理IP (CMIP)

请使用您的首选的方法- scp/sftp/ftp/tftp -复制Hyperflex-Hypercheck.zip到/tmp目录

MAC ：

执行从CLI的SCP (请确认Hyperflex-Hypercheck.zip在同一个文件夹从您运行scp)

# scp Hyperflex-Hypercheck.zip root@<scvm-eth0:mgmtip>:/tmp/

请使用以下识别在您的HX环境Hyperflex剧本的集群管理IP

Windows：

我们能使用WINSCP调用文件如下所示

步骤3.提取Hyperflex-Hypercheck.zip内容

键入cd /tmp更改到/tmp目录 

root@SpringpathController7PVQWP6WV1:~# cd /tmp/

类型解Hyperflex-Hypercheck.zip压缩抽出文件 

root@SpringpathController7PVQWP6WV1:/tmp# unzip Hyperflex-Hypercheck.zip 
Archive:  Hyperflex-Hypercheck.zip
b61c59f7962b72902692ce70548ba3d760efdf06
   creating: Hyperflex-Hypercheck/
  inflating: Hyperflex-Hypercheck/HXTool.py 
  inflating: Hyperflex-Hypercheck/LICENSE.txt 
  inflating: Hyperflex-Hypercheck/ReadMe.txt 
  inflating: Hyperflex-Hypercheck/TestInfo.txt 
  inflating: Hyperflex-Hypercheck/prettytable.py 
  inflating: Hyperflex-Hypercheck/progressbar.py 
root@SpringpathController7PVQWP6WV1:/tmp# 

步骤4.执行HXTool Python脚本

键入cd Hyperflex-Hypercheck去Hyperflex-Hypercheck目录 

root@SpringpathControllerABCDE01234:/tmp# cd Hyperflex-Hypercheck

键入Python HXTool.py执行脚本 

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# python HXTool.py

步骤5.输入SCVM根密码，当提示 

Please enter below info of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the ESX Root Password:

步骤6. Hyperflex-Hypercheck工具开始其检查。将需要大约执行的3-10分钟能根据聚合的节点的编号在簇的完成

步骤7.了解被执行的输出/检查

以下检查由Hyperflex-Hyperchecktool执行

Hyperflex Checks: (Below checks are performed on all the storage controller VMs)
    Cluster services check - Verifies the status of storfs, stMgr and stNodeMgr services. 
    Enospc state check - Checks if the cluster space usage is above the warning threshold or no.     
    Zookeeper check - Checks whether the Zookeeper is running or no.
    Exhibitor check - Verfies the status of the Exhibitor service which manages the ZK.          
    HDD health check - Reports if you have any blacklisted disk in your cluster.        
    DNS check - Checks whether DNS is configured and reachable.                 
    vCenter reachability check - Checks whether the vCenter is reachable on the required ports.
    Timestamp check - Checks if all the controller VMs have the exact same time.           
    NTP sync check - Checks whether NTP is reachable from the storage controller VMs and synced.
    Check package & versions - Checks for packages and versions on Storage Controller VMs.
    Check Iptables count - Checks for Iptables count on and ensure it is same on all Storage Controller VMs.          
    Cluster upgrade check - Checks if there are any previous stale upgrade entries.     
    Extra pnodes check - Looks for any extra/duplicate pnode entries in the cluster.       
    Disk usage(/var/stv) check - Checks if the utilization of /var/stv is more that 80% or no.
    Out of memory check - Checks through the log files if the cluster had any oom event.
    Supported vSphere versions - Shows all the vSphere Versions supported with your current HXDP version.
    Network Checks - Checks the connectivity between Mgmt and Storage network.

ESXi Checks: (Below checks are performed on each ESXI node)
    HX User Account check - Verifies if the HXUser is created on all the esxi hosts and has admin rights.
    vMotion enabled check - Checks if the vMotion network is confirgued.
    vMotion reachability check - Checks the connectivity between vmotion network.
    Check for ESXI Failback timer - Check for ESXi Failback timer on ESXi host.
    Check ping to vmk0, eth0, eth1 - Checks the connectivity between the Mgmt and Storage network.
    No extra controller vm folders check - Checks if there are any extra controller VM folders in the Springpath datastore.
    Check the dumpfile in springpathDS - Check to confirm if SpringpathDS is configured for coredump.

*vMotion reachability check is only performed if vMotion enabled check is successful/PASS*

步骤8.得到脚本输出的报告

详细的每节点输出在目录安置

目录用命名HX_Report_<date>创建

它位于/tmp/ Hyperflex-Hypercheck目录 

键入ls HX_Report_<date>查看Hyperflex-Hypercheck工具创建的文件

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls HX_Report_13_02_2019_16_15
HX_Tool_13-02-2019_04-15-49 PM.log HX_Tool_Main_Report.txt HX_Report_10.197.163.179.txt HX_Report_10.197.163.180.txt HX_Report_10.197.163.181.txt HX_Report_10.197.163.182.txt

类型tar - cvf report.tar放置所有文件的HX_Report_<date>到tarfile 

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# tar -cvf report.tar HX_Report_13_02_2019_16_15/
HX_Report_13_02_2019_16_15/
HX_Report_13_02_2019_16_15/HX_Report_10.197.163.181.txt
HX_Report_13_02_2019_16_15/HX_Report_10.197.163.179.txt
HX_Report_13_02_2019_16_15/HX_Tool_13-02-2019_04-15-49 PM.log
HX_Report_13_02_2019_16_15/HX_Report_10.197.163.180.txt
HX_Report_13_02_2019_16_15/HX_Report_10.197.163.182.txt
HX_Report_13_02_2019_16_15/HX_Tool_Main_Report.txt

确认report.tar文件的类型ls在/tmp/ Hyperflex-Hypercheck目录里被创建

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls
report.tar  prettytable.py  HXTool.py  prettytable.pyc  HX_Report_13_02_2019_16_15  TestInfo.txt  progressbar.pyc  progressbar.py  ReadMe.txt

步骤9.导出Report.tar并且与TAC共享它(若需要)

请使用您的首选的方法- scp/sftp/ftp/tftp -从SCVM导出report.tar

步骤10.示例从3 NODE簇输出的HXTool。

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# python HXTool.py

                 HX Health Check 3.1

Please enter below info of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the ESX Root Password:

SSH connection established to HX Node: 10.X.X.79
SSH connection established to HX Node: 10.X.X.80
SSH connection established to HX Node: 10.X.X.81

HX Cluster Nodes:
+-------+---------------+--------------------------------+
| Nodes | IP Address    | HostName                       |
+-------+---------------+--------------------------------+
| 1     | 10.X.X.79     | SpringpathControllerABCD123456 |
+-------+---------------+--------------------------------+
| 2     | 10.X.X.80     | SpringpathControllerABCD123457 |
+-------+---------------+--------------------------------+
| 3     | 10.X.X.81     | SpringpathControllerABCD123458 |
+-------+---------------+--------------------------------+

SSH connection established to ESX Host: 10.X.X.76
SSH connection established to ESX Host: 10.X.X.75
SSH connection established to ESX Host: 10.X.X.77

HX Controller: 10.X.X.79
Cluster services check      [########################] PASS
ZooKeeper & Exhibitor check [########################] PASS
HDD health check            [########################] PASS
Pre-Upgrade Check           [########################] PASS
Network check               [########################] PASS

HX Controller: 10.X.X.80
Cluster services check      [########################] PASS
ZooKeeper & Exhibitor check [########################] PASS
HDD health check            [########################] PASS
Pre-Upgrade Check           [########################] PASS
Network check               [########################] PASS

HX Controller: 10.X.X.81
Cluster services check      [########################] PASS
ZooKeeper & Exhibitor check [########################] PASS
HDD health check            [########################] PASS
Pre-Upgrade Check           [########################] PASS
Network check               [########################] PASS


HX Controller: 10.X.X.79
Test Summary:
+----------------------------+-----------+----------+
| Name                       | Result    | Comments |
+----------------------------+-----------+----------+
| Cluster services check     | PASS      |          |
+----------------------------+-----------+----------+
| Enospc state check         | PASS      |          |
+----------------------------+-----------+----------+
| Zookeeper check            | PASS      |          |
+----------------------------+-----------+----------+
| Exhibitor check            | PASS      |          |
+----------------------------+-----------+----------+
| HDD health check           | PASS      |          |
+----------------------------+-----------+----------+
| DNS check                  | PASS      |          |
+----------------------------+-----------+----------+
| vCenter reachability check | PASS      |          |
+----------------------------+-----------+----------+
| Timestamp check            | PASS      |          |
+----------------------------+-----------+----------+
| NTP sync check             | PASS      |          |
+----------------------------+-----------+----------+
| Check package & versions   | PASS      |          |
+----------------------------+-----------+----------+
| Check Iptables count       | PASS      |          |
+----------------------------+-----------+----------+
| Cluster upgrade status     | PASS      |          |
+----------------------------+-----------+----------+
| Extra pnodes check         | PASS      |          |
+----------------------------+-----------+----------+
| Disk usage(/var/stv) check | PASS      |          |
+----------------------------+-----------+----------+
| Memory usage check         | PASS      |          |
+----------------------------+-----------+----------+
| Incidence of OOM           | PASS      |          |
+----------------------------+-----------+----------+
| Supported vSphere versions | 6.0.0-U3  |          |
|                            | 6.5.0-U1  |          |
|                            | 6.5.0-U2  |          |
|                            | 6.7.0-UGA |          |
|                            | 6.7.0-U1  |          |
+----------------------------+-----------+----------+

HX Controller: 10.X.X.80
Test Summary:
+----------------------------+-----------+----------+
| Name                       | Result    | Comments |
+----------------------------+-----------+----------+
| Cluster services check     | PASS      |          |
+----------------------------+-----------+----------+
| Enospc state check         | PASS      |          |
+----------------------------+-----------+----------+
| Zookeeper check            | PASS      |          |
+----------------------------+-----------+----------+
| Exhibitor check            | PASS      |          |
+----------------------------+-----------+----------+
| HDD health check           | PASS      |          |
+----------------------------+-----------+----------+
| DNS check                  | PASS      |          |
+----------------------------+-----------+----------+
| vCenter reachability check | PASS      |          |
+----------------------------+-----------+----------+
| Timestamp check            | PASS      |          |
+----------------------------+-----------+----------+
| NTP sync check             | PASS      |          |
+----------------------------+-----------+----------+
| Check package & versions   | PASS      |          |
+----------------------------+-----------+----------+
| Check Iptables count       | PASS      |          |
+----------------------------+-----------+----------+
| Cluster upgrade status     | PASS      |          |
+----------------------------+-----------+----------+
| Extra pnodes check         | PASS      |          |
+----------------------------+-----------+----------+
| Disk usage(/var/stv) check | PASS      |          |
+----------------------------+-----------+----------+
| Memory usage check         | PASS      |          |
+----------------------------+-----------+----------+
| Incidence of OOM           | PASS      |          |
+----------------------------+-----------+----------+
| Supported vSphere versions | 6.0.0-U3  |          |
|                            | 6.5.0-U1  |          |
|                            | 6.5.0-U2  |          |
|                            | 6.7.0-UGA |          |
|                            | 6.7.0-U1  |          |
+----------------------------+-----------+----------+

HX Controller: 10.X.X.81
Test Summary:
+----------------------------+-----------+----------+
| Name                       | Result    | Comments |
+----------------------------+-----------+----------+
| Cluster services check     | PASS      |          |
+----------------------------+-----------+----------+
| Enospc state check         | PASS      |          |
+----------------------------+-----------+----------+
| Zookeeper check            | PASS      |          |
+----------------------------+-----------+----------+
| Exhibitor check            | PASS      |          |
+----------------------------+-----------+----------+
| HDD health check           | PASS      |          |
+----------------------------+-----------+----------+
| DNS check                  | PASS      |          |
+----------------------------+-----------+----------+
| vCenter reachability check | PASS      |          |
+----------------------------+-----------+----------+
| Timestamp check            | PASS      |          |
+----------------------------+-----------+----------+
| NTP sync check             | PASS      |          |
+----------------------------+-----------+----------+
| Check package & versions   | PASS      |          |
+----------------------------+-----------+----------+
| Check Iptables count       | PASS      |          |
+----------------------------+-----------+----------+
| Cluster upgrade status     | PASS      |          |
+----------------------------+-----------+----------+
| Extra pnodes check         | PASS      |          |
+----------------------------+-----------+----------+
| Disk usage(/var/stv) check | PASS      |          |
+----------------------------+-----------+----------+
| Memory usage check         | PASS      |          |
+----------------------------+-----------+----------+
| Incidence of OOM           | PASS      |          |
+----------------------------+-----------+----------+
| Supported vSphere versions | 6.0.0-U3  |          |
|                            | 6.5.0-U1  |          |
|                            | 6.5.0-U2  |          |
|                            | 6.7.0-UGA |          |
|                            | 6.7.0-U1  |          |
+----------------------------+-----------+----------+

################################################################################
			Network check:
################################################################################

ESX vmk0: 10.X.X.75, 10.X.X.76, 10.X.X.77

ESX vmk1: 10.X.X.102, 10.X.X.103, 10.X.X.104

SCVM eth0: 10.X.X.79, 10.X.X.80, 10.X.X.81

SCVM eth1: 10.X.X.98, 10.X.X.99, 10.X.X.100

ESX Host: 10.X.X.76
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Name                                 | Result | Comments                                                                        |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| HX User Account check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion enabled check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion reachability check           | PASS   |                                                                                 | 
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check for ESXI Failback timer        | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check ping to vmk0, eth0, eth1       | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check the dump in springpathDS       | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| No extra controller vm folders check | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+ 

ESX Host: 10.X.X.75
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Name                                 | Result | Comments                                                                        |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| HX User Account check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion enabled check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion reachability check           | PASS   |                                                                                 | 
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check for ESXI Failback timer        | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check ping to vmk0, eth0, eth1       | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check the dump in springpathDS       | PASS   |                                                                                 | 
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| No extra controller vm folders check | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+

ESX Host: 10.X.X.77
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Name                                 | Result | Comments                                                                        |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| HX User Account check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion enabled check                | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| vMotion reachability check           | PASS   |                                                                                 | 
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check for ESXI Failback timer        | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check ping to vmk0, eth0, eth1       | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| Check the dump in springpathDS       | PASS   |                                                                                 | 
+--------------------------------------+--------+---------------------------------------------------------------------------------+
| No extra controller vm folders check | PASS   |                                                                                 |
+--------------------------------------+--------+---------------------------------------------------------------------------------+

Main Report File: HX_Tool_Main_Report.txt

Release Notes: 
https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-release-notes-list.html

Upgrade Guides: 
https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-installation-guides-list.html

Note: Please check the status of Compute nodes manually, script only verifies the config on the converged nodes.

*vMotion reachability check is only performed if vMotion enabled check is successful/PASS*

分析输出的工具-以下步骤

• 工具自动化运行手工的on命令Hyperflex系统的进程。
• 如果工具运行好并且产生在所有测试的PASS。HX系统是有效对于脚本执行了的所有检查。
• 在，在一些的工具FAIL成功检查或不运行的情况下，您能使用CLI命令(如下所示)执行在Hyperflex系统的同样检查象手工完成由脚本。
• 工具不检查其中任一老/新/打开/解决的警告并且是高度推荐的在其中任一前查看Hyperflex版本注释和升级指南升级或维护活动。

注意： 因为脚本没有能运行，请勿开TAC案例。请手工请运行命令，识别问题并且打开SR识别的问题。

CLI命令

在Hyperflex SCVM-

SSH to All Hyperflex SCVMs-

# service_status.sh

# sysmtool --ns cluster --cmd enospcinfo

# echo srvr | nc 0 2181

# pidof exhibitor

# stcli disk list --ip <Corresponding ESXi Mgmt IP Address> |grep -B 2 -A 8 blacklisted

# stcli services dns show (and ping the IPs listed)

# ping <vCenter IP Address>

# date ; compare the time on all SCVMs. They should ideally be identical

# stcli services ntp show

# stcli cleaner info

# ntpq -p -4 

# dpkg -l | grep -i springpath 

# iptables -L -n | wc -l 

# stcli cluster upgrade-status ; only applicable for HX version 2.5 and above

# stcli cluster info

# df -h ; check that /var/stv should not be more than 10-15% used

# zgrep -i “out of memory” /var/log/debug-storfs.*

# ping -I eth0 <eth0> of all SCVMs

# ping -I eth1 <eth1> of all SCVMs

在ESXi系统

SSH to all ESXi hosts

# esxcli system account list

# esxcli network firewall ruleset list | grep -i vMotion

# esxcli software vib list| grep -i spring 

# esxcfg-vmknic -l ; confirm that vMotion VMK2 is created

# vmkping -I vmk0 <eth0> of all SCVMs

# vmkping -I vmk1 <eth1> of all SCVMs

# cd /vmfs/volumes/Springpath-XXXXXXXXXX ; Ensure that it has only one Folder that has the Storage Controller VM