运行UCSM运行状况和升级前检查工具

简介

本文档介绍运行Unified Computing System Manager(UCSM)运行状况和升级前检查工具的过程。 

先决条件

要求

思科建议您在系统上安装Python 3.6或更高版本。 

注意：如果您正在运行Windows OS，则可以安装Python并配置环境路径。

注意：请勿打开Python问题/脚本无法运行的TAC案例。请参阅CLI命令部分，以手动确定问题并按确定的问题打开TAC案例。

使用的组件

本文档不限于特定的软件和硬件版本。

本文档中的信息都是基于特定实验室环境中的设备编写的。本文档中使用的所有设备最初均采用原始（默认）配置。如果您的网络处于活动状态，请确保您了解所有命令的潜在影响。

背景信息

UCSM检查工具是一个实用程序，用于在UCSM上执行主动自检，以确保其稳定性和恢复能力。它有助于在UCS基础设施进行升级和维护操作时，自动执行UCS系统的运行状况和升级前检查列表，从而节省时间。

注意：请始终下载并使用工具的最新版本。由于该工具经常增强，因此当您使用较旧版本时，它可能会遗漏重要的检查。

注意：此脚本是尽力而为、可免费使用的脚本。但是，它无法识别所有问题。

适用场合

• 在UCS基础设施升级之前

• 维护活动前后的UCS运行状况检查
• 当您与Cisco TAC合作时
• 随时主动进行运行状况检查

操作方法

Windows操作系统

步骤1.从Python下载下载最新版本的Python

步骤2.使用常规安装过程并单击Install Now（推荐的安装）下载安装程序。 

注意：确保选中Add Python to PATH。

步骤3.导航到系统上安装Python的目录。

步骤4.打开命令提示符并键入命令Python以验证Python安装。

步骤5.从此处下载最新版本的运行状况检查脚本并将其保存到文件夹中。现在，提取压缩文件，如图所示。

步骤6.下载并将最新的UCSM技术支持日志保存到所创建的文件夹（如图所示）。单击此链接可查找下载UCSM日志捆绑包的步骤：生成UCSM技术支持。

步骤7.打开CMD和cd到UCSMTool.py所在的文件夹，然后运行UCSMTool.py，如图所示。

步骤8.输入UCSM技术支持文件所在的文件路径，然后选择所需选项。

1. UCSM运行状况检查

2.升级前检查

MacOS

步骤1. MacOS随附默认安装的Python。验证安装的Python版本，如下所示：

注意：如果Python版本低于3.6，请升级到3.6及更高版本。

注意：如果Python版本是3.6或更高版本，请跳至步骤5，否则，跳至步骤2。

步骤2.从宏的Python发行版下载最新版本的Python。

步骤3.使用正常安装过程完成/升级Python安装。

步骤4.从此处下载最新版本的运行状况检查脚本并将其保存到文件夹中。现在，解压缩文件，如下图所示：

步骤5.下载并将最新的UCSM技术支持日志保存到所创建的文件夹，如下图所示。单击链接查找下载UCSM日志捆绑包的步骤：生成UCSM技术支持。

步骤6.打开终端，浏览到下载运行状况检查脚本的目录，运行python UCSMTool.py或python3UCSMTool.py，如下所示：

步骤7.输入UCSM技术支持文件所在的文件路径，然后选择所需选项运行脚本。

1. UCSM运行状况检查

2. 升级前检查

了解执行的输出/检查

由UCSM运行状况检查执行的检查

这些检查由UCSM-Healthchecktool执行：

• UCSM HA集群状态：显示交换矩阵互联的集群状态。 
• PMON进程状态：显示CiscoUCS Manager中所有进程的状态。  
• 文件系统装载：显示装载表。   
• 检查/var/ sysmgr大小问题：检查/var/sysmgr使用实例。 
• 检查/var/ tmp大小问题：检查/var/ tmp使用实例。
• 6296 FI在重新通电后无响应，硬件修订版更新：验证交换矩阵互联模块及其硬件修订版号。 
• 严重性为严重性为严重性或严重性为严重性的故障：报告您在UCS Manager中有任何重大或严重警报。 
• 选中Backup Available:验证UCS Manager中是否提供备份。 
• 密钥环证书检查：检查密钥环是否过期或有效。 
• 是否需要安全解决方案：通过验证FI型号及其版本来检查是否需要安全修整解决方法。  
• Cisco UCS Manager版本4.x中已弃用的硬件：检查Cisco UCS Manager 4.x版本中任何已弃用的硬件。 
• 3.1.x及更高版本已弃用的硬件：检查Cisco UCS Manager 3.x版本中任何已弃用的硬件。 
• 检查B200M4是否由于空白MRAID12G字段而重新启动：检查B200M4服务器是否有空的MRAID12G RAID控制器。  
• UCSM 3.1最大功率分配更改会导致刀片发现故障：验证在UCS Manager中配置的电源策略。   
• 存在引导闪存损坏故障代码F1219：检查是否存在引导闪存损坏。 
• 当删除默认密钥环时，检查httpd是否无法启动：检查是否删除了默认密钥环。
• 第3代FI具有不干净的文件系统状态 — “文件系统状态：清理错误":检查文件系统错误。 
• 检查服务器自动安装到4.0(4b)无法激活SAS控制器：验证主机固件版本和SAS扩展器版本。 
• 检查C系列固件升级是否持续较长时间，执行服务器资产、PNU操作系统资产：验证服务器型号及其版本以确定您是否遇到此问题。 
• 检查使用句点或连字符的UCSM身份验证域：验证身份验证域名是否配置了句点或连字符的配置。 
• 本地或回退身份验证失败：检查为特定FI模型配置的身份验证方法并验证其版本。 
• UCSM和UCS中心之间的运行状况检查：验证UCS Manager是否已向UCS中心注册。 
• 保留VLAN检查：检查使用的VLAN是否来自保留的VLAN范围。
• LAN和SAN引脚组：检查集群中的lan/san pinning配置，突出显示以在升级/任何MW活动之前查看配置。 
• 检查UCSM中存在的挂起活动：验证UCS Manager域中是否有任何挂起的活动。 
• IOM运行状况检查：检查IO模块的整体运行状况。   
• UCSM中可用的核心文件检查：验证在60天内是否找到任何核心文件。 
• 分离L2可能配置错误：验证在配置分离L2的情况下是否存在任何配置错误。 
• VIC 1400和6400链路抖动问题：检查此缺陷中存在的条件。   
• 在固件更新期间检查2304 IOM断开连接并重新连接：验证交换矩阵互联和IO模块型号并确定是否存在任何潜在问题。 
• DME运行状况检查：验证数据管理引擎(DME)数据库的运行状况。 
• FI上接口启用和浮动匹配的数量：验证接口和浮动会话的数量。 
• 巨型或标准MTU检查：确定MTU配置。 

UCSM工具输出号示例

akmalla@ucsm_health_check-master % python3 UCSMTool.py 

		 UCS Health Check Tool 2.0

Enter the UCSM file path: /Users/akmalla/Desktop/UCSM health Script/UCSMlog.tar

Press 1 for UCSM Health Check
Press 2 for PreUpgrade Check
Enter your choice (1/2): 1

Log Extraction:  [########################] COMPLETED

UCSM Version: 4.3(2c)


Summary Result:
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| SlNo | Name                                                              | Status                     | Comments                                                          |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 1    | UCSM HA Cluster State                                             | PASS                       |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 2    | PMON Process State                                                | PASS                       |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 3    | File System Mount                                                 | PASS                       |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 4    | Check for /var/sysmgr size issue                                  | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 5    | Check for /var/tmp size issue                                     | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 6    | 6296 FI unresponsive after power cycle, HW revision update        | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 7    | Faults with Severity Major or Severity Critical                   | Found                      | Review the faults and Contact TAC, if needed                      |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 8    | Check Backup Available                                            | Backup Operation Not Found | Backup operation has not been found. Please ensure that the       |
|      |                                                                   |                            | latest backup is captured as a best practice.                     |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 9    | Keyring Cert Check                                                | PASS                       |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 10   | Safeshut Workaround Needed or Not                                 | Not Needed                 |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 11   | Deprecated Hardware in Cisco UCS Manager Release 4.x              | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 12   | Deprecated HW found for 3.1.x onwards                             | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 13   | Check for B200M4 reboot due to blank MRAID12G fields              | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 14   | UCSM 3.1 Change in max power allocation causes blade discovery    | Not Found                  |                                                                   |
|      | failure                                                           |                            |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 15   | Existence of bootflash corruption fault code F1219                | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 16   | Check for httpd fail to start when default keyring is deleted     | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 17   | 3rd GEN FIs has unclean file system states-"Filesystem state:     | Not Found                  |                                                                   |
|      | clean with errors"                                                |                            |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 18   | Check for Server Auto-Install to 4.0(4b) Fails to Activate SAS    | Not Found                  |                                                                   |
|      | Controller                                                        |                            |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 19   | Check for C-Series firmware upgrade stays long in process         | Not Found                  |                                                                   |
|      | "perform inventory of server"  PNU OS Inventory                   |                            |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 20   | Check UCSM Authentication Domain using a Period or Hyphen         | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 21   | Local or fallback Authentication failure                          | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 22   | Health check between UCSM and UCS central                         | Not Found                  | UCS Manager is Not Registered                                     |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 23   | Reserved VLAN Check                                               | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 24   | LAN and SAN Pin Groups                                            | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 25   | Checking Pending Activities Present in UCSM                       | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 26   | Health Check for IOM                                              | PASS                       |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 27   | Core Files available in UCSM Check                                | Not Found                  | No core files were found in last 60 days                          |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 28   | Disjoint L2 potential misconfiguration                            | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 29   | VIC 1400 and 6400 Link Flap Issue                                 | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 30   | Check 2304 IOMs disconnect and re-connect during firmware update  | Not Found                  |                                                                   |
|      | step                                                              |                            |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 31   | Number of Interface up and Flogi Matching on FI                   | ---                        | Primary:                                                          |
|      |                                                                   |                            |   FC Port Trunking Count: 0,                                      |
|      |                                                                   |                            |   Eth up Port: 7,                                                 |
|      |                                                                   |                            |   Flogi Count: 0                                                  |
|      |                                                                   |                            | Secondary:                                                        |
|      |                                                                   |                            |   FC Port Trunking Count: 0,                                      |
|      |                                                                   |                            |   Eth up Port: 7,                                                 |
|      |                                                                   |                            |   Flogi Count: 0                                                  |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+
| 32   | Jumbo or Standard MTU Check                                       | Not Found                  |                                                                   |
+------+-------------------------------------------------------------------+----------------------------+-------------------------------------------------------------------+

Faults with Severity Major:
F0331: Service profile DemoSP cannot be accessed
F0831: lan Member 1/2 of Port-Channel XXXX on fabric interconnect B is down, membership: down
F0858: lan port-channel XXXX on fabric interconnect B oper state: failed, reason: port-channel-members-down
F0831: lan Member 1/2 of Port-Channel XXXX on fabric interconnect A is down, membership: down
F0858: lan port-channel XXXX on fabric interconnect A oper state: failed, reason: port-channel-members-down

NOTE:            
a. All reports and logs will be saved in the same location from where the script was executed.
b. Please visit the Summary Report/ Main Report to view all the Major and Critical Fault alerts.

分析工具输出 — 后续步骤

• 该工具在UCS系统上自动运行手动命令的流程。
• 如果工具运行OK并在所有测试中提供PASS/NOT FOUND。UCS系统适用于脚本执行的所有检查。
• 在某些检查中工具FAIL/FOUND或者未成功运行的情况下，您可以使用CLI命令（此处列出）对UCS系统/交换矩阵互联执行与脚本Manually执行的相同检查。
• 该工具不检查任何旧/新/打开/解决的警告，因此强烈建议在任何升级或维护活动之前查看UCS版本说明和升级指南。

提示：对于您的UCS环境的常规运行状况检查，Cisco TAC不提供此服务。思科的CX客户交付团队（以前称为“高级服务”）确实提供错误清理/风险分析。如果您需要此类服务，请与您的销售/客户团队联系。

CLI命令

到两个交换矩阵互联的SSH:

 # show cluster extended-state, verify HA status is ready.

 # connect local-mgmt ; # show pmon state, Verify the services are in running status.

 # connect nxos ; # show system internal flash, Verify free size in  /var/sysmgr and /var/tmp

 # connect nxos ; # show module, verify HW revision number for 6296 fabric interconnects.

 # show fault detail | include F1219, verify this fault code for bootflash corruption

 # scope eth-uplink; # show reserved-vlan

 # show iom health status, displays health of IOM

 # show server status, verify the status of server.

 # scope monitoring; # scope sysdebug; # show cores , verify if there are any core files.

 # scope security; # scope keyring default; #show detail, verify details for default keyring, expiry etc.

 # connect nxos; # show int br | grep -v down | wc –l, verify the number of active Ethernet interfaces.

 # scope security; # show authentication, review the authentication type.

 # connect nxos; # show flogi database, review the flogi database.