In Secure Workload 3.6.x: Choose Troubleshoot > Cluster Status

In Tetration 3.4.x or 3.5.x: Choose Maintenance > Cluster Status

When you initiate a firmware update, the installer will verify that the update can continue, gracefully shut down the bare-metal host if needed, and then start the upgrade.

The overall update process can take two or more hours for M4 hardware, or one or more hours for M5 hardware.

The following behaviors are temporary and expected:

• When the firmware upgrade is initiated, the Cluster Status page may display errors for up to 10 minutes while backend services fail over.

• During the firmware upgrade, the firmware details normally shown on the Cluster Status page will not be displayed for the bare-metal host being updated, and after the update is complete it may take up to 15 minutes for the firmware details to display again.

• During the firmware upgrade, the node will be shown as inactive on the Cluster Status page.

• Once the firmware update process is initiated, the Service Status page may indicate some services are unhealthy since a bare-metal host, and all the virtual machines running on that bare-metal host, are no longer active in the cluster.

After the firmware update is completed, it may take an additional 30 minutes for the bare-metal host to become active in the cluster again, and additional time may be needed for all services to become healthy again. If services do not recover within two hours following a firmware update, please contact Cisco TAC for assistance.

Once a firmware update is initiated, you can click the View Firmware Upgrade Logs button to view the status of the firmware update. This log will display the overall status of the firmware update at the very top of the listing; this status entry will be one of the following:

• Firmware update has been triggered – The firmware update was requested but has not yet started. Pre-checks are being conducted.

• Firmware update is running – The firmware update has started. When a firmware update reaches this state, CIMC and HUU are in control of the update, and the Secure Workload cluster will report the status information it receives from CIMC about the update.

• Firmware update has timed out – This indicates that some process in the firmware update has exceeded the time alloted to complete it. The overall firmware update process has a 240-minute time limit once it enters the Firmware update is running phase. During the firmware update, CIMC may become unreachable when it reboots into the new version; this unreachable state has a timeout of 40 minutes before the firmware update is declared as “timed out.” Once the firmware update has started, the monitoring of that update will time out after 120 minutes.

• Firmware update has failed with an error – This indicates that an error occurred and the firmware update has failed. CIMC usually does not give an indication of success or failure, so this state usually indicates an error occurred prior to the firmware update actually running.

• Firmware update has finished – The firmware update finished successfully. Since CIMC usually does not give an indication of success or failure, it is best to verify that the UCS firmware versions are updated once those details become available in the Cluster Status page. Note that it can take up to 15 minutes for those details to become available. For expected versions, see CIMC Versions, below.

• If an undetected failure occurs, the upgrade may appear to be running, but will time out after 240 minutes (4 hours). If this occurs, try the upgrade again; if it fails again, contact TAC.

To refresh the status view, close the upgrade status window and then click the View Firmware Upgrade Logs button again.

Below the overall status in the View Firmware Upgrade Logs pop-up window, an Update progress section provides time-stamped log messages indicating the progress of the firmware update. Once the Rebooting Host In Progress status is displayed in these log messages, CIMC is in control of the update and the cluster is monitoring that update—most subsequent log messages come directly from CIMC and are only added to the list of log messages if the status of the update changes.

Below the Update progress section of the View Firmware Upgrade Logs pop-up window, a Component update status section will be shown once CIMC starts providing individual component updates. This section can give a quick overview of the status of the update of the various UCS components on the bare-metal host.