Overview
Before you upgrade the Cisco HX Data Platform and the Cisco UCS server firmware in your Cisco HyperFlex System, consider the guidelines, best practices, and recommendations listed in this chapter.
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Before you upgrade the Cisco HX Data Platform and the Cisco UCS server firmware in your Cisco HyperFlex System, consider the guidelines, best practices, and recommendations listed in this chapter.
See Resolved Caveats and Open Caveats before upgrading and review the New Features for this release. Refer to the latest Cisco HX Data Platform Release Notes.
Review supported versions and system requirements.
Important |
|
Back up the configuration into an All Configuration backup file. See Cisco UCS Manager Backing Up and Restoring the Configuration Guide for the detailed steps.
Before you perform firmware updates, use the Cisco UCS Manager Firmware Management interface to download relevant images to the fabric interconnect. Images are stored in bootflash partitions in the fabric interconnect. See Downloading Software for more details.
A 2-step ESXi upgrade may be required when upgrading from ESXi versions starting with 6.0 GA (Build: 2494585) but before 6.0 P07 (Build: 9239799) or versions starting with 6.5 GA (Build: 4564106) but before 6.5 U2 (Build: 8294253) to 6.5 releases post 23rd July 2020, 6.7 releases post 28th April 2020 or any 7.0 ESXi release. For more information, see Impact on ESXi upgrade to future ESXi releases of 2020 due to expired ESXi VIB Certificate.
Keep SSH enabled on all ESXi Hosts.
Disable Cisco HyperFlex Smart Call Home. For more information, see the Cisco HyperFlex Smart Call Home Quick Start Guide.
Only default TCP/IP stack is supported for vMotion vmkernel adapters.
Enable vMotion so that the VMs can be moved automatically during the upgrade and MTUs are set as required in the environment. See Configuring vMotion Interfaces for details on adding VMkernel interface.
Verify that the HyperFlex cluster is healthy. See HyperFlex Node Upgrade Validations for more details.
Verify that the cluster is in lenient mode. If not, set the cluster to lenient mode, refer Configure Lenient Mode.
Starting with release 4.0(2a), SCVM is no longer needed on a Compute node.
HX REST API Access Token Management – Applications leveraging HX REST APIs should re-use access tokens when making API calls. Once obtained using the AAA Obtain Access Token API, access tokens are valid for 18 days (1,555,200 seconds). In addition, AAA enforces rate limiting on Obtain Access Token API requests: in a 15 minute window, /auth can be invoked (successfully) a maximum of 5 times. A user is allowed to create a maximum of 8 unrevoked tokens. Subsequent call to /auth will automatically revoke the oldest issued token to make room for the new token. A maximum of 16 unrevoked tokens can be present in system. In order to prevent brute-force attacks, after 10 consecutive failed authentication attempts, a user account is locked for a period of 120 seconds. For more information, see Cisco HyperFlex Systems REST API Reference guide.
HxConnect makes use of AAA Authentication REST API for login and the above rate limit applies to HxConnect also.
Single socket stretch cluster nodes are not supported.
Intersight Managed Mode is not currently supported for HyperFlex.
The following list is a highlight of critical criteria for performing an upgrade of your HyperFlex system.
Upgrade Considerations for configurations using SFP-H25G-CU3M or SFP-H25G-CU5M cables— If your configuration is a Fabric Interconnect 6400 connected to VIC 1455/1457 using SFP-H25G-CU3M or SFP-H25G-CU5M cables, then do not use the recommended UCS version of 4.0(4i) release or any other qualified releases. You must use UCS release 4.1(2a) with a qualified HXDP 3.5 or 4.0 version or the cluster may experience an outage. For information on any UCS issues that may affect your environment, see Release Notes for UCS Manager, Firmware/Drivers, and Blade BIOS.
Unsupported HX Data Platform 1.7.x, 1.8.x, 2.0, 2.1x, 2.5x, and 2.6x clusters—Users from any version prior to 2.6(1a) must step through an intermediate version before upgrading to 4.0 or later releases. If you need to upgrade your environment from a Cisco HyperFlex HX Data Platform software release that is past the last date of support, to the latest suggested release on the Cisco Software Download site, see Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HX Releases. For more information, see the Software Advisory for CSCvq66867: WARNING: Only Use HXDP 2.6(1e) Upgrade Package When Upgrading From HXDP 1.8(1a)-1.8(1e).
Hypercheck Health Check Utility— Cisco recommends running this proactive health check utility on your HyperFlex cluster prior to upgrade. These checks provide early visibility into any areas that may need attention and will help ensure a seamless upgrade experience. For more information see the HyperFlex Health & Pre-Upgrade Check Tool TechNote for full instructions on how to install and run Hypercheck.
vSphere 6.7 Software Advisory—Do not upgrade to Cisco HX Data Platform Release 4.0(1a) when running ESXi 6.7U1 EP06 (build # 11675023). Do not upgrade to 6.7U1 EP06 (build # 11675023) if running Cisco HX Data Platform Release 4.0(1a). See the Software Advisory CSCvo56350 for further details.
The software build version posted at release will override any other local versions.
Required vCenter upgrade—For enhanced security, Cisco HX Data Platform Release 3.5(1a) or later requires the use of TLS 1.2. Therefore, vCenter must be upgraded to 6.0 U3f or later before upgrading to Cisco HX Data Platform Release 3.5 or later. In addition, ESXi should be upgraded as required to meet HX Data Platform compatibility requirements.
Minimum HXDP version for upgrade—HX Data Platform clusters running 2.6(1a) or later may upgrade directly to 4.0 using the HX Connect UI.
Cluster Readiness—Ensure that the cluster is properly bootstrapped and the updated plug-in is loaded before proceeding. Manual cluster bootstrap is required for upgrade from a pre-3.5 release.
Cluster Readiness—Ensure that the cluster is properly bootstrapped and the updated plug-in is loaded before proceeding. Manual cluster bootstrap is required for HX releases earlier than 3.5(1a). For more information, see the Manual Bootstrap Upgrade Process in the Cisco HyperFlex Systems Upgrade Guide for VMware ESXi, Release 4.0. Do not skip this cluster bootstrap step, it is required for all upgrades until HX Release 3.5(1a). Auto bootstrap is supported beginning with HX release 3.5(1a). For more information, see the Auto Bootstrap Upgrade Process from HX Connect UI in the Cisco HyperFlex Systems Upgrade Guide for VMware ESXi, Release 4.0.
Manual bootstrap is not supported on Intersight clusters.
Initiating Upgrade―Use the HX
Connect UI or CLI stcli
commands
when upgrading from 2.5(1a) or later releases. Use
either the CLI stcli
commands or
the HX Data Platform Plug-in to the vSphere Web Client when upgrading from a pre-2.5(1a) release. The
vCenter plug-in should not be used for upgrades
starting with the 2.5(1a)
release.
If the current cluster version is at 3.5(1a) or above,
you do not need to use the stcli
command. Direct upgrade to 4.0 is possible.
Complete your Upgrade―The self-healing (or rebalance) capability is disabled temporarily during the upgrade window; If the upgrade fails, you should complete the upgrade as soon as possible.
ESXi and HXDP Compatibility―Ensure your cluster is running a compatible version of ESXi based on the running the HX Data Platform version (see the section Software Requirements for VMware ESXi). ESXi compatibility is determined by the major version and update release of ESXi. It is generally best to upgrade HXDP and ESXi together if combining the upgrade operations into a single optimized reboot. When running a split upgrade, first upgrade the HX Data Platform, then proceed to upgrade ESXi.
Uplinks from the UCS Fabric Interconnects to all top of rack switch ports must configure spanning tree in edge trunk or portfast edge mode depending on the vendor and model of the switch. This extra configuration ensures that when links flap or change state, they do not transition through unnecessary spanning tree states and incur an extra delay before traffic forwarding begins. Failure to properly configure FI uplinks in portfast edge mode may result in network and cluster outages during failure scenarios and during infrastructure upgrades that leverage the highly available network design native to HyperFlex.
vSphere 6.0 VMware’s last day of general support for vSphere 6.0 occurred on March 12, 2020. HXDP will continue to support vSphere 6.0 U3 on both 3.5(2x) and 4.0(2x) long lived releases. However, no bug or security fixes will be provided by VMware or Cisco for ESXi going forward due to reaching the last day of support. Cisco TAC will continue to support customers to the best of their ability on ESXi 6.0 builds that have already been released. Cisco strongly recommends upgrading as soon as possible to a supported VMware vSphere 6.5 or 6.7 release and follow Cisco’s recommendations as outlined in Recommended Cisco HyperFlex HX Data Platform Software Releases - for Cisco HyperFlex HX-Series Systems.
If Upgrading to vSphere 6.5:
Note |
|
vSphere 6.0 Upgrades—Users on vSphere 6.0 migrating to 6.5, upgrade components in the following order:
Upgrade HX Data Platform and UCS firmware.
Upgrade HX Data Platform and ESXi.
Upgrade HX Data Platform only first, then upgrade ESXi or UCS firmware or both.
M4 Server Firmware Upgrades—Upgrade server firmware to ensure smooth operation and to correct known issues. Specifically, newer SAS HBA firmware is available in this release and is recommended for long-term stability.
Users are encouraged to upgrade to 3.1(3c) C-bundle or later whenever possible.
Users running C-bundle versions before 3.1(2f) must upgrade server firmware by performing a combined upgrade of UCS server firmware (C-bundle) to 3.1(3c) or later and HX Data Platform to 2.5. Do not split the upgrade into two separate operations.
If the cluster is already on 3.1(2f) C-bundle or later, you may perform an HX Data Platform only or combined upgrade, as required.
M5 Server Firmware Upgrades—M5 generation servers must run firmware version 3.2(2d) or later.
Firmware Downgrades — Downgrading UCSM from the HX-installer is not supported.
M4/M5 Mixed Domains—A mixed domain occurs when a new, separate M5 cluster is installed under the same UCS domain that contains existing M4 clusters. Under these conditions, orchestrated UCS server firmware upgrade will not operate until Cisco HX Data Platform Release 2.6 or later is installed on the M4 clusters. Therefore, it is best practice to first upgrade UCS server firmware to the latest 3.1(3) or 3.2(2) patch release before adding a new M5 cluster to the existing UCS domain. Additionally, any 1.7 HX Data Platform clusters must first be upgraded before adding any new M5 clusters to the same domain.
Maintenance Window—If upgrading both HX Data Platform and UCS firmware, you can select either a combined or split upgrade through the vSphere HX Data Platform Plug-in depending on the length of the maintenance window. Cisco UCS Manager infrastructure upgrade is only supported using AutoInstall and the direct server firmware upgrade should be performed only through the upgrade orchestration framework provided by the HX Data Platform Plug-in.
Unsupported Self-Encrypting Drives (SEDs)—If adding or replacing self-encrypting drives (SEDs) that have been recently qualified in newer versions of HX Data Platform, insert the new drives only after upgrading HX Data Platform to a compatible version. All drives must be SED drives, mixing SED and non-SED is not supported.
Enabling External Host Access—With Cisco HX Data Platform Release 4.0(1a), port 445 on the management network is blocked for enhanced security. Note that prior to 4.0, port 445 port was open enabling external host access. If you are upgrading to 4.0(1a) from a prior release, and would like to continue external host access, you can use a utility to open select hosts. For more information about enabling external host access, see the "Configuring HyperFlex Share to SCVMM" section in the Installation Guide for Microsoft Hyper-V.
To upgrade from a supported release, see the upgrade recommendations in the Recommended Cisco HyperFlex HX Data Platform Software Releases - for Cisco HyperFlex HX-Series Systems.
If you want to upgrade from a release that is no longer supported, see the Cisco HyperFlex Systems Upgrade Guide for Unsupported Cisco HX Releases.
Cisco HyperFlex does not enforce, or have any dependency on the UCSM upgrade path. For more information about upgrading Cisco USC Manager see the Cisco UCS Install and Upgrade Guides.
Cisco HyperFlex does not enforce, or have any dependency on the VMware ESXi Upgrade path outside of the VMware upgrade guidelines. The recommended VMware ESXi download is located on the Cisco Software Downloads page with your Cisco HyperFlex Software download.
ESXi 5.5 support is deprecated with HXDP 2.5.
If running ESXi 5.5 U3 on HX220, contact TAC for upgrade guidance.
If running ESXi 5.5 U3 on HX240, see Guidelines and Limitations for further details.
If you have the ESXi 6.0 U1 version, we recommend an ESXi upgrade. There is a known VMware issue where the node becomes unresponsive due to a PSOD and OS crash. See the VMware Knowledge Base article, VMware ESXi 6.0, Patch ESXi600-201608401-BG: Updates esx-base, vsanhealth, vsan VIBs (2145664).
Upgrading the VM compatibility version or hardware version of the Storage Cluster Virtual Machine (SCVM) is not supported and should not be performed. This action is detrimental to the SCVM and will require a rebuild of the SCVM if performed.
Attention |
Upgrade to vCenter 6.0 U3f or later is required, due to TLS 1.2 support. Be sure to upgrade vCenter prior to upgrading the HX cluster. |
Cisco UCS Manager Version |
Cisco HX Data Platform |
|||||||||
---|---|---|---|---|---|---|---|---|---|---|
4.0(2x) |
4.0(1x) |
3.5(1x) |
3.0(1x) |
2.6(1x) |
2.5(1x) |
2.1(1x) |
2.0(1x) |
1.8(1x) |
1.7.x |
|
4.1(1c) |
Yes |
— |
— |
— |
— |
— |
— |
— |
— |
— |
4.0(4h) |
Yes |
— |
— |
— |
— |
— |
— |
— |
— |
— |
4.0(4e) |
Yes |
— |
— |
— |
— |
— |
— |
— |
— |
— |
3.2(3g) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
— |
— |
— |
— |
— |
|
3.2(3g) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
— |
— |
— |
— |
— |
|
3.2(3d) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
— |
— |
— |
— |
— |
|
3.2(2d) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash and M4 servers |
Yes Supports hybrid, All Flash, and M4 servers |
Yes Supports hybrid, All Flash and M4 servers |
Yes |
— |
|
3.1(3j) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, and M4 servers |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(3h) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, and M4 servers |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(3f) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, and M4 servers |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(3c) |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, M4 and M5 servers |
Yes Supports hybrid, All Flash, and M4 servers |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(2g) |
— |
— |
— |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(2f) |
— |
— |
— |
— |
— |
— |
Yes Supports hybrid and All Flash |
Yes |
— |
|
3.1(2b) |
— |
— |
— |
— |
— |
— |
Yes Supports hybrid |
Yes |
— |
SED-ready systems are HyperFlex clusters running HXDP 2.1(1b) with self encrypting drives (SEDs) installed. UCS Manager must be upgraded to 3.1(3c) or later. In addition, UCS server firmware (C-bundle) must be upgraded to 3.1(3c) or later. Either combined or split upgrade may be used, so long as all the cluster nodes are upgraded prior to enabling key management in HX Connect.
Caution |
During an upgrade, a flag-based check (True/False) is used to determine is the system is SED capable or not. If the system is SED-ready, this flag ( set to true) will not allow the non-SED systems to become part of the cluster. If there is an issue when SED capability information for cluster is gathered, the upgrade script might toggle this flag to the default value (False). In which case, all the SED drives on the node are replaced with non-SED disks and the upgrade proceeds with non-SED disks as well. Eventually, this may result in the risk of writing data on that node in unencrypted form. |
Before you begin upgrade of a Cisco HyperFlex System, consider the following cautions, guidelines, and limitations.
Important |
|
If you are running HyperFlex release 3.5(1a) or later, you can upgrade the Cisco HX Data Platform by performing the auto-bootstrap process from the HX Connect UI (Auto Bootstrap Upgrade Process from HX Connect UI).
Cisco recommends using GUI upgrade over CLI for ease of use and better reporting.
When Upgrade is complete, for each browser interface you use, empty the cache and reload the browser to page to refresh the HX content.
Ensure that all nodes (including compute nodes) are up and running and the cluster is healthy before starting an upgrade or other maintenance activities.
The Cisco HX Data Platform and Cisco UCS firmware bundles must be compatible. Refer UCS Hardware and Software Compatibility Matrix for more details.
For a split upgrade, Cisco HX Data Platform should be updated first before updating the Cisco UCS firmware.
During online upgrade, as one node is being upgraded (put into maintenance mode), the number of tolerated node failures is reduced based on the Data Replication Factor and Access Policy settings.
Only default TCP/IP stack is supported for vMotion vmkernel adapters.
All endpoints in a Cisco HyperFlex domain must be fully functional and all processes must be complete before you begin a firmware upgrade on those endpoints. For example, the firmware on a server that has not been discovered cannot be upgraded or downgraded. Each endpoint is a component in the Cisco HyperFlex domain that requires firmware to function.
In a three node cluster, if you shut down one node or put into maintenance mode it makes the cluster unhealthy, but the cluster is still online. If you need to perform manual maintenance, put the hosts in maintenance mode one at a time and move to the next host only after the cluster is healthy. For HXDP and UCS server firmware upgrades, this process is automatic.
Note |
You cannot remove a node from 3-node cluster by doing stcli node remove operation. To replace a node on a 3-node cluster, please contact Cisco TAC for assistance with the node replacement procedure. |