Guidelines and Prerequisites

Guidelines, and Best Practices for Firmware Upgrades

Before you upgrade the firmware for any endpoint in a Cisco UCS domain, consider the following guidelines, best practices, and limitations:

Components That Support Firmware Upgrade

The various platforms that are supported by Cisco UCS Manager have different components that support firmware upgrade.

  • Fabric Interconnects:

    • Cisco UCS 6664 Fabric Interconnect

    • Cisco UCS X-Series Direct (UCSX-S9108-100G)

    • Cisco UCS 6536

    • Cisco UCS 64108

    • Cisco UCS 6454

  • Chassis components:

    • Blade server chassis:

      • I/O Modules/FI-IO Modules


        Note

        • I/O Modules are not supported on the Cisco UCS X-Series Direct, which uses the FI-I/O Modules instead.

      • Power Supply Unit

    • Cisco UCS S3260 chassis:

      • Chassis Management Controller (CMC)

      • Chassis Adapter

      • SAS Expander

      • Board Controller

  • Server components:

    • Blade and Rack server:

      • Adapter

      • Cisco Integrated Management Controller (CIMC)

      • BIOS

      • Storage Controller

      • Board Controller

    • Cisco UCS S3260 storage server node:

      • Cisco Integrated Management Controller (CIMC)

      • BIOS

      • Board Controller

      • Storage Controller

Firmware Version Terminology

The firmware version terminology used depends upon the type of endpoint, as follows:

Firmware Versions in CIMC, I/O Modules, BIOS, CIMC, and Adapters

Each CIMC, I/O module, BIOS, CIMC, and Cisco adapter has two slots for firmware in flash. Each slot holds a version of firmware. One slot is active and the other is the backup slot. A component boots from whichever slot is designated as active.

The following firmware version terminology is used in Cisco UCS Manager:

Running Version

The running version is the firmware that is active and in use by the endpoint.

Startup Version

The startup version is the firmware that will be used when the endpoint next boots up. Cisco UCS Manager uses the activate operation to change the startup version.

Backup Version

The backup version is the firmware in the other slot and is not in use by the endpoint. This version can be firmware that you have updated to the endpoint but have not yet activated, or it can be an older firmware version that was replaced by a recently activated version. Cisco UCS Manager uses the update operation to replace the image in the backup slot.

If the endpoint cannot boot from the startup version, it boots from the backup version.

Firmware Versions in the Fabric Interconnect and Cisco UCS Manager

You can only activate the fabric interconnect firmware and Cisco UCS Manager on the fabric interconnect. The fabric interconnect and Cisco UCS Manager firmware do not have backup versions, because all the images are stored on the fabric interconnect. As a result, the number of bootable fabric interconnect images is not limited to two, like the server CIMC and adapters. Instead, the number of bootable fabric interconnect images is limited by the available space in the memory of the fabric interconnect and the number of images stored there.

The fabric interconnect and Cisco UCS Manager firmware have running and startup versions of the kernel and system firmware. The kernel and system firmware must run the same versions of firmware.

Firmware- and Software-Related Guidelines for Upgrades

Before you upgrade any endpoint, consider the following guidelines and limitations:

Determine the Appropriate Type of Firmware Upgrade for Each Endpoint

Some endpoints, such as Cisco adapters and the server CIMC, can be upgraded through either a direct firmware upgrade or a firmware package included in a service profile. The configuration of a Cisco UCS domain determines how you upgrade these endpoints. If the service profiles associated with the servers include a host firmware package, upgrade the adapters for those servers through the firmware package.

Upgrades of an adapter through a firmware package in the service profile associated with the server take precedence over direct firmware upgrades. You cannot directly upgrade an endpoint if the service profile associated with the server includes a firmware package. To perform a direct upgrade, you must remove the firmware package from the service profile.

Do Not Activate All Endpoints Simultaneously in Cisco UCS Manager GUI

If you use Cisco UCS Manager GUI to update the firmware, do not select ALL from the Filter drop-down list in the Activate Firmware dialog box to activate all endpoints simultaneously. Many firmware releases and patches have dependencies that require the endpoints to be activated in a specific order for the firmware update to succeed. This order can change depending upon the contents of the release or patch. Activating all endpoints does not guarantee that the updates occur in the required order, and can disrupt communications between the endpoints and the fabric interconnects and Cisco UCS Manager. For information about the dependencies in a specific release or patch, see the release notes provided with that release or patch.

Determine Available Bootflash and Workspace Partition

The bootflash partition is dedicated solely to firmware images managed by Cisco UCS Manager. To initiate upgrade or downgrade, at least 20 percent of the bootflash partition must be available. When the bootflash partition exceeds 70 percent, faults are raised, but Auto Install proceeds. When the bootflash partition exceeds 80 percent, faults are raised and Auto Install does not proceed.

The workspace partition on the fabric interconnect stores tech support files, core files, and the debug plugin. To initiate upgrade or downgrade, at least 20 percent of the workspace partition must be available.

Determine the Impact of Activation for Adapters and I/O Modules

During a direct upgrade, you should configure Set Startup Version Only for an adapter. With this setting, the activated firmware moves into the pending-next-boot state, and the server is not immediately rebooted. The activated firmware does not become the running version of firmware on the adapter until the server is rebooted. You cannot configure Set Startup Version Only for an adapter in the host firmware package.

If a server is not associated with a service profile, the activated firmware remains in the pending-next-boot state. Cisco UCS Manager does not reboot the endpoints or activate the firmware until the server is associated with a service profile. If necessary, you can manually reboot or reset an unassociated server to activate the firmware.

When you configure Set Startup Version Only for an I/O module, the I/O module is rebooted when the fabric interconnect in its data patch is rebooted. If you do not configure Set Startup Version Only for an I/O module, the I/O module reboots and disrupts traffic. In addition, if Cisco UCS Manager detects a protocol and firmware version mismatch between the fabric interconnect and the I/O module, Cisco UCS Manager automatically updates the I/O module with the firmware version that matches the firmware in the fabric interconnect and then activates the firmware and reboots the I/O module again.

Disable Call Home before Upgrading to Avoid Unnecessary Alerts (Optional)

When you upgrade a Cisco UCS domain, Cisco UCS Manager restarts the components to complete the upgrade process. This restart causes events that are identical to the service disruptions and component failures that trigger Call Home alerts to be sent. If you do not disable Call Home before you begin the upgrade, alerts will be generated by the upgrade-related component, restarts and notifications will be sent out based on your Call Home configuration.

Fabric Interconnect Traffic Evacuation

Upgrading the subordinate fabric interconnect in a system disrupts the traffic that is active on the fabric interconnect. This traffic fails over to the primary fabric interconnect.


Important

  • Fabric interconnect traffic evacuation is supported only in a cluster configuration.

  • You can evacuate traffic only from the subordinate fabric interconnect.

  • The IOM or FEX backplane ports of the fabric interconnect on which evacuation is configured will go down, and their state will appear as Admin down. During the manual upgrade process, to move these backplane ports back to the Up state and resume traffic flow, you must explicitly configure Admin Evac Mode as Off.

You can perform fabric evacuation as follows during the manual upgrade process:

  1. Stop all the traffic that is active through a fabric interconnect by configuring Admin Evac Mode as On.

  2. For vNICs configured with failover, verify that the traffic has failed over by using Cisco UCS Manager or tools such as vCenter.

  3. Upgrade the subordinate fabric interconnect.

  4. Restart all the stopped traffic flows by configuring Admin Evac Mode as Off.

  5. Change the cluster lead to the subordinate fabric interconnect.

  6. Repeat steps 1 to 4 and upgrade the other fabric interconnect.

Fabric Evacuation with Auto Install

Starting with Cisco UCS Manager Release 3.1(3), you can use fabric evacuation during Auto Install. While initiating Auto Install, when you enable fabric evacuation and then begin Auto Install, the following sequence of events occur:

  1. The subordinate fabric interconnect (FI-B) is evacuated and activated.

  2. Failover occurs and the primary fabric interconnect (FI-A) becomes the subordinate fabric interconnect. FI-B now becomes the cluster lead.

  3. FI-A is now evacuated and activated.

If you use fabric evacuation with Auto Install, and fabric evacuation was enabled on the fabric interconnect before Auto Install, fabric evacuation is disabled after Auto Install is complete.

Ensure that you do not initiate Auto Install with fabric evacuation enabled on the primary fabric interconnect. If fabric evacuation was manually enabled on the primary fabric interconnect before Auto Install, it must be manually disabled before initiating Auto Install.


Note

  • Fabric interconnect traffic evacuation is supported only in a cluster configuration.

  • You can evacuate traffic only from the subordinate fabric interconnect.

  • The IOM or FEX backplane ports of the fabric interconnect on which evacuation is configured will go down, and their state will appear as Admin down. These backplane ports will move back to Up state after Auto Install is complete.

Configuring Fabric Interconnect Traffic Evacuation

You can use the steps detailed here, or click Play on this video (http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/videos/3-1/enable_and_disable_fi_traffic_evacuation.html) to watch how to enable and disable fabric interconnect traffic evacuation.

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Fabric Interconnects > Fabric_Interconnect_Name.

Step 3

In the Work pane, click the General tab.

Step 4

In the Actions area of the General tab, click Configure Evacuation.

The Configure Evacuation dialog box appears.

Step 5

To configure evacuation of the traffic through the specified fabric interconnect, click one of the following radio buttons in the Admin Evac Mode field:

  • On—Stops all the traffic that is active through the specified fabric interconnect.
  • Off—Restarts traffic through the specified fabric interconnect.

Step 6

(Optional) To evacuate the traffic through a fabric interconnect irrespective of its current evacuation state, check the Force check box.

Step 7

Click Apply.

A warning dialog box appears. 
Enabling fabric evacuation will stop all traffic through this Fabric Interconnect from servers attached through IOM/FEX. 
The traffic will fail over to the Primary Fabric Interconnect for fail over vnics.
Are you sure you want to continue?

Step 8

Click OK to confirm fabric interconnect traffic evacuation and continue.

Secure Firmware Update

Cisco UCS Manager, Release 3.1(2) introduces secure firmware update, which enables you to update the adapter firmware securely for third-party Intel network and storage adapters. Only server administrators can upgrade or downgrade firmware for the adapters. OS administrators with root privileges are not allowed to downgrade the adapter firmware.

The following Cisco UCS servers support secure firmware update:

Secure Firmware Update Supported Network Adapters and Storage Disks
Supported Storage Disks on Cisco Blade Servers

The following Intel NVMe storage disks support secure firmware update on Cisco UCS B200 M5 Server and Cisco UCS B480 M5 Server.

Table 1. Supported NVMe Storage Disks

NVMe Storage Disks

UCSC-NVMEHW-H800

UCSC-NVMEHW-H1600

UCSC-NVMEHW-H3200

UCSC-NVMEHW-H6400

UCSC-NVMEHW-H7680
Supported Network Adapters and Storage Disks on Cisco Rack Servers

The following NVMe storage disks support secure firmware update on Cisco UCS C220 M5 Server, Cisco UCS C240 M5 Server, and Cisco UCS C480 M5 Server servers:

Table 2. Supported NVMe Storage Disks

NVMe Storage Disks

UCSC-NVMEHW-H800

UCSC-NVMEHW-H1600

UCSC-NVMEHW-H3200

UCSC-NVMEHW-H6400

UCSC-NVMEHW-H7680

UCSC-NVME-H16003 to UCSC-F-H16003

UCSC-NVME-H32003

UCSC-NVME-H38401

UCSC-NVME-H64003

UCSC-NVME-H76801
Guidelines for Secure Firmware Support on Cisco UCS Servers

Cisco UCS Manager Release 3.1(2) introduces support for secure firmware update. For Cisco UCS M5 servers, secure firmware update is introduced in Cisco UCS Manager Release 3.2(2).


Important

Ensure that CIMC is running Version 2.0(13) or later and Cisco UCS Manager is running Release 3.1(2) or later releases. Secure firmware update cannot be done when the CIMC is running a version earlier than 2.0(13) and Cisco UCS Manager is running a release earlier than Release 3.1(2).

Guidelines for Blade Servers

For secure firmware update on , B200 M5, and B480 M5 servers, do the following:

  • Install the UCSB-LSTOR-PT storage controller and insert the NVMe disks on a Cisco UCS B200 M5 or B480 M5 server.

  • Reacknowledge the server. Refer to the Reacknowledging a Blade Server section in the Cisco UCS Manager Infrastructure Management Guide, Release 3.2.


    Note

    Ensure that server discovery does not fail and the NVMe disks are identified by CIMC and BIOS. After the server is associated to the service profile with the default host firmware package, Auto Install is triggered. NVMe disks can be updated with the latest firmware during Auto Install.

    Cisco UCS Manager, Release 3.2(1) supports NVMe boot.

Guidelines for Rack Servers

For secure firmware update on Cisco UCS C460, C240, C220 M5 servers and C480 M5 servers, do the following:

  • Reacknowledge the Cisco UCS servers. Refer to the Reacknowledging a Rack Server section in the Cisco UCS Manager Infrastructure Management Guide, Release 3.2.


    Note

    Ensure that server discovery does not fail and the NVMe disks are identified by CIMC and BIOS. After the server is associated to the service profile with the default host firmware package, Auto Install is triggered. NVMe disks can be updated with the latest firmware during Auto Install.

    Cisco UCS Manager, Release 3.2(1) supports NVMe boot.

Cautions, and Guidelines for Upgrading with Auto Install

Before you use Auto Install to upgrade the firmware for any endpoint in a Cisco UCS domain, consider the following cautions, guidelines, and limitations:


Note

These guidelines are specific to Auto Install and are in addition to those listed in Guidelines, and Best Practices for Firmware Upgrades.

State of the Endpoints

Before you begin an upgrade, all affected endpoints must be as follows:

  • For a cluster configuration, verify that the high availability status of the fabric interconnects shows that both are up and running.

  • For a non-cluster configuration, verify that the Overall Status of the fabric interconnect is Operable.

  • For all endpoints to be upgraded, verify that they are in an Operable state.

  • For all servers to be upgraded, verify that all the servers have been discovered and that discovery did not fail. Install Server Firmware will fail if any server endpoints cannot be upgraded.

  • For each server to be upgraded, check the running firmware version on the storage controller and local disks, and verify that they are in the Ready state.

Recommendations for the Default Host Firmware Policy

After you upgrade Cisco UCS Manager, a new host firmware policy named "default" is created, and is assigned to all service profiles that did not already include a host firmware policy. The default host firmware policy is blank. It does not contain any firmware entries for any components. This default policy is also configured for an immediate reboot rather than waiting for user acknowledgment before rebooting the servers.

During the upgrade of server firmware, you can modify the default host firmware policy to add firmware for the blade and rack-mount servers in the Cisco UCS domain. To complete the upgrade, all servers must be rebooted.

Every service profile that is assigned to the default host firmware policy reboots the associated server according to the maintenance policy included in the service profile. If the maintenance policy is set to immediate reboot, you cannot cancel the upgrade or prevent the servers from rebooting after you complete the configuration in the Install Server Firmware wizard. We recommend that you verify the maintenance policy associated with these service profiles to ensure that they are set for a timed reboot or for user acknowledgment.

Time, Date, and Time Zone on Fabric Interconnects Must Be Identical

To ensure that the fabric interconnects in a cluster configuration are in sync, you must ensure that they are configured for the same date, time, and time zone. We recommend that you configure an NTP server and the correct time zone in both fabric interconnects. If the date, time or time zone in the fabric interconnects are out of sync, the Auto Install might fail.

Cannot Upgrade Infrastructure and Server Firmware Simultaneously

You cannot upgrade the infrastructure firmware at the same time as you upgrade server firmware. We recommend that you upgrade the infrastructure firmware first and then upgrade the server firmware. Do not begin the server firmware upgrade until the infrastructure firmware upgrade is completed.

Required Privileges

Users must have the following privileges to upgrade endpoints with Auto Install:

Privileges Upgrade Tasks User Can Perform

admin

  • Run Install Infrastructure Firmware

  • Run Install Server Firmware

  • Add, delete, and modify host firmware packages

Service profile compute (ls-compute)

Run Install Server Firmware

Service profile server policy (ls-server-policy)

Add, delete, and modify host firmware packages

Service profile config policy (ls-config-policy)

Add, delete, and modify host firmware packages

Impact of Host Firmware Packages on Install Server Firmware

Because Install Server Firmware uses host firmware packages to upgrade the servers, you do not have to upgrade all servers in a Cisco UCS domain to the same firmware versions. However, all servers which have associated service profiles that include the host firmware packages you selected when you configured Install Server Firmware are upgraded to the firmware versions in the specified software bundles.

Effect of Using Install Server Firmware on Servers Whose Service Profiles Do Not Include a Host Firmware Package

If you use Install Server Firmware to upgrade server endpoints on servers that have associated service profiles without host firmware packages, Install Server Firmware uses the default host firmware package to upgrade the servers. You can only update the default host firmware package through Install Server Firmware.

If you want to upgrade the CIMC or adapters in a server with an associated service profile that has previously been updated through the default host firmware package in Install Server Firmware, you must use one of the following methods:

  • Use Install Server Firmware to modify the default host firmware package and then upgrade the server through Install Server Firmware.

  • Create a new host firmware package policy, assign it to the service profile associated with the server, and then upgrade the server through that host firmware package policy.

  • Disassociate the service profile from the server and then directly upgrade the server endpoints.

Upgrading Server Firmware on Newly Added Servers

If you add a server to a Cisco UCS domain after you run Install Server Firmware, the firmware on the new server is not automatically upgraded by Install Server Firmware. If you want to upgrade the firmware on a newly added server to the firmware version used when you last ran Install Server Firmware, you must manually upgrade the endpoints to upgrade the firmware on that server. Install Server Firmware requires a change in firmware version each time. You cannot rerun Install Server Firmware to upgrade servers to the same firmware version.


Note

After you finish the upgrade, you can use the Firmware Auto Sync Server policy in Cisco UCS Manager to automatically update newly discovered servers.

Cautions, and Guidelines Limitations for Managing Firmware in Cisco UCS Central

Before you start managing Cisco UCS Manager firmware from Cisco UCS Central, consider the following cautions, guidelines and limitations:

  • The firmware policies you define for a domain group will be applied to any new Cisco UCS Domain added to this domain group. If a firmware policy is not defined in the domain group, Cisco UCS Domain will inherit the policy from the parent domain group.

  • The global policies will remain global in Cisco UCS Manager even when Cisco UCS Manager loses connection with Cisco UCS Central. If you want to apply any changes to any of the policies that are global in Cisco UCS Manager, you must change the ownership from global to local.

  • When you create a host firmware package from Cisco UCS Central, it must be associated to a service profile to deploy updates in Cisco UCS domains.

  • When you modify a host firmware package in Cisco UCS Central, the changes are applied to Cisco UCS domains during the next maintenance schedule associated with the host firmware update.

  • The host firmware maintenance policies you define in Cisco UCS Central apply to the org-root in Cisco UCS domains. You cannot define separate host maintenance policies for sub organizations in a Cisco UCS Domain from Cisco UCS Central.

  • Any server with no service profile association will get upgraded to the default version of the host firmware pack. Since these servers do not have a maintenance policy, they will reboot immediately.

  • If you specify a maintenance policy in Cisco UCS Central and enable user acknowledgment and do not specify a schedule, you can acknowledge the pending task only from Cisco UCS Manager. To acknowledge pending activities from Cisco UCS Central, you must schedule maintenance using global schedulers and enable user acknowledgment.

  • When you schedule a maintenance policy in Cisco UCS Central and enable user acknowledgment, that task will be displayed on the pending activities tab at the time specified in the schedule.

  • You can view the pending activity for a maintenance policy only from the domain group section.

  • Make sure to enable user acknowledgment for any firmware schedule to avoid any unexpected reboot in the Cisco UCS domains.


Note

For more information on managing firmware in Cisco UCS Central, see the Firmware Management chapters in the Cisco UCS Central Administration Guide and Cisco UCS Central CLI Reference Manual.

Prerequisites for Upgrading and Downgrading Firmware

All endpoints in a Cisco UCS domain must be fully functional and all processes must be complete before you begin a firmware upgrade or downgrade on those endpoints. You cannot upgrade or downgrade an endpoint that is not in a functional state.

For example, the firmware on a server that has not been discovered cannot be upgraded or downgraded. An incomplete process, such as an FSM that has failed after the maximum number of retries, can cause the upgrade or downgrade on an endpoint to fail. If an FSM is in progress, Cisco UCS Manager queues up the update and activation and runs them when the FSM has completed successfully.

Colored boxes around components on the Equipment tab may indicate that an endpoint on that component cannot be upgraded or downgraded. Verify the status of that component before you attempt to upgrade the endpoints.


Note

The Installed Firmware tab in Cisco UCS Manager GUI does not provide sufficient information to complete these prerequisites.

Before you upgrade or downgrade firmware in a Cisco UCS domain, complete the following tasks:

  • Review the release notes for the Cisco UCS Manager version you plan to upgrade/downgrade.


    Note

    Ensure to check the section titled Deprecated Hardware and Software in Cisco UCS Manager in the Release Notes. This section identifies hardware and software features that are no longer supported, helping you avoid issues with unsupported components during your upgrade or operations.

  • Back up the configuration into an All Configuration backup file.

  • For a cluster configuration, verify that the high availability status of the fabric interconnects shows that both are up and running.

  • For a non-cluster configuration, verify that the Overall Status of the fabric interconnect is Operable.

  • Verify that the data path is up and running. For more information, see the Verification that the Data Path is Ready section.

  • Verify that all servers, I/O modules, and adapters are fully functional. An inoperable server cannot be upgraded.

  • Verify that the Cisco UCS domain does not include any critical or major faults. If such faults exist, you must resolve them before you upgrade the system. A critical or major fault may cause the upgrade to fail.

  • Verify that all servers have been discovered. They do not need to be powered on or associated with a service profile.

  • If you want to integrate a rack-mount server into the Cisco UCS domain, follow the instructions in the appropriate C-Series Rack-Mount Server Integration Guide for installing and integrating a rack-mount server in a system managed by Cisco UCS Manager.

  • For Cisco UCS domains that are configured for iSCSI boot, do the following before you upgrade to Cisco UCS, Release 3.1(1) or higher:

    • Ensure that all iSCSI vNICs used across multiple service profiles have unique initiator names.

    • If any iSCSI vNICs have the same initiator name within a service profile, Cisco UCS reconfigures the service profile to have a single unique initiator name.

    • Make the corresponding IQN initiator name changes on any network storage devices to ensure that the boot LUNs are visible to the new IQN.

If Fibre Channel ports on Cisco UCS Fabric Interconnect are connected to non-Cisco products, ensure that these Fibre Channel ports are operating as individual Fibre Channel links and not aggregated into a port channel.


Note
Fibre Channel port channels are not compatible with non-Cisco technology.

Pre-Upgrade Validation Checks

Ensure that you complete the following pre-upgrade validation checks before installing firmware:

Create Backup Files

When you perform a backup through Cisco UCS Manager, you take a snapshot of all or part of the system configuration and export the file to a location on your network. You can perform a backup while the system is up and running. The backup operation only saves information from the management plane. It does not have any impact on the server on network traffic.

Cisco recommends that you create the following backup files before beginning a Cisco UCS firmware upgrade:

  • All Configuration backup file—An XML backup of all the system and logical configuration

  • Full State backup file—A binary snapshot of the entire system

Creating an All Configuration Backup File

This procedure assumes that you do not have an existing backup operation for an All Configuration backup file.

Before you begin

Obtain the backup server IPv4 or IPv6 address and authentication credentials.

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Click the All node.

Step 3

In the Work pane, click the General tab.

Step 4

In the Actions area, click Backup Configuration.

Step 5

In the Backup Configuration dialog box, click Create Backup Operation.

Step 6

In the Create Backup Operation dialog box, do the following:

  1. Complete the following fields:

    • Admin State field—Click the Enabled radio button to run the backup operation as soon as you click OK.

    • Type field—Click the All Configuration radio button to create an XML backup file that includes all system and logical configuration information.

      Click the Full State radio button to create a binary file that includes a snapshot of the entire system.

    • Preserve Identities check box—If the Cisco UCS domain includes any identities derived from pools that you need to preserve, check this check box.

      If this checkbox is selected for Logical Configuration type of backup operation, the backup file preserves all identities derived from pools, including vHBAs, WWPNs, WWNN, vNICs, MACs and UUIDs.

      Note

       

      If this check box is not selected the identities will be reassigned and user labels will be lost after a restore.

    • Location of the Backup File field—To save the backup file on your local file system, click the Local File System radio button. To save the backup file on a remote file system, click the Local File System radio button.

      If the location is set to Local File System, Cisco UCS Manager GUI displays the Filename field. If it is set to Remote File System, Cisco UCS Manager GUI displays the rest of the fields described below.

    • Filename field—Click Browse to navigate to a new location within the local file system.

    • Protocol field—Click the one of the following radio buttons to indicate the protocol you want to use to transfer the file to the backup server:

      • FTP

      • TFTP

      • SCP

      • SFTP

    • Hostname field—Enter the IP address or hostname of the location where the backup file is to be stored. This can be a server, storage array, local drive, or any read/write media that the fabric interconnect can access through the network. If you use a hostname, you must configure Cisco UCS Manager to use a DNS server.

    • Remote File field—Enter the full path to the backup configuration file. This field can contain the filename as well as the path. If you omit the filename, the backup procedure assigns a name to the file.

    • User field—Enter the username that Cisco UCS Manager should use to log in to the backup location. You do not need to complete this field if you selected TFTP for the protocol.

    • Password field—Enter the password associated with the username. You do not need to complete this field if you selected TFTP for the protocol.

  2. Click OK.

Step 7

If Cisco UCS Manager displays a confirmation dialog box, click OK.

If you set the Admin State field to enabled, Cisco UCS Manager takes a snapshot of the configuration type that you selected and exports the file to the network location. The backup operation displays in the Backup Operations table in the Backup Configuration dialog box.

Step 8

(Optional) To view the progress of the backup operation, do the following:

  1. If the operation does not display in the Properties area, click the operation in the Backup Operations table.

  2. In the Properties area, click the down arrows on the FSM Details bar.

The FSM Details area expands and displays the operation status.

Step 9

Click OK to close the Backup Configuration dialog box.

The backup operation continues to run until it is completed. To view the progress, re-open the Backup Configuration dialog box.

Creating a Full State Backup File

Before you begin

Obtain the backup server IPv4 or IPv6 address and authentication credentials.

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Click the All node.

Step 3

In the Work pane, click the General tab.

Step 4

In the Actions area, click Backup Configuration.

Step 5

In the Backup Configuration dialog box, click Create Backup Operation.

Step 6

In the Create Backup Operation dialog box, do the following:

  1. Complete the following fields:

    • Admin State field—Click the Enabled radio button to run the backup operation as soon as you click OK.

    • Type field—Click the Full State radio button to create a binary file that includes a snapshot of the entire system.

    • Preserve Identities check box—If the Cisco UCS domain includes any identities derived from pools that you need to preserve, check this check box.

      If this checkbox is selected for Logical Configuration type of backup operation, the backup file preserves all identities derived from pools, including vHBAs, WWPNs, WWNN, vNICs, MACs and UUIDs.

      Note

       

      If this check box is not selected the identities will be reassigned and user labels will be lost after a restore.

    • Location of the Backup File field—To save the backup file on your local file system, click the Local File System radio button. To save the backup file on a remote file system, click the Remote File System radio button.

      If the location is set to Local File System, Cisco UCS Manager GUI displays the Filename field. If it is set to Remote File System, Cisco UCS Manager GUI displays the rest of the fields described below.

    • Filename field—Click Browse to navigate to a new location within the local file system.

    • Protocol field—Click the one of the following radio buttons to indicate the protocol you want to use to transfer the file to the backup server:

      • FTP

      • TFTP

      • SCP

      • SFTP

    • Hostname field—Enter the IP address or hostname of the location where the backup file is to be stored. This can be a server, storage array, local drive, or any read/write media that the fabric interconnect can access through the network. If you use a hostname, you must configure Cisco UCS Manager to use a DNS server.

    • Remote File field—Enter the full path to the backup configuration file. This field can contain the filename as well as the path. If you omit the filename, the backup procedure assigns a name to the file.

    • User field—Enter the username that Cisco UCS Manager should use to log in to the backup location. You do not need to complete this field if you selected TFTP for the protocol.

    • Password field—Enter the password associated with the username. You do not need to complete this field if you selected TFTP for the protocol.

  2. Click OK.

Step 7

If Cisco UCS Manager displays a confirmation dialog box, click OK.

If you set the Admin State field to enabled, Cisco UCS Manager takes a snapshot of the configuration type that you selected and exports the file to the network location. The backup operation displays in the Backup Operations table in the Backup Configuration dialog box.

Step 8

(Optional) To view the progress of the backup operation, do the following:

  1. If the operation does not display in the Properties area, click the operation in the Backup Operations table.

  2. In the Properties area, click the down arrows on the FSM Details bar.

The FSM Details area expands and displays the operation status.

Step 9

Click OK to close the Backup Configuration dialog box.

The backup operation continues to run until it is completed. To view the progress, re-open the Backup Configuration dialog box.

Configure Cisco Smart Call Home for Firmware Upgrade

Cisco Smart Call Home is a web application that leverages the Call Home feature of Cisco UCS. Smart Call Home offers proactive diagnostics and real-time email alerts of critical system events, which results in higher network availability and increased operational efficiency. Smart Call Home is a secure connected service offered by Cisco Unified Computing Support Service and Cisco Unified Computing Mission Critical Support Service for Cisco UCS. The Cisco UCS Manager Administration Management Guide provides detailed information about configuring Smart Call Home.

When you upgrade firmware, Cisco UCS Manager restarts the components to complete the upgrade process. This restart can trigger email alerts. Disabling Smart Call Home will avoid creating such alerts and automatic support cases with TAC during the firmware upgrade process.

Disabling Smart Call Home

Before you begin

Smart Call Home must previously be enabled.

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Communication Management > Call Home.

Step 3

In the Work pane, click the General tab.

Step 4

In the Admin area, do the following to disable Smart Call Home:

  1. In the State field, click Off.

    Note

     

    If this field is set to On, Cisco UCS Manager GUI displays the rest of the fields on this tab.

Call Home alerts are not generated until Smart Call Home is enabled again.

Fault Suppression During Firmware Upgrade

Fault suppression allows you to suppress SNMP trap and Call Home notifications during a planned maintenance time. You can create a fault suppression task to prevent notifications from being sent whenever a transient fault is raised or cleared.

Faults remain suppressed until the time duration has expired, or the fault suppression tasks have been manually stopped by the user. After the fault suppression has ended, Cisco UCS Manager will send notifications for any outstanding suppressed faults that have not been cleared.

Enabling fault suppression for any component during firmware upgrade suppresses the faults related to that component until the time duration has expired, or until the component comes back up after upgrade. For example, if fabric interconnect faults are configured to be suppressed during firmware upgrade, no faults triggered by the fabric interconnect going down during upgrade will be displayed.

Viewing UCS Manager Faults

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Faults, Events, and Audit Log.

Step 3

Click Faults.

Step 4

In the Work pane, check the All check box.

Step 5

Ensure that there are no service-impacting faults.

Faults Generated Due to Reboot During the Upgrade of a Fabric Interconnect

It is essential to ensure that port configurations and services that go down when the fabric interconnect reboots are re-established after the fabric interconnect comes back up.

Starting with Cisco UCS Manager Release 3.1, Cisco UCS Manager displays any service that is not re-established after the last reboot of a fabric interconnect. Cisco UCS Manager creates a baseline of the outstanding faults before a fabric interconnect is to be rebooted. After the fabric interconnect reboots and comes up, you can view the new faults generated since the last baseline to identify the services that went down because of the fabric reboot.

When a specific interval of time has passed after Cisco UCS Manager created a baseline of the outstanding faults, baselining is cleared and all faults show up as new faults. This interval is called "baseline expiration interval". Modifying Baseline Expiration Interval for Faults, provides detailed information about modifying a baseline expiration interval in Cisco UCS Manager.

Cisco recommends that you resolve service-impacting faults before you continue with the fabric interconnect reboot or evacuation.

Modifying Baseline Expiration Interval for Faults

You can modify a baseline expiration interval in Cisco UCS Manager.

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Faults, Events, and Audit Log.

Step 3

In the Work pane, click the Settings tab, and then click the Global Fault Policy subtab.

Step 4

In the Baseline Expiration Interval area, update the dd:hh:mm:ss field.

The dd:hh:mm:ss field specifies the number of days, hours, minutes, and seconds that should pass before Cisco UCS Manager clears the fault baseline.

The default baseline expiration interval is 24 hours.

Step 5

Click Save Changes.

Viewing UCS Manager Faults

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Faults, Events, and Audit Log.

Step 3

Click Faults.

Step 4

In the Work pane, check the All check box.

Step 5

Ensure that there are no service-impacting faults.

Viewing Faults Generated During the Upgrade of a Fabric Interconnect

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Faults, Events, and Audit Log.

Step 3

In the Work pane, click the Faults tab.

All faults generated after creating the baseline are displayed.

Verifying vNIC Configuration for Fabric Failover

In a Cisco UCS system, fabric failure may happen when one of the following occurs:

  • Fabric Interconnect fails, which results in fabric failure for all chassis connected to the fabric interconnect

  • FEX fails, which results in fabric failure for the chassis connected to the FEX

  • Link between a Fabric Interconnect and a FEX fails, which results in fabric failure for some of the servers in the chassis connected to the specific FEX

  • CNA port failure, which results in fabric failure for a server

When redundant hardware is in place, and vNICs are configured for failover, fabric failure will result in fabric failover. Ensure that vNICs are configured for fabric failover before firmware upgrade.

Procedure

Step 1

In the Navigation pane, click Servers.

Step 2

Expand Servers > Service Profiles > Service_Profile_Name.

Step 3

Expand the specified service profile and select vNICs.

Step 4

Expand vNICs and select the first vNIC configured for the specified service profile.

Step 5

In the Work pane, click the General tab.

Step 6

In the Properties area, verify that the Fabric ID is Fabric A and the Enable Failover checkbox is checked.

Step 7

In the Navigation pane, select the next vNIC configured for the specified service profile.

Step 8

In the Work pane, click the General tab.

Step 9

In the Properties area, verify that the Fabric ID is Fabric B and the Enable Failover checkbox is checked.

Step 10

Repeat Steps 4 through 9 until all the vNICs in the specified service profile are verified.

Important

 

To ensure that failover happens, ensure that alternate vNICs are pinned to Fabric A and

Fabric B.

Verifying the Operability of the Fabric Interconnects

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Fabric Interconnects.

Step 3

Click the node for the fabric interconnect that you want to verify.

Step 4

In the Work pane, click the General tab.

Step 5

In the Status area, verify that the Overall Status is operable.

If the status is not operable, create and download a Tech Support file, and contact Cisco Technical Support. Do not proceed with the firmware upgrade. For more information about Tech Support files, see the Cisco UCS Manager B-Series Troubleshooting Guide.

Verifying the High Availability Status and Roles of a Cluster Configuration

The high availability status is the same for both fabric interconnects in a cluster configuration.

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Fabric Interconnects.

Step 3

Click the node for one of the fabric interconnects in the cluster.

Step 4

In the Work pane, click the General tab.

Step 5

If the fields in the High Availability Details area are not displayed, click the Expand icon to the right of the heading.

Step 6

Verify that the following fields display the following values:

Field Name Required Value

Ready field

Yes

State field

Up

If the values are different, create and download a Tech Support file, and contact Cisco Technical Support. Do not proceed with the firmware upgrade. For more information about Tech Support files, see the Cisco UCS Manager B-Series Troubleshooting Guide.

Step 7

Note the value in the Leadership field to determine whether the fabric interconnect is the primary or subordinate unit.

You need to know this information to upgrade the firmware on the fabric interconnects.

Configuring the Default Maintenance Policy

Some modifications to a service profile or to an updating service profile template can be disruptive and require a reboot of the server. A maintenance policy determines how Cisco UCS Manager reacts when a change that requires a server reboot is made to a service profile associated with a server or to an updating service profile bound to one or more service profiles.

The maintenance policy specifies how Cisco UCS Manager deploys the service profile changes. The deployment can occur in one of the following ways:

  • Immediately

  • When acknowledged by a user with admin privileges

  • Automatically at the time specified in a schedule

  • When the server boots again

You can use the steps detailed here, or click Play on this video (http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/videos/3-1/configure_the_default_maintenance_policy.html)to watch how to configure the default maintenance policy as User Ack.

Procedure

Step 1

In the Navigation pane, click Servers.

Step 2

Expand Servers > Policies.

Step 3

Expand the node for the organization where you want to create the policy.

If the system does not include multi tenancy, expand the root node.

Step 4

Expand Maintenance Policies and click default.

Step 5

In the Work pane, click the Main tab.

Step 6

In the Properties area, select User Ack as the Reboot Policy.

The On Next Boot check box appears.

You must reboot the server manually after the service profile association is complete or after changes are made.

Step 7

(Optional) To enable the On Next Boot option, check the On Next Boot checkbox.

When the On Next Boot option is enabled, the host OS reboot, shutdown, and reset, or server reset and shutdown also triggers the associated FSM to apply the changes waiting for the User Ack maintenance window.

Step 8

Click Save Changes.

Disabling the Management Interface

Before firmware upgrade, shut down the management interface of the secondary fabric interconnect. This ensures that any active KVM connections between any server and the management interface will reset. The GUI flow fails over to the primary fabric interconnect and reduces the time that you are disconnected from the GUI.

If Cisco UCS Manager detects a management interface failure, a failure report is generated. If the configured number of failure reports is reached, the system assumes that the management interface is unavailable and generates a fault. By default, the management interfaces monitoring policy is enabled. The Cisco UCS Manager System Monitoring Guide provides more details about the Management Interfaces Monitoring Policy.

Procedure

Step 1

In the Navigation pane, click Admin.

Step 2

Expand All > Communication Management.

Step 3

Click Management Interfaces.

Step 4

In the Work pane, click the Management Interfaces tab and verify the management IP address of the fabric interconnects.

Step 5

Click the Management Interfaces Monitoring Policy tab and, in the Admin Status field, click the Enabled radio button to enable the monitoring policy for management interfaces.

If Cisco UCS Manager detects a management interface failure, it generates a failure report.

Step 6

Open a Telnet session to the upstream switch connected to the fabric interconnect.

Step 7

Verify the configuration of the interface to which the fabric interconnect management port is connected, and disable it using the shut command on the switch.

Any KVM session that is open through this interface terminates.

Step 8

Reconnect KVM sessions to ensure that these sessions are not impacted by upgrade of the secondary fabric interconnect.

Verifying the Status of I/O Modules

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Chassis.

Step 3

Click on the chassis for which you want to verify the status of the I/O modules.

Step 4

In the Work pane, click the IO Modules tab.

Step 5

For each I/O module, verify that the following columns display the following values:

Field Name Desired Value

Overall Status column

ok

Operability column

operable

If the values are different, create and download a Tech Support file, and contact Cisco Technical Support. Do not proceed with the firmware upgrade. For more information about Tech Support files, see the Cisco UCS Manager B-Series Troubleshooting Guide.

Step 6

Repeat Steps 3 through 5 to verify the status of the I/O modules in each chassis.

Verifying the Status of Servers

If a server is inoperable, you can proceed with the upgrade for other servers in the Cisco UCS domain. However, you cannot upgrade the inoperable server.

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

In the Work pane, click the Servers tab to display a list of all servers in all chassis.

Step 3

For each server, verify that the following columns display the following values:

Field Name Desired Value

Overall Status column

ok, unassociated, or any value that does not indicate a failure.

If the value indicates a failure, such as discovery-failed, the endpoints on that server cannot be upgraded.

Operability column

operable

Step 4

If you need to verify that a server has been discovered, do the following:

  1. Right-click the server for which you want to verify the discovery status and choose Show Navigator.

  2. In the Status Details area of the General tab, verify that the Discovery State field displays a value of complete.

    If the fields in the Status Details area are not displayed, click the Expand icon to the right of the heading.

Verifying the Status of Adapters on Servers in a Chassis

Procedure

Step 1

In the Navigation pane, click Equipment.

Step 2

Expand Equipment > Chassis > Chassis Number > Servers.

Step 3

Click the server for which you want to verify the status of the adapters.

Step 4

In the Work pane, click the Inventory tab.

Step 5

In the Inventory tab, click the Adapters subtab.

Step 6

For each adapter, verify that the following columns display the following values:

Field Name Desired Value

Overall Status column

ok

Operability column

operable

If the fields show a different value and the adapter is inoperable, you can proceed with the upgrade for other adapters on the servers in the Cisco UCS domain. However, you cannot upgrade the inoperable adapter.

UCS Manager Health and Pre-Upgrade Check Tool

The UCS Manager Health and Pre-Upgrade Check Tool provides automated health and pre-upgrade checks that are designed to ensure your clusters are healthy before you upgrade. It is imperative that this healthcheck is not just performed, but that you take corrective action on any cluster that is found to be unhealthy. Correct all issues reported by the UCS Manager health check before continuing.

Verification that the Data Path is Ready


Note

We recommend that you follow the guidelines prior to any processes that require reboot of both Fabric Interconnects.

Ensure that you monitor the VIF paths and counts only from the CLI and not within the Cisco UCS Manager GUI.

The following sections detail the steps to verify that the data path is ready.

Verifying that Dynamic vNICs Are Up and Running

When you upgrade a Cisco UCS that includes dynamic vNICs and an integration with VMware vCenter, you must verify that all dynamic vNICs are up and running on the new primary fabric interconnect. Ensure that the vNICs are up and running before you activate the new software on the former primary fabric interconnect to avoid data path disruption.

Perform this step in the Cisco UCS Manager GUI.

Procedure

Step 1

In the Navigation pane, click VM.

Step 2

Expand All > VMware > Virtual Machines.

Step 3

Expand the virtual machine for which you want to verify the dynamic vNICs and choose a dynamic vNIC.

Step 4

In the Work pane, click the VIF tab.

Step 5

On the VIF tab, verify that the Status column for each VIF is Online.

Step 6

Repeat Steps 3 through 5 until you have verified that the VIFs for all dynamic vNICs on all virtual machines have a status of Online.

Verifying the Ethernet Data Path

Procedure

  Command or Action Purpose

Step 1

UCS-A /fabric-interconnect # connect nxos {a | b}

Enters NX-OS mode for the Fabric Interconnect.

Step 2

UCS-A(nxos)# show int br | grep -v down | wc –l

Returns the number of active Ethernet interfaces.

Verify that this number matches the number of Ethernet interfaces that were up prior to the upgrade.

Step 3

Use the following command:

Option Description
show hardware mac address-table 1 | wc -l

Returns the total number of MAC addresses on Cisco UCS 6600 Series Fabric Interconnect, Cisco UCS X-Series Direct, Cisco UCS 6500 Series, and Cisco UCS 6400 Series Fabric Interconnects.

Verify that this number matches the number of MAC addresses prior to the upgrade.

Example

The following example returns the number of active Ethernet interfaces and MAC addresses for subordinate UCS 6400 Series Fabric Interconnect A so that you can verify that the Ethernet data path for that Fabric Interconnect is up and running: 


UCS-A /fabric-interconnect # connect nxos a
UCS-A(nxos)# show int br | grep -v down  | wc -l
86
UCS-A(nxos)# show hardware mac address-table 1 | wc -l
80

Verifying the Data Path for Fibre Channel End-Host Mode

For best results when upgrading a Cisco UCS domain, we recommend that you perform this task before you begin the upgrade and after you activate the subordinate fabric interconnect, and then compare the two results.

Procedure

  Command or Action Purpose

Step 1

UCS-A /fabric-interconnect # connect nxos {a | b}

Enters NX-OS mode for the fabric interconnect.

Step 2

UCS-A(nxos)# show npv flogi-table

Displays a table of flogi sessions.

Step 3

UCS-A(nxos)# show npv flogi-table | grep fc | wc -l

Returns the number of servers logged into the fabric interconnect.

The output should match the output you received when you performed this verification prior to beginning the upgrade.

Example

The following example displays the flogi-table and number of servers logged into subordinate fabric interconnect A so that you can verify that the Fibre Channel data path for that fabric interconnect in Fibre Channel End-Host mode is up and running: 


UCS-A /fabric-interconnect # connect nxos a
UCS-A(nxos)# show npv flogi-table
--------------------------------------------------------------------------------
SERVER                                                                  EXTERNAL
INTERFACE VSAN FCID             PORT NAME               NODE NAME       INTERFACE
--------------------------------------------------------------------------------
vfc705    700  0x69000a 20:00:00:25:b5:27:03:01 20:00:00:25:b5:27:03:00 fc3/1
vfc713    700  0x690009 20:00:00:25:b5:27:07:01 20:00:00:25:b5:27:07:00 fc3/1
vfc717    700  0x690001 20:00:00:25:b5:27:08:01 20:00:00:25:b5:27:08:00 fc3/1
 
Total number of flogi = 3.

UCS-A(nxos)# show npv flogi-table | grep fc | wc -l
3

Verifying the Data Path for Fibre Channel Switch Mode

For best results when upgrading a Cisco UCS domain, we recommend that you perform this task before you begin the upgrade and after you activate the subordinate fabric interconnect, and then compare the two results.

Procedure

  Command or Action Purpose

Step 1

UCS-A /fabric-interconnect # connect nxos {a | b}

Enters NX-OS mode for the fabric interconnect.

Step 2

UCS-A(nxos)# show flogi database

Displays a table of flogi sessions.

Step 3

UCS-A(nxos)# show flogi database | grep –I fc | wc –1

Returns the number of servers logged into the fabric interconnect.

The output should match the output you received when you performed this verification prior to beginning the upgrade.

Example

The following example displays the flogi-table and number of servers logged into subordinate fabric interconnect A so that you can verify that the Fibre Channel data path for that fabric interconnect in Fibre Channel End-Host mode is up and running: 


UCS-A /fabric-interconnect # connect nxos a
UCS-A(nxos)# show flogi database
--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
--------------------------------------------------------------------------------
vfc726           800   0xef0003  20:00:00:25:b5:26:07:02 20:00:00:25:b5:26:07:00
vfc728           800   0xef0007  20:00:00:25:b5:26:07:04 20:00:00:25:b5:26:07:00
vfc744           800   0xef0004  20:00:00:25:b5:26:03:02 20:00:00:25:b5:26:03:00
vfc748           800   0xef0005  20:00:00:25:b5:26:04:02 20:00:00:25:b5:26:04:00
vfc764           800   0xef0006  20:00:00:25:b5:26:05:02 20:00:00:25:b5:26:05:00
vfc768           800   0xef0002  20:00:00:25:b5:26:02:02 20:00:00:25:b5:26:02:00
vfc772           800   0xef0000  20:00:00:25:b5:26:06:02 20:00:00:25:b5:26:06:00
vfc778           800   0xef0001  20:00:00:25:b5:26:01:02 20:00:00:25:b5:26:01:00
 
Total number of flogi = 8.
UCS-A(nxos)# show flogi database | grep fc | wc -l
8