-
Check the Controller Node status and move the Pacemaker Cluster Stack (PCS) to maintenance mode.
-
Login to the primary Controller Node (controller-0) from the OSP-D Server.
-
Check the state of the Controller Node Pacemaker Cluster Stack (PCS).
sudo pcs status
Note
|
Resolve any issues prior to proceeding to the next step.
|
-
Place the PCS cluster on the Controller Node into standby mode.
sudo pcs cluster standby <controller_name>
-
Recheck the Controller Node status again and make sure that the Controller Node is in standby mode for the PCS cluster.
sudo pcs status
-
Log on to the Ultra M Manager Node.
-
Upgrade the BIOS on the primary UCS server-based Controller Node (controller-0).
./ultram_ucs_utils.py --cfg “controller_0.cfg” --login <cimc_username> <cimc_user_password> --upgrade bios --server <rhel_introspection_ip_address> --timeout 30 --file /firmwares/bios.cap
Example output:
2017-09-29 09:15:48,753 - Updating BIOS firmware on all the servers
2017-09-29 09:15:48,753 - Logging on UCS Server: 192.100.2.7
2017-09-29 09:15:48,758 - No session found, creating one on server: 192.100.2.7
2017-09-29 09:15:50,194 - Login successful to server: 192.100.2.7
2017-09-29 09:16:13,269 - 192.100.2.7 => updating | Image Download (5 %), OK
2017-09-29 09:17:26,669 - 192.100.2.7 => updating | Write Host Flash (75 %), OK
2017-09-29 09:18:34,524 - 192.100.2.7 => updating | Write Host Flash (75 %), OK
2017-09-29 09:19:40,892 - 192.100.2.7 => Activating BIOS
2017-09-29 09:19:55,011 -
---------------------------------------------------------------------
Server IP | Overall | Updated-on | Status
---------------------------------------------------------------------
192.100.2.7 | SUCCESS | NA | Status: success, Progress: Done, OK
Note
|
The Compute Nodes are automatically powered down after this process leaving only the CIMC interface available.
|
-
Upgrade the UCS server using the Host Upgrade Utility (HUU).
./ultram_ucs_utils.py --cfg “controller_0.cfg” --login <cimc_username> <cimc_user_password> --upgrade huu --server <rhel_introspection_ip_address> --file /firmwares/<ucs_huu_iso_filename>
Note
|
This software is available via the HTTP Apache server.
|
If the HUU script times out before completing the upgrade, the process might still be running on the remote hosts. You can
periodically check the upgrade process by entering:
./ultram_ucs_utils.py --cfg “controller_0.cfg” --login <cimc_username> <cimc_user_password> --status huu-upgrade
Example output:
---------------------------------------------------------------------
Server IP | Overall | Updated-on | Status
---------------------------------------------------------------------
192.100.2.7 | SUCCESS | 2017-10-20 07:10:11 | Update Complete CIMC Completed, SasExpDN Completed, I350 Completed, X520 Completed, X520 Completed, 3108AB-8i Completed, UCS VIC 1227 Completed, BIOS Completed,
---------------------------------------------------------------------
-
Verify that the BIOS firmware and HUU upgrade was successful by checking the post-upgrade versions.
./ultram_ucs_utils.py --cfg “controller_0.cfg” --login <cimc_username> <cimc_user_password> --status firmwares
-
Set the package-c-state-limit CIMC setting.
./ultram_ucs_utils.py --mgmt set-bios --bios-param biosVfPackageCStateLimit --bios-values vpPackageCStateLimit=C0/C1 --cfg controller_0.cfg --login <cimc_username> <cimc_user_password>
-
Verify that the package-c-state-limit CIMC setting has been made.
./ultram_ucs_utils.py --status bios-settings --cfg controller_0.cfg --login <cimc_username> <cimc_user_password>
Look for PackageCStateLimit to be set to C0/C1.
-
Modify the Grub configuration on the primary OSD Compute Node.
-
Log on to the OSD Compute (osd-compute-0) and update the grub setting with "processor.max_cstate=0 intel_idle.max_cstate=0".
sudo grubby --info=/boot/vmlinuz-`uname -r`
sudo grubby --update-kernel=/boot/vmlinuz-`uname -r` --args="processor.max_cstate=0
intel_idle.max_cstate=0"
-
Verify that the update was successful.
sudo grubby --info=/boot/vmlinuz-`uname -r`
Look for the "processor.max_cstate=0 intel_idle.max_cstate=0" arguments in the output.
-
Reboot the OSD Compute Nodes.
sudo reboot
-
Recheck all CIMC and kernel settings.
-
Verify the processor c-state.
cat /sys/module/intel_idle/parameters/max_cstate
cpupower idle-info
-
Login to Ultra M Manager Node.
-
Verify CIMC settings.
./ultram_ucs_utils.py --status bios-settings --cfg controller_0.cfg --login <cimc_username> <cimc_user_password>
-
Check the ironic node-list and restore the Controller Node if it went into maintenance mode true state.
-
Login to OSP-D and make sure to "su - stack" and "source stackrc".
-
Perform the check and any required restorations.
ironic node-list
ironic node-set-maintenance $NODE_<node_uuid> off
-
Take the Controller Node out of the PCS standby state.
sudo pcs cluster unstandby <controller-0-id>
-
Wait 5 to 10 minutes and check the state of the PCS cluster to verify that the Controller Node is ONLINE and all services
are in good state.
sudo pcs status
-
Repeat steps 3 through 11 on the second Controller Node (controller-1).
Note
|
Be sure to use the controller_1.cfg file where needed.
|
-
Repeat steps 3 through 11 on the third Controller Node (controller-2).
Note
|
Be sure to use the controller_2.cfg file where needed.
|
-
Proceed to Upgrade Firmware on the OSP-D Server/Ultra M Manager Node.
Feedback