SAP HANA Systems with FusionIO Service Pack Upgrade Example

Introduction

This document describes the steps to upgrade software and systems development (SUSE) Linux Enterprise Server (SLES) 11 for Systems, Applications and Products in Data Processing (SAP) from Service Pack (SP)2 to SP3 on a C460 system with FusionIO cards. SAP High Performance Analytic Appliance (HANA) systems are delivered with the latest version of SLES 11 for SAP applications and driver versions at the time of the installation. Over the lifetime of the system, it is the customer's responsibility to update the environment with the latest security patches, updates, and kernel versions that might be requested by SAP or SUSE.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Linux Administration
• SAP HANA Administration

These components are required:

• SLES 11 for SAP OS that has been registered with Novell or
• SLES 11 SP3 installation media or local access to SP3 repositories
• FusionIO driver and utilities

Components Used

The information in this document is based on these software and hardware versions:

• C460 M2 with firmware 1.5(4a)
• SLES 11 SP2 with kernel 3.0.80-0.7
• SLES 11 SP3 with kernel 3.0.101-0.35
• FusionIO ioDrive 320GB with driver 3.2.3 build 950 and firmware 7.1.13

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Procedure

Whenever you perform major operations on the Operating System (OS), such as an upgrade to the kernel drivers, or a change to the kernel parameters, ensure that you have a backup of your HANA Log partition, preferably stored outside of the appliance. You should also read any pertinent release notes in the SAP, SUSE, and Cisco support channels.

For SAP HANA systems that contain FusionIO cards, such as the C460, the FusionIO drivers are specifically built for the running kernel. Thus, an upgrade to a new kernel requires additional steps.

Lastly, during the update process the HANA database must be stopped.

1. Stop the HANA Database

server01 :~ # su - <SID>adm
server01:/usr/sap/<SID>/HDB00 # HDB stop
hdbdaemon will wait maximal 300 seconds for NewDB services finishing.
Stopping instance using: /usr/sap/HAN/SYS/exe/hdb/sapcontrol
 -prot NI_HTTP -nr 00 -function StopWait 400 2


12.08.2014 23:46:34
Stop
OK

12.08.2014 23:47:02
StopWait
OK
hdbdaemon is stopped.

2. Back Up the Log Partition

After you stop the HANA database, move to the LOG area and back up the log partition.

server01 :~ # cd /hana/log
server01 :/hana/log # find . ?xdev | cpio ?oav > /backup/hana.log.cpio

3. Unmount the Log Partition and Modify fstab

Unmount the log partition and comment it out in /etc/fstab so it does not automatically mount the log partition after the reboot. This is needed because after a kernel upgrade the FusionIO drivers need to be rebuilt before the log partition can be mounted.

server01:~ # umount /hana/log
server01:~ # vi /etc/fstab
server01:~ # cat /etc/fstab | grep "log"
#/dev/md0 /hana/log xfs defaults 1 2

OS Upgrade

4. Upgrade SLES

For Service Pack upgrades follow the process documented at Novell's KB article 7012368.

If required, configure a proxy service so that the server can reach the mirrors.

server01:~ # cd /etc/sysconfig/
server01:/etc/sysconfig # vi proxy
PROXY_ENABLED="yes"
HTTP_PROXY="http://<COMPANY.COM>:8080"
HTTPS_PROXY="http://<COMPANY.COM>::8080"
FTP_PROXY=http://<COMPANY.COM>::8080

A kernel update requires a reboot, but DO NOT REBOOT YET.

Post Upgrade Tasks

5. Check menu.lst

During the kernel update process new GNU (GNU's Not Unix) GRand Unified Bootloader (GRUB) entries are created. Verify /boot/grub/menu.lst to make sure they reflect the specific Redundant Array of Independent Disks (RAID) configurations. 

The relevant entries are below the line root (hd0,0) and the lines that contain the new kernel.

server01:/boot/grub # cat menu.lst
# Modified by YaST2. Last modification on Wed Jul 24 18:27:21 GMT 2013
default 0
timeout 8
##YaST - generic_mbr
gfxmenu (hd0,0)/message
##YaST - activate
###Don't change this comment - YaST2 identifier: Original name: linux###
title SUSE Linux Enterprise Server 11 SP<#> - <VERSION> (default)
     root (hd0,0)
     kernel /vmlinuz-<VERSION>-default root=/dev/rootvg/rootvol
 resume=/dev/rootvg/swapvol splash=silent crashkernel=256M-:128M
 showopts intel_idle.max_cstate=0 vga=0x314 
     initrd /initrd-<VERSION>-default
###Don't change this comment - YaST2 identifier: Original name: failsafe###
title Failsafe -- SUSE Linux Enterprise Server 11 SP<#> - <VERSION> (default)
     root (hd0,0)
     kernel /vmlinuz-<VERSION>-default root=/dev/rootvg/rootvol showopts ide=nodma
 apm=off 
noresume edd=off powersaved=off nohz=off highres=off processor.max_cstate=1
 nomodeset x11failsafe intel_idle.max_cstate=0 vga=0x314 
     initrd /initrd-<VERSION>-default

6. Check Grub

In addition, start grub from the CLI and enter these commands:

GNU GRUB version 0.97 (640K lower / 3072K upper memory)
[ Minimal BASH-like line editing is supported. For the first word,
 TAB lists possible command completions. Anywhere else TAB lists the
 possible completions of a device/filename. ]
grub> device (hd0) /dev/sda
grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs_stage1_5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
Running "embed /boot/grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd0) /boot/grub/stage2 p /boot/grub/menu.lst "...
succeeded
grub>

7. Verify the OS Upgrade

It should now be safe to reboot the server. Enter the uname -a command in order to verify that the kernel was upgraded.

server01:/root # uname -a
Linux server01 3.0.101-0.35-default #1 SMP Wed Jul 9 11:43:04 UTC 2014 (c36987d)
 x86_64 x86_64 x86_64 GNU/Linux

8. Rebuild the FusionIO Drivers

8a. Use source RPM in /usr/src/packages/RPMS/ (Option 1)

server01:/ # find / -name "iomemory*.src.rpm"
/usr/src/packages/RPMS/x86_64/iomemory-vsl-<VERSION>.src.rpm

server01:/
 # rpmbuild --rebuild /usr/src/packages/RPMS/x86_64/iomemory-vsl-<VERSION>.src.rpm   
Installing iomemory-vsl-<VERSION>.src.rpm
...
Wrote:
 /usr/src/packages/RPMS/x86_64/iomemory-vsl-3.0.101-0.35-default-<VERSION>.x86_64.rpm

server01:/
 # rpm -ivh /usr/src/packages/RPMS/x86_64/iomemory-vsl-3.0.101-0.35-
default-<VERSION>.x86_64.rpm
Preparing...                ########################################### [100%]
   1:iomemory-vsl-3.0.101-0.########################################### [100%]

If the old driver is still listed, remove it.

server01:/recover/FusionIO # rpm -qa | grep vsl
iomemory-vsl-3.0.101-0.35-default-<VERSION>
iomemory-vsl-3.0.80-0.7-default-<VERSION>

server01:~ # rpm -e iomemory-vsl-3.0.80-0.7-default-<VERSION>

8b. Use source RPM in /recover partition (Option 2)

These instructions can also be used if newer drivers are desired. The latest FusionIO supported driver version can be found at KB 857 on FusionIO's support site.

server01:/recover # rpm -qa | grep vsl
iomemory-vsl-3.0.80-0.7-default-<VERSION>
libvsl-<VERSION>

server01:/recover # rpm -qi iomemory-vsl-3.0.80-0.7-default-<VERSION>
Name        : iomemory-vsl-3.0.80-0.7-default  Relocations: (not relocatable)
Version     : <VERSION>                         Vendor: Fusion-io
Release     : 1.0                           Build Date: Tue Aug 12 23:22:57 2014
Install Date: Tue Aug 12 23:27:46 2014         Build Host: server91.local
Group       : System Environment/Kernel     Source RPM: iomemory-vsl-<VERSION>.src.rpm
Size        : 5059825                          License: Proprietary
Signature   : (none)
URL         : http://support.fusionio.com/
Summary     : Driver for ioMemory devices from Fusion-io
Description :
Driver for fio devices
Distribution: (none)

The recover partition should have two folders with drivers: CISCO_FusionIO and FusionIO. The vendor for the RPM in this example is FusionIO.

server01:/recover # cd FusionIO
server01:/recover/FusionIO # ls *vsl*src.rpm
iomemory-vsl-<VERSION>.src.rpm

server01:/recover/FusionIO # rpmbuild -?rebuild iomemory-vsl-<VERSION>.src.rpm

Installing iomemory-vsl-<VERSION>.src.rpm
...
Wrote:
 /usr/src/packages/RPMS/x86_64/iomemory-vsl-3.0.101-0.35-default-<VERSION>.x86_64.rpm

server01:/recover/FusionIO # rpm -ivh
 /usr/src/packages/RPMS/x86_64/iomemory-vsl-3.0.101-0.35-default-<VERSION>.x86_64.rpm
Preparing...                ########################################### [100%]
   1:iomemory-vsl-3.0.101-0.########################################### [100%]

server01:/recover/FusionIO # rpm -qa | grep vsl
iomemory-vsl-3.0.101-0.35-default-<VERSION>

If the FusionIO driver is upgraded, the utilities should also be upgraded.

server01:/recover/FusionIO # cd Installed

server01:/recover/FusionIO/Installed # ls
fio-common-<VERSION>.x86_64.rpm    fio-util-<VERSION>.x86_64.rpm  libvsl-<VERSION>.x86_64.rpm
     fio-sysvinit-<VERSION>.x86_64.rpm 

server01:/recover/FusionIO/Installed # rpm -Uvh libvsl-<VERSION>.x86_64.rpm
Preparing...                ########################################### [100%]
   1:libvsl                 ########################################### [100%]

server01:/recover/FusionIO/Installed
 # rpm -Uvh fio-common-<VERSION>.x86_64.rpm fio-sysvinit-<VERSION>.x86_64.rpm
 fio-util-<VERSION>.x86_64.rpm
Preparing...                ########################################### [100%]
   1:fio-util               ########################################### [ 33%]
   2:fio-common             ########################################### [ 67%]
   3:fio-sysvinit           ########################################### [100%]
insserv: Service syslog is missed in the runlevels 4 to use service Framework
iomemory-vsl              0:off  1:on   2:on   3:on   4:on   5:on   6:off

9. Verify FusionIO Driver

After you install the new driver, the system needs to be rebooted in order to activate the driver. After you reboot, verify that the versions are correct.

server01:/root # rpm -qa | grep vsl
iomemory-vsl-3.0.101-0.35-default-<VERSION>
libvsl-<VERSION>

server01:/root # rpm -qa | grep fio
fio-common-<VERSION>
fio-util-<VERSION>
fio-sysvinit-<VERSION>

If the old driver is still listed, remove it.

server01:/root # rpm -qa | grep vsl
iomemory-vsl-3.0.101-0.35-default-<VERSION>
iomemory-vsl-3.0.80-0.7-default-<VERSION>

server01:~ # rpm -e iomemory-vsl-3.0.80-0.7-default-<VERSION>

Verify the driver status. You should see a version and the adapters should be online and attached.

server01:/root # fio-status | egrep -i "(driver|attached|online)"
Driver version: <version>
fct0    Attached
fioa    State: Online, Type: block device
fct1    Attached
fiob    State: Online, Type: block device

10. Undo fstab Changes

Remove the comment (#) and mount /hana/log in order to undo the changes in /etc/fstab.

11. Start the HANA Database

server01 :~ # su - <SID>admserver01:/usr/sap/<SID>/HDB00 # HDB start
StartService
OK
OK
Starting instance using: /usr/sap/HAN/SYS/exe/hdb/sapcontrol
 -prot NI_HTTP -nr 00 -function StartWait 2700 2


12.08.2014 23:57:56
Start
OK

12.08.2014 23:59:10
StartWait
OK