SAP HANA | SUSE Linux Enterprise Server 12 SP1 Migration

Introduction

This document describes an operating system upgrade process for SAP HANA solutions.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• SUSE Linux Enterprise Server (SLES) Administration 
• Cisco Unified Computing System (UCS) Administration

Components Used

The information in this document is based on these software and hardware versions:

• SUSE Linux Enterprise Server for SAP Applications 11 SP4
• Cisco UCS C260M2 server, firmware version 1.5(9d)
• LSI MegaRAID SAS 9261-8i adapter, firmware version 12.15.0-0249

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your SAP HANA deployment is live, ensure that you understand the potential impact of any command.

Related Products

This document can also be used with these hardware versions:

• Cisco UCS C460-M2
• Cisco UCS C460-M4
• CIsco C880-M4

Additional Considerations

The target service pack release needs to be listed as certified and supported in the SAP HANA hardware directory.

In order to be able to connect to the update channels, your SLES has to be registered. A proxy has to be configured if you don't have access to the update channels from within your data center. 

Before you start the operating system update it is recommended to check the related OSS notes, Cisco support channels and the Cisco UCS Hardware and Software compatibility regarding the most recent information. 

Download these items:

• latest Linux driver ISO image for your UCS firmware release
• first installation DVD of SUSE Linux Enterprise Server for SAP Applications 12 SP1
• driver update disk (DUD) from SUSE knowledge base article 7018320 - SLES for SAP migration from SLES 11 SP3/SP4 to SLES 12 GA/SP1 does not work offline.

The migration requires several reboots, and should be applied when SAP HANA is down during a maintenance window.

Note: Whenever you change the operating system or parts of it such as drivers or kernel parameters, ensure you have a valid backup at least of your SAP HANA system, preferably stored outside of the appliance.

Note: Some C460 configurations include a FusionIO card with specifically tailored drivers. Thus, installing a new kernel require additional steps. Ensure you've still the FusionIO drivers available been delivered with the golden image and stored in the /recover partition.

Background Information

The Cisco UCS Cx60 and Cisco C880 M4 SAP Hana Scale-Up appliances consist of the operating system SUSE Linux Enterprise Server for SAP Applications 11 SP3, which includes all software tools and kernel settings required to install and run SAP HANA.

To ensure system security and stability any further security updates, patches or additional software components as well as operating system configuration changes required by future SAP HANA releases need to be applied by the customer. Please refer to SAP HANA and SuSE Linux Enterprise Server (SLES) related SAP OSS notes in addition to this article.

It is best practice to keep the SUSE Linux Enterprise Server (SLES) operating system of your SAP HANA environment on a recent state and within the SUSE Product Support Lifecycle. There are multiple scenarios how to keep the operating system on a recent patch level. This article is intended to list some best practices for the most common scenarios and highlight implications you might encounter when updating the operating system.

SLES for SAP 12 SP1 Migration

The recommendation is to perform a fresh SLES for SAP 12 SP1 installation although it is possible to perform a direct migration to SLES for SAP Applications 12 SP1. This guide describes the online migration procedure. Optionally the system can be configured so the upgrade process starting automatically once the server is rebooted and the installation media KVM mapped.

The online migration can be started either from SLES for SAP Applications 11 SP3 or SP4 and the process is the same regardless of the source operating system release. There is no direct migration path for SLES (for SAP Applications) 11 SP2.

Caution: The update process has to be done completely from beginning to reboot. There is only a limited chance to revert changes. Furthermore, the server has to be connected online during the whole update process.

Procedure

Note: Remember to stop the SAP HANA Platform prior of updating the operating system and have the Linux ISO driver file available which contains the latest driver software. 

Preparation on Environments with FusionIO Cards

1. Ensure HANA is stopped
2. Backup the existing LOG area

# cd /hana/log
# find . -xdev | cpio -oav > /backup/hana.log.cpio

  3. Unmount /hana/log and comment the /hana/log line in /etc/fstab to disable automount after reboot

Apply the Latest Operating System Patches

Run the Online Update either using YAST or zypper from the command line.

zypper ref -s

Refreshing service 'nu_novell_com'.
All services have been refreshed.
Repository 'SLES-for-SAP-Applications 11.3.3-1.17' is up to date.
Repository 'SLE11-HAE-GEO-SP4-Pool' is up to date.
Repository 'SLE11-HAE-GEO-SP4-Updates' is up to date.
Repository 'SLE11-HAE-SP4-Pool' is up to date.
Repository 'SLE11-HAE-SP4-Updates' is up to date.
Repository 'SLE11-SP2-WebYaST-1.3-Pool' is up to date.
Repository 'SLE11-SP2-WebYaST-1.3-Updates' is up to date.
Repository 'SLE11-SP4-SAP-Pool' is up to date.
Repository 'SLE11-SP4-SAP-Updates' is up to date.
Repository 'SLES11-SP4-Pool' is up to date.
Repository 'SLES11-SP4-Updates' is up to date.
All repositories have been refreshed.

Run these commands to check the available patches:

• zypper pchk                            (check for available patches)
• zypper lp                                 (list the available patches)
• zypper lu                                 (list the updates including the version numbers)
• zypper lu | grep -i kernel      (search for kernel updates are included in the patching activity)

Update the system running these command twice: 

zypper update -t patch

Note: If dependencies of the lsi-megaraid_sas_kmp-default driver cannot be resolved during kernel patching choose solution 2. Deinstall the driver and reinstall the driver from the Linux ISO driver file after patching is complete and before rebooting.

Note: Some services and processes require a restart after patches have been applied. Do not reboot without changing the grub menu first to reflect the specific RAID configuration. 

Edit the file /boot/grub/menu.lst and replace all hd(0) entries with hd(0,0) to reflect the specifc RAID configuration:

title SLES for SAP Applications - 3.0.101-100 (default)
   root (hd0,0)
   kernel ...

title SLES for SAP Applications - 3.0.101-100 (default)
   kernel (hd0,0)/vmlinuz...
   initrd (hd0,0)/initrd...

At this point you can reboot the system.

Ensure that it starts with the newly installed kernel with this command uname -a

Migration autostart preparation

1. Mount the installation DVD you downloaded (SUSE Linux Enterprise Server for SAP Applications 12 SP1) to /mnt/dvd.
   
   
2. Copy the kernel and initial ram disk files.

cp /mnt/dvd/boot/x86_64/loader/linux /boot/linux.upgrade
cp /mnt/dvd/boot/x86_64/loader/initrd /boot/initrd.upgrade

   3. Umount the installation DVD again.

   4. Edit the file /boot/grub/menu.lst and add a new section for the upgrade kernel and initrd.

Note: The parameters IP_ADDRESS, NETMASK, GATEWAY and NAMESERVER correspond to your network settings. If your network setup requires a proxy server please specify the proxy server hostname and port as well. Otherwise the proxy parameter in the kernel line below can be skipped.

title Linux Upgrade Kernel
 kernel (hd0,0)/boot/linux.upgrade root=/dev/rootvg/rootvol resume=/dev/rootvg/swapvol splash=silent showopts vga=0x314 upgrade=1 ifcfg=eth3=IP_ADDRESS/NETMASK,GATEWAY,NAMESERVER dud=http://ftp.suse.com/pub/l3/bsc-1004665/bsc1004665-sap-rename-v2.dud proxy=PROXY.domain.com:PORT
 initrd (hd0,0)/boot/initrd.upgrade

  5. Confirm the FTP server is accessible from the command line.  

Migration Procedure

This is a migration to SLES for SAP 12 SP1.

Note: During the whole migration activity SAP HANA must be down.

Step 1. Launch the KVM Console.

Step 2. Map the first SLES for SAP Applications 12 SP1 Installation media. Reboot the server.

Step 3. Select the newly added Linux Upgrade Kernel section from the boot menu to start the SLES 12 SP1 installer. 

Step 4. Confirm the language and keyboard layout. Accept the license terms and continue.

Step 5. Before you select the partition or system to upgrade, select show all paritions from the check box and notice the notation of the /dev/md devices. Click Next.

Step 6. On the error message "The partition /dev/md0 could not be mounted" select Specify Mount Option and change the device name from /dev/md0 to /dev/md/linux:0

Step 7. On the error message "The partition /dev/md3 could not be mounted" select Specify Mount Option and change the device name from /dev/md3 to /dev/md/linux:3

Step 8. Review the repository updates available from the registration server and select those required if any. There might be a seperate registration code required to used those extensions. Click Next to continue. 

Step 9. Click Next on the Add-On Product Installation screen.

Step 10. Review the installation settings and scroll down and click on Booting to change, as shown in the image:

Step 11. Confirm and start the update.

Step 12. At approximately 96% the migration stops with an error regarding the boot configuration. Root cause is the missing post MBR gap in the specific RAID configuration and needs to be worked around.

Step 13. Select ><CTRL>-<ALT>-<F2> to switch to the bash prompt.

Note: Mount points in [ ] below denote mount points from the SLES 11 installation.

1. Identify devices mounted as /mnt [/] and /mnt/boot [/boot].
   
   

   # mount | grep mnt
   /dev/mapper/rootvg-rootvol on /mnt type ext3 (rw,relatime,data=ordered)
   /dev/md126 on /mnt/boot type ext3 (rw,relatime,data=ordered)

2. Umount /mnt/boot [/boot] identified as /dev/md126.
   
   

   # umount /mnt/boot

3. Mount [/boot] as temporary mount point /mnt/boot_tmp.
   
   

   # mkdir /mnt/boot_tmp
   # mount /dev/md126 /mnt/boot_tmp

4. Copy the content of [/boot] to /mnt/boot.
   
   

   # cp -av /mnt/boot_tmp/* /mnt/boot

5. Clean up and umount [/boot].
   
   

   # sync
   # umount /mnt/boot_tmp
   # rmdir /mnt/boot_tmp
   # rmdir /mnt/boot/lost+found

6. Remove /boot mount point from /mnt/etc/fstab.
   
   

7. Analyse the mdraid setup.
   
   

   # mdadm --detail --scan
   ARRAY /dev/md/linux:3 metadata=1.2 name=linux:3 UUID=b0b...
   ARRAY /dev/md/linux:0 metadata=1.0 name=linux:0 UUID=93a...
   ARRAY /dev/md/linux:1 metadata=1.2 name=linux:1 UUID=b6b...
   ARRAY /dev/md/linux:2 metadata=1.2 name=linux:2 UUID=da2...

8. Stop and dissemble the [/boot] raid.
   
   

   # mdadm --stop /dev/md/linux\:0
   # mdadm --zero-superblock /dev/sda1
   # mdadm --zero-superblock /dev/sdf1

9. Fix /mnt/etc/mdadm.conf and remove the line containing the information for /dev/md0
   
   

10. Delete the primary partition
    
    

    # parted /dev/sda rm 1
    # parted /dev/sdf rm 1

11. Confirm disk partitioning
    
    

    # sfdisk -l /dev/sda
     
     Disk /dev/sda: 72824 cylinders, 255 heads, 63 sectors/track
     Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
     
        Device Boot Start   End #cyls   #blocks Id System
     /dev/sda1          0     -     0         0  0 Empty
     /dev/sda2        130 72823 72694 583914555 fd Linux raid autodetect
     /dev/sda3          0     -     0         0  0 Empty
     /dev/sda4          0     -     0         0  0 Empty
     
     # sfdisk -l /dev/sdf
     
     Disk /dev/sdf: 72824 cylinders, 255 heads, 63 sectors/track
     Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
     
        Device Boot Start   End #cyls   #blocks Id System
     /dev/sdf1          0     -     0         0  0 Empty
     /dev/sdf2        130 72823 72694 583914555 fd Linux raid autodetect
     /dev/sdf3          0     -     0         0  0 Empty
     /dev/sdf4          0     -     0         0  0 Empty

Step 14. Select <CTRL>-<ALT>-<F7> to switch back to graphical installer.

Step 15. Review the settings and click ok to continue the update.

Step 16. Retry to write the boot loader configuration by answering yes. This opens the boot loader settings screen, as shown in the image:

Step 17. The system reboots automatically shortly after and finish the installation.

Post-Migration Procedure

Apply the latest operating system patches

Review no patches are in the pending state and run an Online Update either using YAST or zypper from the command line.

zypper ref -s
 
 Refreshing service 'SUSE_Linux_Enterprise_Server_for_SAP_Applications_12_SP1_x86_64'.
 All services have been refreshed.
 Repository 'SLE-12-SP1-SAP-12.1-0' is up to date.
 Retrieving repository 'SLE-12-SP1-SAP-Updates' metadata ...............................[done]
 Building repository 'SLE-12-SP1-SAP-Updates' cache ....................................[done]
 Retrieving repository 'SLE-HA12-SP1-Pool' metadata ....................................[done]
 Building repository 'SLE-HA12-SP1-Pool' cache .........................................[done]
 Retrieving repository 'SLE-HA12-SP1-Updates' metadata .................................[done]
 Building repository 'SLE-HA12-SP1-Updates' cache ......................................[done]
 Retrieving repository 'SLE12-SP1-SAP-Pool' metadata ...................................[done]
 Building repository 'SLE12-SP1-SAP-Pool' cache ........................................[done]
 Retrieving repository 'SLES12-SP1-Pool' metadata ......................................[done]
 Building repository 'SLES12-SP1-Pool' cache ...........................................[done]
 Retrieving repository 'SLES12-SP1-Updates' metadata ...................................[done]
 Building repository 'SLES12-SP1-Updates' cache ........................................[done]
 All repositories have been refreshed.

Update the system running this commandtwiceif required. 

zypper update -t patch
 Refreshing service 'SUSE_Linux_Enterprise_Server_for_SAP_Applications_12_SP1_x86_64'.
 Loading repository data...
 Reading installed packages...
 Resolving package dependencies...
 
 Nothing to do.

Caution: After completing any form of update, run the command rcrpmconfigcheck, then look at the contents of the file /var/adm/rpmconfigcheck. This file contains a list of configuration files that could not be updated automatically. These files must be checked and the configurations adjusted manually.

Review the latest SAP notes and confirm the requirements are fullfilled:

• SAP note 2205917 - SAP HANA DB: Recommended OS settings for SLES for SAP Applications 12 for any configuration changes. 
• SAP note 1984787 - SUSE Linux Enterprise Server 12: Installation notes for software selection and known issues.

Environments with FusionIO cards

1. Re-enable the LOG area in /etc/fstab
2. Mount /hana/log  

Related Information 

• Certified and supported SAP HANA hardware directory
• Cisco UCS Hardware and Software compatibility matrix
• Cisco Technical Support and Downloads
• SUSE Knowledge Base 7018320 - SLES for SAP migration from SLES 11 SP3/SP4 to SLES 12 GA/SP1 does not work offline
• SUSE Linux Enterprise Server for SAP Applications 12 SP1
• SUSE Documentation - Upgrading SUSE Linux Enterprise
• SAP Note 2205917 - SAP HANA DB: Recommended OS settings for SLES for SAP Applications 12
• SAP Note 2235581 - SAP HANA: Supported Operating Systems