Cisco and Hitachi Adaptive Solutions for SAP HANA TDI

Deployment Guide for Cisco and Hitachi Converged Infrastructure with Cisco UCS Blade Servers, Cisco Nexus 9336C-FX2 Switches, Cisco MDS 9706 Fabric Switches, and Hitachi VSP G370 Storage Systems with SUSE Linux Enterprise Server for SAP Applications 12 SP4 and Red Hat Enterprise Linux 7.5

Published: April 2, 2019

Updated: February 7, 2020

NOTE: Works with document’s Advanced Properties “Last Updated” property. Click File | Properties | Advanced Properties | Custom.

About the Cisco Validated Design Program

The Cisco Validated Design (CVD) program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information, go to:

http://www.cisco.com/go/designzone.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS.  CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.  IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE.  USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS.  THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS.  USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS.  RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

CCDE, CCENT, Cisco Eos, Cisco Lumin, Cisco Nexus, Cisco StadiumVision, Cisco TelePresence, Cisco WebEx, the Cisco logo, DCE, and Welcome to the Human Network are trademarks; Changing the Way We Work, Live, Play, and Learn and Cisco Store are service marks; and Access Registrar, Aironet, AsyncOS, Bringing the Meeting To You, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, CCVP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unified Computing System (Cisco UCS), Cisco UCS B-Series Blade Servers, Cisco UCS C-Series Rack Servers, Cisco UCS S-Series Storage Servers, Cisco UCS Manager, Cisco UCS Management Software, Cisco Unified Fabric, Cisco Application Centric Infrastructure, Cisco Nexus 9000 Series, Cisco Nexus 7000 Series. Cisco Prime Data Center Network Manager, Cisco NX-OS Software, Cisco MDS Series, Cisco Unity, Collaboration Without Limitation, EtherFast, EtherSwitch, Event Center, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient, IOS, iPhone, iQuick Study,  LightStream, Linksys, MediaTone, MeetingPlace, MeetingPlace Chime Sound, MGX, Networkers, Networking Academy, Network Registrar, PCNow, PIX, PowerPanels, ProConnect, ScriptShare, SenderBase, SMARTnet, Spectrum Expert, StackWise, The Fastest Way to Increase Your Internet Quotient, TransPath, WebEx, and the WebEx logo are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.

All other trademarks mentioned in this document or website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0809R)

© 2020 Cisco Systems, Inc. All rights reserved.

 

Table of Contents

Executive Summary. 6

Solution Overview.. 7

Introduction. 7

Audience. 7

Purpose of this Document 7

Solution Design. 9

Architecture. 9

Deployment Hardware and Software. 11

Hardware and Software Versions. 11

Configuration Guidelines. 12

Physical Cabling. 14

Cisco Nexus 9000 Series Switch Network Configuration. 20

Cisco Nexus 9000 Initial Configuration. 20

Enable Appropriate Cisco Nexus 9000 Series Switches Features and Settings. 21

Create VLANs for SAP HANA Traffic. 22

Configure Virtual Port-Channel Domain. 22

Configure Network Interfaces for the VPC Peer Links. 23

Configure vPCs with Cisco UCS Fabric Interconnect 25

Configuration of Hitachi Storage. 29

Storage Architecture Overview.. 29

Log into Storage Navigator 34

Check SFP Data Transfer Rate. 34

Create Pool Volumes. 36

Create Dynamic Provisioning Pools. 39

Provision the LUNS (Virtual Volumes) 41

Create Virtual Volumes for the Operating System LUNS and Map Ports. 41

Create Virtual Volumes for HANA Shared File System and Map Ports. 43

Create Virtual Volumes for Log LUNs and Map Ports. 43

Create Virtual Volumes for Data LUNs and Map Ports. 44

Storage Port Configuration. 45

Configure the Host Groups. 49

Cisco UCS Configuration Overview.. 51

Physical Connectivity. 51

Upgrade Cisco UCS Manager Software to Version 4.0(1c) 51

Initial Setup of Cisco UCS 6332-16UP Fabric Interconnects. 51

Cisco UCS 6332-16UP Fabric Interconnect B. 52

Cisco UCS Manager Setup. 53

Log into Cisco UCS Manager 53

Anonymous Reporting. 55

Synchronize Cisco UCS to NTP. 56

Configure Cisco UCS Servers. 56

Chassis Discovery Policy. 56

Configure Server Ports. 57

Configure FC SAN Uplink Ports. 58

Configure Ethernet Uplink Ports. 59

Acknowledge Cisco UCS Chassis. 60

Power Policy. 61

Create New Organization. 61

Create Pools. 62

Add Block of IP Addresses for KVM Access. 62

Create MAC Address Pools. 63

Create WWNN Pool 65

Create WWPN Pool 66

Create UUID Suffix Pool 68

Set Packages and Policies. 70

Create Host Firmware Package. 70

Create Server BIOS Policy. 71

Power Control Policy. 74

Create Serial over LAN Policy. 75

Update Default Maintenance Policy. 76

Network Control Policy. 77

Configure Cisco UCS LAN Connectivity. 78

Set Jumbo Frames in Cisco UCS Fabric. 78

Create LAN Uplink Port Channels. 79

VLAN Configurations. 84

Configure Cisco UCS SAN Configurations. 101

Create FC Port Channels. 101

Create Boot Policy for SAN Boot 112

Create Service Profile Templates for SAP HANA Scale Up Servers. 115

Create Service Profile from the Template. 126

Configure Cisco MDS 9706 Switches. 128

Physical Connectivity. 128

Cisco MDS Initial Configuration Dialogue. 128

Cisco MDS Switch Configuration. 130

Configure Fibre Channel Ports and Port Channels. 130

Configure VSANs. 131

Create and Configure Fiber Channel Zoning. 132

Operating System Installation. 137

Associate Service Profile to Cisco UCS Server 137

SLES for SAP 12 SP4 OS Installation. 137

Network Services Configuration. 158

SLES for SAP 12 SP 4 System Update and OS Customization. 161

Install Cisco eNIC and fNIC Driver 162

Multipath Configuration. 163

Red Hat Enterprise Linux for SAP Solutions 7.5 OS Installation. 165

Network Services Configuration. 182

RHEL 7.5 System Update and OS Customization for SAP HANA. 186

Install Cisco eNIC and fNIC Driver 188

Network Time. 189

Multipath Configuration. 189

Configure HANA Persistent Storage Volume Configuration. 192

SAP HANA Persistent Storage Volume Configuration for Scale-Up Deployments. 193

Configuration Example on SUSE Linux Enterprise Server for SAP Applications. 193

Configuration Example on Red Hat Enterprise Linux. 196

SAP HANA Installation. 199

HDBPARAM Parameters. 199

SAP HANA 1.0. 200

SAP HANA 2.0. 200

References. 201

Certified SAP HANA Hardware Directory. 201

SAP HANA TDI Documentation. 201

Important SAP Notes. 201

SAP HANA IMDB Related Notes. 201

Linux Related Notes. 201

Cisco. 202

Hitachi Storage. 203

Cisco Intersight Registration. 204

Prerequisites. 204

Setup Information. 205

Cisco Intersight Licensing. 205

Deployment Steps. 205

About the Authors. 210

Acknowledgements. 210

 

 

Cisco Validated Designs consist of systems and solutions that are designed, tested, and documented to facilitate and improve customer deployments. These designs incorporate a wide range of technologies and products into a portfolio of solutions that have been developed to address the business needs of our customers.

Cisco and Hitachi are working together to deliver a converged infrastructure solution that helps enterprise businesses meet the challenges of today and position themselves for the future. Leveraging decades of industry expertise and superior technology, this Cisco CVD offers a resilient, agile, and flexible foundation for today’s businesses. In addition, the Cisco and Hitachi partnership extends beyond a single solution, enabling businesses to benefit from their ambitious roadmap of evolving technologies such as advanced analytics, IoT, cloud, and edge capabilities. With Cisco and Hitachi, organizations can confidently take the next step in their modernization journey and prepare themselves to take advantage of new business opportunities enabled by innovative technology.

This document explains the deployment of the Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration, as it was described in Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration Design Guide. The recommended solution architecture is built on Cisco Unified Computing System (Cisco UCS) using the unified software release to support the Cisco UCS hardware platforms for Cisco UCS B-Series blade servers, Cisco UCS 6300 Fabric Interconnects, Cisco Nexus 9000 Series switches, Cisco MDS Fiber channel switches, and Hitachi VSP controllers. This architecture supports Red Hat Enterprise Linux and SUSE Linux Enterprise Server for SAP Applications.

Introduction

Enterprise data centers have a need for scalable and reliable infrastructure that can be implemented in an intelligent, policy driven manner. This implementation needs to be easy to use, and deliver application agility, so IT teams can provision applications quickly and resources can be scaled up (or down) in minutes.

Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration provides a best practice datacenter architecture built on the collaboration of Hitachi Vantara and Cisco to meet the needs of enterprise customers. The solution provides Orchestrate efficiency across the data path with an intelligent system that helps anticipate and navigate challenges as you grow. The architecture builds a self-optimizing data center that automatically spreads workloads across devices to ensure consistent utilization and performance. The solution helps organization to effectively plan infrastructure growth and eliminate the budgeting guesswork with predictive risk profiles that identify historical trends.

Organizations experience a 5-year ROI of 528% with Cisco UCS Integrated Infrastructure solutions, Businesses experience 48% lower IT infrastructure costs with Cisco UCS Integrated Infrastructure solutions. Organizations can realize a 5-year total business benefit of $20.4M per organization with Cisco UCS Integrated Infrastructure solutions. The break-even period with Cisco UCS Integrated Infrastructure solutions is nine months. Businesses experience 67% lower ongoing administrative and management costs with Cisco UCS Manager (UCSM). For more information please refer to IDC #US41084916 2016

This architecture is composed of the Hitachi Virtual Storage Platform (VSP) connecting through the Cisco MDS multilayer switches to Cisco Unified Computing System (Cisco UCS), and further enabled with the Cisco Nexus family of switches.

Audience

The audience for this document includes, but is not limited to; sales engineers, field consultants, professional services, IT managers, partner engineers, and customers who want to modernize their infrastructure to meet SLAs and the business needs at any scale.

Purpose of this Document

This document provides a step by step configuration and implementation guide for the Cisco and Hitachi Adaptive Solutions for Converged Infrastructure solution. This solution features a validated reference architecture composed of:

·       Cisco UCS Compute

·       Cisco Nexus Switches

·       Cisco Multilayer SAN Switches

·       Hitachi Virtual Storage Platform

·       SUSE Enterprise Linux and Red Hat Enterprise Linux Operating System

·       SAP HANA

For the design decisions and technology discussion of the solution, please refer to the Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration Design Guide.

Architecture

Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration provides an end-to-end architecture with Cisco Compute, Networking and Hitachi Storage that demonstrate support for multiple SAP HANA workloads with high availability and secure multi-tenancy. The architecture is built around the Cisco Unified Computing System(UCS) and the Hitachi Virtual Storage Platform(VSP) connected together by Cisco MDS Multilayer SAN Switches, and further enabled with Cisco Nexus Switches. These components come together to form a powerful and scalable design, built on the best practices of Cisco and Hitachi to create an ideal platform for running a variety of enterprise workloads with confidence. Figure 1   illustrates the physical topology of the Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration.

Figure 1     Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration Architecture

The components of this integrated architecture shown in Figure 1   are:

·       Cisco Nexus 9336C-FX2 – 100Gb capable, LAN connectivity to the Cisco UCS compute resources.

·       Cisco UCS 6332-16UP Fabric Interconnect – Unified management of Cisco UCS compute, and the compute’s access to storage and networks.

·       Cisco UCS B200 M5 – High powered, versatile blade server with two CPU for SAP HANA

·       Cisco UCS B480 M5 – High powered, versatile blade server with four CPU for SAP HANA

·       Cisco MDS 9706 – 16Gbps Fiber Channel connectivity within the architecture, as well as interfacing to resources present in an existing data center.

·       Hitachi VSP G370 – Mid-range, high performance storage subsystem with optional all-flash configuration

·       Cisco UCS Manager – Management delivered through the Fabric Interconnect, providing stateless compute, and policy driven implementation of the servers managed by it.

Hardware and Software Versions

Table 1  lists the validated hardware and software versions used for this solution. Configuration specifics are given in this deployment guide for the devices and versions listed in the following tables. Component and software version substitution from what is listed is considered acceptable within this reference architecture, but substitution will need to comply with the hardware and software compatibility matrices from both Cisco and Hitachi, please refer to the following documentation: 

Cisco UCS Hardware Compatibility Matrix:

https://ucshcltool.cloudapps.cisco.com/public/

Cisco Nexus and MDS Interoperability Matrix:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx/Matrix1.html

Cisco Nexus Recommended Releases for Nexus 9K:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/recommended_release/b_Minimum_and_Recommended_Cisco_NX-OS_Releases_for_Cisco_Nexus_9000_Series_Switches.html

Cisco MDS Recommended Releases:

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/b_MDS_NX-OS_Recommended_Releases.html

Hitachi Vantara Interoperability:

https://support.hitachivantara.com/en_us/interoperability.html 

In addition, any substituted hardware or software may have different configurations from what is detailed in this guide and will require a thorough evaluation of the substituted product reference documents.

Table 1    Validated Hardware and Software

Configuration Guidelines

This information in this section is intended to enable you to fully configure the customer environment. In this process, various steps require you to insert customer-specific naming conventions, IP addresses, and VLAN schemes, as well as to record appropriate MAC addresses. Table 2  lists the configuration variables that are used throughout this document. This table can be completed based on the specific site variables and used in implementing the document configuration steps.

The Cisco UCS Fabric Interconnects are similarly configured. Additionally, this document details the steps for provisioning multiple Cisco UCS hosts, and these are identified sequentially: HANA-Server01, HANA-Server02, and so on. Finally, to indicate that you should include information pertinent to your environment in a given step, <text> appears as part of the command structure. Review the following example for the network port vlan create command:

Usage:

Example:

Table 2    Configuration Variables

Physical Cabling

This section explains the cabling examples used in the validated environment. To make connectivity clear in this example, the tables include both the local and remote port locations.

This document assumes that out-of-band management ports are plugged into an existing management infrastructure at the deployment site. The upstream network from the Nexus 9336C-FX2 switches is out of scope of this document, with only the assumption that these switches will connect to the upstream switch or switches with a virtual Port Channel (vPC).

Figure 2  shows the cabling configuration used in this validated design.

Figure 2     Cabling Diagram for Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration

 

Table 3  through Table 8  provide the details of the specific port connections with the cables used in this deployment guide.

Table 3    Cisco Nexus 9336C-FX2 A Cabling Information

Table 4    Cisco Nexus 9336C-FX2 A Cabling Information

Table 5    Cisco UCS 6332-16UP A Cabling Information

Table 6    Cisco UCS 6332-16UP B Cabling Information

Table 7    Cisco MDS 9706 A Cabling Information

Table 8    Cisco MDS 9706 B Cabling Information

The following section provides a detailed procedure for configuring the Cisco Nexus 9000 Switches for SAP HANA environment. The Nexus switch configuration will explain the basic L2 and L3 functionality for the application environment used in the validation environment hosted by the UCS domains. The application gateways are hosted by the pair of Nexus switches, but primary routing is passed onto an existing router that is upstream of the converged infrastructure. This upstream router will need to be aware of any networks created on the Nexus switches, but configuration of an upstream router is beyond the scope of this deployment guide.

The switch configuration in this section based on cabling plan described in the Physical Cabling section. If the systems connected on different ports, configure the switches accordingly following the guidelines described in this section

Cisco Nexus 9000 Initial Configuration

Complete this dialogue on each switch, using a serial connection to the console port of the switch, unless Power on Auto Provisioning is being used.

Enable Appropriate Cisco Nexus 9000 Series Switches Features and Settings

Cisco Nexus 9000 A and Cisco Nexus 9000 B

To enable the IP switching feature and set the default spanning tree behaviors, follow these steps:

1.     On each Nexus 9000, enter configuration mode:

2.     Use the following commands to enable the necessary features:

3.     Configure spanning tree defaults:

4.     Save the running configuration to start-up:

Create VLANs for SAP HANA Traffic

Cisco Nexus 9000 A and Cisco Nexus 9000 B

To create the necessary VLANs, complete the following step on both switches:

1.     From the configuration mode, run the following commands:

Configure Virtual Port-Channel Domain

Cisco Nexus 9000 A

To configure vPCs for switch A, follow these steps:

1.     From the global configuration mode, create a new vPC domain:

2.     Make Nexus 9000A the primary vPC peer by defining a low priority value:

3.     Use the management interfaces on the supervisors of the Nexus 9000s to establish a keepalive link:

4.     Enable following features for this vPC domain:

Cisco Nexus 9000 B

To configure vPCs for switch B, follow these steps:

1.     From the global configuration mode, define the same vPC domain in switch B:

2.     Make Cisco Nexus 9000 B the secondary vPC peer by defining a higher priority value than that of the Nexus 9000 A:

3.     Use the management interfaces on the supervisors of the Cisco Nexus 9000s to establish a keepalive link:

4.     Enable following features for this vPC domain:

1. 

Configure Network Interfaces for the VPC Peer Links

Cisco Nexus 9000 A

1.     Define a port description for the interfaces connecting to VPC Peer <<var_nexus_B_hostname>>.

2.     Apply a port channel to both VPC Peer links and bring up the interfaces.

3.     Define a description for the port-channel connecting to <<var_nexus_B_hostname>>.

4.     Make the port-channel a switchport, and configure a trunk to allow HANA VLANs

5.     Make this port-channel the VPC peer link and bring it up.

Cisco Nexus 9000 B

1.     Define a port description for the interfaces connecting to VPC peer <<var_nexus_A_hostname>>.

2.     Apply a port channel to both VPC peer links and bring up the interfaces.

3.     Define a description for the port-channel connecting to <<var_nexus_A_hostname>>.

4.     Make the port-channel a switchport and configure a trunk to allow HANA VLANs.

5.     Make this port-channel the VPC peer link and bring it up.

Configure vPCs with Cisco UCS Fabric Interconnect

To configure the vPCs for use by the Client zone, Admin zone, and internal zone traffic, follow these steps:

Run on Cisco Nexus 9000 A and Cisco Nexus 9000 B

1.     Define a port description for the interfaces connecting to <<var_ucs_clustername>>-A.

2.     Apply it to a port channel and bring up the interface.

3.     Define a description for the port-channel connecting to <<var_ucs_clustername>>-A.

4.     Make the port-channel a switchport and configure a trunk to allow all HANA VLANs.

5.     Make the port channel and associated interfaces spanning tree edge ports.

6.     Set the MTU to be 9216 to support jumbo frames.

7.     Make this a VPC port-channel and bring it up.

8.     Define a port description for the interface connecting to <<var_ucs_clustername>>-B.

9.     Apply it to a port channel and bring up the interface.

10.  Define a description for the port-channel connecting to <<var_ucs_clustername>>-B.

11.  Make the port-channel a switchport and configure a trunk to allow all HANA VLANs.

12.  Make the port channel and associated interfaces spanning tree edge ports.

13.  Set the MTU to be 9216 to support jumbo frames.

14.  Make this a VPC port-channel and bring it up.

(Optional) Configure SAP HANA Backup Networks to Use Separate vPCs

Configure additional vPCs to be used exclusively by the Backup Network. The following example configures two ports Ethernet 1/31 and Et/hernet1/32 connected to Eth1/31 and Eth1/32 on the UCS Fabric Interconnects.

Run on Cisco Nexus 9000 A and Cisco Nexus 9000 B

1.     Define a port description for the interface connecting to <<var_node01>>.

2.     Apply it to a port channel and bring up the interface.

3.     Define a description for the port-channel connecting to <<var_backup_node01>>.

4.     Make the port-channel a switchport and configure a trunk to allow NFS VLAN for DATA.

5.     Make the port channel and associated interfaces spanning tree edge ports.

6.     Set the MTU to be 9216 to support jumbo frames.

7.     Make this a VPC port-channel and bring it up.

8.     Define a port description for the interface connecting to <<var_node02>>.

9.     Apply it to a port channel and bring up the interface.

10.  Define a description for the port-channel connecting to <<var_node02>>.

11.  Make the port-channel a switchport, and configure a trunk to allow NFS VLAN for DATA

12.  Make the port channel and associated interfaces spanning tree edge ports.

13.  Set the MTU to be 9216 to support jumbo frames.

14.  Make this a VPC port-channel and bring it up.

Set Global NTP Configurations

Run the following commands on both switches to set global configurations:

 

 

A Hitachi Virtual Storage Platform F/G series specialist must install Hitachi Virtual Storage Platform G370. The initial configuration for VSP G370 is done in the Hitachi Distribution Centers.

If IP addresses of the SVP are not known at build time in the distribution center, they will be set to a default value and need change onsite by the Hitachi storage specialist.

Storage Architecture Overview

Each SAP HANA node needs the following storage layout:

·       Operating system (OS) volume

·       SAP HANA shared volume

·       SAP HANA log volume

·       SAP HANA data volume

This SAP HANA TDI setup utilizes the following two dynamic provisioning pools created with Hitachi Dynamic Provisioning for the storage layout. This ensures maximum utilization and optimization at a lower cost than other solutions.

·       OS_SH_DT_Pool for the following:

-      OS volume

-      SAP HANA shared volume

-      SAP HANA data volume

·       LOG_Pool for the following:

-      SAP HANA log volume

The validated dynamic provisioning pool layout options with minimal disks and storage cache on Hitachi Virtual Storage Platform F350, VSP G350, F370, VSP G370, VSP F700, VSP G700, VSP F900 and VSP G900 storage are listed in Table 9  .

Table 9    Dynamic Provisioning Pools with Disks and Storage Cache

Additional parity groups of the same type may need to be added. Drive boxes may be needed if the internal drives on storage are not sufficient, depending on the following:

·       The various combinations of node sizes

·       The number of nodes to meet the capacity requirements

While it is not limited to these systems, this SAP HANA tailored data center integration solution uses the following four active SAP HANA systems, as examples:

·       System 1 — 384 GB

·       System 2 — 768 GB

·       System 3 — 1536 GB

·       System 4 — 3072 GB

Provision the storage for the four SAP HANA systems listed above:

·       Determine the minimum sizes for operating system, data, log, and HANA shared using these formulas in SAP white pager SAP HANA Storage Requirements as following:

-      Every HANA node requires approximately 100 GB capacity for the operating system.

-      /hana/shared size uses formulas:

§  Single node (scale-up) — Size = MIN (1 × RAM; 1 TB)

§  Multi-node (scale-out) — Size = 1 × RAM of every 4 worker nodes

-      Data size requires at least 1 × RAM of each HANA node

-      Log size uses formulas:

§  Systems with equal or less than 512 GB memory — Size = 1/2 × RAM

§  Systems with greater than 512 GB memory — Size = 512 GB

·       Provision the storage:

-      Create two dynamic provisioning pools for the three SAP HANA systems on storage:

§  Use OS_SH_DT_Pool to provision the operating system volume, SAP HANA shared volume, and Data volume.

§  Use LOG_Pool to provision the Log volume.

-      For SSDs, create the parity groups first, as the example shown in Table 10  for Hitachi Virtual Storage Platform G370, using the RAID-10 storage design

Table 10      Dynamic Provisioning Pool with RAID10(2D+2D) for 16 Nodes on VSP F370 and G370 with SSDs

·       Assign all LDEVs to the dedicated pool for VSP G370.

·       Create virtual volumes (VVOLs) for the operating system, SAP HANA shared, log, and data volumes. Table 11  shows examples for HANA systems with memory of 384 GB, 768 GB, 1536 GB, and 3072 GB.

Table 11      VVOLs for SAP HANA Nodes for Four Memory Sizes of HANA Systems

While mapping the LUN path assignment for each node, add VVOLs in the following order:

1.     The operating system volume

2.     The SAP HANA shared volume

3.     The log volume

4.     The data volume

Table 12  lists an example configuration of the LUN path assignment for Node 1. Configure the LUN assignment similarly for all other nodes.

Table 12      Example LUN Path Assignment for the SAP HANA Configuration on Node 1

Log into Storage Navigator

After installing the VSP G370 onsite and running all necessary cable connections and powering up the VSP G370, open Hitachi Storage Navigator to start the configuration:

1.     Access Hitachi Storage Navigator through a web browser.

2.     https://<IP of Storage System SVP>/dev/storage/886000<Serial Number of Storage System>/emergency.do – for example, if Storage System SVP IP address is 192.168.50.21 and Serial Number of Storage System is 456789, the URL would be:
https://192.168.50.21/dev/storage/836000456789/emergency.do

3.     Log into Hitachi Storage Navigator.

Check SFP Data Transfer Rate

When you first log in prior to starting the configuration of the storage, navigate to Port Condition to check the SFP Data Transfer Rate.

To check the SFP data transfer rate, follow these steps:

1.     In the Storage Navigator window click Actions, Components and then View Port Condition.

The Port Condition window opens.

2.     Make sure the transfer rate in the SFP Data Transfer Rate matches the speed of the SFPs in the storage controller. The actual Speed can differ, depending on the configuration of the other components.

3.     Click Close to close the Port Condition window and start with the storage configuration.

Create Pool Volumes

This procedure creates the Parity Groups and LDEVs using Hitachi Storage Navigator for the following:

·       Operating System LUNs

·       SAP HANA Shared LUNs

·       SAP HANA Log LUNs

·       SAP HANA Data LUNs

Use the storage navigator session from the previous section. Repeat these steps to create all the required pool volumes.

To create a pool volume, follow these steps:

1.     Open the LDEV creation window.

2.     In the General Tasks pane, click Create LDEVs. The 1 Create LDEVs dialog box opens.

3.     Create Pool Volume LUN:

a.     Create an LDEV.

b.     Enter the values shown in Table 13  into the Create LDEVs dialog box.

Table 13       Pool Volume Creation for LOG_Pool and OS_SH_DT_Pool

4.     Click Add and then click Finish.

5.     Acknowledge the Warning by clicking OK.

The Confirm window opens.

6.     Confirm the selection again, and then click Apply.

7.     Record the task name for later reference.

8.     Repeat steps 1-7 to create every pool volume required by this installation.

9.     Keep the Storage Navigator session open to Create Dynamic Provisioning Pools.

Create Dynamic Provisioning Pools

Use the Storage Navigator session from previous procedure to perform this procedure to create dynamic provisioning pools. This solution uses two dynamic provisioning pools:

·       LOG_Pool

·       OS_SH_DT_Pool

Follow the steps in this section to create the LOG_Pool and repeat these steps to create the OS_SH_DT_Pool.

To create a dynamic provisioning pool, follow these steps:

1.     From Pools, click Create Pools to open the 1. Create Pools window.

2.     Enter the values shown in Table 6  in the Create Pools dialog box.

Table 14      Dynamic Provisioning Pool Creation: LOG_Pool and OS_SH_DT_Pool

3.     Select the pool volumes for the pool.

4.     Click Select Pool VOLs.

5.     Select the volumes.

-      For LOG_Pool, identify the pool volumes for the pool and select them. Click Add.

-      For OS_SH_DT_Pool, identify the pool volumes for the pool and select them. Click Add.

6.     Click OK.

7.     Click Add.

8.     Click Finish on the 2. Confirm window.

9.     Click Apply.

Provision the LUNS (Virtual Volumes)

This procedure creates the LDEVs using Hitachi Storage Navigator for the following:

·       Operating system LUNS

·       SAP HANA shared LUNS

·       Log LUNs

·       Data LUNs

Assign each of the LUNs to specific MPU for optimal performance, map to LUN paths using specific LUN ID in sequence as listed Table 12   

Create Virtual Volumes for the Operating System LUNS and Map Ports

Use Hitachi Storage Navigator to create the operating system LDEV and map it to specified Hitachi Virtual Storage Platform Fx00 or Gx00 ports.

To create LDEVs for the operating system boot LUN, follow these steps:

1.     From Pools, click OS_SH_DT_Pool.

2.     In the Virtual Volumes pane, click Create LDEVs. The 1 Create LDEVs dialog box opens.

3.     Create operating system boot LUNS.

4.     Create one operating system LUN per HANA node and assign it to the ports following Table 11  . Repeat this step until all operating LUNS are completed.

5.     Create an LDEV.

6.     Enter the values shown in Table 15  in the Create LDEVs dialog box.

Table 15      LDEV Creation Values for Operating System LUN

7.     Click Add and then click Next.

The 2 Select LDEV window displays all configured LDEVs in the right pane.

8.     Select the host ports.

9.     Click Next on the 2 Select LDEVs window. The 3 Select Host Groups/iSCSI Targets window opens.

10.  From the Available Host Groups pane, select the OS LUN ports by referring to Table 11 

11.  Click Add.

12.  The selected ports that were in the Available Hosts Groups pane are now in the Selected Host Groups pane.

13.  Click Next.

14.  The 4 View/Change LUN Paths window displays.

15.  Confirm the selected ports.

16.  Confirm the selected ports and adjust the LUN ID as listed in Table 4 

17.  Click Finish.

The 5 Confirm window opens.

18.  Confirm the selection again and then click Apply.

19.  Record the task name for later reference

20.  Keep the Storage Navigator session open for Create Virtual Volumes for HANA Shared File System and Map Ports.

Create Virtual Volumes for HANA Shared File System and Map Ports

Use Hitachi Storage Navigator to create the HANA shared virtual volumes under dynamic provisioning pool OS_SH_DT_Pool and then map them to specified storage ports.

Repeat this procedure until you create all of the virtual volumes.

To create a virtual volume for the HANA-shared file system and map ports, follow these steps:

1.     From Pools, click OS_SH_DT_Pool.

2.     Enter the values shown in Table 16  in the Create LDEVs dialog box.

Table 16      Virtual Volume Creation for HANA Shared LUNs

3.     Keep the Storage Navigator session open for Create Virtual Volumes for Log LUNs and Map Ports.

Create Virtual Volumes for Log LUNs and Map Ports

This procedure creates and maps LDEVs to the specified storage ports for the log LUNs.

Use the Hitachi Storage Navigator session previously started.

To provision the LDEVs for log LUNs, follow the steps from the previous section with the following changes:

1.     From Pools, click LOG_Pool.

2.     Enter the values shown in Table 17  in the Create LDEVs dialog box.

Table 17      LDEV Creation Values for Log LUN

3.     Keep the Storage Navigator session open for Create Virtual Volumes for Data LUNs and Map Ports.

Create Virtual Volumes for Data LUNs and Map Ports

This procedure creates and maps LDEVs to the specified Hitachi Virtual Storage Platform F370/G370 ports for the Data LUNs.

Use the previously-opened Hitachi Storage Navigator session.

To provision the LDEVs for Data LUNs, follow the steps of the previous sections.

To create virtual volumes for data LUNs and map ports, follow these steps:

1.     From Pools, click OS_SH_DT_Pool.

2.     Enter the values shown in Table 10   in the Create LDEVs dialog box.

Table 18      LDEV Creation Values for Data LUN

3.     Keep the Storage Navigator session open for the Configure the Host Groups procedure.

Storage Port Configuration

The following table lists the configuration and port mapping for Hitachi VSP Fx00 and Gx00 models.

Table 19      Storage Port Mapping for Validated SAP HANA Nodes using SSDs

Configure the Host Groups

To configure the host ports, follow these steps:

1.     Open the Ports/Host Group/iSCSI Targets window.

2.     In Storage Systems under the Explorer pane, expand the VSP Gx00 tree.

3.     Click Ports/Host Groups/iSCSI Targets.

4.     In the right pane of the Ports/Host Groups/iSCSI Targets window, click the Ports tab to see the list of ports.

5.     Select all required ports and click Edit Ports.

6.     Enter the properties in the Edit Ports window, see Table 12 

Table 20      Edit Ports Settings

 

This section describes the specific configurations on Cisco UCS servers to address the SAP HANA requirements.

It is beyond the scope of this document to cover detailed information about the Cisco UCS infrastructure. Detailed configuration guides are at: https://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-manager/products-installation-and-configuration-guides-list.html

Physical Connectivity

Physical cabling should be completed by following the diagram and table references in section Deployment Hardware and Software.

Upgrade Cisco UCS Manager Software to Version 4.0(1c)

This document based on Cisco UCS 4.0(1c). To upgrade the Cisco UCS Manager software and the Cisco UCS Fabric Interconnect software to version 4.0(1c), go to Cisco UCS Manager Install and Upgrade Guides.

Initial Setup of Cisco UCS 6332-16UP Fabric Interconnects

The initial configuration dialogue for the Cisco UCS 6332-16UP Fabric Interconnects will be provide the primary information to the first fabric interconnect, with the second taking on most settings after joining the cluster. 

To start on the configuration of the Fabric Interconnect A, connect to the console of the fabric interconnect and step through the Basic System Configuration Dialogue:

Wait for the login prompt to make sure that the configuration has been saved.

Cisco UCS 6332-16UP Fabric Interconnect B

Continue the configuration on the console of the Fabric Interconnect B:

Wait for the login prompt to make sure that the configuration has been saved.

Cisco UCS Manager Setup

Log into Cisco UCS Manager

To log into the Cisco Unified Computing System environment, follow these steps:

1.     Open a web browser and navigate to the Cisco UCS 6332 Fabric Interconnect cluster IP address.

Figure 3     Accessing Cisco UCS Manager

2.     Click Launch UCS Manager.

3.     If prompted to accept security certificates, accept as necessary.

4.     When prompted, enter admin as the user name and enter the administrative password.

5.     Click Login to log into the Cisco UCS Manager.

Figure 4     Cisco UCS Manager Page

Anonymous Reporting

During the first connection to the Cisco UCS Manager GUI, a pop-up window will appear to allow for the configuration of Anonymous Reporting to Cisco on use to help with future development.  To create anonymous reporting, complete the following step:

1.     In the Anonymous Reporting window, select whether to send anonymous data to Cisco for improving future products, and provide the appropriate SMTP server gateway information if configuring:

If you want to enable or disable Anonymous Reporting at a later date, it can be found within Cisco UCS Manager under:  Admin -> Communication Management -> Call Home, which has a tab on the far right for Anonymous Reporting.

Synchronize Cisco UCS to NTP

To synchronize the Cisco UCS environment to the NTP server, follow these steps:

1.     In Cisco UCS Manager, click the Admin tab in the navigation pane.

2.     Select Timezone Management drop-down list and click Timezone.

3.     In the Properties pane, select the appropriate time zone in the Timezone menu.

4.     Click Save Changes, and then click OK.

5.     Click Add NTP Server.

6.     Enter <<var_oob_ntp>> and click OK.

7.     Click OK.

Configure Cisco UCS Servers

Chassis Discovery Policy

Setting the discovery policy simplifies the addition of B-Series Cisco UCS chassis. To modify the chassis discovery policy, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane and select Equipment in the list on the left.

2.     In the right pane, click the Policies tab.

3.     Under Global Policies, set the Chassis/FEX Discovery Policy to match the number of uplink ports that are cabled between the chassis or fabric extenders (FEXes) and the fabric interconnects. Set the Link Grouping Preference to Port Channel.

4.     Click Save Changes.

5.     Click OK.

Figure 5     Chassis/FEX and Rack Server Discovery Policy

Configure Server Ports

To enable server and uplink ports, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane.

2.     Select Equipment > Fabric Interconnects > Fabric Interconnect A (primary) > Fixed Module.

3.     Click Ethernet Ports.

4.     On the main pane, select the ports that are connected to the chassis and / or to the Cisco C-Series Server (two per FI), right-click them, and select Configure as Server Port.

5.     Click Yes to confirm server ports and click OK.

6.     Verify that the ports connected to the chassis and / or to the Cisco C-Series Server are now configured as server ports.

Figure 6     Cisco UCS – Server Port Configuration Example

7.     Select Equipment > Fabric Interconnects > Fabric Interconnect B (subordinate) > Fixed Module.

8.     Click Ethernet Ports.

9.     On the main pane, select the ports that are connected to the chassis or to the Cisco C-Series Server (two per FI), right-click them, and select Configure as Server Port.

10.  Click Yes to confirm server ports and click OK.

Configure FC SAN Uplink Ports

To configure the FC SAN Uplink ports, follow these steps:

1.     Configure the ports connected to the MDS as FC SAN Uplink Ports. This step creates the first set of ports from the left for example, ports 1-6 of the Fixed Module for FC uplinks and the rest for Ethernet uplinks to N9Ks.

2.     Select Equipment > Fabric Interconnects > Fabric Interconnect A and on the right pane, General > Under Actions > Configure Unified Ports. Choose Yes for the warning pop-up In Cisco UCS Manager, click the Equipment tab in the navigation pane. Move the slider bar to right to enable the first set of 6 ports for FC Uplink Role. Click OK.

Figure 7     Cisco UCS – Configure Fixed Module Ports

3.     Configuring the unified ports require immediate reboot. Click on Yes on the warning pop-up to reboot the Fabric Interconnect.

4.     Select Equipment > Fabric Interconnects > Fabric Interconnect B and on the right pane, General > Under Actions > Configure Unified Ports. Choose Yes for the warning pop-up In Cisco UCS Manager, click the Equipment tab in the navigation pane. Move the slider bar to right to enable the first set of 6 ports for FC Uplink Role. Click OK.

5.     Configuring the unified ports require immediate reboot. Click on Yes on the warning pop-up to reboot the Fabric Interconnect.

6.     After the FIs are accessible after reboot, re-login to Cisco UCS Manager.

Configure Ethernet Uplink Ports

To configure the ethernet uplink ports, follow these steps:

1.     Configure the ports connected to the N9Ks Ethernet Uplink Ports.

2.     In Cisco UCS Manager, click the Equipment tab in the navigation pane.

3.     Select Equipment > Fabric Interconnects > Fabric Interconnect A (primary) > Fixed Module.

4.     Expand Ethernet Ports.

5.     Select ports that are connected to the Cisco Nexus switches, right-click them, and select Configure as Uplink Port.

6.     Click Yes to confirm uplink ports and click OK.

Figure 8     Cisco UCS – Ethernet Uplink Port FI-A Configuration Example

7.     Select Equipment > Fabric Interconnects > Fabric Interconnect B (subordinate) > Fixed Module.

8.     Expand Ethernet Ports.

9.     Select ports that are connected to the Cisco Nexus switches, right-click them, and select Configure as Uplink Port.

10.  Click Yes to confirm the uplink ports and click OK.

Acknowledge Cisco UCS Chassis

To acknowledge all Cisco UCS chassis, follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane.

2.     Expand Chassis and select each chassis that is listed. Right-click each chassis and select Acknowledge Chassis.

3.     After a while, ensure the Discovery completes successfully and there are no major or critical faults reported for any of the servers.

Figure 9     Servers Discovery Status Complete

Power Policy

To run Cisco UCS with two independent power distribution units, the redundancy must be configured as Grid. Follow these steps:

1.     In Cisco UCS Manager, click the Equipment tab in the navigation pane and select Equipment in the list on the left.

2.     In the right pane, click the Policies tab.

3.     Under Global Policies, set the Redundancy field in Power Policy to Grid.

4.     Click Save Changes.

5.     Click OK.

Figure 10         Power Policy

Create New Organization

For secure multi-tenancy within the Cisco UCS domain, a logical entity known as organization is created.

To create an organization unit, follow these steps:

1.     In Cisco UCS Manager, on the Tool bar on right pane top click New.

Figure 11         Cisco UCS – Create Organization

2.     From the drop-down menu select Create Organization.

3.     Enter the Name as T01-HANA

4.     (Optional) Enter the Description as Org for T01-HANA.

5.     Click OK to create the Organization.

Create Pools

Add Block of IP Addresses for KVM Access

To create a block of IP addresses for server Keyboard, Video, Mouse (KVM) access in the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select Pools > root > IP Pools > IP Pool ext-mgmt.

3.     In the Actions pane, select Create Block of IPv4 Addresses.

4.     Enter the starting IP address of the block and the number of IP addresses required, and the subnet and gateway information.

Figure 12         Cisco UCS - Create IP Pool

5.     Click OK to create the IP block.

6.     Click OK in the confirmation message.

Create MAC Address Pools

To configure the necessary MAC address pools for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select Pools > root

3.     In this procedure, two MAC address pools are created, one for each switching fabric.

4.     Right-click MAC Pools under the root

5.     Select Create MAC Pool to create the MAC address pool.

6.     Enter FI-A as the name of the MAC pool.

7.     (Optional) Enter a description for the MAC pool.

8.     Choose Assignment Order Sequential.

9.     Click Next.

10.  Click Add.

11.  Specify a starting MAC address.

12.  The recommendation is to place 0A in the second-last octet of the starting MAC address to identify all of the MAC addresses as Fabric Interconnect A addresses.

13.  Specify a size for the MAC address pool that is sufficient to support the available blade or server resources.

Figure 13         Cisco UCS – Create MAC Pool for Fabric A

14.  Click OK.

15.  Click Finish.

16.  In the confirmation message, click OK.

17.  Right-click MAC Pools under root

18.  Select Create MAC Pool to create the MAC address pool.

19.  Enter FI-B as the name of the MAC pool.

20.  (Optional) Enter a description for the MAC pool. Select ‘Sequential’ for Assignment order.

21.  Click Next.

22.  Click Add.

23.  Specify a starting MAC address.

24.  Specify a size for the MAC address pool that is sufficient to support the available blade or server resources.

25.  Click OK.

26.  Click Finish.

27.  In the confirmation message, click OK.

Figure 14         Cisco UCS – MAC Pools Summary

Create WWNN Pool

To configure the necessary WWNN pool for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Select Pools > root

3.     Right-click WWNN Pools and Select Create WWNN Pool.

4.     Enter HANA-Servers as the name of the WWNN pool.

5.     (Optional) Enter a description for the WWNN pool.

6.     Choose Assignment Order Sequential.

7.     Click Next.

8.     Click Add.

9.     Specify a starting WWNN address.

10.  The recommendation is to place AB in the third-last octet of the starting WWNN address to ensure uniqueness.

11.  Specify a size for the WWNN pool that is sufficient to support the available blade or server resources.

Figure 15         Cisco UCS – Create WWNN Pool

12.  Click OK.

13.  Click Finish.

14.  In the confirmation message, click OK.

Create WWPN Pool

To configure the necessary WWPN pool for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Select Pools > root

3.     In this procedure, two WWPN pools are created, one for each switching fabric.

4.     Right-click WWPN Pools and Select Create WWPN Pool

5.     Enter FI-A as the name of the WWPN pool.

6.     (Optional) Enter a description for the WWPN pool.

7.     Choose Assignment Order Sequential.

8.     Click Next.

9.     Click Add.

10.  Specify a starting WWPN address.

11.  The recommendation is to place 0A in the last bust one octet of the starting MAC address to identify all of the WWPN addresses as Fabric Interconnect A addresses.

12.  Specify a size for the WWPN address pool that is sufficient to support the available blade or server resources.

Figure 16         Cisco UCS – Create WWPN Pool for Fabric A

13.  Click OK.

14.  Click Finish.

15.  In the confirmation message, click OK.

16.  Right-click WWPN Pools and Select Create WWPN Pool.

17.  Enter FI-B as the name of the WWPN pool.

18.  (Optional) Enter a description for the WWPN pool. Select ‘Sequential’ for Assignment order.

19.  Click Next.

20.  Click Add.

21.  Specify a starting WWPN address.

22.  Specify a size for the WWPN address pool that is sufficient to support the available blade or server resources.

23.  Click OK.

24.  Click Finish.

25.  In the confirmation message, click OK.

Figure 17         WWPN Pool Summary

Create UUID Suffix Pool

To configure the necessary universally unique identifier (UUID) suffix pool for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Pools > root

3.     Right-click UUID Suffix Pools.

4.     Select Create UUID Suffix Pool.

5.     Enter HANA-UUID as the name of the UUID suffix pool.

6.     (Optional) Enter a description for the UUID suffix pool.

7.     Keep the Prefix as the Derived option.

8.     Select Sequential for Assignment Order

9.     Click Next.

10.  Click Add to add a block of UUIDs.

11.  Keep the ‘From’ field at the default setting.

12.  Specify a size for the UUID block that is sufficient to support the available blade or server resources.

Figure 18         Cisco UCS – Create UUID Block

13.  Click OK.

14.  Click Finish.

15.  Click OK.

Set Packages and Policies

Create Host Firmware Package

Firmware management policies allow the administrator to select the corresponding packages for a given server configuration. These policies often include packages for adapter, BIOS, board controller, FC adapters, host bus adapter (HBA) option ROM, and storage controller properties.

To create a firmware management policy for a given server configuration in the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Policies > root.

3.     Right-click Host Firmware Packages.

4.     Select Create Host Firmware Package.

5.     Enter HANA-FW as the name of the host firmware package.

6.     Leave Simple selected.

7.     Select the version 4.0(1c)B for the Blade Package and 4.0(1c)C for Rack Packages.

8.     Click OK to create the host firmware package.

9.     Click OK.

Figure 19         Host Firmware Package

Create Server BIOS Policy

To get best performance for HANA it is required to configure the Server BIOS accurately. To create a server BIOS policy for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Policies > root

3.     Right-click BIOS Policies.

4.     Select Create BIOS Policy.

5.     Enter HANA-BIOS as the BIOS policy name.

6.     Select “Reboot on BIOS Settings Change”. Click OK.

7.     Select the BIOS policy selected on the navigation pane.

8.     On the ‘Main’ sub-heading, change the Quiet Boot setting to Disabled.

Figure 20         Create Server BIOS Policy

9.     Click the Advanced Tab.

10.  The recommendation from SAP for SAP HANA is to disable all Processor C States. This will force the CPU to stay on maximum frequency and allow SAP HANA to run with best performance.

11.  On the Advanced tab, under Processor sub-tab, make sure Processor C State is disabled.

12.  Set HPC for CPU Performance, Performance for Power Technology, Performance for Energy Performance.

Figure 21         Processor Settings in BIOS Policy

13.  In the RAS Memory tab, select Performance Mode for LV DDR Mode, enabled for NUMA optimized and maximum-performance for Memory RAS configuration

Figure 22         BIOS Policy – Advanced - RAS Memory

14.  In the Serial Port sub-tab, the Serial Port A enable must be set to Enabled.

15.  On the Server Management tab, select 115.2k for BAUD Rate, Serial Port A for Console redirection, Enabled for Legacy OS redirection, VT100-PLUS for Terminal type. This is used for Serial Console Access over LAN to all SAP HANA servers.

Figure 23         BIOS Policy – Server Management

16.  Click Save Change to modify BIOS Policy.

17.  Click OK.

Power Control Policy

The Power Capping feature in Cisco UCS is designed to save power with a legacy data center use case. This feature does not contribute much to the high-performance behavior of SAP HANA. By choosing the option “No Cap” for power control policy, the SAP HANA server nodes will not have a restricted power supply. It is recommended to have this power control policy to make sure sufficient power supply for high performance and critical applications like SAP HANA.

To create a power control policy for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Policies > root.

3.     Right-click Power Control Policies.

4.     Select Create Power Control Policy.

5.     Enter HANA as the Power Control Policy name. (Optional) provide description.

6.     Set Fan Speed Policy to Performance.

7.     Change the Power Capping setting to No Cap.

Figure 24         Power Control Policy for SAP HANA Nodes

8.     Click OK to create the power control policy.

9.     Click OK

Create Serial over LAN Policy

The Serial over LAN policy is required to get console access to all the SAP HANA servers through SSH from the management network. This is used in case of the server hang or a Linux kernel crash, where the dump is required. To configure Create Serial over LAN Policy, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Policies > root

3.     Right-click the Serial over LAN Policies.

4.     Select Create Serial over LAN Policy.

5.     Enter SoL-Console as the Policy name.

6.     Select Serial over LAN State to Enable.

7.     Change the Speed to 115200.

8.     Click OK.

Figure 25         Serial Over LAN Policy

Update Default Maintenance Policy

It is recommended to update the default Maintenance Policy with the Reboot Policy “User Ack” for the SAP HANA server. This policy will wait for the administrator to acknowledge the server reboot for the configuration changes to take effect.

To update the default Maintenance Policy, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Policies > root.

3.     Select Maintenance Policies > default.

4.     Change the Reboot Policy to User Ack.

5.     Click Save Changes.

6.     Click OK to accept the change.

Figure 26         Maintenance Policy

Network Control Policy

Update Default Network Control Policy to Enable CDP

CDP needs to be enabled to learn the MAC address of the End Point. To update default Network Control Policy, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select LAN > Policies > root > Network Control Policies > default.

3.     In the right pane, click the General tab.

4.     For CDP: select Enabled radio button.

5.     Click Save Changes in the bottom of the window.

6.     Click OK.

Figure 27         Network Control Policy to Enable CDP

Configure Cisco UCS LAN Connectivity

Set Jumbo Frames in Cisco UCS Fabric

The core network requirements for SAP HANA are covered by Cisco UCS defaults. Cisco UCS is based on 40GbE and provides redundancy through the Dual Fabric concept. The Service Profile is configured to distribute the traffic across Fabric Interconnect A and B.

To configure jumbo frames and enable quality of service in the Cisco UCS fabric, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select LAN > LAN Cloud > QoS System Class.

3.     In the right pane, click the General tab.

4.     On the MTU Column, enter 9216 in the box.

5.     Click Save Changes in the bottom of the window.

6.     Click Yes to accept the QoS Change Warning

7.     Click OK.

Figure 28         Cisco UCS – Setting Jumbo Frames

Create LAN Uplink Port Channels

Configure the LAN uplinks from FI-A and FI-B towards northbound Nexus Switches, in port-channel, for use by all of the network zones as prescribed by SAP. For example, we create port-channel 21 on FI-A and port-channel 22 on FI- B. This port channel pair will have corresponding vPCs defined on N9Ks that ensures seamless redundancy and failover for the north-south network traffic

It would suffice to have a port-channel pair on FI with corresponding vPC pair on N9Ks to handle traffic of all network zones provided we have enough ports to account for the desired bandwidth. In the current example, we have used two pairs of 2 x 40GE ports for the FI<->N9K connectivity for port-channels. You could add more based on the need or use-case.

We create port channel pair 21 and 22 with two 40GE ports from FIs to the Nexus switches to cater to SAP HANA’s Client, Admin and Internal zones.

We create another port channel pair 31 and 32 with two 40GE ports from FIs to the Nexus switches that could exclusively handle bandwidth intensive SAP HANA Storage zone traffic comprising of HANA node backup network.

To configure the necessary port channels out of the Cisco UCS environment, follow these steps:

1.     In this procedure, two port channels are created: one each from FI-A to and FI-B to uplink Cisco Nexus switches.

2.     In Cisco UCS Manager, click the LAN tab in the navigation pane

3.     Under LAN > LAN Cloud, expand the Fabric A tree.

4.     Right-click Port Channels.

5.     Select Create Port Channel.

Figure 29         Cisco UCS – Creating Ethernet Port Channel

6.     Enter 21 as the unique ID of the port channel.

7.     Enter Uplink-to-N9K as the name of the port channel.

8.     Click Next.

9.     Select the following ports to be added to the port channel:

-      Slot ID 1 and port 33

-      Slot ID 1 and port 34

Figure 30         Cisco UCS Port Channel – Add ports

10.  Click >> to add the ports to the port channel.

11.  Click Finish to create the port channel.

12.  Click OK.

13.  In the navigation pane, under LAN > LAN Cloud, expand the Fabric B tree.

14.  Right-click Port Channels.

15.  Select Create Port Channel.

16.  Enter 22 as the unique ID of the port channel.

17.  Enter Uplink-to-N9K as the name of the port channel.

18.  Click Next.

19.  Select the following ports to be added to the port channel:

-      Slot ID 1 and port 33

-      Slot ID 1 and port 34

20.  Click >> to add the ports to the port channel.

21.  Click Finish to create the port channel.

22.  Click OK.

23.  In Cisco UCS Manager, click the LAN tab in the navigation pane

24.  Under LAN > LAN Cloud, expand the Fabric A tree.

25.  Right-click Port Channels.

26.  Select Create Port Channel.

Figure 31         Cisco UCS – Creating Ethernet Port Channel

27.  Enter 31 as the unique ID of the port channel.

28.  Enter Uplink-Backup as the name of the port channel.

29.  Click Next.

30.  Select the following ports to be added to the port channel:

-      Slot ID 1 and port 31

-      Slot ID 1 and port 32

Figure 32         Cisco UCS Port Channel – Add Ports

31.  Click >> to add the ports to the port channel.

32.  Click Finish to create the port channel.

33.  Click OK.

34.  In the navigation pane, under LAN > LAN Cloud, expand the Fabric B tree.

35.  Right-click Port Channels.

36.  Select Create Port Channel

37.  Enter 31 as the unique ID of the port channel.

38.  Enter Uplink-Backup as the name of the port channel.

39.  Click Next.

40.  Select the following ports to be added to the port channel:

-      Slot ID 1 and port 31

-      Slot ID 1 and port 32

41.  Click >> to add the ports to the port channel.

42.  Click Finish to create the port channel.

43.  Click OK.

Figure 33         Cisco UCS FI-A Port Channel Overview

Figure 34         Cisco UCS FI-B Port Channel Overview

VLAN Configurations

Within Cisco UCS, all the network types for an SAP HANA system are manifested by defined VLANs. Even though six VLANs are defined, VLANs for all the networks are not necessary if the solution will not use those networks. For example, if the Replication Network is not used in the solution, then VLAN ID 225 need not be created.

The VLAN IDs can be changed if required to match the VLAN IDs in the customer’s network – for example, ID 221 for backup should match the configured VLAN ID at the customer uplink network switches.

Create VLANs

To configure the necessary VLANs for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select LAN > LAN Cloud.

3.     Right-click VLANs.

4.     Select Create VLANs.

5.     Enter HANA-Mgmt as the name of the VLAN to be used for Management network.

6.     Keep the Common/Global option selected for the scope of the VLAN.

7.     Enter <<var_mgmt_vlan_id>> as the ID of the Management network.

8.     Keep the Sharing Type as None.

9.     Click OK, and then click OK again.

Figure 35         Create VLAN for Internode

10.  Repeat steps 1-9 above for each VLAN creation.

11.  Create VLAN for HANA-Backup

Figure 36         Create VLAN for Backup

12.  Create VLAN for HANA-Client.

Figure 37         Create VLAN for Client Network

13.  Create VLAN for HANA-AppServer.

Figure 38         Create VLAN for Application Server

14.  Create VLAN for HANA-DataSource.

Figure 39         Create VLAN for Data Source

15.  Create VLAN for HANA-Replication.

Figure 40         Create VLAN for Replication

The list of created VLANs is shown below:

Figure 41         VLAN Definition in Cisco UCS

Create VLAN Groups

For easier management and bandwidth allocation to a dedicated uplink on the Fabric Interconnect, VLAN Groups are created within the Cisco UCS. SAP groups the networks needed by HANA system into following zones which could be translated to VLAN groups in Cisco UCS configuration:

·       Client Zone – including AppServer, Client and DataSource networks

·       Internal Zone – including Inter-node and System Replication networks

·       Storage Zone – including Backup and IP storage networks 

·       And optional Admin zone – including Management, , OS cluster network, if any

To configure the necessary VLAN Groups for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select LAN > LAN Cloud.

3.     Right-click VLAN Groups.

4.     Select Create VLAN Groups.

5.     Enter Admin-Zone as the name of the VLAN Group used for Infrastructure network.

6.     Select HANA-Mgmt.

Figure 42         Create VLAN Group for Admin Zone

7.     Click Next

8.     Click Next on Add Uplink Ports, since you will use port-channel.

9.     Choose port-channels created [21 & 22 in this example configuration] for uplink network. Click >>

Figure 43         Add Port-Channel for VLAN Group Admin Zone

10.  Click Finish.

11.  Create VLAN Group for Client Zone. Select HANA-AppServer, HANA-Client and HANA-DataSource networks to be part of this VLAN group.

Figure 44         Create VLAN Group for Client Zone

12.  Click Next.

13.  Click Next on Add Uplink Ports, since you will use port-channel.

14.  Choose port-channels (21 and 22 in this example configuration) created for uplink network. Click >>

Figure 45         Add Port-Channel for VLAN Group Internal Zone

15.  Click Finish.

16.  Create VLAN Group for Backup Network. Select HANA-Backup network.

Figure 46         Create VLAN Group for Backup Network

17.  Click Next.

18.  Click Next on Add Uplink Ports, since you will use port-channel.

19.  Choose port-channels (31 and 32 in this example configuration) created for uplink network. Click >>

Figure 47         Add Port-Channel for VLAN Group Internal Zone

20.  Click Finish

21.  Create VLAN Group for Internal-Zone. Select HANA-Replication network

22.  Click Next.

23.  Click Next on Add Uplink Ports, since you will use port-channel.

24.  Choose port-channels (21 and 22 in this example configuration) created for uplink network. Click >>

25.  Click Finish

26.  More VLAN groups, if needed could be created following the above steps. VLAN Groups created in the Cisco UCS.

Figure 48         VLAN Groups in Cisco UCS

Create vNIC Template

Each VLAN is mapped to a vNIC template to specify the characteristic of a specific network. The vNIC template configuration settings include MTU size, Failover capabilities and MAC-Address pools.

To create vNIC templates for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the LAN tab in the navigation pane.

2.     Select Policies > root > Sub-Organization > T01-HANA.

3.     Right-click vNIC Templates.

4.     Select Create vNIC Template.

5.     Enter HANA-Mgmt as the vNIC template name.

6.     For Fabric ID select Fabric A and Check the Enable Failover checkbox.

7.     Under Target, make sure that the VM checkbox is unchecked.

8.     Select Updating Template as the Template Type.

9.     Under VLANs, check the checkboxes for HANA-Mgmt.

10.  Set HANA-Internal as the native VLAN.

11.  For MTU, enter 9000.

12.  In the MAC Pool list, select FI-A.

13.  For Network Control Policy Select default from drop-down list.

Figure 49         Create vNIC Template for HANA-Mgmt

Figure 50         Create vNIC Template for HANA-Mgmt

14.  Click OK to create the vNIC template.

15.  Click OK.

16.  Create vNIC template for each Network.

Create a vNIC Template for Client Network

To create a vNIC template for the client network, follow these steps:

1.     Select Policies > root > Sub-Organization > T01-HANA.

2.     Right-click vNIC Templates and select Create vNIC Template.

3.     Enter HANA-Client as the vNIC template name.

4.     For Fabric ID select Fabric B and Check the Enable Failover checkbox.

5.     Under Target, make sure that the VM checkbox is unchecked.

6.     Select Updating Template as the Template Type.

7.     Under VLANs, check the checkboxes for HANA-Client.

8.     Set HANA-Client as the native VLAN.

9.     For MTU, enter 9000.

10.  In the MAC Pool list, select FI-B

11.  For Network Control Policy Select default from drop-down list.

12.  Click OK to create the vNIC template.

Create a vNIC Template for Application Server Network

To create a vNIC template for the application server network, follow these steps:

1.     Select Policies > root > Sub-Organization > T01-HANA.

2.     Right-click vNIC Templates and select Create vNIC Template.

3.     Enter HANA-AppServer as the vNIC template name.

4.     For Fabric ID select Fabric A and Check the Enable Failover checkbox.

5.     Under Target, make sure that the VM checkbox is unchecked.

6.     Select Updating Template as the Template Type.

7.     Under VLANs, check the checkboxes for HANA-AppServer.

8.     Set HANA-AppServer as the native VLAN.

9.     For MTU, enter 9000.

10.  In the MAC Pool list, select FI-A

11.  For Network Control Policy Select default from drop-down list.

12.  Click OK to create the vNIC template.

Create a vNIC Template for DataSource Network

To create a vNIC template for the DataSource network, follow these steps:

1.     Select Policies > root > Sub-Organization > T01-HANA.

2.     Right-click vNIC Templates and select Create vNIC Template.

3.     Enter HANA-DataSource as the vNIC template name.

4.     For Fabric ID select Fabric A and Check the Enable Failover checkbox.

5.     Under Target, make sure that the VM checkbox is unchecked.

6.     Select Updating Template as the Template Type.

7.     Under VLANs, check the checkboxes for HANA-DataSource.

8.     Set HANA-DataSource as the native VLAN.

9.     For MTU, enter 9000.

10.  In the MAC Pool list, select FI-A

11.  For Network Control Policy Select default from drop-down list.

12.  Click OK to create the vNIC template

Create a vNIC Template for Replication Network

To create a vNIC template for the replication network, follow these steps:

1.     Select Policies > root > Sub-Organization > T01-HANA.

2.     Right-click vNIC Templates and select Create vNIC Template.

3.     Enter HANA-Replication as the vNIC template name.

4.     For Fabric ID select Fabric B and Check the Enable Failover checkbox.

5.     Under Target, make sure that the VM checkbox is unchecked.

6.     Select Updating Template as the Template Type.

7.     Under VLANs, check the checkboxes for HANA-Replication

8.     Set HANA-Replication as the native VLAN.

9.     For MTU, enter 9000.

10.  In the MAC Pool list, select FI-B

11.  For Network Control Policy Select default from drop-down list.

12.  Click OK to create the vNIC template.

Create a vNIC Template for Backup Network

To create a vNIC template for the backup network, follow these steps:

1.     Select Policies > root > Sub-Organization > T01-HANA.

2.     Right-click vNIC Templates and select Create vNIC Template.

3.     Enter HANA-Backup as the vNIC template name.

4.     For Fabric ID select Fabric B and Check the Enable Failover checkbox.

5.     Under Target, make sure that the VM checkbox is unchecked.

6.     Select Updating Template as the Template Type.

7.     Under VLANs, check the checkboxes for HANA-Backup

8.     Set HANA-Backup as the native VLAN.

9.     For MTU, enter 9000.

10.  In the MAC Pool list, select FI-B

11.  For Network Control Policy Select default from drop-down list.

12.  Click OK to create the vNIC template.

The figure below shows the list of vNIC Templates created for SAP HANA.

Figure 51         vNIC Templates Overview

Configure Cisco UCS SAN Configurations

Create FC Port Channels

Create a port channel on FIs A and B for the uplink FC interfaces that connect to respective MDS Fabric Switches, for use by all of the specific VSAN traffic we created earlier in MDS. This port channel pair will have corresponding F-port-channel-trunks defined on MDS switches that would allow for the fabric logins from NPV enabled FIs to be virtualized over the port channel. This provides non-disruptive redundancy should individual member links fail.

To configure the necessary port channels out of the Cisco UCS environment, follow these steps:

1.     In this procedure, two port channels are created: one from fabric A to both Cisco Nexus switches and one from fabric B to both Cisco Nexus switches.

2.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

3.     Under SAN > SAN Cloud, expand the Fabric A tree.

4.     Right-click FC Port Channels.

5.     Select Create FC Port Channel.

Figure 52         Cisco UCS – Creating FC Port Channel

6.     Enter 10 as the unique ID of the port channel.

7.     Enter Uplink-to-MDS-A as the name of the port channel.

8.     Click Next.

9.     Set Port Channel Admin Speed to 16gbps. Select the following ports to be added to the port channel:

-      Slot ID 1 and port 1

-      Slot ID 1 and port 2

-      Slot ID 1 and port 3

-      Slot ID 1 and port 4

Figure 53         Cisco UCS – Port Channel – Add Ports

10.  Click >> to add the ports to the port channel.

11.  Click Finish to create the port channel.

12.  Click OK.

13.  In the navigation pane, under SAN > SAN Cloud, expand the Fabric B tree.

14.  Right-click FC Port Channels.

15.  Select Create FC Port Channel.

16.  Enter 20 as the unique ID of the port channel.

17.  Enter Uplink-to-MDS-B as the name of the port channel.

18.  Click Next.

19.  Set Port Channel Admin Speed to 16gbps. Select the following ports to be added to the port channel:

-      Slot ID 1 and port 1

-      Slot ID 1 and port 2

-      Slot ID 1 and port 3

-      Slot ID 1 and port 4

20.  Click >> to add the ports to the port channel.

21.  Click Finish to create the port channel.

22.  Click OK.

Create VSANs

To configure the necessary VSANs for the Cisco UCS environment, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab in the navigation pane.

2.     Select SAN > SAN Cloud.

3.     Right-click VSANs.

4.     Select Create VSAN.

5.     Enter Fab-A as the name of the VSAN to be used for Fabric A.

6.     Retain ‘Disabled’ for FC Zoning option and select Fabric A.

7.     Enter <<var_fabric-A_vsan_id>> as the ID of the VSAN ID. Use the same value for FCOE VLAN ID.

8.     Click OK and then click OK again.

Figure 54         Create VSAN for Fabric A

9.     Select SAN > SAN Cloud.

10.  Right-click VSANs.

11.  Select Create VSANs.

12.  Enter Fab-B as the name of the VSAN to be used for Fabric-B.

13.  Retain ‘Disabled’ for FC Zoning option and select Fabric B.

14.  Enter <<var_fabric-B_vsan_id>> as the ID of the VSAN ID. Use the same value for FCOE VLAN ID.

15.  Click OK and then click OK again.

Figure 55         VSANs for Fabrics

Assign Respective Fabric FC Channels to Created VSAN

To assign the fc port channels to the fabric VSAN that you just created, follow these steps:

1.     In Cisco UCS Manager, click the SAN tab > SAN Cloud > Fabric A> FC Port Channels>

2.     Select the configured FC Port Channel.

3.     On the right pane, change the VSAN information from default (1) to Fab-A VSAN 10 created for Fabric-A.

Figure 56         VSAN Membership for FI-A FC Uplink Port Channel

4.     Select Save changes. Click OK. After the settings are saved, the Port Channel status changes to Up.

5.     Click the SAN tab > SAN Cloud > Fabric B > FC Port Channels >.

6.     Select the configured FC Port Channel.

7.     On the right pane, change the VSAN information from default (1) to Fab-B VSAN 20 created for Fabric-B.

8.     Select Save changes. Click OK.

Figure 57         VSAN Membership Setting for FI-B FC Uplink Port Channel

Create vHBA Template

1.     In Cisco UCS Manager, click on tab SAN > Policies > root > Sub-Organizations > T01-HANA.

2.     Right-click on vHBA Templates to “Create vHBA Template.”

3.     First create a template for Fabric A. Choose vHBA-A for name.

4.     Optionally provide a description.

5.     Select Fabric ID A

6.     Select VSAN Fab-A

7.     Template Type as Updating template.

8.     Select WWPN Pool FI-A.

9.      Click Ok and Click OK.

Figure 58         Fabric A - vHBA Template

10.  Create a template for Fabric B. Choose vHBA-B for name.

11.  In Cisco UCS Manager, click on tab SAN > Policies > root > Sub-Organizations > HANA.

12.  Right-click on vHBA Templates to “Create vHBA Template.”

13.  Choose vHBA-B for name.

14.  Optionally provide a description.

15.  Select Fabric ID B.

16.  Select VSAN Fab-B

17.  Template Type as Updating template.

18.  Select WWPN Pool as FI-B.

19.   Click Ok and Click OK.

Figure 59         Fabric B - vHBA Template

Create SAN Connectivity Policy

When the physical connectivity is established, the following will configure the zoning for the servers and SAN:

·       Storage connection policies: This configures the storage connectivity taking into account the WWPN Target numbers for the SAN. Since the Zoning is handled by the MDS switches and that FIs aren’t direct attached to the Storage, we do not configure this Storage side connection policy.

·       SAN connectivity policies configuration: This configures vHBAs for the servers which will provide WWPN Initiator numbers for the servers. This server-side configuration is needed to prepare the servers for connection to storage.

To configure the storage connection policy, follow these steps:

1.     Log into UCS Manager.

2.     Click the SAN tab in the Navigation pane.

3.     SAN tab > Policies > root > Sub-Organizations > HANA > SAN Connectivity Policies.

4.     Right-click on SAN Connectivity Policies > Create SAN Connectivity Policy.

5.     Provide name as HANA-SAN.

6.     Optionally add a Description.

7.     Select HANA-Servers for WWNN Assignment

Figure 60         Create SAN Connectivity Policy

8.     Click Add at the bottom for WWPN to add the vHBAs from the vHBA templates previously created.

9.     In the Create vHBA window, provide a name as vhba-a and check “Use vHBA Template” option. Select vHBA-A from the vHBA Template drop-down list and Linux for the Adapter Policy. Click OK.

Figure 61         Create vHBA for Fabric A

10.  Click Add at the bottom for WWPN to add the vHBAs to add another vHBA.

11.  In the Create vHBA window, provide name as vhba-b and check “Use vHBA Template” option. Select vHBA-B from the vHBA Template drop-down list and Linux for the Adapter Policy.

Figure 62         Create vHBA for Fabric B

12.  Click OK.

Figure 63         SAN Connectivity Policy (continued)

13.  Click OK.

Create Boot Policy for SAN Boot

It is strongly recommended to use “Boot from SAN” to realize full benefits of Cisco UCS stateless computing feature such as service profile mobility. The ports on the storage controllers of Hitachi VSP are cross connected with the MDS switches so that we have alternate paths to the LUNs, in addition to the built-in redundancy and path management features of the storage array itself.

You can determine the WWPN information of these storage array target ports from the Hitachi Device Manager.

Configure the SAN primary's primary-target to be port CL1-A and SAN primary's secondary-target to be port CL2-A of the Hitachi VSP Storage. Similarly, the SAN secondary’s primary-target should be port CL3-A and SAN secondary's secondary-target should be port CL4-A

You have to create SAN Boot primary (hba0) and SAN Boot secondary (hba1) in create boot policy by entering WWPN of Hitachi Storage FC Ports.

To create boot policies for the Cisco UCS environments, follow these steps:

1.     Go to tab Servers > Policies > root > Sub-Organizations > T01-HANA > Boot Policies.

2.     Right-click Boot Policies and select Create Boot Policy

3.     Enter HANA-SanBoot as the name of the boot policy

4.     Make sure the “Enforce vNIC/vHBA/iSCSI Name” option is unchecked.

5.     Expand the Local Devices drop-down menu and Choose Add CD-ROM.

6.     Expand the vHBAs drop-down list and Choose Add SAN Boot. In the Add SAN Boot dialog box, select type as ‘Primary’ and enter "hba0" in the vHBA field and Click OK

7.     From the vHBAs drop-down list choose “Add SAN Boot Target.”

8.     Keep 0 as the value for Boot Target LUN. Enter the WWPN for FC port CL1-A of Hitachi VSP Storage and add click OK.

Figure 64         hba0 Primary Boot Target

9.     From the vHBAs drop-down menu choose “Add SAN Boot Target” To add a secondary SAN Boot target into hba0

10.  Enter boot target LUN as 0 and WWPN for FC port CL2-A of Hitachi VSP Storage. Click OK.

11.  From the vHBAs drop-down list and Choose Add SAN Boot. In the Add SAN Boot dialog box, enter "hba1" in the vHBA field. Click OK.

Figure 65         SAN Boot hba1

12.  From the vHBAs drop-down list choose “Add SAN Boot Target.”

13.  Keep 0 as the value for Boot Target LUN. Enter the WWPN for FC port CL2-A of Hitachi VSP Storage and add click OK.

14.  From the vHBAs drop-down list choose “Add SAN Boot Target” to add a secondary SAN Boot target into hba1

15.  Enter boot target LUN as 0 and WWPN for FC port CL4-A of Hitachi VSP Storage. Click OK.

16.  Click OK and click OK for the Create Boot Policy pop-up.

17.  After creating the FC boot policies, you can view the boot order in the Cisco UCS Manager GUI. To view the boot order, navigate to Servers > Policies > root > Sub-Organizations > T01-HANA > Boot Policies> HANA-SanBoot to view the boot order in the right pane of the Cisco UCS Manager as shown below.

Figure 66         SAN Boot Policy

Create Service Profile Templates for SAP HANA Scale Up Servers

The LAN, SAN configurations and relevant SAP HANA policies must be defined prior to creating, a Service Profile Template.

To create the service profile template, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Service Profile Templates > root > Sub-Organization > T01-HANA.

3.     Right-click T01-HANA Select Create Service Profile Template

4.     This will pop-up Create Service Profile Template wizard

5.     Enter HANA-ScaleUp as the name of the service profile template.

6.     Select the Updating Template option from the Type

7.     Under UUID, select HANA-UUID as the UUID pool. Optionally add a Description.

8.     Click Next.

Figure 67         Service Profile Template UUID

9.     In the Storage Provisioning, nothing needs to be configured

10.  Click Next.

11.  In the Networking

12.  Keep the default settings for Dynamic vNIC Connection Policy.

13.  Select the Expert option for ‘How would you like to configure LAN connectivity’ question.

a.     Click Add to add a vNIC to the template.

b.     In the Create vNIC dialog box, enter HANA-AppServer as the name of the vNIC.

c.     Check the Use vNIC Template checkbox.

d.     In the vNIC Template list, select HANA-AppServer.

e.     In the Adapter Policy list, select Linux.

f.      Click OK to add this vNIC to the template.

Figure 68         Service Profile Template vNIC Internal

14.  Repeat step 13 for each vNIC.

15.  Add vNIC for HANA-Backup

Figure 69         Service Profile Template vNIC HANA-Backup

16.  Add vNIC for HANA-Client.

Figure 70         Service Profile Template vNIC Hana-Client

17.  Add vNIC for HANA-DataSource.

Figure 71         Service Profile Template vNIC DataSource

18.  Add vNIC for Mgmt.

Figure 72         Service Profile Template vNIC Mgmt

19.  Add vNIC for HANA-Replication.

Figure 73         Service Profile Template vNIC Replication

20.  Review the table in the Networking pane to make sure that all vNICs were created.

Figure 74         Service Profile Networking

21.  Click Next.

22.  Configure the SAN Connectivity:

23.  Select ‘Use Connectivity Policy’ option for the “How would you like to configure SAN connectivity?” field.

24.  Select HANA-SAN for SAN Connectivity Policy. Click Next.

Figure 75         Service Profile Template - SAN Connectivity (continued)

25.  Zoning – Click Next.

26.  vNIC/vHBA Placement for B480-M5:

a.     In the Select Placement list, choose the Specify Manually.

b.     From the vHBAs tab, assign vhba-a to vCON1.

Figure 76         Service Profile Template – vNIC/vHBA Placement – vHBA Assignment to vCON1

c.     From the vNICs tab, choose vCon1 and assign the vNICs to the virtual network interfaces policy in the following order:

                         i.         HANA-Client

                        ii.         HANA-AppServer

                       iii.         HANA-Replication

Figure 77         Service Profile Template – vNIC/vHBA Placement – vNIC Assignment to vCON1

d.     Select vCON3. From the vHBAs tab, assign vhba-b to vCON3

e.     Choose vCon3 and assign the vNICs to the virtual network interfaces policy in the following order:

                         i.         HANA-Backup

                        ii.         HANA-DataSource

                       iii.         HANA-Mgmt

Figure 78         Service Profile Template – vNIC/vHBA Placement – vNIC Assignment to vCON2

f.      Review the table to verify that all vNICs are assigned to the policy in the appropriate order.

g.     Click Next.

27.  vNIC/vHBA Placement for B200-M5:

a.     In the Select Placement list, choose the Specify Manually.

b.     From the vHBAs tab, assign vhba-a  and vbha-b to vCON1

c.     From the vNICs tab, choose vCon1 and assign the vNICs to the virtual network interfaces policy in the following order:

                         i.         HANA-Client

                        ii.         HANA-AppServer

                       iii.         HANA-Replication

                      iv.         HANA-Backup

                       v.         HANA-DataSource

                      vi.         HANA-Mgmt

d.     Review the table to verify that all vNICs are assigned to the policy in the appropriate order.

f.      Click Next.

28.  No Change required on the vMedia Policy, click Next.

29.  Set the server boot order:

a.     Select HANA-SanBoot for Boot Policy.

Figure 79         Service Profile Template – Server Boot Order

30.  Click Next.

31.  For Maintenance policy:

a.     Select the ‘default’ Maintenance Policy. Click Next.

32.  For Server Assignment: Expand Firmware Management at the bottom of the page and select HANA-FW from the Host Firmware list. Click Next.

Figure 80         Service Profile Template Server Assignment

33.  For Operational Policies:

a.     BIOS Configuration - In the BIOS Policy list, select HANA-BIOS.

34.  External IPMI Management Configuration - Expand the External IPMI Management Configuration. Select SoL-Console in the SoL Configuration Profile.

35.  Management IP Address - In the Outband IPv4 tab choose ext-mgmt in the Management IP Address Policy.

36.  Power Control Policy Configuration - Select HANA from the drop-down list.

37.  Leave the Scrub policy, KVM Management Policy and Graphics Card Policy with default selections.

Figure 81         Service Profile Template Operational Policies

38.  Click Finish to create the service profile template.

39.  Click OK in the confirmation message.

Create Service Profile from the Template

To create service profiles from the service profile template, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Service Profile Templates > root > Sub-Organization > T01-HANA > Service Template HANA-ScaleUp.

3.     Right-click Service Template HANA-ScaleUp and select Create Service Profiles from Template

4.     Enter HANA-ScaleUp-0 as the service profile prefix.

5.     Enter 1 as Name Suffix Starting Number.

6.     Enter 4 as the Number of Instances

7.     Click OK to create the service profile.

Figure 82         Creating Service Profiles from Template

The MDS configuration implements a common redundant physical fabric design with fabrics represented as “A” and “B”.  The validating lab provided a basic MDS fabric supporting VSP Storage Systems that is connected to UCS Fabric Interconnect within the SAN environment.  Larger deployments may require a multi-tier core-edge or edge-core-edge design with port channels connecting the differing layers of the topology.  Further discussion of these kinds of topologies, as well as considerations in implementing more complex SAN environments can be found in this white paper: https://www.cisco.com/c/en/us/products/collateral/storage-networking/mds-9700-series-multilayer-directors/white-paper-c11-729697.pdf

The configuration steps described below are implemented for the Cisco MDS 9706 but are similar to steps required for other Cisco MDS 9000 series switches that may be appropriate for a deployment.  When making changes to the design that comply with the compatibility matrices of Cisco and Hitachi, it is required to consult the appropriate configuration documents of the differing equipment to confirm the correct implementation steps.

Physical Connectivity

Physical cabling should be completed by following the diagram and table references section Deployment Hardware and Software.

Cisco MDS Initial Configuration Dialogue

Complete this dialogue on each switch, using a serial connection to the console port of the switch, unless Power on Auto Provisioning is being used.

Cisco MDS Switch Configuration

Configure Fibre Channel Ports and Port Channels

To configure the fibre channel ports and port channels, follow these steps:

1.     On MDS 9706 A enter the configuration mode and enable the required features as shown below:

2.     Use the following commands to configure the FC Port channel and add all FC ports connected to Cisco UCS Fabric Interconnect A:

3.     On MDS 9706 B enter the configuration mode and enable the required features as shown below:

4.     Use the following commands to configure the FC Port channel and add all FC ports connected to Cisco UCS Fabric Interconnect B:

Configure VSANs

To configure VSANs, follow these steps:

1.     On MDS 9706 A enter the configuration mode and execute the following commands to configure the VSAN:

 

2.     On MDS 9706 B enter the configuration mode and execute the following commands to configure the VSAN:

 

Create and Configure Fiber Channel Zoning

To create the Fiber Channel connections between the Cisco MDS 9706 switches, the Cisco UCS Fabric Interconnects, and the Hitachi Storage, follow these steps:

1.     Log into the Cisco UCS Manager > Servers > Service Profiles > root > Sub-Organizations >T01-HANA > Service Profile HANA–ScaleUp-01. On the right-hand pane, click the Storage tab and vHBA's tab to get the WWPN of HBA's as shown in the figure below.

Figure 83         WWPN of a Server Node

2.     Note the WWPN of the all the configured Servers from their Service Profiles.

In the current example configuration, the WWPN numbers of four server nodes configured are 20:00:00:25:B5:0A:00:00 - 20:00:00:25:B5:0A:00:03 for the Fabric A and 20:00:00:25:B5:0B:00:00 - 20:00:00:25:B5:0B:00:03

3.     Connect to the Hitachi Storage and extract the WWPN of FC Ports connected to the Cisco MDS Switches. We have connected 8 FC ports from Hitachi Storage to Cisco MDS Switches. FC ports CL1-A, CL1-B, CL2-A, CL2-B are connected to MDS Switch-A and similarly FC ports CL3-A, CL3-B, CL3-A, CL3-B are connected to MDS Switch-B.

Figure 84          WWPN of Hitachi Storage  

Create Device Aliases for Fibre Channel Zoning

To configure device aliases and zones for the primary boot paths of MDS switch A, follow this step:

1.     Login as admin user and run the following commands.

To configure device aliases and zones for the primary boot paths of MDS switch B, follow this step:

1.     Login as admin user and run the following commands.

Create Zoning

To configure zones for the MDS switch A, follow these steps:

1.     Create a zone for each service profile.

2.     Login as admin user and run the following commands.

3.     After the zone for the Cisco UCS service profile has been created, create the zone set and add the necessary members.

4.     Activate the zone set by running following commands.

To configure zones for the MDS switch B, follow these steps:

1.     Create a zone for each service profile.

2.     Login as admin user and run the following commands.

3.     After the zone for the Cisco UCS service profile has been created, create the zone set and add the necessary members.

4.     Activate the zone set by running following commands.

This section provides the procedure for Operating System installation using SAN Boot and operating system customizing for SAP HANA requirement.

Associate Service Profile to Cisco UCS Server

To associate service profile created for a specific server, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-01.

3.     Right-click HANA-ScaleUp-01 and select Change Service Profile Association.

4.     For Server Assignment, select the existing Server from the drop-down list.

5.     Click Available Servers.

6.     Select the server, as required. Click OK. Click Yes for the Warning. Click OK.

Figure 85         Creating Service Profiles from Template

7.     Repeat steps 1-6 to associate each Service Profile with a Server.

SLES for SAP 12 SP4 OS Installation

This section provides the procedure for SUSE Linux Enterprise Server for SAP Applications 12 SP 4 Operating System and customizing for SAP HANA requirement.

To install the SLES for SAP 12 SP4, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-01.

3.     Click KVM Console.

4.     When the KVM Console is launched, click Boot Server.

5.     Choose Virtual Media > Activate Virtual Devices.

a.     For Unencrypted Virtual Media Session, select Accept this Session and then click Apply.

6.     Click Virtual Media and choose Map CD/DVD.

7.     Click Browse to navigate to the ISO media location. Select SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.ISO Click Open.

8.     Click Map Device.

9.     At server boot time, during verification of VIC FC boot driver version, it recognizes the Hitachi Storage by its target WWPN numbers. This verifies the server to storage connectivity.

Figure 86         Cisco VIC Boot Driver Recognizes Hitachi Storage

10.  The System will automatically boot from the ISO image. Select the Installation option.

Figure 87         Booting to ISO image

11.  On the first “Language, Keyboard and License Agreement” page, select the Language of choice and Keyboard Layout, “I Agree to license terms” and click Next.

12.  On the Network Settings screen Under Overview, click VNIC Ethernet NIC.

a.     To configure the network interface on the OS, it is required to identify the mapping of the Ethernet device on the OS to vNIC interface on the Cisco UCS.

b.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

c.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-01.

d.     On the main pane click on Network, list of the vNICs with MAC Address are listed.

e.     Note that the MAC Address of the HANA-Mgmt vNIC is “00:25:B5:00:0A:02”

Figure 88         Cisco UCS vNIC MAC Address

f.      By comparing MAC Address on the OS and Cisco UCS, eth0 on OS will carry the VLAN for Management.

13.  Click Edit, under the Address tab.

a.     Click Statically Assigned IP Address:

b.     In the IP Address field enter <<Management IP address>>.

c.     In the Subnet Mask field enter <<subnet mask for Management Interface>>.

d.     In the Hostname field enter the hostname for Management Interface.

Figure 89         Network Settings

14.  Repeat steps 12 and 13 for each vNIC. Alternatively, IP address for vNICs can be set post installation, by using ssh to connect to the server on Management IP.

15.  On the Network Settings screen Select Hostname/DNS:

a.     In the Hostname field enter the Hostname.

b.     In the Domain Name Field enter the Domain Name.

c.     In the Name Server 1 field enter <<DNS server1>> and Name Server 2 field enter <<DNS server2>>

d.     In the Search domains field enter <<domain1.com,domain2.com>>.

Figure 90         Network Settings Hostname

16.  Click Routing.

17.  For the Default IPv4 Gateway enter the <<Default Gateway for>>.

Figure 91         Network Settings Routing

18.  Click Next

19.  System Probing – Select ‘No’ for the pop-up for Do you want to activate multipath?

Figure 92         System Probing – Multipath Activation Choice

20.  Registration – Select Skip Registration. We will do this later as part of post-installation tasks. Click ‘Yes’ for the confirmation warning pop-up to proceed.

21.  Choose Operation System Edition – Select “SUSE Linux Enterprise Server for SAP Applications” option.

Figure 93         Choosing OS Edition

22.  Add On Product: Click Next.

23.  On Suggested Partitioning select Expert Partitioner.

Figure 94         Suggested Partitioning Initial Proposal -Example

24.  On the left ‘System View’ > <<hostname>> > Hard Disks > Select a device from the list which is 100G. In the navigation pane click Delete if found with the suggested partitions which results in an Unpartitioned disk of 100GB.

Figure 95         Expert Partitioner – Choose 100G Hard Disk Device

25.  On the right pane, under Partitions tab, add a new Partition by selecting Add under the Partitions tab for the device. Select Primary Partition for New Partition Type in the next step.

Figure 96         Expert Partitioner – Add Primary Partition on /dev/ device

26.  Select Maximum Size. Click Next.

Figure 97         Add Partition – Specify New Partition Size

27.   Click Next.

28.  Select Operating System Role and click Next.

Figure 98         Add Partition – Specify Role

29.  Select ext3 File system and / or Mount Point. Click Finish.

Figure 99         Add Partition- Formatting and Mounting Options

30.  Click Accept to come back to the Installation Settings page.

Figure 100      Expert Partitioner – Summary

31.  Click Yes to continue setup without swap partition. Click Accept.

32.  Click Next on the final Suggested Partition page.

33.  Clock and Time Zone – choose the appropriate time zone and select Hardware clock set to UTC.

34.  Password for the System Administrator “root” – Key in appropriate password <<var_sys_root-pw>>

35.  On the Installation Settings screen.

Figure 101      Installation Settings – Default Proposal

36.  Customize the software selection. Click Software headline to make the following changes:

a.     Deselect GNOME DE and X Window System.

a.     Make sure C/C++ Compiler and Tools is selected.

b.     Select SAP HANA Server Base.

c.     Deselect SAP Application Sever Base.

Figure 102      Software Selection and System Tasks - Customized

37.  Click OK.

38.  Under the Firewall and SSH headline, click ‘disable’ for Firewall. This will automatically enable SSH service.

Figure 103      Firewall and SSH - Customized

39.  Leave the default selections unchanged.

Figure 104      Installation Settings – Final Selections

40.  Click Install and select Install again for the subsequent ‘Confirm Installation’ prompt. The installation is started, and you can monitor the status.

Figure 105      Perform Installation

41.  After the Operating System is installed the system will reboot.

Figure 106      Booting from Hard Disk

Network Services Configuration

To configure the server with Network services, follow these steps:

Hostnames

The operating system must be configured such a way that the short name of the server is displayed for the command ‘hostname’ and Full Qualified Host Name is displayed with the command ‘hostname –d’.

1.     ssh to the Server using Management IP address assigned to the server during installation.

2.     Login as root and password.

3.     Set the hostname using hostnamectl

IP Address

Each SAP HANA Server is configured with 6 vNIC device. Table 21  lists the IP Address information required to configure the IP address on the Operating System.

Table 21      List the IP Address for SAP HANA Server

1.     To configure the network interface on the OS, it is required to identify the mapping of the ethernet device on the OS to vNIC interface on the Cisco UCS.

2.     From the OS execute the below command to get list of Ethernet device with MAC Address.

3.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

4.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-01.

5.     On the main pane, click Network; the list of the vNICs with MAC Address are listed.

Figure 107      Cisco UCS vNIC MAC Address

6.     Note the MAC Address of the HANA-Client vNIC is “00:25:B5:00:0B:00”.

7.     By comparing MAC Address on the OS and Cisco UCS, eth5 on OS will carry the VLAN for HANA-Client.

8.     Go to network configuration directory and create a configuration for eth5

9.     Repeat the steps 9 to 11 for each vNIC interface.

10.  Add default gateway.

DNS

Domain Name Service configuration must be done based on the local requirements.

1.     Add DNS Servers entry:

Hosts file

HANA nodes should be able to resolve internal network IP address, below is an example of Scale Up HANA System host file with the entire network defined in the /etc/hosts file.

Network Time

It is important that the time on all components used for SAP HANA is in sync. The configuration of NTP is important and to be performed on all systems.

1.     Configure NTP by adding at least one NTP server to the NTP config file /etc/ntp.conf.

SLES for SAP 12 SP 4 System Update and OS Customization

To updated and customize the SLES 12 SP 4 System for HANA Servers, follow these steps:

1.     Register SUSE Linux Enterprise installations with the SUSE Customer Center:

2.     Execute the below command to update the SLES4SAP 12 SP 4 to latest patch level.

3.     Follow the on-screen instruction to complete the update process.

4.     Disable transparent hugepages, Configure C-States for lower latency in Linux, Auto NUMA settings. Modify /etc/default/grub search for the line starting with "GRUB_CMDLINE_LINUX_DEFAULT" and append to this line:

5.     Save your changes and run:

6.     Add the following line into /etc/init.d/boot.local, for CPU Frequency, Energy Performance Bias, Kernel samepage merging settings:

7.     Activate tuned:

8.     Enable tuned profile:

9.     Reboot the OS by issuing reboot command.

Install Cisco eNIC and fNIC Driver

To download the Cisco UCS Drivers ISO bundle, which contains most of the Cisco UCS Virtual Interface Card drivers, follow these steps:

1.     In a web browser, navigate to https://software.cisco.com/download/home/283853163/type/283853158/release/suse

2.     After the download is complete browse to:

a.     cisco-ucs-drivers-1.1901.1.0-suse.iso\12.4\network\cisco\vic\3.1.142.369 and copy cisco-enic-usnic-kmp-default-3.1.142.369_k4.12.14_94.41-700.19.x86_64.rpm to HANA server

b.      cisco-ucs-drivers-1.1901.1.0-suse.iso\12.4\storage\cisco\vic\1.6.0.47 and copy cisco-fnic-kmp-default-1.6.0.47_k4.12.14_94.37-1.x86_64.rpm to HANA server

3.     ssh to the Server as root.

4.     Update the enic driver with below command:

rpm -Uvh cisco-enic-usnic-kmp-default-3.1.142.369_k4.12.14_94.41-700.19.x86_64.rpm

5.     Update the fnic driver with below command:

rpm -Uvh cisco-fnic-kmp-default-1.6.0.47_k4.12.14_94.37-1.x86_64.rpm

Multipath Configuration

This reference architecture uses Device-mapper Multipath, a native component of the Linux operating system. Using Device-mapper Multipath allows the configuration of multiple I/O paths between the server blades and storages.

Each node has two I/O paths connected with the storage. Multipathing aggregates all physical I/O paths into a single logical path. The LUNs are always available unless both paths fail.

Device-mapper Multipath is used for the following I/O paths:

·       SAP HANA server boot volume

·       SAP HANA data volume

·       SAP HANA log volume

·       SAP HANA shared volume

1.     ssh to the Server as root.

2.     Create the following entry in /etc/multipath.conf

3.     Start the multipath daemon and enable to start at the boot

4.     Check the status of multipath devices using multipath -ll

5.     Use dracut to include multipath in the initrd image:

Red Hat Enterprise Linux for SAP Solutions 7.5 OS Installation

This section provides the procedure for RedHat Enterprise Linux 7.5 Operating System and customizing for SAP HANA requirement.

To install the RHEL 7.5 system, follow these steps:

1.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

2.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-03.

3.     Click KVM Console.

4.     When the KVM Console is launched, click Boot Server.

5.     Choose Virtual Media > Activate Virtual Devices:

a.     For Unencrypted Virtual Media Session, select Accept this Session and then click Apply.

6.     Click Virtual Media and choose Map CD/DVD.

7.     Click Browse to navigate to the ISO media location. Select rhel-server-7.5-x86_64-dvd.iso Click Open.

8.     Click Map Device.

9.     At server boot time, during verification of VIC FC boot driver version, it recognizes the Hitachi Storage by its target WWPN numbers. This verifies the server to storage connectivity.

Figure 108      Cisco VIC Boot Driver recognize Hitachi Storage

10.  On the Initial screen choose Install Red Hat Enterprise Linux 7.5 to begin the installation process.

Figure 109      Red Hat Enterprise Linux 7.5 Installation screen

11.  Choose Language and click Continue.

12.  The Installation Summary page displays. Click Date & Time; choose the appropriate timezone and click Done.

Figure 110      Red Hat Enterprise Linux 7.5 Installation Summary Screen

13.  Click Keyboard; choose Keyboard layout and click Done.

14.  Under Software Menu, click Software selection.

15.  In the Base Environment choose Infrastructure Server.

16.  For Add-Ons for Selected Environment choose Large Systems Performance, Network File System Client, Performance Tools, Compatibility Libraries and click Done.

Figure 111      Red Hat Enterprise Linux 7.5 Installation Software Selection

17.  Under System; click Installation destination. Select Specialized & Network Disks’s “Add a disk.”

Figure 112      Red Hat Enterprise Linux 7.5 Installation Destination Disk

18.  Under Multipath Devices, select the lone 100G device identifies by its WWID. Click Done.

Figure 113      Red Hat Enterprise Linux 7.5 Installation Destination Multipath Device

19.   From the Other Storage Options choose ‘I will configure partitioning’ and click Done.

Figure 114      Red Hat Enterprise Linux 7.5 Installation Device Selection

20.  In the Manual Partitioning Screen, choose Standard Partition for New mount points will use the following partitioning scheme.

Figure 115      Red Hat Enterprise Linux 7.5 Installation Disk Partitioning

21.  Click the + symbol to add a new partition.

22.  Choose the mount point as ‘/boot.

23.  Enter the Desired capacity as 1024 MiB and click Add Mount Point.

Figure 116      Red Hat Enterprise Linux 7.5 Installation Disk Partitioning for /boot

24.  Choose the filesystem ext3.

25.  Click the + symbol to add a new partition.

26.  Choose the mount point swap.

27.  Enter the Desired capacity 2048 MiB and click Add Mount Point.

28.  Choose the filesystem swap.

29.  Click the + symbol to add / (root) partition.

30.  Choose the mount point as /.

31.  Enter the Desired capacity blank and click Add Mount Point.

32.  Choose the filesystem ext3.

Figure 117      Red Hat Enterprise Linux 7.5 Installation Disk Partitioning Summary

33.  Click Done.

34.  Review the partition layout and the size.

35.   Click Accept Changes to proceed to the next steps.

36.   Click KDUMP.

Figure 118      Red Hat Enterprise Linux 7.5 Installation Disable kdump

37.  Deselect Enable kdump and click Done

38.  Click Network & Hostname.

Figure 119      Red Hat Enterprise Linux 7.5 Installation Network and Hostname

39.  Enter the Host name and click Apply.

40.  On the NETWORK & HOSTNAME, click Ethernet:

a.     To configure the network interface on the OS, it is required to identify the mapping of the Ethernet device on the OS to vNIC interface on the Cisco UCS.

b.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

c.     Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-03.

d.     On the main pane click on Network, list of the vNICs with MAC Address are listed.

e.     Note that the MAC Address of the HANA-Mgmt vNIC is “00:25:B5:00:0A:0B”

Figure 120      Cisco UCS vNIC MAC Access

f.      By comparing MAC Address on the OS and Cisco UCS, Ethernet (enp55s0f0) on OS will carry the VLAN for Management.

Figure 121      Red Hat Enterprise Linux 7.5 Installation Network and Hostname

41.  Click Configure:

a.     Click IPv4 Settings, and choose Manual for Method.

b.     Under Addresses Click Add.

c.     In the Address field enter <<Management IP address>>.

d.     In the Netmask field enter <<subnet mask for Management Interface>>.

e.     In the Gateway field enter <<default gateway for Management Interface>>.

f.      Click Save.

Figure 122      Red Hat Enterprise Linux 7.5 Installation Network IP Address Settings

42.  Enter the Host name and click Apply.

Figure 123      Red Hat Enterprise Linux 7.5 Installation Hostname Settings

43.  Click Done at the top left corner of the screen.

44.  IP address for rest of the Ethernet will be set post installation, by using ssh to connect to the server on Management IP.

45.  Review the installation summary and click Begin Installation.

Figure 124      Red Hat Enterprise Linux 7.5 Installation Summary

46.  The next screen will show the start of the OS installation.

47.  Click Root Password.

Figure 125      Red Hat Enterprise Linux 7.5 Installation Set Root Password

48.  Enter the Root Password and Confirm.

49.  Click Done.

50.  The installation will start and continue.

Figure 126      Red Hat Enterprise Linux 7.5 Installation Progress

51.  When the installation is complete click Reboot to finish the installation.

Network Services Configuration

To configure the server with Network services, follow these steps:

Hostnames

The operating system must be configured such a way that the short name of the server is displayed for the command ‘hostname’ and Full Qualified Host Name is displayed with the command ‘hostname –d’.

1.     Use the KVM console to log in to the installed system as the user root and the password <<var_sys_root-pw>>.

2.     Set the hostname using hostnamectl:

IP Address

Each SAP HANA Server is configured with 6 vNIC device. Table 22  lists the IP Address information required to configure the IP address on the Operating System.

Table 22      IP Addresses for SAP HANA Server

1.     To configure the network interface on the OS, it is required to identify the mapping of the ethernet device on the OS to vNIC interface on the Cisco UCS.

2.     In RHEL 7, systemd and udev support a number of different naming schemes. By default, fixed names are assigned based on firmware, topology, and location information, like ‘enp72s0’. With this naming convention, though names stay fixed even if hardware is added or removed it is often harder to read unlike traditional kernel-native ethX naming “eth0”.  Another way to name network interfaces, “biosdevnames”, is already available with installation.

3.     Configure boot parameters "net.ifnames=0 biosdevname=0" to disable both, to get the original kernel native network names.

4.     Also, IPV6 support could be disabled at this time as we use IPV4 in the solution. This can be done by appending ipv6.disable=1 to GRUB_CMDLINE_LINUX as shown below:

5.     To Run the grub2-mkconfig command to regenerate the grub.cfg file:

6.     Finally reboot system to effect the changes.

7.     To configure the network interface on the OS, it is required to identify the mapping of the ethernet device on the OS to vNIC interface on the Cisco UCS.

8.     From the OS, run the following command to get list of Ethernet device with MAC Address:

9.     In Cisco UCS Manager, click the Servers tab in the navigation pane.

10.  Select Service Profile > root > Sub-Organization > T01-HANA > HANA-ScaleUp-03

11.  On the main pane click Network; the list of the vNICs with MAC Address are listed.

Figure 127      Cisco UCS vNIC MAC Address

12.  Note the MAC Address of the HANA-Client vNIC is “00:25:B5:00:0B:0A”.

13.  By comparing MAC Address on the OS and Cisco UCS, eth5 on OS will carry the VLAN for HANA-Client.

14.  Go to network configuration directory and create a configuration for eth3:

15.  Repeat steps 12 through 18 for each vNIC interface.

16.  Add default gateway:

DNS

Domain Name Service configuration must be done based on the local requirements.

Add DNS Servers entry:

Hosts File

HANA nodes should be able to resolve internal network IP address, below is an example of Scale Up HANA System host file with the entire network defined in the /etc/hosts file.

RHEL 7.5 System Update and OS Customization for SAP HANA

To update and customize SAP HANA, follow these steps:

1.     In order to patch the system, the repository must be updated. Note that the installed system doesn’t include any update information. In order to patch the RedHat System, it must be registered and attached to a valid Subscription. The following line will register the installation and update the repository information.

2.     If proxy server is required to access the internet, please update the proxy settings using

3.     Update only the OS kernel and firmware packages to the latest release that appeared in RHEL 7.5. Set the release version to 7.5

4.     Add the repos required for SAP HANA.

5.     Apply the latest updates for RHEL 7.5 Typically, the kernel is updated as well:

yum -y update

6.     Install dependencies in accordance with the SAP HANA Server Installation and Update Guide. The numactl package if the benchmark HWCCT is to be used.

7.     Disable SELinux:

8.     For compatibility reasons, four symbolic links are required:

9.     The Linux kernel shipped with RHEL 7 includes a cpuidle driver for recent Intel CPUs: intel_idle. This driver leads to a  different behavior in C-states switching. The normal operating state is C0, when the processor is put to a higher C state, which saves power. But for low latency applications, the additional time needed to stop and start the execution of the code will cause performance degradations. Modify the file /etc/default/grub and append the following parameter to the line starting with GRUB_CMDLINE_LINUX:

10.  To implement these changes, rebuild the GRUB2 configuration:

grub2-mkconfig -o /boot/grub2/grub.cfg

11.  Turn off auto-numa balancing: SAP HANA is a NUMA (non-uniform memory access) aware database. Thus it does not rely on the Linux kernel's features to optimize NUMA usage automatically. Depending on the workload, it can be beneficial to turn off automatically NUMA balancing. For this purpose, add "kernel.numa_balancing = 0" to /etc/sysctl.d/sap_hana.conf (please create this file if it does not already exist) and reconfigure the kernel by running:

12.  The "numad" daemon must be disable:

13.  Configure tuned to use profile "sap-hana." The tuned profile "sap-hana", which is provided by Red Hat as part of RHEL 7 for SAP Solutions, contains many of the configures some additional settings. Therefore the "sap-hana" tuned profile must be activated on all systems running SAP HANA:

14.  Disable ABRT, Crash Dump:

15.  Disable core file creation. To disable core dumps for all users, open /etc/security/limits.conf, and add the line:

16.  Enable group "sapsys" to create an unlimited number of processes:

17.  Disable Firewall:

18.  Reboot the OS by issuing reboot command.

19.  Optional: old kernels can be removed after OS update:

Install Cisco eNIC and fNIC Driver

To download the Cisco UCS Drivers ISO bundle, which contains most of the Cisco UCS Virtual Interface Card drivers, follow these steps:

1.     In a web browser, navigate to https://software.cisco.com/download/home/283853163/type/283853158/release/redhat

2.     After the download is complete browse to

a.     cisco-ucs-drivers-1.1901.1.0-redhat.iso\7.5\network\cisco\vic\3.1.137.5 and copy kmod-enic-3.1.137.5-700.16.rhel7u5.x86_64.rpm to HANA server

b.     cisco-ucs-drivers-1.1901.1.0-redhat.iso\7.5\storage\cisco\vic\1.6.0.47 and copy kmod-fnic-1.6.0.47-rhel7u5.el7.x86_64.rpm to HANA server

3.     ssh to the Server as root.

4.     Update the enic driver with below command

5.     Update the fnic driver with below command

Network Time

The configuration of NTP is important and must be performed on all systems. To configure network time, follow these steps:

1.     Install NTP-server with utilities.

2.     Configure NTP by adding at least one NTP server to the NTP config file /etc/ntp.conf.

3.     Stop the NTP services and update the NTP Servers.

4.     Start NTP service and configure it to be started automatically.

Multipath Configuration

This reference architecture uses Device-mapper Multipath, a native component of the Linux operating system. Using Device-mapper Multipath allows the configuration of multiple I/O paths between the server blades and storages.

Each node has two I/O paths connected with the storage. Multipathing aggregates all physical I/O paths into a single logical path. The LUNs are always available unless both paths fail.

Device-mapper Multipath is used for the following I/O paths:

·       SAP HANA server boot volume

·       SAP HANA data volume

·       SAP HANA log volume

·       SAP HANA shared volume

1.     ssh to the Server as root.

2.     Create the following entry in /etc/multipath.conf:

3.     Restart the multipath daemon and enable to start at the boot:

4.     Check the status of multipath devices using multipath –ll:

Configure HANA Persistent Storage Volume Configuration

For both operating systems, SUSE Linux Enterprise Server for SAP Applications and Red Hat Enterprise Linux, Hitachi uses an LVM-based storage layout. Once installing the operating system and correctly configuring multipathing, you can see the assigned LUNs in the directories:

For example:

The last 6 digits of this number indicates the LDEV ID you have used during the LUN assignment. In the example above, 000101 maps to LDEV ID: 00:01:01.

For all the LUNs besides of the one hosting the operating system, you need to initialize the LUNs for use by LVM, running the pvcreate command, which is part of the lvm2 rpm package, for example:

pvcreate -ff -y /dev/mapper/360060e801227fc00504027fc00000101

After you have prepared all the LUNs, you need to configure the volume groups using the vgcreate command. The names for the volume group differs between scale-up and scale-out installations.

·       The volume groups for scale-up use vgdata, vglog, vgshared.

The command to create the volume group takes no specific options. The following example creates the volume group for SAP HANA log in a scale-up scenario using 4 physical disks / LUNs:

vgcreate vglog /dev/mapper/360060e801227fc00504027fc0000010[2,3,4,5]

For creating other volume groups, use the same syntax, exchanging the volume group name as well as the physical disks or LUNs.

Once creating the volume groups, you need to create a logical volume on top. The general syntax is the following:

lvcreate --yes --extents=100%VG --stripes <# luns> --stripesize 1024 --name <lv name> <volume group>

Use Table 23  to complete the creation of logical volumes.

Table 23      Details for Creating Logical Volumes

Create the file system on top of the logical volume. Hitachi storage arrays use the XFS file system. In Table 24  , find the options to create and mount the file system.

Table 24      File System Create and Mount Options

To create a file system, use the following command:

For example:

SAP HANA Persistent Storage Volume Configuration for Scale-Up Deployments

For scale-up systems, you need to persist the file systems, including the mount options, in one of the operating system's startup file, /etc/fstab/, to mount the file systems automatically during boot operations.

Add the following entry for each filesystem to /etc/fstab:

Refer to Table 22  and Table 22  for volume group and logical volume names as well as the mount options.

Configuration Example on SUSE Linux Enterprise Server for SAP Applications

List of assigned LUNs:

Initialize the LUNs using pvcreate:

Create volume group for data:

Create volume group for log:

Create volume group for shared:

Create logical volume for data:

Create logical volume for log:

Create logical volume for shared:

Create filesystem for data:

Create filesystem for log:

Create filesystem for shared:

Create mount directories for the data, log, and HANA shared file systems:

Add the following entry to /etc/fstab:

Use the following command to mount the file systems from /etc/fstab:

 Use the df –h command to check the status of all mounted volumes:

Change the directory permissions before installing SAP HANA. Use the chmod command on each volume after the file systems are mounted:

Configuration Example on Red Hat Enterprise Linux

List of assigned LUNs:

Initialize the LUNs using pvcreate:

Create volume group for data:

Create volume group for log:

Create volume group for shared:

Create logical volume for data:

Create logical volume for log:

Create logical volume for shared:

Create filesystem for data:

Create filesystem for log:

Create filesystem for shared:

Create mount directories for the data, log, and HANA shared file systems:

Add the following entry to /etc/fstab:

Use the following command to mount the file systems from /etc/fstab:

 Use the df –h command to check the status of all mounted volumes:

Change the directory permissions before installing SAP HANA. Use the chmod command on each volume after the file systems are mounted:

SAP HANA Installation

Please refer to the official SAP documentation which describes the installation process with and without the SAP unified installer.

SAP HANA Server Installation Guide

All SAP installation and administration documentation is available here: http://service.sap.com/instguides

HDBPARAM Parameters

The following parameters were set on the HANA system. These parameters change I/O behavior and enhance the database behavior for the Hitachi storage.

For Data and Log Volumes use the following hdbparams:

SAP HANA 1.0

In order to use these parameters in SAP HANA you need to run the following commands in the Linux shell as <sid>adm user:

SAP HANA 2.0

With HANA 2.0, global.ini is used to set the parameter vector for optimal storage performance. Add the following parameters in the /hana/shared/<SID>/global/hdb/custom/config/global.ini:

Certified SAP HANA Hardware Directory

Certified SAP HANA Hardware Directory: Enterprise Storage

SAP HANA TDI Documentation

·       SAP HANA TDI: Overview

·       SAP HANA TDI: FAQ

·       SAP HANA TDI: Storage Requirements

·       SAP HANA TDI: Network Requirements

Important SAP Notes

Read the following SAP Notes before you start the HANA installation. These SAP Notes contain the latest information about the installation, as well as corrections to the installation documentation.

The latest SAP Notes can be found here: https://service.sap.com/notes.

SAP HANA IMDB Related Notes

SAP Note 1514967   - SAP HANA: Central Note

SAP Note 1523337   - SAP HANA Database: Central Note

SAP Note 2000003   - FAQ: SAP HANA

SAP Note 1780950 - Connection problems due to host name resolution

SAP Note 1755396 - Released DT solutions for SAP HANA with disk replication

SAP Note 1890444 - HANA system slow due to CPU power save mode

SAP Note 1681092   - Support for multiple SAP HANA databases on a single SAP HANA appliance

Linux Related Notes

SAP Note 2235581 - SAP HANA: Supported Operating Systems

SAP Note 2205917 - SAP HANA DB: Recommended OS settings for SLES 12 / SLES for SAP Applications 12

SAP Note 1984787 - SUSE LINUX Enterprise Server 12: Installation notes

SAP Note 1275776 - Linux: Preparing SLES for SAP environments

SAP Note 2382421- Optimizing the Network Configuration on HANA- and OS-Level

SAP Note 2002167 - Red Hat Enterprise Linux 7.x: Installation and Upgrade

SAP Note 2292690 - SAP HANA DB: Recommended OS settings for RHEL 7

SAP Note 2009879   - SAP HANA Guidelines for RedHat Enterprise Linux (RHEL)

SAP Note 1944799   - SAP HANA Guidelines for SLES Operating System

SAP Note 1731000 - Non-recommended configuration changes

SAP Note 1557506 - Linux paging improvements

SAP Note 1829651 - Time zone settings in SAP HANA scale out landscapes

SAP Application Related Notes

SAP Note 1658845 - SAP HANA DB hardware check

SAP Note 1681092 - Support for multiple SAP HANA databases one HANA aka Multi SID

SAP Note 1577128   - Supported clients for SAP HANA

SAP Note 2186744   - FAQ: SAP HANA Parameters

SAP Note 1943937   - Hardware Configuration Check Tool - Central Note

SAP Note 2267798   - Configuration of the SAP HANA Database during Installation Using hdbparam

SAP Note 2156526   - Parameter constraint validation on section indices does not work correctly with hdbparam

SAP Note 2399079   - Elimination of hdbparam in HANA 2

Cisco

MDS Best Practices: https://www.cisco.com/c/en/us/products/collateral/storage-networking/mds-9700-series-multilayer-directors/white-paper-c11-738426.html

Cisco MDS 9000 Series Interfaces Configuration Guide, Release:  8.xhttps://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/sw/8_x/config/interfaces/cisco_mds9000_interfaces_config_guide_8x.html

Nexus vPC Best Practices: https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/interfaces/configuration/guide/b_Cisco_Nexus_9000_Series_NX-OS_Interfaces_Configuration_Guide_7x.html

Cisco UCS Best Practices: https://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-manager/whitepaper_c11-697337.html

Cisco UCS Performance and Tuning: https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/whitepaper_c11-740098.pdf

Cisco UCS 6454 Spec Sheet https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/ucs-6454-fab-int-specsheet.pdf

Cisco UCS 6300 Spec Sheet https://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/6332-specsheet.pdf

Cisco UCS: Design Zone for SAP Applications (technical documentation)

Cisco UCS: Data Center Solutions for SAP (customer references)

Hitachi Storage

Hitachi Virtual Storage Platform F Series:

https://www.hitachivantara.com/en-us/pdf/datasheet/vsp-f-series-all-flash-enterprise-cloud-solutions-datasheet.pdf

Hitachi Virtual Storage Platform G Series:

https://www.hitachivantara.com/en-us/pdf/datasheet/vsp-g-series-hybrid-flash-midrange-cloud-solutions-datasheet.pdf

SAP HANA Tailored Data Center Integration with Hitachi VSP F/G Storage Systems and SVOS RF

https://www.hitachivantara.com/en-us/pdfd/architecture-guide/sap-hana-tdi-on-vsp-g-series-vsp-f-series-with-svos-reference-architecture-guide.pdf

Cisco Intersight gives manageability and visibility to multiple UCS domains through a common interface, regardless of location. The Base addition is available for UCSM starting at release 3.2(1) at no additional cost.

To add the Cisco UCS Fabric Interconnects into Intersight, follow these steps:

1.     Connect to https://www.intersight.com.

Prerequisites

The following prerequisites are necessary to setup access to Cisco Intersight:

1.     An account on cisco.com.

2.     A valid Cisco Intersight account. This can be created by navigating to https://intersight.com and following the instructions for creating an account. The account creation requires at least one device to be registered in Intersight and requires Device ID and Claim ID information from the device. See Collecting Information From Cisco UCS Domain for an example of how to get Device ID and Claim ID from Cisco UCS Fabric Interconnect devices.  

3.     Valid License on Cisco Intersight – see Cisco Intersight Licensing section below for more information.

4.     Cisco UCS Fabric Interconnects must be able to do a DNS lookup to access Cisco Intersight.

5.     Device Connectors on Fabric Interconnects must be able to resolve svc.ucs-connect.com.

6.     Allow outbound HTTPS connections (port 443) initiated from the Device Connectors on Fabric Interconnects to Cisco Intersight. HTTP Proxy is supported.

Setup Information

To setup access to Cisco Intersight, the following information must be collected from the Cisco UCS Domain. The deployment steps provided below will show how to collect this information.

·       Device ID

·       Claim Code

Cisco Intersight Licensing

Cisco Intersight is offered in two editions:

·       Base license which is free to use, and offers a large variety of monitoring, inventory and reporting features.

·       Essentials license, at an added cost but provides advanced monitoring, server policy and profile configuration, firmware management, virtual KVM features, and more. A 90-day trial of the Essentials license is available for use as an evaluation period.

New features and capabilities will be added to the different licensing tiers in future release.

Deployment Steps

To setup access to Cisco Intersight from a Cisco UCS domain, complete the steps outlined in this section.

Connect to Cisco Intersight

To connect and access Cisco Intersight, follow these steps:

1.     Use a web browser to navigate to Cisco Intersight at https://intersight.com/.

2.     Login with a valid cisco.com account or single sign-on using your corporate authentication.

Collect Information from UCS Domain

To collect information from Cisco UCS Fabric Interconnects to setup access to Cisco Intersight, follow these steps:

1.     Use a web browser to navigate to the UCS Manager GUI. Login using the admin account.

2.     From the left navigation menu, select the Admin icon.

3.     From the left navigation pane, select All > Device Connector.

4.     In the right window pane, for Intersight Management, click Enabled to enable Intersight management.

5.     From the Connection section, copy the Device ID and Claim ID information. This information will be required to add this device to Cisco Intersight. 

6.     (Optional) Click Settings to change Access Mode and to configure HTTPS Proxy.

Add Cisco UCS Domain to Cisco Intersight

To add Cisco UCS Fabric Interconnects to Cisco Intersight to manage the UCS domain, follow these steps:

1.     From Cisco Intersight, in the left navigation menu, select Devices.

2.     Click the Claim a New Device button in the top right-hand corner.

3.     In the Claim a New Device  pop-up window, paste the Device ID and Claim Code collected in the previous section.

4.     Click Claim.

5.     On Cisco Intersight, the newly added UCS domain should now have a Status of Connected.

6.     On Cisco UCS Manager, the Device Connector should now have a Status of Claimed.

7.     Dashboard will present an overview of the managed UCS domains:

Shailendra Mruthunjaya, Cisco Systems, Inc.

Shailendra is a Technical Marketing Engineer with Cisco UCS Solutions and Performance Group. Shailendra has over eight years of experience with SAP HANA on Cisco UCS platform. Shailendra has designed several SAP landscapes in public and private cloud environment. Currently, his focus is on developing and validating infrastructure best practices for SAP applications on Cisco UCS Servers, Cisco Nexus products and Storage technologies.

Dr. Stephan Kreitz, Hitachi Vantara

Stephan Kreitz is a Master Solutions Architect in the Hitachi Vantara Converged Product Engineering Group. Stephan has worked at SAP and in the SAP space for Hitachi Vantara since 2011. He started his career in the SAP space as a Quality specialist at SAP and has worked in multiple roles around SAP at Hitachi Data Systems and Hitachi Vantara. He is currently leading the virtualized SAP HANA solutions at Hitachi Vantara, responsible for Hitachi Vantara’s certifications around SAP and SAP HANA and the technical relationship between Hitachi Vantara and SAP.

Acknowledgements

For their support and contribution to the design, validation, and creation of this Cisco Validated Design, the authors would like to thank:

·       Caela Dehaven, Cisco Systems, Inc.

·       Erik Lillestolen, Cisco Systems, Inc.

·       Pramod Ramamurthy, Cisco Systems, Inc.

·       Michael Lang, Cisco Systems, Inc.

·       Ramesh Isaac, Cisco Systems, Inc.

·       Joerg Wolters, Cisco Systems, Inc.

·       YC Chu, Hitachi Vantara

·       Tim Darnell, Hitachi Vantara

·       Markus Berg, Hitachi Vantara

·       Maximilian Weiss, Hitachi Vantara