SSC Installation

This chapter provides information and procedures to install and configure Subscriber Service Controller (SSC), a component of Cisco Policy Charging and Control (PCC) solution.

SSC provides a Graphical User Interface (GUI) based installer that can be used for single host as well as cluster installation. This installer performs installations of necessary database storage infrastructure as well as database related applications such as In Memory Database (IMDB) application, that are required to maintain the database gird for the cluster deployment of SSC. SSC installer creates the users and groups with appropriate access privileges that are required to administer the SSC deployment. The installer updates required environment variables as well as provides necessary scripts for post installation configuration and administration tasks.

SSC installation chapter provides network planning as well as cluster deployment guide lines, along with procedures for enabling High Availability (HA) and Geo Redundancy(GR) features for a cluster deployment as well as adding and removing a node from an existing SSC cluster. This chapter briefly describes the SAN storage infrastructure installation prerequisites and procedures for the data base that is being used by the SSC deployment.

This chapter also provides un-installation information for the SSC deployment. SSC Installation chapter includes following sections:

Before You Begin

This section includes the information that is required for initiating the installation procedure. It contains following sub-sections:

Pre- installation Checklist

You must verify following requirements before starting the installation procedure.

  • IBM blades and chassis or Cisco UCS C210M2 is available and properly mounted.
  • Latest versions of firm-ware and Basic Input Output System (BIOS) are installed on blades.
  • X-windows application such as Xming or Xterm is installed and active on the machines where SSC is being installed.
  • X11 forwarding is enabled in the putty configuration. Xterm application can be started from the shell and the DISPLAY variable is set correctly.
  • The Cisco MITG SSC RHEL v5.5. The 64- bit operating system customized to run on selected hardware platform is installed on blades.
  • Prior to installation for a multi host set-up with High Availability (HA) and Geo Redundancy (GR) features, the file /localhome/oracle/.ssh/known_hosts is deleted. On the host where SSC installation is complete, entry of the current host from this file is removed.For example if SSC installation is completed for host 1and host 2and it is being performed on host 3. Then verify that the file /localhome/oracle/.ssh/known_hosts on host 1 and host 2 does not have any entry for host 3. Remove such entry from host1 and host 2 if it exists.
  • Hostname is same as the interface name that is used to connect to the database, for e.g. if hostname of the box is datablade1 then “ping datablade1” resolves to the IP address on which IPCF will connect to SSC.
  • Ensure that execution of hostname command returns short host name and not the FQDN by setting the parameter HOSTNAME in the file etc/sysconfig/network to short host name and executing the command sysctl kernel.hostname =<short_hostname>.
  • SSH is set-up between the host machines on which the SSC is being deployed.
  • NTP daemon is properly configured. The server parameter in the file etc/ntp/conf has correct address of the NTP server. The OPTIONs parameter in the file etc/sysconfig//ntpd is set to -x-u ntp:ntp – p /var/run/ntpd.pid
  • IP map for management, database and application interfaces for SSC clearly defines the ethernet interfaces such as eth0, eth1, eth2 that are being used by SSC.
  • During installation you can access the var/log/messages file for any SSC installation related errors. You can use troubleshooting information to resolve these errors. After resolving them use Previous and Next buttons to re-execute the failed steps in the installation procedure.

IMPORTANT:

Refer to relevant IBM blade center or Cisco UCS C210M2 documentation for system hardware details.

Hardware Requirements

This section lists the recommended hardware requirements for single host as well as cluster installation.

Following are the recommended hardware requirements:
  • Processors: Up to 2 x Dual-core Intel Xeon Processor up to 3.0GHz.
  • Memory: 96GB Fully Buffered DIMM. Minimum: 16GB.
  • Hard drives: 140GB, minimum 40GB.
  • Networking: Dual Gigabit Ethernet up to 8 ports. Minimum - 2 ports for single host installation, 4 ports for cluster installation.
  • Max cache : 4MB L2 shared cache.
  • Front side bus: 1333 MHz.

IMPORTANT:

Hardware requirements may vary as per SSC deployment configuration.

RHEL File System Partitioning Guide Lines

This section lists guidelines for RHEL partitioning.

Following are the recommended values:
  • Root (/): Hosts the root file system. Minimum Requirement is 2Gb.
  • localhome (/localhome): Hosts SSC application. In some deployments it can host primary data base along with application. Minimum requirement is 80 Gb.The partition size scales as per number of subscribers in the database.
  • Storage Array(/u01): This is a separate file system over the storage array to host raw file system of the database.

Users and Groups

This section lists various users and their groups created by the SSC installer. Following table lists their names and descriptions.


Table 1. SSC Users and Groups
Sr.No User Name Group Name Description

1

sscadmin

sscadmin, oinstall,timesten,dba

A Linux administrative user for SSC application. Who can administer SSC application, log onto all databases that are associated with this software installation using DBA privileges. Start or stop database services and perform other database related activities.

2

Policy_Operator

.

An SSC application user, who can manage policies.

3

Sys_Operator

 .

An SSC application user, who can manage SSC application.

4

Sys_Admin

 .

An application user, who can act as a super user for SSC application.


If the groups oinstall, timesten and dba does not exist on the host, then create these groups by using command #groupadd <group_name> by logging in with root administrative privileges.

Environment Variables

This section describes the environment variables required for SSC installation. Following table lists required environment variables along with their description.


Table 2. Environment Variables
Sr.No Variable Name Description

1

$SSC_HOME

Stores the path of the directory where SSC is installed by the user.

2

SSC_BIN

Stores the path of the directory where SSC binaries are available.

3

database_HOME

Stores home directory path for database.

4

IMDB_App_HOME

Stores home directory path for the In Memory Database (IMDB) application.


Network Configuration For Cluster Installation

This section describes guidelines to design network configuration for cluster installation.

A blade cluster contains multiple blades. In a cluster installation you need to install SSC components on all the blades in a cluster. As per your IP map you can install the SSC application, primary and stand-by database along with the IMDB application on respective blades.

This section contains following sub sections:

Network Configuration

This section briefly describes the network configuration required for the SSC deployed in a cluster environment.

While configuring the network you need to ensure that the database remains available even if one of the blades is not accessible for some time.

You can create such a highly available clustered database solution by installing SSC components across various blades using network interfaces. You need to configure these interfaces in such a way that they are highly available from the ethernet interface as well as from the cabling prospective.

IMPORTANT:

Before proceeding with SSC cluster installation you need to define a detailed IP map that co-relates the SSC management and application interfaces as well as primary and standby databases with available IP addresses for entire SSC deployment. During installation you must ensure that eth0 is selected as a management interface. The IP map should list Blade, Eth Interface associated with the blade, Alias or blade name, validations and description.

Following figure describes a default network configuration for an SSC cluster with two blades.
Figure 1. Network Configuration for a Cluster

To provide a highly available clustered database solution, SSC services, In Memory Database (IMDB) and RDBMS use various interfaces across the blades. These interfaces need to be highly available from the Ethernet interface and cabling perspective.

This can be achieved by constructing three VLANs in high availability configuration mode. During the installation process you need to provide IP address of the host for each interface.

These three VLANs are:
  • Traffic VLAN: It carries data traffic for the SSC cluster. Maximum bandwidth should be configured for this VLAN. It serves the Sh, En and SPR API interfaces along with a standby interface for the Oracle database.
  • Management VLAN: It carries management traffic for the SSC cluster and used for following interfaces:
    • UI access to SSC.
    • SSH access to the blades for SSH login for installation and troubleshooting.
    • Web access to IBM chassis management console.
  • PPT Interface : It provides access to PPT server component to exchange information related to data plans, notification templates, subscription tires, dynamic profile attributes, area, region and region lists with the PPT application component of Cisco PCC solution.
  • Private VLAN: It provides Inter-Process Communication (IPC) between the application nodes as well as carries the private traffic between IMDB instances.

Sample 3 Node Cluster IP Plan

This section briefly describes a three node cluster IP plan and associated interfaces for the SSC deployment.

A cluster IP plan describes:
  • Deployment of SSC install set components on the blades.
  • Ethernet Interface usage.
Following is the deployment of SSC install set components on three blades:
  • Blade one hosts SSC application as well as primary database.
  • Blade two hosts only SSC application.
  • Blade three hosts only SSC application.
Following ethernet (eth) interfaces are used in this deployment:
  • Eth0: For management VLAN. This IP address needs to be configured manually.
  • Eth1: for the traffic VLAN
  • Eth2: for the private VLAN for IMDB
  • Eth3: for the private VLAN for IPC
To install SSC in a cluster environment:
  • Ensure that you have configured the network and have a detailed IP map as described in the previous section.
  • Ensure that all the binaries required for installation are present on all blades in the cluster.
  • Select the blade or host on which to install primary database.
  • Create In Memory Database Application (IMDB) grid for primary database.
  • Select the blade on which to install the secondary database.
  • Synchronize primary database with secondary database.
  • Initiate SSC and bind the controllers using SSC console.

Access Installation Files

The SSC installer constitutes five zip files namely:
  • ssc_<version>_rhel_x86_qa1.zip
  • ssc_<version>_rhel_x86_qa2.zip
  • ssc_<version>_rhel_x86_qa3.zip
  • ssc_<version>_rhel_x86_qa4.zip
  • ssc_<version>_rhel_x86_qa5.zip

IMPORTANT:

SSC installer also includes a binary file for upgrading SSC along with above mentioned files.

Copy and extract these files in a temporary directory.

During the installation use tail command to view SSC installation logs being created in/var/log/messages file, to verify the success or failure of commands being executed by the SSC installer.

After installation refer to the installation summery captured in Subscriber_Services_Controller_InstallLog.log file which is located in /localhome/install folder.

IMPORTANT:

For the cluster installation, the installer executable must be available on all the blades where you want to install the SSC.

If there are any failures in system logs then, fix those errors by referring to Troubleshooting chapter, and re-execute the installation steps using Previous and Next buttons, provided by installer GUI.

Single Host Installation

This section describes the procedure to install SSC on a single host.

Installing SSC on a single host

Perform following procedure for a single host installation:

  1. On Linux host login as a user with root administrative privileges.
  2. Locate the SSC installer archive files as explained in previous section.
  3. Extract all zip files that constitute the SSC installer.
  4. Initiate the installation process by issuing following command: 
    /. start.sh

    IMPORTANT:

    As the SSC installer is a GUI based installer, ensure that x windows application such as Xming or Xterm is active and running, before you execute the start script.

    The installer displays following messages:
    SSC installation config
directory is /tmp/sscCurrent directory is /root/tools/SSC/SC_12.1.225Extracting
installdata.tar in /localhome/install. Please wait
....Extracting Sinstalldata.tar
in /localhome/install. Please wait ....Extracting upgradedata.tar
in /localhome/installChange setup.bin
as executableCalling setup binPreparing to install...Extracting
the JRE from the installer archiveExtracting the installation
resources from the installer archive...

    Configuring the installer
for this system's environment...Launching installer...
  5. SSC Installer displays the Introduction screen.
  6. Click Next SSC Installer displays License Agreement screen. Select the option I accept terms of license agreement.
  7. Click Next SSC Installer displays Cluster Installation screen. As this installation is for single host. Select Number of Blades as 1. The Choose Blade field displays the available blade. Using Choose Interface field select the interface for this blade.

    IMPORTANT:

    In a single host as well as cluster installation do not change the value of Eth0, as this is the interface that is being used to connect to the box, otherwise connection to the installer will be lost. IP addresses assigned to all the Ethernet interfaces are stored in ../etc/hosts and in ../etc/sysconfig/networking-scripts/ifcfg-eth.

  8. Click Add to add this interface Add interfaces for database and application, as per your network configuration strategy and IP map.

    IMPORTANT:

    For multi node cluster installation refer Cluster Installation section.

  9. Click Next SSC Installer displays Installation Set screen. The options Primary and Stand-by site are applicable only if you are implementing geo redundancy feature. Select whether you want to install Application, Primary Database or Secondary Database components from SSC installation set on the available blades.
    Primary Site and Stand-by site options are used to implement the geo redundancy feature. In a multi-host cluster environment, if you are implementing the geo-redundancy, then as per your IP map specify whether you want to use this SSC installation as primary site or as a stand-by site.

    IMPORTANT:

    Primary and standby databases cannot be configured on same blade.

  10. Click Next Following step completes the installation of SSC database. Installer configures database and IMDB instances. After installing selected Installation Set, SSC installer creates database schema and IMDB cache. You need to check the ssctmp.properties file. You can edit this file as per your requirements. Refer following note as a guide line:

    IMPORTANT:

    Ensure that INSTALL_DB in /localhome/install/ssctmp.properties is set to true, if database schema is to be installed on this blade. database_DB_HOST in /localhome/ssc/install/spr_install/createSSC_grid.cfg is set to the host name of box where primary database instance is created. IMDB_* in /localhome/SSC/install/spr_install/cratessc_grid.cfg is set to the host names of the boxes that need to be part of the IMDB grid. InstanceName in /localhome/ssc/etc/system.cfg is set to the database instance that is to be configured on this box.

  11. Click Next Following step completes the creation of database schema and IMDB configuration.
  12. SSC Installer configures Kernel Parameters, sets network configuration files, adds users and groups. Time required to configure these parameters depends upon your hardware and the options that you have selected. During this period the Next button will be grayed out. Click Next when that button is available. If any errors or suggested changes regarding certain configurations are displayed by the SSC installer, then resolve such errors before proceeding with installation. Refer Troubleshooting chapter for more information.
  13. SSC Installer displays Pre- Installation Summary screen.
  14. Click Install. SSC Installer displays Important Information such has important steps that are being performed by the installer. Number of users that are being created. Along with a list of manual steps that need to be performed to complete the SSC installation. The installer waits for user input. You need to check and edit the /localhome/install/ssctmp.properties file as per your SAN and geo-redundancy requirements.
  15. Click Next SSC Installer displays Installation Complete screen, when SSC application is installed on your system. After installation refer to overall summary of installation steps along with success or failure indicators in /localhome/install/Subscriber_Services_Controller_InstallLog.log

    IMPORTANT:

    After successful installation, maintenance mode is enabled for the SSC deployment which affects the execution of cron jobs. Refer Controlling Maintenance Mode in Before You Begin section of SSC Administration chapter.

Salient Installation

A user with root administrative privileges can use the sscinstall.sh script to perform the silent SSC installation using parameter values from the file ssctmp.properties.

Installing SSC Using Script

This section briefly describes the non-GUI or script based installation.

Perform following procedure to saliently install SSC:
  1. Log in with root administrative privileges.

    IMPORTANT:

    Before proceeding with salient installation, ensure that X11 forwarding option is disabled.

    IMPORTANT:

    The sscinstall.sh script performs silent installation using parameter values from the file ssctmp.properties, before proceeding with installation ensure that interface values and other parameters are updated in this file for the respective node as per your network plan.

  2. Create a place holder directory, outside the /localhome directory.
  3. Copy all the zip files of SSC installer archive to this directory.

    IMPORTANT:

    Refer section Accessing Installation Files for more information.

  4. Extract all the installation zip files and access Silent Installation directory.
  5. Execute the script ./sscinstall.sh to perform silent installation.

SAN Based Disk Installation

This section briefly describes SAN installation and configuration for the SSC deployment.

SAN installation and configuration is required for multi host, cluster deployments that support HA and GR features. This section includes following sub-sections:
  • Prerequisites and Assumptions
  • User Group and ASM Directory Creation
  • SAN Disk Configuration
  • ASM Libraries Configuration
  • ASM Disk Value Creation
  • Oracle Infrastructure Installation

Prerequisites and Assumptions for SAN Infrastructure

This section lists the prerequisites and assumptions for the SAN infrastructure installation.

Following are the prerequisites:
  • Linux kernel version is 2.6.18-194.el5.
  • Any previously installed Oracle infrastructure has been removed.
  • Using root administrative privilege and commands from /usr/sbin/getenforce. It is possible to check the status of SELinux parameter. This status should be Disabled. It can be changed by accessing the /etc/selinux/configset file.
Following are the assumptions:
  • SAN is connected to the Linux machine that is hosting SSC Oracle database.
  • SAN disks have the RAID -5 configuration and are represented as devices in /dev directory.
  • During SSC grid installation the primary database is to be installed on the first node.

User, Group and ASM Installation Library Directory Creation

This section briefly describes account and ASM directory creation.

This section briefly describes how to create user, group and ASM installation library. This section includes following sub-sections:
  • Adding a Group
  • Creating Oracle User
  • Resetting Password for the Oracle User
  • Creating ASM Installation Directory and Setting System Limits

Adding a Group

This section describes how to add various groups. Using root administrative privileges you need to create following groups for installing SAN infrastructure:
  • oinstall
  • oracle
  • timesten
  • dba
Use the command cat/tec/group ! grep “^<groupname> to check the existence of groups listed above. If these groups does not exist then create them using following commands with root administrative privileges:
  • groupadd oinstall
  • groupadd oracle
  • groupadd timesten
  • groupadd dba

Creating Oracle User

This section describes how to create an Oracle user. To create an Oracle User:
  • Ensure that localhome directory exists: If it does not exist then create it and change its permission using the commands , mkdir /localhome and chmod 775/localhome.
  • Ensure that localhome/oracle directory exists: If it does not exist then create it and change its permission using commands mkdir/localhome/oracle and chmod 775 .
  • Ensure that the user oracle exists: By using following command with root administrative privileges, Cat/etc/passwd |grep “^oracle”. If it does not exist then crate it using following command, USERADD –C Database software owner –d/localhome/oracle –g oinstall –G oracle,timesten,dboracle.
  • Ensure that ownership of /localhome/oracle is changed : By using the command chown oracle: oracle/localhome/oracle.

Resetting Password of Oracle User

This section describes how to reset the password for Oracle user. To rest the password:
  • Log in with root administrative privileges.
  • Enter the command #passwd oracle.
  • Specify new user password using which the oracle user can access the system.

Creating ASM Installation Directory and Setting System Limits

This section describes how to configure the ASM installation directories and set system limits. To create installation directories:
  • Log in with root administrative privileges.
  • Enter command mkdir/oracleinfra
  • Enter command chown oracle:oinstall/oracleinfra
System limits need to be set for providing maximum number of open file descriptor as well as process for the user oracle. To set system limits execute following commands:
  • echo “oracle soft nofile 4096” >>/ etc/security /linits.conf
  • echo “oracle hard nofile 65536 >> /etc/security/limits.conf
  • echo oracle soft nproc 16384 >> /etc/security/limits.conf
  • echo oracle hard nproc 16384 >> /etc/security/limits.conf

SAN Disk Configuration

This section briefly describes how to configure volumes for SAN disk.

Ensure that multi-pathing in SAN is enabled and following parameters are set as described:
  • Ensure that device-mapper multioath 0.4.7-34.el5 rpm is installed. By using command #rpm-qa ! grep multipath.
  • Ensure that multi-pathing is enabled by checking value of MULTIPATH parameter, by using following command # cat/etc/sysconfig/mkinitrd/multipath.
  • Using fdisk command create three disk volumes such as /dev/sdd1, /dev/sdd2 and /dev/sdd3 and identify where to configure SAN disk.

If volume /dev/sdd1 is identified for SAN configuration. Enter command #fdisk/dev/sdd1

Ensure parameter settings as described below:
  • The number of cylinders for the disk is set to 364722. As this number is larger than 1024 it may cause problems for boot-time and partitioning software from other operating system.
  • Use option (m for help):n
  • Use option action
  • Use option e for extended
  • Use option p for primary partition (1-4)
  • Specify partition number (1-4):1
  • First cylinder (1-36472, default 1):1
  • Last cylinder or + size or sizeM or sizeK (1-36472, default 36472):12000

    IMPORTANT:

    This value should be approximately one third of 36472 hence it is 12000.

For second volume ensure following parameters:
  • Use option (m for help):n
  • Use option action
  • Use option e for extended
  • Use option p for primary partition
  • Specify parathion number (1-4):2
  • First cylinder(12001-36472, default 12001):

    IMPORTANT:

    Using default value 12001.

  • Last cylinder or +size or +sizeM or +sizeK (12001-36472, default 36472):24000
For third volume ensure following parameters:
  • Use option (m for help):n
  • Use option action
  • Use option e for extended
  • Use option p for primary partition
  • Specify parathion number (1-4):3

    IMPORTANT:

    The system might respond as Invalid parathion number for type 3.

  • Use option action
  • Use option e for extended
  • Use option p for primary partition
  • Specify parathion number (1-4):3
  • First cylinder (24001-36472, default 24001):

    IMPORTANT:

    Using default value 24001

  • Last cylinder or +size or +sizeM or +sizeK(24001-36472, default 36472):

    IMPORTANT:

    Using default value 36472

Complete the configuration using command, Command (m for help) w.

ASM Libraries Configuration

This section describes how to configure ASM libraries.

Log in with root administrative privileges and execute the command #/etc/init.d/oracleasm configure.

Ensure that following values are set for the parameters of this command:
  • Default user to own the driver interface []:oracle
  • Default group to own the driver interface []:dba
  • Start Oracle ASM library driver on boot (y/n)[n]:y
  • Scan for Oracle ASM disks on boot (y/n)[y]”y
The command configures ASM libraries and displays following output:
  • Writing Oracle ASM library driver configuration :done
  • Initializing the Oracle ASMLib driver: [ok]
  • Scanning the system for Oracle ASMLib disks: [ok]

ASM Disk Volumes Creation

This section describes ASM disk value creation.

This section briefly describes the procedure to create ASM disk volumes when:
  • Multi-pathing is enabled.
  • Multi-pathing is disabled.

Creating ASM Disk Volumes when Multi-pathing is Enabled

This task describes how to create the disk volume when multi-pathing is enabled.

To create the disk volumes when multi-pathing is enabled:
  1. Edit the file /etc/sysconfig/oracleasm and set the parameters are described. Following are the recommended values for parameters:
    • ORACLEASM_ENABLED=true
    • ORACLEASM_UID=oracle
    • ORACLEASM_GID=dba
    • ORACLEASM_SCANBOOT=true
    • ORACLEASM_SCANORDER=""
    • ORACLEASM_SCANEXCLUDE="sd*"
  2. Change current directly to /dev/mapper directory and list files and directories present in it. Ensure that there exist entries for each partition.
  3. Create the ASM disk volumes for three previously configures disk volumes. Following are the recommended command parameter values:
    • # /etc/init.d/oracleasm createdisk ASMDISK1 /dev/mapper/mpath0p1 - Marking disk "ASMDISK1" as an ASM disk [ok]:
    • # /etc/init.d/oracleasm createdisk ASMDISK2 /dev/mapper/mpath0p2 - Marking disk "ASMDISK2" as an ASM disk: [ok]
    • # /etc/init.d/oracleasm createdisk ASMDISK3 /dev/mapper/mpath0p3 - Marking disk "ASMDISK3" as an ASM disk: [ok]
  4. If due to some reasons above mentioned command fails then use first deletedisk and then cretedisk commands. Use these commands in following sequence:
    • #/etc/init.d/oracleasm createdisk ASMDISK1 /dev/sdd1 - Marking disk "ASMDISK1" as an ASM disk: [ok]
    • # /etc/init.d/oracleasm deletedisk ASMDISK3 /dev/mapper/mpath0p3 - Removing ASM disk "ASMDISK3":[ok]
    • # /etc/init.d/oracleasm createdisk ASMDISK3 /dev/mapper/mpath0p3 - Marking disk "ASMDISK3" as an ASM disk: [ok]

Creating ASM Disk Volumes when Multi-pathing is Disabled

This task describes how to create the ASM disk volumes when multi-pathing is disabled.

To create ASM disk volumes when multi-pathing is disabled:
  1. Follow step 1 as described in previous procedure.
  2. Follow step 2 as described in previous procedure.
  3. Create the ASM disk volumes for three previously configures disk volumes. Following are the recommended command parameter values:
    • #/etc/init.d/oracleasm createdisk ASMDISK1 /dev/sdd1 - Marking disk "ASMDISK1" as an ASM disk: [ok]
    • #/etc/init.d/oracleasm createdisk ASMDISK2 /dev/sdd2 - Marking disk "ASMDISK2" as an ASM disk: [ok]
    • #/etc/init.d/oracleasm createdisk ASMDISK3 /dev/sdd3 - Marking disk "ASMDISK3" as an ASM disk: [ok]
  4. If due to some reasons above mentioned command fails then use firstdeletedisk and then cretedisk commands. Following are the recommended command parameter values:
    • #/etc/init.d/oracleasm createdisk ASMDISK3 /dev/sdd3 - Marking disk "ASMDISK3" as an ASM disk: [FAILED]
    • #/etc/init.d/oracleasm deletedisk ASMDISK3 /dev/sdd3 - Removing ASM disk "ASMDISK3": [ok]
    • #/etc/init.d/oracleasm createdisk ASMDISK3 /dev/sdd3 - Marking disk "ASMDISK3" as an ASM disk:[ok]

Oracle Infrastructure Installation

This section briefly describes how to install the Oracle storage infrastructure for the SSC deployment.

Infrastructure Installation Pre-requisites

Following are the prerequisites for Oracle installation:
  • You can access the system with Oracle user privileges.
  • DISPLAY environment variable is set correctly, by verifying that X11 forwarding option is selected for the ssh session with the host.
  • The directory /localhome/inastall/installdata/oracle/griddirectory exist.

Verifying Existence of /oracle/griddirctory

This section briefly describes how to create the /localhome/install/installdata/oracle/griddirectory if it does not exists. Verify existence of this directory before proceeding with infrastructure installation. If it does not exists then:
  • If /localhome/install directory is also not present, the create it first by using command mkdir –p /localhome/install.
  • Ensure that root is the owner of the above mentioned directory and group as well as world have read and execute permissions for it.
  • Down load all five SSC installation zip archives and extract them.
  • After extracting all the zip files ensure that you have following tar files , installdata1.tar, installdata2.tar, installdata3.tar, installdata4.tar and installdata5.tar. Copy all these tar files to /localhome/install directory and extract all the tar files in this directory.
  • The /localhome/install/installdatadirectory will be created after extraction of all these inastalldata tar files.
  • The localhome/install/installdatadirectory contains following tar files oracle1.tar, oracle2.tar, oracle3.tar and oracle4.tar
  • Extract all the oracle.tar files from localhome/install/installdatadirectory to create the /localhome/install/installdata/oracle/griddirectory.
  • Ensure that the /localhome/install directory and its sub-directories have read and execute permissions for group and world user groups.

Installing Oracle Infrastructure Software

To install oracle infrastructure software:
  1. After ensuring that DISPLAY environment variable is set as explained in previous section. Execute the export commands. Following export commands need to be executed before initiating the installer:
    • export ORACLE_HOME=/oracleinfra/product/11.2.0/grid
    • export ORACLE_BASE=/oracleinfra/base
    • export PATH =/oracleinfra/product/11.2.0/grid/bin:$PATH
    • cd/localhome/install/installdata/oracle/grid
  2. Initiate the installer by issuing following command ./runInstaller. Installer may display a warning message indicating auto check for display colors, ignore such warning message by clicking yes .
  3. Installer displays Download Software Updates screen . Select the option Skip software updates and click Next.
  4. Installer displays Select Installation Option screen. Select the option InstallOracle_Grid infrastructure Software Only. Click Next to specify the product Language. In the Select Product Languages screen select English.
  5. Click Next Installer displays Privileged Operating System Groups screen.
    Select oinstall from the drop-down lists for all three groups. The installer may display warning indicating that one and the same account is selected for operator, administrator and DBA groups, ignore this warning by clicking yes.
  6. Click Next installer displays Specify Installation Location screen. For Oracle Base specify the directory /oracleinfra/base. For Software location specify the directory /oracleinfra/product/11.2.0/grid. The installer may display a warning indicating that grid infrastructure software for the cluster should not be under an oracle base directory. Ignore this warning by clicking yes.
  7. Click Next the installer displays Create Inventory screen. Specify /oracleinfra/oraInventory directory as the Inventory directory. Click Next the installer performs prerequisite checks while displaying Perform Prerequisite Checks screen. Some prerequisite checks may fail. In such scenario, click more details link and try to fix the errors using provided information. If any warning messages are displayed then select yes option to continue installation.
  8. After completing the prerequisites the installer displays summary of the selected option. Ensure that the options areas per your requirement. Save this configuration information using Save Response option .Click Install to initiate the installation.
  9. Installer displays the Install Product screen indicating status of grid infrastructure packages that are being installed for the cluster.
  10. Installer displays Execute Configuration scripts screen. When this screen appears, then log-into a separate window and execute the scripts displayed on the screen. After competing their execution access the screen again and click OK and then Close.
    These scripts set the environment variables, execute generic as well as product specific root actions. As a root user:
    • Issue following command to configure grid for stand-along server - /oracleinfra/product/11.2.0/grid/perl/bin/perl -I/oracleinfra/product/11.2.0/grid/perl/lib -I/oracleinfra/product/11.2.0/grid/crs/install /oracleinfra/product/11.2.0/grid/crs/install/roothas.pl
    • Issue following command to configure grid for a cluster - /oracleinfra/product/11.2.0/grid/crs/config/config.sh
  11. After successful installation the installer displays Finish screen.
    To configure grid infrastructure for a stand-along server issue the command #/oracleinfra/product/11.2.0/grid/perl/bin/perl -I/oracleinfra/product/11.2.0/grid/perl/lib -I/oracleinfra/product/11.2.0/grid/crs/install /oracleinfra/product/11.2.0/grid/crs/install/roothas.pl Installation logs can be accessed using /localhome/oracle/app/orainventory/logs/location directory or in the log directory at Oracle inventory location.

SAN Infrastructure Un-installation

This section describes how to un-install the SAN infrastructure.

Un-installing SAN Infrastructure

To un-install SAN infrastructure:
  1. Log in with database administrative privileges and access the un-installation script by issuing following commands:$export ORACLE_HOME=/oracleinfra/product/11.2.0/grid$cd $ORACLE_HOME/deinstalldeinstall]$./deinstall Specify following parameters:
    • Specify the ASM Diagnostic Destination as :/oracleinfra/product/diag/asm/+asm/+ASM
    • Specify the diskgroups that are managed by this ASM instance such as ASMDISK1,ASMDISK2,ASMDISK3

      IMPORTANT:

      De-configuring ASM drops all the diskgroups at cleanup time. Specify ywhen prompted.

  2. Installer displays Check Operation Summary. Specify y when prompted. Summary displays grid infrastructure directories created by Oracle infrastructure. It also specifies where the un-installation log files will be stored.
  3. If in the previous step you have selected to proceed with un-installation, then the installer displays Clean Operation Start Information. The installer proceeds to remove listener and naming methods configurations and maintains un-installation logs in /localhome/oracle/app/oraInventory/logs directory.
  4. Issue following command with root administrative privileges, from the blade on which the Oracle infrastructure is installed:/tmp/deinstall2011-05-13_01-05-01-PM/perl/bin/perl -I/tmp/deinstall2011-05-13_01-05-01-PM/perl/lib -I/tmp/deinstall2011-05-13_01-05-01-PM/crs/install /tmp/deinstall2011-05-13_01-05-01-PM/crs/install/roothas.pl -force -delete -paramfile /tmp/deinstall2011-05-13_01-05-01-PM/response/deinstall_Ora11g_gridinfrahome1.rsp

    IMPORTANT:

    Press Enter to proceed with un-installation.

  5. Log into the blade as root user and execute *.rsp i.e. the output file mentioned in previous step. Info installer displays messages indicating successful deletion of Oracle configuration stack and removal of associated temporary directories
  6. With root administrative privileges issue following command #oracleasm listdisks. This command lists the ASM disk partitions such as ASMDISK1 and ASMDISK2.
  7. Delete each ASM disk parathion by issuing command #oracleasm deletedisk <Disk ParationName>

Cluster Installation

This section briefly describes how to install SSC cluster, configure high availability in a multi host deployment and add or remove a node from cluster.

Primary Node Installation - Cluster Setup

Before proceeding with cluster installation ensure that:
  • All the nodes on both sites are reachable to each other by providing host names as well as by their IP addresses.
  • Version of SSC components namely SSC application, IMDB and database on all nodes is same.
  • Physical memory on both the nodes where IMDB and standby database is being installed is identical.

Ensure that you have a cluster plan, before initiating the cluster installation. A plan also helps in maintenance of the cluster when you need to add or remove a node from cluster.

Refer to the installation procedure documented in the section Installing SSC on a Single Host. On each host or blade, follow this procedure from step 1 up to the cluster installation step.

Following figure displays the cluster installation screen:

While performing the cluster installation, on each blade:

  1. Select the check box Cluster Installation. Select Number of Blades as 2. You can specify the Cluster Name to identify the unique cluster, otherwise installer assigns a default cluster name SSCCLUSTER. Ensure that the cluster name is same on all the nodes in a given cluster. UsingChoose Interface field you can specify the Ethernet interfaces from eth0 up to eth5 for each blade. Refer your IP map and network planning for assigning the interfaces. If you do not have NIC bonding then you have to configure only eth1, eth2 and eth3 interfaces as eth4 and eth5 interfaces are reserved for NIC bonding. Use Add and Remove buttons to add or remove a network interface.
  2. Click Next after completing the network configuration for primary node.
  3. SSC Installer displays Installation Set screen. Refer step 9 in the Single Host Installation section to view this screen. Select the blade to be configured on this host. Select components Application and Primary Db.
  4. Click Next, the installer displays Installation Slot Information screen. Specify the Slot number for each blade and press Enter. Ensure that all the blades have respective slot numbers. By default the blade number should match the slot number.

    IMPORTANT:

    The installer displays an error message Slot interface can neither be empty nor be null, if the slot numbers are not provided.

  5. Click Next button SSC installer displays Pre-Installation Summary screen. Refer step 13 in the Single Host Installation section to view this screen. It displays information such as available and used space, product name and install folders.
  6. Before initiating installation, the installer displays Important Information screen. Refer step 11 in the Single Host Installation Section to view this screen. Update appropriate parameters of ssctmp.properties file located in localhome/ssc/install folder. Update the file /localhome/ssc/etc/system.cfg ensure that parameter SscInstanceMode is set to MASTER Following are the suggested values for important parameters of ssctmp.properties file for Primary node:
    • INSTALL_SAN_ENABLED = false (If this is a local disk installation otherwise set it to True for SAN installation.
    • INSTALL_EXT_PROFILE_INTERFACE = <change to floating IP>
    • INATALL_EXT_EVENT_INTERFACE = <change to floating IP>
    • ISTALL_EXT_MGT_INTERFACE = <change to floating IP>
    • INSTALL_EXT_SH_INTERFACE = <change to floating IP>
    On the next screen, SSC installer halts for user input, before clicking Next update required parameters in file ./localhome/ssc/install/spr_install/createSSC_grid.cfg.
  7. Login as database (Oracle) user, and updated the file ../localhome/ssc/install/spr_install/createSSC_grid.cfg Following are the suggested values specification of IMDB data store parameters:
    • IMDB_HOST_1 = < First IMDB host name>
    • IMDB_HOST_2 = < Second IMDB host name>
    • IMDB_HOST_3 =
    • IMDB_HOST_4 =
    • IMDB_HOST_5 =
    • IMDB_HOST_6 =
    • IMDB_HOST_7 =
    • IMDB_HOST_8 =
    • IMDB_HOST_9=
    • IMDB_HOST_10 =
    Following are the suggested values for specification of active standby pair parameters:
    • INSTANCE = 1
    • IMDB_AS_PAIR_1= 1:2:9990:9991
    • IMDB_AS_PAIR_2 =
    • IMDB_AS_PAIR_3 =
    • IMDB_AS_PAIR_4 =
    • IMDB_AS_PAIR_5 =
    • IMDB_AS_PAIR_6=

Secondary Node Installation - Cluster Setup

In a two node cluster, SSC Application is installed on the second note. Refer to previous task Primary Node Installation – Cluster Setup.
  1. Refer Cluster Installation Set-up screen displayed in the previous task. Specify cluster installation information for the second node.
  2. Click Next after completing the network configuration for secondary node.
  3. SSC installer displays Installation Set Screen. Select Application for blade 2.
  4. Click Next. SSC installer displays the Pre-Installation Summary screen. It displays information such as available and used space, product name and installation folders.
  5. Before initiating the installation, SSC installer displays Important Information screen. Update appropriate parameters of ssctmp.properties file located in localhome/ssc/install folder. Following are the suggested values for secondary node:
    • INSTALL_SSC_INSTANCE_ID=2
    • INSTALL_DB_INSTANCE_NAME=SSC
    • INSTALL_DB=false
    • INSTALL_SAN_ENABLED=false (false if local disk otherwise set it to true if SAN installation)
    • INSTALL_HA_ENABLED=1
    • INSTALL_EXT_PROFILE_INTERFACE=<Change to floating IP>
    • INSTALL_EXT_EVENT_INTERFACE=<Change to floating IP>
    • INSTALL_EXT_MGMT_INTERFACE=<Change to floating IP>
    • INSTALL_EXT_SH_INTERFACE=<Change to floating IP>
  6. Login as database (Oracle) user, and updated the file ../localhome/ssc/install/spr_install/createSSC_grid.cfg
    • IMDB_HOST_1 = < First IMDB host name>
    • IMDB_HOST_2 = < Second IMDB host name>
    • IMDB_HOST_3 =
    • IMDB_HOST_4 =
    • IMDB_HOST_5 =
    • IMDB_HOST_6 =
    • IMDB_HOST_7 =
    • IMDB_HOST_8 =
    • IMDB_HOST_9=
    • IMDB_HOST_10 =
    • INSTANCE = 1
    • IMDB_AS_PAIR_1= 1:2:9990:9991
    • IMDB_AS_PAIR_2 =
    • IMDB_AS_PAIR_3 =
    • IMDB_AS_PAIR_4 =
    • IMDB_AS_PAIR_5 =
    • IMDB_AS_PAIR_6=

Adding a Node to Cluster

This section describes the configurations required on a node to add it to a cluster.

While installing components of SSC application, before initiating the installation, installer displays important information. Refer step 5 of the section Primary Node Installation - Cluster Setup. Perform following configurations to add this node to cluster:
  • ORACLE_DB_HOST parameter in the file /localhome/ssc/install/spr_install/createSSC_grid.cfg is set to the host name of the machine where primary database is installed.
  • IMDB_DB_* parameter in the file /localhome/ssc/install/spr_install/crateSSC_grid.cfg is set as per the cluster plan.
  • InstanceName parameter in the file /localhome/ssc/etc/system.cfg is set to the database instance configured.

IMPORTANT:

Increment the instance Id for every node or blade that is being added to the cluster. Instance id for the master blade is reserved to 1.

Removing a Node from Cluster

You may require removing a node from the SSC cluster for maintenance or trouble shooting purpose. This section describes how to remove a node from a cluster.
  1. Shutdown all components of SSC application from the node that you want to remove form the cluster.
  2. Login to the node as the database (oracle) user.
  3. Access IMDB (TimesTen) application by issuing following command:ttIsql “UID=cacheuser;PWD=timesten;DSN=<Your InstanceName from etc/system.cfg>;OraclePwd=oracle
  4. Execute the GridDetach script by issuing following commend:Call ttGridDetach ()
  5. Exit IMDB command prompt.

createSSC_grid.cfg Parameters

This section briefly describes important parameters of crateSSC_grid.cfg file.

A cluster deployment involves multiple SSC installations or nodes. Features such as High Availability (HA) and Geo Redundancy (GR) can be implemented in an SSC cluster deployment using the database and IMDBgrid. cerateSSC_Grid.cfg file stores parameters that are used to enable these features. This file is located in /localhome/ssc/install/spr_install directory.

In a cluster deployment following TimesTen (IMDB) and log related parameters need to be defined appropriately:
  • T10CacheCleanupRetryCount: This indicates the maximum retry attempts for the cleanup of the cache information for a failed TimesTen site. Minimum one and by default 12 retries are allowed for the clean-up attempt. During a fail-over the cache information of the failed TimesTen (IMDB) site is cleaned to delete the event logs for the updates and hence to avoid filling up the disk space. This parameter indicates number of clean-up attempts.
  • T10CacheCleanupRetriesSleepTime: This indicates the sleep time in seconds between two consecutive retry attempts for the clean-up of the TimesTen cache information. The default sleep time is 5 seconds. During a fail-over the cache information of the failed TimesTen (IMDB) site is cleaned to delete the event logs for the updates and hence to avoid filling up the disk space. This parameter indicates default interval between two successive attempts.
  • T10HealthCheckCountForTimeout: This indicates the maximum retries allowed for the health check to be performed for the timeout error encountered. This parameter covers the scenario in which the failover is in progress for the failed TimesTen site and simultaneously the database monitoring script is checking the TimesTen health. This parameter indicates maximum retries allowed for the health check. Minimum and default values for this parameter are 1 and 3 respectively.
  • ArchiveLogKeep: This parameter indicates time in hours. It is referred by the archive log clean-up script when the data base is in archive log mode. The archive log cleanup job retains archive logs on both primary as well as standby database for the period specified by this parameter. This parameter is to be configured as per available disk space. The default value is one hour.

Configuring HA in Cluster

This section briefly describes how to configure high availability feature.

Configuring High Availability (HA) in Cluster

IMPORTANT:

For a single host SSC deployment, High Availability (HA) is not supported, however failure detection and recovery mechanisms exist for application processes, database, IMDB and network link for such deployment.

In a multi-host SSC deployment, High Availability ensures availability of SSC application in case of node, network or power failure. High Availability is implemented using RedHat Cluster Suit (RHCS). The RHCS protects SSC components or services in case of network disruption.

IMPORTANT:

Before attempting the HA configuration for both Cisco UCS and IBM Blade Center, you need to ensure that the hardware is configured for recovery from eth card failure.

Depending upon your deployment configuration, RHCS can protect following SSC services and related IP addresses in case of network description:
  • Services that are used for managing SSC deployment such as System controller, Log daemon and Scheduler.
  • Sh controller.
  • Event controller.
  • Profile controller.
  • Ud controller.

In a cluster deployment high availability can be configured by setting some parameters values in the file/localhome/install/ssctmp.properties. Ensure that before starting installation of cluster RPMs, the keys RPM-GPG-KEY-redhat-beta and RPM-GPG-KEY-redhat-release are imported.

To configure high availability:

  1. Install the RPMs. Ensure that following RPMs are installed:
    • Cluster_Administration-en-US-5.2-1.noarch.rpm
    • cluster-cim-0.12.1-2.el5.x86_64.rpm
    • cluster-snmp-0.12.1-2.el5.x86_64.rpm
    • cman-2.0.115-29.el5.x86_64.rpm
    • cman-devel-2.0.115-29.el5.x86_64.rpm
    • luci-0.12.2-10.el5.x86_64.rpm
    • modcluster-0.12.1-2.el5.x86_64.rpm
    • rgmanager-2.0.52-3.el5.x86_64.rpm
    • ricci-0.12.2-10.el5.x86_64.rpm
    • system-config-cluster-1.0.57-3.noarch.rpm
  2. Enable High Availability (HA) feature by setting parameter INSTAL_HA_ENABLED parameter to 1. This parameter is located in /localhome/istall/ssctmp.properties file.
  3. Set floating point IPs for Sh, Management, Event and Profile Controller interfaces. Update following parameters in the file in /localhome/istall/ssctmp.properties:
    • INSTALL_EXT_PROFILE_INTERFACE=<Change to floating IP >
    • INSTALL_EXT_EVENT_INTERFACE=<Change to floating IP >
    • INSTALL_EXT_MGMT_INTERFACE=<Change to floating IP >
    • INSTALL_EXT_SH_INTERFACE=<Change to floating IP >
  4. Set Management Module details. By updating the file in /localhome/istall/ssctmp.properties as follows:
    • INSTALL_MM_IP=<MM hostname or IP address>
    • INSTALL_MM_USER=USERID
    • INSTALL_MM_PWD=PASSW0RD

    IMPORTANT:

    You need to change the user id and password only if you have changed them during chassis set-up, otherwise keep the default values.

Geo Redundancy Setup

This section briefly describes how to enable geo redundancy feature, enable it along with cluster and how to install a secondary node in the site.

IMPORTANT:

Currently the Geo Redundancy (GR) feature is available only for the deployments that are using IBM Blade Center platform for hosting SSC database and application.

The geo redundancy feature allows you to deploy SSC in two geographically distinct sites, namely:
  • Primary site.
  • Stand-by site.

Geo Redundancy(GR) future supports failover to stand-by site for the subscriber data in case of catastrophic failure of primary site. This is achieved by using a database technology supported by RDBMS that allows maintaining a redundant repository for primary database.

Following are the prerequisites for enabling Geo Redundancy (GR) feature:
  • All nodes from both sides are reachable to each other.
  • No local stand by database is configured on any of the geo redundant site, as the stand by data base will be installed on one of these sites.
  • All SSC nodes participating the geo-redundant configuration contain same version of the SSC product components i.e. SSC application, IMDB and database.
  • Primary and stand-by sites have independent IMDB grids, and only the IMDB grid of active site is up, this is important as both the grids are attached to same database.
  • Physical memory on both the nodes where active and stand-by pair for the GR feature is to be configured, must be same.
At both primary and secondary sites, the SSC installer can be used to:
  • Enable only geo redundancy feature.
  • Enable geo redundancy with cluster.
  • Install a secondary node on a site in a cluster setup.

IMPORTANT:

While enabling any one of these geo-redundancy options on a stand-by site, follow same procedures but use the data base and ssctmp.properties as well as createSSC_Grid.cfg parameters for the stand-by site.

Enabling Geo Redundancy

This task describes how to configure geo redundancy feature without cluster. It assumes you have detailed IP map that describes the network planning.

  1. Refer your IP map. Follow the procedure to install single host up to step 7 – The cluster installation and geo redundancy screen.
  2. Select Geo Redundancy feature. The Number of Blades and Chose Blade fields will be already populated. As Geo Redundancy feature is being enabled without cluster, keep Cluster Installation and Cluster Name fields blank. Configure the Network using Choose Interface option. If NIC bonding is not available then configure interfaces eth1, eth2 and eth3 only, as the interfaces eth4 and eth5 are used for NIC bonding.
  3. Click Next after completing the network configuration. SSC installer displays the Installation Set screen. Select options Primary Site, Application and Primary Db.
  4. Click Next, SSC installer displays Geo Redundancy Information screen.
    Provide the Host Name and IP Address for Primary as well as Stand-by sites. Verify the ping results from each machine.
  5. Click Next. The installer waits for user input. Verify that the file /localhome/install/ssctmp.properties contains following parameters. Recommended values for the parameters are as follows:
    • INSTALL_SSC_INSTANCE_ID=1
    • INSTALL_DB_INSTANCE_NAME=SSC
    • INSTALL_DB=true
    • INSTALL_STANDBY_DB=false
    • INSTALL_GEO_REDUNDANCY=true
    • INSTALL_PRIMARY_INSTANCE=true
    • INSTALL_STANDBY_INSTANCE=false
    • INSTALL_PRIMARY_HOST_NAME=<primary db host>
    • INSTALL_PRIMARY_HOST_IPADDR=<primary ip address>
    • INSTALL_STANDBY_HOST_NAME=<standby db host>
    • INSTALL_STANDBY_HOST_IPADDR=<standby ip address>
    • INSTALL_SAN_ENABLED=false (false if local disk otherwise set it to true if SAN installation)
    • INSTALL_IMDB_GRID=SPRGRIDPRI

    IMPORTANT:

    ignore the parameters INSTALL_GEO_REMOTE_SITE_ID and INSTALL_GEO_SITE_ID if present.

  6. Click Next the SSC installer displays Important Information screen and wait for usr input. Update parameters in the file /localhome/ssc/install/spr_install/crateSSC_grid.cfg Following are the recommended values:
    • ORACLE_DB_HOST=<Primary DB Short Hostname>
    • IMDB_HOST_1=<Primary site IMDB node short host name>
    • SBY_DB_HOST_1=<Standby DB Short Hostname>
  7. In the crateSSC_grid.cfg set the SITE FAILOVER as per requirement. It is not recommended to disable this flag. Following are the recommended values for the parameters:
    • #For failover on any TimesTen blade or Oracle database failure.
    • SITE_FAILOVER=ALWAYS
    • #For failover of majority of TimesTen blade or Oracle database failure.
    • SITE_FAILOVER=DBONLY
    • #For no failover (default)
    • SITE_FAILOVER=DISABLED
  8. In /localhome/ssc/install/spr_install/crateSSC_grid.cfg set the flag T10HealthCheckCountForTimeout as required. If the db site monitoring script is checking the health of TimesTen (IMDB), then value of this parameter specifies number of times the health check needs to be done. If this parameter is not defined or wrongly defined then the check is performed only once.
  9. Update the file localhome/ssc/etc/system.cfg Following are the recommended values for the parameters:
    • SiteName=PRIMARY
    • InstanceName=SSC
    • SscInstanceId=1
  10. Perform remaining installation steps.

Enabling Geo Redundancy with Cluster

This task describes how to configure geo redundancy with clustering enabled. It assumes you have detailed IP map that describers the network planning.

  1. Refer your IP map. Follow procedure to install single host up to step 7 – The cluster installation and geo redundancy step.
  2. Select Cluster Installation as well as Geo Redundancy options. Select Number of Blades as 2. Specify the Cluster Name. Ensure that same cluster name is used across all nodes. Default cluster name is SSCCLUSTER. Configure network using Chose Interface option and your IP map as described in previous task.
  3. After completing network configuration, click Next. SSC installer displays the Installation Set screen. Select options Primary Site, Application and Primary Db.

    IMPORTANT:

    Installation on multiple nodes is not supported. Select only blade that is to be configured on this host.

  4. Click Next, SSC installer displays Geo Redundancy Information screen. Provided the Host Name and IP address for Primary as well as Stand- By sties. Verify the ping results from each machine.
  5. Click Next. The installer waits for the user input. Verify that the file /loalhome/install/ssctmp.properties contains following parameters. For recommended parameter values refer step 5 of previous task.
  6. Click Next the installer displays Important Information screen. Login as Oracle user and edit the file /localhome/ssc/install/spr_install/createSSC_grid.cfg Following are the recommended values for dome of the parameters of this file:
    • SBY_DB_HOST_1=<Standby DB short hostname>
    • ORACLE_DB_HOST=<Primary DB short host name>
    • IMDB_HOST_1=<Primary site IMDB node short host name>
    • IMDB_HOST_2=<Primary site IMDB node short host name>
    • IMDB_HOST_3=
    • IMDB_HOST_4=
    • IMDB_HOST_5=
    • IMDB_HOST_6=
    • IMDB_HOST_7=
    • IMDB_HOST_8=
    • IMDB_HOST_9=
    • IMDB_HOST_10=
    • INSTANCE=1
    • IMDB_AS_PAIR_1=1:2:9990:9991
    • IMDB_AS_PAIR_2=
    • IMDB_AS_PAIR_3=
    • IMDB_AS_PAIR_4=
    • IMDB_AS_PAIR_5=
    • IMDB_AS_PAIR_6=

    IMPORTANT:

    Refer section Configuring HA Cluster for enabling high availability feature.

  7. In /localhome/ssc/install/spr_install/crateSSC_grid.cfg set the flag T10HealthCheckCountForTimeout as required. If the db site monitoring script is checking the health of TimesTen (IMDB), then value of this parameter specifies number of times the health check needs to be done. If this parameter is not defined or wrongly defined then the check is performed only once.
  8. Update the file localhome/ssc/etc/system.cfg Following are the recommended values for the parameters:
    • SiteName=PRIMARY
    • InstanceName=SSC
    • SscInstanceId=1
  9. Perform remaining installation steps.

Installing a Secondary Node on a Site in cluster Setup

Primary node on the primary site has been installed successfully.

  1. Refer your IP map. Follow procedure to install single host up to step 7 – The cluster installation and geo redundancy step.
  2. Select Cluster Installation as well as Geo Redundancy options. Select Number of Blades as 2. Specify the Cluster Name. Ensure that same cluster name is used across all nodes. Default cluster name is SSCCLUSTER. Configure network using Chose Interface option and your IP map as described in previous task.
  3. After completing network configuration, click Next. SSC installer displays the Installation Set screen. Select options Primary Site, Application and Primary Db.

    IMPORTANT:

    Installation on multiple nodes is not supported. Select only blade that is to be configured on this host.

  4. Click Next, SSC installer displays Geo Redundancy Information screen. Provided the Host Name and IP address for Primary as well as Stand- By sties. Verify the ping results from each machine.
  5. Click Next. The installer waits for the user input. Verify that the file /loalhome/install/ssctmp.properties contains following parameters. Following are the recommended values for the parameters:
    • INSTALL_SSC_INSTANCE_ID=2
    • INSTALL_DB_INSTANCE_NAME=SSC
    • INSTALL_DB=false
    • INSTALL_STANDBY_DB=false
    • INSTALL_GEO_REDUNDANCY=true
    • INSTALL_PRIMARY_HOST_NAME=<primary db host>
    • INSTALL_PRIMARY_HOST_IPADDR=<primary ip address>
    • INSTALL_STANDBY_HOST_NAME=<standby db host>
    • INSTALL_STANDBY_HOST_IPADDR=<standby ip address>
    • INSTALL_SAN_ENABLED=false (false if local disk otherwise set it to true if SAN installation)
    • INSTALL_IMDB_GRID=SPRGRIDPRI
  6. Click Next the installer displays Important Information screen. Login as Oracle user and edit the file /localhome/ssc/install/spr_install/createSSC_grid.cfg Following are the recommended values for important parameters:
    • ORACLE_DB_HOST=<Primary DB short host name >
    • SBY_DB_HOST_1=<Standby DB short hostname>
    • IMDB_HOST_1=<Primary site IMDB node short host name>
    • IMDB_HOST_2=<Primary site IMDB node short host name>
    • IMDB_HOST_3=
    • IMDB_HOST_4=
    • INSTANCE=1
    • IMDB_AS_PAIR_1=1:2:9990:9991
    • IMDB_AS_PAIR_2=
    • IMDB_AS_PAIR_3=

    IMPORTANT:

    Refer section Configuring HA Cluster for enabling high availability feature.

  7. Update the flag SITE_FAILOVER in the file /localhome/ssc/install/spr_install/createSSC_grid.cfg as per requirement. It is recommended to not to set is as DISABLED. Following are recommended values for this flag:
    • #For failover on any TimesTen blade or Oracle database failure.
    • SITE_FAILOVER=ALWAYS
    • #For failover of majority of TimesTen blade or Oracle database failure.
    • SITE_FAILOVER=DBONLY
    • #For no failover (default)
    • SITE_FAILOVER=DISABLED
  8. In /localhome/ssc/install/spr_install/crateSSC_grid.cfg set the flag T10HealthCheckCountForTimeout as required. If the db site monitoring script is checking the health of TimesTen (IMDB), then value of this parameter specifies number of times the health check needs to be done. If this parameter is not defined or wrongly defined then the check is performed only once.
  9. Ensure that the file /localhome/ssc/install/spr_install/createSSC_grid is identical on both primary as well as secondary nodes.
  10. Update the file localhome/ssc/etc/system.cfg Following are the recommended values for the parameters:
    • SiteName=PRIMARY
    • InstanceName=SSC
    • SscInstanceId=2
  11. Perform remaining installation steps.

Database Migration

This section describes procedures required to migrate SSC data.

Exporting Database Schema From One SSC Instance to Another

This section describes the procedure to export database schema from instance SSC1 on host 1 to another SSC instance SSC2 on host 2.

To export database schema from SSC1 to SSC2:
  1. Shut down the IMDB and SSC application instance on host 1. Ensure that there is no activity happening on host1database.
  2. Log-in to host 1 as database administrative user and create a working directory under /home directory by issuing following command $mkdir dbdump Ensure that you have sufficient disk space for the export operation.
  3. From the home directory execute following command:exp system/sscsystem@SSC1 FILE=scc_dump.dmp log=scc_export_dump.log STATISTICS=none direct=Y owner=spradm
  4. Verify connectivity of host 2 database from host 1 by issuing following command:$tnsping SSC2 If you are not able to ping SSC2 database from hosit1, then FTP the dump file on host2 server.
  5. Shutdown the IMDB and SSC applications on host2
  6. Ensure that no activity is happening on host 2 database. Execute the clean up script on SSC2 database by issuing following command:./cleanuo_all.sh <SID>
  7. and then execute the following command: select 'DROP SEQUENCE ' || SEQUENCE_NAME || ';' seq from USER_SEQUENCES
  8. Execute the output of SQL statement mentioned in the previous step to drop all the sequences.
  9. Ensure that there are no sequences by executing following SQL command: SELECT COUNT(*) FROM USER_SEQUENCES A zero count for this SQL indicates that there are no sequences.
  10. Execute the output of following SQL statement to drop all the object types: SELECT 'DROP TYPE ' || OBJECT_NAME || ' VALIDATE;' A FROM USER_OBJECTS WHERE OBJECT_TYPE = 'TYPE
  11. Ensure that there are no object types by executing following SQL command: SELECT COUNT(*) FROM USER_OBJECTS WHERE OBJECT_TYPE = 'TYPE' A zero count for this SQL indicates that there are no object types.
  12. Execute following SQL statement:SELECT 'ALTER TABLE ' || TABLE_NAME || ' DISABLE CONSTRAINT ' || CONSTRAINT_NAME || ';' A FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R' Execute the output of this SQL statement to disable foreign keys.
  13. Ensure that the foreign key constraints are disabled by issuing following SQL statement:SELECT CONSTRAINT_NAME, STATUS FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R'
  14. import the data on the host2 that is in the SSC2 application instance by issuing following command: imp system/sscsystem@SSC2 FILE=scc_dump.dmp log=scc_import_dump.log FROMUSER=spradm TOUSER=spradm IGNORE=y STATISTICS=none buffer=99999
  15. Login to SSC2 database using sqlplus by issuing the following command: sqlplus spradm/spr_adm@SSC2and then execute following command:SELECT 'ALTER TABLE ' || TABLE_NAME || ' ENABLE CONSTRAINT ' || CONSTRAINT_NAME || ';' A FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R'; Execute the output of this SQL command to enable foreign keys.
  16. Execute the following SQL command:SELECT CONSTRAINT_NAME, STATUS FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R'; From the output of this SQL command ensure that the status of foreign key constraints is enabled.
  17. Truncate the session tracker table by issuing following command TRUNCATE TABLE SPR_SESSION_TRACKER
  18. Gather SSC2 database statistics by first log-in to the database by issuing following command: sqlplus system/sscsystem@SSC2and then collect the schema statistics by issuing following command:execute dbms_stats.gather_schema_stats('SPRADM',60)
  19. Initiate the IMDB for SSC2.

Migrating a Single Host SSC installation to multi-host

This section describes the procedure to migrate a single host SSC installation to multi-host installation.

Following are prerequisites for migration:
  • You have full filled prerequisites of SSC installation.
  • Blades with SAN set-up are available.
  • Live single host SSC instance is available with disk based SPR database.
  • Both the blades, one with live SSC instance and another with SAN connection are housed in same IBM blade center chassis.
  1. Installing SSC instance on a blade connected with SAN.
  2. From the blade with active SSC instance, exporting SPR database.
  3. On newly installed SAN enabled blade importing the SPR database.
  4. Configuring Sh and other components for SAN based SPR after it has been brought –up with positive results, for the test call in the previous step.
  5. Un-installing SSC on earlier blade of disk based SPR.
  6. Installing SSC on the blade from where it was un-installed.
  7. Completing the migration process.

Installing SSC Instance on SAN Connected Blade

Perform following procedure to install SSC instance on SAN connected blade.

  1. Perform cluster installation.
  2. Provide the IP addresses of both blades i.e the current blade as well as a blade that has active SSC instance with disk based SPR.
  3. Select for the primary database on the current blade.
  4. In ssctemp.properites chose for SAN based installation and HA enabled options.
  5. In createGrid.cfg for the stand-by database give the name of blade with disk based SPR.
  6. Complete the installation

Exporting SPR database

Perform following procedure to export SPR data base.

  1. Access the blade with active SSC instance.
  2. Shut down SSC application.
  3. Export SPR database schema.
  4. Start SSC application.

Importing SPR Database on Newly Installed SAN Enabled Blade.

Perform following procedure to import SPR database on SAN enabled blade.

  1. Upgrade IMDB application to appropriate version.
  2. Start IMDB application.
  3. Upgrade SSC instance to appropriate version.
  4. Edit etc/system.cfg and set value of the parameter HaEnabled to 0.
  5. Start SSC.
  6. Ensure that sample profile is exported correctly.
  7. Bind Sh controller. Ensue it is working properly by trying test call for a sample subscriber profile.
  8. If results of the test mentioned in previous step are positive, then un-bind Sh controller, shut down SSC and in system.cfg change value of the parameter HaEnabled to 1. Start SSC application.
  9. If test results are negative then troubleshoot the issues with SAN based SPR.

Completing Migration

Perform following procedure to complete single host to multi-host migration.

  1. Confirm that grid is attached properly with two entries one for each blade.
  2. Confirm that standby database is replicating from active database.
  3. Ensure that the files /etc/cluster/cluster.conf and etc/host are identical for both blades.
  4. Start SSC application.

RAC Support

This section briefly describes the RAC support along with RAC-HA installation prerequisites and procedures.

This section contains following sub-sections:
  • RAC Overview
  • RAC-HA Installation Prerequisites
  • RAC-HA Primary Node Installation
  • RAC-HA Secondary Node Installation

RAC Overview

This section briefly describes the RAC support in the enhanced architecture.

Enhanced SSC architecture supports Oracle Real Application Cluster (RAC). The RAC allows Oracle data base to run any packaged or custom application un-changed across the server pool. It provides facility to add more servers and instances to the pool without taking the users offline. In the previous SSC architecture, the data base can become single point of failure as well as performance bottle neck as it is installed on a single blade in any site.

With RAC Oracle de-couples the Oracle instance i.e. the processes and memory structures that are running on the server to access the data, from the data files i.e. the physical structure that is actually storing the data. A clustered database can be accessed by multiple instances running on separate servers.

IMPORTANT:

In a non-RAC deployment, when the data base server fails in the first site the fail-over occurs in the second site. In a RAC deployment failure of one database instance fail-over does not occurs in second site, as another database instance form the first site takes over.

RAC –HA Installation Prerequisites

This section briefly describes RAC-HA installation prerequisites.

Following are the RAC-HA installation prerequisites:
  • The Virtual IPs (VIPs) used in the installation should be registered in the DNS and they must be on the same sub-net as the public host network addresses. Each Configured VIP requires an un-used and resolvable IP address.
  • RAC installation needs seven partitions on the sheared disk. Partition the shared disk as described in the following table.

Table 3. RAC-HA Partition Table
Partition Name Partition Type Partition Content (Size of Partition in %)

/dev/sde1

Primary

Data disk, Redo logs, Control file 1 (25%)

/dev/sde2

Primary

Index disk, Redo logs, Control file 2 (15%)

/dev/sde3

Primary

System file disk, Control file 3 (5%)

/dev/sde4

Extended

/dev/sde5

Logical Extended 1

Flashback file disk (10%)

/dev/sde6

Logical Extended 2

Archive file disk (35%)

/dev/sde7

Logical Extended 3

Cluster file disk (5%)

/dev/sde8

Logical Extended 4

Cluster file disk (5%)


Primary Node Installation – RAC HA

This section briefly describes primary node installation procedure for RAC – HA deployment. Refer RAC-HA Prerequisites as well as Before You Begin section of this chapter, before proceeding with RAC-HA set-up.

Installing Primary Node for RAC-HA

To install primary node for RAC-HA:

  1. Follow the procedure mentioned in the section, Installing SSC on a Single Host upto step 7, the Cluster Installation step to Add or Remove the Network Configuration. Ensure that:
    • Number of Blades is 2 as this is a two node setup. In the Choose Blade field, the blade number on which this installation is being performed is selected.
    • Cluster Name is same for all the nodes in a cluster.
    • In the Choose Interface field there are 6 entries viz eth0 upto eth5 for each blade. Refer you IP map and network planning section for choosing the interface and the interface name.
    • If you do not have NIC bonding then you need to configure only et1, et2 and eth3. The Ethernet cards et4 and eth5 can be used for NIC bonding if they are not being used as management interface.
  2. Click Next to configure DatabaseCluster and Disk Information for the RAC-HA set-up. Specify the disk partitioning information refer table tiled RAC-HA Partition from previous section.
  3. Click Nextto specify the Virtual IP Address. Provide Virtual IP address and the interface names for both the blades.
  4. Click Next to specify the Installation Set. Select Application and Primary Database, along with the blade on which you want to install this application. Refer step 9 of the procedure Installing SSC on single host.

    IMPORTANT:

    Installation on multiple nodes is not supported in the current release.

  5. Click Next to specify the Slot Numbers for the blade.

    IMPORTANT:

    If you are not aware of the slot numbers, then specify slot number 1 for blade 1 and slot number 2 for blade 2 respectively.

  6. Click Next to specify the Floating Point Management IP.
    The Floating Point Management IP is used to manage the SSC instance running on multiple blades.
  7. Click Next to specify the Management Information.
    The management information includes:
    • Management user Id
    • Management Password
    • AMM URL
    • Instance Mode - Select the instance mode as Master for Primary Node installation for RAC-HA.
  8. Click Next to specify the Port Details for the TimesTen Active and Standby node.
  9. Click Next to view the Pre-Installation Summary panel. It includes following information:
    • Product Name
    • Install Folder
    • Link Folder
    • Available as well as required disk space in bytes.
  10. Click Next to view the important information. Read the information regarding the interface values. Access the ssctmp.properties file from /localhome/install folder. Set value of INSTALL_SAN_ENSBLED parameter to true.
  11. Click Next whenever the installer prompts and complete the installation.

Secondary Node Installation RAC-HA

Perform the task described in the previous section on the second node. In step 4 while specifying the Installation Set, select only Application for blade 2.

SSC Uninstallation

The section lists the procedure to un-install SSC.

The un-installation wizard removes SSC application along with all database records and deletes the database schema.

IMPORTANT:

If your SSC cluster deployment is using multiple hosts or blades, then you need to run this un-installation wizard on each such host.

Un-installing SSC

Perform following procedure to un-install the SSC:

  1. Log in as a user with root administration privileges.
  2. Ensure that your X windows application such as Xterm or Xming is active and running. If Putty application is being used to access SSC Console, then ensure that the Putty setting Enable X11 forwarding is selected.
  3. Access SSC installation directory by issuing following command 
     cd../localhome/install/uninstall
  4. Execute the script uninstallSSC, by issuing following command 
    ./uninstallSSC.sh
  5. SSC initiates the GUI based un-installer.
  6. Click Next, the un-installer first stops and then removes SSC related database and applications, by executing the clean-up scripts and displaying appropriate messages.
  7. Click Done to exit SSC un-installer.
  8. You can access detailed log of SSC un-installation process in ../var/log/messagesfile.

    IMPORTANT:

    After executing un-installation wizard, you can ensure that the un-installation was complete, by checking that processes related to database, IMDB and SSC application are not active.