> SSC Installation
> This chapter provides information and procedures to install and configure Subscriber Service Controller (SSC) components for the product Intelligent Policy Control Function. (IPCF), a Policy Charging and Control (PCC) solution.
> SSC provides a Graphical User Interface (GUI) based installer that can be used for single host as well as cluster installation. This installer performs installations of necessary database and database related applications such as In Memory Database (IMDB) application, that are required to maintain the database gird for the cluster deployment of SSC.
> It also creates the users and groups with appropriate access privileges that are required to administer the SSC deployment. The installer updates required environment variables as well as provides necessary scripts for post installation configuration and administration tasks. It also provides brief information about net work planning required for installing SSC clusters, along with procedures for providing high availability to a cluster and adding a node to an existing SSC cluster.
> This chapter also provides un-installation information for the SSC deployment.
> This chapter includes following sections:

> • | Before You Begin

> • | Access Installation Files

> • | Single Host Installation

> • | Cluster Installation

> • | SSC Uninstallation

> Before You Begin
> This section includes the information that is required for initiating the installation procedure. It contains following sub-sections:

> • | Pre- installation Checklist

> • | Hardware Requirements

> • | Users and Groups

> • | Environment Variables

> • | Network Configuration For Cluster Installation

> Pre- installation Checklist
> You must verify following requirements before starting the installation procedure.

> • | IBM blades and chassis or Cisco UCS C210M2 is available and properly mounted.

> • | Latest versions of firm-ware and Basic Input Output System (BIOS) are installed on blades.

> • | The Cisco MITG RHEL v5.5, 64- bit operating system customized to run on selected hardware platform is installed on blades.

> • | Hostname is same as the interface name that is used to connect to the database, for e.g. if hostname of the box is datablade1 then “ping datablade1” resolves to the IP address on which IPCF will connect to SSC.

> • | IP map for management, database and application interfaces for SSC clearly defines eth0, eth1, eth2, eth3, eth4 and eth5 interfaces.

> Important: Refer to relevant IBM blade center or Cisco UCS C210M2 documentation for system hardware details.
> Hardware Requirements
> This section lists the hardware requirements for single host as well as cluster installation.
> Following are the hardware requirements:

> • | Processors: Up to 2 x Dual-core Intel Xeon Processor up to 3.0GHz.

> • | Memory: 32GB Fully Buffered DIMM. Minimum: 4GB.

> • | Hard drives: 140GB, minimum 40GB.

> • | Networking: Dual Gigabit Ethernet up to 8 ports. Minimum - 2 ports for single host installation, 4 ports for cluster installation.

> • | Max cache : 4MB L2 shared cache.

> • | Front side bus: 1333 MHz.

> Important: Hardware requirements may vary as per SSC deployment configuration.
> Users and Groups
> This section lists various users and their groups created by the SSC installer. Following table lists their names and descriptions.
> SSC Users and Groups

> Sr.No | User Name | Group Name | Description
> 1 | sscadmin | sscadmin, oinstall,dba | A Linux administrative user for SSC application. Who can administer SSC application, log onto all databases that are associated with this software installation using DBA privileges. Start or stop database services and perform other database related activities.
> 4 | Policy_Operator |  | An SSC application user, who can manage policies.
> 5 | Sys_Operator |  | An SSC application user, who can manage SSC application.
> 6 | Sys_Admin |  | An application user, who can act as a super user for SSC application.

> ‌
> Environment Variables
> This section describes the environment variables required for SSC installation. Following table lists required environment variables along with their description.
> Environment Variables

> Sr.No | Variable Name | Description
> 1 | $SSC_HOME | Stores the path of the directory where SSC is installed by the user.
> 2 | SSC_BIN | Stores the path of the directory where SSC binaries are available.
> 3 | database_HOME | Stores home directory path for database.
> 4 | IMDB_App_HOME | Stores home directory path for the In Memory Database (IMDB) application.

> ‌
> Network Configuration For Cluster Installation
> This section describes cluster installation.
> A blade cluster contains multiple blades. In a cluster installation you need to install SSC on all the blades in a cluster.
> This section contains following sub sections:

> • | Network Configuration

> • | Installation Strategy for a Cluster

> Network Configuration
> This section briefly describes the network configuration required for the SSC deployed in a cluster environment.
> While configuring the network you need to ensure that the database remains available even if one of the blades is not accessible for some time.
> You can create such a highly available clustered database solution by installing SSC components across various blades using network interfaces. You need to configure these interfaces in such a way that they are highly available from the ethernet interface as well as from the cabling prospective.
> Important: Before proceeding with SSC cluster installation you need to define a detailed IP map that co-relates the SSC management and application interfaces as well as primary and standby databases with available IP addresses for entire SSC deployment. During installation you must ensure that eth0 is selected as a management interface.
> Following figure describes a default network configuration for an SSC cluster with two blades.
> Network Configuration for a Cluster
> To provide a highly available clustered database solution, SSC services, In Memory Database (IMDB) and RDBMS use various interfaces across the blades. These interfaces need to be highly available from the Ethernet interface and cabling perspective.
> This can be achieved by constructing three VLANs in high availability configuration mode. During the installation process you need to provide IP address of the host for each interface.
> These three VLANs are:

> • | Traffic VLAN: It carries data traffic for the SSC cluster. Maximum bandwidth should be configured for this VLAN. It serves the Sh, En and SPR API interfaces along with a standby interface for the database.

> • | Management VLAN: It carries management traffic for the SSC cluster. This VLAN provides access to the user interface for the deployed SSC cluster along with the access for an interface to Policy Provisioning Tool (PPT). As well as web access to the console that manages IBM/Cisco chassis.

> • | Private VLAN: It provides inter-process communication between the application nodes as well as carries the private traffic between the database instances.

> Installation Strategy for a Cluster
> This section briefly describes the strategy that can be used to install SSC in a cluster environment.
> To install SSC in a cluster environment:

> • | Ensure that you have configured the network and have a detailed IP map as described in the previous section.

> • | Ensure that all the binaries required for installation are present on all blades in the cluster.

> • | Select the blade or host on which to install primary database.

> • | Create In Memory Database Application (IMDB) grid for primary database.

> • | Select the blade on which to install the secondary database.

> • | Synchronize primary database with secondary database.

> • | Initiate SSC and bind controllers using SSC console.

> Access Installation Files
> SSC installation wizard is available in an archived format as ssc_<version>_rhel_x86_qa1.tar. Create a temporary directory where this wizard can be extracted, by issuing following command:
> mkdir/tmpinst
> Copy the installer in this directory.
> During the installation use tail command to view SSC installation logs being created in/var/log/messages file, to verify the success or failure of commands being executed by the SSC installer. After installation refer to the installation summery captured in Subscriber_Services_Controller_InstallLog.log file which is located in /localhome/install folder.
> Important: For the cluster installation, the installer executable must be available on all the blades where you want to install the SSC.
> SSC Installation logs are stored in system logs /var/log/messages file. If there are any failures in system logs then, fix those errors by referring to Troubleshooting chapter, and re-execute the installation steps using Previous and Next buttons, provided by installer GUI.
> Single Host Installation
> This section describes the procedure to install SSC on a single host.
> Installing SSC on a single host
> Perform following procedure for a single host installation:

> Step 1 | On Linux host login as a user with root administrative privileges.

> Step 2 | Locate the SSC installer tar archive as explained in previous section.

> Step 3 | Extract the archive by issuing following command:

> tar -xvf ssc_<version>_rhel_x86_qa1.tar

> Step 4 | Set the DISPLAY variable by using IP address of the workstation from where this installer is being installed by issuing following command:

> # echo DISPLAY 10.4.2.141:0.0
> Verify that your Xwindows application is active and running, to access the SSC installer GUI.

> Step 5 | Initiate the installation process by issuing following command:

> /. start.sh

> Step 6 | SSC Installer displays the Introduction screen.

> Step 7 | Click Next

> SSC Installer displays License Agreement screen. Select the option I accept terms of license agreement.

> Step 8 | Click Next

> SSC Installer displays Cluster Installation screen. As this installation is for single host. Select Number of Blades as 1. The Choose Blade field displays the available blade. Using Choose Interface field select the interface for this blade.

> Step 9 | Click Add to add this interface

> Add interfaces for database and application, as per your network configuration strategy and IP map.

> Step 10 | Click Next

> SSC Installer displays Installation Set screen. Select whether this is a primary site or a stand -by site. Select whether you want to install Application, Primary Database or Secondary Database on this blade.
> Important: In a multi-host cluster environment, if you are implementing the geo-redundancy, then as per your IP map specify whether you want to use this SSC installation as primary site or as a stand-by site.
> Important: Primary and standby databases cannot be configured on same blade.

> Step 11 | Click Next

> Installer configures database and IMDB instances. After installing selected Installation Set, SSC installer creates database schema and IMDB cache. Ensure that INSTALL_DB in /localhome/install/ssctmp.properties is set to true, if database schema is to be installed on this blade. database_DB_HOST in /localhome/ssc/install/spr_install/createSSC_grid.cfg is set to the host name of box where primary database instance is created. IMDB_* in /localhome/SSC/install/spr_install/cratessc_grid.cfg is set to the hostnames of the boxes that need to be part of the IMDB grid. InstanceName in /localhome/ssc/etc/system.cfg is set to the database instance that is to be configured on this box.

> Step 12 | SSC Installer configures Kernel Parameters, Sets network configuration files, adds users and groups. Time required to configure these parameters depends upon your hardware and the options that you have selected. During this period the Next button will be grayed out. Click Next when that button is available.

> If any errors or suggested changes regarding certain configurations are displayed by the SSC installer, then resolve such errors before proceeding with installation.

> Step 13 | SSC Installer displays Pre- Installation Summary screen.

> Step 14 | Click Install.

> SSC Installer displays Important Information such has important steps that are being performed by the installer. Number of users that are being created. Along with a list of manual steps that needs to be performed to complete the SSC installation.

> Step 15 | Click Next

> SSC Installer displays Installation Complete screen, when SSC application is installed on your system. After installation refer to overall summary of installation steps along with success or failure indicators in /localhome/install/Subscriber_Services_Controller_InstallLog.log
> Cluster Installation
> This section briefly describes how to install SSC cluster, configure high availability in a multi host deployment and add or remove a node from cluster.
> Installing SSC Cluster
> Ensure that you have a cluster plan, before initiating the cluster installation. A plan also helps in maintenance of the cluster when you need to add or remove a node from cluster.
> Refer to the installation procedure documented in the section Installing SSC on a Single Host. On each host or blade, follow this procedure from step 1 up to the cluster installation step.
> While performing the cluster installation, on each blade:

> Step 1 | Select the number of blades to be used in this cluster.

> Step 2 | Choose the blade from the drop-down list.

> Step 3 | Choose the interface that is to be associated with this blade.

> Step 4 | Specify the host name for this blade.

> Step 5 | Specify IP address of this blade.

> Step 6 | Click on Add button to include this blade in the cluster.

> Installer displays, the host name, IP address and the associated interface for all the blades included in the cluster. Follow remaining steps of the installation procedure.
> Adding SSC Node
> This section briefly describes how to add a node in a cluster installation.
> Refer to step 12,the Cluster Installation step in single host installation section. When the installer prompts for data base and IMDB configuration for this node, ensure that:

> • | ORACLE_DB_HOST in /localhome/ssc/install/spr_install/createSSC_grid.cfg is set to the hostname of box where primary database instance is created.

> • | IMDB_DB_* in /localhome/ssc/install/spr_install/createSSC_grid.cfg is set appropriately as per cluster plan

> • | InstanceName in /localhome/ssc/etc/system.cfg is set to db instance configured on database box

> Important: Increment the instance id for every node or blade.
> Configuring High Availability (HA) in Cluster
> Important: For both Cisco UCS and IBM Blade Center, you need to ensure that the hardware is configured for recovery from eth card failure, before attempting the HA configuration.
> In a multi-host SSC deployment, high availability ensures availability of SSC application in case of node, network or power failure. High availability is implemented using RedHat Cluster Suit (RHCS). The RHCS protects SSC components or services in case of network disruption.
> Depending upon your deployment configuration, RHCS can protect following SSC services and related IP addresses in case of network description:

> • | Services that are used for managing SSC deployment such as System controller, Log daemon and Scheduler.

> • | Sh controller.

> • | Event controller.

> • | Profile controller.

> • | Ud controller.

> In a cluster deployment high availability can be configured by setting some parameters values in the file/localhome/install/ssctmp.properties. Ensure that before starting installation of cluster RPMs, the keys RPM-GPG-KEY-redhat-beta and RPM-GPG-KEY-redhat-release are imported.
> Important: For a single host SSC deployment, High Availability (HA) is not supported, however failure detection and recovery mechanisms exist for application processes, database, IMDB and network link for such deployment.
> To configure high availability:

> Step 1 | With root administrative privilege access the file /localhome/install/ssctmp.properties .

> Step 2 | Enable HA option by setting value of INSTALL_HA_ENABLED parameter to 1.

> Step 3 | Set Floating point IPs for Sh Management, Event Notification and Profile Controller interfaces by setting values of INSTALL_EXT_PROFILE_INTERFACE,INSTALL_EXT_EVENT_INTERFACE,INSTALL_EXT_MGMT_INTERFACEand INSTALL_EXT_SH_INTERFACE parameters to floating point IPs.

> Step 4 | Set Management Module details, by setting values of INSTALL_MM_IP,INSTALL_MM_USER and INSTALL_MM_PWD parameters to MM host name or IP address, MM user id and MM password respectively.

> Exporting Database Schema
> This section describes the procedure to export database schema from one SSC instance to another with same build version of the database.
> Exporting Database Schema From One SSC Instance to Another
> This section describes the procedure to export database schema from instance SSC1 on host 1 to another SSC instance SSC2 on host 2.
> To export database schema from SSC1 to SSC2:

> Step 1 | Shut down the IMDB and SSC application instance on host 1. Ensure that there is no activity happening on host1database.

> Step 2 | Log-in to host 1 as database administrative user and create a working directory under /home directory by issuing following command $mkdir dbdump

> Ensure that you have sufficient disk space for the export operation.

> Step 3 | From the home directory execute following command:exp system/sscsystem@SSC1 FILE=scc_dump.dmp log=scc_export_dump.log STATISTICS=none direct=Y owner=spradm

> Step 4 | Verify connectivity of host 2 database from host 1 by issuing following command:$tnsping SSC2

> If you are not able to ping SSC2 database from hosit1, then FTP the dump file on host2 server.

> Step 5 | Shutdown the IMDB and SSC applications on host2

> Step 6 | Ensure that no activity is happening on host 2 database. Execute the clean up script on SSC2 database by issuing following command:./cleanuo_all.sh <SID>

> Step 7 | and then execute the following command: select 'DROP SEQUENCE ' || SEQUENCE_NAME || ';' seq from USER_SEQUENCES

> Step 8 | Execute the output of SQL statement mentioned in the previous step to drop all the sequences.

> Step 9 | Ensure that there are no sequences by executing following SQL command: SELECT COUNT(*) FROM USER_SEQUENCES

> A zero count for this SQL indicates that there are no sequences.

> Step 10 | Execute the output of following SQL statement to drop all the object types: SELECT 'DROP TYPE ' || OBJECT_NAME || ' VALIDATE;' A FROM USER_OBJECTS WHERE OBJECT_TYPE = 'TYPE

> Step 11 | Ensure that there are no object types by executing following SQL command: SELECT COUNT(*) FROM USER_OBJECTS WHERE OBJECT_TYPE = 'TYPE'

> A zero count for this SQL indicates that there are no object types.

> Step 12 | Execute following SQL statement:SELECT 'ALTER TABLE ' || TABLE_NAME || ' DISABLE CONSTRAINT ' || CONSTRAINT_NAME || ';' A FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R'

> Execute the output of this SQL statement to disable foreign keys.

> Step 13 | Ensure that the foreign key constraints are disabled by issuing following SQL statement:SELECT CONSTRAINT_NAME, STATUS FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R'

> Step 14 | import the data on the host2 that is in the SSC2 application instance by issuing following command: imp system/sscsystem@SSC2 FILE=scc_dump.dmp log=scc_import_dump.log FROMUSER=spradm TOUSER=spradm IGNORE=y STATISTICS=none buffer=99999

> Step 15 | Login to SSC2 database using sqlplus by issuing the following command: sqlplus spradm/spr_adm@SSC2and then execute following command:SELECT 'ALTER TABLE ' || TABLE_NAME || ' ENABLE CONSTRAINT ' || CONSTRAINT_NAME || ';' A FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R';

> Execute the output of this SQL command to enable foreign keys.

> Step 16 | Execute the following SQL command:SELECT CONSTRAINT_NAME, STATUS FROM USER_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'R';

> From the output of this SQL command ensure that the status of foreign key constraints is enabled.

> Step 17 | Truncate the session tracker table by issuing following command TRUNCATE TABLE SPR_SESSION_TRACKER

> Step 18 | Gather SSC2 database statistics by first log-in to the database by issuing following command: sqlplus system/sscsystem@SSC2and then collect the schema statistics by issuing following command:execute dbms_stats.gather_schema_stats('SPRADM',60)

> Step 19 | Initiate the IMDB for SSC2.

> Monitoring Status of Standby Database
> This section lists the queries that can be used to monitor the database status.
> Database administration privileges are required to execute following queries:

> • | Viewing Protection Mode: This query displays the level and mode of data protection as well as database role and switch over status for a primary physical standby or snapshot standby database.

> SELECT PROTECTION_MODE, PROTECTION_LEVEL, DATABASE_ROLE ROLE, SWITCHOVER_STATUS FROM V$DATABASE;

> • | Viewing Redo Apply and Transport Status: This query displays redo apply and redo transport status of a physical standby database.

> SELECT PROCESS, STATUS, THREAD#, SEQUENCE#,BLOCK#, BLOCKS FROM V$MANAGED_STANDBY;

> • | Viewing Archive Redo Log Files Information: This query displays information about archived redo log files that have been received by a physical or snapshot standby database from the primary database.

> SELECT THREAD#, SEQUENCE#, FIRST_CHANGE#, NEXT_CHANGE# FROM V$ARCHIVED_LOG;

> • | Viewing Archived Log History: This query displays the archived log history information.

> SELECT THREAD#, SEQUENCE#, FIRST_CHANGE#, NEXT_CHANGE# FROM V$LOG_HISTORY;

> • | Viewing Data Guard Status Messages: This query displays the messages generated due to data guard events, these messages are stored in alter log or server process trace file

> SELECT MESSAGE FROM V$DATAGUARD_STATUS;

> • | Viewing Status of Redo Transport Destination: This query displays the status of each redo transport destination, for the destinations that are standby databases.

> SELECT DEST_ID, APPLIED_SCN FROM V$ARCHIVE_DEST WHERE TARGET='STANDBY';
> Migrating Single Host Installation to Multi-Host Installation
> This section describes the procedure to migrate single host installation to multi- host installation.
> Following are prerequisites for migration:

> • | You have full filled prerequisites of SSC installation.

> • | Blades with SAN set-up are available.

> • | Live single host SSC instance is available with disk based SPR database.

> • | Both the blades, one with live SSC instance and another with SAN connection are housed in same IBM blade center chassis.

> Migrating a Single Host SSC installation to multi-host
> Single host to multi-host SSC migration procedure involves following tasks:

> Step 1 | Installing SSC instance on a blade connected with SAN.

> Step 2 | From the blade with active SSC instance, exporting SPR database.

> Step 3 | On newly installed SAN enabled blade importing the SPR database.

> Step 4 | Configuring Sh and other components for SAN based SPR after it has been brought –up with positive results, for the test call in the previous step.

> Step 5 | Un-installing SSC on earlier blade of disk based SPR.

> Step 6 | Installing SSC on the blade from where it was un-installed.

> Step 7 | Completing the migration process.

> Installing SSC Instance on SAN Connected Blade
> Perform following procedure to install SSC instance on SAN connected blade.

> Step 1 | Perform cluster installation.

> Step 2 | Provide the IP addresses of both blades i.e the current blade as well as a blade that has active SSC instance with disk based SPR.

> Step 3 | Select for the primary database on the current blade.

> Step 4 | In ssctemp.properites chose for SAN based installation and HA enabled options.

> Step 5 | In createGrid.cfg for the stand-by database give the name of blade with disk based SPR.

> Step 6 | Complete the installation

> Exporting SPR database
> Perform following procedure to export SPR data base.

> Step 1 | Access the blade with active SSC instance.

> Step 2 | Shut down SSC application.

> Step 3 | Export SPR database schema.

> Step 4 | Start SSC application.

> Importing SPR Database on Newly Installed SAN Enabled Blade.
> Perform following procedure to import SPR database on SAN enabled blade..

> Step 1 | Upgrade IMDB application to appropriate version.

> Step 2 | Start IMDB application.

> Step 3 | Upgrade SSC instance to appropriate version.

> Step 4 | Edit etc/system.cfg and set value of the parameter HaEnabled to 0.

> Step 5 | Start SSC.

> Step 6 | Ensure that sample profile is exported correctly.

> Step 7 | Bind Sh controller. Ensue it is working properly by trying test call for a sample subscriber profile.

> Step 8 | If results of the test mentioned in previous step are positive, then un-bind Sh controller, shut down SSC and in system.cfg change value of the parameter HaEnabled to 1. Start SSC application.

> Step 9 | If test results are negative then troubleshoot the issues with SAN based SPR.

> Completing Migration
> Perform following procedure to complete single host to multi-host migration.

> Step 1 | Confirm that grid is attached properly with two entries one for each blade.

> Step 2 | Confirm that standby database is replicating from active database.

> Step 3 | Ensure that the files /etc/cluster/cluster.conf and etc/host are identical for both blades.

> Step 4 | Start SSC application.

> SSC Uninstallation
> The section lists the procedure to un-install SSC.
> The un-installation wizard removes SSC application along with all database records and deletes the database schema.
> Important: If your SSC cluster deployment is using multiple hosts or blades, then you need to run this un-installation wizard on each such host.
> Un-installing SSC
> Perform following procedure to un-install the SSC:

> Step 1 | Log in as a user with root administration privileges.

> Step 2 | Ensure that your X windows application such as xterm or xming is active and running.

> Step 3 | Access SSC installation directory by issuing following command

> cd../localhome/install/uninstall

> Step 4 | Execute the script uninstallSSC, by issuing following command

> ./uninstallSSC.sh

> Step 5 | SSC initiates the GUI based un-installer.

> Step 6 | Click Next, the un-installer first stops and then removes SSC related database and applications , by executing the clean-up scripts and displaying appropriate messages.

> Step 7 | Click Done to exit SSC un-installer.

> Step 8 | You can access detailed log of SSC un-installation process in ../var/log/messagesfile.

> Important: After executing un-installation wizard, you can ensure that the un-installation was complete, by checking that processes related to database, IMDB and SSC application are not active.

Cisco Systems Inc.
Tel: 408-526-4000
Fax: 408-527-0883