Upgrading and Rolling Back Prime Network
This section covers tasks on how to upgrade from Prime Network 4.2.2, 4.2.3, 4.3, 4.3,1, 4.3.2, or 5.0 to 5.1 or roll back from Prime Network 5.1 to 5,0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2. If you want to upgrade from an earlier version of Prime Network, you must first upgrade to Prime Network 4.0 and then you can upgrade to Prime Network 5.1.
To upgrade 4.0 from earlier versions of Prime Network, refer Prime Network 4.0 DVD contents. For the upgrade procedure, see Cisco Prime Network 4.0 Installation Guide.
This section contains the following topics:
– In-line RHEL Upgrade of 6.7, 6.8, 6.9 and 7.4 with Prime Network 5.1 and Oracle 12.1.0.2
– If you have Unit server attached with Gateway, first upgrade the Gateway as mentioned in the above steps.
– Upgrading to Prime Network 5.1 in Suite Mode
Prime Network Upgrade Overview
The upgrade procedure backs up the existing user directory and then adds any new Prime Network 5.1 libraries, files, and code to the existing installation. Any changes to the database are made automatically as part of the upgrade. The majority of your customizations and user-defined information remain intact and available after upgrading. A list of what is migrated is provided in Table 10-1.
If Operations Reports is installed, it will be upgraded automatically during the upgrade process.
The amount of time required to upgrade Prime Network depends on your deployment size and system performance. During upgrade, the system will be down. Contact your Cisco account representative for an estimated upgrade duration.
Table 10-1 shows the components affected by the Prime Network upgrade and whether those components are upgraded automatically. If they are not updated automatically, the manual procedure you must perform is provided.
Table 10-1 Components Affected by the Prime Network Upgrade
|
|
|
|
VNE AVMs |
avm*.xml files with managed element definitions |
Yes |
— |
Third-party VNE support |
Support for non-Cisco VNEs |
No |
Prime Network supports third-party devices through Cisco Advanced Services engagement. As of release 5.1, Prime Network will not natively support third-party devices, and a Cisco Advanced Services contract will be required for their enablement and support. |
Database schema changes |
Add, change, or remove database schema tables to meet the Cisco Prime Network 5.1 schema definition |
Yes |
— |
Database data preservation |
Migrates the old data representation to the Cisco Prime Network 5.1 representation, where applicable |
Yes |
All tickets and events are available after upgrading. All other data (such as maps, users, and so on) are preserved and migrated. |
Database (general) |
— |
No |
You must retain the same database type after migration. In other words, you cannot upgrade from:
- A database located on the gateway server to a database located on a remote server (and vice versa)
- A customer-provided database to an embedded database.
|
Users and scopes |
— |
Yes |
All users and scopes are maintained. |
Northbound API trap forwarding and SNMP |
Out-of-box support for the SNMP trap forwarding mechanism |
Yes |
The Cisco-EPM-NOTIFICATION-MIB structure includes a running index in the object identifier (OID) suffix, instead of a constant number as in previous releases. For more information, contact Cisco Advanced Services. |
Northbound API: IMO and BQL |
Changes made to information model objects (IMOs) |
Yes |
IMOs might change between versions to support new features. For more information, contact Cisco Advanced services. |
Customizations: Business objects |
— |
Yes |
Review IMO changes to verify that the OID associated with the business object did not change. |
Customizations: Soft properties |
Soft properties remain backward compatible and are available in Prime Network 5.1 after upgrading. |
Yes |
— |
Customizations: Command Builder |
User-defined commands |
Yes |
— |
Built-in Command Builder scripts |
Prime Network built-in activation scripts |
Yes |
The upgrade procedure updates the built-in changes and removes scripts that are no longer part of the product. See Prime Network Post-upgrade Tasks to understand which commands require installation after the upgrade. |
Customizations: Drools rules |
— |
Yes |
The Post.drl rule is available after upgrading. |
Customizations: crontab files |
Prime Network crontabs are configured as part of the installation |
Yes, if in proper location |
If you have user-defined cron jobs, place them in PRIME_NETWORK_HOME /local/cron/crontab.user.list. The upgrade will automatically add the user-defined cron jobs. User-defined cron jobs that are not placed in this directory will be removed. See Prime Network Post-upgrade Tasks. |
Customizations: External launch points |
External launch configuration |
Yes |
Review IMO changes to verify that the OID associated with the launch command did not change. |
Customizations: Message of the Day |
Message of the Day configuration |
Yes |
|
Registry |
— |
Yes |
New Prime Network 5.1 registry files are available automatically after the upgrade. Customizable registry files, including avm+.xml and site*, are available and upgraded automatically. Review any customized registry configurations in site.xml and avm*.xml to understand whether they are relevant to Prime Network 5.1. Contact your Cisco account representative, if necessary. |
pnuser _admin user |
User with database administrator permissions who can run maintenance tasks—such as gathering statistics—on the other Prime Network database schemas. |
Yes |
— |
Security: SSH and SSL keys |
Prime Network SSL keystore and truststore keys, SSH keys, and registry encryption keys |
Yes |
Prime Network SSL keystore and truststore keys are maintained. These keys are used by all SSL sockets, including BQL and PTP clients. Prime Network SSH keys and registry encryption keys are also maintained. |
Prime Network persistency files |
Inventory, events, and link persistency data |
Yes |
All persistency files are available after the upgrade. |
Standby units |
— |
Yes |
Standby units complete their upgrade when they are restarted by the gateway (when an active unit goes down and the standby unit is brought online). |
GUI client |
— |
No |
If you had an installed client, you need to reinstall it after upgrade. If you access the clients via Web Start, no action is required. |
Network Service Activation (NSA) |
— |
No |
Cisco Prime Network Activation functionality is no longer available in Prime Network 5.1. Transaction Manager replaces the Prime Network Workflow and Activation features that were available in previous releases. For details on setting up Transaction Manager, see Setting Up Transaction Manager. For information on how to use Transaction Manager, see the Cisco Prime Network 5.1 Customization Guide. |
Change and Configuration Management |
Software image and device configuration files |
Yes |
All the software and device configuration changes are retained as part of the upgrade. |
High availability configuration |
Upgrades for RHCS/Oracle Active Data Guard gateway high availability |
No |
If you have gateway high availability, move the Prime Network and Oracle services to maintenance mode before you run the upgrade, then move them back to normal mode after it. |
Operations Reports |
User-defined reports |
Yes |
All user defined reports created prior to the upgrade will be available post-upgrade. |
Preparing to Upgrade Prime Network (Pre-Upgrade Checklist)
Table 10-2 shows the pre-upgrade tasks that must be performed before upgrading to Prime Network 5.1.
Table 10-2 Gateway Pre-Upgrade Tasks
|
|
Referred Topic/Action Required
|
Step 1 |
If you are managing third-party devices, make note of them. You will need to give this information to your Cisco representative to enable the support after the upgrade. |
Prime Network supports third-party devices through Cisco Advanced Services engagement. As of release 5.1, Prime Network will not natively support third-party devices, and a Cisco Advanced Services contract will be required for their enablement and support. |
Step 2 |
Insert Disk 3: Upgrade Files into the DVD drive. |
Copy these files from the DVD to the temporary upgrade directory you created:
- ivne-drivers.tar file
- Prime_Network_upgrade directory and its dependent contents
|
Step 3 |
Familiarize yourself with the upgrade process and identify areas that may require manual changes. |
Components affected by upgrade are listed in Table 10-1 . |
Step 4 |
Back up your database and files stored on the gateway. |
External database:
- Back up your gateway data by logging into the gateway and running this command from PRIME_NETWORK_HOME/Main/scripts:
- Back up the Oracle database using your Oracle documentation.
Embedded database: a. Log in to the gateway as pnuser. b. Change to the embedded database directory:
# cd $PRIME_NETWORK_HOME/Main/scripts/embedded_db
c. Execute the backup script:
For information on emdbctl utility used in the above procedure, refer to the Cisco Prime Network 5.1 Administrator Guide.
Note If you would like to rollback embedded database later, please ensure that the procedure described in database rollback Prerequisites is done.
|
Step 5 |
Apply the database configurations and recommendations. |
Preparing the Oracle External Database |
Step 6 |
Verify that the server machines comply with the system hardware and software requirements. |
Installation Requirements Gateway: CPU and Memory Requirements for Different Network Sizes |
Step 7 |
Verify that the backup directory has at least 6000 MB of free space for pnuser. |
Example: df –k /backup_dir |
Step 8 |
Verify that the database has at least 8 GB of RAM available (the minimum requirement). |
For the database storage sizing guidelines, contact your Cisco account representative. |
Step 9 |
Verify that all required ports are free. |
Required Ports for Prime Network. |
Step 10 |
Make sure all database sessions (such as TOAD, SQL, and so on) are closed. |
Other TOAD/SQL sessions apart from Prime Network established session should be closed. |
Step 11 |
Place any customized crontab files in PRIME_NETWORK_HOME /local/cron/crontab.user.list. User-defined cron jobs that are not placed in this directory will be removed. |
— |
Step 12 |
(External database only) Restart Prime Network and the Oracle database. |
1. As pnuser, stop Prime Network:
2. As oracle user, stop and restart Oracle:
3. As pnuser, restart Prime Network:
|
Step 13 |
Verify that the gateway and units are powered up and connected by opening an SSH session between gateway and all units. |
__ |
Step 14 |
Verify that Oracle and the Oracle listener are running. |
Starting the Oracle Listener (External Database) |
Step 15 |
Drop the TMP_BIG_TICKET2 table if it is already created. |
Prior to Prime Network 5.1 upgrade, run the below query in Data base (DB): 1. Log in to the Prime Network DB and do the following: a. As pnuser, execute sqlplus <PN Username>/<PN User Password>@"[<Gateway IP>]:1521/<SID>"
Example: sqlplus pn43/Admin123#@"[10.76.80.19]:1521/mcdb"
EXECUTE IMMEDIATE 'DROP TABLE TMP_BIG_TICKET2';
|
Step 16 |
(Only for NAT units) Stop the Prime Network application and remove the current crontab. |
Enter the following commands on each of the NAT units: networkctl stop;
Note To restart the crontab later, see Restarting Crontab Jobs for NAT Units. |
Step 17 |
(Local and geographic gateway high availability) Verify that the gateways and units with Red Hat installed have rsync 3.0.6 or newer. |
Verify the rsync version installed on the gateway/units using the command: [root@primebgl01-lnx ~]# rpm -qa rsync rsync-3.0.6-9.el6_4.1.x86_64 [root@primebgl01-lnx ~]# |
Step 18 |
If using an external database, verify your database settings. Note Prime Network 5.1 requires the Oracle JVM and partitioning options. |
See Chapter 4, “Preparing the Oracle External Database” |
Supported Prime Network Upgrade and Rolling back versions
Refer the following table for supported Prime Network Upgrade and rolling back versions.
Table 10-3 Supported Prime Network Upgrade and Rolling back versions
Upgrading to Prime Network 5.1 from 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 (Intermediate Steps)
Note The steps provided below are intermediate steps that are to be followed while Upgrading to Prime Network 5.1 and Oracle 12.1.0.2.
During the upgrade of Prime Network 5.1, it is highly recommended to use Oracle 12.1.0.2 version. In case, if you want to use 12.1.0.1 Oracle version, then upgrade only Prime Network, do not upgrade the Oracle.
If you are running Prime Network with RHEL 5.8 before upgrading Prime Network 5.1, upgrade the RHEL version to RHEL 6.x or 7.x version based on your current Prime Network RHEL version support.
Use the procedure described in this section to upgrade from Prime Network 5.0, 4,3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2, to Prime Network 5.1.
Caution
Do
not apply any service patches during any phase of the upgrade to Prime Network 5.1. Apply them after the upgrade is completed.
Before You Begin
Note While upgrading Prime Network in a HA setup, you should always start the upgrade from the Primary gateway as active gateway. The active gateway should not be the secondary gateway when starting the upgrade process.
To upgrade the Prime Network gateway:
Step 1 Create a temporary upgrade directory on the gateway.
Note Make sure that upgrade directory is not a subdirectory of $PRIME_NETWORK_HOME (which is /export/home/pnuser by default).
Step 2 Insert Disk 3: Upgrade Files into the DVD drive.
Step 3 Copy these files from the DVD to the temporary upgrade directory you created:
- ivne-drivers.tar file
- Prime_Network_upgrade directory and its dependent contents
Step 4 Assign pnuser:pngroup owner permissions to the Prime_Network_upgrade directory and its contents:
chown –R pnuser:pngroup Prime_Network_upgrade
Step 5 To verify the group name, run the following command as pnuser: id --group --name
Step 6 As pnuser, move to the following location in your temporary upgrade directory:
Step 7 If you have not upgraded from fresh install of Prime Network 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 to Prime Network 5.1, as PN user, run status command to check if Compliance Manager is UP, if not, run:
cmctl start
For example, pn51@imeir-pn50-01 [~]% cmctl status
Step 8 Start the upgrade:
Note Compliance server should be up and running for performing the upgrading process.
Note While exporting custom policies, if you are prompted with the following message,
Export failed, Do you want to continue (YES/NO),
then you can follow the below conditions based on your requirements:
Choose NO to stop the upgrading process and exit, or YES to continue.
When you choose YES, the following message appears:
Warning ! All the custom policies has been wiped out, Do you want to continue (YES/NO).
Choose NO to stop the upgrading process and exit, or YES to continue the upgrade process.
Step 9 Enter the required information as shown in the following table.
|
|
|
Password for OS root user |
Operating system root password |
Linux root password In a high availability environment, you will be required to enter the OS root user for each machine in the setup. |
Verifying whether you have completed database backup |
yes |
This prompt is to check whether you have recently completed database backup. Default is yes. If you enter no, the upgrade process will stop and will ask you to back up the database. For information on backing up your database, see Step 4 in the pre-upgrade checklist. |
Destination location for backing up the existing installation tar file |
directory |
Specify a directory with at least 6000 MB of free space. Verify that the backup directory is available for pnuser. The backup directory needs write permission. Enter the following command to add write permission to the backup directory: chmod 777 <directory> |
Disabling Configuration Audit |
yes |
Configuration Audit is deprecated and replaced by Compliance Audit. If you still want to use Configuration Audit, enter no and it will remain available from Change and Configuration Management. |
Path to the ivne-drivers.tar file |
full pathname |
Provide the full pathname to the temporary upgrade location from Step 1. |
Prime Network root password |
root password |
The root password used to log into the Prime Network GUI applications. |
Step 10 After the upgrade is complete, Prime Network restarts. Log in as pnuser for the environment changes to take effect.
Note While importing the custom policies, if the number of custom policies exported is zero, then the importing process is skipped with a message No Custom Policies to import. If the custom policies exported is not zero and if the compliance server is up, then the importing process begins. If the compliance server is not up within 30 seconds, the following message is prompted to the user:
Failed : Run <PN_Home>/utils/independent/compliance/bin/importPolicies.sh manually
Step 11 If any of the preceding steps fail, the following error message is shown:
Failed to execute hook-type for hook-name. See log for further details.
- Hook hook-name terminated with failure
- Please choose one of the following:
1. Abort the upgrade process
In the error message, hook-type and hook-name are the type and name of the procedure that failed.
a. Check the upgrade log ( PRIME_NETWORK_HOME /Main/upgrade- timestamp.log) to identify the reason for the failure.
b. If you can identify the problem and fix it manually, do so; then, choose option 2 to rerun the hook. The upgrade procedure continues from the procedure that failed.
c. If you cannot fix the problem, choose option 1 to cancel the upgrade. After canceling the upgrade, Prime Network cannot be started. Contact your Cisco account representative to fix the problem; then, rerun the upgrade. The upgrade procedure continues from the procedure that failed.
Note If you decide not to rerun the upgrade, you must roll back to your base Prime Network environment, including rolling back the database. See Rolling Back to Earlier Prime Network Version.
Step 12 If you upgraded a gateway configured with local high availability, take the ana and oracle_db services out of maintenance mode:
Step 13 Clear the web browser cache.
Step 14 Perform the necessary tasks listed in Prime Network Post-upgrade Tasks.
Note To remove previous device package reference errors in avm file: 11.out, execute the following command as a Prime Network user: networkctl restart –avm 11.
After the upgrade of Prime Network, you can implement SNMPv2 protocol on all devices. Follow the below steps to run the script:
1. Navigate to ‘<PRIME_NETWORK_HOME>/local/scripts’ directory in the GateWay (GW).
2. Run the script as pnuser from the GW.
Note To run script, Prime Network should be down in both GW and Units. This condition is applicable for all VNEs and not applicable to a specific VNE.
3. Run the script file using the command below:
perl updateSNMPver.pl <currentSnmpver> <newSnmpVer>
Ex1: perl updateSNMPver.pl v2 v2only
Ex2: perl updateSNMPver.pl v1 v2only
Note Valid SnmpVer parameters for the above script are v1,v2,v2 only and the parameters are case sensitive. The log files are available in the following path; ‘<PRIME_NETWORK_HOME>/SnmpVerchange_logs’ in the GW.
4. Start the Prime Network to reflect the changes.
Upgrading to Prime Network 5.1 and Oracle 12.1.0.2
In-line RHEL Upgrade of 6.7, 6.8, 6.9 and 7.4 with Prime Network 5.1 and Oracle 12.1.0.2
To perform the in-line upgrade from RHEL 6.x to 6.7, 6.8, 6.9, and from 7.2 to 7.4 with Prime Network 5.1 and Oracle 12.1.0.2, follow the procedure provided below:
Step 1 Upgrade to Prime Network 5.1 using Prime_Network_upgrade directory from Disk 3 to the temporary upgrade directory you created. See Upgrading to Prime Network 5.1 from 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 (Intermediate Steps).
Step 2 Upgrade embedded Oracle 12 using the embedded_upgrade_12.1.zip file from Disk 3. See Upgrading the Embedded Database to Oracle 12.1.0.2.
Step 3 To perform the in-line upgrade from 6.x to 6.7, 6.8, 6.9,and from 7.2 to 7.4 with latest Open ssl package, contact your System Admin for RHEL in-line upgrade.
Step 4 After upgrading the RHEL, login with pnuser and verify the web server status and the compliance engine status.
Step 5 Login as pnuser and restart AVM11 using the following command:
Note If you have Unit server attached with Gateway, first upgrade the Gateway as mentioned in the above steps.
To upgrade the RHEL, follow the below steps:
Step 1 Note down the pnuser name and Password, and Oracle username and Database profile that you had selected while installing Prime Network lower version.
Step 2 Upgrade to Prime Network 5.1 from Prime Network lower version using Prime_Network_upgrade directory from Disk 3. See Upgrading to Prime Network 5.1 from 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 (Intermediate Steps).
Step 3 Upgrade embedded Oracle 12 using the embedded_upgrade_12.1.zip file. See Upgrading the Embedded Database to Oracle 12.1.0.2.
Step 4 Login as pnuser and backup the embedded oracle database and Prime Network data using the commands below:
cd $Prime_Network_Home/Main/scripts/embedded_db
cd $Prime_Network_Home/Main/scripts
run: backup.pl backup-folder
Note If you have operations reports in Gateway, uninstall it before performing PN Database backup.
Step 5 Copy the latest backup folder in $ Prime_Network_Home/backup to your local server (for example, other than the server you are currently using).
Step 6 Re-image the Gateway server to RHEL 6.7 or 6.8 or RHEL 7.2 or RHEL 7.4. If you have a Unit server attached in the Gateway, re-image the Unit server to RHEL 6.7 or 6.8, or 7.2 or 7.4.
Note After you complete the re-imaging of the Gateway server to RHEL 6.x or 7.4, ensure to restore the backup data.
Step 7 Install the Prime Network 5.1 Gateway, Oracle 12.1.0.2 and the Unit server. If you have unit Gateway setup in PN lower version, use the pnuser name and Password, and Oracle username and Database profile that you had chosen while installing PN Gateway lower version.
Note If you have installed the embedded Oracle in remote server for Prime Network lower version, install embedded database 12.1.0.2 on the same server for Prime Network 5.1.
Step 8 Once installation is complete, login as a Prime user, back up the Prime network Gateway data and embedded database $ PRIME_NETWORK_HOME/Main/scripts/embedded_db # emdbctl --backup . Please refer Prime Network 5.1 Administrator guide to know more on how to back up the Gateway data and the embedded database.
Step 9 Navigate to $ANA_HOME/backup location, and remove the back up file folder in the location.
Step 10 Paste the backup file folder which you already have in your local machine to the location $ANA_HOME/backup.
Step 11 Provide the group owner permissions to the backup file directory and its contents as follows:
chown - R pnuser: pngroup.
Example: chown -R pn51:ana
Step 12 Login as pnuser and restore the embedded oracle database and Prime Network data using the commands below:
cd $Prime_Network_Home/Main/scripts/embedded_db
cd $Prime_Network_Home/Main/scripts
run: restore.pl restore-folder
Step 13 Once the restoring process is completed, check the status of Prime Network using the command:
Step 14 Ensure that the status of both compliance engine and web server is up using the commands:
Step 15 Start the Unit server as a pnuser using the command:
Upgrading to Prime Network 5.1 in Suite Mode
To upgrade to Prime Network 5.1 in suite mode, follow the procedure provided below:
Step 1 Follow the upgrade procedures described below:
Upgrading to Prime Network 5.1 from 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 (Intermediate Steps)
In-line RHEL Upgrade of 6.7, 6.8, 6.9 and 7.4 with Prime Network 5.1 and Oracle 12.1.0.2
If you have Unit server attached with Gateway, first upgrade the Gateway as mentioned in the above steps.
Step 2 Integrate Prime Network in suite mode with Prime Central 2.1. Refer to the Integrating Prime Network with Prime Central topic of the Cisco Prime Central Quick Start Guide.
Step 3 Upgrade to Prime Network Integration Layer 2.1.0 from PN-IL earlier release. Refer to the Appendix 10, “Upgrading PN-IL in Standalone Mode” e topic of the Cisco Prime Network 5.1 Installation Guide
Step 4 Integrate Prime Network Integration Layer 2.1.0 in suite mode with Prime Central 2.1. Refer to the Integrating the Prime Network Integration Layer with Prime Central topic of the Cisco Prime Central Quick Start Guide.
Upgrading or Downgrading OS in HA Environment
You can upgrade or downgrade RHEL version on the local cluster and install HA on all VMs. For example, you can install VM1 and VM2 in a local cluster and VM3 as Geo/DR in a Local with Geographical setup or Install VM1 in a local cluster and VM3 as Geo/DR in a Geo only setup. VM1 is considered as Local or Primary VM, VM2 as secondary local cluster VM where both PN and oracle services not running, and VM3 as standby and distant Geo/DR.
Upgrade of OS in HA Environment
To perform the upgrade, follow the steps:
Step 1 Install HA on a Local cluster VM with Geographical setup or Geographical only setup that has RHEL5.8 or 6.x on all VMs.
Step 2 Shutdown the Primary VM (VM1) in case of both Local+HA local clusters without loss of generality.
Step 3 Execute the following script on the StandBy VM (VM3):
Note After execution, VM3 will be your new Primary, and either VM1 or VM2 will be your new Geo/DR.
Step 4 Upgrade the RHEL from 5.8 to 6.x or 7.4 on the local cluster.
-or-
Upgrade the RHEL from 6.x to 7.4 on the local cluster.
Complete the below steps:
a. Take the backup of Prime Network, re-image the server with RHEL 6.x or RHEL7.4, install the Prime Network 5.1, and then restore the Prime Network backup.
b. In case, if you are re-imaging the server with RHEL 7.4, ensure to configure the cluster with Pacemaker. For more information, see Configuring Clusters for Pacemaker and Corosync Setup.
Note In Prime Network 5.1, in-line upgrade is supported from RHEL 6.8 to RHEL 6.9. For support on new RHEL 6.9 installation with Prime Network 5.1, contact the account manager and the Advance Services representative.
Step 5 Setup VM cluster (VM1 or VM2) for HA installation as shown below:
a. Create /etc/hosts file
b. Set permissions for both /tmp and /etc/shadow
c. Mount build locations
d. Mount again various 4 disk partitions without loss of generality on the primary VM as shown below:
– mount/dev/sdb1/export1/ana-home/ana
– mount/dev/sdb2/ora/opt/ora1
– mount/dev/sdb3/directio
– mount/dev/sdb4/datafiles/dbf
Step 6 Log in to the Primary VM (VM1) without loss of generality, and then navigate to /tmp path to unzip RH_ha.zip.
Note Your new Geo/DR VM will be the new DR.
Step 7 Navigate to /tmp/RH_ha path and then execute the following script on VM1:
#"perl resumeFromFailOver.pl –- reinstall_setup" from /tmp/RH_ha on the primary VM
Note When the script fails, do the following:
a. Add OVERRIDE_SWAP=true to the file /tmp/RH_ha/auto_install_RH.ini
b. Execute perl install_Prime_HA.pl-autoconf auto_install_RH.ini
Step 8 Execute perl resumeFromFailOver.pl --reconfigure_setup also on the primary VM1.
Step 9 Log in to standby VM (VM3) and navigate to /tmp/RH_ha path.
Step 10 Execute “perl resumeFromFailOver.pl --setup_replication” on the standby VM (VM3).
Step 11 To upgrade OS on your new primary VM(VM3) to RHEL 6.5 or 6.7 or 6.8 or 7.4, repeat steps 2 through 10.
a. Shutdown VM3 and execute perl primeha –fail script on Local VM (VM1).
b. Upgrade OS on VM3 to 6.7 or 6.8 or 6.9 or 7.4.
Note In Prime Network 5.1, in-line upgrade is supported from RHEL 6.8 to RHEL 6.9. For support on new RHEL 6.9 installation with Prime Network 5.1, contact the account manager and the Advance Services representative.
c. Setup VM3 to install HA If VM3.
d. Execute the scripts perl resumeFromFailOver.pl –-reinstall_setup and perl resumeFromFailOver.pl --reconfigure_setup on VM3.
e. Execute perl resumeFromFailOver.pl --setup_replication on VM1.
Downgrade OS in HA Environment
To perform the downgrade follow the steps:
Step 1 Install HA on a Local cluster VM with Geographical setup or Geographical only setup that has RHEL 6.5, 6.7, 6.8 or 7.4 on all VMs.
Step 2 Shutdown the Primary VM without loss of generality in case of both Local +HA clusters.
Step 3 Execute the following script on the StandBy VM (VM3):
Note After execution, VM3 will be your new Primary, and either VM1 or VM2 will be your new Geo/DR.
Step 4 Downgrade the RHEL from 7.4 to 6.x on the local cluster.
Note Downgrade to RHEL 5.8 is not supported.
Step 5 Setup VM cluster for the HA installation as shown below:
a. Create /etc/hosts file
b. Set permissions for both /tmp and /etc/shadow
c. Mount build locations
d. Mount again various 4 disk partitions without loss of generality on the primary VM as shown below:
– mount/dev/sdb1/export1/ana-home/ana
– mount/dev/sdb2/ora/opt/ora1
– mount/dev/sdb3/directio
– mount/dev/sdb4/datafiles/dbf
Step 6 Login to the Primary VM without loss of generality, and then navigate to /tmp path to unzip RH_ha.zip.
Note Your new Geo/DR VM will be the new DR.
Step 7 Navigate to /tmp/RH_ha path and then execute the following script:
#"perl resumeFromFailOver.pl –- reinstall_setup" from /tmp/RH_ha on the primary VM
Note When the script fails, do the following:
a. Add OVERRIDE_SWAP=true to the file /tmp/RH_ha/rf_auto_install_RH.ini
b. Execute perl install_Prime_HA.pl-autoconf rf_auto_install_RH.in
Step 8 Execute perl resumeFromFailOver.pl --reconfigure_setup also on the primary VM.
Step 9 Login to standby VM and navigate to /tmp/RH_ha path.
Step 10 Execute perl resumeFromFailOver.pl--setup_replication on the standby VM.
Step 11 To downgrade OS on your new primary VM to RHEL6.x, repeat steps 2 through 10.
a. Shutdown VM3 and execute perl primeha –fail script on Local VM (VM1)
b. Downgrade OS on VM3 to RHEL 6.x
c. Setup VM3 to install HA
d. Execute the scripts perl resumeFromFailOver.pl –-reinstall_setup and perl resumeFromFailOver.pl --reconfigure_setup on VM3
e. Execute perl resumeFromFailOver.pl --setup_replication on VM1.
Configuring Clusters for Pacemaker and Corosync Setup
From RHEL7.2 onwards Pacemaker is the default cluster resource manager. Pacemaker provides maximum availability for your cluster services and resources by detecting and recovering node and resource-level failures. It uses messaging and membership capabilities provided by Corosync to keep the resource availability on any of the cluster nodes
Corosync is the open source cluster engine that manages cluster interconnect and maintains the same cluster configuration across all the cluster nodes. All the configuration changes will be replicated to other node using corosync cluster engine. Pacemaker and Corosync are powerful open source technologies that completely replaces the CMAN and RGManager technologies from previous Redhat cluster releases.
Use the following procedure to configure clusters:
Step 1 Install the required packages for Pacemaker on both nodes.
yum-config-manager --enable rhel-7-server-optional-rpms
yum install pcs fence-agents-all
yum install lvm2-cluster gfs2-utils
yum install nfs-utils rpcbind
yum install pacemaker pcs
Step 2 (Optional) Verify Pacemaker /Corosync /pcs is available or not-a | grep pacemaker.
rpm -q -a | grep corosync
Step 3 Stop the firewalld services and Network Manager on both nodes.
systemctl disable firewalld.service
systemctl stop firewalld.service
systemctl stop NetworkManager.service
Step 4 Disable selinux on both nodes.
update SELINUX=enforcing permissive
[root@pn51-qa-ha-01 /]# setenforce 0
[root@pn51-qa-ha-01 /]# getenforce
Configure SELINUX=disabled in the /etc/selinux/config file:
Step 5 (Optional) Verify Hostnames on both nodes.
[root@pn51-qa2-ha-01 /]# cat /etc/hostname
[root@pn51-qa2-ha-02 /]# cat /etc/hostname
Step 6 (Optional) Verify system configurations.
cat /etc/sysconfig/network-scripts/ifcfg-ens192
Step 7 Add Hostnames, IP addresses and Virtual IPs on both nodes (Vi /etc/hosts)
[root@pn51-qa2-ha-01 /]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.56.57.204 pn51-qa2-ha-01 pn51-qa2-ha-01.cisco.com
10.56.57.205 pn51-qa2-ha-02 pn51-qa2-ha-02.cisco.com
10.56.57.212 ana-cluster-ana
10.56.57.211 ana-cluster-oracle
[root@pn51-qa2-ha-02 /]# vi /etc/hosts
Step 8 Set the cluster password on both the nodes.
Step 9 Start and enable the pcsd sevice on both nodes.
systemctl start pcsd.service
systemctl enable pcsd.service
Step 10 Authenticate and authorize the nodes, any one node
pcs cluster auth <Node1> <Node2> (Proper “HOSTNAME” should be given as it is case sensitive)
Eg: pcs cluster auth pn51-qa2-ha-01 pn51-qa2-ha-02.
Step 11 Start the cluster on one of the nodes. Execute the below command:
pcs cluster setup --start --name <Cluster_Name> <Node1> <Node2>
Note Node1 and Node2 can accept hostname and is case sensitive. For example, pcs cluster setup --start --name hacluster pn51-qa2-ha-01 pn51-qa2-ha-02
Step 12 Enable a cluster with the specified command on both nodes.
Step 13 Log in to the Cluster GUI.
For example, https:// <Anynode>:2224;https://10.56.57.204:2224
a. Enter the username and password.
b. Click Login o access the HAcluster. The High Availability Management window appears.
Figure 10-1 High Availability Login Screen
Step 14 To add a Hostname to an existing Cluster GUI, follow the below steps:
a. Click the MANAGE CLUSTERS tab, and then click Add Existing.
Figure 10-2 Add Existing Clusters
b. In the Add Existing Cluster dialog box, enter the relevant hostname/IP of a node, and then click Add Existing.
Note To create a new cluster, click Create New.
Step 15 To view individual Cluster information, follow the below steps:
a. In the Left pane, check a cluster name to view the nodes, resources, and fence-devices information of the selected cluster.
Note Prime Network installation does not support fencing. Fencing can be configured manually as per the instructions in RHEL documentation.
Step 16 In the right pane, under the Nodes area, click any one node and view the Cluster Status in GUI or in CLI.
Verifying the Prime Network Operations Reports Upgrade from 4.0 to 5.1
Note This section is applicable only for users who has purchased the Prime Network Operation Reports tool prior to May 2018 and are upgrading from previous releases.
When upgrading the Prime Network Operations Reports from 4.0 to 5.1, you must manually enter the following URL of Operations Reports in the Address field:
https:// < gateway-IP >:< port-number >/ prime-network-reports
Where,
Gateway-IP—gateway IP of the Operations Reports portal.
Port-number—SSL port number that was configured during installation. The default SSL port is 8445.
Note In Prime Network, Operations reports application uses TSL 1.0. This is because Pentaho does not allow TSL 1.1 and 1.2 versions and SSL 3.0 and 2.0 versions with RHEL 6.8.
Rolling Back to Prime Network 4.3.2 from Prime Network 5.1 Including Oracle Rollback
Note This procedure assumes configuration with Prime Network 4.3.2 with embedded Oracle 12.1.0.1 only and no High Availability.
Prerequisites
Step 1 Gateway Pre-Upgrade Tasks.
Step 2 Execute the following command as root user to switch to oracle user:
su - oracle
Step 3 Change directory to ORACLE_BASE/backup and record the time stamp of Prime Network 4.3.2 backup (which is Year: 2019 Month: 02 Date: 26 HR:15 MIN: 54 in the example below) and the file name.
For example,
oracle@pn-sgw-02-lnx [~]#cd $ORACLE_BASE/
oracle@pn-sgw-02-lnx [~]# cd backup
oracle@pn-sgw-02-lnx [~/backup]# ls -la
drwxr-xr-x. 2 oracle oinstall 4096 Feb 26 15:54.
drwxr-x---. 16 oracle oinstall 4096 Feb 25 20:41..
-rw-r-----. 1 oracle dba 180060160 Feb 25 15:38 ANADB_before_upgrade_cold_07tqpc3p_1_1
-rw-r-----. 1 oracle dba 10125312 Feb 26 15:45 cfc-4025296251-20190226-01
-rw-r-----. 1 oracle dba 10125312 Feb 26 15:54 cfc-4025296251-20190226-02
-rw-r-----. 1 oracle dba 1114112 Feb 25 20:21 CONTROLFILE_before_upgrade.rman
-rw-r--r--. 1 oracle dba 1393 Feb 25 20:18 PFILE_before_upgrade.manual
-rw-r-----. 1 oracle dba 10043392 Feb 25 20:21 snapcf_anadb.f
-rw-r-----. 1 oracle dba 98304 Feb 25 20:20 SPFILE_before_upgrade.rman
Step 4 As root user, execute the following command:
cp –rp /export/home/oracle/product/12.1.0 /<Directory to be written to>/
Procedure to Rollback Oracle from 12.1.0.2 to 12.1.0.1
Step 1 Execute the following commands as a root user to restore Oracle home directory:
cp -rp /<Directory that stores the backed up 12.1.0.1 home directory as defined in Step 4 of Prerequisites> /export/home/oracle/product/12.1.0
anactl stop
Step 2 To rollback Oracle 12.1.0.2 to Oracle 12.1.0.1:
a. Execute the following commands:
su – oracleuser
cd $ORACLE_HOME/rdbms/admin
sqlplus / as sysdba
SHUTDOWN IMMEDIATE
startup downgrade pfile=<pfilename>;
To retrieve the pfilename, switch to oracle user and go to pfile location:
oracle@pn-sgw-02-lnx [~]# cd /export/home/oracle/admin/anadb/pfile/
oracle@pn-sgw-02-lnx [~/admin/anadb/pfile]# ls
init.ora.1242019133348
For example,
<pfilename> = /export/home/oracle/admin/anadb/pfile/init.ora.1242019133348
SQL> startup downgrade pfile=/export/home/oracle/admin/anadb/pfile/init.ora.1242019133348
SPOOL downgrade.log
@catdwgrd.sql
SPOOL OFF
SHUTDOWN IMMEDIATE
Exit
b. Copy the three files, namely cshrc, inventory.xml, and oratab, and execute source command as follows:
oracle@pn-sgw-02-lnx [~]# cd /etc
oracle@pn-sgw-02-lnx [/etc]# ls -la | grep oratab
-rw-rw-r--. 1 oracle dba 791 Feb 26 12:03 oratab
-rw-r--r--. 1 root root 789 Feb 26 12:03 oratab.12.1.0.1
oracle@pn-sgw-02-lnx [/etc]# cp oratab.12.1.0.1 oratab
oracle@pn-sgw-02-lnx [/etc]# cd
oracle@pn-sgw-02-lnx [~]# ls -la | grep.cshrc
-rw-r--r--. 1 oracle oinstall 1306 Feb 26 12:03.cshrc
-rw-r--r--. 1 root root 1304 Feb 26 12:03.cshrc.12.1.0.1
oracle@pn-sgw-02-lnx [~]# cp.cshrc.12.1.0.1.cshrc
oracle@pn-sgw-02-lnx [~]# cd
oracle@pn-sgw-02-lnx [~]# cd oraInventory/ContentsXML/
oracle@pn-sgw-02-lnx [~/oraInventory/ContentsXML]# ls -la | grep inventory.xml
-rw-rw----. 1 oracle dba 573 Feb 26 12:00 inventory.xml
-rw-rw----. 1 oracle dba 478 Feb 26 11:59 inventory.xml.12.1.0.1
oracle@pn-sgw-02-lnx [~/oraInventory/ContentsXML]# cp inventory.xml.12.1.0.1 inventory.xml
oracle@pn-sgw-02-lnx [~/oraInventory/ContentsXML]# cd
oracle@pn-sgw-02-lnx [~]# source.cshrc
c. Execute the following commands:
cd $ORACLE_HOME/rdbms/admin
sqlplus / as sysdba
STARTUP UPGRADE
SPOOL reload.log
@catrelod.sql
SPOOL OFF
SHUTDOWN IMMEDIATE
STARTUP
@utlrp.sql
d. Switch to Prime Network user and execute the following command:
pn432@pn-sgw-02-lnx [~]% cd Main/scripts/embedded_db/
pn432@pn-sgw-02-lnx [~/Main/scripts/embedded_db]% emdbctl --update_oracle_home
e. Switch to oracle user and restart the lsnrctl status:
lsnrctl stop
lsnrctl start
Step 3 Rollback Prime Network and restore the database as described in Rolling Back to Earlier Prime Network Version.
Rolling Back to Earlier Prime Network Version
Rollback to Prime Network 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, 4.2.2, 4.2.1, or 4.2 is available if you encounter problems during the upgrade, or if you want to roll back to the previous version after the upgrade completes.
Before You Begin
- Verify that the gateway and units are powered up and connected; that is, you can open an SSH session between the gateway and all units.
- Disconnect standby and NAT units from the gateway using the Administration GUI.
- Verify that the Prime Network application is not running with networkctl status.
- Before performing the rollback, stop PN integration layer and watchdog monitoring process. For stopping the Integration layer, refer Chapter 9, “Installing the Prime Network Integration Layer”.
- Rollback Oracle from 12.1.0.2 to 12.1.0.1. Refer Procedure to Rollback Oracle from 12.1.0.2 to 12.1.0.1.
To Roll back Prime Network gateway:
Step 1 If your deployment has units that are connected to the gateway, roll back the units (before rolling back the gateway). The rollback will remove redundant units from the registry and the golden source.
Step 2 Configure all units using the following command:
network-conf -rollback
Step 3 Enter no at the prompt to start the unit.
Step 4 Restore the backed-up database and start the database services and the listener. Because the database table structure changes after the upgrade, the database is backed up as part of the upgrade process. The old table structure must be recovered.
Note If you have a gateway high availability deployment, the services ana and oracle_db services should be moved to maintenance state.
- To restore an external database, contact your database administrator.
- To restore an embedded database:
– Log into the gateway as pnuser.
– Change to the directory PRIME_NETWORK_HOME /Main/scripts/embedded_db:
# cd $PRIME_NETWORK_HOME/Main/scripts/embedded_db
– Gateway Pre-Upgrade Tasks :
Note While executing emdctl --restore_db, enter the time stamp and ensure that your restore is from the time stamp you recorded in Step 3 of Prerequisites (which is Year: 2019 Month: 02 Date: 26 HR:15 MIN: 54 in the step’s example).
For more information on prompts that appear while restoring an embedded database, see the
Cisco Prime Network 5.1 Administrator Guide.
After restoring the database, enter no at the prompt to start Prime Network.
Step 5 As pnuser, move to the temporary upgrade directory (created in Step 1 of the procedure in Upgrading to Prime Network 5.1 from 5.0, 4.3.2, 4.3.1, 4.3, 4.2.3, or 4.2.2 (Intermediate Steps)).
Step 6 Enter the following command to change to the upgrade directory:
Step 7 Enter the following command on the gateway (only):
Step 8 Perform the rollback by entering the required information as shown in the following table.
|
|
|
Confirm that you have restored the database |
yes |
Confirm that you performed Configure all units using the following command:. Note If you have not restored the database, enter no and exit the script. Restore the database and begin again. |
Confirm whether you have reinstalled units |
yes |
Confirm that you performed Step 5. Note If you have not rolled back the units, enter no and exit the script. Rollback the units and begin again. |
Confirm whether you want to roll back to the older version |
yes |
— |
Full path to the backup file |
full pathname |
Location of the backup file (it is not deleted during the rollback). An example is: /export/home/PrimeNetworkBackUp _xxxxxxxxxx.tar.gz |
Step 9 When the rollback is complete, log in as the pnuser to apply the environment changes.
Step 10 Start the unit:
- networkctl start (without running network-conf again)
Step 11 Reconnect standby and NAT units to the gateway using the Administration GUI.
Note Rollback logs can be found in the Prime_Network_upgrade folder under PRIME_NETWORK_HOME.
Upgrading the Prime Network Integration Layer (PN-IL)
If the PN-IL is installed on your system, you can upgrade using the instructions in these topics:
Note If the PN-IL is not installed on your system, you can install it using the instructions in Installing the PN-IL (CLI Method)
Upgrading PN-IL in Standalone Mode
Before You Begin
Perform these tasks as pnuser:
- Disable the health monitor to disable the PN-IL services permanently otherwise the services will start automatically after a delay of 3 minutes.
$PRIMEHOME/local/scripts/il-watch-dog.sh disable
- Back up the $PRIMEHOME directory.
For example,./ilUpgradeUtility.sh backup
- Stop the PN-IL using the following command:
To upgrade a standalone PN-IL:
Step 1 As the root user, launch a terminal on the Prime Network gateway server where you want to install PN-IL.
Step 2 Insert Disk 3: Upgrade Files in the DVD drive.
Step 3 Mount the inserted DVD using mount and move to the mount location.
Step 4 Log in as pnuser :
Step 5 Create a temporary PN-IL upgrade directory.
mkdir -p $PRIME_NETWORK_HOME/pnilupgrade
Step 6 Copy the PN-IL upgrade tar file from the mount location to the pnilupgrade directory.
cp /mnt/**/Upgrade/PNIntegrationLayerUpgrade_1.0.0.0-2.1.0.tar.gz $PRIME_NETWORK_HOME/pnilupgrade
Step 7 Navigate to the directory in which the tar file was copied and extract the PN-IL upgrade tar.
cd $PRIME_NETWORK_HOME/pnilupgrade
tar -zxf PNIntegrationLayerUpgrade_1.0.0.0-2.1.0.tar.gz
Step 8 Navigate to the extracted files directory.
cd
PNIntegrationLayerUpgrade_1.0.0.0-2.1.0
Step 9 Run the upgrade script
./upgradeIntegrationLayer.sh
Step 10 Enter yes at the prompt to continue the upgrade process. The upgrade process is completed and the log file directory changes based on the PNIL versions changes.
Step 11 Perform the following post-upgrade tasks:
a. As pnuser, reload the user profile:
source $PRIME_NETWORK_HOME/.cshrc
b. Configure the PN-IL in standalone mode:
c. Start the PN-IL:
$PRIMEHOME/bin/itgctl start
d. Enable the health monitor:
$PRIMEHOME/local/scripts/il-watch-dog.sh enable
Upgrading PN-IL in Suite Mode
If you have been working with Prime Network 5.1, you will have PN-IL 2.1 installed on your system. The procedure for upgrading to PN-IL 2.1 in suite mode is the same as upgrading in standalone mode. See Upgrading PN-IL in Standalone Mode.
If you have been working with a release prior to Prime Network 5.1, follow the instructions below to upgrade to PN-IL 2.1.
Step 1 Upgrade PN-IL in standalone mode as described in Upgrading the Prime Network Integration Layer (PN-IL).
Step 2 Perform these tasks on the Prime Central Server to create a backup of the PN-IL configuration data.
a. Log in to the Prime Central server as root.
ssh root@Prime-Central-host-IP-address
b. Create Prime Central upgrade directory
mkdir -p $PRIMEHOME/upgrade
c. Copy the PN-IL upgrade tar file (example: PNIntegrationLayerUpgrade_1.0.0.0-2.1.0.tar.gz) from the upgrade directory on the Prime Network server to the upgrade directory on the Prime Central server.
d. Extract the files.
tar -zxf PNIntegrationLayerUpgrade_1.0.0.0-2.1.0.tar.gz
e. Run the PN-IL upgrade utility script to create a backup tar file in $PRIMEHOME/backup.
./ilUpgradeUtility.sh backup
Step 3 Perform these tasks on the Prime Network server to restore the PN-IL configuration.
a. As pnuser, copy the backup tar from the Prime Central upgrade directory to Prime Network server.
b. Extract the files:
tar -zxf il_backup_2.1.0.0.tar.gz
c. Run the PN-IL utility script to restore the PN-IL configuration:
./ilUpgradeUtility.sh restore untar-files-directory
Step 4 Perform these tasks on Prime Central as described in C isco Prime Central Quick Start Guide.
- Upgrade Prime Central
- Integrate Prime Network and PN-IL with Prime Central
Step 5 Start the upgraded PN-IL:
$PRIMEHOME/bin/itgctl start
Prime Network Post-upgrade Tasks
After the upgrade to Prime Network 5.1 is complete, perform the post-upgrade tasks that apply to your deployment.
Enable Units to Restart Automatically After they are Rebooted
After upgrade, you need to perform the following steps on each unit in your setup otherwise the units will not restart automatically after they are rebooted.
Step 1 Log into the unit as pnuser.
Step 2 Copy rootdeploy.cmd from the gateway, as follows:
remote_copy.cmd "<Gateway_IP>:.deploy/independent/on_boot/rootdeploy.cmd" ".deploy/independent/on_boot/rootdeploy.cmd"
Step 3 Switch to the root user:
As the root user, execute the root deploy command:
cd $PRIME_NETWORK_HOME/.deploy/independent/on_boot ;./rootdeploy.cmd
Restoring Customized Crontabs
If you saved user-defined cron jobs in PRIME_NETWORK_HOME /local/cron/crontab.user.list, they are restored. User-defined cron jobs that are not placed in this directory must be manually recreated.
Restarting Crontab Jobs for NAT Units
Cron jobs on NAT units must be manually restarted.
Step 1 Log into the unit as pnuser.
Step 2 Copy the upgrade_restart_crons.pl script from the gateway, as follows:
remote_copy.cmd [gw-ip]:$PRIME_NETWORK_HOME/Main/scripts/upgrade_restart_crons.pl Main/scripts
Step 3 Run the upgrade_restart_crons.pl script. It will display output similar to the following:
./Main/scripts/upgrade_restart_crons.pl
+ Updating the unit's cronjobs
- Writing log to ~/Main/logs/upgrade_crons.log
- Copying the files from the gateway (gateway's_ip)
- Restarting the cronjobs
+ Please wait while the unit is being updated.................................Done.
Step 4 Verify that the crontab list is not empty:
Step 5 The upgrade is now complete. Run the status command and check the version number to make sure that the upgrade has been successful.
Fixing the Database Entry for Vision Clients with NAT
If you are using network address translation (NAT) with the Prime Network Vision client, update the database host in the Prime Network registry to contain the hostname instead of the IP address.
If you already use a hostname instead of an IP address, you do not have to repeat this procedure.
Step 1 Make sure Prime Network is running.
Step 2 Verify that the client workstations have the correct Domain Name System (DNS) mapping.
Step 3 From PRIME_NETWORK_HOME/Main, run the following commands:
./runRegTool.sh -gs 127.0.0.1 set 0.0.0.0 site/persistency/nodes/main/Host database-server-hostname
./runRegTool.sh -gs 127.0.0.1 set 0.0.0.0 site/persistency/nodes/ep/Host database-server-hostname
Step 4 Enter the following command to restart Prime Network:
Updating the Port Watchdog (AVM Protection) Scripts
After upgrading to Prime Network 5.1, copy the port watchdog scripts to /var/adm/cisco/prime-network/scripts. Enter the following commands as the root user:
mkdir –p /var/adm/cisco/prime-network/scripts
cp PRIME_NETWORK_HOME/Main/scripts/port_watchdog.pl /var/adm/cisco/prime-network/scripts
cp PRIME_NETWORK_HOME/Main/scripts/keep_alive_port_watchdog.pl /var/adm/cisco/prime-network/scripts
chmod -R 700 /var/adm/cisco/prime-network/scripts
chown -R pnuser:network /var/adm/cisco/prime-network/scripts
Restore Links Between Devices and Cloud VNEs
If your deployment had cloud VNEs that were connected to devices with static links, the connection between the cloud VNE and the device may be lost after the upgrade. Delete and recreate the link using the Administration GUI.
Support for Third-Party VNEs
Prime Network supports third-party devices through Cisco Advanced Services engagement. As of release 5.0, Prime Network will not natively support third-party devices, and a Cisco Advanced Services contract will be required for their enablement and support.
Command Builder Scripts
If you had customized Command Builder scripts (which you should have uninstalled), you may need to update your scripts if your deployment:
- Executes command scripts using the Prime Network northbound APIs (for example, BQL)
- Includes references to IMOs or to the Prime Network internal model
Verify whether the command names, parameters, or IMO references have changed, in which case you must update your scripts. The reinstall your customized scripts.
Gathering DB Statistics in First 24 Hours
The pnuser _admin user performs maintenance tasks—such as gathering statistics—on the other Prime Network database schemas. After this user is created, a cron job runs every 24 hours to gather statistics on the Fault Database tables.
However, if you expect a high scale in the first 24 hours, you might need to manually force statistics gathering twice during the first day, 1 and 5 hours after noise start. To force statistics gathering, enter the following UNIX command as pnuser :
cd $PRIME_NETWORK_HOME/Main/scripts ;./call_update_ana_stats.pl >& /dev/null
If you deploy Prime Network to handle a high event rate, disabling Oracle’s automatic maintenance jobs is recommended. Automatic maintenance significantly affects Oracle performance and increases event processing time. See Disabling Automatic Maintenance Jobs.
Prime Network supports the resetVneSSHv2Algorithms.pl to clean up the ‘algorithms’ key in avm registry files.
It is recommended to run this script if you are managing the VNE’s from Prime Network3.7 or earlier.
To run the script, follow the below steps:
Step 1 Navigate to ‘<PRIME_NETWORK_HOME>/local/scripts’ directory in GW.
Ensure the following:
– Script runs as PN user from the GW.
– PN should be down in both GW and Units.
Step 2 Run the script file using the below command:
perl resetVneSSHv2Algorithms.pl
Step 3 Start the PN in GW and Units to reflect the changes.
Adding Managed Elements to the Database Manually for PC-FM Resync
After upgrading Prime Network, you can execute BQL commands to invoke a VNE insert operation in a new MANAGED ELEMENTS table for all the existing MANAGED ELEMENTS.
Execute the below BQL commands, which has a VNE name “CopyAllManagedElementsToDB” and IP “0.0.0.0”.
Note Make sure to execute the BQL command before restarting PNIL. BQL execution will not introduce any new VNE, but only performs DB refreshing for all the existing VNE’s; inserts all Managed Elements to DB.
<?xml version="1.0" encoding="UTF-8"?>
<management.IElementManagement type="management.IElementManagement" instance_id="0">
<ID type="Oid">{[MCNetwork][MCVM(IP=X.X.X.X)][ElementManagement(Key=CopyAllManagedElementsToDB)]}</ID>
<IP type="com.sheer.types.IPAddress">0.0.0.0</IP>
<ElementName type="String">CopyAllManagedElementsToDB</ElementName>
</management.IElementManagement>
Note Replace X.X.X.X in the above BQL with Gateway IP Address.
To terminate the further processing of BQL, an Exception that will be returned as part of the response to the BQL must be invoked (Invocation of this Exception is an already available approach used for Validating the input values while creating a new VNE through Modelling tabs.)
Note The below exception message is expected after executing the BQL:
“<Description type="String">ERROR (5133): The VNE's name contains invalid characters. valid chars are: A-Z, a-z, 0-9, _, '@', '!', '~', '.', ' '.</Description>”
For details on BQL and other integrations after the upgrade, refer to the Cisco Developer Network at https://developer.cisco.com/site/prime-network/.
Upgrading the Embedded Database to Oracle 12.1.0.2
You must upgrade the embedded Oracle database to version 12.1.0.2 if:
- You have been using Prime Network 4.1 or a lower version and you want to upgrade to Prime Network 5.1.
AND
- You are planning to upgrade your operating system to Red Hat 6.
If the conditions specified are not met, there is no need to upgrade to Oracle 12.1.0.2 and the upgraded Prime Network 5.1 can run with Oracle 12.1.0.1 as well.
While upgrading to Oracle 12.1.0.2, follow the steps:
1. First upgrade to Prime Network 4.1.
2. Upgrade Oracle from earlier version to Oracle 12.1.0.1.
3. Upgrade your Operating System.
4. Upgrade from Prime Network 4.1 to Prime Network 5.1.
5. Upgrade to Oracle 12.1.0.2
Before you Begin
- Copy the following Oracle installation.zip files from Prime Network 5.1, Disk 4 : Database Binaries to a directory on the machine on which the embedded database is installed (either on the local gateway server or a remote server):
– linuxamd64_12c_database_1of2.zip
– linuxamd64_12c_database_2of2.zip
Step 1 If your setup has cluster, freeze the cluster configured services (ana and oracle_db) using the following command:
Step 2 Start the database upgrade by entering the following command:
# perl upgrade_embedded_oracle_12.pl
Example-Upgrading the Embedded Database to Oracle 12.1.0.2
Step 1 In the database server, perform the following steps:
a. Copy the following two zip files to /tmp/upg12c.
– linuxamd64_12c_database_1of2.zip
– linuxamd64_12c_database_2of2.zip
b. Create the staging directory by entering the following commands:
c. Upgrade to Oracle 12.1.0.2 by entering the following command:
# perl upgrade_embedded_oracle_12.pl
Enter the name of the OS user of the database [oracle]
Enter the staging/upgrade directory. This directory should have at least 9GB free space [/export/home/stg]
Running pre-upgrade validations
Extracting /tmp/upg12c/linuxamd64_12c_database_2of2.zip
Extracting /tmp/upg12c/linuxamd64_12c_database_1of2.zip
Diagnosing the database status
Running pre-upgrade tasks
Copying files to new Oracle home
Verifying no files needs media recovery and no backup is running
Before proceeding with the upgrade, this procedure will take a backup of the database. you may choose between
1. Offline (Cold) backup (requires database downtime) [default]
The database is about to be shutdown. Please stop PrimeNetwork and any other application using the database.
Hit the 'Enter' key when ready to continue
Stopping the database & listener
Stopping the database & listener
Upgrading the database. This step may take at least 40 minutes.
Executing post upgrade tasks.
Identifying new invalid objects
Copying PrimeNetwork scripts to new Oracle home
Restarting Oracle cronjobs
Upgrade completed successfully. Logs can be found under /opt/ora/oracle/ana_logs/upgrade
To complete the upgrade, enter the following command as the Prime Network user:
cd ~/Main/scripts/embedded_db ; emdbctl --update_oracle_home
You have new mail in /var/spool/mail/root
Step 2 Enter the required information as shown in the following table.
|
|
|
OS username |
Username for the Oracle database user. |
Default is oracle. |
Staging/upgrade directory |
Path to the directory from which the upgrade will run and to which the database zip files will be extracted. |
Default is /export/home/stg |
Location of zip files |
Path to the directory to which the Oracle zip files were copied. |
— |
Database backup method |
Offline (Cold) backup or Online (Hot) backup |
With cold backup, the database is down during the backup. With hot backup, the database continues to run until the upgrade starts. Downtime is shorter but the backup might take longer. Default is cold backup. |
Step 3 Login to Oracle, and restart the embedded Oracle by following command:
#lsnrctl stop
#lsnrctl start
Upgrading the Embedded Database to Oracle 12.1.0.2 in a HA Setup with Geographical Redundancy and Oracle ADG
You must upgrade the embedded Oracle database to version 12.1.0.2 if:
- You have been using Prime Network 3.9 or a lower version and you want to upgrade to Prime Network 5.1.
AND
- You are planning to upgrade your operating system to Red Hat 6.
If the conditions specified are not met, there is no need to upgrade to Oracle 12.1.0.2, and the upgraded Prime Network 5.1 can run with Oracle 12.1.0.1 as well.
While upgrading to Oracle 12.1.0.2, follow the steps:
1. First upgrade to Prime Network 4.1.
2. Upgrade Oracle from earlier version to Oracle 12.1.0.1.
3. Upgrade your Operating System.
4. Upgrade from Prime Network 4.1 to Prime Network 5.1.
5. Upgrade to Oracle 12.1.0.2
Before you Begin
- To be performed on both primary and standby gateway servers: Copy the following Oracle installation.zip files from Prime Network 5.1 Disk 4: Database Binaries to a directory on the machines on which the embedded database is installed (both the primary and standby gateway servers):
– linuxamd64_12c_database_1of2.zip
– linuxamd64_12c_database_2of2.zip
- Ensure that there is a minimum of 12 GB free disk space on each of the servers. This space is freed up after the upgrade has completed successfully.
- Verify that database replication works properly prior to starting the database upgrade by performing the geographical verification tests described in the Cisco Prime Network 5.1 High Availability Guide.
Step 1 On the standby gateway server, run the Oracle software upgrade and prepare the standby server for database upgrade.
Navigate to the upgrade scripts directory and enter the following command:
# perl standby_db_prepare_for_upgrade_12.1.pl
Step 2 On the primary gateway server, start the database upgrade by entering the following command:
# perl upgrade_embedded_oracle_12.pl
Step 3 Enter the required information as shown in the following table.
|
|
|
OS user name |
Username for the Oracle database user. |
Default is oracle. |
Staging/upgrade directory |
Path to the directory to which the upgrade zip file was copied. |
— |
Location of zip files |
Path to the directory to which the Oracle zip files were copied. |
— |
Database backup method |
Offline (Cold) backup or Online (Hot) backup |
With cold backup, the database is down during the backup and the gateway is stopped. With hot backup, the database continues to run until the upgrade starts. Downtime is shorter but the backup might take longer. Default is cold backup. |
Step 4 On the primary gateway server, verify that the Oracle listener is running by entering the following command as the root user:
su - oracle -c "lsnrctl status"
Step 5 On the standby gateway server, set back the replication redo apply by running the standby_post_upgrade.pl to perl./standby_db_post_upgrade12.1.pl.
Example-Upgrading the Embedded Database to Oracle 12.1.0.2 in a HA Setup with Geographical Redundancy and Oracle ADG
Step 1 Stop the Prime Network.
Step 2 Verify if the replication between databases work.
Step 3 In the STANDBY database server, perform the following steps:
a. Navigate to the location where the embedded Oracle software is available.
b. Copy the two zip files to /tmp/upg12c:
– linuxamd64_12c_database_1of2.zip
– linuxamd64_12c_database_2of2.zip
c. Create the staging directory by entering the following commands:
d. Upgrade to Oracle 12.1.0 by entering the following command:
# perl standby_db_prepare_for_upgrade_12.1.pl
- Enter the name of the OS user of the database [oracle]
- Enter the staging/upgrade directory. This directory should have at least 9GB free space [/export/home/stg]
- Running pre-upgrade validations
- Extracting /tmp/upg12c/linuxamd64_12c_database_2of2.zip
- Extracting /tmp/upg12c/linuxamd64_12c_database_1of2.zip
- Installing the software
- Copying files to new Oracle home
- Enter the name of the prime network user :pn400
- Backing up system files
- Starting the standby database in mount mode.
- Copying PrimeNetwork scripts to new Oracle home
- Restarting Oracle cronjobs
Standby database is ready for upgrade. Please run the upgrade procedure for the primary database. Logs can be found under /opt/ora/oracle/ana_logs/upgrade
Step 4 In the PRIMARY database server, perform the following steps:
a. Navigate to the location where the embedded Oracle software is available.
b. Copy the two zip files to /tmp/upg12c:
– linuxamd64_12c_database_1of2.zip
– linuxamd64_12c_database_2of2.zip
c. Create the staging directory by entering the following commands:
d. Upgrade to Oracle 12.1.0.2 by entering the following command:
# perl upgrade_embedded_oracle_12.pl
- Enter the name of the OS user of the database [oracle]
- Enter the staging/upgrade directory. This directory should have at least 9GB free space [/export/home/stg]
- Running pre-upgrade validations
- Extracting /tmp/upg12c/linuxamd64_12c_database_2of2.zip
- Extracting /tmp/upg12c/linuxamd64_12c_database_1of2.zip
- Diagnosing the database status
- Installing the software
- Running pre-upgrade tasks
- Copying files to new Oracle home
- Verifying no files needs media recovery and no backup is running
- Before proceeding with the upgrade, this procedure will take a backup of the database. you may choose between
1. Offline (Cold) backup (requires database downtime) [default]
The database is about to be shutdown. Please stop PrimeNetwork and any other application using the database.
Hit the 'Enter' key when ready to continue
- Stopping the database and listener
- Backing up the database
- Stopping the database and listener
- Backing up system files
- Upgrading the database. This step may take at least 40 minutes.
- Executing post upgrade tasks
- Upgrading timezone file
- Enter the name of the prime network user :pn400
- Running Oracle patch installation
- Identifying new invalid objects
- Copying PrimeNetwork scripts to new Oracle home
- Restarting Oracle cronjobs
Upgrade completed successfully. Logs can be found under /opt/ora/oracle/ana_logs/upgrade
To complete the upgrade, enter the following command as the Prime Network user:
cd ~/Main/scripts/embedded_db; emdbctl --update_oracle_home
You have new mail in /var/spool/mail/root.
----------------------------------------------------------------------------------------
.-= Welcome to pn-ha-p1-s5, running Cisco Prime Network gateway (v5.1 (build 119)) =-.
----------------------------------------------------------------------------------------
+ Checking for services integrity:
- Checking if host's time server is up and running [DOWN]
- Checking if webserver daemon is up and running [OK]
- Checking if secured connectivity daemon is up and running [OK]
- Checking Prime Network Web Server Status [DOWN]
- Checking Compliance Engine Status [DOWN]
- Detected AVM99 is down, skipping AVMs check
+ Checking for latest installed device packages:
- Cisco: PrimeNetwork-5.0-DP0
- Third party: No third party device package installed.
Step 5 In the STANDBY database server, perform the following steps:
a. Enter the following command:
b. Upgrade the Oracle version by entering the following command:
# perl standby_db_post_upgrade12.1.pl
- Enter the name of the OS user of the database [oracle]
- Setting standby DB for redo apply
- Enter the staging/upgrade directory, same one that was provided earlier [/export/home/stg]
- Enter the name of the prime network user :pn400
- Starting the STANDBY database in mount mode.
- Standby database is ready. Please verify replication.