Feature Summary and Revision History

Feature Description

In CPS 22.1.1 and later releases, if you want to upgrade CPS from prior releases, change the storage engine from MMapV1 to WiredTiger. The prerequisites for upgrading to 22.1.x release are:

• Set the 4.0 replica set to CompatibilityVersion 4.0. To ensure that all members of the replica set have featureCompatibilityVersion set to 4.0, connect to each replica set member and check the featureCompatibilityVersion:

  db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

  All members return a result that includes "featureCompatibilityVersion" : { "version" : "3.6" }.

• To set or update featureCompatibilityVersion, run the following command on the primary. Most of the data-bearing members must be available:

  db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } )

Note	In CPS 22.1.1, PCRF runs with Mongo DB 4.0.27 and Mongo Java Driver 3.12.9. The compatible driver for both Mongo 4.0 and 4.2 is 3.11 and above.

For more information, see the Configuration Parameters - HA System table in the CPS Installation Guide for OpenStack and Modification of Storage Engine before Upgrade.

Memory and Performance Impact

Wired Tiger Storage engine change in MongoDB Server 4.0 requires additional CPU resources of ~15% and additional memory (RAM) resources of ~40% in the Session Manager VMs. Up to ~40% more memory being consumed more by wiredtiger from total memory(RAM) than MMapV1.

For example, if the session manager VM(150GB) with MMapV1 utilizes 60GB then, wiredtiger requires 120GB(MMapV1 usage 60GB + 40% of total memory 150GB).

Upgrade, Migrate, and Backward Compatibility Considerations

You can upgrade CPS 21.1.0 or 21.2.0 to CPS 22.1.1, and then to CPS 22.2.0 with ISSM process. Upgrade from prior to CPS 21.1.0 releases is not supported.

During the ISSM process, a new VM is created, and then a replica set member is newly created with only a new mongo version.

Prerequisites for the ISSM Process

The following are the Prerequisites:

• Take a copy of the mongoConfig.cfg file in both old Cluster Managers.

• Update the following values in mongoConfig.cfg file:

  • WT_CACHESIZEGB=12

  • WT_CACHEARBSIZEGB=1

• Execute import_deploy.sh before performing ISSM procedure.

• Make sure that the system is running mongo 3.6 and Java driver patch is applied on the system.

After the prerequisites conditions are met, perform the ISSM process.

Roll back Procedure

Before performing a rollback, restore the copied mongoConfig.cfg file in older Cluster Managers.

Execute the import_deploy.sh before performing ISSM rollback procedure.

Feature Summary and Revision History

Feature Description

The CPS vDRA supports the following vDRA release versions for upgrading to vDRA 22.1.1.

• vDRA 22.1.0 (MongoDB version 4.0.27 MMAP) to vDRA 22.1.1 (MongoDB version 4.0.27 WT).

• vDRA 22.1 Patch 1 (MongoDB version 4.0.27 MMAP) to vDRA 22.1.1 (MongoDB version 4.0.27 WT).

• vDRA 21.1 Patch4 (MongoDB version 3.6.9 MMAP) to vDRA 22.1.1 (MongoDB version 4.0.27 WT).

Prerequisites for Upgrading to 22.1.1 and Rollback from 22.1.1

The following are the common prerequisites for upgrade and roll back:

• Run the following CLI before upgrade:

  #database fcvcheck

  Note	Make sure to run the above CLI before upgrade and / or downgrade on all sites.

• Specify any one of the CLI options:

  • Set: This option checks and sets FCV only on primary.

    Note

    We recommended to use Set option first and then Check to make sure that FCV is replicated on secondary members. Upgrade/downgrade should not be triggered if any error is found in above CLI or FCV is not replicated on secondary members. Make sure to resolve the CLI error, rerun the CLI, and then only proceed for upgrade or downgrade.

  • Check: This option only checks FCV on all members (primary, secondary, and arbiter).

Upgrade to 22.1.1

1. Run the prerequisite steps.

2. Follow the standard documented procedure for upgrade.

3. After upgrade is successful, execute the following additional post checks:

   show running-config database | include storage

   The output displays the storage-engine WT.

   admin@orchestrator[fPAS-site3-master-1]# docker exec mongo-s104 "ps -efww"
   ==========output from container mongo-s104===========
   root        32     1 88 May19 ?        12-15:30:18 mongod --keyFile=/mongodb.key --enableMajorityReadConcern false --ipv6 --bind_ip_all --port 27021 --dbpath=/data/db/wt-27021 --replSet rs-app_shardCD-ipv6-1 --quiet --slowms 500 --logpath /data/db/mongo-27021.log --setParameter diagnosticDataCollectionEnabled=true --logappend --oplogSize 3221 --logRotate reopen --wiredTigerCacheSizeGB 4.40
   output of above cli should contain 4.40 value for–wiredTigerCacheSizeGB parameter.
   admin@orchestrator[fPAS-site3-master-1]# docker exec mongo-s104 "df -hT"
   ==========output from container mongo-s104===========
   Filesystem     Type     Size  Used Avail Use% Mounted on
   tmpfs          tmpfs    7.6G  1.6G  6.0G  21% /data/db/wt-27026
   tmpfs          tmpfs    500M  301M  200M  61% /data/db/wt-27029
   Output of above CLI contains 7.6G size (for non-arbiters) and 500M (for arbiters) size tmpfs partitions.
   

Downgrade from 22.1.1

1. Run the steps mentioned in the prerequisite section.

2. Execute the below CLI to convert database config from WT to MMAP.

    #database changeConfigToMMAP

3. After running the above two prerequisites, follow the standard documented procedure for downgrade.

Roll back Procedure

Before performing a rollback, restore the copied mongoConfig.cfg file in older Cluster Managers.

Execute the beforeimport_deploy.sh performing ISSM rollback procedure.

The following KPIs are added in the Database Monitor in binding VNF:

• WT Cache Usage %: (mongo_current_bytes_into_cache/mongo_max_bytes_configured)*100 – Monitors WT cache usage in %age.

• WT Tracked Dirty Usage %: (mongo_tracked_dirty_bytes_in_cache/mongo_max_bytes_configured)*100 – Monitors dirty bytes in WT cache in %age.

• WT Pages Read into Cache (MB): mongo_pages_read_into_cache/(1024*1024) – Monitors number of pages that are read into cache in MB.

• WT Pages Written from Cache (MB): mongo_pages_written_from_cache/(1024*1024) – Monitors number of pages that are written from the cache in MB.

Limitations

In the 22.1.1 release, runtime database configuration change is not supported and not recommended. Runtime database configuration is performed to support in service storage engine conversion from MMAP to WT and vice versa. To perform any changes, you must stop the database and reconfigure with appropriate configuration changes.

Additional Information

WiredTiger storage engine change in MongoDB Server 4.0 requires additional resources:

• CPU utilize ~10-15 [under the threshold] on DBP VMs while no significant change observed in rest of the VMs [distributor / director / worker].

• DBP VMs consume up to ~20–30% extra memory more by wiredtiger from total memory (RAM) than MMapV1.