Operations

API Additions or Changes

No changes were introduced in this release.

Log Additions or Changes

Enhancement on Logging and Logback

Feature Summary and Revision History

Table 1. Summary Data

Applicable Product(s) or Functional Area

CPS

Applicable Platform(s)

Not Applicable

Default Setting

Enabled - Always ON

Related Changes in This Release

Not Applicable

Related Documentation

Not Applicable
Table 2. Revision History

Revision Details

Release

First introduced

20.2.0

Feature Description

CPS now supports enhancements to logback xml file.

A new script logCollector.sh is introduced which performs the following operations:

  • Provides options to enable and disable the log levels for specific components, class, and interfaces

  • Collects the enabled debug logs from all VMs or Specific VMs and store provided log path.

  • Displays proper error message when the user does not provide valid inputs.

  • Enables alias functionality for each function which helps user to provide only the operation name which is needed to execute the script.

  • Adds the timer function to ensure the collection of required logs in the amount of time passed to the script.

Logging system provides more information with exception in a user-friendly and readable format. This feature is applicable for logging messages in both Core and CustRefData modules to print the clear context of the source such as process, subsystem, and exception when occurs.

MIB Additions or Changes

No changes were introduced in this release.

SNMP Alarm Additions or Changes

The following table provides information on new/modified alarms:

Table 3. Alarm Additions or Changes

New/Modified Alarms

Release Introduced/

Modified

Applicable Product(s)/

MongoPrimaryDB fragmentation exceeded the threshold value

20.2.0

CPS

PrimaryDB fragmentation percent conforms to threshold

20.2.0

CPS

SVNnotinsync

20.2.0

CPS

SVNinsync

20.2.0

CPS

DOCKER_ENGINE_DOWN

20.2.0

vDRA

For more information, see the following sections:

  • Application Notifications table in the CPS SNMP, Alarms, and Clearing Procedures Guide

  • Clearing Procedures chapter in the CPS SNMP, Alarms, and Clearing Procedures Guide

  • Testing Traps Generated by CPS in the CPS Troubleshooting Guide

Configuration for SNMP Gets and Walks

As CPS 20.2.0 is built on CentOS 8.1, snmpwalk command has limitations and hence cannot perform a direct snmpwalk on the OID such as .1.3.6.1.4.1.26878.200.3.2.70. Instead of snmpwalk, you need to use snmpget command along with the complete OID such as .1.3.6.1.4.1.26878.200.3.2.70.1.1. The list of OIDs for the individual machines are available in /etc/snmp/snmpd.conf file. The OIDs are part of the line containing the word proxy.

Here is an example:

proxy -e 0x0102030405060708 -v 3 -u cisco_snmpv3 -a SHA -m 
0x71d8d544a7447e377fa5fc355d8f08f81f1a901c -x AES -m 0x71d8d544a7447e377fa5fc355d8f08f8 
-l authPriv localhost .1.3.6.1.4.1.26878.200.3.2.70.1.1.0  .1.3.6.1.4.1.2021.11.9.0

Here .1.3.6.1.4.1.26878.200.3.2.70.1.1.0 is the OID and hence the snmpget must be triggered as follows: 

snmpget -e 0x0102030405060708 -v 3 -u cisco_snmpv3 -a SHA -A cisco_12345 
-x AES -l authNoPriv -m +/etc/snmp/mibs/BROADHOP-MIB.txt:/etc/snmp/mibs/CISCO-QNS-MIB.txt 
lb01 ".1.3.6.1.4.1.26878.200.3.3.70.11.2.0"
CISCO-QNS-MIB::kpiLBPCRFProxyInternalCurrentSessions.0 = STRING: 0

For more information, see Configuration for SNMP Gets and Walks section in the CPS SNMP, Alarms, and Clearing Procedures Guide.

Statistics/KPI Additions or Changes

The following table provides information on new/modified statistics:

Table 4. Statistics Additions or Changes

Statistics Name

Description

Applicable Product(s)

node1.counters.

total_tags_added

The total number of new tags added in overall sessions.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.counters.

total_tags_removed

The total number of tags removed in overall sessions.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.counters.session_

count_exceeding_tag_size

The total number tags exceeding the predefined size.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.counters.

session_count_exceeding_

predefined_number_of_tags

The total number of sessions containing the number of tags in TagsList more than predefined size.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.counters.total_

session_with_padding

The total number of sessions created with padding.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.counters.total_

session_without_padding

The total number of sessions created without padding.

The source of the statistics is Policy Server (QNS) VM.

CPS

indexSize

Indicates the total size of all indexes created on a database.

The source of the statistics is Policy Server (QNS) VM.

CPS

storageSize

The total amount of space allocated to collections in database for document storage.

The source of the statistics is Policy Server (QNS) VM.

CPS

fileSize

The total size (in bytes) of the data files that hold the database. This value includes pre-allocated space and the padding factor.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.jms.PolicyEngineJmsSender.

qns_jms_senders.

MessagesSentCount

Number of async messages sent.

The source of the statistics is Policy Server (QNS).

CPS

node1.jms.PolicyEngineJmsReceiver-

Cluster.qns_jms_receivers.

MessagesReceived

Number of messages received.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.jms.PolicyActionJmsSender.

qns_jms_receivers.

MessagesSentCount

Number of PolicyAction messages sent.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.jms.PolicyActionJmsReceiver-

Global.qns_jms_receivers.

MessagesReceived

Number of PolicyAction messages received.

The source of the statistics is Policy Director (LB) VM.

CPS

node1.jms.FlowControl.

qns_jms_flowcontrols.

NumberOfFlowControlledMessages

Number of messages that were flow controlled.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.jms.FlowControl.

qns_jms_flowcontrols.QueueSize

Flow control queue size.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.jms.FlowControl.

qns_jms_flowcontrols.QueueSizeLimit

Flow control queue size limit.

The source of the statistics is Policy Server (QNS) VM.

CPS

rcv_ttl_drop_<fqdn>

Number of messages discarded due to exceeding SLA in inbound direction.

The source of the statistics is Policy Director (LB) VM.

CPS

send_ttl_drop_<fqdn>

Number of messages discarded due to exceeding SLA in outbound direction.

The source of the statistics is Policy Director (LB) VM.

CPS

node1.cdr.<CDRName>.write

Number of CDRs written to the database for the CDR name

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.cdr.<CDRName>.drop

Number of CDRs dropped without writing to database for the CDR name.

The source of the statistics is Policy Server (QNS) VM.

CPS

node1.cdr.<CDRName>.

replTaskOverrun

Number of times the replication task could not be run as the previous task was still running for the CDR name.

The source of the statistics is Policy Server (QNS)/Policy Director (LB) VM.

CPS

node1.cdr.<CDRName>.

replSkipNearCurrentTime

Number of times the replication task was skipped as the replication time is near current time for the CDR name.

The source of the statistics is Policy Server (QNS)/Policy Director (LB) VM.

CPS

node[x].classes.gauge-

loaded_classes

Number of loaded classes in the JVM.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].classes.gauge-

unloaded_classes

Number of unloaded classes in JVM.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].thread.gauge-

daemon_thread_count

Total number of daemon threads in the JVM.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].thread.gauge-

live_thread_count

Total number of live threads in the JVM.

The source of the statistics is Policy Server (QNS), Policy Director (LB) VMs.

CPS

node[x].thread.gauge-peak_

live_thread_count

Peak count of the live thread in the JVM.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].thread.gauge-total_

started_thread_count

Total number of threads started by the JVM.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].gc-ConcurrentMark

Sweep.invocations

Total number of times ConcurrentMarkSweep GC occurred.

The source of the statistics is Policy Server (QNS) VM.

CPS

node[x].gc-ConcurrentMarkSweep.total_

time_in_ms-collection_time

Time taken in millisecods for the ConcurrentMarkSweep GC.

The source of the statistics is Policy Server (QNS) VM.

CPS

node[x].gc-ParNew.

invocations

Total number of times ParNew GC occurred.

The source of the statistics is Policy Server (QNS) VM.

CPS

node[x].gc-ParNew.total_

time_in_ms-collection_time

Time taken in millisecods for the ConcurrentMarkSweep GC.

The source of the statistics is Policy Server (QNS) VM.

CPS

node[x].gc-PS_MarkSweep.

invocations

Total number of times PS MarkSweep GC occurred.

The source of the statistics is Policy Director (LB) VM.

CPS

node[x].gc-PS_MarkSweep.total_

time_in_ms-collection_time

Time taken in millisecods for the PS MarkSweep GC.

The source of the statistics is Policy Director (LB) VM.

CPS

node[x].gc-PS_Scavenge.

invocations

Total number of times PS Scavenge GC occurred.

The source of the statistics is Policy Director (LB) VM.

CPS

node[x].gc-PS_Scavenge.total_

time_in_ms-collection_time

Time taken in milliseconds for the PS Scavenge GC.

The source of the statistics is Policy Director (LB) VM.

CPS

skdb_cache_get_total.

qns_stat.success

The total number of success queries on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

total.qns_stat.error

The total number of error/fail queries on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_total.

qns_stat.total_time_in_ms

The total time in millisecond to query on all SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

total.qns_stat.avg

The average time taken by the queries on all SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get.

qns_stat.success

The number of success queries on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_

get.qns_stat.error

The number of error/fail query on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get.qns_

stat.total_time_in_ms

The total time in millisecond to query on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get.

qns_stat.avg

The average number of queries on SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_remote.

qns_stat.success

The total number of success queries on remote SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

remote.qns_stat.error

The total number of error/fail query on remote SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_remote.qns_

stat.total_time_in_ms

The time in millisecond to query on remote SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

pri.qns_stat.avg

The average number of queries on primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

pri.qns_stat.success

The total number of success queries on primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_

pri.qns_stat.error

The total number of error/fail query on primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri.qns_

stat.total_time_in_ms

The time in millisecond to query on primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri.

qns_stat.avg

The average number of queries on primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri_

remote.qns_stat.success

The number of success queries on remote site for primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri_

remote.qns_stat.error

The number of error/fail query on remote site for primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri_remote.

qns_stat.total_time_in_ms

The total time in millisecond to query on remote site for primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

skdb_cache_get_pri_

remote.qns_stat.avg

The average number of queries on remote site for primary SK database cache.

The source of the statistics is Policy Server (QNS) VM.

CPS

parallel_query_

skdb_fail

Parallel query to get secondary key record from the local site secondary member if SK database fails.

The source of the statistics is Policy Server (QNS) VM.

CPS

svn_status.records,1.0

This statistics shows that SVN is in sync on the pcrfclient VM’s.

Note 

New SVN KPI stats are added in

/var/broadhop/stats/

bulk-pcrfclient-*.csv.

CPS

svn_status.records,0.0

This statistics shows that SVN is not in sync on the pcrfclient VM’s.

Note 

New SVN KPI stats are added in

/var/broadhop/stats/

bulk-pcrfclient-*.csv.

CPS

node[x].actions.

ISendRealTimeNotificationRequest.

qns_stat.avg

Rolling 5 minute average of sending of outbound real time notifications.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].actions.

ISendRealTimeNotificationRequest.

qns_stat.error

Count of errors sent in outbound real time notifications.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].actions.

ISendRealTimeNotificationRequest.

qns_stat.success

Count of real time notifications sent out successfully.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].actions.

ISendRealTimeNotificationRequest.

qns_stat.total_time_in_ms

Total time in milliseconds required to sent out successful outbound realtime notifications.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].counters.r.n_

<realtime_notification_template_name>

_fail.qns_count

Number of failed <realtime_notification_template_name> notifications sent to primary URL.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].counters.r.n.f_

<realtime_notification_template_name>

_fail.qns_count

Number of failed <realtime_notification_template_name> notifications sent to fallback URL.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].counters.r.n_

<realtime_notification_template_name>

_success.qns_count

Number of successful <realtime_notification_template_name> notifications sent to primary URL.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

node[x].counters.r.n.f_

<realtime_notification_template_name>

_success.qns_count

Number of successful <realtime_notification_template_name> notifications sent to fallback URL.

The source of the statistics are Policy Server (QNS) and Policy Director (LB) VMs.

CPS

db_cpu_threshold_

breach_total

This statistics displays the total number of requests rejected/forwarded due to database CPU usage threshold breach.

CCR-I requests are rejected in case of database CPU threshold breach and bindings are not marked as best effort bindings.

Requests are forwarded in case of database CPU threshold breach and bindings are marked as best effort bindings. For CCR-I, bindings are not stored. For CCR-T/ Gx RAR, bindings are not deleted.

Field in statistics:

status = discard/forward

operation = create/read/update/delete

vDRA

dra_api_binding_

sharddetails_count

Total number of shard details requests that are successful or failures.

Details of field in statistics.

  • binding_type = session/ipv6/ipv4/imsi/msisdn

  • status = error_500/error_404/success

vDRA

Support to Configure Database Fragmentation Threshold

Feature Summary and Revision History

Table 5. Summary Data

Applicable Product(s) or Functional Area

CPS

Applicable Platform(s)

Not Applicable

Default Setting

Enabled – Configuration Required

Default value - 40 %

Related Changes in This Release

Not Applicable

Related Documentation

CPS Operations Guide
Table 6. Revision History

Revision Details

Release

First introduced

20.2.0

Feature Description

CPS now supports configuring custom database fragmentation threshold percentage for the list of databases present in /etc/collectd.d/dbMonitorList.cfg file on sessionmgr VMs. By default, the threshold is set to 40 % for all the databases in /etc/collectd.d/dbMonitorList.cfg file.

For more information, see Configure Custom Database Fragmentation Threshold Percentage section in the CPS Operations Guide.

Support to Configure Threshold Values for Gx and LDAP Alarms

Feature Summary and Revision History

Table 7. Summary Data

Applicable Product(s) or Functional Area

CPS

Applicable Platform(s)

Not Applicable

Default Setting

Enabled – Configuration Required

Related Changes in This Release

Not Applicable

Related Documentation

CPS Installation Guide for VMware

CPS Installation Guide for OpenStack
Table 8. Revision History

Revision Details

Release

First introduced

20.2.0

Feature Description

CPS now supports:

  • To configure different threshold values for CCR-I/U/T response time exceeded alarms.

  • To configure LDAP retry, request and result alarm threshold values using Configuration.csv in VMware environment and YAML file in OpenStack environment.

To support the threshold values, following parameters are added:

  • Under Configuration Parameters - HA System section in the CPS Installation Guide for OpenStack:

    • gxAlarmCcrIAvgThreshold

    • gxAlarmCcrUAvgThreshold

    • gxAlarmCcrTAvgThreshold

    • ldapAlarmRetryThreshold

    • ldapAlarmCcrIReqThreshold

    • ldapAlarmResultThreshold

    • ldapAlarmRequestThreshold

  • Under General Configuration section in the CPS Installation Guide for VMware:

    • gx_alarm_ccr_i_avg_threshold

    • gx_alarm_ccr_t_avg_threshold

    • gx_alarm_ccr_u_avg_threshold

    • ldap_alarm_ccr_i_req_threshold

    • ldap_alarm_request_threshold

    • ldap_alarm_result_threshold

    • ldap_alarm_retry_threshold

For more information, refer to the concerned sections in CPS Installation Guide for OpenStack and CPS Installation Guide for VMware.