High CPU and virtual memory issues

Monitor high CPU and virtual memory issues using Unified Operations Manager

Unified Operations Manager 8.6 and 8.7 provides an overview of CPU and virtual memory usage on your IM and Presence Service node. If usage is high, debug further using Unified RTMT.

Unified Operations Manager 9.0 reports on the memory usage of the IM and Presence Service node A Cisco DB service or cmoninit processes, the system overview, and also the CPU usage of the following services:
  • Cisco Tomcat
  • Cisco Presence Engine
  • Cisco SIP Proxy
  • Cisco XCP Router
  • Cisco XCP Connection Manager
  • Cisco XCP Web Connection Manager
  • Cisco XCP SIP Federation Connection Manager
  • Cisco XCP XMPP Federation Connection Manager

Monitor high CPU and virtual memory issues using Unified RTMT

Unified RTMT provides an overview of CPU and Virtual Memory usage using the CPU and Memory tool. This provides overall system usage statistics for all nodes in an IM and Presence cluster.

Unified RTMT allows monitoring of usage statistics, and individual processes using the Process tool. Every process on the server reports data for the following:
PID
The task's unique process ID, which periodically wraps, though never restarts at zero.
% CPU
The task’s share of the elapsed CPU time since the last update.
Status
The task's process status: 0 - Running, 1 - Sleeping, 2 - Uninterruptible disk sleep, 3 - Zombie, 4 - Traced or stopped (on a signal), 5 - Paging, 6 - Unknown.
Shared Memory (KB)
The amount of shared memory, in kilobytes (KB), that a task is using. Other processes could potentially share the same memory.
Nice (Level)
The nice value of the task. A negative nice value indicates that the process has a higher priority, while a positive nice value indicates that the process has a lower priority. If the nice value equals zero, do not adjust the priority when you are determining if the task can be dispatched.
VmRSS (KB)
The virtual memory (Vm) resident set size (RSS) that is currently in physical memory in KB, including Code, Data, and Stack.
VmSize (KB)
The total amount of virtual memory, in KB, that the task is using. It includes all code, data, shared libraries, and pages that have been swapped out: Virtual Image = swapped size + resident size.
VmData (KB)
The virtual memory usage of the heap for the task in KB.
Thread Count
The number of threads that are currently grouped with the task. The negative value -1 indicates that this counter is currently not available because thread statistics (including all performance counters in the Thread object as well as the Thread Count counter in the Process object) have been turned off because the system's total processes and threads have exceeded the default threshold value.
Data Stack Size
The stack size for task memory status.
Page Fault Count
The number of major page faults that a task encountered that required the data to be loaded into memory.

High CPU issues

On IM and Presence Service, when you experience high overall CPU usage, Cisco recommends that you check the usage of the following processes that have historically caused high CPU on IM and Presence:

Process Service
tomcat Cisco Tomcat
jabberd Cisco XCP Router
pe Cisco Presence Engine
cm Cisco XCP Connection Manager
cm_web Cisco XCP Web Connection Manager
cm_sip_fed Cisco XCP SIP Federation Connection Manager
cm_xmpp_fed Cisco XCP XMPP Federation Connection Manager
cmoninit A Cisco DB
sipd Cisco SIP Proxy

Note

The cmoninit and sipd processes will both have 20+ individual instances, any one of which could be responsible for high CPU usage.

If the process consuming CPU is not in the preceding table, consult the following table for a list of other processes and their corresponding services. If the process causing high CPU is not in either table, the problem may reside with a system or platform service. Consult Cisco TAC for further assistance.

Process Service
amc Cisco AMC Service
AuditLog Cisco Audit Event Service
auth Cisco XCP Authentication Service
BPS Cisco Bulk Provisioning Service
cdpd Cisco CDP
cdpAgt Cisco CDP Agent
certM Cisco Certificate Expiry Monitor
CiscoDRFLocal Cisco DRF Local
CiscoDRFMaster Cisco DRF Master
CiscoLicenseMgr Cisco License Manager
CiscoSyslogSubAgt Cisco Syslog Agent
dblrpc A Cisco DB Replicator
dbmon Cisco Database Layer Monitor
EspConfigAgent Cisco Config Agent
hostagt Host Resources Agent
interClusterSyncAgent Cisco Intercluster Sync Agent
jds Cisco XCP Directory Service
LpmTool Cisco Log Partition Monitoring Tool
ma Cisco XCP Message Archiver
Mib2agt MIB2 Agent
oamagent Cisco OAM Agent
replWatcher Cisco Replication Watcher
RisDC Cisco RIS Data Collector
rtmtreporter Cisco Serviceability Reporter
sappagt System Application Agent
snmpdm SNMP Master Agent
srm Cisco Server Recovery Manager
syncAgent Cisco Sync Agent
tc Cisco XCP Text Conference Manager
tracecollectionservice Cisco Trace Collection Service
ttlogin Cisco Login Datastore
ttreg Cisco SIP Registration Datastore
ttroute Cisco Route Datastore
ttsoft Cisco Presence Datastore
xcpConfigManager Cisco XCP Config Manager

High virtual memory issues

When experiencing high Virtual Memory usage on IM and Presence, Cisco recommends that you monitor the VmSize of the following processes:
  • tomcat
  • jabberd
  • pe
  • all of the Connection Manager processes (cm, cm_web, cm_sip_fed & cm_xmpp_fed)
  • all of the sipd processes
  • all of the cmoninit processes