Cisco Unified Real-Time Monitoring Tool Administration Guide for Cisco Unified Contact Center Express and Cisco Unified IP IVR Release 9.0(1)
Performance monitoring
Downloads: This chapterpdf (PDF - 470.0 KB) The complete bookPDF (PDF - 2.35 MB) | Feedback

Performance monitoring

Performance monitoring

Unified CCX provides Performance counters (called PerfMon counters) for application performance monitoring. The PerfMon counters help expose critical performance values, making it possible to track application performance in real time.

The PerfMon counters contain counter-based information, such as the name and index of the counter, the scale, the type, sub-counters to set when setting a counter, the current values, and a map containing counter instance data. Each performance counter instance object contains instance-based data, like the instance ID and current values.

You can monitor the performance of the components of the system and the components for the application on the system by choosing the counters for any object by using RTMT. The counters for each object display when the folder expands.

You can log perfmon counters locally on the computer and use the performance log viewer in RTMT to display the perfmon CSV log files that you collected or the Realtime Information Server Data Collection (RISDC) perfmon logs.

This chapter contains information on the following topics:

RTMT for performance monitoring

RTMT displays performance information for all the Unified CCX components, along with all the System Components.

RTMT provides alert notifications for troubleshooting performance. It also periodically polls performance counter to display data for that counter. Refer to Display counter description for examples on displaying perfmon counters in a chart or table format.

Performance monitoring allows you to perform the following tasks:

  • Continuously monitor a set of preconfigured objects AND receive notification in the form of an email message.
  • Associate counter threshold settings to alert notification. An email or popup message provides notification to the administrator.
  • Save and restore settings, such as counters being monitored, threshold settings, and alert notifications, for customized troubleshooting tasks.
  • Display up to six perfmon counters in one chart for performance comparisons.
  • Use performance queries to add a counter to monitor. See Performance Queries for more information.

Performance counters

RTMT contains ready-to-view, predefined performance counters. You can also select and add counters to monitor in RTMT.

RTMT displays performance counters in chart or table format. Chart format looks like a miniature window of information. You can display a particular counter by double clicking the counter in the perfmon monitoring pane.

Attributes for predefined performance counters, such as format and category, remain fixed. You can define attributes for counters that you configure in RTMT. Because chart view represents the default, you configure the performance counters to display in table format when you create a category.

Category tabs

A category comprises a group of monitored performance counters. A tab in the RTMT monitoring pane contains the category name. All performance counters that are monitored in this tab belong to a category. RTMT displays any categories that you access during a RTMT session in the bottom toolbar.

The system polls the performance counters in the tab at the same rate, with each category configured to have its own polling rate.

You can create custom categories in the RTMT monitoring pane to view information that helps you troubleshoot specific performance, system, or device problems. If your system is experiencing performance problems with specific objects, create custom categories to monitor the performance of the counters within the object. In addition, you can create alert notifications for counters in these custom categories. To create custom categories, you add a new category tab. When the tab is created, you specify the specific performance counters, devices, and alerts within that tab and then save your custom category by using Profile.

Sample Rate

The application polls the counters to gather status information.

The polling rate in each precanned monitoring window remains fixed, and the default value specifies 30 seconds. If the collecting rate for the AMC (Alert Manager and Collector) service parameter changes, the polling rate in the precanned window also updates. In addition, the local time of the RTMT client application and not the backend server time, provides the basis for the time stamp in each chart. For more information on Service Parameters, refer to Cisco Unified Contact Center Express Serviceability Administration Guide.

In the RTMT monitoring pane, you configure the polling intervals for the applicable performance counters for each category tab that you create.


High-frequency polling rate affects the performance on the server. The minimum polling rate for monitoring a performance counter in chart view equals 5 seconds; the minimum rate for monitoring a performance counter in table view equals 1 second. The default for both specifies 10 seconds.

Check perfmon counters

To get a closer look at perfmon counters, you can zoom a perfmon monitor counter in the RTMT. See also Highlight charts and graphs.

    Step 1   To zoom in a counter, perform one of the following tasks:
    1. To zoom predefined objects, such as System Summary, perform one of the following tasks:
      • Drag the mouse over the plot area in the counter to frame the data and release the mouse button. The counter zooms in the chart.
      • Click the counter. The counter zooms in.
    2. To zoom counters in the Performance pane, perform one of the following tasks (and resize the window, if necessary):
      • Double-click the counter that you want to zoom. The box with the counter appears highlighted and the Zoom window launches. The minimum, maximum, average, and last fields show the values for the counter since the monitoring began for the counter.
      • Click the counter to select the counter to zoom. The box with the counter appears highlighted. Right-click the counter and select Zoom Chart or choose System > Performance > Zoom Chart. The Zoom window launches. The minimum, maximum, average, and last fields show the values for the counter since the monitoring began for the counter.
    Step 2   To zoom out a counter, perform one of the following tasks:
    1. To zoom out predefined objects, such as System Summary, click the counter and press Z in the active counter to return the counter to original size.
    2. To zoom out counters in the Performance pane, click OK to close the Zoom window.

    Highlight charts and graphs

    The highlight feature helps to distinguish hosts and counters when multiple nodes or counters display on color-coded graphs. This feature is active in the System Summary, CPU and Memory, Disk Usage, and Performance Log Viewer windows. See also Check perfmon counters.

      Step 1   To highlight charts and graphs, perform one of the following tasks:
      1. To highlight charts and graphs for predefined objects, such as System Summary, right-click in a plot area to highlight the nearest data series or point.
      2. To highlight charts and graphs in the Performance Log Viewer, perform one of the following tasks
        • Right-click on any color code in the table below the chart in the Performance Log Viewer and choose Highlight to highlight the data series for that counter.
        • Right-click on any color code in the table below the chart in the Performance Log Viewer and choose Change Color to select a different color for the counter.
      Step 2   To return a highlighted item to its original appearance in the Performance Log Viewer, select another item to highlight.

      Counter properties

      Counter properties allow you to display a description of the counter and configure data-sampling parameters.

      The Counter Property window contains the option to configure data samples for a counter. The performance counters that display in the RTMT performance monitoring pane contain green dots that represent samples of data over time. You can configure the number of data samples to collect and the number of data points to show in the chart. After the data sample is configured, view the information by using the View All Data/View Current Data menu option to view all the data that a perfmon counter collected.

      Related Information

      Alert notification

      Using the alert notification feature, the application notifies you of system problems. Perform the following configuration setup to activate alert notifications for a system counter:

      • From the RTMT Perfmon Monitoring pane, choose the system perfmon counter.
      • Set up an email or a message popup window for alert notification.
      • Determine the threshold for the alert (for example, an alert activates when calls in progress exceed the threshold of over 100 calls or under 50 calls).
      • Determine the frequency of the alert notification (for example, the alert occurs once or every hour).
      • Determine the schedule for when the alert activates (for example, on a daily basis or at certain times of the day).

      Perfmon logs

      You can log perfmon counters locally on the computer and use the performance log viewer in RTMT to display the perfmon CSV log files that you collected or the Realtime Information Server Data Collection (RISDC) perfmon logs.

      RTMT allows you to choose different perfmon counters to log locally. You can then view the data from the perfmon CSV log by using the performance log viewer.

      See View and troubleshoot perfmon logs for more information.

      Troubleshooting Perfmon Data Logging

      The troubleshooting perfmon data logging feature assists Cisco TAC in identifying system problems. When you enable troubleshooting perfmon data logging, you initiate the collection of a set of the applicable Unified CCX and operating system performance statistics on the selected server. The statistics that are collected include comprehensive information that can be used for system diagnosis.

      The system automatically enables troubleshooting perfmon data logging to collect statistics from a set of perfmon counters that provides comprehensive information on the system state. When Troubleshooting Perfmon Data Logging is enabled, Cisco estimates that the system experiences a less than 5-percent increase in CPU utilization and an insignificant increase in the amount of memory that is being used, and it writes approximately 50 MB of information to the log files daily.

      You can perform the following administrative tasks with the troubleshooting perfmon data logging feature:

      • Enable and disable the trace filter for Troubleshooting perfmon data logging.
      • Monitor the applicable set of predefined System, Unified CCX performance objects and counters on each server.
      • Log the monitored performance data in CSV file format on the server in the active log partition in the var/log/active/cm/log/ris/csv directory. The log file uses the following naming convention: PerfMon_<server>_<month>_<day>_<year>_<hour>_<minute>.csv; for example, PerfMon_172.19.240.80_06_15_2005_11_25.csv. Specify the polling rate. This rate specifies the rate at which performance data gets gathered and logged. You can configure the polling rate down to 5 seconds. Default polling rate equals 15 seconds.
      • View the log file in graphical format by using the Microsoft Windows performance tool or by using the Performance Log viewer in the RTMT.
      • Specify the maximum number of log files that will be stored on disk. Log files exceeding this limit get purged automatically by removal of the oldest log file. The default specifies 50 files.
      • Specify the rollover criteria of the log file based on the maximum size of the file in megabytes. The default value specifies 5 MB.
      • Collect the Cisco RIS Data Collector PerfMonLog log file by using the Trace & Log Central feature of the RTMT or Command Line Interface.

      For more information on configuring Troubleshooting Perfmon Data Logging, see Configure Troubleshooting Perfmon Data Logging.

      The troubleshooting perfmon data-logging feature collects information from the following counters within the following perfmon objects.

      Refer to the System performance objects and counters for a description of the system counters:

      • Database Change Notification Server Object
        • Clients
        • CNProcessed
        • QueueDelay
        • QueuedRequestsInDB
        • QueuedRequestsInMemory
      • Database Local DSN Object
        • CcmDbSpace_Used
        • CcmtempDbSpace_Used
        • CNDbSpace_Used
        • LocalDSN
        • RootDbSpace_Used
        • SharedMemory_Free
        • SharedMemory_Used
      • Enterprise Replication DBSpace Monitors Object
        • ERDbSpace_Used
        • ERSBDbSpace_Used
      • IP Object
        • In Receives
        • In HdrErrors
        • In UnknownProtos
        • In Discards
        • In Delivers
        • Out Requests
        • Out Discards
        • Reasm Reqds
        • Reasm Oks
        • Reasm Fails
        • Frag OKs
        • Frag Fails
        • Frag Creates
        • InOut Requests
      • Memory Object
        • % Page Usage
        • % VM Used
        • % Mem Used
        • Buffers Kbytes
        • Cached Kbytes
        • Free Kbytes
        • Free Swap Kbytes
        • High Free
        • High Total
        • Low Total
        • Low Free
        • Page Faults Per Sec
        • Page Major Faults Per Sec
        • Pages
        • Pages Input
        • Pages Input Per Sec
        • Pages Output
        • Pages Output Per Sec
        • SlabCache
        • SwapCached
        • Shared Kbytes
        • Total Kbytes
        • Total Swap Kbytes
        • Total VM Kbytes
        • Used Kbytes
        • Used Swap Kbytes
        • Used VM Kbytes
      • Network Interface Object
        • Rx Bytes
        • Rx Packets
        • Rx Errors
        • Rx Dropped
        • Rx Multicast
        • Tx Bytes
        • Tx Packets
        • Tx Errors
        • Tx Dropped
        • Total Bytes
        • Total Packets
        • Tx QueueLen
      • Number of Replicates Created and State of Replication Object
        • Replicate_State
      • Partition Object
        • % CPU Time
        • %Used
        • Await Read Time
        • Await Time
        • Await Write Time
        • Queue Length
        • Read Bytes Per Sec
        • Total Mbytes
        • Used Mbytes
        • Write Bytes Per Sec
      • Process Object
        • % Memory Usage
        • Data Stack Size
        • Nice
        • PID
        • STime
        • % CPU Time
        • Page Fault Count
        • Process Status
        • Shared Memory Size
        • VmData
        • VmRSS
        • VmSize
        • Thread Count
        • Total CPU Time Used
      • Processor Object
        • Irq Percentage
        • Softirq Percentage
        • IOwait Percentage
        • User Percentage
        • Nice Percentage
        • System Percentage
        • Idle Percentage
        • %CPU Time
      • System Object
        • Allocated FDs
        • Freed FDs
        • Being Used FDs
        • Max FDs
        • Total Processes
        • Total Threads
        • Total CPU Time
      • TCP Object
        • Active Opens
        • Passive Opens
        • Attempt Fails
        • Estab Resets
        • Curr Estab
        • In Segs
        • Out Segs
        • Retrans Segs
        • InOut Segs
      • Thread Object—This object is currently not supported on Unifed CCX.
        • %CPU Time

      Refer to the Performance Objects and Counters for Unified CCX for a description of the Unified CCX counters.