Table Of Contents
Understanding Performance Monitoring
Using RTMT for Performance Monitoring
Category Tabs
Sample Rate
Adding Counters to Monitor
Alert Notification for Counters
Zoom Counter
Counter Properties
Troubleshooting Perfmon Data Logging
Related Topics
Understanding Performance Monitoring
Cisco Unified Communications Manager directly updates Performance counters (called PerfMon counters). The counters contain simple, useful information on the system and devices on the system, such as number of registered phones, number of active calls, and number of available conference bridge resources.
The Cisco CallManager object contains most of the Cisco Unified Communications Manager performance counters, and these counters have only one instance. The instance-based counters that belong to the other objects can have zero or more instances. For example, if two phones are registered to Cisco Unified Communications Manager, two instances of each counter that belong to the Cisco phones object exist.
You can monitor the performance of the components of the system and the components for the application on the system by choosing the counters for any object by using RTMT. The counters for each object display when the folder expands.
You can log perfmon counters locally on the computer and use the performance log viewer in RTMT to display the perfmon CSV log files that you collected or the Realtime Information Server Data Collection (RISDC) perfmon logs.
This chapter contains information on the following topics:
•
Using RTMT for Performance Monitoring
•Troubleshooting Perfmon Data Logging
Using RTMT for Performance Monitoring
RTMT integrates with the administration and serviceability software for both Cisco Unified Communications Manager and Cisco Unity Connection. RTMT displays performance information for all Cisco Unified Communications Manager, Cisco Unity Connection Administration, and Connection components. RTMT provides alert notification for troubleshooting performance. It also periodically polls performance counter to display data for that counter. Refer to "Displaying a Counter Description" section on page 5-7 for examples on displaying perfmon counters in a chart or table format.
Perfmon monitoring allows you to perform the following tasks:
•Monitor Cisco Unified Communications Manager servers, TFTP servers, and database servers.
•Continuously monitor a set of preconfigured objects AND receive notification in the form of an e-mail message.
•Associate counter threshold settings to alert notification. An e-mail or popup message provides notification to the administrator.
•Save and restore settings, such as counters being monitored, threshold settings, and alert notifications, for customized troubleshooting tasks.
•Display up to six perfmon counters in one chart for performance comparisons.
RTMT displays performance counters in chart or table format. Chart format looks like a miniature window of information. Up to six charts display in the RTMT performance monitoring pane for each category tab that you create. You can display a particular counter by double clicking the counter in the perfmon monitoring pane. Because chart view represents the default, you configure the performance counters to display in table format when you create a category.
You can remove a counter chart (table entry) with the Remove Chart/TableEntry menu item in the Perfmon menu in the menu bar.
Tip The polling rate in each precanned monitoring window remains fixed, and the default value specifies 30 seconds. If the collecting rate for the AMC (Alert Manager and Collector) service parameter changes, the polling rate in the precanned window also updates. In addition, the local time of the RTMT client application and not the backend server time, provides the basis for the time stamp in each chart.
For more information on Service Parameters, refer to Cisco Unified Communications Manager Administration Guide.
See the following sections for configuration options in the RTMT perfmon monitoring pane:
•Category Tabs
•Sample Rate
•Using RTMT for Performance Monitoring
•Zooming a Counter, page 5-6
Category Tabs
A category comprises a group of monitored performance counters. A tab in the RTMT monitoring pane contains the category name. All performance counters that are monitored in this tab belong to a category. The system polls the performance counters in the tab at the same rate, with each category configured to have its own polling rate.
You can create custom categories in the RTMT monitoring pane to view information that helps you troubleshoot specific performance, system, or device problems. If your system is experiencing performance problems with specific objects, create custom categories to monitor the performance of the counters within the object. If the system is experiencing problems with specific devices, create custom categories to monitor the devices in your system. In addition, you can create alert notifications for counters and gateways in these custom categories. To create custom categories, you add a new category tab. When the tab is created, you specify the specific performance counters, devices, and alerts within that tab and then save your custom category by using Profile.
Sample Rate
Note The application polls counters, devices, and gateway ports to gather status information. In the RTMT monitoring pane, you configure the polling intervals for the performance counters, devices, and gateway ports for each category tab that you create. High-frequency polling rate affects the performance on the server. The minimum polling rate for monitoring a performance counter in chart view equals 5 seconds; the minimum rate for monitoring a performance counter in table view equals 1 second. The default for both specifies 10 seconds.
Adding Counters to Monitor
To troubleshoot system performance problems, you add the counter that is associated with the perfmon object to the RTMT performance monitoring pane, which displays a chart for the counter. Before you add counters, see the "Category Tabs" section.
Category tabs contain up to six perfmon counter charts.
Alert Notification for Counters
Using the alert notification feature, the application notifies you of system problems. Perform the following configuration setup to activate alert notifications for a system counter:
•From the RTMT Perfmon Monitoring pane, choose the system perfmon counter.
•Set up an e-mail or a message popup window for alert notification.
•Determine the threshold for the alert (for example, an alert activates when calls in progress exceed the threshold of over 100 calls or under 50 calls).
•Determine the frequency of the alert notification (for example, the alert occurs once or every hour).
•Determine the schedule for when the alert activates (for example, on a daily basis or at certain times of the day).
Zoom Counter
To get a closer look at performance monitors, zoom the monitor counter in the RTMT perfmon monitoring pane by highlighting the counter chart and choosing System > Performance > Zoom Chart.
Counter Properties
Counter properties allow you to display a description of the counter and configure data-sampling parameters.
The Counter Property window contains the option to configure data samples for a counter. The performance counters that display in the RTMT performance monitoring pane contain green dots that represent samples of data over time. You can configure the number of data samples to collect and the number of data points to show in the chart. After the data sample is configured, view the information by using the View All Data/View Current Data menu option to view all the data that a perfmon counter collected.
Additional Information
See the "Related Topics" section.
Troubleshooting Perfmon Data Logging
The troubleshooting perfmon data logging feature assists Cisco TAC in identifying system problems. When you enable troubleshooting perfmon data logging, you initiate the collection of a set of Cisco Unified Communications Manager, Cisco Unity, and operating system performance statistics on the selected server. The statistics that are collected include comprehensive information that can be used for system diagnosis.
The system automatically enables troubleshooting perfmon data logging to collect statistics from a set of perfmon counters that provides comprehensive information on the system state. When Troubleshooting Perfmon Data Logging is enabled, Cisco estimates that the system experiences a less than 5-percent increase in CPU utilization and an insignificant increase in the amount of memory that is being used, and it writes approximately 50 MB of information to the log files daily.
You can perform the following administrative tasks with the troubleshooting perfmon data logging feature:
•Enable and disable the trace filter for Troubleshooting perfmon data logging.
•Monitor a set of predefined System, Cisco Unified Communications Manager, and Cisco Unity performance objects and counters on each server.
•Log the monitored performance data in CSV file format on the server in the active log partition in the var/log/active/cm/log/ris/csv directory. The log file uses the following naming convention: PerfMon_<node>_<month>_<day>_<year>_<hour>_<minute>.csv; for example, PerfMon_172.19.240.80_06_15_2005_11_25.csv. Specify the polling rate. This rate specifies the rate at which performance data gets gathered and logged. You can configure the polling rate down to 5 seconds. Default polling rate equals 15 seconds.
•View the log file in graphical format by using the Microsoft Windows performance tool or by using the Performance Log viewer in the Real-Time Monitoring Tool.
•Specify the maximum number of log files that will be stored on disk. Log files exceeding this limit get purged automatically by removal of the oldest log file. The default specifies 50 files.
•Specify the rollover criteria of the log file based on the maximum size of the file in megabytes. The default value specifies 2 MB.
•Collect the Cisco RIS Data Collector PerfMonLog log file by using the Trace & Log Central feature of the Real-Time Monitoring Tool or Command Line Interface.
For more information on configuring Troubleshooting Perfmon Data Logging, see "Configuring Troubleshooting Perfmon Data Logging" section on page 5-12.
The troubleshooting perfmon data-logging feature collects information from the following counters within the following perfmon objects.
Refer to the Performance Objects and Counters for Cisco Unified Communications Manager for a description on the counters:
•Cisco CallManager Object
–Communications ManagerHeartBeat
–CallsActive
–CallsAttempted
–CallsCompleted
–InitializationState
–RegisteredHardwarePhones
–RegisteredMGCPGateway
•Cisco CallManager System Performance Object
–AverageExpectedDelay
–CallsRejectedDueToThrottling
–CodeRedEntryExit
–CodeYellowEntryExit
–QueueSignalsPresent 1-High
–QueueSignalsPresent 2-Normal
–QueueSignalsPresent 3-Low
–QueueSignalsPresent 4-Lowest
–QueueSignalsProcessed 1-High
–QueueSignalsProcessed 2-Normal
–QueueSignalsProcessed 3-Low
–QueueSignalsProcessed 4-Lowest
–QueueSignalsProcessed Total
–SkinnyDevicesThrottled
–ThrottlingSampleActivity
–TotalCodeYellowEntry
•Cisco SIP Stack Object
–CCBsAllocated
–SCBsAllocated
–SIPHandlerSDLQueueSignalsPresent
•Cisco TFTP
–BuildAbortCount
–BuildCount
–BuildDeviceCount
–BuildDialruleCount
–BuildDuration
–BuildSignCount
–BuildSoftkeyCount
–BuildUnitCount
–ChangeNotifications
–DeviceChangeNotifications
–DialruleChangeNotifications
–EncryptCount
–GKFoundCount
–GKNotFoundCount
–HeartBeat
–HttpConnectRequests
–HttpRequests
–HttpRequestsAborted
–HttpRequestsNotFound
–HttpRequestsOverflow
–HttpRequestsProcessed
–HttpServedFromDisk
–LDFoundCount
–LDNotFoundCount
–MaxServingCount
–Requests
–RequestsAborted
–RequestsInProgress
–RequestsNotFound
–RequestsOverflow
–RequestsProcessed
–SegmentsAcknowledged
–SegmentsFromDisk
–SegmentsSent
–SEPFoundCount
–SEPNotFoundCount
–SIPFoundCount
–SIPNotFoundCount
–SoftkeyChangeNotifications
–UnitChangeNotifications
•Process Object
–PID
–STime
–% CPU Time
–Page Fault Count
–Process Status
–VmData
–VmRSS
–VmSize
–Thread Count
•Memory Object
–Used Kbytes
–Free Kbytes
–Total Kbytes
–Shared Kbytes
–Buffers Kbytes
–Cached Kbytes
–Free Swap Kbytes
–Total Swap Kbytes
–Used Swap Kbytes
–Pages Input
–Pages Output
–Pages
–Used VM Kbytes
–Total VM Kbytes
–% Page Usage
–% VM Used
–% Mem Used
•Processor Object
–Irq Percentage
–Softirq Percentage
–IOwait Percentage
–User Percentage
–Nice Percentage
–System Percentage
–Idle Percentage
–%CPU Time
•Thread Object—Troubleshooting Perfmon Data Logger only logs Cisco Unified Communications Manager threads.
–%CPU Time
•Partition Object
–Used Mbytes
–Total Mbytes
–%Used
–Await Read Time
–Await Write Time
–Await Time
–% CPU Time
–Read Bytes Per Sec
–Write Bytes Per Sec
–Queue Length
•IP Object
–In Receives
–In HdrErrors
–In UnknownProtos
–In Discards
–In Delivers
–Out Requests
–Out Discards
–Reasm Reqds
–Reasm Oks
–Reasm Fails
–Frag OKs
–Frag Fails
–Frag Creates
–InOut Requests
•TCP Object
–Active Opens
–Passive Opens
–Attempt Fails
–Estab Resets
–Curr Estab
–In Segs
–Out Segs
–Retrans Segs
–InOut Segs
•Network Interface Object
–Rx Bytes
–Rx Packets
–Rx Errors
–Rx Dropped
–Rx Multicast
–Tx Bytes
–Tx Packets
–Tx Errors
–Tx Dropped
–Total Bytes
–Total Packets
–Tx QueueLen
•System Object
–Allocated FDs
–Freed FDs
–Being Used FDs
–Max FDs
–Total Processes
–Total Threads
–Total CPU Time
Refer to the "Performance Objects and Counters for Cisco Unity Connection" appendix for a description on the following counters:
•CUC Data Store
–Allocated Memory [kb]
–Buffer Cache Hit Ratio
–Buffer Reads
–Buffer Waits
–Buffer Writes
–Cache Reads/second
–Cache Writes/second
–Checkpoint Waits/second
–Compilations/second
–Data Path Scans
–Database Connections
–Deadlocks/second
–Disk Free/chunk [kb]
–Disk Reads
–Disk Reads/chunk
–Disk Reads/second
–Disk Writes
–Disk Writes/chunk
–Disk Writes/second
–Index Read-aheads
–Index to Data Read-aheads
–Latch Requests/second
–Latch Wait Time [ms]
–Latch Waits/second
–Lock Requests/second
–Lock Wait Time [ms]
–Lock Waits/second
–Page Reads
–Page Reads/second
–Page Writes
–Page Writes/second
–PDQ Calls
–Percent Read Cache
–Percent Write Cache
–Read-ahead Pages Used
–Shared Memory [kb]
–Temporary Space Used [kb]
–Virtual Processor: I/O Reads/second
–Virtual Processor: User Time [ms]
•CUC Data Store: Databases
–Disk Free/chunk [kb]
–Disk Reads/chunk
–Disk Writes/chunk
–Transactions/second
•CUS Data Store: Sessions
–Active Transactions
–Log File Used Size [kb]
–Threads
•CUC Diagnostics
–Diagnostic Data Written [kb]
•CUC Directory Services
–Directory Search Duration [s]
–Directory Search Duration Average [s]
–Directory Searches
–New Message Events
–Read Message Events
•CUC Message Store
–Bad Mail Total
–Delivery Receipts Total
–Failures Total
–Message Size Average [kb]
–Messages Delivered Total
–Messages Received Total
–Non-delivery Receipts Total
–Queued Messages Current
–Read Receipts Total
–Retries Total
–VPIM Contacts Added Total
–VPIM Contacts Deleted Total
–VPIM Contacts Modified Total
–VPIM Message Decode Duration [s]
–VPIM Message Decode Duration Average [s]
–VPIM Message Encode Duration [s]
–VPIM Message Encode Duration Average [s]
–VPIM Message Send Queue
–VPIM Messages Delivered Total
–VPIM Messages Received Total
–VPIM Messages Total
•CUC Personal Call Transfer Rules
–Applicable Rule Found
–Destinations Tried
–PCTR Calls
–Rules Evaluated
–Subscriber Reached
–Transfer Failed
–Voicemail Reached
•CUC Phone System
–AGC Percent Average
–Call Count Current
–Call Count Total
–Call Duration Average [s]
–Call Duration Total [s]
–Calls Unanswered Total
–Incoming Calls CFB Current
–Incoming Calls CFB Total
–Incoming Calls CFNA Current
–Incoming Calls CFNA Total
–Incoming Calls Current
–Incoming Calls Direct Current
–Incoming Calls Direct Total
–Incoming Calls Duration Average [s]
–Incoming Calls Duration Total [s]
–Incoming Calls No Info Total
–Incoming Calls Total
–MWI Request Duration Average [ms]
–MWI Request Duration Total [ms]
–MWI Requests Failed Total
–MWI Requests Total
–Notification Duration Average [s]
–Notification Duration Total [s]
–Notifications Failed
–Notifications Total
–Outgoing Calls Duration Average [s]
–Outgoing Calls Duration Total [s]
–Outgoing Calls Release Transfers Completed
–Outgoing Calls Release Transfers Failed
–Outgoing Calls Release Transfers Total
–Outgoing Calls Supervised Transfers Completed
–Outgoing Calls Supervised Transfers Dropped
–Outgoing Calls Supervised Transfers Failed
–Outgoing Calls Supervised Transfers Total
–Outgoing Calls Transfers Total
–Pager Notifications Duration Average [s]
–Pager Notifications Duration Total [s]
–Pager Notifications Failed
–Pager Notifications Total
–Port Idle Duration [s]
–Port Idle Duration Average [s]
–Ports Idle Current
–Ports In Use Current
–Ports Locked
–SMS Notifications Duration Average [s]
–SMS Notifications Duration Total [s]
–SMS Notifications Failed
–SMS Notifications Total
–SMTP Notifications Duration Average [s]
–SMTP Notifications Duration Total [s]
–SMTP Notifications Failed
–SMTP Notifications Total
•CUC Phone System: Ports
–Port Calls
–Port Idle Percent
–Port Usage Duration Average [s]
–Port Usage Duration Total [s]
–Port Usage Percent
•CUC Sessions: E-mail Access
–Delay - Authentication [ms]
–Delay - Expunge [ms]
–Delay - Fetch Body [ms]
–Delay - Fetch Message Information [ms]
–Delay - Search [ms]
–Delay - Set Flag [ms]
–Messages Read - Total
–Session Duration Average [ms]
–Session Duration Total [ms]
–Sessions - Current
–Sessions - Total
•CUC Sessions: IMAP Server
–Commands/second Average
–Connection Length Average [s]
–Errors Total
–EXAMINE Requests Total
–Failed Login Requests Total
–FETCH Requests Total
–Login Requests Total
–Logout Requests Total
–Messages Read Total
–Messages Read/hour
–Messages/fetch Average
–NOOP Requests Total
–Response Time Average [ms]
–Socket Connections Current
–Socket Connections Total
–STARTTLS Requests Total
–STATUS Requests Total
–TLS Connections Current
–TLS Errors Total
–Unsolicited Notify Response Time Average [ms]
–Unsolicited Notify Responses Total
•CUC Sessions: TRaP
–Session Duration Average [s]
–Session Duration Total [s]
–Sessions Current
–Sessions Total
•CUC Sessions: TTS
–Session Duration Average [s]
–Session Duration Total [s]
–Sessions Current
–Sessions Total
•CUC Sessions: Unified Client
–Connections Current
–Connections Total
•CUC Sessions: Voice
–Delay - Directory Search [ms]
–Delay - Opening Greeting [ms]
–Delay - Subscriber Delete Message [ms]
–Delay - Subscriber Logon [ms]
–Delay - Subscriber Message Access [ms]
–Delay - Subscriber Message Count [ms]
–Delay - Subscriber Message Header [ms]
–Delay - Subscriber Record Message [ms]
–Failsafes Total
–G.711 Sessions Current
–G.711 Sessions Total
–G.729 Sessions Current
–G.729 Sessions Total
–Messages Deleted
–Messages Forwarded
–Messages Read
–Messages Replied
–Messages Sent
–MRCP Define Grammar Delay [ms]
–MRCP Define Grammar Delay Average [ms]
–MRCP Define Grammar Delay Max [ms]
–MRCP Delay [ms]
–MRCP Delay Average [ms]
–MRCP Delay Max [ms]
–Prompts Bytes
–Prompts/minute
–Record Bytes
–Records/minute
–Session Duration Average/call [s]
–Session Duration Total [s]
–Sessions Current
–Sessions Total
–Subscriber lookup delay [ms]
–TUI Accounts Locked
•CUC Sessions: VUI
–Delay - Subscriber Message Access [ms]
–Matches Total
–Messages Deleted
–Messages Forwarded
–Messages Read
–Messages Replied
–Messages Sent
–No-matches Total
–Session Duration Average/call [s]
–Session Duration Total [s]
–Sessions Current
–Sessions Total
•CUC Sessions: Web
–CPCA Authentication Delay [s]
–CPCA Authentication Delay Max [s]
–CPCA Failed Authentications Total
–CPCA Messages Deleted Total
–CPCA Messages Forwarded Total
–CPCA Messages Read Total
–CPCA Messages Replied Total
–CPCA Messages Sent Total
–CPCA Pages Saved Total
–CPCA Pages Served Total
–CPCA Requests In Queue Total
–CPCA Server Busy Pages Total
–CPCA Sessions Current
–CPCA Sessions Total
–CUCA Authentication Delay [s]
–CUCA Authentication Delay Max [s]
–CUCA Failed Authentications Total
–CUCA Pages Saved Total
–CUCA Response Time [ms]
–CUCA Response Time Max [ms]
–CUCA Sessions Current
–CUCA Sessions Total
Related Topics
•Using RTMT for Performance Monitoring
•Troubleshooting Perfmon Data Logging
•Configuring and Displaying Performance Counters, page 5-1
•Performance Objects and Counters for the System, page A-1
•Performance Objects and Counters for Cisco Unified Communications Manager, page B-1
•Performance Objects and Counters for Cisco Unity Connection, page C-1
Feedback
