Cisco Unity Connection Troubleshooting Guide, Release 1.x
Diagnostic Traces and Event Logs
Downloads: This chapterpdf (PDF - 183.0KB) The complete bookPDF (PDF - 2.77MB) | Feedback

Diagnostic Traces and Event Logs

Table Of Contents

Diagnostic Traces and Event Logs

About the Diagnostic Utilities for Cisco Unity Connection

Event Log

Macro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

Available Macro Traces

Enabling Macro Traces

Micro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

Available Micro Traces

Enabling Micro Traces

Viewing Individual Micro Trace Logs

How to Interpret Micro Trace Information in the Cisco Unity Diagnostic Tool (UDT)

Gathering Micro Trace Logs into Files

Dr. Watson Logs

Reporting Problems to Cisco TAC

System Information

Problem Description


Diagnostic Traces and Event Logs


In this chapter, you will find information about the diagnostic utilities that enable you to troubleshoot problems and to maintain Cisco Unity Connection, and instructions that will help you in reporting problems to Cisco Technical Assistance Center (Cisco TAC).

See the following sections:

About the Diagnostic Utilities for Cisco Unity Connection

Event Log

Macro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

Micro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

Dr. Watson Logs

Reporting Problems to Cisco TAC

About the Diagnostic Utilities for Cisco Unity Connection

Table 1-1 describes the diagnostic utilities available for Cisco Unity Connection.

Table 1-1 Diagnostic Utilities for Cisco Unity Connection 

Utility
Uses

Event log

The Event log should be the first resource you search for information when troubleshooting a problem. The Event log is used by Windows applications to report information events, warnings, and errors. Reviewing the Event log for Connection events provides a good overview of how the system is functioning.

For details, see the "Event Log" section.

Macro trace logs in the UDT

In the Cisco Unity Diagnostic tool (UDT), you can enable a preselected group of individual macro trace levels to obtain diagnostic trace output on several Connection components at once.

For details, see the "Macro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)" section.

Micro trace logs in the UDT

The Cisco Unity Diagnostic tool (UDT) can enable most Connection components to write diagnostic traces to a log. The diagnostic trace output is essential to troubleshooting problems that involve individual components.

For details, see the "Micro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)" section.

Dr. Watson logs

The Dr. Watson utility is invoked by Windows 2000 when a serious problem occurs that is not handled by Connection. When invoked, the Dr. Watson utility displays a dialog box that contains an error message (for example, "Dr. Watson encountering an error in the AvCsMgr.exe process"). Dr. Watson errors may occur in other processes as well.

For details, see the "Dr. Watson Logs" section.


Event Log

The Event log is the first resource you should search for information when troubleshooting a problem. Cisco Unity Connection components report information events, warnings, and errors in the Event log. Reviewing the Event log for Connection events provides a good overview of how the system is functioning.


Note The raw data within the files in the Event log is stamped with time stamps recorded in GMT (Greenwich mean time) rather than in the local time of the Connection server. The time stamps for the Event log files themselves, however, are in the local time of the Connection server. Using GMT for the time stamps of the raw data provides for an accurate comparison of events when Connection servers are not all in the same time zone. The Connection reports convert the GMT time stamps to local time.


To Obtain an Event Log Trace


Step 1 On the Windows Start menu, click Programs > Administrative Tools > Event Viewer.

Step 2 In the Tree pane, click Application.

Step 3 Search for Connection events.


Note For further instructions on Event Viewer functions, see the Event Viewer Help.



Macro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

The Cisco Unity Diagnostic tool (UDT) lets you create and view diagnostic trace logs for troubleshooting problems. Diagnostic trace logs of a problem that is occurring can be critical to determining the cause of the problem.

Macro traces in the UDT let you enable preselected groups of micro traces. For details on viewing, interpreting and gathering the micro traces that the macro traces use, see the "Micro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)" section.


Caution Diagnostic traces that are set before a Cisco Unity Connection software upgrade are not preserved and must be reset after the upgrade.

See the following sections:

Available Macro Traces

Enabling Macro Traces

Available Macro Traces

Table 1-2 lists the macro traces that are available and what each macro trace analyzes.

Table 1-2 Macro Traces 

Macro Trace Name
What the Trace Analyzes

Call Flow Diagnostics

The flow of a call through Connection

Message Objectid Tracking Traces

Message handing; the objects that handle messages from delivery to deletion

Call Control (Miu) Traces

Call control functions

Traces for MWI Problems

Turning message waiting indicators (MWIs) on and off

Traces for Other Notification Problems

Notification and outdial functions

Skinny TSP Traces

The Skinny networking layer; useful only when Connection is integrated with Cisco Unified CallManager

Unity Startup

Connection startup functions

Voice User Interface/Speech Recognition Traces

Voice User Interface (VUI)

Media (Wave) Traces 1 - High-Level

Basic media and WAV file usage

Media (Wave) Traces 2 - Medium-Level

Media and WAV file usage; logs more information than high-level traces

Media (Wave) Traces 3 - Low-Level

Media and WAV file usage; logs detailed information and should be used only when there is significant free hard drive space

Text to Speech (TTS) Traces

The Text to Speech (TTS) feature; also can log traces on other Connection components that interact with TTS


Enabling Macro Traces

Enable the macro trace diagnostics when you are troubleshooting problems with Cisco Unity Connection features. For example, if there are MWI problems, enable the Traces for MWI Problems macro trace. However, keep in mind that running diagnostics can affect system performance and hard drive space.

To Enable Macro Trace Diagnostics


Step 1 On the Windows Start menu, click Programs > Cisco Unity > Cisco Unity Diagnostic Tool. The Cisco Unity Diagnostic Viewer window appears.

Step 2 In the right pane of the Cisco Unity Diagnostic Viewer window, click Configure Macro Traces. The Configure Macro Traces wizard appears.

Step 3 On the Welcome page, click Next.

Step 4 On the Configure Macro Traces page, check the check boxes for the applicable traces.

Step 5 Click Next.

Step 6 On the Completing page, click Finish.

Step 7 In the right pane of the Cisco Unity Diagnostic Viewer window, click Start New Log Files.

Step 8 Reproduce the problem.


Note After obtaining the diagnostic trace logs that you want, disable the traces that you enabled.



Micro Trace Logs in the Cisco Unity Diagnostic Tool (UDT)

The Cisco Unity Diagnostic tool (UDT) lets you create and view diagnostic trace logs for troubleshooting problems. Diagnostic trace logs of a problem that is occurring can be critical to determining the cause of the problem.

Micro traces in the UDT let you enable specific Cisco Unity Connection components and trace levels, which makes the trace logs as precise as possible. This is particularly critical when the problem occurs seldom, such as only once a day, as it can be difficult to find the actual occurrence of the problem in a diagnostic trace log.


Caution Diagnostic traces that are set before a Cisco Unity Connection software upgrade are not preserved and must be reset after the upgrade.

See the following sections:

Available Micro Traces

Enabling Micro Traces

Viewing Individual Micro Trace Logs

How to Interpret Micro Trace Information in the Cisco Unity Diagnostic Tool (UDT)

Gathering Micro Trace Logs into Files

Available Micro Traces

Table 1-3 lists the micro traces that are available and describes what each micro trace analyzes.

Table 1-3 Micro Traces 

Micro Trace Name
What the Trace Analyzes

Address Searcher (Address Searcher)

Addressing user-to-user messages

Arbiter (Arbiter)

Conversations, ports, and call routing rules that are used for calls

Bulk Administration Manager (BulkAdministrationManager)

Bulk Administration Manager for creating, updating, and deleting multiple users or system contacts

Certificate Manager (CuCertMgr)

Private secure messaging

Client Data Library (CDL)

 

Common Messaging Layer (CML)

 

CommServer Manager (AvCsMgr)

Main Cisco Unity Connection process; starting and stopping Connection

ConfigData (ConfigData)

 

Conversation Development Environment (CDE)

Conversation engine and conversation events

Database SysAgent Tasks (DataSysAgentTasks)

SysAgent tasks

DbEvent Tasks (DbEvent)

Component notification of database changes

Encryption Library (CuEncrypt)

Encryption (except for messaging) and the encryption audit logs

Failure Conversation (FailureConv)

 

GAL: Cache (CuGalCach)

 

GAL: Data (CuGalData)

 

GAL: Distributed Authoring and Versioning (CuGalDav)

 

GAL: SQL (CuGalSql)

 

GAL: Test (CuGalTest)

 

Gateway (AvCsGateway)

Starting and stopping Cisco Unity Connection; access to AvCsMgr; access to Cisco Unity Connection components

Groupware Access Library (CuGal)

 

Licensing (Licensing)

Licensing for per-seat licensed features

Log Manager (AvLogMgr)

Writing diagnostic traces and Event log

Media: Call (MiuCall)

The process between the Miu and conversations

Media: COM Methods (MiuMethods)

Handing of incoming calls; call control; turning messaging waiting indicators (MWIs) on and off; notification and outdial functions; media or WAV file usage

Media: Database (MiuDatabase)

 

Media: General (MiuGeneral)

Tracking calls through the telephone user interface (TUI); call control functions; turning message waiting indicators (MWIs) on and off; notification and outdial functions; basic media or WAV file usage

Media: Input/Output (MiuIO)

Media or WAV file usage with TAPI (circuit-switched or Cisco Unified CallManager) integrations

Media: Integration (MiuIntegration)

Integrations with circuit-switched phone systems; call information in integrations with circuit-switched phone systems; turning message waiting indicators (MWIs) on and off in integrations with circuit-switched phone systems

Media: SC Bus (MiuSCBus)

Fax engine and fax tone detection

Message Transfer Agent (MTA)

Delivery of voice messages to the message store

Notifier and Notification Devices (Notifier)

Notification of messages and selected events; turning message waiting indicators (MWIs) on and off

Performance Monitor (PerfMonitor)

Performance of system objects that Cisco Unity Connection uses

PHGreeting Conversations (ConvPH Greeting)

Opening greeting and user greetings

PHInterview Conversations (ConvPH Interview)

Interview handler

Phrase Server (PhraseServer)

The prompts that play and the user DTMF input; the logs are written to a file

Phrase Server to Monitor (PhraseServer to Monitor)

The prompts that play and the user DTMF input; the logs are written to the monitor

PHTransfer Conversations (ConvPH Transfer)

Phone transfers

Report Data Library (RDL)

 

Resource Loader (Resource Loader)

Using the selected language in the GUI; filling strings with product or message information

Resource Manager (Resource Manager)

Monitoring and providing available resources to the Arbiter as needed

Routing Rules (Routing Rules)

Call routing decisions

Routing Rules Conversation (ConvRoutingRules)

The conversation to which the Arbiter routes calls

Rules Engine (Rules Engine)

 

Scheduler (Scheduler)

Currently active Cisco Unity Connection schedule (whether during normal business hours or during nonbusiness hours) or holiday

Server Roles Manager (SRM)

Server Role Manager, which monitors and manages all server roles

Server Status App (CuStatusTray)

The Cisco Unity Connection Server Status utility

Skinny TSP (SkinnyTSP)

(Circuit-switched or Cisco Unified CallManager phone system integrations) Media or WAV file usage

(Cisco Unified CallManager SCCP integrations only) Skinny networking layer

Stream Server (StreamServer)

 

Subscriber Conversations (Subscriber Conversation)

User activities and usage

System Agent (SysAgent)

System Agent role, which schedules system tasks that the administrator enters (such as resynchronizing MWIs)

Telephone Record and Playback (TRAP)

Telephone Record and Playback (TRAP), which lets clients use the phone as a recording and playback device

Text to Speech (Text to Speech)

Text to Speech feature

UMSS IMAP Server (UMSSIMAPServer)

Access to voice messages by IMAP clients

UMSS Messaging Abstraction Layer (CsMalUmss)

 

UMSS SysAgent Tasks (UmssSysAgentTasks)

 

Unity Reports Scavenger Service (Scavenger)

Report Data Collector role, which extracts data from log files and periodically cleans up the database

Virtual Queue (VirtualQueue)

Call queuing


Enabling Micro Traces

Enable the micro trace diagnostics when you are troubleshooting problems with specific Cisco Unity Connection components. For example, if there are notification errors in the Event log, enable the Notifier and Notification Devices (Notifier) diagnostics. However, keep in mind that running diagnostics can affect system performance and hard drive space.

To Enable Micro Trace Diagnostics


Step 1 On the Windows Start menu, click Programs > Cisco Unity > Cisco Unity Diagnostic Tool. The Cisco Unity Diagnostic Viewer window appears.

Step 2 In the right pane of the Cisco Unity Diagnostic Viewer window, click Configure Micro Traces. The Configure Micro Traces wizard appears.

Step 3 On the Welcome page, click Next.

Step 4 On the Configure Micro Traces page, check the check boxes to select the component traces and the trace levels that you want to enable.

Step 5 Click Next.

Step 6 On the Completing page, click Finish.

Step 7 In the right pane of the Cisco Unity Diagnostic Viewer window, click Start New Log Files.

Step 8 Reproduce the problem.


Note After obtaining the diagnostic trace logs that you want, disable the traces that you enabled.



Viewing Individual Micro Trace Logs

Do the following procedure to use the UDT to view individual micro trace logs. For information on interpreting the micro trace information in the UDT, see the "How to Interpret Micro Trace Information in the Cisco Unity Diagnostic Tool (UDT)" section. For instructions on gathering trace logs into files, see the "To Gather Micro Trace Logs into Files" procedure.

To View Individual Micro Trace Logs


Step 1 On the Windows Start menu, click Programs > Cisco Unity > Cisco Unity Diagnostic Tool. The Cisco Unity Diagnostic Viewer window appears.

Step 2 In the left pane of the Cisco Unity Diagnostic Viewer window, expand the Processes node.

Step 3 In the left pane, expand the process (or component) that you enabled traces for, and click the log file that you want to view. The log file is formatted and appears in the right pane.


How to Interpret Micro Trace Information in the Cisco Unity Diagnostic Tool (UDT)

When you open a trace log in the UDT, the information is formatted and displayed in columns in the right pane. Table 1-4 lists the information contained in each column of trace logs.

Table 1-4 Information in Trace Logs 

Column Name
Information Contained in the Column

#

The line number in the trace log. This number is provided by the UDT and is not contained in the trace log.

Timestamp

The date and time of the trace log.

Source

The source of the trace log text.

Trace No.

The number of the message string that was used from the source identified in the Source column.

Component

The micro trace component that was selected in the Configure Micro Traces wizard.

Level

The trace level that was selected in the Configure Micro Traces wizard.

Trace

The raw data, delimited by commas, from the micro trace.


Gathering Micro Trace Logs into Files

When you are requested to send micro trace logs for examination, you must gather the logs into files. Do the following procedure.

To Gather Micro Trace Logs into Files


Step 1 On the Windows Start menu, click Programs > Cisco Unity > Cisco Unity Diagnostic Tool. The Cisco Unity Diagnostic Viewer window appears.

Step 2 In the right pane of the Cisco Unity Diagnostic Viewer window, click Gather Log Files. The Gather Logs wizard appears.

Step 3 On the Welcome page, click Select Logs.

Step 4 If you want to change the directory where the files are saved, do the following sub-steps. Otherwise, skip to Step 5.

a. Click Browse to select a destination for the files. The Browse for Folder dialog box appears.

b. Click the destination directory where you want the files to be saved, and click OK.

Step 5 On the Welcome page, click Next.

Step 6 On the Select Logs to Gather page, expand the micro trace processes that you enabled and check the check box for the most recent log for each micro trace.

Step 7 Click Next.

Step 8 On the Completing page (after the logs are gathered and formatted), click View Directory to open the directory where the files were saved.

Step 9 On the Completing page, click Finish to exit the wizard.

Step 10 Close the Cisco Unity Diagnostic Viewer window.


Dr. Watson Logs

Dr. Watson is a program invoked by Windows 2000 when a serious problem occurs that is not handled by Cisco Unity Connection. When Dr. Watson is invoked, a dialog box that contains an error message appears (for example, "Dr. Watson encountering an error in the CuCsMgr.exe process").

To Obtain a Dr. Watson Log


Step 1 From a command prompt, enter drwtsn32 and press Enter.

Step 2 In the Dr. Watson for Windows dialog box, in the Log File Path field, note the location of the log file.

Step 3 Browse to the Drwtsn32.log file and make a copy of the file in another location.

Step 4 In the Dr. Watson for Windows dialog box, in the Number of Instructions field, enter 50.

Step 5 In the Number of Errors to Save field, enter the number of errors you want to record. The default is 10.

Step 6 Under Options, confirm that the Dump All Thread Contexts, Append to Existing Log File, Visual Notification, and Create Crash Dump File check boxes are checked.

Step 7 Click OK to close the dialog box.

Step 8 Reproduce the problem.

Step 9 Browse to the Drwtsn32.log file and make a copy of the file in another location.


Reporting Problems to Cisco TAC

When you report a problem to the Cisco Technical Assistance Center (Cisco TAC), you will be asked to provide information about your system and about the problem. This section provides procedures for gathering the system information and problem descriptions that may be requested.

System Information

Have the following system information ready when you call:

The results from running the Gather Unity System Info utility. See the "To Collect the Cisco Unity Connection System Information" procedure.

Number and type of voice ports installed.

Number of users in the Cisco Unity Connection database.

Number, type, and speed of processors.

Memory and pagefile size.

Hard disk size and free space available.

Phone system integration, including the manufacturer, model, and version (if applicable).

Other telephony software or hardware installed, such as fax.

Microsoft Windows 2003 service packs installed.

Approximate normal Connection server CPU utilization. (For example, does the Windows task manager often show 100 percent CPU utilization, or is it usually less than 80 percent?)

To Collect the Cisco Unity Connection System Information


Step 1 On the Windows Start menu, click Programs > Cisco Unity > Gather Unity System Info.

Step 2 In the Gather Unity System Info window, note the system information that is displayed.


Problem Description

Be prepared to give a complete description of the problem, including:

Symptoms such as lost ports, Event log errors, or Dr. Watson errors.

Problem frequency under normal load conditions (for example, every call, once per hour, or once only).

Problem frequency when specific attempts are made to reproduce it.

Detailed sequence of steps to reproduce the problem.

Date and time of last known occurrence of the problem.

Which digits were entered by the caller (for example, menu selections or user extensions, or the extension of the caller or called port), if known.

Which port(s) were affected by the problem, if known.

Applicable logs and traces. For information on how to obtain log and trace files, see the preceding sections of this chapter.