Document ID: 22374
This document provides a procedure to identify the specific problem within a system that lands in one of these states:
Ports lock up
System slows down
The document provides steps to gather information and troubleshoot. The information that this document describes is required for engineering review. Failure to provide this information increases the amount of time to resolution.
Note: The information in this document is based on Cisco Unity for Exchange.
Cisco recommends that you have knowledge of these topics:
For Cisco Unity 2.4(x) and 3.0(x)—Refer to the Configuring Unity Traces with MaestroTools.exe document.
For Cisco Unity 4.x—Refer to the Logs and Traces section of the Cisco Unity Troubleshooting Guide.
For Cisco Unity 5.x and 7.x —Refer to the Diagnostic Trace Utilities and Logs section of the Cisco Unity Troubleshooting Guide. The Cisco Unity 5.x troubleshooting guide applies to Cisco Unity 7.x as well.
For Cisco Unity 8.x—Refer to the Diagnostic Trace Utilities and Logs in Cisco Unity 8.x section of the Cisco Unity Troubleshooting Guide.
The information in this document is based on these software versions:
Cisco Unity for Exchange 2.4(6)
Cisco Unity 3.x
Cisco Unity 4.x
Cisco Unity 5.x
Cisco Unity 7.x
Cisco Unity 8.x
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Refer to Cisco Technical Tips Conventions for more information on document conventions.
The steps in this section help you identify the problem. For example, a system in which ports lock up is often described as a system crash.
Note: The call does not reach Cisco Unity if the caller hears a fast busy. Cisco Unity cannot generate a fast busy signal.
Check for event log errors.
This can immediately help to identify the problem.
Determine if the system answers calls.
In order to do this, dial each port individually and note which ones answer, if any.
Check the Port Status window in the Status Monitor.
A port is most likely locked up if it is in a non-idle state and has been in this state for an hour or longer.
Try to reset each port and see if the ports begin to respond.
Reset the Cisco Unity-CallManager Telephony Application Programming Interface (TAPI) service provider (TSP). In order to do this, select Start > Settings > Control Panel > Phone and Modem options > Advanced > Cisco Unity-CallManager TSP > Configure.
Another option is to launch Port Status Monitor. In order to launch, right-click the Cisco Unity icon on the system tray.
Determine if there were any noticeable problems with the behavior of the system before the problem occurred.
For example, did the system experience a slowdown before it totally stopped? If any such problems occurred, describe exactly what happened.
Determine if you can access the System Administrator (SA).
Determine the status of all the avcs* processes.
In order to do this, go to the Microsoft Windows Task Manager, and click the Processes tab. Provide a screen capture. What is the CPU utilization?
If you are able to leave the system down for another few minutes, skip to the Prepare to Gather Data for the Next Failure section and complete step 1 and step 4.
Determine if you can stop and restart Cisco Unity with the Unity status tray icon.
Check to see if a complete restart of the system solves the problem.
If any of these items are true, port lockup most likely affects the system:
The SA can be accessed.
Some ports answer, and some ports do not.
Ports begin to respond after you reset ports in Status Monitor or reset the TSP, as in step 4 of Identify the Problem.
Note: A system that experiences the lockup of ports can appear to experience a system crash when all the ports lock up.
If any of these items are true, the problem is a system crash:
The system is unresponsive.
If you restart the Cisco Unity service, it does not resolve the problem.
One or all of the Cisco Unity services no longer run.
You cannot access the SA or Status Monitor.
CPU utilization is at or near 100 percent.
It is important that you provide Cisco Technical Support with these basic details:
Version and build information for Cisco Unity
Phone system type and version
Type of Cisco Unity installation—Voice Messaging or Unified Messaging
The number of mail users
If you have Unified Messaging, the number of mail users homed on each remote Exchange server
The platform on which you have installed Cisco Unity, along with voice boards, if applicable
Cisco Unity features you have enabled, such as text-to-speech (TTS) and Audio Messaging Interchange Specification (AMIS)
Frequency of failures
Version of TSP
Complete these steps:
The first time port lockup occurs, gather the standard Message Interface Unit (MIU) traces and application logs for initial investigation.
Check to be sure that your system currently runs the latest version of Cisco Unity-CallManager TSP.
Perform a search for the file named avskinny.tsp.
Right-click avskinny.tsp, and choose the Version tab.
If you find that it is not the latest version of Cisco Unity-CallManager TSP, perform an upgrade.
Refer to the appropriate document in order to be prepared to gather data for another failure:
While in the failed state, select TechTools > StatusMonitor.exe (not the HTML version) and capture the screen that displays the status of the locked port.
This identifies the part of the system conversation that was active when the call failed.
For any outgoing call troubleshooting such as AMIS, MWI, message notification, or Telephony Record and Playback (TRAP), use one of the last ports because the higher-numbered ports are most likely being used for outgoing tasks. Cisco Unity searches for available ports for outgoing tasks starting with the last port.
Refer to Troubleshooting Cisco Unity System Crashes if the problem is a system crash.
This issue exists in Cisco Unity 3.1(x). The workaround for this issue is in Cisco Unity 3.1(3) and 3.1(2)c.
If you use playback speed controls during message retrieval, it results in various symptoms. The symptoms you experience depend on the system configuration. Cisco Unity might drop the call immediately, disable the port after the call ends, or fail to answer any calls to any Unity port.
This issue exists in Cisco Unity 3.1(x). An upgrade to Cisco Unity 3.1(5) or 3.1(6) resolves the issue.
The port is busy, according to the Status Monitor. The call might be long, and the port does not accept other calls.
The event log shows almost 20 MIU errors at the time that the call originated. The first error appears as:
Event Type: Error Event Source: AvMiu_MC Event Category: Error Event ID: 530 Description: Component Miu: Thread 0x000009B8 had a Failure on Port 1 in AvWav DESCRIPTION: e:\views\cs_ue22.214.171.124\un_Miu\UnityAvWav\WAV.C(4688) : Exception caught in Method WavWriteFormatRecordAGC.
Reboot the system to release the port.
You cannot reset the port with Status Monitor. If you attempt to stop it from the Cisco Unity status tray icon, Unity hangs.
Upgrade to Cisco Unity 3.1(3).
This issue exists in all versions of Cisco Unity. The workaround for this issue is in Cisco Unity 126.96.36.199 and later.
You get dead air or Ring No Answer (RNA) when you call Cisco Unity. This appears to occur on busy systems set up to use G.729 decoding. Also, you might see it on a G.711 system. The event log contains MIU errors. Also, ports disconnect and reconnect from the Cisco CallManager. The event log lists numerous identical AvWav errors from two ports at once.
If you look further, you find two ports make the call at the same time to the Microsoft Win32 application programming interface (API) function, which is acmStreamConvert(), inside the AvWav function, AcmStreamConvert(). Both throw an exception. From that point, callers hear either dead air or RNA.
Complete these steps in Cisco Unity version 188.8.131.52 and later:
Navigate to the registry.
Add the key HKLM/Software/Active Voice/UnityAvWav/1.0/ForceGlobalAcmThreadSafety=1.
Restart the Cisco Unity server.
Note: Refer to Cisco Unity Ports Lock Up After Enabling G729a Codec for step-by-step instructions to enable this registry setting.
Cisco CallManager Leaves Voice Mail Ports in a Transfer State, Ports Are Busied Out—CSCdw94340, CSCdw70477, CSCdw36431, CSCdw19730, CSCdw47532, CSCdw55180
This issue exists in Cisco CallManager 3.1(2). It is resolved in Cisco CallManager 3.1(3)a—Engineering Special 3.
Cisco Unity attempts to perform a transfer, and Cisco CallManager returns this error:
Transferring - ERROR splitting_primary_call: time out on waiting response.
The transfer does not complete. Cisco Unity does not receive the stimulus necessary to put the port back on hook, which leaves the port in a busy state.
The caller receives RNA or busy when calling Cisco Unity.
Upgrade to Cisco CallManager 3.1(3)a—Engineering Special 3.
This issue exists in Cisco Unity 3.1. This issue is resolved in Cisco Unity 3.1(3).
The Cisco Unity system might lock a port after the caller leaves a message. Cisco Unity might also drop the call after the caller leaves a message.
This occurs in Cisco Unity 3.1(x) when you have Automatic Gain Control (AGC) enabled on Unity and G.729 set as the message record format.
The Event Viewer errors can include these errors:
12:00:33:559,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: e:\views\cs_UE184.108.40.206\un_Miu\UnityAvWav\WAV.C(4651) Method: WavWriteFormatRecordAGC Failure: call to AcmConvertGetSizeDst failed,0x00000BD0 12:00:33:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: e:\views\cs_UE220.127.116.11\un_Miu\UnityAvWav\WAV.C(6223) Method: WavNotify Failure: call to WavWriteFormatRecordAGC failed,0x00000BD0 12:01:03:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: e:\views\cs_UE18.104.22.168\un_Miu\UnityAvWav\WAVIN.C(725) Method: WavInClose Failure: call to WaitForSingleObject (hEventDeviceClosed) failed,0x00000D68 12:01:03:561,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: e:\views\cs_UE22.214.171.124\un_Miu\UnityAvWav\WAV.C(5437) Method: WavStopRecord Failure: call to WavInClose failed,0x00000D68 12:01:03:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: e:\views\cs_UE126.96.36.199\un_Miu\UnityAvWav\WAV.C(2254) Method: WavStop Failure: call to WavStopRecord failed,0x00000D68 12:01:03:561,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, e:\views\cs_UE188.8.131.52\un_Miu\UnityAvWav\WAVIN.C(1064) : Exception caught in Method WavInStop | Data hWavIn 021E4CC4,0x00000BD0 12:01:03:560,AvMiu_MC,683,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(), AvWav WavStop failed with 0xFFFFFFFF,0x00000D68 12:01:03:561,AvMiu_MC,686,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(), FileName: C:\DOCUME~1\ADMINI~1.RAC\LOCALS~1\Temp\WAV1.tmp HWAV: 0x021E493C WavState: WAV_STOPPING,0x00000D68 12:01:03:560,AvMiu_MC,688,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(), 1,[11:54:25:938 - 0x00000D78] Drop() - S_OK,0x00000D68 ------------------------------------------------------------------ Source: AvMiu_MC Category: Error EventId: 530 Description: Component Miu: Thread 0x00000BD0 had a Failure on Port 1 in AvWav DESCRIPTION: File: e:\views\cs_UE184.108.40.206\un_Miu\UnityAvWav\WAV.C(4651) Method: WavWriteFormatRecordAGC Failure: call to AcmConvertGetSizeDst failed.
Reset the port from the Cisco Unity Status Monitor, as this section shows.
Note: Status Monitor is located in the C:\Commserver\TechTools directory.
Open Status Monitor.
Locate the port that is locked and click Reset Port.
Note: If more than one port is locked, stop and restart the Cisco Unity services.
Disable AGC on the Cisco Unity system from the registry.
Locate the AGCUseCompression key under this directory, and set it to 0:
Restart the system for the disablement to take effect.
The action disables AGC for both G.711 and G.729 messages.
This issue exists in all versions of Cisco Unity.
You must create a service request with Cisco Technical Support.
Cisco Unity ports lock up one by one and stop taking calls.
If there is a section of Resource Interchange File Format (RIFF) header of the .wav file that is corrupt, the AvWav ExAcmFormatGetText function fails. This eventually becomes a critical section that is not released in AvWav. Subsequent calls to the failed port might answer to silence or might not be answered at all.
A string of AvWav errors appear in the event log. The first error appears as this:
Component Miu: Thread 0x000009F0 had a Failure on Port 5 in AvWav File: D:\commsvr\Sources\UnityAvWav\ACM.C(581) Method: ExAcmFormatGetText Failure: call to acmFormatDetails failed with error(200)
This issue exists in Cisco Unity 2.4(x).
Note: This issue does not exist in Cisco Unity 3.x.
Upgrade to Cisco Unity 3.x, or create a service request with Cisco Technical Support.
The event log displays Timed out waiting for Drop completed event errors, which is usually followed by a lost port. This can happen on any integration. However, if you have an IP integration, the event log may display Previous call not deallocated messages in the Skinny traces.
Note: If either of these messages appears in the event log, it does not necessarily mean that your system has this problem. Many problems can lead to either of these symptoms.
This issue exists in Cisco Unity 2.4(x) and 3.0(3).
Note: This bug occurs if you have Windows 9x clients that run any application that uses Telephony Record and Playback (TRaP). These include ViewMail and Auto Attendant (AA).
You must create a service request with Cisco Technical Support.
Cisco Unity stops the answer of calls, but it appears to run. Cisco Unity actually answers a call on each port, but it is dead air. After it has done this for each port, it does not answer any more calls. The SA, Status Monitor, StatusMonitor.exe, and many other Cisco Unity tools do not open.
This is a bug with the implementation of remote-procedure call (RPC) in Windows 9x.
This issue occurs on the TSP version 8.3(1). This issue depends on the TSP version used, not on the Cisco Unity version. This issue is documented by Cisco bug ID CSCth05576 (registered customers only) .
Disable the SkinnyTSP diagnostics in the Unity Diagnostic Tool. Correct incoming ASCII Alerting Name format to include only valid ASCII characters.
Cisco Unity stops answering calls from all Cisco Unified CallManager integrations only when you have the SkinnyTSP diagnostics enabled in the Unity Diagnostic Tool. The Application Log shows this error message:
Event Type: Error Event Source: Application Error Event Category: (100) Event ID: 1000 Date: 5/28/2010 Time: 5:34:44 AM User: N/A Computer: UNITY80 Description: Faulting application svchost.exe, version 5.2.3790.3959, faulting module ntdll.dll, version 5.2.3790.4455, fault address 0x0001a3e1.
- Field Notice: Cisco Unity Playback Speed Controls May Drop Callers or Lock Up All Unity Ports
- Field Notice: Cisco Unity Stops Responding Because of RPC Hang
- Voice Technology Support
- Voice and Unified Communications Product Support
- Troubleshooting Cisco IP Telephony
- Technical Support & Documentation - Cisco Systems
|Updated: Aug 24, 2011||Document ID: 22374|