Guest

Cisco Unity

Troubleshooting Port Lockups in Cisco Unity

Cisco - Troubleshooting Port Lockups in Cisco Unity

Document ID: 22374

Updated: Aug 24, 2011

   Print

Introduction

This document provides a procedure to identify the specific problem within a system that lands in one of these states:

  • Ports lock up

  • System slows down

  • System crashes

The document provides steps to gather information and troubleshoot. The information that this document describes is required for engineering review. Failure to provide this information increases the amount of time to resolution.

Note: The information in this document is based on Cisco Unity for Exchange.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

Components Used

The information in this document is based on these software versions:

  • Cisco Unity for Exchange 2.4(6)

  • Cisco Unity 3.x

  • Cisco Unity 4.x

  • Cisco Unity 5.x

  • Cisco Unity 7.x

  • Cisco Unity 8.x

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Identify the Problem

The steps in this section help you identify the problem. For example, a system in which ports lock up is often described as a system crash.

Note: The call does not reach Cisco Unity if the caller hears a fast busy. Cisco Unity cannot generate a fast busy signal.

  1. Check for event log errors.

    This can immediately help to identify the problem.

  2. Determine if the system answers calls.

    In order to do this, dial each port individually and note which ones answer, if any.

  3. Check the Port Status window in the Status Monitor.

    A port is most likely locked up if it is in a non-idle state and has been in this state for an hour or longer.

    Note: Refer to The Status Monitor section of Maintaining Cisco Unity for more information.

  4. Try to reset each port and see if the ports begin to respond.

    Reset the Cisco Unity-CallManager Telephony Application Programming Interface (TAPI) service provider (TSP). In order to do this, select Start > Settings > Control Panel > Phone and Modem options > Advanced > Cisco Unity-CallManager TSP > Configure.

    Another option is to launch Port Status Monitor. In order to launch, right-click the Cisco Unity icon on the system tray.

  5. Determine if there were any noticeable problems with the behavior of the system before the problem occurred.

    For example, did the system experience a slowdown before it totally stopped? If any such problems occurred, describe exactly what happened.

  6. Determine if you can access the System Administrator (SA).

  7. Determine the status of all the avcs* processes.

    In order to do this, go to the Microsoft Windows Task Manager, and click the Processes tab. Provide a screen capture. What is the CPU utilization?

  8. If you are able to leave the system down for another few minutes, skip to the Prepare to Gather Data for the Next Failure section and complete step 1 and step 4.

  9. Determine if you can stop and restart Cisco Unity with the Unity status tray icon.

  10. Check to see if a complete restart of the system solves the problem.

System Experiences Port Lockup

If any of these items are true, port lockup most likely affects the system:

  • The SA can be accessed.

  • Some ports answer, and some ports do not.

  • Ports begin to respond after you reset ports in Status Monitor or reset the TSP, as in step 4 of Identify the Problem.

Note: A system that experiences the lockup of ports can appear to experience a system crash when all the ports lock up.

System Experiences a System Crash

If any of these items are true, the problem is a system crash:

  • The system is unresponsive.

  • If you restart the Cisco Unity service, it does not resolve the problem.

  • One or all of the Cisco Unity services no longer run.

  • You cannot access the SA or Status Monitor.

  • CPU utilization is at or near 100 percent.

Provide Basic Details

It is important that you provide Cisco Technical Support with these basic details:

  • Version and build information for Cisco Unity

  • Phone system type and version

  • Type of Cisco Unity installation—Voice Messaging or Unified Messaging

  • The number of mail users

  • If you have Unified Messaging, the number of mail users homed on each remote Exchange server

  • The platform on which you have installed Cisco Unity, along with voice boards, if applicable

  • Cisco Unity features you have enabled, such as text-to-speech (TTS) and Audio Messaging Interchange Specification (AMIS)

  • Frequency of failures

  • Version of TSP

Prepare to Gather Data for the Next Failure

For Port Lockup

Complete these steps:

  1. The first time port lockup occurs, gather the standard Message Interface Unit (MIU) traces and application logs for initial investigation.

  2. Check to be sure that your system currently runs the latest version of Cisco Unity-CallManager TSP.

    1. Perform a search for the file named avskinny.tsp.

    2. Right-click avskinny.tsp, and choose the Version tab.

    3. If you find that it is not the latest version of Cisco Unity-CallManager TSP, perform an upgrade.

      Customers with contracts should obtain upgraded software through their normal update channels. For most customers, upgrades are obtained from the Software Center on Cisco.com.

  3. Refer to the appropriate document in order to be prepared to gather data for another failure:

  4. While in the failed state, select TechTools > StatusMonitor.exe (not the HTML version) and capture the screen that displays the status of the locked port.

    This identifies the part of the system conversation that was active when the call failed.

  5. For any outgoing call troubleshooting such as AMIS, MWI, message notification, or Telephony Record and Playback (TRAP), use one of the last ports because the higher-numbered ports are most likely being used for outgoing tasks. Cisco Unity searches for available ports for outgoing tasks starting with the last port.

For a System Crash

Refer to Troubleshooting Cisco Unity System Crashes if the problem is a system crash.

System Crash Known Problems

The known problems that this section lists exist in Cisco Unity software. You can view the Cisco bug IDs with the Bug Toolkit (registered customers only) .

Speed Control—CSCdw88890

This issue exists in Cisco Unity 3.1(x). The workaround for this issue is in Cisco Unity 3.1(3) and 3.1(2)c.

Problem Description

If you use playback speed controls during message retrieval, it results in various symptoms. The symptoms you experience depend on the system configuration. Cisco Unity might drop the call immediately, disable the port after the call ends, or fail to answer any calls to any Unity port.

Workaround

Refer to Field Notice: Cisco Unity Playback Speed Controls May Drop Callers or Lock Up All Unity Ports.

MIU Errors and Port Locked After AvWav Failure on Cisco CallManager Port—CSCdw11833

This issue exists in Cisco Unity 3.1(x). An upgrade to Cisco Unity 3.1(5) or 3.1(6) resolves the issue.

Problem Description

The port is busy, according to the Status Monitor. The call might be long, and the port does not accept other calls.

Symptom

The event log shows almost 20 MIU errors at the time that the call originated. The first error appears as:

Event Type: Error
Event Source: AvMiu_MC
Event Category: Error
Event ID: 530
Description: Component Miu: Thread 0x000009B8 had a Failure on Port 1 in AvWav
DESCRIPTION: e:\views\cs_ue3.1.0.56\un_Miu\UnityAvWav\WAV.C(4688) :
Exception caught in Method WavWriteFormatRecordAGC.

Workaround

Reboot the system to release the port.

You cannot reset the port with Status Monitor. If you attempt to stop it from the Cisco Unity status tray icon, Unity hangs.

Solution

Upgrade to Cisco Unity 3.1(3).

Thread Safety—CSCae08069

This issue exists in all versions of Cisco Unity. The workaround for this issue is in Cisco Unity 2.4.6.135 and later.

Problem Description

You get dead air or Ring No Answer (RNA) when you call Cisco Unity. This appears to occur on busy systems set up to use G.729 decoding. Also, you might see it on a G.711 system. The event log contains MIU errors. Also, ports disconnect and reconnect from the Cisco CallManager. The event log lists numerous identical AvWav errors from two ports at once.

If you look further, you find two ports make the call at the same time to the Microsoft Win32 application programming interface (API) function, which is acmStreamConvert(), inside the AvWav function, AcmStreamConvert(). Both throw an exception. From that point, callers hear either dead air or RNA.

Workaround

Complete these steps in Cisco Unity version 2.4.6.135 and later:

  1. Navigate to the registry.

  2. Add the key HKLM/Software/Active Voice/UnityAvWav/1.0/ForceGlobalAcmThreadSafety=1.

  3. Restart the Cisco Unity server.

    Note: Refer to Cisco Unity Ports Lock Up After Enabling G729a Codec for step-by-step instructions to enable this registry setting.

Cisco CallManager Leaves Voice Mail Ports in a Transfer State, Ports Are Busied Out—CSCdw94340, CSCdw70477, CSCdw36431, CSCdw19730, CSCdw47532, CSCdw55180

This issue exists in Cisco CallManager 3.1(2). It is resolved in Cisco CallManager 3.1(3)a—Engineering Special 3.

Problem Description

Cisco Unity attempts to perform a transfer, and Cisco CallManager returns this error:

Transferring - ERROR splitting_primary_call:
	 time out on waiting response.

The transfer does not complete. Cisco Unity does not receive the stimulus necessary to put the port back on hook, which leaves the port in a busy state.

Symptom

The caller receives RNA or busy when calling Cisco Unity.

Workaround

Upgrade to Cisco CallManager 3.1(3)a—Engineering Special 3.

Enable of AGC for G729 MSG-Type Recording May Lock Up Cisco Unity Ports—CSCdw92062

This issue exists in Cisco Unity 3.1. This issue is resolved in Cisco Unity 3.1(3).

Problem Description

The Cisco Unity system might lock a port after the caller leaves a message. Cisco Unity might also drop the call after the caller leaves a message.

Symptom

This occurs in Cisco Unity 3.1(x) when you have Automatic Gain Control (AGC) enabled on Unity and G.729 set as the message record format.

The Event Viewer errors can include these errors:

12:00:33:559,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, 
File: e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAV.C(4651) 
Method: WavWriteFormatRecordAGC Failure: 
call to AcmConvertGetSizeDst failed,0x00000BD0

12:00:33:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, File: 
e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAV.C(6223) 
Method: WavNotify Failure: 
call to WavWriteFormatRecordAGC failed,0x00000BD0

12:01:03:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, 
File: e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAVIN.C(725) 
Method: WavInClose Failure: 
call to WaitForSingleObject (hEventDeviceClosed) failed,0x00000D68

12:01:03:561,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, 
File: e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAV.C(5437) 
Method: WavStopRecord Failure: call to WavInClose failed,0x00000D68

12:01:03:560,AvMiu_MC,690,-1,-1,MiuGeneral,0,1, 
File: e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAV.C(2254) 
Method: WavStop Failure: call to WavStopRecord failed,0x00000D68

12:01:03:561,AvMiu_MC,690,-1,-1,MiuGeneral,0,1,
e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAVIN.C(1064) : 
Exception caught in Method WavInStop | 
Data hWavIn 021E4CC4,0x00000BD0

12:01:03:560,AvMiu_MC,683,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(),
AvWav WavStop failed with 0xFFFFFFFF,0x00000D68

12:01:03:561,AvMiu_MC,686,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(), 
FileName: C:\DOCUME~1\ADMINI~1.RAC\LOCALS~1\Temp\WAV1.tmp 
HWAV: 0x021E493C WavState: WAV_STOPPING,0x00000D68

12:01:03:560,AvMiu_MC,688,-1,-1,MiuGeneral,0,1,CAvMiuWave::Record(),
1,[11:54:25:938 - 0x00000D78] 
Drop() - S_OK,0x00000D68

------------------------------------------------------------------

Source: AvMiu_MC
Category: Error
EventId: 530
Description:
Component Miu: Thread 0x00000BD0 had a Failure on Port 1 in AvWav
DESCRIPTION:
File: e:\views\cs_UE3.1.2.17\un_Miu\UnityAvWav\WAV.C(4651)
Method: WavWriteFormatRecordAGC
Failure: call to AcmConvertGetSizeDst failed.

Workaround

Reset the port from the Cisco Unity Status Monitor, as this section shows.

Note: Status Monitor is located in the C:\Commserver\TechTools directory.

  1. Open Status Monitor.

  2. Locate the port that is locked and click Reset Port.

    Note: If more than one port is locked, stop and restart the Cisco Unity services.

  3. Disable AGC on the Cisco Unity system from the registry.

    Locate the AGCUseCompression key under this directory, and set it to 0:

    \\HKLM\Software\ActiveVoice\MIU\1.0\Initialization
  4. Restart the system for the disablement to take effect.

    The action disables AGC for both G.711 and G.729 messages.

AvWav: Failure in ExAcmFormatGetText Can Cause Port Lock—CSCdx15569

This issue exists in all versions of Cisco Unity.

Resolution

You must create a service request with Cisco Technical Support.

Problem Description

Cisco Unity ports lock up one by one and stop taking calls.

Symptom

If there is a section of Resource Interchange File Format (RIFF) header of the .wav file that is corrupt, the AvWav ExAcmFormatGetText function fails. This eventually becomes a critical section that is not released in AvWav. Subsequent calls to the failed port might answer to silence or might not be answered at all.

A string of AvWav errors appear in the event log. The first error appears as this:

Component Miu: Thread 0x000009F0 had a Failure on Port 5 in AvWav

File: D:\commsvr\Sources\UnityAvWav\ACM.C(581)
Method: ExAcmFormatGetText
Failure: call to acmFormatDetails failed with error(200)

Race Condition in MIU Leads to Calls Not Deallocated—CSCdw54752

This issue exists in Cisco Unity 2.4(x).

Note: This issue does not exist in Cisco Unity 3.x.

Resolution

Upgrade to Cisco Unity 3.x, or create a service request with Cisco Technical Support.

Symptom

The event log displays Timed out waiting for Drop completed event errors, which is usually followed by a lost port. This can happen on any integration. However, if you have an IP integration, the event log may display Previous call not deallocated messages in the Skinny traces.

Note: If either of these messages appears in the event log, it does not necessarily mean that your system has this problem. Many problems can lead to either of these symptoms.

Cisco Unity Fails to Respond Because of RPC Hang-CSCdw22615

This issue exists in Cisco Unity 2.4(x) and 3.0(3).

Note: This bug occurs if you have Windows 9x clients that run any application that uses Telephony Record and Playback (TRaP). These include ViewMail and Auto Attendant (AA).

Resolution

You must create a service request with Cisco Technical Support.

Symptom

Cisco Unity stops the answer of calls, but it appears to run. Cisco Unity actually answers a call on each port, but it is dead air. After it has done this for each port, it does not answer any more calls. The SA, Status Monitor, StatusMonitor.exe, and many other Cisco Unity tools do not open.

This is a bug with the implementation of remote-procedure call (RPC) in Windows 9x.

Malformed ASCII Alerting Name Can Terminate svchost.exe

This issue occurs on the TSP version 8.3(1). This issue depends on the TSP version used, not on the Cisco Unity version. This issue is documented by Cisco bug ID CSCth05576 (registered customers only) .

Resolution

Disable the SkinnyTSP diagnostics in the Unity Diagnostic Tool. Correct incoming ASCII Alerting Name format to include only valid ASCII characters.

Symptom

Cisco Unity stops answering calls from all Cisco Unified CallManager integrations only when you have the SkinnyTSP diagnostics enabled in the Unity Diagnostic Tool. The Application Log shows this error message:

Event Type:	Error
Event Source:	Application Error
Event Category:	(100)
Event ID:	1000
Date:		5/28/2010
Time:		5:34:44 AM
User:		N/A
Computer:	UNITY80
Description:
Faulting application svchost.exe, version 5.2.3790.3959, faulting module ntdll.dll, 
version 5.2.3790.4455, fault address 0x0001a3e1.

Related Information

Updated: Aug 24, 2011
Document ID: 22374