Guest

Cisco Unified Communications Manager (CallManager)

AVVID TAC Cases: Collecting Troubleshooting Information

Cisco - AVVID TAC Cases: Collecting Troubleshooting Information

Document ID: 18281

Updated: Oct 08, 2006

   Print

Introduction

When you open a case with Cisco Technical Support, some preliminary information is required in order to identify and qualify the issue in a better way. Some of this information is always required

Other information depends on the nature of the issue. If you wait until the engineer requests that you collect the information after you open your case, it causes the delay of resolution results.

This document identifies the required preliminary information with regard to the type of issue so that it can be provided to the engineer immediately. This document also provides general guidelines that you can follow when information is collected for the Technical Support engineer in order to avoid repetitive testing and recollection of identical data.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

For more information on document conventions, refer to the Cisco Technical Tips Conventions.

General Guidelines

This section defines the guidelines for the various tools, products, and processes involved in order to troubleshoot AVVID Technical Support cases.

Technical Support Site Area

The Technical Support site area contains a detailed collection of tools and technical documents written by engineers in order to analyze common issues and provide solutions. Technical Support Training Resources contains links to various training resources that include Cisco Learning Connection and various seminars.

View the Voice Technology Support and Voice and Unified Communications Product support pages before you contact Technical Support.

Cisco Support Community

Cisco Support Community is a forum for you to ask and answer questions, share suggestions, and collaborate with your peers. View the Collaboration, Voice and Video Communities before you contact Technical Support.

Cisco.com Cases

When you open a case through Contact Technical Support, it gives you priority over all other case-opening methods. Make sure you use the web in order to open and follow up your cases. High priority cases (P1/P2) are an exception to this rule.

Note: It is important to provide an accurate description of the problem when you open a Cisco.com case as you are presented with several URLs, the selection of which is based on that description. Follow each and every one of these links while you wait for the Technical Support engineer to contact you, as in many cases they can provide you with an immediate solution.

Attachments

Any document larger than 100 Kb needs to be zipped. You can attach documents to a case and send it by e-mail to the engineer. In order to do so, use the web-based Update a Technical Support Case option.

Remote Access

Note: When you set up dial-in, do not use login:cisco or password:cisco as they constitute a vulnerability.

Issues can be resolved quickly when you allow the Technical Support engineer to remotely access the devices through one of these methods:

  • Equipment with public IP address

  • Dial-in access

    • In the decreasing order of preference—analog modem, ISDN modem, Microsoft PPTP, VPN

  • NAT (Network Address Translation)

Remote access needs to provide the ability to establish Terminal Services (remote port 3389), HTTP (remote port 80), and Telnet (remote port 23) sessions to all concerned equipment.

Pay special attention in order to ensure that firewalls do not obstruct Cisco IOS Software traffic and PIX traffic during the intervention of the engineer and that all necessary services, such as Terminal Services, start on the servers.

All access information is handled with the utmost discretion and no changes are made to the system without the consent of the customer.

Sniffer Traces

Sniffer traces can typically be collected by the connection of a laptop or other sniffer-equipped device on a Catalyst port, configured to span the VLAN or port(s) (CatOS, Cat6K-IOS, XL-IOS) that contains the remarkable information. If no free port is available, another possibility is to connect the sniffer-equipped device on a hub inserted between the switch and the device.

Note: For issues with IP Telephony, connect the sniffer to the PC port off the back of the Cisco IP Phone.

Note: You can trace the collection with the help of Sniffer Pro software as it is widely used in Cisco Technical Support. It also facilitates the comprehension and interpretation of the traces by the engineer. You must mention the IP or MAC addresses of all equipment involved, such as, IP phones, gateways, and Cisco CallManagers.

Debugs

Note: Collection of debugs in a live environment during operation hours needs to be avoided. It is preferable that debugs are collected during non-working hours. If they have to be collected in a live environment, issue the no logging console and logging buffered commands. In order to collect the debugs, issue the show log command.

Before you run any Cisco IOS Software voice gateway debugs, make sure that you configure the service timestamps debug datetime msec command globally on the gateway.

Some debugs are very verbose. Therefore, collect them directly on the console port (default logging console or on the buffer (logging buffer). If you collect them over a Telnet session it can have an impact on the performance of the device and the resultant debugs can be incomplete. This results in the need to recollect them.

CallManager Traces

Cisco CallManager traces are very important in the process used to troubleshoot an IP Telephony issue. For guidelines to collect Cisco CallManager traces, refer to Set Up Cisco CallManager Traces for Cisco Technical Support.

Phone Statistics

Press the I button twice during an active call in order to obtain phone statistics on a Cisco IP Phone 79xx.

You can also configure the "QRT" service within the Cisco CallManager Administration page and subscribe your phones to this service. For more information, refer to Phone Problem Reports Viewer.

General

When you run a test in order to reproduce the issue and produce information, this data is crucial in order to understand this issue:

  • Calling number/Called number.

  • Any other number involved along with its role in the specific scenario.

  • Whether or not the call is placed from an H.323 gateway or Media Gateway Control Protocol (MGCP) gateway.

  • Whether the problem is experienced on inbound/outbound calls to/from the PSTN or IP phone to IP phone.

  • Time of call—Time synchronization of all equipment is important.

Standard Required Information

For all issues, the information always needs to be provided to Cisco Technical Support. You also need to collect and save this information for further use upon opening a case. Additionally, you need to update it regularly with any changes.

Network Layout

This is a detailed description of the physical and logical setup, as well as all the network elements involved in the voice network (if applicable).

  • Cisco CallManager(s)—Version (CCMAdmin > Details), any service patches, number of CallManagers, setup (stand-alone, cluster)

  • Cisco Unity—Version (main admin page), integration type

  • Applications—Version

  • IP/Voice gateways—Operating System (OS) version, show tech command (Cisco IOS gateway), CM load (Skinny gateway), H.323 or MGCP

  • Switch—OS version, VLAN configuration

  • Dial plan—Numbering scheme, call routing

Ideally, a visual or other detailed diagram, such as JPG, needs to be submitted. The diagram can also be provided through a Cisco Live! session, with use of the whiteboard of the tool.

Problem Description

Provide step-by-step detail of actions performed by the user when the issue occurs. The detailed information needs to include:

  • Expected behavior

  • Detailed observed behavior

General Information

This information needs to be readily available.

  • Is this a new installation?

    • If it is an old installation, has this issue occurred since it was installed?

      • If not, what changes were made lately to the system?

  • Is the issue reproducible?

    • If it is reproducible, is it under normal or special circumstances?

    • If it is not reproducible, is there anything special about the time it does occur?

    • What is the frequency of occurrence?

  • What are the affected devices?

    • If specific devices are affected (not random), what do they have in common?

  • What devices are on the Call-Path (if applicable)?

Problem Types and Specific Required Information

This section provides information on the type of problems and specific information required in order to submit a case to Technical Support.

Note: The shaded areas in these tables represent information that is needed when you submit a case to Cisco Technical Support.

CallManager: System/OS

Provide this information:

  System/OS
Services not Starting High CPU Event Log Errors System Crash/Freeze Memory Leak
Event Log (app/sys/sec) XXX     XXX  
IIS and Cisco Services Status XXX   XXX    
Performance Monitor Log (MS - Q248345) leavingcisco.com   CPU percent counter   CPU percent and Memory counters Memory counter
Common Software versions, installed service packs, add-ons, hot fixes, patches
Additional Information     RED X'ed errors    

CallManager: Administration

This information needs to be provided:

  Cisco CallManager Administration
Access to CCMAdmin Configuration
Event Log (app/sys/sec) XXX  
IIS and Cisco Services Status XXX  
IIS Logs XXX  
DBL Traces   XXX
Common Software versions, installed service packs, add-ons, hot fixes, patches
Additional Information    

CallManager: Call Processing/Signaling

This information needs to be provided:

Perform one call and collect all adequate information simultaneously based on this table. Follow the general guidelines in the Debugs section of this document.

  Cisco CallManager: Call Processing/ Establishment / Drops
IP Phone to IP Phone 1 MGCP Gateway 2 Skinny Gateway 3 H.323 Gateway 4
debug show version IOS gateway   IOS gateway
show run IOS gateway   IOS gateway
Cisco CallManager Traces All Cisco CallManagers involved
cch323 h225     IOS gateway
voice ccapi inout     IOS gateway
mgcp all IOS gateway    
isdn q931     IOS gateway with Primary Rate Interface(PRI)
vtsp all IOS gateway/ Foreign Exchange Station (FXS)   IOS gateway with R2, channel associated signaling (CAS), or FXS
vpm all IOS gateway/FXS   IOS gateway with R2, CAS, or FXS
      Non-IOS gateway traffic Cisco CallManager <->gateway
Common Calling number, called number, other number(s) involved, time of call, observed behavior

Note: When you collect multiple debugs, collect them all at the same time.

Additional Information      

1 IP Phone—skinny protocol—CallManager—skinny protocol—IP Phone

2 IP Phone—skinny protocol—CallManager—MGCP protocol—MGCP gateway—PSTN

3 IP Phone—skinny protocol—CallManager—skinny protocol—Skinny gateway—PSTN

4 IP Phone—skinny protocol—CallManager—H.323—H.323 gateway—PSTN

CallManager: One-way Audio

This information needs to be provided:

Perform one call and collect all adequate information simultaneously based on this table. Ensure you follow the debug general guidelines.

    CallManager: One-way Voice
IP Phone to IP Phone 1 MGCP Gateway 2 Skinny Gateway 3 H.323 Gateway 4
debug show version   IOS gateway   IOS gateway
show run   IOS gateway   IOS gateway
CallManager Traces Attenuation settings for gateway
cch323 h245       IOS gateway
cch323 rtp       IOS gateway
voice ccapi inout       IOS gateway
mgcp packets   IOS gateway    
vtsp all   IOS gateway   IOS gateway
show call active voice brief       IOS gateway if call over PRI, FXS, Foreign Exchange Office(FXO)
  If 7960/7940 present behavior of RxCnt, TxCnt, RxLost, MaxJtr
  Traffic between audio endpoints and endpoints <->CallManager
Common Calling number, called number, other number(s) involved, time of call, observed behavior

Note: When you collect multiple debugs, collect them all at the same time.

Additional Information Which party does not hear the audio? Is one-way audio permanent? If not, does it happen at the launch of the call? After a while? How long does it last?

1. IP Phone—skinny protocol—CallManager—skinny protocol—IP Phone

2. IP Phone—skinny protocol—CallManager—MGCP—MGCP gateway—PSTN

3. IP Phone—skinny protocol—CallManager—skinny protocol—Skinny gateway—PSTN

4. IP Phone—skinny protocol—CallManager—H.323—H.323 gateway—PSTN

CallManager: Echo

This information needs to be provided:

Perform one call and collect all adequate information simultaneously based on this table. Follow the general guidelines in the Debugs section of this document.

   
IP Phone to IP Phone 1 MGCP Gateway 2 MGCP Gateway 3 MGCP Gateway 4
show version   IOS gateway   IOS gateway
show run   IOS gateway   IOS gateway
CallManager config Attenuation settings for gateway
show call active voice   IOS gateway (while tone testing)   IOS gateway (while tone testing)
show port voice active     Catalyst 6000 (while tone testing)  
  Traffic between audio endpoints and endpoints <-> Cisco CallManager
Common Calling number, called number, other number(s) involved, time of call, observed behavior

Note: When you collect multiple debugs, collect all of them at the same time.

Additional Information Which party hears the echo? Is the echo permanent? If not, does it happen at the start of the call? After a while? How long does it last? Is it long or short echo? Is it loud or low?

1 IP Phone—skinny protocol—CallManager—skinny protocol—IP Phone

2 IP Phone—skinny protocol—CallManager—MGCP—MGCP gateway—PSTN

3 IP Phone—skinny protocol—CallManager—skinny protocol—Skinny gateway—PSTN

4 IP Phone— skinny protocol —CallManager—H.323—H.323 gateway—PSTN

CallManager: Sub-optimal Voice Quality

This information needs to be provided:

Perform one call and collect all adequate information simultaneously based on this table. Follow the general guidelines in the Debugs section of this document.

  Cisco CallManager: Sub-optimal Voice Quality
IP Phone to IP Phone 1 MGCP Gateway 2 MGCP Gateway 3 MGCP Gateway 4
show version   IOS gateway   IOS gateway
show run   IOS gateway   IOS gateway
Cisco CallManager Traces All Cisco CallManagers involved
CallManager config Attenuation settings for gateway
show call active voice | inc los       IOS gateway if call over PRI, FXS, FXO
  If 7960/7940 present behavior of RxCnt, TxCnt, RxLost, MaxJtr
  Traffic between audio endpoints and endpoints <-> Cisco CallManager
Common Calling number, called number, other number(s) involved, time of call, observed behavior

Note: When you collect multiple debugs, collect them all at the same time.

Additional Information Which party hears sub-optimal audio? Is the issue permanent? If not, does it happen at the start of the call? After a while? How long does it last?

1 IP Phone—skinny protocol—CallManager—skinny protocol—IP Phone

2 IP Phone—skinny protocol—CallManager—MGCP—MGCP gateway—PSTN

3 IP Phone—skinny protocol—CallManager—skinny protocol—Skinny gateway—PSTN

4 IP Phone—skinny protocol—CallManager—H.323—H.323 gateway—PSTN

CallManager: Upgrade or Install Failure

In case of an upgrade or install failure, this information needs to be provided:

  • Original version of Cisco CallManager

  • Upgrade/Installation version

  • Any error messages that appeared during the process

Collect the logs under these paths:

  1. C:\*.log

  2. C:\*.txt

  3. C:\Winnt\sti*.*

  4. C:\dcdsrvr\log\*.*

  5. C:\Install\DBInstall\*.*

  6. C:\Program Files\Common Files\Cisco\Logs\*.*

If this is a Service Pack or Engineering Special install, include the file C:\Program Files\Cisco\ciscoupdate.txt.

CallManager: Backup Utility

This information needs to be provided:

  • Cisco CallManager version

  • C:\Winnt\sti*.txt

  • C:\Winnt\Catalog.txt

In order to create this file, right click on the Backup utility icon at the bottom right hand corner of the screen and select Create Catalog of Backup.

CallManager: Other

This information needs to be provided:

  Cisco CallManager: Other
IP Phones Unregistering/Reregistering 1 IP Phones Rebooting 1
Cisco CallManager Traces All Cisco CallManagers involved
  Traffic between Cisco CallManagers and IP phones
Common IP and MAC address of involved phones
Additional Information How are the phones powered (inline power or external power supply)?

1. IP Phone—skinny protocol—CallManager

Applications: Attendant Console

Before you open a case, refer to Cisco CallManager Attendant Console Frequently Asked Questions.

Collect these informations:

  • Cisco CallManager version

  • Attendant Console version

  • Pilot point DN, Hunt group members in order

  • From the server side, provide the complete logs folder located at C:\ProgramFiles\Cisco\CallManagerAttendant\

  • Cisco CallManager traces from all involved CallManagers

  • The computer telephony integration (CTI) traces from all involved Cisco CallManagers

  • Telephony Call Dispatcher (TCD) traces from all involved Cisco CallManagers

  • From the client side, provide the logs located at:

    • C:\ProgramFiles\Cisco\CallManager Antendant Console\*.txt

    • C:\ProgramFiles\Cisco\CallManager Antendant Console\jtapi\*.txt

    • UserID used to log into the client

Note: For the client side, this path is valid only if you chose the default path during installation. If this is not the case, substitute C:\ProgramFiles\ with the path that you had chosen.

Applications: Customer Response Solutions (CRA/CRS, IPCC Express, IP Autoattendant, ICD)

Trace masks for:

  • AutoAttendant: SS_TEL

  • ICD: SS_RM, SS_CM, SS_TEL, SS_RMCM, ICD_CTI, ICD_RTDM, STEP_ICD

  • Extension Mobility: GENERAL_STEPS*, LIB_LDAP, CCNUSER_STEPS*, SS_HTTP*, LIB_DIRECTORY*

* Only available in full CRA/CRS versions.

Applications: Issue-specific Required Information

Administrative Reporting Tool (ART)

On the ART server, extract these application, system, and security event logs:

  • ErrorLog.txt—Located at C:/ciscowebs/art

  • ARTSchError.log—Located at C:/ciscowebs/art/SchedulerService

  • ARTSchOutput.log—Located at C:/ciscowebs/art/SchedulerService

  • wrapper.properties—Located at C:/ciscowebs/art/SchedulerService

Tool for Auto-Registered Phone Support (TAPS)

These logs are generated for TAPS:

  • C:\CiscoWebs\BAT\Taps\TAPSTrace.txt

  • C:\CiscoWebs\BAT\Taps\LogFiles\Taps log file.txt

Related Information

Updated: Oct 08, 2006
Document ID: 18281