Troubleshooting Guide
Preface
Downloads: This chapterpdf (PDF - 409.0KB) The complete bookPDF (PDF - 10.87MB) | Feedback

Preface

Table Of Contents

Preface

Introduction

Objective

Audience

Organization

Conventions

Updates to this Document for Release 5.0

New Events and Alarms (Release 4.5.1 to Release 5.0)

Modified Events and Alarms (Release 4.5.1 to Release 5.0)

Deleted Events and Alarms (Release 4.5.1 to Release 5.0)

Obtaining Documentation and Submitting a Service Request


Preface


Revised: July 22, 2009, OL-8723-19

Introduction

This document provides detailed troubleshooting procedures for the Cisco BTS 10200 Softswitch. It provides the procedures for troubleshooting network, subscriber, billing, operations and maintenance, system administrative problems. It also includes details for aiding troubleshooting by utilizing diagnostic and trace procedures.

Objective

The maintenance personnel or troubleshooters of a BTS 10200 can use this document to better understand how to troubleshoot the BTS 10200 and quickly clear network problems.

Audience

This document is designed for engineers, technicians, and system administrators who troubleshoot and repair the BTS 10200.

Organization

This Troubleshooting Guide contains the following chapters:

Chapter 1, "Troubleshooting Overview"—Provides an overview of troubleshooting the BTS 10200.

Chapter 2, "Audit Troubleshooting"—Provides the information needed to monitor and troubleshoot Audit events and alarms.

Chapter 3, "Billing Troubleshooting"—Provides the information needed to monitor and troubleshoot Billing events and alarms.

Chapter 4, "Call Processing Troubleshooting"—Provides the information needed to monitor and troubleshoot Call Processing events and alarms.

Chapter 5, "Configuration Troubleshooting"—Provides the information needed to monitor and troubleshoot Configuration events and alarms.

Chapter 6, "Database Troubleshooting"—Provides the information needed to monitor and troubleshoot Database events and alarms.

Chapter 7, "Maintenance Troubleshooting"—Provides the information needed to monitor and troubleshoot Maintenance events and alarms.

Chapter 8, "Operations Support System Troubleshooting"—Provides the information needed to monitor and troubleshoot Operations Support System events and alarms.

Chapter 9, "Security Troubleshooting"—Provides the information needed to monitor and troubleshoot Security events and alarms.

Chapter 10, "Signaling Troubleshooting"—Provides the information needed to monitor and troubleshoot Signaling events and alarms.

Chapter 11, "Statistics Troubleshooting"—Provides the information needed to monitor and troubleshoot Statistics events and alarms.

Chapter 12, "System Troubleshooting"—Provides the information needed to monitor and troubleshoot System events and alarms.

Chapter 13, "Network Troubleshooting"—Provides the information needed to conduct network troubleshooting on the BTS 10200.

Chapter 14, "General Troubleshooting"—Provides the general troubleshooting information needed to conduct troubleshooting on the BTS 10200.

Chapter 15, "Diagnostic Tests"—Describes the diagnostic tests that can be performed on media gateways, subscriber terminations, and trunk terminations.

Chapter 16, "Disaster Recovery Procedures"Describes how to recover the database in a disaster situation, how to recover the database from another database, and how to recover data from the Call Agent shared memory.

Chapter 17, "Disk Replacement"—Describes how to manually recover a BTS 10200 system.

Appendix A, "Recoverable and Nonrecoverable Error Codes"—Lists normal, recoverable and nonrecoverable error codes for the BTS 10200.

Appendix B, "System Usage of MGW Keepalive Parameters, Release 5.0"—Explains how the BTS 10200 determines the connectivity status between itself and a media gateway (MGW).

Appendix C, "Overload Control"—Overload is a switch condition that exists when system resources cannot handle system tasks. Increases in call traffic or messages indirectly related to call traffic usually cause overload.

Conventions

This document uses the following conventions:


Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.



Caution Means reader be careful. In this situation, you might do something that could result in equipment damage or loss of data.

Updates to this Document for Release 5.0

This document was updated from Version OL-8723-01 to Version OL-8723-02 to incorporate the alarms and events deltas between Release 4.5.1 and Release 5.0.

This document was updated from Version OL-8723-02 to Version OL-8723-03 to incorporate additional information and remove the release cause code information.

This document was updated from Version OL-8723-03 to Version OL-8723-04 to incorporate updated threshold and throttle information and to incorporate the pre-switch feature information.

This document was updated from Version OL-8723-04 to Version OL-8723-05 to incorporate additional alarm in the following chapters:

Chapter 6, "Database Troubleshooting"

Chapter 8, "Operations Support System Troubleshooting"

Chapter 10, "Signaling Troubleshooting"

This document was updated from Version OL-8723-05 to Version OL-8723-06 to incorporate updated alarm information.

This document was updated from Version OL-8723-06 to Version OL-8723-08 to incorporate updated keepalive information.

This document was updated from Version OL-8723-08 to Version OL-8723-09 to incorporate additional Signaling (151) troubleshooting information and the "Viewing Trace Logs for Throttled Flood of MGCP Messages from Specific Endpoint" section in Chapter 14, "General Troubleshooting."

Updated Audit (25) and Signaling (124) information for OL-8723-09 to OL-8723-10.

This document was updated from Version OL-8723-10 to Version OL-8723-11 to incorporate information about "Platform Core File Alarm" feature.

This document was updated from Version OL-8723-11 to Version OL-8723-12 to remove the erroneous Throttle/Threshold note in Chapter 1, "Troubleshooting Overview."

This document was updated from Version OL-8723-12 to Verison OL-8723-13 to update the disk replacement procedures in Chapter 17, "Disk Replacement."

This document was updated from Version OL-8723-13 to Version OL-8723-14 to update the shared memory recovery procedure and include the call agent database download and restore procedure.

The update to Version OL-8723-15 included updating the Audit 4 alarm information.

The update to Version OL-8723-16 included adding the "PSTN Trunk Testing" section on page 15-113.

The update to Version OL-8723-17 included updating the "Automatic Shared Memory Backup Restore" section on page 16-35 and the "Signaling System 7 Trunk Termination Tests" section on page 15-6.

The update to Verison OL-8723-18 included adding the "Oracle Database Tool Restart" section on page 13-9.

The update to Version OL-8723-19 included updating the "Recovering Shared Memory" section on page 16-30.

New Events and Alarms (Release 4.5.1 to Release 5.0)

lists the new events and alarms that were incorporated into the BTS 10200 system as part of the update from Release 4.5.1 to Release 5.0.

Table 1 New Events and Alarms (Release 4.5.1 to Release 5.0) 

Type and Number
Description

AUDIT (21)

Quality of Service Gate Memory Audit Complete (QoS Gate Memory Audit Complete)

AUDIT (22)

Quality of Service Gate Status Audit Complete (QoS Gate Status Audit Complete)

AUDIT (23)

Recover Memory of Dangling Gate

AUDIT (24)

No Gate in the Cable Modem Termination System for Active Connection (No Gate in CMTS for Active Connection)

AUDIT (25)

Core File Present

BILLING (59)

Billing File Name Type Change in the Command Line Interface is Inconsistent

CALLP (42)

Call Failed after Local Number Portability Query with Location Routing Number of this BTS 10200 and the Directory Number (Call Failed after LNP Query with LRN of this switch and the DN)

CALLP (43)

Call Processing Session Initiation Protocol Trigger Provisioning Error (CALLP SIP Trigger Provisioning Error)

CALLP (44)

Call Processing No Session Initiation Protocol Trigger Context Found (CALLP No SIP Trig Context Found)

CALLP (45)

Context in Call from Application Server not Found (Context in Call from App Server not Found)

CONFIG (6)

FIMXML Parse Error

CONFIG (7)

Application Server Provisioning Error

CONFIG (8)

BTS 10200 Provisioning for External Applications is not Complete

DATABASE (22)

Unprovisioned Language

DATABASE (23)

Element Management System Oracle Database-Minor Error (EMS Oracle Database (ORA)-Minor Error)

DATABASE (24)

Element Management System Oracle Database-Minor Error (EMS Oracle Database (ORA)-Major Error)

DATABASE (27)

Failure Setting the Index Table Soft Limit (Failure Setting the IDX Table Soft Limit)

MAINTENANCE (112)

Congestion Status

MAINTENANCE (113)

Central Processing Unit Load of Critical Processes (CPU Load of Critical Processes)

MAINTENANCE (114)

Queue Length of Critical Processes

MAINTENANCE (115)

Inter-Process Communication Buffer Usage Level (IPC Buffer Usage Level)

MAINTENANCE (116)

Call Agent Reports the Congestion Level of the Feature Server (CA Reports the Congestion Level of FS)

MAINTENANCE (117)

Side Automatically Restarting Due to Fault

MAINTENANCE (118)

Domain Name Server Zone Database does not Match Between the Primary Domain Name Server and the Internal Secondary Authoritative Domain Name Server (DNS Zone Database does not Match Between the Primary DNS and the ISADS)

MAINTENANCE (121)

Invalid SOAP Request

OSS (12)

User Session Count is Approaching Threshold Limit

OSS (14)

User Session Count Exceeds Major Threshold Limit

OSS (16)

Session Has Been Removed by Session Control Policy

SIGNALING (153)

Emergency Trunks Become Locally Blocked

SIGNALING (154)

Emergency Trunks Become Remotely Blocked

SIGNALING (155)

Packet Cable Multi-Media Unsolicited Gate Delete (PCMM Unsolicited Gate Delete)

SIGNALING (156)

Integrated Services Digital Network Signaling Gateway Down (ISDN Signaling Gateway Down)

SIGNALING (157)

Integrated Services Digital Network Signaling Gateway Inactive (ISDN Signaling Gateway Inactive)

SIGNALING (158)

Invalid Integrated Services Digital Network Interface Identification (Invalid ISDN Interface ID)

SIGNALING (159)

Integrated Services Digital Network User Adaptation Layer can not Go Active (IUA can not Go Active)

SIGNALING (160)

Integrated Services Digital Network User Adaptation Layer can not Go Standby (IUA can not Go Standby)

SIGNALING (161)

Session Initiation Protocol Update not Allowed for Operator Service Position System Calls (SIP Update not Allowed for OSPS Calls)

SIGNALING (162)

Session Initiation Protocol Server Group Element Operationally Out of Service (SIP Server Group Element Operationally out of Service)

SIGNALING (163)

Routing Key Inactive

SIGNALING (164)

Signaling Gateway Traffic Mode Mismatch

SIGNALING (165)

No Session Initiation Protocol P-DCS Billing Information Header Received (No SIP P-DCS Billing Info Hdr Rcvd)

SIGNALING (168)

A Session Initiation Protocol Server Group has no Child Elements Provisioned (A SIP Server Group has no Child Elements Provisioned)

SIGNALING (169)

Session Initiation Protocol Element Provisioned with Service Enabled is Internally Disabled (SIP Element Provisioned with SRV Enabled is Internally Disabled)

SIGNALING (170)

Residential Gateway Endpoints are out of Service at the Gateway (Residential Gateway Endpoints are out of Service at the GW)

SIGNALING (171)

Residential Gateway Unreachable

SIGNALING (172)

Multimedia Terminal Adapter Effective-Aggr-Id Becomes Unavailable Due to its IP Address (MTA Effective-Aggr-Id Becomes Unavailable Due to its IP Address)

SIGNALING (173)

ENUM Server Domain Cannot be Resolved into Any IP Address

SIGNALING (174)

ENUM Server Unavailable

SIGNALING (175)

ENUM Server Farm Unavailable

SIGNALING (176)

No Resources Available to Launch ENUM Query

SIGNALING (177)

ISDN Unable to Restore D-Channel into In-Service Active State

STATISTICS (14)

MDII Trunk

STATISTICS (15)

Threshold Crossing Alert

STATISTICS (16)

Trunk Group has Reached the MDII Alarm Threshold


Modified Events and Alarms (Release 4.5.1 to Release 5.0)

lists the BTS 10200 modified events and alarms that were modified as part of the upgrade from Release 4.5.1 to Release 5.0.

Table 2 Modified Events and Alarms (Release 4.5.1 to Release 5.0) 

Type and Number
Description

Monitoring Billing Events

Bad File Detected During Startup

DATABASE (3)

There Are Errors in Element Management System Database DefError Queue; Contact Database Administrator (There are Errors in EMS Database DefError Queue; Contact DBA)

DATABASE (11)

Element Management System Database Alert.log Alerts (EMS Database Alert.log Alerts)

MAINTENANCE (63)

Outgoing Heartbeat Period Exceeded Limit (Outgoing HB Period Exceeded Limit)

MAINTENANCE (74)

Local Domain Name System Service not Responsive (Local DNS Service not Responsive)

SIGNALING (111)

Stream Control Transmission Protocol Association Degraded (One of Two Internet Protocol Connections Down) (SCTP Association Degraded (One of Two IP connections Down))

SIGNALING (121)

Message Transfer Part 3 User Adapter Cannot Go Standby (M3UA/SUA Cannot Go Standby)

SIGNALING (122)

Message Transfer Part 3 User Adapter Cannot Go Active (M3UA/SUA Cannot Go Active)

SIGNALING (141)

Address not Resolved by Domain Name System Server (Address not Resolved by DNS Server)

SIGNALING (142)

Session Initiation Protocol Trunk Operationally Out-of-Service (SIP Trunk Operationally out of Service)

SYSTEM (9)

Timer Start Failure


Deleted Events and Alarms (Release 4.5.1 to Release 5.0)

No events or alarms are deleted as the result of the upgrade from Release 4.5.1 to Release 5.0.

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.