Table Of Contents
Preface
Introduction
Objective
Audience
Organization
Conventions
Updates to this Document for Release 5.0
New Events and Alarms (Release 4.5.1 to Release 5.0)
Modified Events and Alarms (Release 4.5.1 to Release 5.0)
Deleted Events and Alarms (Release 4.5.1 to Release 5.0)
Obtaining Documentation and Submitting a Service Request
Preface
Revised: July 22, 2009, OL-8723-19
Introduction
This document provides detailed troubleshooting procedures for the Cisco BTS 10200 Softswitch. It provides the procedures for troubleshooting network, subscriber, billing, operations and maintenance, system administrative problems. It also includes details for aiding troubleshooting by utilizing diagnostic and trace procedures.
Objective
The maintenance personnel or troubleshooters of a BTS 10200 can use this document to better understand how to troubleshoot the BTS 10200 and quickly clear network problems.
Audience
This document is designed for engineers, technicians, and system administrators who troubleshoot and repair the BTS 10200.
Organization
This Troubleshooting Guide contains the following chapters:
•
Chapter 1, "Troubleshooting Overview"—Provides an overview of troubleshooting the BTS 10200.
•Chapter 2, "Audit Troubleshooting"—Provides the information needed to monitor and troubleshoot Audit events and alarms.
•Chapter 3, "Billing Troubleshooting"—Provides the information needed to monitor and troubleshoot Billing events and alarms.
•Chapter 4, "Call Processing Troubleshooting"—Provides the information needed to monitor and troubleshoot Call Processing events and alarms.
•Chapter 5, "Configuration Troubleshooting"—Provides the information needed to monitor and troubleshoot Configuration events and alarms.
•Chapter 6, "Database Troubleshooting"—Provides the information needed to monitor and troubleshoot Database events and alarms.
•Chapter 7, "Maintenance Troubleshooting"—Provides the information needed to monitor and troubleshoot Maintenance events and alarms.
•Chapter 8, "Operations Support System Troubleshooting"—Provides the information needed to monitor and troubleshoot Operations Support System events and alarms.
•Chapter 9, "Security Troubleshooting"—Provides the information needed to monitor and troubleshoot Security events and alarms.
•Chapter 10, "Signaling Troubleshooting"—Provides the information needed to monitor and troubleshoot Signaling events and alarms.
•Chapter 11, "Statistics Troubleshooting"—Provides the information needed to monitor and troubleshoot Statistics events and alarms.
•Chapter 12, "System Troubleshooting"—Provides the information needed to monitor and troubleshoot System events and alarms.
•Chapter 13, "Network Troubleshooting"—Provides the information needed to conduct network troubleshooting on the BTS 10200.
•Chapter 14, "General Troubleshooting"—Provides the general troubleshooting information needed to conduct troubleshooting on the BTS 10200.
•Chapter 15, "Diagnostic Tests"—Describes the diagnostic tests that can be performed on media gateways, subscriber terminations, and trunk terminations.
•Chapter 16, "Disaster Recovery Procedures"—Describes how to recover the database in a disaster situation, how to recover the database from another database, and how to recover data from the Call Agent shared memory.
•Chapter 17, "Disk Replacement"—Describes how to manually recover a BTS 10200 system.
•Appendix A, "Recoverable and Nonrecoverable Error Codes"—Lists normal, recoverable and nonrecoverable error codes for the BTS 10200.
•Appendix B, "System Usage of MGW Keepalive Parameters, Release 5.0"—Explains how the BTS 10200 determines the connectivity status between itself and a media gateway (MGW).
•Appendix C, "Overload Control"—Overload is a switch condition that exists when system resources cannot handle system tasks. Increases in call traffic or messages indirectly related to call traffic usually cause overload.
Conventions
This document uses the following conventions:
Note Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.
Caution Means
reader be careful. In this situation, you might do something that could result in equipment damage or loss of data.
Updates to this Document for Release 5.0
This document was updated from Version OL-8723-01 to Version OL-8723-02 to incorporate the alarms and events deltas between Release 4.5.1 and Release 5.0.
This document was updated from Version OL-8723-02 to Version OL-8723-03 to incorporate additional information and remove the release cause code information.
This document was updated from Version OL-8723-03 to Version OL-8723-04 to incorporate updated threshold and throttle information and to incorporate the pre-switch feature information.
This document was updated from Version OL-8723-04 to Version OL-8723-05 to incorporate additional alarm in the following chapters:
•Chapter 6, "Database Troubleshooting"
•Chapter 8, "Operations Support System Troubleshooting"
•Chapter 10, "Signaling Troubleshooting"
This document was updated from Version OL-8723-05 to Version OL-8723-06 to incorporate updated alarm information.
This document was updated from Version OL-8723-06 to Version OL-8723-08 to incorporate updated keepalive information.
This document was updated from Version OL-8723-08 to Version OL-8723-09 to incorporate additional Signaling (151) troubleshooting information and the "Viewing Trace Logs for Throttled Flood of MGCP Messages from Specific Endpoint" section in Chapter 14, "General Troubleshooting."
Updated Audit (25) and Signaling (124) information for OL-8723-09 to OL-8723-10.
This document was updated from Version OL-8723-10 to Version OL-8723-11 to incorporate information about "Platform Core File Alarm" feature.
This document was updated from Version OL-8723-11 to Version OL-8723-12 to remove the erroneous Throttle/Threshold note in Chapter 1, "Troubleshooting Overview."
This document was updated from Version OL-8723-12 to Verison OL-8723-13 to update the disk replacement procedures in Chapter 17, "Disk Replacement."
This document was updated from Version OL-8723-13 to Version OL-8723-14 to update the shared memory recovery procedure and include the call agent database download and restore procedure.
The update to Version OL-8723-15 included updating the Audit 4 alarm information.
The update to Version OL-8723-16 included adding the "PSTN Trunk Testing" section on page 15-113.
The update to Version OL-8723-17 included updating the "Automatic Shared Memory Backup Restore" section on page 16-35 and the "Signaling System 7 Trunk Termination Tests" section on page 15-6.
The update to Verison OL-8723-18 included adding the "Oracle Database Tool Restart" section on page 13-9.
The update to Version OL-8723-19 included updating the "Recovering Shared Memory" section on page 16-30.
New Events and Alarms (Release 4.5.1 to Release 5.0)
lists the new events and alarms that were incorporated into the BTS 10200 system as part of the update from Release 4.5.1 to Release 5.0.
Table 1 New Events and Alarms (Release 4.5.1 to Release 5.0)
Type and Number
|
Description
|
AUDIT (21)
|
Quality of Service Gate Memory Audit Complete (QoS Gate Memory Audit Complete)
|
AUDIT (22)
|
Quality of Service Gate Status Audit Complete (QoS Gate Status Audit Complete)
|
AUDIT (23)
|
Recover Memory of Dangling Gate
|
AUDIT (24)
|
No Gate in the Cable Modem Termination System for Active Connection (No Gate in CMTS for Active Connection)
|
AUDIT (25)
|
Core File Present
|
BILLING (59)
|
Billing File Name Type Change in the Command Line Interface is Inconsistent
|
CALLP (42)
|
Call Failed after Local Number Portability Query with Location Routing Number of this BTS 10200 and the Directory Number (Call Failed after LNP Query with LRN of this switch and the DN)
|
CALLP (43)
|
Call Processing Session Initiation Protocol Trigger Provisioning Error (CALLP SIP Trigger Provisioning Error)
|
CALLP (44)
|
Call Processing No Session Initiation Protocol Trigger Context Found (CALLP No SIP Trig Context Found)
|
CALLP (45)
|
Context in Call from Application Server not Found (Context in Call from App Server not Found)
|
CONFIG (6)
|
FIMXML Parse Error
|
CONFIG (7)
|
Application Server Provisioning Error
|
CONFIG (8)
|
BTS 10200 Provisioning for External Applications is not Complete
|
DATABASE (22)
|
Unprovisioned Language
|
DATABASE (23)
|
Element Management System Oracle Database-Minor Error (EMS Oracle Database (ORA)-Minor Error)
|
DATABASE (24)
|
Element Management System Oracle Database-Minor Error (EMS Oracle Database (ORA)-Major Error)
|
DATABASE (27)
|
Failure Setting the Index Table Soft Limit (Failure Setting the IDX Table Soft Limit)
|
MAINTENANCE (112)
|
Congestion Status
|
MAINTENANCE (113)
|
Central Processing Unit Load of Critical Processes (CPU Load of Critical Processes)
|
MAINTENANCE (114)
|
Queue Length of Critical Processes
|
MAINTENANCE (115)
|
Inter-Process Communication Buffer Usage Level (IPC Buffer Usage Level)
|
MAINTENANCE (116)
|
Call Agent Reports the Congestion Level of the Feature Server (CA Reports the Congestion Level of FS)
|
MAINTENANCE (117)
|
Side Automatically Restarting Due to Fault
|
MAINTENANCE (118)
|
Domain Name Server Zone Database does not Match Between the Primary Domain Name Server and the Internal Secondary Authoritative Domain Name Server (DNS Zone Database does not Match Between the Primary DNS and the ISADS)
|
MAINTENANCE (121)
|
Invalid SOAP Request
|
OSS (12)
|
User Session Count is Approaching Threshold Limit
|
OSS (14)
|
User Session Count Exceeds Major Threshold Limit
|
OSS (16)
|
Session Has Been Removed by Session Control Policy
|
SIGNALING (153)
|
Emergency Trunks Become Locally Blocked
|
SIGNALING (154)
|
Emergency Trunks Become Remotely Blocked
|
SIGNALING (155)
|
Packet Cable Multi-Media Unsolicited Gate Delete (PCMM Unsolicited Gate Delete)
|
SIGNALING (156)
|
Integrated Services Digital Network Signaling Gateway Down (ISDN Signaling Gateway Down)
|
SIGNALING (157)
|
Integrated Services Digital Network Signaling Gateway Inactive (ISDN Signaling Gateway Inactive)
|
SIGNALING (158)
|
Invalid Integrated Services Digital Network Interface Identification (Invalid ISDN Interface ID)
|
SIGNALING (159)
|
Integrated Services Digital Network User Adaptation Layer can not Go Active (IUA can not Go Active)
|
SIGNALING (160)
|
Integrated Services Digital Network User Adaptation Layer can not Go Standby (IUA can not Go Standby)
|
SIGNALING (161)
|
Session Initiation Protocol Update not Allowed for Operator Service Position System Calls (SIP Update not Allowed for OSPS Calls)
|
SIGNALING (162)
|
Session Initiation Protocol Server Group Element Operationally Out of Service (SIP Server Group Element Operationally out of Service)
|
SIGNALING (163)
|
Routing Key Inactive
|
SIGNALING (164)
|
Signaling Gateway Traffic Mode Mismatch
|
SIGNALING (165)
|
No Session Initiation Protocol P-DCS Billing Information Header Received (No SIP P-DCS Billing Info Hdr Rcvd)
|
SIGNALING (168)
|
A Session Initiation Protocol Server Group has no Child Elements Provisioned (A SIP Server Group has no Child Elements Provisioned)
|
SIGNALING (169)
|
Session Initiation Protocol Element Provisioned with Service Enabled is Internally Disabled (SIP Element Provisioned with SRV Enabled is Internally Disabled)
|
SIGNALING (170)
|
Residential Gateway Endpoints are out of Service at the Gateway (Residential Gateway Endpoints are out of Service at the GW)
|
SIGNALING (171)
|
Residential Gateway Unreachable
|
SIGNALING (172)
|
Multimedia Terminal Adapter Effective-Aggr-Id Becomes Unavailable Due to its IP Address (MTA Effective-Aggr-Id Becomes Unavailable Due to its IP Address)
|
SIGNALING (173)
|
ENUM Server Domain Cannot be Resolved into Any IP Address
|
SIGNALING (174)
|
ENUM Server Unavailable
|
SIGNALING (175)
|
ENUM Server Farm Unavailable
|
SIGNALING (176)
|
No Resources Available to Launch ENUM Query
|
SIGNALING (177)
|
ISDN Unable to Restore D-Channel into In-Service Active State
|
STATISTICS (14)
|
MDII Trunk
|
STATISTICS (15)
|
Threshold Crossing Alert
|
STATISTICS (16)
|
Trunk Group has Reached the MDII Alarm Threshold
|
Modified Events and Alarms (Release 4.5.1 to Release 5.0)
lists the BTS 10200 modified events and alarms that were modified as part of the upgrade from Release 4.5.1 to Release 5.0.
Table 2 Modified Events and Alarms (Release 4.5.1 to Release 5.0)
Type and Number
|
Description
|
Monitoring Billing Events
|
Bad File Detected During Startup
|
DATABASE (3)
|
There Are Errors in Element Management System Database DefError Queue; Contact Database Administrator (There are Errors in EMS Database DefError Queue; Contact DBA)
|
DATABASE (11)
|
Element Management System Database Alert.log Alerts (EMS Database Alert.log Alerts)
|
MAINTENANCE (63)
|
Outgoing Heartbeat Period Exceeded Limit (Outgoing HB Period Exceeded Limit)
|
MAINTENANCE (74)
|
Local Domain Name System Service not Responsive (Local DNS Service not Responsive)
|
SIGNALING (111)
|
Stream Control Transmission Protocol Association Degraded (One of Two Internet Protocol Connections Down) (SCTP Association Degraded (One of Two IP connections Down))
|
SIGNALING (121)
|
Message Transfer Part 3 User Adapter Cannot Go Standby (M3UA/SUA Cannot Go Standby)
|
SIGNALING (122)
|
Message Transfer Part 3 User Adapter Cannot Go Active (M3UA/SUA Cannot Go Active)
|
SIGNALING (141)
|
Address not Resolved by Domain Name System Server (Address not Resolved by DNS Server)
|
SIGNALING (142)
|
Session Initiation Protocol Trunk Operationally Out-of-Service (SIP Trunk Operationally out of Service)
|
SYSTEM (9)
|
Timer Start Failure
|
Deleted Events and Alarms (Release 4.5.1 to Release 5.0)
No events or alarms are deleted as the result of the upgrade from Release 4.5.1 to Release 5.0.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.
Feedback
