Table Of Contents

Troubleshooting

Troubleshooting

---

Following is a list of trouble symptoms and error messages that might arise as you configure and use Cisco SIP proxy server (Cisco SPS), with possible causes and recommended actions.

Also helpful for troubleshooting problems is the list of call flows in "SIP Call-Flow Scenarios."

Symptom    The error_log file contains the following line, and users can no longer register:

ACE_INET_Addr::ACE_INET_Addr: localhost: Invalid argument send_n failed: Broken pipe, expected len = 4, actual len = -1.

Possible Cause    A farm member is configured whose name or IP address cannot be resolved.

Recommended Action    Determine why the machine cannot resolve the name or address and fix that problem.

Symptom    Cisco SPS does not start.

Recommended Action   

1. Verify that the ./sip directory has the proper read-write permissions so that you can start sipd.

2. Verify that the proper environment variables are enabled.

3. Verify that the Linux RPM Package Manager (rpm) is correctly installed.

4. Verify that an old version of SIP proxy server (sipd) is not still running (enter ps -ef | grep sps). If necessary, kill the old-version processes.

5. Verify that Cisco SPS can use Domain Name System (DNS) to resolve its hostname.

6. Verify that your system has enough shared memory (at least the amount specified during installation).

Symptom    Cisco SPS takes a long time to start.

Possible Cause    A DNS is used before local files. Cisco SPS times out attempting to resolve DNS.

Recommended Action    To override this behavior, create /etc/irs.conf:

#############################################################

# /etc/irs.conf

#############################################################

# Get password entries from local file

passwd local

# Build group membership from local file

group local

# Services comes from just the local file

services local

protocols local

# Hosts comes first from local, failing that, DNS

hosts local continue

hosts dns

# Networks comes first from the local file, and failing # that the, irp daemon

 networks local continue

 networks irp

netgroup local

Symptom    Cisco SPS does not allow devices to register.

Possible Cause    Registration might be disabled in the GUI or the sipd.conf file.

Possible Cause    If authentication is required, one of the following might be the problem:

•The SIP-UA and password might not be defined in MySQL or the RADIUS server.

•Cisco SPS might be unable to connect to the MySQL database or RADIUS server.

Possible Cause    The type of authentication that the SIP-UA is trying to use might be set incorrectly. Cisco SPS supports only HTTP Basic and Digest authentication locally for SIP user agents; Challenge-Handshake Authentication Protocol (CHAP) is also supported on RADIUS.

Possible Cause    Access control lists prevent SIP user agents from registering.

Symptom    Cisco SPS routes calls improperly.

Recommended Action   

•Verify that the num-expansion statements are configured correctly.

•Verify that the various translation mechanisms (Static Registry, Dynamic Registry, ENUM, GKTMP) are configured correctly and are populated correctly.

•Verify that the Static Routing table shows correct routes.

•Verify that the DNS server is provisioned correctly for DNS SRV and DNS A records of the devices to be routed.

•Check the error_log to see what errors are reported (example: bad SIP messages, process errors).

Symptom    Cisco SPS farming works improperly.

Recommended Action   

•Verify that farm members have the same GUI or sipd.conf file configuration.

•Verify that each farm member contains an entry for the other farm member in its GUI or sipd.conf file configuration.

•Verify that both farm members are running the same version of Cisco SPS.

•Verify, using Network Time Protocol (NTP), that both farm members are synchronized to the same clock source.

Symptom    The GUI-based provisioning system cannot connect to pserver.

Recommended Action    Change the debug level in <ServerRoot>/sip/conf/prov/ps.conf to LOG_DEBUG. Then do the following.

•Verify that the pserver process is running. If it is not, restart it with ./sip restart.

•Verify that the pserver port is not used by another process.

•Verify that your session has not timed out. Log back in to the GUI and reauthenticate.

•Verify that the database information in ps.conf is correct. It should look like the following.

###########################################################

#  Provisioning Server (ps) Configuration File

###########################################################

DebugLevel					string			LOG_ERR

LogFilename					string			/usr/local/sip/logs/pserver_log

LocalPort					int			26005

DaemonMode					bool			True

NumThreads					int			1

DataStoreDir					string			/usr/local/sip/conf/prov/newdata

WebStoreDir					string			/usr/local/sip/conf/prov/www

Authentication					bool			True

UserAccountName					string			csps

GroupAccountName					string			csps

#

# DB info

#

# DB Backend

DBConnection					string			MYSQL

#MySql info sample

database					string			sip

host					string			localhost

user					string			guest

password					string			nobody

Symptom    pserver cannot connect to MySQL.

Recommended Action   

1. Manually attempt to connect to MySQL:

Login:  mysql -h<host> -u<user> -p<password>

show databases;

use sip; (sip should be listed as an option)

show tables;

select * from <table>;

2. If you can log in manually, verify that the correct MySQL IP address is configured.

3. Verify the permission (that is, the MySQL password).

Symptom    SIP provisioning agent (spa) does not receive updates from pserver.

Recommended Action   

1. Check error logs for any issues.

2. Log in to MySQL manually to confirm that the changes are present.

3. If changes are in MySQL, do the following:

a. Change the log level in <ServerRoot>/sip/conf/prov/spa.conf to LOG_DEBUG.

b. Stop and restart spa.

c. Tail -f spa_log.

d.Verify that spa receives the update.

e. If spa receives the update but problems persist, contact the Cisco Technical Assistance Center.

4. If changes are not in MySQL, verify that the tables are set up correctly and that the data is not corrupted.

Symptom    SIP provisioning agent (spa) cannot write to the sipd.conf file.

Possible Cause    The sipd.conf file may have been modified by a different user (such as root) so that, when spa tries to write to the file, it lacks the correct permissions.

Recommended Action   

1. Save the sipd.conf file under a different name.

2. Use the GUI-based provisioning system to apply the changes again so that spa writes a new sipd.conf file.

3. Restart Cisco SPS so that it picks up the changes.

Symptom    The error message sps_setup:not found appears when you attempt to run the SPS setup (sps_setup) script after successful installation of Cisco SPS on a Solaris machine.

Possible Cause    Bash needs to be installed.

Recommended Action    Install Bash.

Symptom    You are unable to invoke the csps_provision provisioning application.

Recommended Action    Verify that MySQL is up and running (use the ps -ef | grep mysql command). If command output indicates that MySQL is running, do the following:

1. Raise the debug level on the GUI and pserver by going to the sip/conf/prov/www directory and editing the log configuration (logConfig) file to uncomment the following line:

log4j.category.org.vovida.prov.psLib.HttpPSConnection=DEBUG

2. Change the log level in sip/conf/prov/ps.conf to LOG_DEBUG.

3. Perform a sip restart.

4. Run the sysadmin_csps_provision script and check the contents of both sysadmin_csps_provision_log and pserver_log for helpful information.

Symptom    When you start the provisioning server and enter the default password, the system rejects you.

Recommended Action   

1. Verify that pserver is running.

2. Verify that you are running the GUI-based provisioning system on the same box where pserver is running (use the ps -ef | grep pserver command).

3. Check the ./logs/pserver_log file for helpful information. If there is none, change the log level of pserver by editing ./conf/prov/ps.conf. Change the log level to LOG_DEBUG and then perform a ./sip restart. Afterwards, try opening the GUI again and see if more useful debugging information appears in the pserver_log file.

Symptom    Bulk import of data into MySQL stops before anything is written to the database.

Possible Cause    The pserver, upon checking the imported information, finds fields or values other than what it expects. When importing through the GUI-based provisioning system, the pserver checks the number of fields and some of the values. For example, it checks whether the port is within the correct range of values. If it finds a problem, it stops the import before writing anything to the database.

Recommended Action    Fix any problems and then import the data again.

Symptom    Error 2002: Can't connect to local MySQL server through socket....

Possible Cause    The MySQL database is not installed.

Recommended Action    Install the database before running the tool.

Symptom    Error 1045: Access denied for user.... Operation failed.

Possible Cause    Your MySQL username and password are invalid or have insufficient permission to access the database.

Recommended Action    Enter the correct or properly enabled username and password. If you have forgotten your password, assign a new one.

Symptom    Error 1116: Table `sip.subscriber...' doesn't exist. Operation failed.

Possible Cause    The database whose name you entered does not exist.

Recommended Action    Enter a valid name or reinstall the database.

Symptom    ERROR: Invalid user_id syntax.

Possible Cause    Your subscriber ID has invalid syntax.

Recommended Action    Check your subscriber ID and reenter it with valid syntax.

Symptom    ERROR: Invalid dest_url_cfna syntax.

Possible Cause    Your call-forwarding destination URL has invalid syntax.

Recommended Action    Check the URL and reenter it with valid syntax.

Symptom    When you use the regroute tool to list everything in the registry database, you get a segmentation fault.

Possible Cause    Your registration database might contain some garbage that the regroute tool cannot parse, causing the tool to crash, possibly due to a power outage or some other hard reset of the machine.

Recommended Action    If you can tolerate losing the data in your registration and route databases, create a new (empty but not corrupted) database by entering the following commands on each Cisco SPS in the farm:

1. ./bin/sip stop

2. rm ..data/*_db

3. ./bin/sip start

Then refresh registrations and add routes again.

Symptom    When you use the regroute tool, writes to MySQL fail.

Possible Cause    The regroute tool and the MySQL database might be from different Cisco SPS releases.

Recommended Action    Upgrade the earlier-release component.

Symptom    The regroute tool is unable to push a registration into MySQL.

Possible Cause    The regroute tool works under the assumption that the subscriber table is named "subscriber."

Recommended Action    Rename the subscriber table to the default "subscriber."

Symptom    The SIP proxy server (sipd) process stops and cannot be restarted. Or, when you start the server, the provisioning GUI works fine and then gets blocked after an update (particularly an update of a subscriber profile) or after waiting for awhile.

Possible Cause    You might have changed the IP address for one of your proxy servers. The new information displays in the GUI, but the old information remains in MySQL in addition to the new information, causing an IP address mismatch between what you set with the GUI (which updates sipd.conf) and what appears in the spa.conf file. (The GUI changes only certain things such as sipd.conf, subscriber data, static routes, and static registries; it does not change other things such as spa.conf, ps.conf, and dynamic registries.) The pserver tries to contact a SIP provisioning agent (spa) that no longer exists, and then hangs.

Recommended Action    Check the SPS setup log (sps_setup_log) to see which IP address you set up during installation. Use the ifconfig -a command to see what the IP address is now. If a mismatch exists, update both the GUI and the <server_root>/sip/conf/prov/spa.conf file and do a graceful restart.

Recommended Action    Check that any directives that contain IP addresses are set correctly. As necessary, modify them with the GUI. IP addresses appear in the following places:

•sipd.conf farm directives

•sipd.conf MySQL host directive on the secondary server

•spa.conf pointer to PServerHost on the secondary server

•spa.conf (if your system runs on a server with two NIC cards)

•The following MySQL tables: Seeding, License, TempLicense, and DBSubscriberTable

Symptom    Call forwarding does not configure properly.

Possible Cause    You might need to define the corresponding subscribers.

Recommended Action    Rather than editing an existing subscriber, add a new one:

user --> 5100

domain --> cisco.com

Add a call-forwarding URL for the new user and enable the corresponding call-forwarding feature in the sipd.conf file.

Symptom    When you add a static route, an error message states "Could not find out how many accounts are stored on the server." Static routes disappear from the page after system reboot.

Possible Cause    MySQL might be having problems accessing the data file for the routes.

Recommended Action    Try the following:

1. Check the MySQL file permissions in /opt/sip/data/sip. The files there should have MySQL read-write permission. If they do not, something went wrong with the MySQL installation.

2. Check to see if the CspsStaticRoute.MYD MySQL file is present. If it is not, some type of corruption may have occurred.

3. Log in to MySQL directly using the command line and grab all static routes that are stored in MySQL using the following commands:

%  mysql -hlocalhost -uguest -pnobody sip;

mysql> select * from CspsStaticRoute;

If the command line fails, the problem is with the MySQL tables.

4. Get more logging information from the pserver by changing the LOG_LEVEL in /opt/sip/conf/prov/ps.conf from LOG_ERR to LOG_DEBUG and restart the pserver.

Symptom    You experience a delay of several seconds from when Cisco SPS sends a 100 trying message to when it processes an INVITE message. If you disable the access-control list (ACL), the delay goes away.

Possible Cause    The delay might be caused by the Cisco SPS trying to match the caller's IP address with a hostname listed in the ACL.

Recommended Action    Use IP addresses rather than hostnames in the ACL.