Troubleshooting Cisco Emergency Responder System and Administration Problems
This chapter covers topics that help you troubleshoot problems related to the Cisco ER system and its administration, such as server and web server problems:
•Cannot Validate Publisher, page 5-1
•Troubleshooting Login Problems, page 5-1
•Using Cisco Unified Operations Manager, page 5-2
•Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems, page 5-2
•Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration, page 5-4
•Replacing the Publisher Server and Subscriber Servers, page 5-4
•Using the Cisco Emergency Responder Admin Utility, page 5-5
•Troubleshooting the Database and Enterprise Replication, page 5-6
•Troubleshooting Cisco Emergency Responder System Problems, page 5-7
•Troubleshooting Cisco Unified Communications Manager Configuration Problems, page 5-8
Cannot Validate Publisher
If the installation cannot validate the Publisher (Step 5 of the "Installing the Cisco Emergency Responder Subscriber" section on page 2-8), check the following:
1. Verify that the Publisher hostname is correct and that the Publisher is reachable by hostname.
2. Verify that the Publisher and Subscriber servers are running the same version of Cisco ER.
3. Verify that the database password that you entered is correct. This password was specified on the Database Access Security Configuration page during installation.
4. Make sure that the Subscriber has been configured correctly on the Publisher.
Troubleshooting Login Problems
These are some issues you might encounter while logging into Cisco ER:
Symptom You cannot log in to the Cisco ER Administration website.
Recommended Action Log in to CLI and run the utils service list command. Check if the status "Cisco IDS" is STARTED. If not, start the service using the utils service start service name command.
Symptom You cannot open multiple Cisco ER sessions using Netscape Navigator.
Recommended Action Netscape/Mozilla Navigator uses the same session ID across multiple windows. This creates problems if you try to log into Cisco ER using different IDs. Normally, you can open multiple windows when logged in as system administrator. With Internet Explorer, if you open separate IE session by starting a new IE instance (rather than by opening a new window from an existing session), IE uses different session IDs, and you should be able to log in using separate IDs (for example, as a user and an administrator, or as LAN switch and ERL administrators).
•Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration, page 5-4
Using Cisco Unified Operations Manager
Use Cisco Unified Operations Manager 2.01 to continuously monitor the health of the Cisco ER system.
For information on setting up Cisco ER to use Cisco Unified Operations Manager, see the Configuring Test ERLs.
For information on installing and using Cisco Unified Operations Manager, see the documentation at:
Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems
Some issues that you might encounter while configuring switches or switch ports in Cisco ER are shown below:
Symptom Cisco ER is configured with Cisco Unified CM information, but no phones get discovered.
Recommended Action Ensure that the Cisco Unified CM servers are reachable on the network. Then, ensure that the SNMP read community strings are configured correctly for the switches and Cisco Unified CM servers (see the Configuring the SNMP Connection.) Then, manually run the switch port and phone update process (see the Manually Running the Switch-Port and Phone Update Process.) Use the CLI-based utils snmp command to determine if the Cisco Unified CM is SNMP reachable.
Symptom Cisco ER does not show the ports on a switch configured in Cisco ER.
Recommended Action If you add a supported switch to Cisco ER and run phone tracking on the switch after adding it, you should be able to view the list of Ethernet ports on the switch. If Cisco ER does not list the ports, check the SNMP settings in Cisco ER for the switch (see the Configuring the SNMP Connection.) Also, verify that the switch is reachable over the network. Retry the selective phone tracking process on the switch (click Locate Switch Ports when viewing the switch details; see the LAN Switch Details.)
If the problem persists, ensure that the switch is supported (see the Network Hardware and Software Requirements.) Also, check the Event Viewer for error messages.
Symptom Some phones do not appear in the switch port list.
Recommended Action Check if the phone is found under configured IP subnets or in synthetic phones. If it is not found in either of those places, then they will be place un unlocated phones. See the Too Many Unlocated Phones for a list of reasons that a phone could not be located.
Symptom Cannot delete a switch from the Cisco ER configuration.
Recommended Action You cannot delete a switch when a phone tracking process is in progress. Retry the deletion after the process has ended. If this is not the problem, the Cisco ER server might not be running. Check the control center and restart the server (see the Starting and Stopping a Cisco Emergency Responder Server.)
Symptom Import or export of the switch port details fails.
Recommended Action If a switch port import or export attempt fails, it might be due to these reasons: the first switch-port and phone update process has not yet ended (wait for it to finish); the Cisco ER server is not running (use the control center to restart it, see the Starting and Stopping a Cisco Emergency Responder Server); the Cisco ER server is not completely initialized (wait for it to initialize).
Symptom The import of some switch port configurations fail.
Recommended Action To import switch port configurations, Cisco ER must already be configured with the switch and Cisco ER must first discover the ports on the switch using the switch-port and phone update process. If you try to import a configuration for ports not yet discovered in Cisco ER, the importation of those settings fails. See the Manually Running the Switch-Port and Phone Update Process for information on the process. Run it on the switches whose port configurations you could not import, then retry the import.
Symptom Phones moved from other Cisco ER groups to this Cisco ER group, and then moved back, are still showing up in the switch port details for the Cisco ER group.
Recommended Action This types of phones are not removed from the switch port details until the next full switch-port and phone update process is run. If this is an issue for you, you can run the process on the switch (or on all switches) manually. See the Manually Running the Switch-Port and Phone Update Process.
Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration
The ERL Debug Tool takes a phone extension as the search criteria and displays the ERL(s) currently being used for routing emergency calls for the phone(s).
Use this diagnostic tool to verify the Cisco ER configuration during the ERL creation and the ERL assignment phase, and to troubleshoot calls directed to incorrect ERLs.
For example, you configured the phone in ERL_1 as a manually configured phone, however a misconfigured IP subnet matches this phone's IP address, and associates it with ERL_2. Now that you have found the configuration problem using the Debug Tool, you can correct it.
To use the ERL Debug Tool, follow these steps.
Step 1 Select Tools > ERL Debug Tool.
Cisco ER displays the ERL Debug Tool page.
Step 2 At the Find Phones field, to list specific phones, select the search criteria and click Find.
Cisco ER displays the ERL currently being used for routing emergency calls for the phone.
Step 3 If the configurations are not correct, make the required changes.
Note Cisco ER displays a maximum of 1,000 records.
Replacing the Publisher Server and Subscriber Servers
If you need to replace a faulty Publisher server or a faulty Subscriber server, perform the appropriate procedure:
•Replacing a Faulty Subscriber, page 5-4
•Replacing a Faulty Publisher, page 5-5
Replacing a Faulty Subscriber
To replace a faulty Subscriber, go to Cisco ER administration and delete the faulty Subscriber. Install a new Cisco ER Subscriber for the Publisher (see the Installing Cisco Emergency Responder 8.0 on a New System).
Note If the same host name is not going to be used by the replacement Subscriber server, you must delete the faulty Subscriber using the Cisco ER administration screen on the Publisher server.
Replacing a Faulty Publisher
You can restore the Publisher only if you have backed up the Publisher using the Disaster Recovery System available as part of the Cisco ER. See the Backing Up and Recovering Data.
To replace a faulty Publisher, follow these steps:
Step 1 Install the same version of the Cisco ER Publisher on a server with the same host name as the one you used previously.
Step 2 Choose the same configuration options (such as the Cisco Unified CM version, and so on) during the installation.
Step 3 Restore the old configuration data using the Disaster Recovery System.
Using the Cisco Emergency Responder Admin Utility
You can use the Cisco ER Admin Utility tool to perform the following tasks:
•To point a Subscriber to a different Publisher
•To update Cisco ER cluster database host details
•To upgrade the CCM version
This section describes the following topic:
•How to Use the Cisco ER Admin Utility Tool, page 5-5
How to Use the Cisco ER Admin Utility Tool
To use the Cisco ER Admin Utility tool, follow these steps:
Step 1 Log in to the Cisco ER Admin Utility web interface.
Step 2 Using the menu bar, choose a task to perform:
a. To change the Publisher that the Subscriber server points to, select Update > Publisher.
b. To update the Cisco Unified CM version, select Update > CCM Version.
c. To update the cluster settings on both the Publisher and Subscriber servers, select Cluster > DBHost.
Note This action updates the Cisco ER cluster DB details for this server group only. Other servers in this Cisco ER cluster will NOT be updated automatically.
Step 3 To save the changes that you have made, restart both the Publisher and the Subscriber servers.
Troubleshooting the Subscriber Database Setup
To configure the Publisher-Subscriber setup again if you have an issue with the Subscriber (apart from DB replication), follow these steps:
Step 1 Log in to the Cisco ER Admin Utility web interface on the Subscriber server.
Step 2 Select Update > Publisher.
Step 3 Specify the same Publisher Host Name, IP address (already being pointed to) and database access security password.
Step 4 Click Go.
This step might a take a while setup.
Troubleshooting the Database and Enterprise Replication
Use the following CLI commands for troubleshooting the Informix Dynamic Server (IDS) database:
•utils service list—Used to check whether the IDS service is running or not
•show tech dbstateinfo—Gives the DB state information which is helpful in debugging database issues
•show tech dbinuse—Displays the currently used database
•show tech dbintegrity—Shows database integrity information
•show tech database—Creates a .csv file with contents of all the tables in the database
Use the following CLI commands for troubleshooting Enterprise Replication:
•utils dbreplication status—Used to show the status of the database replication
•utils dbreplication reset—Resets and restarts the database replication between the Publisher and Subscriber
•utils dbreplication repair—Compares the data on replication servers (Publisher and Subscriber) and create a report listing data inconsistencies and repairs the data inconsistencies. This command also tries to repair replication by rebuilding the corrupted .rhosts file if it is corrupted for some reason.
For troubleshooting database problems using logs, download logs from the Cisco ER Serviceability website or through CLI.
The following logs provide information for debugging database related issues
•Install DB logs—/var/log/active/er/trace/dbl/sdi/
Symptom Replication fails to start after the Subscriber is installed with DNS and the CLI command utils dbreplication status shows replication not working.
Possible Cause The .rhosts will have the Host Name for the Subscriber instead of FQDN (Fully Qualified Domain Name) of the Subscriber.
Use the CLI command utils dbreplication repair to repair the replication issue. This command tries to repair replication by rebuilding the corrupted .rhosts file.