Cisco Emergency Responder Administration Guide 1.2(3)
Troubleshooting Cisco Emergency Responder
Downloads: This chapterpdf (PDF - 510.0KB) The complete bookPDF (PDF - 2.59MB) | Feedback

Troubleshooting Cisco Emergency Responder

Table Of Contents

Troubleshooting Cisco Emergency Responder

Troubleshooting Phone-Related Problems

Undiscovered Phones

Too Many Unlocated Phones

Cisco IP SoftPhone Movements Not Tracked

Phone Sometimes Disappears in Cisco Emergency Responder

Wrong ERL is Used for a Shared Line

802.11b Endpoints Using Wrong ERL

Troubleshooting Emergency Call Problems

Emergency Calls are Not Being Intercepted by Cisco Emergency Responder

ELIN not Transmitted to the PSAP

ELIN For Default ERL Used For Calls From Other ERLs

Emergency Calls Not Routed to the Correct PSAP

Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

PSAP Call Back Errors

Onsite Alert Personnel Are Not Getting Telephone Alerts

Prompts for Phone Alerts Not Getting Played

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

Incorrect Location Information Sent To Onsite Alert Personnel

Emergency Call History Problems

Troubleshooting Email Alerts

Emergency Call Alert

Transition Alert

Tracking Failure

Failed To Get Provider

Failed to Establish Communication with CER Phone Tracking Engine

Lost Communication with CER Phone Tracking Engine

Failed to Send Unlocated Phone Details to Remote Cisco ER Server Group

Emergency Call Could Not be Routed

Calling Party Modification Failed

Troubleshooting Web Alerts

Troubleshooting Cisco Emergency Responder System and Administration Problems

Cannot Validate Publisher

Troubleshooting Login Problems

Using CiscoWorks IP Telephony Environment Monitor

Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems

Using The ERL Debug Tool to Verify Cisco ER Configuration

Replacing the Publisher Server and Subscriber Servers

Replacing a Faulty Subscriber

Replacing a Faulty Publisher

Using the Cisco ER Admin Utility

How to Use the Cisco ER Admin Utility Tool

Troubleshooting the Subscriber Database Setup

Troubleshooting Subscriber Database Replication

Troubleshooting the MSDE Database

Troubleshooting Cisco Emergency Responder System Problems

Troubleshooting Cisco CallManager Configuration Problems

Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster

Phones Moving Between Clusters

Starting and Stopping a Cisco Emergency Responder Server

Troubleshooting ALI Data Uploads

Fixing ALI Data Records

Editing NENA 2.0 and 2.1 File Formats

Editing NENA 3.0 File Formats

Collecting Call History Logs

Collecting Trace and Debug Information

Installing Syslog

Enabling Syslog

Viewing Windows Event Messages

Managing Performance

Integrating with Network Management Systems

Understanding CDP Support

Monitoring Cisco Emergency Responder Subsystem Status

Collecting System Logs with Syslog

Backing Up and Recovering Data


Troubleshooting Cisco Emergency Responder


These topics address problems you might encounter with Cisco Emergency Responder (Cisco ER), and provide ways to resolve them; also included are other tasks associated with problem identification and resolution.

Troubleshooting Phone-Related Problems

Troubleshooting Emergency Call Problems

Troubleshooting Cisco Emergency Responder System and Administration Problems

Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster

Starting and Stopping a Cisco Emergency Responder Server

Troubleshooting ALI Data Uploads

Collecting Call History Logs

Collecting Trace and Debug Information

Viewing Windows Event Messages

Managing Performance

Integrating with Network Management Systems

Backing Up and Recovering Data

Troubleshooting Phone-Related Problems

These topics help you troubleshoot problems related to assigning phones to ERLs and managing the phones:

Undiscovered Phones

Too Many Unlocated Phones

Cisco IP SoftPhone Movements Not Tracked

Phone Sometimes Disappears in Cisco Emergency Responder

Wrong ERL is Used for a Shared Line

802.11b Endpoints Using Wrong ERL

Undiscovered Phones

If Cisco Emergency Responder (Cisco ER) is not discovering the phones homing to Cisco CallManager, check that all Cisco CallManagers are SNMP-reachable and that the SNMP settings are correct. Cisco ER will log an event if Cisco CallManager is SNMP-unreachable.

To verify the Cisco CallManager SNMP settings, perform the following steps:

Procedure


Step 1 Try to ping the Cisco CallManager server from Cisco ER.

Step 2 If you can ping the Cisco CallManager, make sure the SNMP settings are correct on Cisco CallManager:

a. Open the services on Cisco CallManager. Go to:

Start > Settings > Control Panel > Administrative Tools >
Services Properties > SNMP > Properties > Security Tab

b. Make sure the community string is set to read_only access.


Tip Verify that the community string setting is read_only on all the Cisco CallManagers in the cluster.


c. Check that the radio button for Cisco ER host readability is clicked.

d. If the radio button "Accept SNMP Packets from these hosts" has been selected, make sure the IP addresses of Master and Backup Cisco ER servers are visible in the bottom window.


Too Many Unlocated Phones

Cisco Emergency Responder (Cisco ER) obtains a list of registered phones from Cisco CallManager and tries to locate all phones. If Cisco ER cannot locate a phone behind a switch port or in any configured IP subnets, and the phone is not a configured synthetic phone, the phone will be placed in the list of unlocated phones.

If there are a lot of unlocated phones, first try running the switch port and phone update process to see if Cisco ER can resolve some of the problems automatically. See the "Manually Running the Switch-Port and Phone Update Process" section for more information.

These are some things that can prevent Cisco ER from locating a phone:

If more than one switch port reports the phone as a CDP (Cisco Discovery Protocol) neighbor, then the phone will be placed in unlocated phones. This condition will be corrected in the next phone tracking when only one switch port reports this phone as its CDP neighbor.

The phone is attached to a switch that is not defined in Cisco ER. See the "Identifying the LAN Switches" section for information on defining switches.

The phone is connected to an unsupported device, such as a router port, a hub connected to a router, or an unsupported switch. See the "Network Hardware and Software Requirements" section for a list of supported switches. See the "Manually Defining a Phone" section for information on configuring these types of phones if you cannot connect them to a supported device.

The phone is connected to a hub, which is connected to a supported switch port, but it does not support CDP. Cisco ER can consistently discover CDP-enabled phones attached to hubs (which are attached to supported switch ports), but cannot always track non-CDP phones attached in this manner. For non-CDP phones, ensure the phones are attached directly to supported switch ports.

The switch to which the phone is connected is currently unreachable, for example, it does not respond to SNMP queries. This could be for several reasons:

The SNMP read community string on the switch does not match the string configured in Cisco ER. Correct the Cisco ER configuration. See the "Configuring the SNMP Connection" section.

The phone requires CAM table access, but CAM tracking is not enabled for the switch in Cisco ER. See the "Identifying the LAN Switches" section.

There is a network outage preventing communication between the Cisco ER server and the switch. Locate and resolve the network outage problem.

Unreachable switches are not retried until Cisco ER runs the next full switch-port and phone update process, unless you run it against the individual switch (see below).

The phone has moved to a switch served by a different Cisco ER group. If this is the case, the Cisco ER group name is shown for the phone in the unlocated phones list. If the phone is not locatable in the next incremental phone tracking process after it is moved, the phone remains unlocated in any Cisco ER group until a full switch-port and phone update process is run.

The phone requires CAM-based tracking, but CAM-based tracking is not enabled on the switch to which the phone is connected. Cisco IP SoftPhone and some other phone models require CAM-based tracking. See the "Identifying the LAN Switches" section for information on enabling CAM-based tracking, and "Network Hardware and Software Requirements" section for a list of phones that require CAM-based tracking.

After fixing the problems that are preventing Cisco ER from locating phones, run the switch-port and phone update process on the affected switches, or on all switches:

To run the process on a specific switch—Select Phone Tracking > LAN Switch Details and select the switch in the left-hand column; then click Locate Switch Ports.

To run the process on all switches—Select Phone Tracking > Run Switch-Port & Phone Update.

Related Topics

Identifying Unlocated Phones

IP Subnet Phones

Cisco IP SoftPhone Movements Not Tracked


Note This section applies to Cisco Emergency Responder running Cisco IP SoftPhone with Cisco CallManager 3.2 or earlier. Cisco Emergency Responder running Cisco CallManager 3.3 is able to track Cisco IP SoftPhones using the incremental phone tracking process.


If the computer that hosts a Cisco IP SoftPhone moves, Cisco Emergency Responder (Cisco ER) tracks the movement only during the full switch-port and phone update process. The movements are not tracked by the incremental phone tracking process. Any emergency calls from the Cisco IP SoftPhone are routed based on the last ERL assigned during the switch-port and phone update process.

If your organization uses Cisco IP SoftPhones extensively, and users move them frequently, you might consider running the full switch-port and phone update process more than once per day.


Tip Check your Cisco IP SoftPhone version; Cisco ER supports Cisco IP SoftPhone 1.2 or later.


These are some other points to keep in mind when supporting Cisco IP SoftPhone:

You must configure the Cisco IP  SoftPhone software to work with Cisco ER. See the "Setting up Cisco IP SoftPhone for Cisco Emergency Responder" section.

You must enable CAM tracking on the switches to which Cisco IP SoftPhones are attached. See the "Identifying the LAN Switches" section for more information.

Cisco IP SoftPhones are not tracked if they are attached through a hub, even if the hub is attached to a supported switch port.

Cisco IP SoftPhones are only tracked by the incremental phone update process if the Cisco IP SoftPhone is detached from the network for more than 10 minutes. That is, at least 10 minutes must pass between unplugging the computer from the network and plugging it into a different port. If the move takes less than 10 minutes, the move is only discovered during the next full switch-port and phone update process.

Cisco IP SoftPhones, like other phones, must be attached to the network using Ethernet. Cisco ER does not support Cisco IP SoftPhones attached to other types of network such as Token-Ring or ATM.

A single Cisco ER group can support up to 500 Cisco IP SoftPhones.

With Cisco Emergency Responder 1.1(3) and later, if you change the IP address on a computer running Cisco IP SoftPhone, the IP address change will be reflected in the Cisco ER web interface.

The IP address change is reflected in the Cisco ER web interface only if you used Cisco IP SoftPhone's Automatic Selection and you used the correct URL in Network Audio Settings.

A changed IP address on a computer running Cisco IP SoftPhone will not be reflected in the Cisco ER web interface. To fix this problem, restart Cisco IP SoftPhone.

Related Topics

Defining the Phone Tracking and Switch Update Schedules

Phone Sometimes Disappears in Cisco Emergency Responder

If Cisco Emergency Responder (Cisco ER) is in the middle of a phone tracking process, and a phone is in the middle of homing to a different Cisco CallManager cluster, no Cisco CallManager cluster has a record of the phone. Thus, Cisco ER does not know the phone exists, and you will not be able to look up the phone in the Cisco ER interface. However, assuming the phone successfully connects to a Cisco CallManager cluster, Cisco ER tracks the phone during the next incremental phone tracking process, and the phone should then appear in the Cisco ER interface.

This problem can also occur if phones are reconnecting to a primary Cisco CallManager server from a backup server during the Cisco ER phone tracking process.

Wrong ERL is Used for a Shared Line

When two or more phones with a shared line appearance move from switches that are monitored by one Cisco Emergency Responder (Cisco ER) group to switches that are monitored by a different Cisco ER group, then Cisco ER may assign an incorrect ERL to these phones during an emergency call. This can occur when the phones move to a different campus that has a different Cisco CallManager cluster (although the moved phones are still registered with the original Cisco CallManager cluster), and it can also occur when the phones move within a single large campus that is served by multiple Cisco CallManager clusters.

Because the moved phones are still registered to their original Cisco CallManager cluster, emergency calls from these phones are routed to the original Cisco ER group. In this case, the Cisco ER group detects that the calling phone is connected to a switch that is monitored by a different Cisco ER group, and the call is forwarded to the appropriate Cisco ER group through an H.323 inter-cluster trunk. Because the inter-cluster trunk does not pass the MAC address of the calling phone, the receiving Cisco ER group does not know the MAC address of the calling phone and must associate the phone to an ERL based on the calling party number.

In cases with a single phone connected to the switches monitored by the receiving Cisco ER group, this is not a problem. However, when multiple phones with a shared line appearance connect to switches monitored by the receiving Cisco ER group, then Cisco ER must guess which phone has placed the emergency call. If all of the phones with a shared line appearance are in the same ERL, the guess is correct. If the phones span multiple ERLs, then the guess might be incorrect.

Related Topics

Deploying Cisco Emergency Responder In Two Main Sites

Creating Route Patterns for Inter-Cisco Emergency Responder-Group Communications

802.11b Endpoints Using Wrong ERL

802.11b endpoints (such as Cisco Wireless IP 7920 Phones and Cisco IP SoftPhones running on 802.11b) are using switch port-based ERL instead of the configured subnet-based ERL.

Cisco Emergency Responder (Cisco ER) give a higher priority to switch port association for call routing. If Cisco ER finds a switch port mapping for any endpoint (including 802.11b endpoints), it uses the switch port mapping to route emergency calls. If the switch port mapping is not found or if the ERL is not configured for the corresponding switch port, Cisco ER 1.2 routes emergency calls using subnet-ERL configuration.

Be aware that Cisco ER 1.2 will locate 802.11b endpoints behind a switch port under the following conditions:

CDP (Cisco Discovery Protocol) is disabled on the access point or the switch port on which it is connected; and

CAM tracking is enabled in Cisco ER for that particular switch.

See the switch port screen or the ERL debug tool (see Using The ERL Debug Tool to Verify Cisco ER Configuration) to check if the 802.11b endpoint is associated with a switch port.

It is recommended that you track 802.11b endpoints using subnet-based ERLs. Therefore, enable CDP on the switch port and the access points to route emergency calls from 802.11b endpoints using subnet-based ERLS.

Related Topics

Configuring IP Subnet-based ERLs

Troubleshooting Emergency Call Problems

These topics help you troubleshoot problems related to the routing of emergency calls and the information supplied with the calls:

Emergency Calls are Not Being Intercepted by Cisco Emergency Responder

ELIN not Transmitted to the PSAP

ELIN For Default ERL Used For Calls From Other ERLs

Emergency Calls Not Routed to the Correct PSAP

Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

PSAP Call Back Errors

Onsite Alert Personnel Are Not Getting Telephone Alerts

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

Incorrect Location Information Sent To Onsite Alert Personnel

Emergency Call History Problems

Emergency Calls are Not Being Intercepted by Cisco Emergency Responder

If Cisco Emergency Responder (Cisco ER) is not intercepting emergency calls, there is probably a mistake in your Cisco CallManager configuration or its representation in the Cisco ER configuration. Check these items (based on the names used in the examples in Chapter 3 "Configuring Cisco CallManager for Cisco Emergency Responder.")

The emergency call number (911) is in the Phones partition and uses the E911CSS calling search space. Ensure this number was identified during Cisco ER installation (see the "Installing Cisco Emergency Responder 1.2(1) on a New System" section.) This ensures that users can dial the emergency number. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number.

The standby Cisco ER server's route point (912) is in the E911 partition and uses the E911CSS calling search space. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number. Ensure this number is defined as the standby server's route point in the Cisco ER configuration (see the "Configuring Group Telephony Settings For the Cisco Emergency Responder Server" section.)

The PSAP callback route point pattern (913XXXXXXXXXX) is in the E911 partition and uses the E911CSS calling search space. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number. Ensure this number is defined as the PSAP callback route point pattern in the Cisco ER configuration, and that the strip prefix (913) is also identified (see the "Configuring Group Telephony Settings For the Cisco Emergency Responder Server" section.)

All ELIN route patterns are in the E911 partition. See the "Creating the Route Patterns for ELINs" section for information on setting up the Cisco CallManager configuration for these numbers.

All phones and CTI ports (both device and line) are in the Phones partition and use the PhoneCSS calling search space. You can use additional partitions, but they must be set up with relationship to the Cisco ER partitions and calling search spaces in the same manner as these partitions in the examples described in the "Setting Up Cisco Emergency Responder to Handle Emergency Calls" section.

All gateways to the service provider's network use the E911CSS calling search space. See the "Configuring the Calling Search Space for the Gateways Used to Connect to the PSAP" section for more information.

ELIN not Transmitted to the PSAP

If the ELIN is not transmitted to the PSAP, and you are using a PRI connection to route emergency calls to the PSAP, check the configuration of the gateway. The PRI must be configured to send the real calling party number (which will be the ELIN) rather than a static number, such as the main site number. See the "Obtain CAMA or PRI Trunks to the PSTN" section.

ELIN For Default ERL Used For Calls From Other ERLs

If an emergency call is assigned an ELIN defined for the Default ERL rather than an ELIN assigned to the ERL whence the call was made:

Check the Cisco CallManager configuration for the route pattern for the ELIN you expected to be used. See the Creating the Route Patterns for ELINs.

Check the ERL definition in Cisco ER to ensure that the ELIN is correctly configured for the ERL. See the "Setting Up an Individual ERL and Its Automatic Location Information (ALI)" section.

If the route pattern for an ERL fails, Cisco ER uses the route pattern defined for the Default ERL.

Emergency Calls Not Routed to the Correct PSAP

If an emergency call is not routed to any PSAP, check whether the route patterns used for the ERL from which the call was made and for the default ERL are configured and use the correct partitions and calling search spaces (see the "Creating the Route Patterns for ELINs" section). Ensure that the partitions and calling search spaces for the gateways are correct (see the "Configuring the Calling Search Space for the Gateways Used to Connect to the PSAP" section).

If an emergency call successfully leaves your network but does not get routed to the correct PSAP, look at these possible points of failure:

Is Cisco ER configured to assign the correct ELIN to the ERL assigned to the phone? Emergency calls are routed based on the ELIN, so if you assign the wrong ELIN, the call will not be routed correctly. See the "Creating ERLs" section.

If the ELIN is correct, is the ELIN's route pattern configured to use the correct gateway? If you select the wrong gateway, the call might be routed to a part of the service provider's network that cannot connect to the desired PSAP. Consult with your service provider to determine gateway requirements.

See these topics:

Setting Up the ELIN Numbers to Route Emergency Calls and Enable PSAP Callbacks

Deploying Cisco Emergency Responder in One Main Site with Two or More PSAPs

Does the service provider's ALI database contain the correct information for the ELIN? Emergency call routing outside your network is based on the information in the service provider's database, not on the information in your local network. See the "Exporting ERL Information" section.

Does the emergency caller's phone register with a Cisco CallManager cluster supported by a different Cisco ER group than the Cisco ER group that supports the originating switch port? Then you might have a miss-configured Cisco ER cluster. See these topics:

Installing Cisco Emergency Responder 1.2(1) on a New System

Creating Route Patterns for Inter-Cisco Emergency Responder-Group Communications

Configuring Group Telephony Settings For the Cisco Emergency Responder Server


Note If the call reaches the PSAP, but the PSAP cannot talk to the caller, ensure that the Cisco CallManager for the remote Cisco ER group has the Cisco CallManager for the local Cisco ER group defined as a gateway.


Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

If callers hear a busy signal when calling the emergency call number, or if emergency calls sometimes do not get routed, there is probably a problem with the configuration of your standby Cisco Emergency Responder (Cisco ER) server:

If you have only configured a primary Cisco ER server, install and configure a standby Cisco ER server. If CPU utilization on the primary server reaches 100%, Cisco ER cannot handle emergency calls. In this case, the standby server handles the calls.

Check the route point configuration for the standby server. Ensure the emergency call route point's call forward settings are configured to forward calls to this number. See the "Creating the Emergency Call Route Points" section for information on the Cisco CallManager configuration, and the "Configuring Group Telephony Settings For the Cisco Emergency Responder Server" section for the Cisco ER configuration.

PSAP Call Back Errors

You might encounter these problems if a PSAP operator tries to call back an emergency caller using the ELIN provided by caller ID:

Symptom    PSAP could not reach the original emergency call extension.

Recommended Action    Cisco ER caches a mapping between the caller's true extension and the ELIN you define for an ERL. If more calls get made than the number of ELINs you define for an ERL, Cisco ER must reuse these numbers and thus overwrites the original caller's extension. You can view the call history to determine the extension of the original caller. See the "What Happens When an Emergency Call Is Made" section.

If this is not the problem, check the configuration of the PSAP callback route point in Cisco CallManager and Cisco ER (see the "Creating the Emergency Call Route Points" section and the "Configuring Group Telephony Settings For the Cisco Emergency Responder Server" section), and the ELIN translation patterns in Cisco CallManager (see the "Creating the Translation Patterns for ELINs" section).

Symptom    Onsite alert (security) personnel get callbacks from the PSAP.

Recommended Action    Cisco ER routes PSAP callbacks to the onsite alert personnel for the default ERL if ELIN-to-extension mapping for the emergency call has expired from the cache. By default, this is three hours, although you can configure expiration to be a longer or shorter time. See the "Cisco ER Group Settings" section.

Onsite Alert Personnel Are Not Getting Telephone Alerts

If the onsite alert personnel are not getting telephone alerts when an emergency call is made in an ERL they are covering, ensure that all phones and CTI ports (both device and line) are in the Phones partition and use the PhoneCSS calling search space. You can use additional partitions, but they must be set up with relationship to the Cisco ER partitions and calling search spaces in the same manner as these partitions in the examples described in the "Setting Up Cisco Emergency Responder to Handle Emergency Calls" section.

Also, ensure that the Cisco ER configuration for the Cisco CallManager clusters is correct. The Cisco ER configuration should show the correct begin address for the telephony ports you defined as CTI ports in Cisco CallManager, and the number of telephony ports should be the correct number and it must be greater than 0 for any calls to occur. Cisco ER uses this CTI ports to place the telephone calls to onsite alert personnel.

If the Windows Event Viewer shows the error message "No port to place call," this means that there were not enough CTI ports defined to initiate all the calls to onsite alert personnel. Define more ports.

Prompts for Phone Alerts Not Getting Played

Symptom    Prompts do not get played at the onsite alert phone when the call is initiated from the CTI ports.

Explanation    This problem can occur when a single CTI port is configured with multiple lines. Prompts may not get played from one or more of these lines when the onsite alert notifications call is initiated through them.

Recommended Action    To avoid this problem, configure only one line per CTI port in the Cisco CallManager that is configured for Cisco Emergency Responder.

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

If the onsite alert personnel are not getting email, or email-based pages, even though you configure email addresses for them (see the "Onsite Alert Settings" section), check the Cisco ER configurations SMTP settings. Ensure that the SMTP server address and source mail ID are correct (see the "Cisco ER Group Settings" section), and that there is an account for the mail ID in the SMTP server.

Incorrect Location Information Sent To Onsite Alert Personnel

If your onsite alert (security) personnel are receiving incorrect location information for an emergency call, consider these potential problems:

Is the ALI data for the ERL correct? See the "Creating ERLs" section.

Is the phone location data for the switch port correct? See the "Configuring Switch Ports" section.

Is the correct ERL assigned to the switch port to which the phone is connected? If not, there could be two problems:

Someone switched wires on the switch, so your formerly correct configuration is no longer correct. Wires cannot be moved from port to port without potentially invalidating the ERL assignment. See the "Data Integrity and Reliability Considerations" section.

The wiring closet is secure, the ERL assignment is simply incorrect. See the "Configuring Switch Ports" section.

Did the call come from the Default ERL (assuming you do not use the Default ERL for any permanent ERL)? This could indicate these problems:

The phone is connected to an unsupported port and is not defined as a manual phone. See the "Manually Defining a Phone" section.

The phone is not supported and it is not defined as a manual phone. See the "Manually Defining a Phone" section.

The phone is supported but Cisco ER could not locate it. You might have to manually assign the phone to an ERL if you cannot resolve the problem. See the "Too Many Unlocated Phones" section.

Did the call come from a manually-defined phone extension? If so, it is likely the incorrect ERL is assigned, perhaps because the phone moved. See the "Manually Defining a Phone" section.

Emergency Call History Problems

These are some issues you might encounter when viewing the emergency call history information (see the "Viewing the Emergency Call History" section):

Symptom    Emergency call information does not show up in call history right away.

Recommended Action    Cisco ER writes call history information to the MSDE database every 15 seconds. You should be able to view history information after 15 seconds.

Symptom    The call history does not show the ELIN and route pattern used for a call.

Recommended Action    If the call could not be routed to the PSAP, you will not see an ELIN or route pattern. Check to determine why the call could not be routed. See the "Emergency Calls Not Routed to the Correct PSAP" section.

Troubleshooting Email Alerts

These topics help you troubleshoot problems related to the email alerts that Cisco Emergency Responder (Cisco ER) generates:

Emergency Call Alert

Transition Alert

Tracking Failure

Failed To Get Provider

Failed to Establish Communication with CER Phone Tracking Engine

Lost Communication with CER Phone Tracking Engine

Failed to Send Unlocated Phone Details to Remote Cisco ER Server Group

Emergency Call Could Not be Routed

Calling Party Modification Failed

Emergency Call Alert

Whenever a user makes a 911(Emergency) call, Cisco Emergency Responder (Cisco ER) generates an email alert. Cisco ER sends the email alert to all of the onsite alert (security) personnel whose email ids are configured for the ERL from which the call was made. (See the "Configuring a Cisco Emergency Responder Server Group" section.)

Security personnel are expected to respond to that user. For detailed call information, refer to the following URL:

http://<<CERServer HostName>>/ceruserreports

When a 911 call is made and the backup Cisco ER server handles the call, Cisco ER sends the following alert:

Subject: Emergency Call Alert -- Extn # 332101 (Generated by Backup 
Cisco ER)
Message: EMERGENCY CALL DETAILS (Generated by Cisco ER)
Caller Extension:332101
Zone/ERL         :Z1
Location         :ddd    
Call Time        :June 2, 2003 3:47:30 PM IST

Transition Alert

Cisco Emergency Responder (Cisco ER) sends a Transition Alert to the Cisco ER administrator when the standby Cisco ER server takes control and becomes the active server. This occurs under any of the following circumstances:

If the primary Cisco ER server is stopped.

If the Cisco ER service is stopped on that server.

If the connectivity between primary and standby Cisco ER servers is broken.

The administrator should diagnose the cause and fix the problem as soon as possible.

When the Cisco ER backup server takes control, Cisco ER sends the following alert:

Subject: Transition Alert: Cisco ER Backup is active
Message:
Backup Cisco ER <<CERServer HostName>> has taken control as Active 
Cisco ER.
Transition Time  :June 2, 2003 3:57:12 PM IST

When the master Cisco ER server takes control., Cisco ER sends the following alert:

Subject: Transition Alert: Cisco ER Master is active
Message:
Master Cisco ER <<CERServer HostName>> has taken control as Active 
Cisco ER.
Transition Time  :June 2, 2003 3:57:12 PM IST
 
   

Tracking Failure

At the end of a switch-port and phone tracking process, if there are any devices that could not be tracked, Cisco ER sends a Tracking Failure email to the Cisco ER administrator.

The administrator should look at the event log on the Cisco ER server to find the list of devices that were not tracked. Then the administrator should check the following and make any required corrections:

1. Make sure that the correct SNMP Community String is configured in Cisco ER.

2. Check that the device is connected.

3. Check that the host name for the Cisco ER server is resolvable, that is, it can be found.

4. Check that the SNMP service is enabled on that particular device (Switch / Cisco CallManager).

Here is an example of a tracking failure alert.

Subject: CER Phone Tracking failed to track some devices
Message:
CER Phone Tracking could not get information [using SNMP] from 2 
Cisco CallManager(s) and 1 Switch(es)
Check Event Viewer on CER Server for details.

Failed To Get Provider

Cisco Emergency Responder (Cisco ER) sends a Failed to Get Provider Alert to the Cisco ER administrator if Cisco ER is not able register to one of the configured Cisco CallManager clusters. Cisco ER will continue trying registration until it succeeds. Cisco ER sends the Failed to Get Provider email after a few retries.

The message provides information on how to clear the problem, as shown in the following example.

Subject: Failed to get JTAPI Provider for Cisco CallManager <<CCM 
IP/Host Name>> (Generated by Backup CiscoER)
Message:
Please check the following:
1) Check if the Cisco CallManager is connected to the CER server.
2) Check if the configured Call Manager is running a version supported 
by the CER server.
3) Check if the given login credentials are correct:
	CTI Manager Host Name:<<CCM IP/HostName>>

Failed to Establish Communication with CER Phone Tracking Engine

Cisco Emergency Responder (Cisco ER) sends this email alert to the Cisco ER administrator if the Cisco ER server fails to establish communication with the Phone Tracking Engine for some time. This can occur if the CER Phone Tracking Engine service is down. The administrator should perform the following steps:

1. If the CER Phone Tracking Engine service is down, start the service.

2. Make sure that the Host Name of the Cisco ER server does not contain any underscore(_) characters.

Here is an example of a tracking failure alert.

Subject: CER Server failed to establish communication with CER Phone 
Tracking Engine.
Message:
CER Server could not communicate with CER Phone Tracking Engine.

Lost Communication with CER Phone Tracking Engine

Cisco Emergency Responder (Cisco ER) sends this email alert to the Cisco ER administrator if the Cisco ER server loses communication with the CER Phone Tracking Engine. This is most liked to occur if the Cisco ER Phone Tracking Engine service goes down when the Cisco ER server is running.

The administrator should restart the CER Phone Tracking Engine service.

The following shows an example of a tracking failure alert.

Subject: CER Server lost communication with CER Phone Tracking Engine
Message:
CER Server could not communicate with CER Phone Tracking Engine.

Failed to Send Unlocated Phone Details to Remote Cisco ER Server Group

If Cisco Emergency Responder (Cisco ER) fails to send unlocated entries to a server group because it is already in the process of sending entries to that server group, this alert is sent.

This alert will occur very rarely. It can occur when a Cisco ER server is found in more than one Cisco ER server group. To resolve this problem, check to see which server group is an old configuration and remove that server group.

Subject: CER Server failed to send Unlocated Phones details to Remote 
CER Server Group.
Message:
CERServer failed to send Unlocated Phones to Remote CER Server Group. 
Please ensure that the CER servers are not found under more than one 
CER Server Group.
CER Servers in Remote Server Group:<< CERServer HostNames >>
 
   

Emergency Call Could Not be Routed

If the emergency call routing to some route patterns configured in the ERL fails, Cisco ER sends an email to the system administrator.

Subject: Emergency call could not be routed using some route patterns (CERServer:<server hostname>)

Message Body: Emergency call from :<Caller Extn> could not be routed using some Route Patterns. Check Event Log.

The Event Log displays the following message:

Emergency call from <extn> could not be routed using the following 
route patterns
 
   
<RoutePattern1>
<RoutePattern2>
*****************
Call Routed to <RoutePattern-X>
 
   
Please check the availability of the above routes. Also, check for the 
following error conditions:
 
   
1. If FAC and/or CMC are configured on the route patterns used for 
Cisco ER, please disable them.
2. If the "Calling Party Number Modification" flag on the CER user 
page in the CallManager is not enabled, please enable it.

Solution   

1. If you are running Cisco CallManager 4.1(2), check to make sure that Forced Authorization Code (FMC) and/or Client Matter Code (CMC) is not configured on the route patterns. Cisco ER does not support FAC or CMC.

2. If you are running Cisco CallManager 4.0 or later, check to make sure that the Calling Party Number flag on the Cisco ER User page is enabled.

3. For all releases of Cisco CallManager, check to make sure that the routes are available.

Calling Party Modification Failed

If the calling party modification was not successful, Cisco ER sends the following email to the system administrator:

Subject: Emergency Calling Party Modification Failed (CERServer: <server>)

Message Body: Emergency call from :<Caller Extn> cannot be routed with calling party modification. Check Event Log.

The Event Log displays the following message:

Emergency Call from <Caller Extn> has been routed to default ERL 
because the calling party modification failed.
Please make sure that the checkbox "Enable Calling Party Number 
Modification: is checked on the Cisco CallManager user page for the 
CER user. PSAPO callbacks MAY NOT work correctly. The CER service will 
need to be restarted once the flag is checked on the CallManager User 
page.

Solution   Check the box for the "Enable Calling Party Number Modification" in the Cisco ER user page in Cisco CallManager 4.0(1) Administration. After you enable this flag, restart the Cisco ER service for the changes to take effect.

Troubleshooting Web Alerts

Symptom    Web alert continues to refresh every 30 seconds. You can see this problem by checking the status in the browser. The status will display the seconds remaining before refresh if it is in this mode.

Recommended Action    Check if there are other web alert screens open on the same client machine. Only one browser from a client machine can operate in the real-time mode. Remove any extra browsers.

Troubleshooting Cisco Emergency Responder System and Administration Problems

These topics help you troubleshoot problems related to the Cisco Emergency Responder system and its administration, such as server and web server problems:

Cannot Validate Publisher

Troubleshooting Login Problems

Using CiscoWorks IP Telephony Environment Monitor

Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems

Using The ERL Debug Tool to Verify Cisco ER Configuration

Replacing the Publisher Server and Subscriber Servers

Using the Cisco ER Admin Utility

Troubleshooting the MSDE Database

Troubleshooting Cisco Emergency Responder System Problems

Troubleshooting Cisco CallManager Configuration Problems

Cannot Validate Publisher

If the installation cannot validate the publisher (Step 5 of the "Installing the Cisco ER Subscriber" section), check the following:

1. Verify that the publisher hostname is correct and that the publisher is reachable by hostname. If not, add the publisher's hostname and password to the hosts file on the subscriber.

2. Verify that the publisher and subscriber servers are running the same version of Cisco Emergency Responder.

3. Verify that the CERSQLAdmin password that you entered is correct.

4. Make sure that none of the publishers shares (particularly, the C$ share) are mounted on the subscriber:

Access the Tools > Disconnect any mounted share (of the publisher).

Then, proceed with the installation.

Troubleshooting Login Problems

These are some issues you might encounter while logging into Cisco Emergency Responder (Cisco ER):

Symptom    You occasionally cannot log in as LAN switch administrator or ERL administrator.

Recommended Action    Check the error message for the reason for the failed login. If the system administrator is logged in, you cannot log in as a LAN switch or ERL administrator. This is because the system administrator can configure the same data as the LAN switch and ERL administrators; you are prevented from logging in simultaneously to ensure data integrity. If the system administrator is not logged in, separate LAN switch and ERL administrators can log in simultaneously.

Symptom    You cannot open multiple Cisco ER sessions using Netscape Navigator.

Recommended Action    Netscape Navigator uses the same session ID across multiple windows. This creates problems if you try to log into Cisco ER using different IDs. Normally, you can open multiple windows when logged in as system administrator. With Internet Explorer, if you open separate IE session by starting a new IE instance (rather than by opening a new window from an existing session), IE uses different session IDs, and you should be able to log in using separate IDs (for example, as a user and an administrator, or as LAN switch and ERL administrators).

Symptom    Cisco ER web interface says it cannot connect to LDAP.

Recommended Action    Cisco ER maintains its cluster information in the Cisco CallManager LDAP database. Check to ensure there is connectivity to the computer running the LDAP directory. If there are no connectivity problems, check the computer running the directory to ensure that the directory service has been started and is running properly.

Related Topics

Using The ERL Debug Tool to Verify Cisco ER Configuration

Using CiscoWorks IP Telephony Environment Monitor

Use CiscoWorks IP Telephony Environment Monitor (ITEM) 2.0 to continuously monitor the health of the Cisco Emergency Responder (Cisco ER) system.

For information on setting up Cisco ER to use CiscoWorks ITEM, see the "Configuring Test ERLs" section.

For information on installing and using ITEM, see the CiscoWorks ITEM documentation at:

http://www.cisco.com/en/US/products/sw/cscowork/index.html

Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems

Some issues that you might encounter while configuring switches or switch ports in Cisco Emergency Responder (Cisco ER) are shown below:

Symptom    Cisco ER is configured with Cisco CallManager information, but no phones get discovered.

Recommended Action    Ensure that the Cisco CallManager servers are reachable on the network. Then, ensure that the SNMP read community strings are configured correctly for the switches and Cisco CallManager servers (see the "Configuring the SNMP Connection" section.) Then, manually run the switch port and phone update process (see the "Manually Running the Switch-Port and Phone Update Process" section.)

Symptom    Cisco ER does not show the ports on a switch configured in Cisco ER.

Recommended Action    If you add a supported switch to Cisco ER and run phone tracking on the switch after adding it, you should be able to view the list of Ethernet ports on the switch. If Cisco ER does not list the ports, check the SNMP settings in Cisco ER for the switch (see the "Configuring the SNMP Connection" section.) Also, verify that the switch is reachable over the network. Retry the selective phone tracking process on the switch (click Locate Switch Ports when viewing the switch details; see the "LAN Switch Details" section.)

If the problem persists, ensure that the switch is supported (see the "Network Hardware and Software Requirements" section.) Also, check the Windows Event Viewer for error messages.

Symptom    Some phones do not appear in the switch port list.

Recommended Action    Check if the phone is found under configured IP subnets or in synthetic phones. If it is not found in either of those places, then they will be place un unlocated phones. See the "Too Many Unlocated Phones" section for a list of reasons that a phone could not be located.

Symptom    Cannot delete a switch from the Cisco ER configuration.

Recommended Action    You cannot delete a switch when a phone tracking process is in progress. Retry the deletion after the process has ended. If this is not the problem, the Cisco ER server might not be running. Check the control center and restart the server (see the "Starting and Stopping a Cisco Emergency Responder Server" section.)

Symptom    Import or export of the switch port details fails.

Recommended Action    If a switch port import or export attempt fails, it might be due to these reasons: the first switch-port and phone update process has not yet ended (wait for it to finish); the Cisco ER server is not running (use the control center to restart it, see the "Starting and Stopping a Cisco Emergency Responder Server" section); the Cisco ER server is not completely initialized (wait for it to initialize).

Symptom    The import of some switch port configurations fail.

Recommended Action    To import switch port configurations, Cisco ER must already be configured with the switch and Cisco ER must first discover the ports on the switch using the switch-port and phone update process. If you try to import a configuration for ports not yet discovered in Cisco ER, the importation of those settings fails. See the "Manually Running the Switch-Port and Phone Update Process" section for information on the process. Run it on the switches whose port configurations you could not import, then retry the import.

Symptom    Phones moved from other Cisco ER groups to this Cisco ER group, and then moved back, are still showing up in the switch port details for the Cisco ER group.

Recommended Action    This types of phones are not removed from the switch port details until the next full switch-port and phone update process is run. If this is an issue for you, you can run the process on the switch (or on all switches) manually. See the "Manually Running the Switch-Port and Phone Update Process" section.

Using The ERL Debug Tool to Verify Cisco ER Configuration

The ERL Debug Tool takes a phone extension as the search criteria and displays the ERL(s) currently being used for routing emergency calls for the phone(s).

Use this diagnostic tool to verify the Cisco ER configuration during the ERL creation and the ERL assignment phase, and to troubleshoot calls directed to incorrect ERLs.

For example, you configured the phone in ERL_1 as a manually configured phone, however a misconfigured IP subnet matches this phone's IP address, and associates it with ERL_2. Now that you have found the configuration problem using the Debug Tool, you can correct it.

To use the ERL Debug Tool, perform the following steps.

Procedure


Step 1 Select Tools >ERL Debug Tool.

Cisco ER displays the ERL Debug Tool page.

Step 2 At the Find Phones field, to list specific phones, select the search criteria and click Find.

Cisco ER displays the ERL currently being used for routing emergency calls for the phone.

Step 3 If the configurations are not correct, make the required changes.



Note Cisco ER displays a maximum of 1,000 records.


Replacing the Publisher Server and Subscriber Servers

If you need to replace a faulty publisher server or a faulty subscriber server, perform the appropriate procedure:

Replacing a Faulty Subscriber

Replacing a Faulty Publisher

Replacing a Faulty Subscriber

To replace a faulty subscriber, go to Cisco Emergency Responder (Cisco ER) administration and delete the faulty subscriber. Install a new Cisco ER subscriber for the publisher (see the "Installing Cisco Emergency Responder 1.2(1) on a New System" section).


Note If the same host name is not going to be used by the replacement subscriber server, you must delete the faulty subscriber using the Cisco ER administration screen on the publisher server.


Replacing a Faulty Publisher

You can restore the publisher only if you have backed up the publisher using the The Cisco IP Telephony Applications Backup Utility Version 3.5.44. See the "Backing Up and Recovering Data" section.

To replace a faulty publisher, perform the following steps:

Procedure


Step 1 Install the same version of the Cisco ER publisher on a server with the same host name as the one you used previously.

Step 2 Choose the same configuration options (such as the CERSqlAdmin password, Cisco CallManager version, and so on) during the installation.

Step 3 Restore the old configuration data using the Cisco IP Telephony Applications Backup Utility.


Using the Cisco ER Admin Utility

You can use the Cisco ER Admin Utility tool to perform the following tasks:

To change the password for the CERSQLAdmin account.

To point a subscriber to a different publisher.

To restart Cisco ER services.

To restart database services.

To update Cisco ER cluster details on the Cisco CallManager LDAP directory.

This section describes the following topics:

How to Use the Cisco ER Admin Utility Tool

Troubleshooting the Subscriber Database Setup

Troubleshooting Subscriber Database Replication

How to Use the Cisco ER Admin Utility Tool

To use the Cisco ER Admin Utility tool, perform the following steps:

Procedure


Step 1 Log in locally to the Cisco ER publisher server.

Step 2 Open the Cisco ER Admin Utility on the Cisco ER server at:

C:\Program Files\Cisco Systems\CiscoER\bin\AdminUtils\
CiscoERAdminUtility.exe.

Step 3 At the Choose a Task drop-down list, select one of the following tasks and click Go:

a. To change the CERSQLAdmin server password on both the publisher server and the subscriber server, select To change the password for CERSQLAdmin account.

b. To change the publisher that the subscriber server points to, select To point a Subscriber to another Publisher.

c. To restart the Cisco ER services (CERServer, CERAdminServer, and CERPhoneTracking services), select To restart Cisco ER services.

d. To restart the MSDE services (MSSQLSERVER and SQSERVERAGENT), select To restart Database services.

e. To update the cluster LDAP settings on both the publisher and subscriber servers select To update Cisco ER Cluster LDAP details.


Note This updates the Cisco ER cluster details for this server group only. Other servers in this Cisco ER cluster will NOT be updated automatically.


Step 4 To save the changes that you have made, restart both the publisher and the subscriber servers.


Tip The Cisco ER Admin Utility tool generates log files in the C:\\CERInstall folder.



Troubleshooting the Subscriber Database Setup

The local Administrator password must be the same for the Cisco ER publisher server and subscriber server. If the passwords are different, the subscriber installation may report problems setting up the subscriber database. If you encounter this problem, Perform the following steps to re-initialize the subscriber database:

1. Change the subscriber local administrator password to the publisher's local administrator password.

2. Reboot the machine when prompted by the install.

3. Log in as local administrator.

4. Launch the Cisco ER Admin Utility and run the "To point a Subscriber to another Publisher" task.

5. Enter the name of the publisher.

6. Restart both the publisher and subscriber server for the changes to take effect.

Troubleshooting Subscriber Database Replication

You can use the "To point a subscriber to a different publisher" task if you failed to add the publisher hostname and the IP address to the host file on the subscriber before you began the Cisco ER installation or upgrade. After the subscriber installation completes, you will receive errors about a failure to set up replication for the subscriber database.

Perform the following steps to correct the problem:

1. Reboot the machine when prompted by the installation.

2. Add the publisher host information to the "hosts" file.

3. Select the To point a Subscriber to another Publisher task.

4. Enter the name of the publisher.

5. Restart both the publisher and the subscriber server for the changes to take effect.

Troubleshooting the MSDE Database

When you install Cisco Emergency Responder (Cisco ER) 1.2(x), the MSDE (Microsoft SQL Server Desktop Engine) database is set up on both the Cisco ER publisher server and the subscriber server. The MSDE database stores the Cisco ER configuration information. The publisher and subscriber databases are set up to be synchronized using Merge replication.

The Cisco ER MSDE database behaves as described here:

Both the publisher server and the subscriber server use the publisher database for all reads and writes whenever possible.

If the publisher is the active database, the user can configure all data.

If the publisher database is unavailable:

The Cisco ER servers use the subscriber database to read data.

All configuration options are disabled except for modifying the trace settings.

If neither database is available, Cisco ER displays a "Failed to Contact Database Error" message.

Some issues that you might encounter with general operation of the MSDE database are shown below.

Symptom    The publisher and subscriber databases can become out of synchronization with each other; when the publisher database is down and the Cisco ER administration screens are displaying old configuration details, then the databases are no longer synchronized.

Recommended Action    If the publisher and subscriber databases become out of synchronization, perform the following steps:

1. Make sure that there is connectivity between the publisher server and the subscriber server.

2. Make sure the SQL Server Agent is running on both servers.

3. Make sure that the CERSQLAdmin user is found and that it is a member of the Administrator's group with the same password in both the servers.

4. Run the Cisco ER Admin Utility. For information on the Cisco ER Admin Utility, see the "Using the Cisco ER Admin Utility" section.

Symptom    The database replication between the Cisco ER publisher server and subscriber server may not take place for some time if the subscriber system time is changed to an older value.

Recommended Action    The database replication between the publisher server and the subscriber server will not take place until the subscriber system time reaches the time that was originally set.

If the system time changes are so large that it is unacceptable to wait until the system time reaches the original value, you can run the "Point subscriber to another publisher" task from the Cisco ER Admin Utility to recreate the subscription. For information on the Cisco ER Admin Utility, see the "Using the Cisco ER Admin Utility" section.

Troubleshooting Cisco Emergency Responder System Problems

These are some issues you might encounter with general operation of the Cisco Emergency Responder (Cisco ER) system and the configuration screens that involve the Cisco ER server, group, and cluster:

Symptom    Cisco ER intra-cluster call routing fails or Cisco ER does not discover phones correctly.

Recommended Action    Ensure that all the Cisco ER servers in a Cisco ER cluster can be found by their host name, and ensure that all are reachable on the network by all the other Cisco ER servers.

Recommended Action    Ensure that all the Cisco ER servers can reach the Cisco ER cluster Directory.

Symptom    Cisco ER exits after starting.

Possible Cause    You have configured Cisco ER to use a TCP port that is already in use.

Recommended Action    Check the Windows Event Viewer for the message "Cisco ER could not open socket at port peer-tcp-port, Exiting." If you see this message, change the Cisco ER group configuration to use a different TCP port. See the "Configuring a Cisco Emergency Responder Server Group" section for instructions.

Symptom    The Cisco ER Groups in Cluster screen does not load, and exhibits the error "Cannot connect to primary LDAP."

Recommended Action    Check the connectivity to the computer running the primary LDAP directory for the cluster to which the Cisco ER group belongs. If there is connectivity, check the computer to ensure that the directory service is started and running correctly. If this does not resolve the problem, Cisco ER might not have been able to resolve the host name of the primary LDAP directory during Cisco ER installation. See the "Installing Cisco Emergency Responder 1.2(1) on a New System" section for information on Cisco CallManager LDAP database requirements.

Recommended Action    Cisco ER might be uninstalled or not running on the remote machine. If Cisco ER is uninstalled, delete the corresponding entry from the Cisco ER group using the Cisco ER groups in Cluster page (see the "Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster" section). If the Cisco ER server is not running, restart it using the control center (see the "Starting and Stopping a Cisco Emergency Responder Server" section.)

Related Topics

Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster

Starting and Stopping a Cisco Emergency Responder Server

Viewing Windows Event Messages

Managing Performance

Backing Up and Recovering Data

Troubleshooting Cisco CallManager Configuration Problems

These are some issues that you might encounter with Cisco Emergency Responder's (Cisco ER's) communications with Cisco CallManager. Additional problems with symptoms that involve emergency call failures are discussed in the "Troubleshooting Emergency Call Problems" section.

Symptom    Cisco ER does not register with the route points and CTI ports configured for its use.

Recommended Action    Ensure that the route points and CTI ports are associated with the Cisco CallManager Cisco ER user (see the "Creating a Cisco Emergency Responder Cisco CallManager User" section.) Ensure that the CTI Manager and DC Directory on the Cisco CallManager server are running properly.

Symptom    When trying to delete a Cisco CallManager from the Cisco ER configuration, Cisco ER prevents me and displays the message "Phone tracking in progress."

Recommended Action    You cannot delete a Cisco CallManager server from the Cisco ER configuration while a phone tracking process is in progress. Retry the deletion after the process has ended.

Updating Cisco Emergency Responder After You Add Devices

You must create a Cisco CallManager user for Cisco ER's use and CTI ports and route points that need to be assigned to the user before Cisco ER tries to create a provider with the Cisco ER cluster. Cisco ER only registers the CTI ports and route points that are associated with the user when the provider is created. Thus, any devices you add to the user after starting Cisco ER will not be registered by Cisco ER.

If you add devices to the Cisco ER user in Cisco CallManager, you can force Cisco ER to recreate the provider using any of these techniques:

Restart the Cisco ER server.

Delete the Cisco CallManager server from the Cisco ER configuration and re-enter it.

Change the backup CTI Manager setting for the Cisco CallManager server in the Cisco ER configuration and click Update. This forces Cisco ER to log off the provider and recreate it.

Change the name of the user in Cisco CallManager, or create a new user, and associate all devices with it. Then update the Cisco ER configuration to use the new user.

Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster

If you are connected to the administrator's interface on a Cisco Emergency Responder (Cisco ER) server, you can view the details of the server and the Cisco ER group's standby server by selecting Cisco ER Group > Server Settings.

You can also identify the Cisco ER groups and their Cisco ER servers that are in the same Cisco ER cluster. To view the other Cisco ER groups in the cluster, select Reports > Cisco ER Groups in Cluster. From the Cisco ER Groups in Cluster page, select the group you want to view from the left column, and Cisco ER displays the Cisco ER servers that are in the group. To view the details for these servers, you must connect to the Cisco ER administrator's interface running on one of the servers.

If you need to uninstall a Cisco ER group, first delete the group from the Cisco ER cluster using this page. You must log in as a system administrator to delete the group. Deleting the group from the cluster simply removes the entries for the group from the Cisco ER cluster's LDAP directory; it does not remove Cisco ER from the group's servers.

Related Topics

Cisco ER Server Groups in Cluster

Phones Moving Between Clusters

The following scenario illustrates how Cisco ER clusters work and how Cisco ER treats phones moving between clusters:

Server Group A (SGA) has a phone (Phone_1) that is moving out of SGA.

Cisco ER discovers Phone_1 in Server Group B (SGB).

The Unlocated Phones page in SGA will display the phone in SGB.

Now, if both the Cisco ER servers (master and standby) in SGB go down, SGA will still display Phone_1 in SGB.

Calls made from Phone_1 during this time will be redirected to SGB and Cisco ER will take the same steps to route this emergency call when Cisco ER servers are not there in SGB.

Phone_1 will also be treated like any other phone in SGB when both the SGB Cisco ER servers are down.

If Phone_1 moves to Server Group C (SGC):

Then it will be discovered after the next incremental phone tracking on SGA and then in SGC.

And, the Unlocated Phones page will change the association of Phone_1 to SGC.

If Phone_1 moves back to SGA, it will be discovered in the next incremental phone tracking and displayed under the corresponding switch port.

Starting and Stopping a Cisco Emergency Responder Server

When you install Cisco Emergency Responder (Cisco ER), the Cisco ER server is set up to automatically start whenever the computer is powered up or rebooted. However, you can stop and then restart a Cisco ER server through the Cisco ER administrator's interface without powering down or rebooting the computer. You might find this helpful if you are trying to debug a problem, or if you are trying to move a Cisco ER server to another computer.

To start or stop a Cisco ER server, perform the following steps:

Procedure


Step 1 Select Cisco ER Group > Control Center.

All the Cisco ER servers in the Cisco ER server group are displayed in a tree in the left pane.

Step 2 In the left pane, click a Cisco ER server.

The Control Center page displays the status of both the Cisco ER server and the CER Phone Tracking Engine.

Table 6-1 explains the meaning of the icons you might see on this page.

Step 3 From the control center, you can click Start to restart the Cisco ER server or the CER Phone Tracking Engine, or Stop to stop the server or service. The buttons only appear if the action is possible; for example, Start only appears if the server is stopped.

Step 4 To see the status of other servers and their services, go to the left pane and click that server.

Table 6-1 Cisco Emergency Responder Control Center Icons

Icon
Meaning

The Cisco ER server or the CER Phone Tracking Engine is started and functioning normally.

The Cisco ER server CER Phone Tracking Engine was stopped by the administrator.

The Cisco ER server is experiencing connectivity or permission errors.

Verify that the Cisco ER server is connected to the network and running properly.

If the problem is due to permission errors, ensure that both Cisco ER servers in the group are in the same Windows domain, or if they are in different domains, that they are logged in with the same user name and password.



Related Topics

Control Center

Troubleshooting ALI Data Uploads

Periodically, you must export your ALI data and submit it to your service provider. The ALI data is used to route emergency calls from your network to the correct PSAP, and provide the PSAP with information about the location of the emergency call.

Cisco Emergency Responder (Cisco ER) lets you export the ALI data in a variety of NENA formats. Ask your service provider which format you should use.

During the upload process, you might find that some ALI data records did not upload correctly. Your service provider should be able to provide you with a list of errors, or you might see these when using your service provider's data upload software. You must fix any mistaken records and resubmit the ALI data export file. To fix the records, you might need to manually edit the records in error.

These sections describe the general procedure for fixing ALI data records, and explain how to edit the various types of NENA formatted files:

Fixing ALI Data Records

Editing NENA 2.0 and 2.1 File Formats

Editing NENA 3.0 File Formats

Fixing ALI Data Records

If you receive data errors when uploading ALI records to your service provider, use this procedure to correct the errors.

Before You Begin

Obtain NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, from NENA or your service provider. This document explains the various NENA formats in detail.

Procedure


Step 1 Look through the error reports to determine the problems you encountered.


Step 1 Cisco Emergency Responder (Cisco ER) web interface, change the fields that were in error for the ERL/ALI records that failed. For example, if the Street Suffix was an unacceptable abbreviation, change it to an acceptable one. Save all of your changes.

Step 2 Export the ALI data again (see the online help).

Step 3 If any of the records in error were new, you must change the database function for the records. Because Cisco ER has already exported these records, Cisco ER will label them as updates rather than new insertions. However, because these records failed on upload, the service provider's database will view them as new.

Open the ALI export file in a text editor and change the function code for the records that you are fixing. Use an editor that will not add formatting or other extra characters. See these sections for details about editing the files:

Editing NENA 2.0 and 2.1 File Formats

Editing NENA 3.0 File Formats

Step 4 Submit the edited file to your service provider.


Editing NENA 2.0 and 2.1 File Formats

The NENA 2.0 and 2.1 file formats have these characteristics:

Fixed-length records

Fields are in a specific order

Unused fields are filled with blanks

End of record is indicated by an asterisk (*)

Use NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, to determine the byte location and length of each field. When you edit the file, ensure that you are not lengthening the records. Delete any extra spaces that get added. If the length of an item is less than the length of a field, pad the field with blanks. Depending on the field, padding might be on the right or the left.

The file contains one header and one trailer record. The ALI data records are contained between these records.

Table 6-2 describes the fields you are most likely to edit. You should use the Cisco ER web interface to change the other fields.

Table 6-2 NENA 2.0 and 2.1 Common Fields 

Field
Description

Function Code

Location: Byte 1.

Length: 1 character.

Description: The database function for the record. One of:

I—Insert new ALI record

C—Change existing record. You must have successfully uploaded the record once before you can use C. If you are correcting a record that has never been successfully uploaded, change the C to an I.

D—Delete the record. Cisco ER only generates a deletion record once, in the export file created after you deleted the ALI from the Cisco ER configuration. If you need to regenerate the record, cut and paste it from the previous export file (and adjust the record count), or recreate the ALI in Cisco ER, save it, export the data, then delete the ALI and export the data again.

Cycle Counter (sequence number)

Location: Byte 62 to 67.

Length: 6 characters.

Description: The sequence number of the file you are submitting to the service provider (for example, 1, 2, etc.) The number is right-aligned with leading spaces. Your service provider might ignore this field.

Record count

Location: Byte 62 to 70 in the trailer record.

Length: 9 characters.

Description: The total number of records in the file you are submitting to the service provider (for example, 1, 2, etc.) The number is right-aligned with leading spaces.


Editing NENA 3.0 File Formats

The NENA 3.0 file format has these characteristics:

Variable-length records.

Fields are a tag and data combination, and can be in any order.

Unused fields are not included. The presence or absence of a tag has this effect:

If the tag is not included, the previous value of the element, if any, is left unchanged.

If the tag is included with a blank value, any previous value for the element is removed.

If the tag is include with a non-blank value, the value of the element is changed to the new value.

Tags are separated by a vertical bar (|).

End of record is indicated by a pre-defined character.

Use NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, to determine tag name and values for each field. Ensure that your values do not exceed the maximum length for the field. You do not need to pad fields with extra blanks.

The file contains one header and one trailer record. The ALI data records are contained between these records.

Table 6-2 describes the fields you are most likely to edit. You should use the Cisco ER web interface to change the other fields.

Table 6-3 NENA 3.0 Common Fields 

Field
Description

Function Code

Tag: FOC.

Description: The database function for the record. One of:

I—Insert new ALI record (FOCI)

C—Change existing record (FOCC). You must have successfully uploaded the record once before you can use C. If you are correcting a record that has never been successfully uploaded, change the C to an I.

D—Delete the record (FOCD). Cisco ER only generates a deletion record once, in the export file created after you deleted the ALI from the Cisco ER configuration. If you need to regenerate the record, cut and paste it from the previous export file (and adjust the record count), or recreate the ALI in Cisco ER, save it, export the data, then delete the ALI and export the data again.

Cycle Counter (sequence number)

Tag: CYC.

Description: The sequence number of the file you are submitting to the service provider (for example, CYC1, CYC2, etc.) Your service provider might ignore this field.

Record count

Tag: REC in the header and trailer records.

Description: The total number of records in the file you are submitting to the service provider (for example, REC1, REC2, etc.)


Collecting Call History Logs

Cisco Emergency Responder (Cisco ER) maintains extensive call history logs, which include entries for each emergency call handled. You can view call history information from the administration and user interfaces. Cisco ER 1.2 maintains and can display detailed records of the most recent 10,000 Cisco ER emergency calls. There is no restriction on when these calls were placed.

When you upgrade from Cisco ER 1.1 to Cisco ER 1.2, your detailed call history records will be copied from the Cisco ER 1.1 directory to the Cisco ER 1.2 database.

If you need to maintain detailed records for liability purposes, you can view Cisco ER's raw log information and copy and back up these files for your records.

Cisco ER maintains log files in the /callHistory directory (relative to the Cisco ER installation directory). Each server maintains logs of the calls it handles. For example, if the standby Cisco ER server in a Cisco ER group takes over for the primary server, emergency calls made are reflected only in the standby server's logs. When the primary becomes available again, the primary server takes over logging, and the calls handled by the standby server are not reflected in the primary server's logs.

Cisco ER can create up to 99,999 logs, called callRecordsxxxxx.csv, where xxxxx is a digit from 00001 to 99999. Cisco ER starts a new log whenever the Cisco ER server is started, for example, by rebooting the server, or starting and stopping the server using the control center (see the "Starting and Stopping a Cisco Emergency Responder Server" section). Cisco ER also starts a new log if a log reaches 10,000 records.

Cisco ER uses these logs in numerical sequence, that is, 00001, 00002, 00003, and so forth. After finishing 99999, Cisco ER reuses 00001.

The files are in comma-separated (CSV) format, and the first record of each log explains the fields. You can view the file with any spreadsheet program that supports CSV formats, or you can use a text editor.

Collecting Trace and Debug Information

When you contact Cisco Technical Support for help with a problem that you are having with Cisco Emergency Responder (Cisco ER), Cisco might request that you collect trace and debug information.

Because collecting trace and debug information will affect Cisco ER's performance, you should only turn on tracing and debugging at Cisco's request. The generated information is for Cisco's use in resolving product problems.

Use the following sections to learn about:

Installing Syslog

Enabling Syslog

Installing Syslog

To install the syslog collector service on the Cisco ER server, perform the following steps:

Procedure


Step 1 Download the latest sacntservice from the CiscoWorks 2000 server.

Step 2 To install the syslog collector service on the Cisco ER server, run sacntservice/install.

Step 3 Reboot the machine.


Note If you do not reboot, the service may fail to start.


Step 4 Modify the SaEnvProperties.ini properties as shown below:

At SAC_PORT, enter 514

At SAC_SERVER, enter FGDN BINDNAME = <your_server_hostname>::SaReceiver<your_domain>


Note The format is <your_server_hostname>.<your_domain>.



Note The server name is case-sensitive.


To get more information, at DEBUG_LEVEL, change the debug level from 4 to 1 or 2.

Step 5 Map the RME server's domain name to the IP address of the RME server in the client's machine (in case you have name resolution problems):

c:\winnt\system32\drivers\etc\hosts file

Step 6 Enable the SNMP service on the machine.

Step 7 Add the machine as a managed device to the RME inventory Login to the CiscoWorks server. Go to:

Resource Manager>Essentials>Administration>Inventory>Add Device

and enter the appropriate details.

Step 8 To verify that the device was added successfully, look in the following location:

Resource Manager Essentials>Administration>Inventory>Import Status> Managed

Step 9 Enable syslog from the Cisco ER web interface. See the "Enabling Syslog" section.


Enabling Syslog

You need to enable syslog for Cisco Emergency Responder (Cisco ER) to collect trace and debug information. Perform the following steps.

Procedure


Step 1 From the Cisco ER web interface, select Cisco ER Group > Server Settings.

Cisco ER opens the Server Settings page.

Step 2 From the left column, select the server from which you need to collect debug or trace information.

Cisco ER displays the settings for the server.

Step 3 Scroll down to the debug package and trace package sections and select the packages that Cisco Technical Support has requested.

The lists in each section are identical; make sure that you select the package in the list that Cisco requested. Packages selected in the Debug list generate trace information plus extra debug data. If Cisco requests that you select all packages, click Select All for the appropriate list.

The available packages include:

CER_DATABASE—The database subsystem, covers the log information generated by the database access code.

CER_TELEPHONY—The telephony subsystem, used for interactions with Cisco CallManager.

CER_ONSITEALERT—The onsite alert subsystem for notifying onsite alert personnel.

CER_CLUSTER—The server cluster subsystem, used for communicating between Cisco ER groups in a cluster.

CER_SYSADMIN—The system administration web interface subsystem.

CER_PHONETRACKINGENGINE—The phone tracking subsystem, which runs the phone tracking and switch-port and phone update processes.

CER_GROUP—The Cisco ER server group subsystem, used for communicating between servers within a group.

CER_REMOTEUPDATE—The remote update subsystem, which manages updates between servers.

CER_AGGREGATOR—The aggregator module covers all Cisco ER server communication and data handling with the phone tracking engine. The module includes the search and lookup of tracked data for the subsystems like cluster, Administration, Cisco IP SoftPhone and call routing.

CER_CALLENGINE—The call engine subsystem, which routes and processes calls.

Step 4 Click Update to save and activate your changes.

Cisco ER begins generating the requested trace and debug information.

The information is placed in a log file in the /logs subdirectory of the Cisco ER installation directory. Or, if you configured Cisco ER to use the CiscoWorks 2000 Syslog facility, the data is sent to syslog. Send this information to the Cisco Technical Support group with which you are working.


Note See the "Collecting System Logs with Syslog" section for information on using syslog.


Step 5 When you have finished generating debug and trace information, turn off debug and trace by clicking Clear All for each section in which you have made a selection. Then, click Update to complete the change.


Related Topics

Server Settings (for Cisco ER Group)

Viewing Windows Event Messages

You can view Cisco Emergency Responder (Cisco ER) messages in the Windows Event Viewer to help diagnose problems with the software. In Event Viewer, look for messages from the source "CiscoER."

See the Microsoft documentation for information on using Event Viewer. Event Viewer is an administration tool.

Managing Performance

Refer to the Table 1-6 on page 1-24 for supported MCS platforms and their Cisco ER scalability.

Cisco ER performance can be affected if Cisco ER is managing switches across a WAN link. Cisco ER must send SNMP requests to the managed switches, and WAN delays can lead to SNMP timeouts and increase the time needed to track phone and switch changes. You might need to tune the SNMP parameters. See the "Configuring the SNMP Connection" section for more information.

Integrating with Network Management Systems

You can manage the status of the Cisco Emergency Responder (Cisco ER) server remotely using CiscoWorks2000 or another SNMP-based network management system. CiscoWorks2000 is the standard Cisco network management system, but it is not bundled with Cisco ER. For more information about CiscoWorks2000, Campus Manager, and Topology Services, refer to the documentation, available at the following URL:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/cw2000/index.htm

These topics provide information to assist you in integrating Cisco ER with network management systems:

Understanding CDP Support

Monitoring Cisco Emergency Responder Subsystem Status

Collecting System Logs with Syslog

Understanding CDP Support

Cisco Emergency Responder (Cisco ER) uses the Cisco Discovery Protocol (CDP) to periodically send out CDP messages, on the active interface, to a designated multicast address. These messages contain information such as device identification, interface name, system capabilities, SNMP agent address, and time-to-live. Any Cisco device with CDP support can locate a Cisco ER server by listening to these periodic messages.

Using information provided through CDP, the CiscoWorks2000 Server can detect the Cisco ER server, and the Campus Manager application, Topology Services, can build topology maps displaying the Cisco ER server.

In addition to sending out CDP messages, the Cisco ER server uses CDP to locate phones that support CDP. You must ensure CDP is enabled on your switches so that Cisco ER can obtain this information through SNMP queries to the switches.

Monitoring Cisco Emergency Responder Subsystem Status

Cisco Emergency Responder (Cisco ER) supports the SYSAPPL-MIB that allows you to use CiscoWorks2000 or a third-party SNMP browser to remotely access information about the following Cisco ER components:

Cisco ER Server

CERServer.exe

Cisco ER Web Administration

CERAdminServer.exe

Cisco PhoneTrackingEngine

CERPhoneTracking.exe

MSQL Server-related Services

The SYSAPPL-MIB uses the Simple Network Management Protocol (SNMP). Cisco ER supports the following SYSAPPL-MIB tables:

SysApplInstallPkgTable—provides installed application information such as Manufacturer, Product Name, Version installed, Date installed, and Location, which is a partial URL for accessing the associated Application Administration web page (when applicable).

SysApplRunTable—describes the application starting time and run-time status.

SysApplInstallElmtTable—describes the individual application elements, or associated executables, which comprise the applications defined in the SysApplInstallPkgTable.

SysApplElmtRunTable—describes the processes, or executables, that are currently running on the host system.

Collecting System Logs with Syslog

You can configure Cisco Emergency Responder (Cisco ER) to use the Cisco Syslog Collector. Cisco Syslog Collector and Cisco Syslog Analyzer are offered with CiscoWorks2000 as part of the Resource Management Essentials package. You can also adapt Syslog output from Cisco ER for use with other network management systems.

The Cisco Syslog Collector keeps common system logs of messages reported to Cisco ER.

The Cisco Syslog Analyzer controls and displays all events efficiently so they can easily be read, interpreted, and used for system maintenance and problem solving.

To install and configure the Cisco Syslog Collector, refer to the CiscoWorks2000 documentation.

Procedure


Step 1 Select Cisco ER Groups > Cisco ER Group Settings.

Cisco ER opens the Cisco ER Group Settings page.

Step 2 Select enable in Enable Syslog.

Step 3 Enter the fully-qualified DNS name of the CiscoWorks2000 server in Syslog Server, for example, server.domain.com.

Step 4 Click Update Settings to save your changes.

Cisco ER immediately begins writing messages to syslog.


Related Topics

Cisco ER Group Settings

Backing Up and Recovering Data

Cisco Emergency Responder (Cisco ER) 1.2 stores its configuration data in its own MSDE database. The Cisco IP Telephony Applications Backup Utility Version 3.5.44 provides a reliable and convenient way to perform regularly scheduled automatic or user-invoked backups of your data. Use the utility with Cisco ER 1.2 to manually perform periodic data backups. Ensure that you back up both the Cisco ER publisher and subscriber.

For instructions on using the utility, see the following URL:

http://www.cisco.com/univercd/cc/td/doc/product/voice/backup/index.htm

Related Topics

Collecting Call History Logs