Cisco Emergency Responder Administration Guide 1.1
Troubleshooting Emergency Responder
Downloads: This chapterpdf (PDF - 307.0KB) The complete bookPDF (PDF - 2.21MB) | Feedback

Troubleshooting Emergency Responder

Table Of Contents

Troubleshooting Emergency Responder

Troubleshooting Phone-Related Problems

Too Many Unlocated Phones

Moved Phone Located On Wrong Switch

Cisco IP Softphone Movements Not Tracked

Phone Sometimes Disappears in CER

Wrong ERL is Used for a Shared Line

Troubleshooting Emergency Call Problems

Emergency Calls are Not Being Intercepted by CER

ELIN not Transmitted to the PSAP

ELIN For Default ERL Used For Calls From Other ERLs

Emergency Calls Not Routed to the Correct PSAP

Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

PSAP Call Back Errors

Onsite Alert Personnel Are Not Getting Web Alerts

Onsite Alert Personnel Are Not Getting Telephone Alerts

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

Incorrect Location Information Sent To Onsite Alert Personnel

Emergency Call History Problems

Troubleshooting CER System and Administration Problems

Troubleshooting Login Problems

Troubleshooting CER Switch and Port Configuration Problems

Troubleshooting CER System Problems

Troubleshooting Cisco CallManager Configuration Problems

Identifying the CER Groups and Servers in a CER Cluster

Starting and Stopping a CER Server

Collecting Call History Logs

Collecting Trace and Debug Information

Viewing Windows Event Messages

Managing Performance

Integrating with Network Management Systems

Understanding CDP Support

Monitoring Emergency Responder Subsystem Status

Collecting System Logs with Syslog

Backing Up and Recovering Data


Troubleshooting Emergency Responder


These topics address problems you might encounter, and provide ways to resolve them; also included are other tasks associated with problem identification and resolution.

Troubleshooting Phone-Related Problems

Troubleshooting Emergency Call Problems

Troubleshooting CER System and Administration Problems

Identifying the CER Groups and Servers in a CER Cluster

Starting and Stopping a CER Server

Collecting Call History Logs

Collecting Trace and Debug Information

Viewing Windows Event Messages

Managing Performance

Integrating with Network Management Systems

Backing Up and Recovering Data

Troubleshooting Phone-Related Problems

These topics help you troubleshoot problems related to assigning phones to ERLs and managing the phones:

Too Many Unlocated Phones

Moved Phone Located On Wrong Switch

Cisco IP Softphone Movements Not Tracked

Phone Sometimes Disappears in CER

Wrong ERL is Used for a Shared Line

Too Many Unlocated Phones

Emergency Responder obtains a list of registered phones from Cisco CallManager and tries to locate all phones. If Emergency Responder cannot locate a phone, it places the phone in the Default ERL and puts it in a list of unlocated phones. If there are a lot of unlocated phones, first try running the switch port and phone update process to see if CER can resolve some of the problems automatically. See the "Manually Running the Switch-Port and Phone Update Process" section for more information.

These are some things that can prevent Emergency Responder from locating a phone:

The phone is attached to a switch that is not defined in Emergency Responder. See the "Identifying the LAN Switches" section for information on defining switches.

The phone is connected to an unsupported device, such as a router port, a hub connected to a router, or an unsupported switch. See the "Network Hardware and Software Requirements" section for a list of supported switches. See the "Manually Defining a Phone" section for information on configuring these types of phones if you cannot connect them to a supported device.

The phone is connected to a hub, which is connected to a supported switch port, but it does not support CDP. CER can consistently discover CDP-enabled phones attached to hubs (which are attached to supported switch ports), but cannot always track non-CDP phones attached in this manner. For non-CDP phones, ensure the phones are attached directly to supported switch ports.

The switch to which the phone is connected is currently unreachable, for example, it does not respond to SNMP queries. This could be for several reasons:

The SNMP read community string on the switch does not match the string configured in CER. Correct the CER configuration. See the "Configuring the SNMP Connection" section.

The phone requires CAM table access, but CAM tracking is not enabled for the switch in CER. See the "Identifying the LAN Switches" section.

There is a network outage preventing communication between the CER server and the switch. Locate and resolve the network outage problem.

Unreachable switches are not retried until CER runs the next full switch-port and phone update process, unless you run it against the individual switch (see below).

The phone has moved to a switch served by a different CER group. If this is the case, the CER group name is shown for the phone in the unlocated phones list. If the phone is not locatable in the next incremental phone tracking process after it is moved, the phone remains unlocated in any CER group until a full switch-port and phone update process is run.

The phone requires CAM-based tracking, but CAM-based tracking is not enabled on the switch to which the phone is connected. Cisco IP Softphone and some other phone models require CAM-based tracking. See the "Identifying the LAN Switches" section for information on enabling CAM-based tracking, and "Network Hardware and Software Requirements" section for a list of phones that require CAM-based tracking.

After fixing the problems that are preventing CER from locating phones, run the switch-port and phone update process on the affected switches, or on all switches:

To run the process on a specific switch—Select Phone Tracking > LAN Switch Details and select the switch in the left-hand column; then click Locate Switch Ports.

To run the process on all switches—Select Phone Tracking > Run Switch-Port & Phone Update.

Related Topics

Identifying Unlocated Phones

Unlocated Phones

Moved Phone Located On Wrong Switch

When you move a phone from one port to another port, it takes some time for the switch to age out the entry for the phone. If CER's incremental phone tracking process inspects the switch before the entry times out, CER thinks that the phone is still connected to the switch.

This situation should occur only rarely, because the switch ages out the entry within 180 seconds or so.

This miss-assignment is resolved the next time CER runs the switch-port and phone update process. If you need to resolve it sooner, manually run the process on the switch. Select Phone Tracking > LAN Switch Details and select the switch in the left-hand column; then click Locate Switch Ports.

Related Topics

LAN Switch Details

Cisco IP Softphone Movements Not Tracked

If the computer that hosts a Cisco IP Softphone moves, CER only tracks the movement during the full switch-port and phone update process. The movements are not tracked by the incremental phone tracking process. Any emergency calls from the Cisco IP Softphone are routed based on the last ERL assigned during the switch-port and phone update process.

If your organization uses Cisco IP Softphones extensively, and users move them frequently, you might consider running the full switch-port and phone update process more than once per day.

These are some other points to keep in mind when supporting Cisco IP Softphone:

You must configure the Cisco IP Softphone software to work with CER. See the Cisco IP Softphone documentation for more information.

You must enable CAM tracking on the switches to which Cisco IP Softphones are attached. See the "Identifying the LAN Switches" section for more information.

Cisco IP Softphones are not tracked if they are attached through a hub, even if the hub is attached to a supported switch port.

Cisco IP Softphones are only tracked by the incremental phone update process if the Cisco IP Softphone is detached from the network for more than 10 minutes. That is, at least 10 minutes must pass between unplugging the computer from the network and plugging it into a different port. If the move takes less than 10 minutes, the move is only discovered during the next full switch-port and phone update process.

Cisco IP Softphones, like other phones, must be attached to the network using Ethernet. CER does not support Cisco IP Softphones attached to other types of network such as Token-Ring or ATM.

A single CER group can support up to 500 Cisco IP Softphones.

If you change the IP address on a computer running Cisco IP Softphone, the change is not reflected in the CER GUI, because Cisco IP Softphone continues to advertise the old IP address. If you change the IP address, close and then restart Cisco IP Softphone so that Cisco IP Softphone advertises the correct address.

Related Topics

Defining the Phone Tracking and Switch Update Schedules

Phone Sometimes Disappears in CER

If CER is in the middle of a phone tracking process, and a phone is in the middle of homing to a different Cisco CallManager cluster, no Cisco CallManager cluster has a record of the phone. Thus, CER does not know the phone exists, and you will not be able to look up the phone in the CER interface. However, assuming the phone successfully connects to a Cisco CallManager cluster, CER tracks the phone during the next incremental phone tracking process, and the phone should then appear in the CER interface.

This problem can also occur if phones are reconnecting to a primary Cisco CallManager server from a backup server during the CER phone tracking process.

Wrong ERL is Used for a Shared Line

When two or more phones with a shared line appearance move from switches that are monitored by one CER group to switches that are monitored by a different CER group, then CER may assign an incorrect ERL to these phones during an emergency call. This can occur when the phones move to a different campus that has a different Cisco CallManager cluster (although the moved phones are still registered with the original Cisco CallManager cluster), and it can also occur when the phones move within a single large campus that is served by multiple Cisco CallManager clusters.

Because the moved phones are still registered to their original Cisco CallManager cluster, emergency calls from these phones are routed to the original CER group. In this case, the CER group detects that the calling phone is connected to a switch that is monitored by a different CER group, and the call is forwarded to the appropriate CER group through an H.323 inter-cluster trunk. Because the inter-cluster trunk does not pass the MAC address of the calling phone, the receiving CER group does not know the MAC address of the calling phone and must associate the phone to an ERL based on the calling party number. In cases with a single phone connected to the switches monitored by the receiving CER group, this is not a problem. However, when multiple phones with a shared line appearance connect to switches monitored by the receiving CER group, then CER must guess which phone has placed the emergency call. If all of the phones with a shared line appearance are in the same ERL, the guess is correct. If the phones span multiple ERLs, then the guess might be incorrect.

Related Topics

Deploying CER In Two Main Sites

Creating Route Patterns for Inter-CER-Group Communications

Troubleshooting Emergency Call Problems

These topics help you troubleshoot problems related to the routing of emergency calls and the information supplied with the calls:

Emergency Calls are Not Being Intercepted by CER

ELIN not Transmitted to the PSAP

ELIN For Default ERL Used For Calls From Other ERLs

Emergency Calls Not Routed to the Correct PSAP

Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

PSAP Call Back Errors

Onsite Alert Personnel Are Not Getting Web Alerts

Onsite Alert Personnel Are Not Getting Telephone Alerts

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

Incorrect Location Information Sent To Onsite Alert Personnel

Emergency Call History Problems

Emergency Calls are Not Being Intercepted by CER

If CER is not intercepting emergency calls, there is probably a mistake in your Cisco CallManager configuration or its representation in the CER configuration. Check these items (based on the names used in the examples in "Configuring Cisco CallManager for Emergency Responder.")

The emergency call number (911) is in the Phones partition and uses the E911CSS calling search space. Ensure this number was identified during CER installation (see the "Installing Cisco Emergency Responder on a New System" section.) This ensures that users can dial the emergency number. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number.

The standby CER server's route point (912) is in the E911 partition and uses the E911CSS calling search space. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number. Ensure this number is defined as the standby server's route point in the CER configuration (see the "Configuring CER Server Group Telephony Settings" section.)

The PSAP callback route point pattern (913XXXXXXXXXX) is in the E911 partition and uses the E911CSS calling search space. See the "Creating the Emergency Call Route Points" section for information on setting up the Cisco CallManager configuration for this number. Ensure this number is defined as the PSAP callback route point pattern in the CER configuration, and that the strip prefix (913) is also identified (see the "Configuring CER Server Group Telephony Settings" section.)

All ELIN route patterns are in the E911 partition. See the "Creating the Route Patterns for ELINs" section for information on setting up the Cisco CallManager configuration for these numbers.

All phones and CTI ports (both device and line) are in the Phones partition and use the PhoneCSS calling search space. You can use additional partitions, but they must be set up with relationship to the CER partitions and calling search spaces in the same manner as these partitions in the examples described in the "Setting Up Emergency Responder to Handle Emergency Calls" section.

All gateways to the service provider's network use the E911CSS calling search space. See the "Configuring the Calling Search Space for the Gateways Used to Connect to the PSAP" section for more information.

ELIN not Transmitted to the PSAP

If the ELIN is not transmitted to the PSAP, and you are using a PRI connection to route emergency calls to the PSAP, check the configuration of the gateway. The PRI must be configured to send the real calling party number (which will be the ELIN) rather than a static number, such as the main site number. See the "Obtain CAMA or PRI Trunks to the PSTN" section.

ELIN For Default ERL Used For Calls From Other ERLs

If an emergency call is assigned an ELIN defined for the Default ERL rather than an ELIN assigned to the ERL whence the call was made:

Check the Cisco CallManager configuration for the route pattern for the ELIN you expected to be used. See the Creating the Route Patterns for ELINs.

Check the ERL definition in CER to ensure that the ELIN is correctly configured for the ERL. See the "Setting Up an Individual ERL and Its Automatic Location Information (ALI)" section.

If the route pattern for an ERL fails, CER uses the route pattern defined for the Default ERL.

Emergency Calls Not Routed to the Correct PSAP

If an emergency call is not routed to any PSAP, check whether the route patterns used for the ERL from which the call was made and for the default ERL are configured and use the correct partitions and calling search spaces (see the "Creating the Route Patterns for ELINs" section). Ensure that the partitions and calling search spaces for the gateways are correct (see the "Configuring the Calling Search Space for the Gateways Used to Connect to the PSAP" section).

If an emergency call successfully leaves your network but does not get routed to the correct PSAP, look at these possible points of failure:

Is CER configured to assign the correct ELIN to the ERL assigned to the phone? Emergency calls are routed based on the ELIN, so if you assign the wrong ELIN, the call will not be routed correctly. See the "Creating ERLs" section.

If the ELIN is correct, is the ELIN's route pattern configured to use the correct gateway? If you select the wrong gateway, the call might be routed to a part of the service provider's network that cannot connect to the desired PSAP. Consult with your service provider to determine gateway requirements. See these topics:

Setting Up the ELIN Numbers to Route Emergency Calls and Enable PSAP Callbacks

Deploying CER in One Main Site with Two or More PSAPs

Does the service provider's ALI database contain the correct information for the ELIN? Emergency call routing outside your network is based on the information in the service provider's database, not on the information in your local network. See the "Exporting ERL and ALI Information for Submission to Your Service Provider" section.

Does the emergency caller's phone register with a Cisco CallManager cluster supported by a different CER group than the CER group that supports the originating switch port? Then you might have a miss-configured CER cluster. See these topics:

Installing Cisco Emergency Responder on a New System

Creating Route Patterns for Inter-CER-Group Communications

Configuring CER Server Group Telephony Settings


Note If the call reaches the PSAP, but the PSAP cannot talk to the caller, ensure that the Cisco CallManager for the remote CER group has the Cisco CallManager for the local CER group defined as a gateway.


Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed

If callers hear a busy signal when calling the emergency call number, or if emergency calls sometimes do not get routed, there is probably a problem with the configuration of your standby CER server:

If you have only configured a primary CER server, install and configure a standby CER server. If CPU utilization on the primary server reaches 100%, CER cannot handle emergency calls. In this case, the standby server handles the calls.

Check the route point configuration for the standby server. Ensure the emergency call route point's call forward settings are configured to forward calls to this number. See the "Creating the Emergency Call Route Points" section for information on the Cisco CallManager configuration, and the "Configuring CER Server Group Telephony Settings" section for the CER configuration.

PSAP Call Back Errors

You might encounter these problems if a PSAP operator tries to call back an emergency caller using the ELIN provided by caller ID:

Symptom    PSAP could not reach the original emergency call extension.

Action    CER caches a mapping between the caller's true extension and the ELIN you define for an ERL. If more calls get made than the number of ELINs you define for an ERL, CER must reuse these numbers and thus overwrites the original caller's extension. You can view the call history to determine the extension of the original caller. See the "What Happens When an Emergency Call Is Made" section.

If this is not the problem, check the configuration of the PSAP callback route point in Cisco CallManager and CER (see the "Creating the Emergency Call Route Points" section and the "Configuring CER Server Group Telephony Settings" section), and the ELIN translation patterns in Cisco CallManager (see the "Creating the Translation Patterns for ELINs" section).

Symptom    Onsite alert (security) personnel get callbacks from the PSAP.

Action    CER routes PSAP callbacks to the onsite alert personnel for the default ERL if ELIN-to-extension mapping for the emergency call has expired from the cache. By default, this is three hours, although you can configure expiration to be a longer or shorter time. See the "CER Group Settings" section.

Onsite Alert Personnel Are Not Getting Web Alerts

If the onsite alert personnel are not seeing web alerts in the CER user interface, the CERAdminServer service is probably not yet initialized. This system is initialized when someone logs into the CER administration interface.

After you start the CER server, make sure you log into the administration interface at least once. You can then log out of the interface. All that is required is that someone log in once, not that someone is always logged in. You only have to log in again if you restart the server or if the service gets restarted somehow on its own (for example, a power outage takes down all computers, then the computers are restarted when power comes up; or someone unplugs the server then plugs it in again, thus restarting the server).

Onsite Alert Personnel Are Not Getting Telephone Alerts

If the onsite alert personnel are not getting telephone alerts when an emergency call is made in an ERL they are covering, ensure that all phones and CTI ports (both device and line) are in the Phones partition and use the PhoneCSS calling search space. You can use additional partitions, but they must be set up with relationship to the CER partitions and calling search spaces in the same manner as these partitions in the examples described in the "Setting Up Emergency Responder to Handle Emergency Calls" section.

Also, ensure that the CER configuration for the Cisco CallManager clusters is correct. The CER configuration should show the correct begin address for the telephony ports you defined as CTI ports in Cisco CallManager, and the number of telephony ports should be the correct number and it must be greater than 0 for any calls to occur. CER uses this CTI ports to place the telephone calls to onsite alert personnel.

If the Windows Event Viewer shows the error message "No port to place call," this means that there were not enough CTI ports defined to initiate all the calls to onsite alert personnel. Define more ports.

Onsite Alert Personnel Not Getting Email (or Paging) Notifications

If the onsite alert personnel are not getting email, or email-based pages, even though you configure email addresses for them (see the "Onsite Alert Settings" section), check the CER configurations SMTP settings. Ensure that the SMTP server address and source mail ID are correct (see the "CER Group Settings" section), and that there is an account for the mail ID in the SMTP server.

Incorrect Location Information Sent To Onsite Alert Personnel

If your onsite alert (security) personnel are receiving incorrect location information for an emergency call, consider these potential problems:

Is the ALI data for the ERL correct? See the "Creating ERLs" section.

Is the phone location data for the switch port correct? See the "Configuring Switch Ports" section.

Is the correct ERL assigned to the switch port to which the phone is connected? If not, there could be two problems:

Someone switched wires on the switch, so your formerly correct configuration is no longer correct. Wires cannot be moved from port to port without potentially invalidating the ERL assignment. See the "Data Integrity and Reliability Considerations" section.

The wiring closet is secure, the ERL assignment is simply incorrect. See the "Configuring Switch Ports" section.

Did the call come from the Default ERL (assuming you do not use the Default ERL for any permanent ERL)? This could indicate these problems:

The phone is connected to an unsupported port and is not defined as a manual phone. See the "Manually Defining a Phone" section.

The phone is not supported and it is not defined as a manual phone. See the "Manually Defining a Phone" section.

The phone is supported but CER could not locate it. You might have to manually assign the phone to an ERL if you cannot resolve the problem. See the "Too Many Unlocated Phones" section.

Did the call come from a manually-defined phone extension? If so, it is likely the incorrect ERL is assigned, perhaps because the phone moved. See the "Manually Defining a Phone" section.

Emergency Call History Problems

These are some issues you might encounter when viewing the emergency call history information (see the "Viewing the Emergency Call History" section):

Symptom    Emergency call information does not show up in call history right away.

Action    CER writes call history information to the LDAP directory every 45 seconds. You should be able to view history information after 45 seconds.

Symptom    The call history does not show the ELIN and route pattern used for a call.

Action    If the call could not be routed to the PSAP, you will not see an ELIN or route pattern. Check to determine why the call could not be routed. See the "Emergency Calls Not Routed to the Correct PSAP" section.

Symptom    The ELIN for calls made from remote CER groups is empty.

Action    When a phone is moved to a remote CER group, CER only writes the inter-CER-group route pattern in the call history, and does not include the ELIN.

Troubleshooting CER System and Administration Problems

These topics help you troubleshoot problems related to the CER system and its administration, such as server and GUI problems:

Troubleshooting Login Problems

Troubleshooting CER Switch and Port Configuration Problems

Troubleshooting CER System Problems

Troubleshooting Cisco CallManager Configuration Problems

Troubleshooting Login Problems

These are some issues you might encounter while logging into CER:

Symptom    You occasionally cannot log in as LAN switch administrator or ERL administrator.

Action    Check the error message for the reason for the failed login. If the system administrator is logged in, you cannot log in as a LAN switch or ERL administrator. This is because the system administrator can configure the same data as the LAN switch and ERL administrators; you are prevented from logging in simultaneously to ensure data integrity. If the system administrator is not logged in, separate LAN switch and ERL administrators can log in simultaneously.

Symptom    You cannot open multiple CER sessions using Netscape Navigator.

Action    Netscape Navigator uses the same session ID across multiple windows. This creates problems if you try to log into CER using different IDs. Normally, you can open multiple windows when logged in as system administrator. With Internet Explorer, if you open separate IE session by starting a new IE instance (rather than by opening a new window from an existing session), IE uses different session IDs, and you should be able to log in using separate IDs (for example, as a user and an administrator, or as LAN switch and ERL administrators).

Symptom    The login page takes a long time to load.

Action    When you first connect to CER, CER goes through an initialization process before the login screen can be loaded. Once the initialization process is completed, subsequent logins (while your system is running) should not take as long.

Symptom    CER GUI says it cannot connect to LDAP.

Action    CER maintains its configuration in the Cisco CallManager LDAP database. If CER cannot access the database, you cannot use the configuration screens. Check to ensure there is connectivity to the computer running the LDAP directory. If there are no connectivity problems, check the computer running the directory to ensure that the directory service has been started and is running properly.

Related Topics

Troubleshooting CER System Problems

Troubleshooting CER Switch and Port Configuration Problems

These are some issues you might encounter while configuring switches or switch ports in CER:

Symptom    CER is configured with Cisco CallManager information, but no phones get discovered.

Action    Ensure that the Cisco CallManager servers are reachable on the network. Then, ensure that the SNMP read community strings are configured correctly for the switches and Cisco CallManager servers (see the "Configuring the SNMP Connection" section.) Then, manually run the switch port and phone update process (see the "Manually Running the Switch-Port and Phone Update Process" section.)

Symptom    CER does not show the ports on a switch configured in CER.

Action    If you add a supported switch to CER and run phone tracking on the switch after adding it, you should be able to view the list of 10/100 Ethernet ports on the switch. If CER does not list the ports, check the SNMP settings in CER for the switch (see the "Configuring the SNMP Connection" section.) Also, verify that the switch is reachable over the network. Retry the selective phone tracking process on the switch (click Locate Switch Ports when viewing the switch details; see the "LAN Switch Details" section.)

If the problem persists, ensure that the switch is supported (see the "Network Hardware and Software Requirements" section.) Also, check the Windows Event Viewer for error messages.

Symptom    Some phones do not appear in the switch port list.

Action    Any phones that cannot be located on a switch configured in CER will appear on the unlocated phones list (see the "Unlocated Phones" section.) See the "Too Many Unlocated Phones" section for a list of reasons that a phone could not be located.

Symptom    Cannot delete a switch from the CER configuration.

Action    You cannot delete a switch when a phone tracking process is in progress. Retry the deletion after the process has ended. If this is not the problem, the CER server might not be running. Check the control center and restart the server (see the "Starting and Stopping a CER Server" section.)

If the problem persists, there might have been an LDAP failure the last time you tried to update the switch configuration information in CER. Try updating the switch information again (for example, by changing the notes). If the update is successful, you should then be able to delete the switch from the CER configuration.

Symptom    Import or export of the switch port details fails.

Action    If a switch port import or export attempt fails, it might be due to these reasons: the first switch-port and phone update process has not yet ended (wait for it to finish); the CER server is not running (use the control center to restart it, see the "Starting and Stopping a CER Server" section); the CER server is not completely initialized (wait for it to initialize).

Symptom    The import of some switch port configurations fail.

Action    To import switch port configurations, CER must already be configured with the switch and CER must first discover the ports on the switch using the switch-port and phone update process. If you try to import a configuration for ports not yet discovered in CER, the importation of those settings fails. See the "Manually Running the Switch-Port and Phone Update Process" section for information on the process. Run it on the switches whose port configurations you could not import, then retry the import.

Symptom    Port location information is lost after restarting the CER server.

Action    CER waits five minutes before saving port location information. If the CER server goes down, or you restart it, within five minutes of changing location information, you loose the changes you made.

Symptom    Phones moved from other CER groups to this CER group, and then moved back, are still showing up in the switch port details for the CER group.

Action    This types of phones are not removed from the switch port details until the next full switch-port and phone update process is run. If this is an issue for you, you can run the process on the switch (or on all switches) manually. See the "Manually Running the Switch-Port and Phone Update Process" section.

Troubleshooting CER System Problems

These are some issues you might encounter with general operation of the CER system and the configuration screens that involve the CER server, group, and cluster:

Symptom    The CER server does not start.

Action    Check whether the CER server is configured. See the "Configuring CER Servers" section for more information.

Symptom    CER exits after starting.

Possible Cause    You have configured CER to use a TCP port that is already in use.

Action    Check the Windows Event Viewer for the message "Cisco ER could not open socket at port peer-tcp-port, Exiting." If you see this message, change the CER group configuration to use a different TCP port. See the "Configuring a CER Server Group" section for instructions.

Symptom    The CER Groups in Cluster screen does not load, and exhibits the error "Cannot connect to primary LDAP."

Action    Check the connectivity to the computer running the primary LDAP directory for the cluster to which the CER group belongs. If there is connectivity, check the computer to ensure that the directory service is started and running correctly. If this does not resolve the problem, CER might not have been able to resolve the host name of the primary LDAP directory during CER installation. See the "Installing Cisco Emergency Responder on a New System" section for information on Cisco CallManager LDAP database requirements.

Symptom    I can see the entry for a CER group in a drop-down list, but when I select the group, no data appears.

Action    CER might be uninstalled or not running on the remote machine. If CER is uninstalled, delete the corresponding entry from the CER group using the CER groups in Cluster page (see the "Identifying the CER Groups and Servers in a CER Cluster" section). If the CER server is not running, restart it using the control center (see the "Starting and Stopping a CER Server" section.)

Related Topics

Identifying the CER Groups and Servers in a CER Cluster

Starting and Stopping a CER Server

Viewing Windows Event Messages

Managing Performance

Backing Up and Recovering Data

Troubleshooting Cisco CallManager Configuration Problems

These are some issues that you might encounter with CER's communications with Cisco CallManager. Additional problems with symptoms that involve emergency call failures are discussed in the "Troubleshooting Emergency Call Problems" section.

Symptom    CER does not register with the route points and CTI ports configured for its use.

Action    Ensure that the route points and CTI ports are associated with the Cisco CallManager CER user (see the "Creating an Emergency Responder Cisco CallManager User" section.) Ensure that the CTI Manager and DC Directory on the Cisco CallManager server are running properly.

Symptom    When trying to delete a Cisco CallManager from the CER configuration, CER prevents me and displays the message "Phone tracking in progress."

Action    You cannot delete a Cisco CallManager server from the CER configuration while a phone tracking process is in progress. Retry the deletion after the process has ended.

Identifying the CER Groups and Servers in a CER Cluster

If you are connected to the administrator's interface on a CER server, you can view the details of the server and the CER group's standby server by selecting CER Group > Server Settings.

You can also identify the CER groups and their CER servers that are in the same CER cluster. To view the other CER groups in the cluster, select Reports > CER Groups in Cluster. From the CER Groups in Cluster page, select the group you want to view from the left column, and Emergency Responder displays the CER servers that are in the group. To view the details for these servers, you must connect to the Emergency Responder administrator's interface running on one of the servers.

If you need to uninstall a CER group, first delete the group from the CER cluster using this page. You must log in as a system administrator to delete the group. Deleting the group from the cluster simply removes the entries for the group from the CER cluster's LDAP directory; it does not remove CER from the group's servers.

Related Topics

CER Server Groups in Cluster

Starting and Stopping a CER Server

When you install CER, the CER server is set up to automatically start whenever the computer is powered up or rebooted. However, you can stop and then restart a CER server through the CER administrator's interface without powering down or rebooting the computer. You might find this helpful if you are trying to debug a problem, or if you are trying to move a CER server to another computer.

To start or stop a CER server, select CER Group > Control Center. From the control center, you can click Start to restart the server, or Stop to stop the server. The buttons only appear if the action is possible; for example, Start only appears if the server is stopped. Table 6-1 explains the meaning of the icons you might see on this page.

The control center only lists the servers within a CER group.

Table 6-1 CER Control Center Icons

Icon
Meaning

The CER server is started and functioning normally.

The CER server was stopped by the administrator.

The CER server is experiencing connectivity or permission errors.

Verify that the CER server is connected to the network and running properly.

If the problem is due to permission errors, ensure that both CER servers in the group are in the same Windows domain, or if they are in different domains, that they are logged in with the same user name and password.


Related Topics

Control Center

Collecting Call History Logs

Emergency Responder maintains extensive call history logs, which include entries for each emergency call handled.

You can view call history information from the administration and user interfaces. The interface shows you detailed records for the past three months, or summary information for three to twelve months ago. However, if you need detailed information for periods prior to three months ago, or if you need to maintain detailed records for liability purposes, you can view Emergency Responder's raw log information. You can copy and back up these files for your records.

Emergency Responder maintains log files in the /callHistory directory (relative to the CER installation directory). Each server maintains logs of the calls it handles. For example, if the standby CER server in a CER group takes over for the primary server, emergency calls made are reflected only in the standby server's logs. When the primary becomes available again, the primary server takes over logging, and the calls handled by the standby server are not reflected in the primary server's logs.

Emergency Responder can create up to 99,999 logs, called callRecordsxxxxx.csv, where xxxxx is a digit from 00001 to 99999. Emergency Responder starts a new log whenever the CER server is started, for example, by rebooting the server, or starting and stopping the server using the control center (see the "Starting and Stopping a CER Server" section). Emergency Responder also starts a new log if a log reaches 10,000 records.

Emergency Responder uses these logs in numerical sequence, that is, 00001, 00002, 00003, and so forth. After finishing 99999, Emergency Responder reuses 00001.

The files are in comma-separated (CSV) format, and the first record of each log explains the fields. You can view the file with any spreadsheet program that supports CSV formats, or you can use a text editor.

Collecting Trace and Debug Information

When you contact Cisco Technical Support for help with a problem you are having with Cisco Emergency Responder, Cisco might request that you collect trace and debug information.

Because collecting trace and debug information will affect Emergency Responder's performance, you should only turn on tracing and debugging at Cisco's request. The generated information is for Cisco's use in resolving product problems.

To collect trace and debug information, perform the following steps.

Procedure


Step 1 From the Emergency Responder web interface, select CER Group > Server Settings.

Emergency Responder opens the Server Settings page.

Step 2 From the left column, select the server from which you need to collect debug or trace information.

Emergency Responder displays the settings for the server.

Step 3 Scroll down to the debug package and trace package sections. Select the packages that Cisco Technical Support has requested. The lists in each section are identical; make sure that you select the package in the list Cisco requested. Packages selected in the Debug list generate trace information plus extra debug data. If Cisco request you select all packages, click Select All for the appropriate list.

The available packages include:

CER_DIRECTORY—The directory update subsystem, used for updating the LDAP directories.

CER_TELEPHONY—The telephony subsystem, used for interactions with Cisco CallManager.

CER_GROUP—The CER server group subsystem, used for communicating between servers within a group.

CER_SYSADMIN—The system administration web interface subsystem.

CER_TRACKING—The phone tracking subsystem, which runs the phone tracking and switch-port and phone update processes.

CER_CALLENGINE—The call engine subsystem, which routes and processes calls.

CER_REMOTEUPDATE—The remote update subsystem, which manages updates between servers.

CER_ONSITEALERT—The onsite alert subsystem for notifying onsite alert personnel.

CER_CLUSTER—The server cluster subsystem, used for communicating between CER groups in a cluster.

Step 4 Click Update to save and activate your changes.

Emergency Responder begins generating the requested trace and debug information.

The information is placed in a log file in the /logs subdirectory of the Emergency Responder installation directory. Or, if you configured Emergency Responder to use the CiscoWorks 2000 Syslog facility, the data is sent to syslog. Send this information to the Cisco Technical Support group with which you are working. See the "Collecting System Logs with Syslog" section for information on using syslog.

Step 5 When you have finished generating debug and trace information, turn off debug and trace by clicking Clear All for each section in which you have made a selection. Then, click Update to complete the change.


Related Topics

Server Settings (for CER Group)

Viewing Windows Event Messages

You can view CER messages in the Windows Event Viewer to help diagnose problems with the software. In Event Viewer, look for messages from the source "CiscoER."

See the Microsoft documentation for information on using Event Viewer. Event Viewer is an administration tool.

Managing Performance

Emergency Responder stores most configuration information in the Cisco CallManager database (directory). The size of the configuration can affect the performance of the product. Specifically, these things can hurt system performance:

A large number of manually-defined phones—Replace the phones with models that CER supports for automatic tracking (either CDP- or CAM-based). See the "Network Hardware and Software Requirements" section for the supported phone models.

A large number of unlocated phones—See the "Too Many Unlocated Phones" section for tips on resolving these issues.

A large number of ERLs—The "Determining the Required Number of CER Groups" section describes the maximum number of ERLs a CER group can support without affecting performance. Try to keep the number of ERLs within this range.

In addition to these directory-performance issues, CER performance can be affected if CER is managing switches across a WAN link. CER must send SNMP requests to the managed switches, and WAN delays can lead to SNMP timeouts and increase the time needed to track phone and switch changes. You might need to tune the SNMP parameters. See the "Configuring the SNMP Connection" section for more information.

Integrating with Network Management Systems

You can manage the status of the CER server remotely using CiscoWorks2000 or another SNMP-based network management system. CiscoWorks2000 is the standard Cisco network management system, but it is not bundled with Emergency Responder. For more information about CiscoWorks2000, Campus Manager, and Topology Services, refer to the documentation, available at the following URL:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/cw2000/index.htm

These topics provide information to assist you in integrating Emergency Responder with network management systems:

Understanding CDP Support

Monitoring Emergency Responder Subsystem Status

Collecting System Logs with Syslog

Understanding CDP Support

Emergency Responder uses the Cisco Discovery Protocol (CDP) to periodically send out CDP messages, on the active interface, to a designated multicast address. These messages contain information such as device identification, interface name, system capabilities, SNMP agent address, and time-to-live. Any Cisco device with CDP support can locate an Emergency Responder server by listening to these periodic messages.

Using information provided through CDP, the CiscoWorks2000 Server can detect the CER server, and the Campus Manager application, Topology Services, can build topology maps displaying the CER server.

In addition to sending out CDP messages, the CER server uses CDP to locate phones that support CDP. You must ensure CDP is enabled on your switches so that CER can obtain this information through SNMP queries to the switches.

Monitoring Emergency Responder Subsystem Status

Emergency Responder supports the SYSAPPL-MIB that allows you to use CiscoWorks2000 or a third-party SNMP browser to remotely access information about the following Emergency Responder components:

Cisco Emergency Responder Server

CERServer.exe

Cisco Emergency Responder Web Administration

CERAdminServer.exe

The SYSAPPL-MIB uses the Simple Network Management Protocol (SNMP). Emergency Responder supports the following SYSAPPL-MIB tables:

SysApplInstallPkgTable—provides installed application information such as Manufacturer, Product Name, Version installed, Date installed, and Location, which is a partial URL for accessing the associated Application Administration web page (when applicable).

SysApplRunTable—describes the application starting time and run-time status.

SysApplInstallElmtTable—describes the individual application elements, or associated executables, which comprise the applications defined in the SysApplInstallPkgTable.

SysApplElmtRunTable—describes the processes, or executables, that are currently running on the host system.

Collecting System Logs with Syslog

You can configure Emergency Responder to use the Cisco Syslog Collector. Cisco Syslog Collector and Cisco Syslog Analyzer are offered with CiscoWorks2000 as part of the Resource Management Essentials package. You can also adapt Syslog output from Emergency Responder for use with other network management systems.

The Cisco Syslog Collector keeps common system logs of messages reported to Emergency Responder.

The Cisco Syslog Analyzer controls and displays all events efficiently so they can easily be read, interpreted, and used for system maintenance and problem solving.

Procedure


Step 1 Select CER Groups > CER Group Settings.

Emergency Responder opens the CER Group Settings page.

Step 2 Select enable in Enable Syslog.

Step 3 Enter the fully-qualified DNS name of the CiscoWorks2000 server in Syslog Server, for example, server.domain.com.

Step 4 Click Update Settings to save your changes.

Emergency Responder immediately begins writing messages to syslog.


Related Topics

CER Group Settings

Backing Up and Recovering Data

Most CER configuration data is stored in the Cisco CallManager database (LDAP directory). If you already have backup procedures for this database, the CER configuration is protected.

However, some configuration or log information is stored on the CER servers. Ensure that you add these files and locations to your network backup procedures. If you must restore your CER configuration, first restore the LDAP information (if necessary), then restore these files:

/callHistory directory—This contains the raw call history logs. The logs are not consolidated between a primary and standby server: backup and recover these logs independently.

Export and import files—If you created export or import files, include the directories used in your backup. You might be able to restore a lost configuration using your import files.

/location/location.txt—The phone location information associated to switch ports. If you made changes to port location information immediately before the primary server went down, the copy of this file on the backup server might not have your latest changes.

/nena_msag_record directory—Contains the MSAG files.

Related Topics

Collecting Call History Logs