User Guide for CiscoWorks QoS Policy Manager 3.0
Troubleshooting QPM
Download this chapter
Troubleshooting QPMTroubleshooting QPM
Download the complete book
PDF version of the User Guide for QoS Policy Manager 3.0PDF version of the User Guide for QoS Policy Manager 3.0 (PDF - 6 MB)
give_us_feedback

Table of Contents

Troubleshooting QPM

Troubleshooting QPM

The following topics can help you troubleshoot problems you might encounter when using QPM:

Obtaining System Status Information for Troubleshooting

If unusual exceptions occur or error windows are displayed while running QPM, you can obtain system status information by running the QPM Diagnostic Tool on the QPM server. This tool generates a report in a browser window of the system status with its diagnostics, and suggests possible solutions where applicable.

You can access the Diagnostic Tool only from the QPM server.

Note   In general, if you come across troubleshooting problems that do not have an obvious solution, you should try logging out of QPM and CiscoWorks, and restarting the QPM server. If the problem persists, run the QPM Diagnostic Tool on the QPM server to create the troubleshooting log file, and consult your system administrator.

If you want to send the diagnostics results to a TAC representative, you can run the MDCSupport.exe command-line utility, which collects configuration and system information in a zip file, called MDCSupportInformation.zip. This zip file includes any problems that occurred during the installation or the running of QPM. You can send this file to the Cisco Technical Assistance Center (TAC) support staff to assist in diagnosing the problems.

Procedure

Step 1   On the QPM server, select Start > Programs > Cisco Systems > QoS Policy Manager > Diagnostic Tool.

A report is generated and displayed in a browser window for you to view.

Step 2   To send the diagnostics results to a TAC representative:

   a. At the command line, enter MDCSupport.exe and press Enter.

A zip file named MDCSupportInformation.zip is created under c:\Program Files\CSCOpx\MDC\etc.

   b. Send this file to the TAC representative by email.

Problems Starting QoS Policy Manager

The following topics describe causes and solutions for problems you might encounter when trying to start QoS Policy Manager:

Troubleshooting Problems Starting Common Services

Common Services might not start for any of the following reasons:

Changing Windows Account

Problem—If you install Common Services and QPM using a specific admin account/password everything works as planned. However, if you change the password to this Windows account then installed Services fail to start.

Recommended Action—Change the password for all services to match the current password of the account which they were installed. Common Services services include Tomcat, fms, lm, and da-framework.

Port Conflict

Problem—You cannot start Common Services because port 1741, which is used by Common Services is in use by another application.

Recommended Action—Try the following:

  • Restart the QPM server.

  • To run CiscoWorks, enter http://<QPMinstall>:1741/login.html, where <QPMinstall> is the name or IP address of the QPM server.

Troubleshooting Problems Starting QPM

QPM might not start for any of the following reasons:

Changed Database Password

Problem—If you changed the QPM database password, and then try to start QPM without restarting the QPM server, the connection to the database is lost.

Recommended Action—Restart the QPM server after changing the QPM database password.

Old Version of Java Plug-In

Problem—QPM might not start if there is a older version of the Java plug-in, than required by QPM.

Recommended Action—Uninstall the old java plug-in.When you start CiscoWorks it automatically installs the java plug-in.

Unknown Cause

Recommended Action—Restart the QPM server.

Troubleshooting User Interface Problems

This topic describes causes and solutions for your user interface problems:

Many buttons in the user interface are grayed out

Problem—You might not have the correct user permissions to perform the tasks associated with the grayed out buttons.

Recommended Action—Verify your user permissions in the CiscoWorks2000 desktop (Server Configuration > Setup > Security), or in ACS (depending on the method you are using for user authentication). For more information about user permissions, and working with ACS user permissions, see the Installation Guide for QoS Policy Manager 3.0.

Options for QoS features do not appear in the user interface

Problem—You cannot see options for QoS features that are not valid for the defined device constraints.

Explanation—When you define policy groups and policies, QPM presents you with only those QoS properties, and policy options that are valid for the defined device constraints.

Recommended Action—Verify the defined device constraints. For information about the devices and software releases that QPM supports, and the QoS features you can configure on the supported platforms, see the device support tables at the following URL:

http://www.cisco.com/univercd/cc/td/doc/product/rtrmgmt/qos/qpm3_0/
qpm30dev/index.htm

Troubleshooting Device Management Problems

The following sections help you troubleshoot problems with QPM device management:

QPM cannot log into a device

Problem—QPM can connect to a device, but cannot log into it.

Recommended Action—Verify that the device access parameters in the QPM inventory match those configured on the device. If the device access parameters are correct but QPM still cannot log into the device, enable the blind login option, which causes QPM to send login information to the device (including access parameters) without waiting for or evaluating return prompts from the device. For instructions for enabling blind login, see Viewing and Editing Device Properties. You can also configure blind login as the default for all devices in a device group; see Editing Device Group Properties.

A device is not added to the expected device group when imported into QPM

Problem—The device might be defined in ACS as an AAA server and an AAA client, which is supported in ACS but not in QPM. In this case, the device is added to only the QPM AAA client device group in QPM.

Recommended Action—Remove the device from the ACS AAA server device group.

A device has a status that indicates an error

Problem—The device statuses described in Table 11-1 indicate that the device is in an error state. You cannot deploy to devices with these statuses.


Table 11-1: Device Status Errors
Device Status Description Explanation

Unreachable

The QPM server cannot establish basic network connectivity to the device.

Establish basic network connectivity to the device.

SNMP Error

The device has an SNMP error that is preventing QPM from gathering the data it needs to work with the device.

These are the common causes:

  • The device public community string entered in QPM is incorrect. Correct the community string in QPM.

  • QPM can't read all of the necessary SNMP information from the device, possibly because there are corrupted or missing MIBs.

  • The device does not have a functioning SNMP engine.

  • The SNMP request timed out, typically because the device or network was too congested to respond before the timeout limit. The possible resolutions are to retry the SNMP connection, or increase the SNMP timeout value.

Telnet Error

QPM cannot connect to the device using Telnet.

These are the common causes:

  • The device Telnet password entered in QPM is incorrect. Correct the Telnet password in QPM.

  • SSH is enabled but SSH login failed because SSH is not configured correctly on the device. Fix the SSH configuration on the device.

  • The login to the device failed.

  • There is no Telnet connection to the device.


Troubleshooting Deployment Problems

Policies can only be deployed to devices if QPM can correctly contact and connect to the device. QPM can usually provide a specific error message that clearly indicates the problem. However, sometimes providing a clear error message is not possible.

If QPM cannot configure a device, and the error is not clear in the device log for the device, this might be due to any of the following reasons:

Incorrect device read community string

Problem—An SNMP error is returned due to an incorrect read community string. This device access parameter is required by QPM to log into and configure the device.

Recommended Action—Check in the Access Parameters area of the Device Properties page that the device's read community string is correct.

Incorrect software version or device type

Problem—QPM does not support an imported device's Cisco IOS version. The device is assigned the status "Unsupported", and no mapped OS version is assigned to it. You cannot perform any tasks on a device that has been assigned this status.

Recommended Action—If the device model is supported by QPM but the Cisco IOS version is not, you can upgrade the device to a supported Cisco IOS version, and then rediscover the device to make it available in QPM. Click Rediscover in the Device Table to have QPM query the device for the correct information.

Deployment fails because QPM could not connect to a device

There are several possible explanations for this problem:

  • Problem—Unable to log into the device due to an incorrect access passwords.

Recommended Action—Check in the Device Properties page that the correct Telnet, TACACS, and Enable passwords are defined. QPM does not display the passwords (to maintain security), so carefully retype the passwords.

  • Problem—There is no connectivity between QPM and the device—the device status is "Unreachable".

Recommended Action—Check the device to ensure that it is powered on and that it is connected to the network. Login to the server and telnet to the device to see if you can connect to it (from the Device Properties page).

  • Problem—There might be a problem with the login prompts returned by the device.

Recommended Action—Enable the Blind Login check box in the Access Parameters area of the Device Properties page. This feature lets you bypass the login prompts (such as, username and password) that wait for a response when the device returns a string. Then try to log in again.

In the system configuration file of the QPM installation directory (CSCOpx\MDC\share\config\system.cfg), you can configure blind login for all devices, by setting devcomm.blind_login=on. To use blind login for a specific device, set <hostname>.blind_login=On,where hostname is the IP address or host name of the device. This will override the global blind login setting. If required, you can also increase the login timeout delay parameter for the devices (devcomm/hostname.blind_login_timeout).

Configuration messages from the device indicate that distributed Cisco Express Forwarding (dCEF) is not configured on VIP cards

Problem—Some types of QoS configurations require that dCEF is configured on VIP cards and QPM cannot detect this.

Recommended Action—Before distributing policies to VIP interfaces, configure dCEF on the device's VIP card interfaces through the device's commands.

Configuration messages from the device indicate that Cisco Express Forwarding (CEF) is not configured on the device

Problem—Some QoS features, such as DWFQ and NBAR, require CEF to be configured on the devices (not on the VIP cards) and QPM cannot detect this.

Recommended Action—Before distributing policies to devices, configure CEF on the device through the device's commands.

Related Topics

Troubleshooting QoS Analysis Problems

The following sections help you troubleshoot problems with QoS analysis:

Disk Space Shortage Problems

Cannot perform monitoring; a Disk Space Shortage message appears

Problem—You do not have enough disk space on the QPM server because you have already performed monitoring tasks, and the database is full.

Recommended Action—Follow the instructions in the Disk Space Shortage message to free disk space by rebuilding the QPM database. See Freeing Disk Space for QoS Analysis for details.

Note
  • Occasionally, you might not have enough space in the QPM database for monitoring because the percentage of free disk space defined during installation is too high.
Your system administrator can change the percentage of free disk space defined in the installation procedure:

a. On the QPM server, locate the qpm.cfg file under mdc\qpm\config in the Common Services installation folder.

b. Open qpm.cfg in WordPad or other text editor.

c. In the command line, database.trigger.percentage=0, change the 0 to the percentage of free disk space required.

d. Save and close the file.

e. Reboot the QPM server. A new file, qpm.local.cfg, has been created.

f. Open qpm.local.cfg.

The original command line, database.trigger.percentage=0, reappears, and a new command line appears, database.trigger.percentage.actual=<%>, where % is the percentage you defined.




Monitoring Error Messages

The following sections describe monitoring error messages and their fixes or workarounds:

Error Message: Monitoring system error

Problem—This error message indicates that an unspecified error occurred when attempting to run a report.

Recommended Action—Restarting the QPM server might resolve the problem.

Error Message: Collection service error

Problem—This error message indicates that there was a collection service error when trying to run a real-time report or complete the definition of a historical analysis task. The error message contains additional explanation of the error. Each variation of the error is described in Table 11-2.


Table 11-2: Collection Service Error Troubleshooting
Error Variation Description Fix or Workaround

Connection to collector refused

The QPM collector service is not working properly.

Restarting the QPM Collector service might resolve the problem.

Failed to load task to the collector

The QPM collector service is not working properly.

Restarting the QPM Collector service might resolve the problem.

Failed to get settings from the collector

The QPM collector service is not working properly.

Restarting the QPM Collector service might resolve the problem.

Failed to get data from the collector

The QPM collector service is not working properly.

Restarting the QPM Collector service might resolve the problem.

SNMP request timeout

SNMP requests sent to the device timed out.

Try the following fixes in this order:

1. Verify basic device connectivity using ping or Telnet.

2. Verify that the device access parameters stored in QPM inventory agree with those configured on the device.

3. Check device CPU load. If it's abnormally high, SNMP requests might not get processed. Lower the CPU load to restore monitoring.

The device: <IP address> is not configured with the policy: "<Policy>" on interface with MIB index y

The device does not contain the policy or MIB index that QPM needs to monitor a policy.

This happens when the device is changed in certain ways after the monitoring task was created.

These are possible causes of this error. To resolve the error, fix the applicable problem, then either rerun the real-time analysis report, or redefine the historical analysis task.

  • The interface is shut down. Enable the interface.

  • There was an undetected deployment error. Redeploying the policies to the device might resolve the problem.

  • You deployed policies to a file rather than to the network. QPM cannot detect that policies are deployed to a file, so it updates monitoring tasks that are affected by a policy deployment, even if the changes were not deployed to the actual device. There is no workaround.

  • You changed the device hardware configuration or created subinterfaces on the device, but did not rediscover the device. Rediscover the device.

  • You selected a CAR policy with the Mark and Transmit option configured. QPM cannot monitor this QoS configuration. Remove the CAR policy with the Mark and Transmit option from the monitoring task definition.


Error Message: Cannot run task

Problem—The device is not responding because the QPM SNMP timeout was changed to larger than 2 minutes.

Note   It is recommended that you use the default QPM SNMP timeout value.

Recommended Action—Try the following fixes in this order:

1. Verify basic device connectivity using ping or Telnet.

2. Verify that the device access parameters stored in QPM inventory agree with those configured on the device.

3. Check device CPU load. If it's abnormally high, SNMP requests might not get processed. Lower the CPU load to restore monitoring.

Error Message: The task does not contain a device from the current device group

Problem—The analysis report cannot run because none of the devices that it is assigned to monitor are still in the current device group. The likely cause is that the devices were removed from the current device group. This error will also occur if you move a device that is being monitored from the Default device group to another device group.

If any of the devices assigned to the monitoring task are still in the current device group, you can run the report, which will contain data only for the devices that are still in the current device group.

Recommended Action—To fix this, add at least one of the devices that are assigned to the monitoring task back into the current device group.

Troubleshooting Display Problems

The following sections describe display problems and their fixes or workarounds:

Graphs Do Not Appear in Analysis Reports

Problem—The browser's missing graphic icon appears where graphs should appear in analysis reports. The likely cause is that the Common Services session has expired.

Recommended Action—Log out of QPM and log back into Common Services using the Login dialog box in the desktop.

Subinterface Graphs Display Incorrect Bandwidth Percentages

Problem—When a subinterface is displayed in monitoring graphs with the units of display set to percentage, the graph will display incorrect values for the subinterface in the following cases:

  • The subinterface has CIR or Minimum CIR configured on it and the value assigned to these Qos features is different than the actual rate configured on the parent interface of the subinterface.

  • The bandwidth of the parent interface was changed on the device but the device was not rediscovered by QPM.

Recommended Action—To fix this, rediscover the device. See Rediscovering Device Information.

Troubleshooting the IP Telephony Network

The following topics help you troubleshoot problems with the IP telephony network:

Deployment fails because interface does not support "mls qos vlan-based" command

Problem—This occurs while trying to configure VLAN-based QoS on a Catalyst 6000 device running IOS. The reason it occurs is because you must first configure the switchport command on the interface to configure the VLAN-based QoS command.

Recommended Action—Configure the switchport command (switchport<cr>) manually on the interface before configuring VLAN-based QoS (mls qos vlan-based).

Shaping interval is 0 milliseconds. Intervals below 4 milliseconds rejected

Problem—This message displays while trying to deploy policies to a Frame Relay interface on a VIP card. The reason it occurs is because Modular shaping on VIP cards requires the BC/CIR interval to be in units of 4 milliseconds.

Recommended Action—Adjust the shaping parameters according to the link speed. If you have several rates, you can create a group for each rate. Note that the IP Telephony wizard will assign the interfaces to only one of the groups, and you will have to move the others manually.

Deployment fails for a serial PPP interface

Problem—After assigning a serial Point-to-Point interface with a rate equal to or less than 768kbs, this error occurs on deployment. The reason it occurs is that serial Point-to-Point cannot support LFI, which is applied on interfaces with link speeds below 768kbs.

Recommended Action—Select a multilink interface instead of a serial one. The following provides an example of a Multilink configuration:

  • interface Multilink20

    • bandwidth 256

    • ppp multilink

    • multilink-group 20

  • interface Serial5/2

    • bandwidth 256

    • encapsulation ppp

    • multilink-group 20

Selected interfaces were not assigned

Problem—After selecting an interface in a configuration step of the IP Telephony wizard, it was not assigned to any voice policy group. It appears in the "Selected but not assigned" counter. The reason could be that the interface has no policies for the current voice role.

Recommended Action—There is no recommended action since this is expected behavior for an interface with no policies for the current voice role. See Assignment Summary for more details about interface assignments.

Troubleshooting Database Backup Problems

This topic describes causes and solutions for problems you might encounter when trying to backup the QPM database:

Scheduled Incremental Backup Fails

Problem—If the time or date was changed on the QPM server, and the server was not restarted, the scheduled incremental backups following the time/date change fail.

Recommended Action—Restart the QPM server.