-
Cisco CDA Visual Quality Experience Application User Guide, Release 2.1
-
Preface
-
Introduction to Cisco CDA Visual Quality Experience Application
-
Getting Started with the VQE Startup Configuration Utility
-
Using the VQE Channel Provisioning Tool
-
Using the VQE-S Application Monitoring Tool
-
Troubleshooting VQE Software Components
-
Configuring VQE-S
-
Using Net-SNMP
-
VQE System Messages
-
Manual Initial VQE System Configuration
-
Configuring DHCP and DNS Servers for VCDS
-
Feedback
Table Of Contents
Troubleshooting VQE Software Components
VQE Server Logging and Log Files
VQE-S AMT and VCPT Logging and Log Files
VQE Files, Directories, and Permissions
VQE Server, VQE-S AMT, and VCPT Troubleshooting
Monitoring Received Multicast Streams
Repeated Message on CDE110 Hosting VQE-S or VCPT
VQE-S AMT and VCPT Unavailable
Channels Are Not Displayed in VQE-S AMT
VQE-S AMT Channels Status Summary Has No Graphs
Internal Server Error When VCPT Tries to Send Channel Information to VQE-S
Troubleshooting VQE Software Components
This chapter describes how to identify and remedy problems related to Cisco VQE Server (VQE-S), VQE-S Application Monitoring Tool (VQE-S AMT or AMT), and VQE Channel Provisioning Tool (VCPT). This chapter contains the following major topics:
•
VQE Files, Directories, and Permissions
•
Useful Linux Tools
Table 5-1 provides information on some Linux commands that are particularly useful for troubleshooting VQE software components.
VQE Logging and Log Files
VQE-related log entries can provide useful information for troubleshooting. The VQE-related log files are described in these sections:
•
•
VQE Server Logging and Log Files
VQE Server logging is based on the Berkeley syslogd utility, which is part of the inetutils package in Linux. The following is a typical VQE-S log entry:
Oct 22 08:42:03 minnie vqes_mlb: <<%VQES_MLB-3-MLB_NIC_DOWN>> Network interface eth3 went down.Every logged message contains five fields in this order:
•
•
•
•
•
For information on VQE system messages, see Appendix C, "VQE System Messages."
By default, VQE-S syslog messages are written to the file /var/log/vqes/vqes.log. The location of the file is configured in the /etc/syslog.conf file. For information on configuring VQE-S logging, see the "Configuring VQE-S Logging" section on page 4-15.
VQE-S log files are rotated by logrotate. The logrotate facility is configured to rotate a log file when it is larger than 20MB. The old versions of the log files are rotated up to 20 times before being removed.
Note
VQE-S AMT and VCPT Logging and Log Files
VQE-S AMT and VCPT are both web applications that make use of the Apache Tomcat application server. Apache Tomcat uses the log4j logging system. For AMT and VCPT, the logging output is configured by these log4j configuration files:
•
•
By default, the log files for AMT and VCPT are saved in these locations:
•
•
Table 5-2 lists the most important default log4j configuration settings for AMT and VCPT logging.
For information on configuring the log4j.properties file, see the log4j documentation at these URLs:
http://tomcat.apache.org/tomcat-5.5-doc/logging.html
http://logging.apache.org/log4j/1.2/manual.html
VQE Files, Directories, and Permissions
If you encounter a problem related to VQE-related files, directories, or permissions, use the information in the following tables to resolve the issues.
•
•
These sections provide additional information on resolving problems:
Changing Permissions
To change permissions on a file or directory, use the chmod command:
Step 1
Step 2
chmod mode file_or_dir_name
In the preceding command, the two arguments are as follows:
•
•
For example:
[root@system]# chmod 555 /opt/vqes/bin/vqes_cp[root@system]# chmod 640 /opt/vqes/bin/vqes.confFor information on setting permissions, see the man page for the chmod command.
Replacing a Lost VQE File
Two Red Hat Package Manager (RPM) packages containing the VQE software files can be obtained by registered Cisco.com users:
•
•
To replace a lost VQE file, follow these steps:
Step 1
Step 2
http://www.cisco.com/public/sw-center/content-delivery/cda.shtml
Step 3
•
•
For example: rpm2cpio< vqes-2.1.1-1.i386.rpm | cpio -id ./opt/vqes/bin/vqes_cp
Step 4
chown owner.group filename
For example: chown vqes.vqes /opt/vqes/bin/vqes_cp
Step 5
VQE Server, VQE-S AMT, and VCPT Troubleshooting
This section provides information on troubleshooting the following specific problems that can occur with VQE Server, VQE-S AMT, and VCPT.
•
•
•
•
•
•
Monitoring Received Multicast Streams
For troubleshooting VQE-S, using VQE-S AMT to monitor received multicast streams is recommended. Because there are normal cases where VQE-S will intentionally drop packets from its socket buffers, using Linux commands to monitor multicast streams can be misleading.
We recommend that you use AMT to monitor received multicast streams. In the AMT Channel Statistics window for each channel (Figure 5-1), the Lost RTP Packets field for the Channel Input Stream is a per-channel counter that provides the number of missing RTP packets. The Lost RTP Packets counter increments when an RTP packet is sent for the channel's original source stream but is not received by the VQE Server.
Figure 5-1 Lost RTP Packets
Repeated Message on CDE110 Hosting VQE-S or VCPT
Symptom: During initial system configuration, a new Cisco CDE110 hosting VQE-S or VCPT boots successfully, the root password is set successfully, but the following message displays repeatedly:
SIOCGIFADDR: Cannot assign requested addressSIOCGIFADDR: Cannot assign requested addressSIOCGIFADDR: Cannot assign requested addressSIOCGIFADDR: Cannot assign requested addressSIOCGIFADDR: Cannot assign requested addressSIOCGIFADDR: Cannot assign requested addressExplanation: This error can occur when you configure the CDE110 appliance manually (rather than use the VQE Startup Configuration Utility). The message may display so frequently that it is difficult to edit a file to continue the initial system configuration.
The message can be caused by an Ethernet interface configuration issue in the ifcfg-eth# files (ifcfg-eth1, ifcfg-eth2, and so on).
Remedy: Look for Ethernet interface address or other errors in the four ifcfg-eth# files. Use a text editor to correct any errors.
Use the following command to verify that the addresses specified in the ifcfg-eth# files have been assigned to the interfaces:
ifconfig -a
Continue with the initial configuration tasks.
VQE-S AMT and VCPT Unavailable
Symptom: To browse to VQE-S AMT or VCPT, you enter a valid IP address or fully qualified domain name, but the browser displays the message "Page Cannot Be Found"(status code 404) or "Service Temporarily Unavailable" (status code 503).
Explanation: Both AMT and VCPT require the Apache HTTP server service (httpd) and the Apache Tomcat application server service (tomcat5) to be running. When you attempt to browse to AMT or VCPT and one of these services is not running, the status codes are typically as follows:
•
•
To verify that httpd is not running, log in as root and issue following command:
[root@system]# ps -ef | grep httpdIf httpd is running, the output is similar to the following. If httpd is not running, this output will not be present.
[root@system]# ps -ef | grep httpdroot 2880 1 0 Jul18 ? 00:00:00 /usr/sbin/httpdapache 4881 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4882 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4883 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4884 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4885 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4886 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4887 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdapache 4888 2880 0 04:03 ? 00:00:00 /usr/sbin/httpdTo verify that tomcat5 is running, log in as root and issue following command:
[root@system]# ps -ef | grep tomcat5If tomcat5 is running, the output is similar to the following. If tomcat5 is not running, this output will not be present.
root 2915 1 0 Jul18 ? 00:00:11 /usr/java/default/bin/java -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=/usr/share/tomcat5/conf/logging.properties -Djava.endorsed.dirs=/usr/share/tomcat5/common/endorsed -classpath :/usr/share/tomcat5/bin/bootstrap.jar:/usr/share/tomcat5/bin/commons-logging-api.jar -Dcatalina.base=/usr/share/tomcat5 -Dcatalina.home=/usr/share/tomcat5 -Djava.io.tmpdir=/usr/share/tomcat5/temp org.apache.catalina.startup.Bootstrap startRemedy: If either httpd or tomcat5 is not running, log in as root and start the services that are stopped as follows:
[root@system]# service httpd restart[root@system]# service tomcat5 restartVerify that the httpd and tomcat5 services are running as shown earlier in this Explanation.
If the httpd or tomcat5 processes continue to fail, check the log files for the services to determine the cause. The tomcat5 log files are in /var/log/tomcat5/catalina.date.log. For information on the log files for each service, see the vendor documentation for the Apache HTTP server or for the Apache Tomcat application server.
Channels Are Not Displayed in VQE-S AMT
Symptom: After channels are defined in VCPT and channel information is pushed to servers, VQE-S AMT does not display the channels.
Explanation: The most likely reason why AMT does not display the channels is that the VQE Server was not associated with the channels in VCPT.
To determine whether the channels are associated with this VQE Server, follow these steps:
Step 1
Step 2
Step 3
The channels should be in the Selected group. Also, make sure the VQE Server is defined with the role VQE-S. If the role is wrong, correct it with VCPT from the Servers tab. For information on updating server information, see the "Viewing or Updating Server Information" section on page 3-14.
If the channels are correctly associated with the VQE Server, it is possible that there were problems when VQE Server attempted to process the channels. Check the VQE Server log files for errors in channel processing. For information on these log files, see the "VQE Server Logging and Log Files" section.
Remedy: If the channels have not been associated with the VQE Server in VCPT, associate the channels with VCPT from the Association tab. For information on associating channels with servers, see the "Defining Channel Associations for a Server" section on page 3-15.
VQE-S AMT Channels Status Summary Has No Graphs
Symptom: In the AMT VQE-S Status window (Figure 4-2), the Channels Status Summary does not have graphs for the channels.
Explanation: If one or more channels are successfully created in AMT, the channels should appear in a Channels Status Summary graph of active, inoperative, and inactive channels in the VQE-S Status window. To be displayed, the graphs require that Adobe Flash Player be installed on the computer that hosts the browser accessing AMT.
Remedy: Download and install Adobe Flash Player on the computer that hosts the browser used to access AMT. The software is free and can be found at this URL:
http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash
It is recommended that you close all other open browser windows before performing the installation.
Internal Server Error When VCPT Tries to Send Channel Information to VQE-S
Symptom: When VCPT tries to send channel information to VQE Server, you get an "Internal Server Error." Java exceptions in the catalina.out file indicate that there is an address conflict or unreachable address.
Explanation: The Internal Server Error is displayed when you use VCPT to send channel information to a VQE Server. The Status of Last column on the VCPT Servers tab shows the error.
From the CDE110 that hosts VQE-S, you are able to successfully ping the machine running the web browser used for VQE-S AMT, and ping the CDE110 hosting VCPT.
In the /usr/share/tomcat5/logs/catalina.out file of the Apache Tomcat application server, there are Java exceptions indicating an address conflict or unreachable address. For example:
java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(Unknown Source) ...java.net.SocketException: Network is unreachable at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(Unknown Source)Remedy: The Internal Server Error can be caused by one or more errors in the /etc/hosts file on the CDE110 that hosts VQE-S. Check the /etc/hosts file for typing or syntax errors, correct them, and save the file. Reboot the CDE110.
If you still receive the same Internal Server Error and Java exceptions, check the following files to see if you can find some indication for what is causing the error.
•
•
•
•
•
Using the vqereport Command
The vqereport command can be useful for VQE troubleshooting. You can use the vqereport command to gather information on the VQE system's software configuration and hardware. The information can be used by the VQE user or Cisco technical support personnel to diagnose problems with VQE software or with the Cisco Content Delivery Engine 110 (CDE110) hardware.
Caution
The syntax for the vqereport command is as follows:
vqereport [-h | -help]
Syntax Description
Usage Guidelines
The vqereport command generates a report file that, when the command finishes, is located in /root/hostname.dateandtime.tar.bz2. If appropriate, the report file can be attached to field issue reports for Cisco technical support.
Note
The vqereport command can be executed on the CDE110 that hosts VQE Server or on the CDE110 that hosts VCPT. The vqereport executable is located at /opt/vqes/bin/vqereport.
To have the operating system find the vqereport executable, you must set the PATH environment variable to include the location of the /opt/vqes/bin directory, or use the full path to vqereport when executing the command.
All information gathered is considered confidential, and Cisco will use this information for diagnostic purposes only.
Examples
The following example shows the execution of the vqereport command and some abbreviated output.
[root@system ~]# /opt/vqes/bin/vqereportThis utility will go through and collect some detailed information about the hardware and setup of your VQE system. This information will be used to diagnose problems with your system and will be considered confidential information. Cisco will use this information for diagnostic purposes ONLY.Please wait while we collect information about your system.This process may take a while to complete.... No changes will be made to your system during this process. NOTE: You can safely ignore a failed message. This only means a file we were checking for did not exist.Press ENTER to continue, or CTRL-C to quit.********************************************************************** Collect Redhat sysreport********************************************************************** Collect information of installed RPM packages...output omitted...Report file /root/<hostname>.20071115164635.tar.bz2 has been generated, please send it to Cisco support.
