Cisco CDA Visual Quality Experience Application User Guide, Release 3.1 - Chapter 5: Troubleshooting VQE Software Components [Cisco Videoscape Distribution Suite for Internet Streaming]

Table Of Contents

Troubleshooting VQE Software Components

Useful Linux Tools

VQE Logging and Log Files

VQE Server and VQE Tools Logging and Log Files

VQE-S AMT and VCPT Logging and Log Files

VQE Files, Directories, and Permissions

Changing Permissions

Replacing a Lost or Corrupt VQE File

VQE-S and VQE Tools Server Troubleshooting

Monitoring Received Multicast Streams

Static Route for a Management Network Is Missing on CDE110 Hosting VQE-S or VQE Tools

Repeated Message on CDE110 Hosting VQE-S or VQE Tools

Changing the System Time Causes Unicast Retransmission and RCC Disruptions

Performing a Date and Time Change with NTP

Performing a Date and Time Change with the Linux date Command

VQE-S AMT or VCPT Unavailable

Channels Are Not Displayed in VQE-S AMT

VQE-S AMT Channels Status Summary Has No Graphs

Channel-Provisioning Server Cannot Send Channel Information to VQE Server:
Trusted VCPT Host(s) Problem

Channel-Provisioning Server Cannot Send Channel Information to VQE Server: SSL Certificates Problems

Unable to Connect Error When VCPT Tries to Send Channel Information

Internal Server Error When VCPT Tries to Send Channel Information to VQE-S

Using the vqereport Command

Stopping, Starting, and Restarting VQE-S

Stopping, Starting, and Restarting VCDS

Troubleshooting VQE Software Components

This chapter describes how to identify and remedy problems related to Cisco VQE Server (VQE-S), VQE-S Application Monitoring Tool (VQE-S AMT or AMT), and VQE Channel Provisioning Tool (VCPT). This chapter contains the following major topics:

•Useful Linux Tools

•VQE Logging and Log Files

•VQE Files, Directories, and Permissions

•VQE-S and VQE Tools Server Troubleshooting

•Using the vqereport Command

•Stopping, Starting, and Restarting VQE-S

•Stopping, Starting, and Restarting VCDS

Useful Linux Tools

Table 5-1 provides information on some Linux commands that are particularly useful for troubleshooting VQE software components.

Table 5-1 Useful Linux Commands

Command

Description

Notes

arp

Manipulates the kernel's ARP cache

—

ethtool

Displays or changes Ethernet card settings

Detailed status on an Ethernet interface

ifconfig

Configures a network interface

Traditional interface configuration and status

ip

Shows or manipulates routing, devices, policy routing and tunnels

Most useful interface to all Linux networking

netstat

Displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

Provides traditional network state information

ping

Sends ICMP ECHO_REQUEST to network hosts

—

tcpdump

Dumps and analyzes traffic on a network

Captures packet data

VQE Logging and Log Files

VQE-related log entries can provide useful information for troubleshooting. The VQE-related log files are described in these sections:

•VQE Server and VQE Tools Logging and Log Files

•VQE-S AMT and VCPT Logging and Log Files

VQE Server and VQE Tools Logging and Log Files

VQE Server logging is based on the Berkeley syslogd utility, which is part of the inetutils package in Linux. The following is a typical VQE-S log entry:
Oct 22 08:42:03 minnie vqes_mlb: <<%VQES_MLB-3-MLB_NIC_DOWN>> Network interface eth3 went down. 
Every logged message contains five fields in this order:

•Timestamp (Oct 22 08:42:03)

•Host name (minnie)

•Name of process logging the message (vqes_mlb)

•Identity string (<<%VQES_MLB-3-MLB_NIC_DOWN>>)

•Log message (Network interface eth3 went down.)

For information on VQE system messages, see Appendix C, "VQE System Messages."

By default, VQE-S and VQE Tools system messages are written to the file /var/log/vqe/vqe.log. The location of the file is configured in the /etc/syslog.conf file. For information on configuring VQE-S logging, see the "Configuring VQE-S Logging" section on page 4-17.

VQE-S and VQE Tools log files are rotated by logrotate. The logrotate facility is configured to rotate a log file when it is larger than 20 MB. The old versions of the log files are rotated up to 20 times before being removed.

Note If you move the VQE log file to another directory, be sure to stop the syslogd daemon before moving the vqe.log file. If syslogd is started with no vqe.log file in the /var/log/vqe directory, syslogd creates the vqe.log file.

VQE-S AMT and VCPT Logging and Log Files

VQE-S AMT and VCPT are both web applications that make use of the Apache Tomcat application server. Apache Tomcat uses the log4j logging system. For AMT and VCPT, the logging output is configured by these log4j configuration files:

•For AMT, the configuration file is located at /usr/share/tomcat5/webapps/ems/WEB-INF/classes/ log4j.properties.

•For VCPT, the configuration file is located at /usr/share/tomcat5/webapps/vcpt/WEB-INF/classes/ log4j.properties.

By default, the log files for AMT and VCPT are saved in these locations:

•For AMT, the log files (vqe.log) are saved in the /usr/share/tomcat5/logs directory.

•For VCPT, the log files (vcpt.log) are saved in /usr/share/tomcat5/logs directory.

Table 5-2 lists the most important default log4j configuration settings for AMT and VCPT logging.

Table 5-2 Log4j Default Configuration for AMT and VCPT

Configuration Property

Default Setting

Logging priority level

Warning

Logging output

Log messages are written to both the console and the log file

Maximum size of the log file before it rotates

100 KB

Maximum number of backup files

4

For information on configuring the log4j.properties file, see the log4j documentation at these URLs:

http://tomcat.apache.org/tomcat-5.5-doc/logging.html

http://logging.apache.org/log4j/1.2/manual.html

VQE Files, Directories, and Permissions

If you encounter a problem related to VQE-related files, directories, or permissions, use the information in the following tables to resolve the issues.

•Table 5-3 provides information for the CDE110 that hosts VQE-S.

•Table 5-4 provides information for the CDE110 that hosts VCPT and VQE Client Channel Configuration Delivery Server.

These sections provide additional information on resolving problems:

•Changing Permissions

•Replacing a Lost or Corrupt VQE File

Table 5-3 VQE-S Files, Directories, and Permissions

File or Directory

Required Permissions, Owner,

and Group

Description

/etc/opt/vqes (directory)

drwx------ vqes vqes

--

/etc/opt/vqes/vcdb.conf

-rw-r--r-- root root

VCDB configuration file

/etc/opt/vqes/vcdb.conf.sample

-r-------- vqes vqes

VCDB sample configuration file

/etc/opt/vqes/vqe_channels.cfg

-rw-r--r-- vqes vqes

VQE-S channel configuration file

/etc/opt/vqes/vqes_syslog.conf

-rw-r--r-- vqes vqes

VQE-S syslog configuration file

/opt/vqes (directory)

drwxr-xr-x vqes vqes

--

/opt/vqes/bin (directory)

drwxr-xr-x vqes vqes

VQE-S binary directory

/opt/vqes/bin/fbt_flush.sh

-r-xr----- root vqes

VQE-S feedback target flush script

/opt/vqes/bin/mlb

-r-xr-xr-x root root

VQE-S Multicast Load Balancer process

/opt/vqes/bin/mlb_ethtool

-r-sr-x--- root vqes

VQE-S Ethernet utility for Multicast Load Balancer

/opt/vqes/bin/process_monitor

-r-xr-xr-x root root

VQE-S Process Monitor process

/opt/vqes/bin/stun_server

-r-xr-xr-x root root

VQE-S STUN Server process

/opt/vqes/bin/vqe_cfgtool

-r-x------ vqes vqes

vqe_cfgtool command

/opt/vqes/bin/vqereport

-r-x------ root root

vqereport command

/opt/vqes/bin/vqes_cp

-r-x------ root root

VQE-S Control Plane process

/opt/vqes/bin/vqes_dp

-r-xr-xr-x root root

VQE-S Data Plane process

/opt/vqes/bin/vqes_bin.sha1sum

-r--r----- vqes vqes

SHA-1 checksums list

/opt/vqes/bin/vqes_syslogd

-r-xr-xr-x vqes vqes

VQE-S syslogd process

/opt/vqes/bin/xmlrpc

-r-xr-xr-x vqes vqes

XML-RPC client utility

/var/log/vqe (directory)

drwxr-xr-x vqes vqes

VQE log file directory

/var/log/vqe/vqe.log

-rw------- vqes vqes

VQE system messages log file

Table 5-4 VQE Tools Files, Directories, and Permissions

File or Directory

Required Permissions, Owner,

and Group

Description

/etc/opt/vqes (directory)

drwx------ root root

--

/etc/opt/vqes/vcdb.conf

-rw-r--r-- root root

VCDB configuration file

/etc/opt/vqes/vcdb.conf.sample

lrwxrwxrwx root root

A link to the VCDB sample configuration file

/etc/opt/vqes/vcdb.conf.tools.sample

-r-------- vqes vqes

VCDB sample configuration file

/etc/opt/vqes/VCDServer.cfg

-rw-r--r-- root root

VQE Client Channel Configuration Delivery Server (VCDS) configuration file

/etc/opt/vqes/vqec_channels.cfg

-rw--r--r-- root root

VCDS channel configuration file

/etc/opt/vcpt (directory)

drwxr-xr-x root root

--

/etc/opt/vcpt/data (directory)

drwxr-xr-x root root

Directory for VCPT configuration files

VCPT configuration files in /etc/opt/vcpt/data

-rw-r--r-- root root

VCPT configuration files (Filenames are user-defined and vary.)

/opt/vqes (directory)

drwxr-xr-x vqes vqes

--

/opt/vqes/bin (directory)

drwxr-xr-x vqes vqes

--

/opt/vqes/bin/vqe_cfgtool

-r-x------ vqes vqes

vqe_cfgtool command

/opt/vqes/bin/vqereport

-r-x------ root root

vqereport command

/var/log/vqe (directory)

drwxr-xr-x vqes vqes

VQE log directory

/var/log/vqe/vqe.log

-rw------- vqes vqes

VQE system messages log file

Changing Permissions

To change permissions on a file or directory, use the chmod command:

Step 1 If needed, log in as root on the CDE110 that hosts VQE-S or that hosts VCPT.

Step 2 To change permissions, issue the following command:

chmod mode file_or_dir_name

In the preceding command, the two arguments are as follows:

•mode specifies the correct permissions.

•file_or_dir_name specifies the file or directory whose permissions will be set.

For example:
[root@system]# chmod 555 /opt/vqes/bin/vqes_cp 
For information on setting permissions, see the man page for the chmod command.

Replacing a Lost or Corrupt VQE File

If a required VQE file that has not been backed up becomes lost or corrupt, you must reinstall the version of the VQE software that the CDE110 server is running. To preserve the current VQE configuration, the software reinstallation should be an ISO full upgrade. For information on VQE software installation, see the Release Notes for Cisco CDA Visual Quality Experience Application, Release 3.1.

If the lost or corrupt file that was not backed up is a configuration file located in the directories under /etc, you can get a copy of the factory default version of the configuration file from the /vqe-etc/etc-pristine directory. To restore the lost or corrupt file, copy the factory default configuration file to its correct location under /etc, and apply the current VCDB configuration to the /etc configuration files using the vqe_cfgtool command and the -apply option.

For information on the vqe_cfgtool command, see the "Using the VQE Configuration Tool Command-Line Options" section on page 6-16.

VQE-S and VQE Tools Server Troubleshooting

This section provides information on troubleshooting the following specific problems that can occur with a VQE-S or VQE Tools server. Table 5-5 lists the troubleshooting topics by the category of the problem.

Table 5-5 Troubleshooting Topics by Problem Category

Problem Category

Troubleshooting Topics

Monitoring a VQE-S server

Monitoring Received Multicast Streams

VQE-S and VQE Tools servers

Static Route for a Management Network Is Missing on CDE110 Hosting VQE-S or VQE Tools

Repeated Message on CDE110 Hosting VQE-S or VQE Tools

VQE-S server

Repeated Message on CDE110 Hosting VQE-S or VQE Tools

VQE-S AMT

VQE-S AMT or VCPT Unavailable

Channels Are Not Displayed in VQE-S AMT

VQE-S AMT Channels Status Summary Has No Graphs

VCPT or other channel-provisioning servers*

VQE-S AMT or VCPT Unavailable

Channel-Provisioning Server Cannot Send Channel Information to VQE Server: Trusted VCPT Host(s) Problem

Channel-Provisioning Server Cannot Send Channel Information to VQE Server: SSL Certificates Problems

Unable to Connect Error When VCPT Tries to Send Channel Information

Internal Server Error When VCPT Tries to Send Channel Information to VQE-S

* Some VCPT troubleshooting topics apply to other channel-provisioning servers, such as Cisco IPTV Services Delivery Server (ISDS).

Monitoring Received Multicast Streams

For troubleshooting VQE-S, using VQE-S AMT to monitor received multicast streams is recommended.

We recommend that you use AMT to monitor received multicast streams. In the AMT Channel Statistics window for each channel (Figure 5-1), the Lost RTP Packets field for the Channel Input Stream is a per-channel counter that provides the number of missing RTP packets. The Lost RTP Packets counter increments when an RTP packet is sent for the channel's original source stream but is not received by the VQE Server.

Figure 5-1 Lost RTP Packets

Static Route for a Management Network Is Missing on CDE110 Hosting VQE-S or VQE Tools

Symptom: A static route for a management network, which was configured in /etc/sysconfig/static-routes-iputil, are missing, and the VQE-S host or VQE Tools host can no longer reach the management network.

Explanation: This condition can occur when CDE110 interfaces are manually shutdown by the operator using the Linux ifdown ifname command and then manually brought back up using the ifup ifname command. This condition can also occur when the CDE110 interfaces are reset for any reason.

If the next hop routers for a static route configured in /etc/sysconfig/static-routes-iputil become unreachable due to CDE110 interfaces being manually shut down in this manner, the corresponding routes are removed from the routing table and will not be reinstalled even when one or more of the CDE110 interfaces are brought back up.

(The VQE Startup Utility and the VQE Configuration Tool can be used to configure a static route for a management network. These facilities use /etc/sysconfig/static-routes-iputil for the static route to a management network.)

Remedy: To reinstall a management-network static route that has been removed from the routing table, use the route add command:

route add -net management-network-addr/prefix-length gw gateway-addr

For example:
[root@system]# route add -net 192.0.2.0/16 gw 10.86.21.1 
In the preceding, the management-network-addr/prefix-length is the IP address and prefix length for the management network. The gateway-addr is the IP address of the router interface that is directly attached to the CDE110 Ethernet port that will be used for management network traffic.

To avoid this problem in the future, instead of using the ifup ifname command to bring each of the interfaces up individually, bring them all back up at once using the service network start command. This will bring up all interfaces that were shut down, and will also reinstall all of the routes contained in the file /etc/sysconfig/static-routes-iputil.

Repeated Message on CDE110 Hosting VQE-S or VQE Tools

Symptom: During initial system configuration, a new Cisco CDE110 hosting VQE-S or VQE Tools boots successfully, the root password is set successfully, but the following message displays repeatedly:
SIOCGIFADDR: Cannot assign requested address
SIOCGIFADDR: Cannot assign requested address
SIOCGIFADDR: Cannot assign requested address
SIOCGIFADDR: Cannot assign requested address
SIOCGIFADDR: Cannot assign requested address
SIOCGIFADDR: Cannot assign requested address
Explanation: This error can occur when you configure the CDE110 server manually (rather than use the VQE Startup Configuration Utility). The message may display so frequently that it is difficult to edit a file to continue the initial system configuration.

The message can be caused by an Ethernet interface configuration issue in the ifcfg-eth# files (ifcfg-eth1, ifcfg-eth2, and so on).

Remedy: Look for Ethernet interface address or other errors in the four ifcfg-eth# files. Use a text editor to correct any errors.

Use the following command to verify that the addresses specified in the ifcfg-eth# files have been assigned to the interfaces:

ifconfig -a

Continue with the initial configuration tasks.

Changing the System Time Causes Unicast Retransmission and RCC Disruptions

Symptom: When the system time is changed on a VQE-S server that is actively repairing network errors, all Unicast Retransmissions stop indefinitely, and output gaps are seen on the VQE Clients.

Explanation: When the system time is changed, the behavior of the VQE-S is as follows:

•When the system time is moved forward, the VQE-S receives requests for Unicast Retransmission and Rapid Channel Change (RCC) but does not send the repairs/RCCs to the VQE Clients on the set-top boxes.

•When the system time is moved backward, all channels go to an inactive state and the VQE-S does not perform Unicast Retransmission and RCC operations.

For a VQE-S server that is actively repairing network errors, an explicit system time change (that is, by using the date command) always results in the failure of Unicast Retransmission and RCC operations until corrective action is taken.

Remedy: Any time change performed on the VQE-S system should be done during a maintenance window. The procedures for changing the date and time vary depending on whether Network Time Protocol (NTP) or the Linux date command is used. See one of the following sections:

•"Performing a Date and Time Change with NTP" section

•"Performing a Date and Time Change with the Linux date Command" section

Note Using the local clock is not the recommended procedure for running with accurate time. Using NTP is recommended to keep the VQE-S services operational.

Performing a Date and Time Change with NTP

When performing a date and time change with NTP, do the following:

Step 1 Log in as root.

Step 2 Stop the VQE-S services by issuing the following command:
[root@system]# service vqes stop
Step 3 Stop the ntpd service by issuing the following command:
[root@system]# service ntpd stop 
Step 4 If needed, set the time zone with the vqe_cfgtool command's -config option. Use the Configuration Tool's System Parameters menu and the Timezone choice.

Step 5 Set the system date and time to a date and time close to the NTP server date and time by issuing the following command:

date -s "date_time_string"

For example:
[root@system]# date -s "16:55:30 July 7, 2008"
Step 6 Synchronize the clock to the configured NTP servers by issuing the following command:
[root@system]# ntpd -q
If the system clock is off by a lot, the command will take considerable time to return.

Step 7 Start the ntpd service by issuing the following command:
[root@system]# service ntpd start 
Step 8 Synchronize the hardware clock by issuing the following command:
[root@system]# /sbin/hwclock --systohc 
Step 9 Check NTP synchronization
[root@system]# ntpq -p 
Step 10 Reboot the VQE-S server by issuing the following command:
[root@system]# init 6 
Performing a Date and Time Change with the Linux date Command

When performing a time/date change with the Linux date command only, perform the following commands:

Step 1 Log in as root.

Step 2 Stop the VQE-S services by issuing the following command:
[root@system]# service vqes stop
Step 3 If needed, set the time zone with the vqe_cfgtool command's -config option. Use the Configuration Tool's System Parameters menu and the Timezone choice.

Step 4 Set the system date and time by issuing the following command:

date -s "date_time_string"

For example:
[root@system]# date -s "16:55:30 July 7, 2008"
Step 5 Synchronize the hardware clock by issuing the following command:
[root@system]# /sbin/hwclock --systohc 
Step 6 Reboot the VQE-S server by issuing the following command:
[root@system]# init 6 
VQE-S AMT or VCPT Unavailable

Symptom: To browse to VQE-S AMT or VCPT, you enter a valid IP address or fully qualified domain name, but the browser displays the message "Page Cannot Be Found" (status code 404) or "Service Temporarily Unavailable" (status code 503).

Explanation: Both AMT and VCPT require the Apache HTTP server service (httpd) and the Apache Tomcat application server service (tomcat5) to be running. When you attempt to browse to AMT or VCPT and one of these services is not running, the status codes are typically as follows:

•"Page Cannot Be Found" (status code 404) indicates that httpd is not running.

•"Service Temporarily Unavailable" (status code 503) indicates that tomcat5 is not running.

To check whether the httpd service is running, log in as root and issue following command:
[root@system]# /sbin/service httpd status
httpd (pid 15836 13179 2593 2592 2591 2590 2589 2588 2587 2586 2576) is running...
To check whether the tomcat5 service is running, log in as root and issue following command:
[root@system]# service tomcat5 status 
Tomcat is running...
Remedy: If either httpd or tomcat5 is not running, log in as root and start the services that are stopped as follows:
[root@system]# service httpd start 
[root@system]# service tomcat5 start
Verify that the httpd and tomcat5 services are running as shown earlier in the Explanation. If the httpd or tomcat5 processes continue to fail, check the log files for the services to determine the cause.

•The httpd log file is /etc/httpd/logs/error_log.

•The tomcat5 log files are /var/log/tomcat5/catalina.date.log.

For information on the log files for each service, see the vendor documentation for the Apache HTTP server or for the Apache Tomcat application server.

Channels Are Not Displayed in VQE-S AMT

Symptom: After channels are defined in VCPT and channel information is pushed to servers, VQE-S AMT does not display the channels.

Explanation: The most likely reason why AMT does not display the channels is that the VQE Server was not associated with the channels in VCPT.

To determine whether the channels are associated with this VQE Server, follow these steps:

Step 1 Log in to VCPT.

Step 2 Click the Association tab

Step 3 Select the VQE Server in the Select Server pull-down menu.

The channels should be in the Selected group. Also, make sure the VQE Server is defined with the role VQE-S. If the role is wrong, correct it with VCPT from the Servers tab. For information on updating server information, see the "Viewing or Updating Server Information" section on page 3-15.

If the channels are correctly associated with the VQE Server, it is possible that there were problems when VQE Server attempted to process the channels. Check the VQE Server log files for errors in channel processing. For information on these log files, see the "VQE Server and VQE Tools Logging and Log Files" section.

Remedy: If the channels have not been associated with the VQE Server in VCPT, associate the channels with VCPT from the Association tab. For information on associating channels with servers, see the "Defining Channel Associations for a Server" section on page 3-16.

VQE-S AMT Channels Status Summary Has No Graphs

Symptom: In the AMT VQE-S Status window (Figure 4-2 on page 4-4), the Channels Status Summary does not have graphs for the channels.

Explanation: If one or more channels are successfully created in AMT, the channels should appear in a Channels Status Summary graph of active, inoperative, and inactive channels in the VQE-S Status window. To be displayed, the graphs require that Adobe Flash Player be installed on the computer that hosts the browser accessing AMT.

Remedy: Download and install Adobe Flash Player on the computer that hosts the browser used to access AMT. The software is free and can be found at this URL:

http://www.adobe.com/shockwave/download/download.cgi?P1_Prod_Version=ShockwaveFlash

It is recommended that you close all other open browser windows before performing the installation.

Channel-Provisioning Server Cannot Send Channel Information to VQE Server:
Trusted VCPT Host(s) Problem

Symptom: The channel-provisioning server (for example, VCPT or Cisco IPTV Services Delivery Server [ISDS]) cannot send channel information to a VQE Server.

Explanation: When the channel-provisioning server attempts to send channel information to a VQE Server, the send operation fails. On the VQE-S server, the VQE-S parameter Trusted VCPT Host(s) is not configured to allow packets from the channel-provisioning server.

If VCPT is the channel-provisioning server and the send operation fails because the Trusted VCPT Host(s) parameter is not configured with the IP address of the VCPT host, VCPT displays the following in the Status of Last Send column in its VCPT Servers Summary window:
Failed - Connection refused 
In addition, the following error message is written to the VCPT log file (/usr/share/tomcat5/logs/vcpt.log):
ERROR: Unable to send the SDP File java.net.ConnectException: Connection refused
Remedy: Use the VQE Configuration Tool to configure the VQE-S parameter "Trusted VCPT Host(s)" with the IP addresses of the channel-provisioning servers. For information on using the Configuration Tool, see the "Using the VQE Configuration Tool" section on page 6-4.

Note If ISDS is the channel-provisioning server, for the Trusted VCPT Host(s), use the Broadcast System IP Address, which is configured on the ISDS VASP List. The VASP List can be found on the ISDS Administrative Console under ISDS -> Network Element Provisioning -> VASP.

Channel-Provisioning Server Cannot Send Channel Information to VQE Server: SSL Certificates Problems

Symptom: The channel-provisioning server (for example, VCPT or ISDS) cannot send channel information to a VQE Server.

Explanation: When the channel-provisioning server attempts to send channel information to a VQE Server, the send operation fails. On the VQE-S server or the channel-provisioning server or both, the Secure Sockets Layer (SSL) certificates are not valid or the needed items have not been copied to the correct locations.

If VCPT is the channel-provisioning server and the send operation fails because of SSL certificate problems, VCPT displays the following in the Status of Last Send column in its VCPT Servers Summary window:
Failed - Unable to find valid certification path to requested target 
In addition, the following error message is written to the VCPT log file (/usr/share/tomcat5/logs/vcpt.log):
ERROR: Unable to send the SDP File javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid 
certification path to requested target
Remedy: Make sure all SSL certificate-related items have been created correctly and the needed items have been copied to the correct locations on the VQE-S server and on the channel-provisioning server. For information on setting up SSL certificates, see the "Setting Up SSL Certificates" section on page 2-4.

Once the certificate-related items have been copied to the correct locations on the VQE-S server and on the channel-provisioning server, restart the httpd and tomcat5 services on both servers. For example, login as root and issue the following commands on both servers:
[root@system]# service httpd restart 
[root@system]# service tomcat5 restart
Note If the tomcat5 service is not running when you try to restart it, an exception is displayed on the console when the attempt is made to stop the service. The exception can be ignored.

Unable to Connect Error When VCPT Tries to Send Channel Information

Symptom: When VCPT tries to send channel information to a VQE Server or VCDS, the send operation fails. In VCPT, the Servers tab displays "Failed - Unable to connect with Server" in the Status of Last Send column.

Explanation: Possible causes for the failed send operation are that the Apache HTTP server service (httpd) or the Apache Tomcat application server service (tomcat5) on the target VQE-S or VQE Tools server (hosting VCDS) is not running.

To check whether the httpd service is running, log in as root and issue following command:
[root@system]# service httpd status
httpd (pid 15836 13179 2593 2592 2591 2590 2589 2588 2587 2586 2576) is running...
To check whether the tomcat5 service is running, log in as root and issue following command:
[root@system]# service tomcat5 status 
Tomcat is running...
Remedy: If either httpd or tomcat5 is not running, log in as root and start the services that are stopped as follows:
[root@system]# service httpd start 
[root@system]# service tomcat5 start
Verify that the httpd and tomcat5 services are running as shown earlier in the Explanation. If the httpd or tomcat5 processes continue to fail, check the log files for the services to determine the cause.

•The httpd log file is /etc/httpd/logs/error_log.

•The tomcat5 log files are /var/log/tomcat5/catalina.date.log.

For information on the log files for each service, see the vendor documentation for the Apache HTTP server or for the Apache Tomcat application server.

Internal Server Error When VCPT Tries to Send Channel Information to VQE-S

Symptom: When VCPT tries to send channel information to VQE Server, you get an "Internal Server Error." Java exceptions in the catalina.out file indicate that there is an address conflict or unreachable address.

Explanation: The Internal Server Error is displayed when you use VCPT to send channel information to a VQE Server. The Status of Last Send column on the VCPT Servers tab shows the error.

From the CDE110 that hosts VQE-S, you are able to successfully ping the machine running the web browser used for VQE-S AMT, and ping the CDE110 hosting VCPT.

In the /usr/share/tomcat5/logs/catalina.out file of the Apache Tomcat application server, there are Java exceptions indicating an address conflict or unreachable address. For example:
java.net.BindException: Address already in use 
at java.net.PlainSocketImpl.socketBind(Native Method) 
at java.net.PlainSocketImpl.bind(Unknown Source) 
...
java.net.SocketException: Network is unreachable 
at java.net.PlainSocketImpl.socketConnect(Native Method) 
at java.net.PlainSocketImpl.doConnect(Unknown Source)
Remedy: The Internal Server Error can be caused by one or more errors in the /etc/hosts file on the CDE110 that hosts VQE-S. Check the /etc/hosts file for typing or syntax errors, correct them, and save the file. Reboot the CDE110.

If you still receive the same Internal Server Error and Java exceptions, check the following files to see if you can find some indication for what is causing the error.

•/var/log/vqe/vqe.log

•/var/log/httpd/error_log

•/var/log/httpd/access_log

•/usr/share/tomcat5/logs/catalina.out

•/var/log/messages

Using the vqereport Command

The vqereport command can be useful for VQE troubleshooting. You can use the vqereport command to gather information on the VQE system's software configuration and hardware. The information can be used by the VQE administrator or Cisco technical support personnel to diagnose problems with VQE software or with the Cisco Content Delivery Engine 110 (CDE110) hardware.

Caution The vqereport command can take up to 15 minutes to complete and may cause VQE services (such as Unicast Retransmission) to be degraded for the duration of the execution.

The syntax for the vqereport command is as follows:

vqereport [-h | -help]
Syntax Description

-h | -help

Displays help information.

Usage Guidelines

The vqereport command generates a report file that, when the command finishes, is located in /root/hostname.dateandtime.tar.bz2. If appropriate, the report file can be attached to field issue reports for Cisco technical support.

Note You must log in as root to execute the vqereport command.

The vqereport command can be executed on the CDE110 that hosts VQE Server or on the CDE110 that hosts VCPT. The vqereport executable is located at /opt/vqes/bin/vqereport.

To have the operating system find the vqereport executable, you must set the PATH environment variable to include the location of the /opt/vqes/bin directory, or use the full path to vqereport when executing the command.

All information gathered is considered confidential, and Cisco will use this information for diagnostic purposes only.

Examples

The following example shows the execution of the vqereport command and some abbreviated output.
[root@system ~]# /opt/vqes/bin/vqereport 
This utility will go through and collect some detailed information about the hardware and 
setup of your VQE system. This information will be used to diagnose problems with your 
system and will be considered confidential information. Cisco will use this information 
for diagnostic purposes ONLY. 
Please wait while we collect information about your system. 
This process may take a while to complete.... 
No changes will be made to your system during this process. 
NOTE: You can safely ignore a failed message. This only means a file we were checking for 
did not exist. 
Press ENTER to continue, or CTRL-C to quit.
********************************************************************** 
Collect Redhat sysreport 
 
********************************************************************** 
Collect information of installed RPM packages 
... 
output omitted 
... 
Report file /root/<hostname>.20071115164635.tar.bz2 has been generated, please send it to 
Cisco support.
Stopping, Starting, and Restarting VQE-S

The VQE-S application is a service that is started with the Linux service command.

To stop, start, or restart the VQE-S application, follow these steps:

Step 1 Log in as root.

Step 2 Depending on what is required, issue one of the following commands:

•To stop VQE-S, issue the following command:
[root@system]# service vqes stop 
•To start VQE-S, issue the following command:
[root@system]# service vqes start 
•To restart (stop and then start) VQE-S, issue the following command:
[root@system]# service vqes restart 
Stopping, Starting, and Restarting VCDS

The VQE Client Channel Configuration Delivery Server (VCDS) application is a service that is started with the Linux service command.

To stop, start, or restart the VCDS application, follow these steps:

Step 1 Log in as root.

Step 2 Depending on what is required, issue one of the following commands:

•To stop VCDS, issue the following command:
[root@system]# service vcds stop 
•To start VCDS, issue the following command:
[root@system]# service vcds start 
•To restart (stop and then start) VCDS, issue the following command:
[root@system]# service vcds restart 