Cisco CDA Visual Quality Experience Application User Guide, Release 3.0
Chapter 4: Using the VQE-S Application Monitoring Tool

Table Of Contents

Using the VQE-S Application Monitoring Tool

Logging in to and Navigating in VQE-S AMT

Monitoring the Health of the VQE-S Processes

Viewing Channel Configuration and Status

Viewing Channel Configuration

Viewing Channel Statistics

Monitoring Unicast Retransmission and RCC

Viewing Error Repair and RCC Configuration

Viewing Error Repair and RCC Statistics

Monitoring Multicast Load Balancer

Monitoring RTCP Exporter

Monitoring the STUN Server

Configuring VQE-S Logging

Configuring VQE-S Debugging


Using the VQE-S Application Monitoring Tool


This chapter describes how to use Cisco VQE-S Application Monitoring Tool (VQE-S AMT or AMT). The tasks that you can perform with VQE-S AMT are listed in Table 4-1.

Table 4-1 VQE-S AMT Tasks

VQE-S AMT Task
Section Where Described

Logging in to VQE-S AMT

Logging in to and Navigating in VQE-S AMT

Monitor the health of the VQE-S processes

Monitoring the Health of the VQE-S Processes

View channel configuration details, status, and statistics

Viewing Channel Configuration and Status

Monitor statistics for Unicast Retransmission and RCC

Monitoring Unicast Retransmission and RCC

View configuration details, status, and statistics for Multicast Load Balancer

Monitoring Multicast Load Balancer

View configuration details, status, and statistics for VQE-S RTCP Exporter

Monitoring RTCP Exporter

View STUN Server Statistics

Monitoring the STUN Server

Configure VQE-S logging levels

Configuring VQE-S Logging

Configure VQE-S debugging levels

Configuring VQE-S Debugging



Note VQE-S AMT supports configuration capabilities for logging levels and debugging options. However, the configured values do not persist when AMT is restarted.


Logging in to and Navigating in VQE-S AMT

Before logging into VQE-S AMT, you need a valid Linux username and password on the Cisco CDE110 hosting AMT. The username does not have to belong to any special group. Creation of the username is the responsibility of the Cisco CDE110 system administrator.

VQE-S AMT supports two web browsers: Microsoft Internet Explorer version 6.0 or later, and Mozilla Firefox version 2.0 or later. The minimum screen resolution required for VQE-S AMT is 1024 x 768 pixels.

VQE-S AMT uses secure HTTPS. Access by multiple simultaneous browsers is supported.

To display the Channels Status Summary graph of active, inoperative, and inactive channels in the AMT VQE-S Status window, Adobe Flash Player must be installed on the computer that hosts the browser accessing AMT. Adobe Flash Player is free and can be found at this URL:

http://get.adobe.com/flashplayer/

To login to VQE-S AMT, follow these steps:


Step 1 Point your web browser to the Cisco CDE110 that hosts VQE-S AMT using an IP address or fully qualified domain name:

https://ip_address 

           or

https://fully_qualified_domain_name 

The AMT login dialog box is displayed.

Step 2 Enter a Linux username and password.

Step 3 Click OK.

If the username and password are valid, the AMT VQE-S Status window is displayed.


Figure 4-1 shows the VQE-S AMT navigation controls and buttons.

Figure 4-1 VQE-S AMT Navigation

VQE-S AMT (Figure 4-1) provides these controls and buttons for navigating and displaying VQE data:

Navigation tree—Use the collapsible and expandable tree to display the VQE-S AMT items that can be viewed or configured. The navigation tree can be hidden by clicking the < arrow in its right border.

A parent node in the tree (such as Channels) usually provides summary data.

A child node in the tree (such as an individual channel name) provides data on the specific child node.

Navigation Tabs—Click the tabs to display VQE data for viewing or VQE parameters for configuring in the main window. Some but not all VQE items have Configuration and Statistics tabs that display the relevant data or parameters.

Advanced—Click to get additional, detailed information on the current VQE-S AMT items. The advanced data is intended for Cisco Technical Support and are not described in this document.

Refresh —Click to update the displayed data. VQE-S AMT statistical data are not automatically updated when VQE-S real-time counters are incremented. The Last refresh date-and-time information to the right of the Refresh button indicates when the last update of displayed data occurred.

Monitoring the Health of the VQE-S Processes

When you click the System node in the navigation tree and the VQE-S Status tab, VQE-S AMT displays the VQE-S Status window (Figure 4-2). The Application Health Monitor and a Channel Status Summary are displayed in the VQE-S Status window.

Figure 4-2 VQE-S Status Window

In the Application Health Monitor, a green icon with a checkmark indicates the VQE-S system is running and healthy. A red icon with an X indicates one or more problems with the VQE-S processes.

Using the Application Health Monitor, you can view the following status information:

System Up Time—Time that Cisco CDE110 hosting VQE-S has been running.

Status Message—Current status of the VQE-S software or information on problems (for example, the message "Config File Parsing Failed")

VQE-S Version—Current version of the VQE-S software.

Process Table—Information on these VQE-S processes:

Multicast Load Balancer

STUN Server

Data Plane

Control Plane

In the Process Table, Failure Count is the number of times the process has failed and been restarted by VQE-S Process Monitor. Process Monitor is responsible for starting and monitoring the preceding processes.

The Channels Status Summary section provides the number of and a graph for active, inoperative, and inactive channels.

In addition to the VQE-S Status tab, you can get information on the VQE-S system on the five other tabs. Table 4-2 lists the information that is available from each tab accessible from the System node.

Table 4-2

System Node Tab
Information Provided

VQE-S Status

Application Health Monitor and Channel Status Summary

Hardware

Processor and memory on the Cisco CDE110 that hosts VQE-S

Each CDE110 has two physical processors and four virtual processors.

System Info

Hostname, operating system version, date and time, Network Time Protocol servers, Domain Name System servers, and iptables information (trusted channel-provisioning servers).

Network

Output from the ifconfig and ip addr show lo commands.

The configuration information displayed by ifconfig is for the four CDE110 Ethernet interfaces and the CDE110 loopback addresses.

The loopback addresses displayed by ip addr show lo are the Feedback Target addresses for the configured channels.

System Status

Detailed information on host uptime, currently running processes, and file system disk space.

SNMP

SNMP information on the VQE-S host taken from the snmpd.conf file.

Histograms

Set-top box IGMP latency data is displayed in graphs and tables.

Logs

Most recent 300 lines from the following VQE-related log files are displayed:

VQE-S log (/var/log/vqe/vqe.log)

System messages log (/var/log/messages)

HTTPD error log (/var/log/httpd/error_log)

SSL error log (/var/log/httpd/ssl_error_log)

Tomcat log (/usr/share/tomcat5/logs/catalina.out)


Viewing Channel Configuration and Status

When you click the Channels node in the navigation tree, VQE-S AMT displays the Channel Lineups window (Figure 4-3).

Figure 4-3 Channel Lineups Window

Using the Channel Lineups window, you can do the following:

Click on a column heading to reorder the channels in the list. For example, clicking on Feedback Target orders the entries by feedback target: the combination of an IP address and RTCP port number.

Double-click on any item in a channel's row to display the full details for the channel in a new window.

Click the Re-Activate button to cause VQE-S to create all channels that are not currently created in the VQE-S software.

In Figure 4-3, the channel-lineup summary data indicates when the lineup was last updated (for example, with VCPT) and provides totals for all channels and active channels as well as aggregated bandwidth and total receivers:

 Last update: 2007-08-06T14:15:31, Total Channels: 10, Active Channels: 9 
Aggregated Bandwidth: 88000 (kbits/sec), Total Receivers: 0

The Channel Lineups window has the same information that is defined for channels using VCPT but has the following additions. For information on the other data that appears in the Channel Lineups table, see Table 3-3 on page 3-8.

Status—Indicates channel status as follows:

Green with a checkmark—Channel is active. VQE-S is receiving the multicast stream.

Yellow with exclamation mark (!)—Channel is inactive. The channel is successfully initialized, but VQE-S is not receiving the multicast stream.

Red with an X—Channel is inoperative (for example, the channel is not configured correctly).

Member Receiver Population—Provides the number of VQE Clients that are currently receiving this multicast stream.

Viewing Channel Configuration

When you click on a channel name in the navigation tree and then click the Configuration tab, VQE-S AMT displays the Channel Configuration window (Figure 4-4).

Figure 4-4 Channel Configuration Window

The Channel Configuration window has the information listed in Table 4-3.

Table 4-3 Channel Configuration  

Field
Description

                            Channel Configuration

Channel Name

Name of the channel.

Feedback Target Address

Feedback Target IP address assigned on the VQE-S host.

Rapid Channel Change

true—RCC is enabled for this channel.

false—RCC is not enabled for this channel.

Mode

Currently only Lookaside mode is supported.

Error Repair

true—Unicast Retransmission is enabled for this channel.

false—Unicast Retransmission is not enabled for this channel.

Post Repair Loss RLE Enabled

true—RTCP Extended Reports will include the Loss RLE (run-length encoded) report block type.

false—RTCP Extended Reports will not include the Loss RLE report block type.

                            Original Source Stream

Multicast IP Address

Multicast IP address for the channel's original source stream.

Protocol

Currently only RTP is supported.

IP Source Filter

Source IP address of the channel's original source stream.

RTCP XR Enabled

true—RTCP Extended Reports are enabled.

false—RTCP Extended Reports are not enabled.

Port

Number of the port for the original source stream. When the protocol is RTP, this is the RTP port number.

RTCP Port

Number of the RTCP port for the original source stream.

Bit Rate (Kbps)

Bit rate configured for the original source stream.

                            Unicast Retransmission Stream

Retransmission IP Address

IP address of the Unicast Retransmission stream.

RTCP Port

Number of the RTCP port for the Unicast Retransmission stream.

RTP Port

Number of the RTP port for the Unicast Retransmission stream.


Viewing Channel Statistics

When you click on a channel name in the navigation tree and then click the Statistics tab, VQE-S AMT displays the Channel Statistics window (Figure 4-5).

Figure 4-5 Channel Statistics Window

The Channel Statistics window has the information listed in Table 4-4.

Table 4-4 Channel Statistics  

Field
Description

                            Channel Input Stream

Status

active—VQE-S is receiving the multicast stream.

inactive—The channel is successfully initialized, but VQE-S is not receiving the multicast stream.

inoperative—The channel is not, for example, configured correctly.

Received Packets

Number of RTP packets received by VQE-S for this multicast stream.

Lost RTP Packets

Number of RTP packets sent but not received by VQE-S for this multicast stream.

Average Stream Rate (Mbps)

Average megabits per second received by VQE-S for this multicast stream.

Received Bytes

Number of bytes received by VQE-S for this multicast stream.

                     Error Repair Statistics—Generic NACK Messages Received

Total Messages

Number of NACK messages received by VQE-S from VQE-C for this multicast stream.

Invalid Messages

Number of invalid messages received by VQE-S from VQE-C for this multicast stream. Invalid messages are received messages that, for example, cannot be parsed.

                            Repair Packets

Requested

Number of RTP packets VQE Clients have requested for Error Repair for this multicast stream.

Sent

Number of RTP packets sent by VQE-S that have succeeded in repairing an error for this multicast stream.

Not Sent

Number of failed RTP packets that were not repaired by VQE-S for this multicast stream. VQE-S was not able to send an Error Repair packet.

                            Error Repair Average Rate (packets per second)

5 second, 5 minute, 1 minute, 15 minute

For each time interval, the average number of packets per second that VQE-S has sent to VQE Clients to repair errors for this multicast stream. Averages are calculated for each time interval over a period of time that is greater than the interval. For example, the five second calculation might be an average of 12 five-second intervals over a one minute period.

                            RCC Statistics—Rapid Channel Changes

Requested

Number of Rapid Channel Changes requested for this channel.

Accepted

Number of Rapid Channel Changes requested for this channel where the VQE-S has accepted the request and sent IPTV packets and other channel data to a VQE Client on the set-top box.

Refused

Number of Rapid Channel Changes requested for this channel where VQE-S has refused the request. For example, VQE-S may refuse an RCC request if no I-frame is available that is a suitable starting point for the unicast burst, or if the VQE-S has reached a maximum load limit.

Failed

Number of Rapid Channel Changes that have failed. A VQE-C has made an RCC request, and VQE-S has accepted the request and sent IPTV packets and other channel data to a VQE Client. But the RCC failed. For example, an RCC can fail if the VQE-C stops receiving the packets from the VQE-S earlier than expected.

                            RTCP Packets

Received

Number of RTCP packets received by VQE-S for this multicast stream.

Sent Downstream

Number of RTCP packets that VQE-S sent downstream for this multicast stream.

Sent Upstream

Number of RTCP packets that VQE-S sent upstream for this multicast stream.


Monitoring Unicast Retransmission and RCC


Note In VQE-S AMT, Error Repair is the term used for Unicast Retransmission.


You can use VQE-S AMT to view Error Repair and RCC statistics.

Viewing Error Repair and RCC Configuration

When you click Error Repair and RCC in the navigation tree and click the Configuration tab, VQE-S AMT displays the Error Repair and RCC Configuration window (Figure 4-6).

Figure 4-6 Error Repair and RCC Configuration Window

The Error Repair and RCC Configuration window has the information listed in Table 4-5. Each of the parameters listed in the table can be configured with the VQE Configuration Management System. For information on configuring VQE-S, see Chapter 6, "Configuring VQE Server and VQE Tools."

Table 4-5 Error Repair and RCC Configuration  

Field
Description

                            Error Repair/RCC Configuration

RCC Mode

RCC mode can be aggressive or conservative. This value is configured with the vqe.vqes.rcc_mode parameter in VCDB. For information on this parameter, see Table A-3 on page A-5.

IGMP Join Variability (msec)

Amount of variability (in milliseconds) between the fastest and slowest IGMP joins for RCC. This value is configured with the vqe.vqes.igmp_join_variability parameter. For information on this parameter, see Table A-3 on page A-5.

Excess Bandwidth Fraction

Percent of channel bandwidth that will be available on the access link for Unicast Retransmission and Rapid Channel Change. This value is configured with the vqe.vqes.excess_bw_fraction parameter. For information on this parameter, see Table A-4 on page A-7.

Maximum Interval between MPEG RAPs (msec)

Maximum time interval (in milliseconds) between MPEG RAPs (random access points) for the entire channel lineup. This value is configured with the vqe.vqes.rap_inteval parameter in VCDB. For information on this parameter, see Table A-3 on page A-5.

                            Differentiated Services Code Point Configuration

RTCP Packets

DCSP value for RTCP packets that VQE-S transmits. This value is configured with the vqe.vqes.rtcp_dscp parameter in VCDB. For information on this parameter, see Table A-1 on page A-2.

Retransmitted RTP Packets

DCSP value for RTP packets that VQE-S transmits. This value is configured with the vqe.vqes.rtp_trans_dscp parameter in VCDB. For information on this parameter, see Table A-1 on page A-2.


Viewing Error Repair and RCC Statistics

When you click Error Repair and RCC in the navigation tree and click the Statistics tab, VQE-S AMT displays the Error Repair and RCC Statistics window (Figure 4-7). The statistics are for this VQE-S and are similar to the Error Repair and RCC statistics displayed for individual channels but are for all channels in the current channel lineup.

The Error Repair and RCC statistics are reset to 0 when VQE-S restarts or when the channel lineup changes.

Figure 4-7 Error Repair and RCC Statistics Window

The Error Repair and RCC Statistics window has the information listed in Table 4-6.

Table 4-6 Error Repair and RCC Statistics  

Field
Description

                            Generic NACK Messages Received

Total Messages

Number of NACK messages received by this VQE-S from VQE Clients.

Invalid Messages

Number of invalid messages received by this VQE-S from VQE Clients. Invalid messages are received messages that, for example, cannot be parsed.

                            Repaired RTP Packets

Requested

Number of RTP packets VQE Clients have requested for Error Repair from this VQE-S.

Sent

Number of RTP packets sent by this VQE-S that have succeeded in repairing an error.

Not Sent

Number of failed RTP packets that were not repaired by VQE-S.

                            Inbound and Outbound Error Repair Average Rate (packets per second)

5 second, 5 minute, 1 minute, 15 minute

For each time period, the average number of packets per second that VQE-S has received (inbound) or sent (outbound) to VQE Clients to repair errors (Unicast Retransmission).

                            RCC Statistics—Rapid Channel Changes

Requested

Number of Rapid Channel Changes requested for this channel.

Accepted

Number of Rapid Channel Changes requested for this channel where the VQE-S has accepted the request and sent IPTV packets and other channel data to a VQE Client on the set-top box.

Refused

Number of Rapid Channel Changes requested for this channel where VQE-S has refused the request. For example, VQE-S may refuse an RCC request if no I-frame is available that is a suitable starting point for the unicast burst, or if the VQE-S has reached a maximum load limit.

Failed

Number of Rapid Channel Changes that have failed. A VQE-C has made an RCC request, and VQE-S has accepted the request and sent IPTV packets and other channel data to a VQE Client. But the RCC failed. For example, an RCC can fail if the VQE-C stopped receiving RCC packets from the VQE-S earlier than was expected.


Monitoring Multicast Load Balancer

When you click Multicast Load Balancer in the navigation tree, VQE-S AMT displays two tables associated with the Multicast Load Balancer component of VQE-S: the Interface Table and the Multicast Table. Figure 4-8 shows both tables.

Multicast Load Balancer distributes traffic for incoming multicast streams and outgoing Error Repair over the Ethernet interfaces on the Cisco CDE110 that hosts VQE-S. The interfaces must be correctly configured and operational (up) to participate in this load balancing.

Multicast Load Balancer determines the best interface on which to join or leave the multicast group, distributing the joins across available interfaces to avoid oversubscription. Load Balancer also monitors the status of these interfaces, moving the streams to other interfaces in case of interface failure.

Figure 4-8 Interface Table and the Multicast Table

The Interface Table (Figure 4-8) has the information listed in Table 4-7. The information is about the Ethernet interfaces on the Cisco CDE110 that are currently participating in multicast load balancing.

Table 4-7 Interface Table  

Column
Description

IP

IP address assigned to this Ethernet interface.

Name

Name that is assigned to this Ethernet interface by Linux. For information on these names, see "Configuring the Linux Operating System for VQE-S" section on page D-3.

Status

Either up or down.

Max BW

Maximum speed of the interface.

Available BW

Interface bandwidth that is currently not being used.

Ingress BW Reserved for Unicast

Percent of the interface's ingress bandwidth that is reserved for Error Repair requests from VQE Clients. This value is configured by the burst-rate option in the VQE-S configuration file.

Multicast Membership

Number of channels that are currently using this interface for Error Repair.


The Multicast Table (Figure 4-8) has the information listed in Table 4-8. The information is about the multicast streams on this VQE-S that are currently participating in multicast load balancing.

Table 4-8 Multicast Table  

Column
Description

Multicast Address

IP address for this multicast group.

Name

Name of the interface that is being used for this multicast stream.

Source Filter

IP address of the multicast group's original source stream.

Data Bandwidth

Bandwidth specified for this multicast steam in VCPT's Bit Rate field.


Monitoring RTCP Exporter

When you click RTCP Exporter in the navigation tree and click the Configuration tab, VQE-S AMT displays the VQE-S RTCP Exporter parameters. RTCP Exporter currently has no parameters that can be configured with VQE-S AMT, and the parameters are for viewing only. For information on configuring RTCP Exporter, see the "Configuring VQE-S RTCP Exporter" section on page 2-25.

When you click RTCP Exporter in the navigation tree and click the Statistics tab, VQE-S AMT displays the RTCP Exporter Statistics window (Figure 4-9).

Figure 4-9 RTCP Exporter Statistics Window

The RTCP Exporter Statistics window has the information listed in Table 4-9. The statistics are for this VQE Server's RTCP Exporter.

Table 4-9 RTCP Exporter Statistics

Field
Description

                            VQM Application Configuration

VQM Host

IP address of the host on which the video-quality monitoring (VQM) application resides.

VQM TCP Port

TCP port number on which the VQM application listens for video quality data from RTCP Exporter.

Enabled

Either true (enabled) or false (not enabled).

Configuration Status

Either enabled or not enabled. It is possible for the value in the Enabled field to be true, and the value in the Configuration Status field to be not enabled (for example, if the host IP address or TCP port are not valid).

Operational Status

Message providing information on the operational status of RTCP Exporter.

                            RTCP Exporter Stats

Packets Exported

Number of RTCP compound packets that have been exported to a VQM host.

Packets Dropped

Number of RTCP compound packets that have been dropped rather than exported to a VQM host. In heavy load situations, RTCP Exporter may intentionally drop RTCP packets.

Details on dropped packets (number of dropped packets and the position and time where this loss occurred) is contained in the collection of reports exported by this VQE-S.


For information on RTCP Exporter and the video-quality monitoring application, see the "VQE-S RTCP Exporter for Video-Quality Monitoring" section on page 1-19.

Monitoring the STUN Server

When you click STUN Server in the navigation tree, VQE-S AMT displays the STUN Server Statistics window (Figure 4-10).

Figure 4-10 STUN Server Statistics Window

The STUN Server Statistics window has the information listed in Table 4-10.

Table 4-10 STUN Server Statistics

Field
Description

Total Requested

Number of STUN binding requests that the VQE-C STUN Client sent to the STUN Server.

Successfully Sent

Number of responses to STUN binding requests that the STUN Server successfully sent to the VQE-C STUN Client.

Response Failed

Number of instances where the STUN Server fails to generate a binding response because of an internal error.

Sent Failed

Number of instances where the STUN Server successfully generates a binding response but fails to send it to the VQE-C STUN Client (for example, because of a socket error).


Configuring VQE-S Logging

When you click Logging in the navigation tree, VQE-S AMT displays the logging priority levels (Figure 4-11) for VQE-S. The priority levels allow you to control the logging level for system messages for all VQE-S processes (Process Monitor, Control Plane, Data Plane, and Multicast Load Balancer). By default, all VQE-S system messages are written to the file /var/log/vqe/vqe.log.

Figure 4-11 Logging Priority Levels

To configure a VQE-S logging priority level, click the button next to the level you want and click Apply.

In VQE-S AMT, the logging priority levels are listed from least verbose to most verbose. The Emergency level generates the smallest number of messages, and the Info level generates the greatest number of messages. The default value is Warning.

When you select a logging level, log messages are generated for the levels below that level. For example, when the level is set to Error, messages are generated for Emergency, Alert, Critical, and Error.

The selected logging priority level does not persist when VQE-S is stopped. When VQE-S restarts, the logging priority level goes back to the default. Logging priority levels that do persist can be set using the process_monitor command options or the VQE-S configuration file. For information on these methods, see Appendix A, "VQE, System, and Network Parameters."

Configuring VQE-S Debugging

When you click Debugging in the navigation tree and click a debug component, the debugging options for that component are displayed. Figure 4-12 shows the debug components. The options allow you to control the types of debugging messages that are written to the syslog file. By default, debug messages are written to the file /var/log/vqe/vqe.log.

Figure 4-12 Debug Components

To configure VQE-S debugging, follow these steps:


Step 1 In the VQE-S AMT navigation tree, click the + sign next to Debugging and click the component for which you want debug messages.

The debug flags and filters for that component are displayed. For example, Figure 4-13 shows the flags and filters for the RTP/RTCP category.

Figure 4-13 RTP/RTCP Flags and Filters

Step 2 To enable a debugging flag, click the check box in the On/Off column, or to disable an enabled flag click the check box to uncheck it.


Note For the next step, be aware that not all debug flags support debug filters (in the Filter Status column). If debug filters are not supported, the check box next to the flag is grayed out.


Step 3 For an enabled debugging flag, if you want to filter debugging messages by a specific channel or VQE Client, do the following:

a. In the Filter Status column, click the check box for the option.

b. In the Debug Filters section, click either Multicast IP or Client IP:

For Multicast IP, enter the channel's source stream IP address and (RTP/UDP) port number.

For Client IP, enter the VQE Client's IP address.

Step 4 Click Apply.

The selected debugging options are enabled.

The selected debugging options do not persist when VQE-S is stopped. When VQE-S restarts, debugging goes back to the default—no options are enabled.