Cisco Video Quality Monitoring Configuration Guide
Video Quality Monitoring
Downloads: This chapterpdf (PDF - 280.0KB) The complete bookPDF (PDF - 569.0KB) | Feedback

Video Quality Monitoring

Table Of Contents

Video Quality Monitoring

Information About Video Quality Monitoring

Video Quality Monitoring Index

Key Metrics Collected Using VQM

Basic Video Attributes

Video Compression

Packet Loss

Video Entropy

Delay and Jitter

Metrics Produced by Other Products

Configuring Video Quality Monitoring

Turning On the Video Monitoring

Creating Flow Records for the Match and Collect Fields

Creating a Flow Exporter

Creating a Flow Monitor with Flow Records and Monitoring Thresholds

Creating a Class Map to Match the Monitored Video Stream

Creating Monitoring Policy Map with the Class Map and Flow Monitor

Attaching the Monitoring Policy Map to Interfaces

Troubleshooting Video Quality Monitoring


Video Quality Monitoring


First Published: July 30, 2013, OL-29868-01

Information About Video Quality Monitoring

The Video Quality Monitoring (VQM) module monitors the quality of the video calls delivered over a network. The VQM solution offered in the Cisco Integrated Services Routers Generation 2 (Cisco ISR G2) provide a complete set of video quality metrics for calls with H.264 codec.

The VQM module parses the Real-Time Transport Protocol (RTP) header and the first few bytes of the video packet payloads to retrieve critical information about the video stream. The first few bytes parsed by the VQM module includes Sequence Parameter Sets (SPS), Picture Parameter Sets (PPS), and slice headers (SH). In Cisco ISR G2, VQM measures the quality of the RTP video flows crossing the WAN link (Ingress or Egress). The VQM reports the video-quality related fields through Cisco Flexible NetFlow Version 9 (FNFv9) and IP Flow Information Export (IPFIX).

Video Quality Monitoring Index

The video quality monitoring index is calculated using the Cisco proprietary model called estimated Mean Opinion Score (eMOS). eMOS is a quantitative number assigned to the quality of the video stream based on internal measurements and calculations. The following aspects of a video is used to calculate the eMOS:

Uncompressed Video Quality

Compression Distortion

Packet Loss and Delay Distortion

Table 1-1 lists the three parts of the Video Quality Model and integrated parameters.

Table 1-1 Three Parts of the Video Quality Model and Integrated Parameters

 
Uncompressed Video Quality
Compression Distortion
Packet Loss and Delay Distortion

Video Content

Y

Y

Resolution

Y

Frame Rate and Fluctuation

Y

Y

Video Encoder Implementation

Y

Bits Per Pixel

Y

GOP Structure

Y

Y

Packet Loss Pattern

Y

Packet Loss Concealment

Y

Packet Delay

Y

Packet Jitter

Y


The eMOS value calculated using the above model is measured on a continuous scale of 0 to 100, as shown in Figure 1-1.

Figure 1-1 Scale to Measure eMOS

Key Metrics Collected Using VQM

The VQM module produces a list of metrics along with the eMOS score. These metrics are the result of eMOS calculation, and they provide a complete view of all the characteristics of a video stream. The VQM module monitors the quality of a video stream in real time, but reporting of eMOS and other metrics can be based on a configured period. The configured period should be between 1 and 10 seconds. The recommended period is 5 seconds.

The VQM module produces the following major categories of metrics:

Basic Video Attributes

Video Compression

Packet Loss

Video Entropy

Delay and Jitter

Metrics Produced by Other Products

Basic Video Attributes

Basic Video Attributes consist of the following metrics:

Most Recent Resolution Width

Most Recent Resolution Height

Frame Rate

Video Payload Average Bit Rate

Video Payload Bit Rate Fluctuation

These metrics can be used to monitor the actual resolution and frame rate of a video stream and raise flags when it is not consistent with the values configured or signaled. Also, they are used to monitor the average video bit rate and fluctuation for the purpose of cross-stream bandwidth allocation. You can adjust the buffer and queue length based on the bit rate fluctuation calculated using these metrics.

Table 1-2 lists the metric details.

Table 1-2 Basic Video Attributes Metric Details

Name of Metric
Description
Unit
Range

Most Recent Resolution Width

The most recently observed width of the video frame.

pixels

1-8000

Most Recent Resolution Height

The most recently observed height of the video frame.

pixels

1-8000

Frame Rate

Actual frame rate observed in the period.

frames per second (fps)

0-120

Video Payload Average Bit Rate

Average bit rate of the actual video payload (excluding Network, Layer 1-5, PHY+IP+UDP+RTP headers) observed in the period.

kbps

0-20000

Video Payload Bit Rate Fluctuation

Standard deviation in the bit rate of the actual video payload (excluding Network, Layer 1-5, PHY+IP+UDP+RTP headers) observed in the period.

kbps

0-20000


Video Compression

Video Compression consists of the following metrics:

Number I Frames

Number I Packets

I Packet Sizes

Number Short Term Reference (STR) Frames

Number STR Packets

STR Packet Sizes

Number Long Term Reference (LTR) Frames

Number LTR Packets

LTR Packet Sizes

Number Super-P Frames

Number Super-P Packets

Super-P Packet Sizes

Number Non Referenced (NR) Frames

Number NR Packets

NR Packet Sizes

Quantization Level of I Slices

Quantization Level of STR Slices

Quantization Level of LTR Slices

Quantization Level of Super-P Slices

Quantization Level of NR Slices

eMOS Compression Bitstream

eMOS Compression Network

The Video Compression metrics are used for the following purposes:

Monitor the video traffic for each frame type. You can set flags for many or few frames for each type. This helps maintain fairness across different streams with different frame type distributions.

Help troubleshoot video quality by providing visibility into the video endpoints' performance and codec implementation.

Monitor the compression level of video to isolate the compression defect from the overall quality outage. When the compression level is too high, you can use this metric to troubleshoot which type of video frames are causing the problem.

Table 1-3 lists the Video Compression metric details.

Table 1-3 Video Compression Metric Details

Name of Metric
Description
Unit
Range

Number I Frames

Total number of I frames observed in the period.

0-0xFFFFFFFF

Number I Packets

Total number of I packets observed in the period.

0-0xFFFFFFFF

I Packet Sizes

Total number of I packets payload (excluding Network, Layer 1-5, PHY+IP+UDP+RTP headers) sizes observed in the period.

Kilobytes (KB)

0-0xFFFFFFFF

Number Short Term Reference (STR) Frames

Total number of STR frames observed in the period.

0-0xFFFFFFFF

Number STR Packets

Total number of STR packets observed in the period.

0-0xFFFFFFFF

STR Packet Sizes

Total number of STR packets payload sizes observed in the period.

Kilobytes (KB)

0-0xFFFFFFFF

Number Long Term Reference (LTR) Frames

Total number of LTR frames observed in the period.

0-0xFFFFFFFF

Number LTR Packets

Total number of LTR packets observed in the period.

0-0xFFFFFFFF

LTR Packet Sizes

Total number of LTR packet payload sizes observed in the period.

Kilobytes (KB)

0-0xFFFFFFFF

Number Super-P Frames

Total number of super-P frames (frames that reference exclusively the LTR frames for error resilience) observed in the period.

0-0xFFFFFFFF

Number Super-P Packets

Total number of super-P packets observed in the period.

0-0xFFFFFFFF

Super-P Packet Sizes

Total number of super-P packet payload sizes observed in the period.

Kilobytes (KB)

0-0xFFFFFFFF

Number Non Referenced (NR) Frames

Total number of NR frames (P frames that are not used as reference for coding other frames) observed in the period.

0-0xFFFFFFFF

Number NR Packets

Total number of I packets observed in the period.

0-0xFFFFFFFF

NR Packet Sizes

Total number of I packet payload sizes in kilobytes observed in the period.

Kilobytes (KB)

0-0xFFFFFFFF

Quantization Level of I Slices

Average quantization level of I slices from a range defined in the H.264 standard.

0-51

Quantization Level of STR Slices

Average quantization level of STR slices from a range defined in the H.264 standard.

0-51

Quantization Level of LTR Slices

Average quantization level of LTR slices from a range defined in the H.264 standard.

Kilobytes (KB)

0-51

Quantization level of Super-P Slices

Average quantization level of super-P slices from a range defined in the H.264 standard.

0-51

Quantization Level of NR Slices

Average quantization level of NR slices from a range defined in the H.264 standard.

0-51

eMOS Compression Bitstream

Estimated Mean Opinion Score of video in the observed period after compression (bitstream model).

0-100

eMOS Compression Network

Estimated Mean Opinion Score of video in the observed period after compression (network model).

0-100


Packet Loss

Packet Loss consists of the following metrics:

Number I Packets Lost

Number STR Packets Lost

Number LTR Packets Lost

Number Super-P Packets Lost

Number NR Packet Lost

Severely Damaged Frame Percentage

eMOS Packet Loss Bitstream

eMOS Packet Loss Network

You can use these metrics to monitor packet loss and the effect of packet loss at three levels: packets, frame, and overall quality.

Table 1-4 lists the metric details.

Table 1-4 Packet Loss Metric Details

Name of Metric
Description
Unit
Range

Number I Packets Lost

Total number of I packets lost in the period.

0-0xFFFFFFFF

Number STR Packets Lost

Total number of STR packets lost in the period.

0-0xFFFFFFFF

Number LTR Packets Lost

Total number of LTR packets lost in the period.

0-0xFFFFFFFF

Number Super-P Packets Lost

Total number of super-P packets lost in the period.

0-0xFFFFFFFF

Number NR Packet Lost

Total number of NR packets lost in the period.

0-0xFFFFFFFF

Severely Damaged Frame Percentage

Total percentage of MBs impacted by packet loss and loss propagation in the observed period.

%

0-100

eMOS Packet Loss Bitstream

Deterioration of the eMOS score of video in the observed period due to packet loss (bitstream model).

0-100

eMOS Packet Loss Network

Deterioration of the eMOS score of video in the observed period due to packet loss (network model).

0-100


Video Entropy

Video Entropy consists of the following metrics:

Scene Complexity

Level of Motion

These metrics help you optimize the bandwidth of different video streams.

Table 1-5 lists the metric details.

Table 1-5 Video Entropy Metric Details

Name of Metric
Description
Unit
Range

Scene Complexity

Scene complexity from 0 to 100 as computed by the VQM algorithm.

0— Not complex.

100— Maximally complex.

0-100

Level of Motion

Level of motion from 0 to 100 as computed by the VQM algorithm.

0— No motion.

100—Maximally high motion.

0-100


Delay and Jitter

Delay and Jitter consists of the following metrics:

Frame Arriving Times Difference

Frame Arriving Times Difference Variation

These metrics help you monitor the video frame delay and jitter relative to the frame display time indicated by RTP timestamps without RTCP reports.

Table 1-6 lists the metric details.

Table 1-6 Delay and Jitter Metric Details

Name of Metric
Description
Unit
Range

Frame Arriving Times Difference

The average difference of the actual packet arriving time and the expected packet arriving time calculated using the RTP timestamp.

milliseconds(ms)

-4000-4000

Frame Arriving Times Difference Variation

The standard deviation in the difference of the actual packet arriving time and the expected packet arriving time calculated using the RTP timestamp.

milliseconds(ms)

-4000-4000


Metrics Produced by Other Products

The metrics produced by the other products consists of the following:

First Packet RTP Sequence Num

Last Packet RTP Sequence Num

These metrics help you troubleshoot synchronization issues with other producers.

Table 1-7 lists the metric details.

Table 1-7 Metrics Produced by Other Products

Name of Metric
Description
Unit
Range

First Packet RTP Sequence Num

The RTP sequence number of the first packet inspected in the period.

0-0xFFFF

Last Packet RTP Sequence Num

The RTP sequence number of the last packet inspected in the period.

0-0xFFFF


Configuring Video Quality Monitoring

Configuring VQM involves configuring the performance monitor policies on the hub router and the branch router. The performance monitor policy should be enabled on the interface. Note that the video quality monitoring depends upon the Flow Meta Data (FMD) of the video stream.

Configuring the VQM involves the following steps:


Step 1 Turn on Video Monitoring.

Step 2 Create flow records for the match and collect fields.

Step 3 Create a flow exporter.

Step 4 Create a flow monitor with flow records and monitoring thresholds.

Step 5 Create an appropriate class map to match the video streams that are monitored.

Step 6 Create a monitoring policy map with the class map and flow monitor.

Step 7 Attach the monitoring policy map to the interfaces.

Turning On the Video Monitoring

The following example shows how to turn on Video Monitoring:

video monitoring
  maximum-sessions 100
  no shutdown
 
   

The default maximum session is 100. The default mode is shutdown.

Creating Flow Records for the Match and Collect Fields

The following perf-monitor example configures a monitor to create flow records for the match and collect fields:

flow record type performance-monitoring vqm-rec
  match ipv4 protocol
  match ipv4 source address
  match ipv4 destination address
  match transport source-port
  match transport destination-port	
  match transport rtp ssrc
  collect application video resolution [ width | height ] last
  collect application video frame rate
  collect application video payload bitrate [ average | fluctuation ]
  collect application video frame [ I | STR | LTR | super-P | NR ] counter frames
  collect application video frame [ I | STR | LTR | super-P | NR ] counter packets [lost]
  collect application video frame [ I | STR | LTR | super-P | NR ] counter bytes
  collect application video frame [ I | STR | LTR | super-P | NR] slice-quantization-level
  collect application video eMOS compression [ network | bitstream ]
  collect application video eMOS packet-loss [ network | bitstream ]
  collect application video frame percentage damaged
  collect application video scene-complexity
  collect application video level-of-motion
  collect transport rtp sequence-number [ last ]

Creating a Flow Exporter

The following example shows how to create a flow exporter configuration:

flow exporter vqm-exporter 
  description PAM or CPCM interface
  destination < ip address>

Creating a Flow Monitor with Flow Records and Monitoring Thresholds

The following example shows how to create a flow monitor and monitoring thresholds:

flow monitor type performance-monitoring vqm-mon 
  record vqm-rec
  exporter vqm-exporter
  cache type synchronized 
  cache entries 1000
  cache timeout synchronized 5
  history size 60 timeout 30 

Creating a Class Map to Match the Monitored Video Stream

The following example shows how to create a class map to match a monitored video stream:

class-map match-all vqm-class
  match access-group 101

Creating Monitoring Policy Map with the Class Map and Flow Monitor

The following example shows how to create a policy map with the class map and the flow monitor you created earlier:

policy-map type performance-monitoring vqm-policy
  class vqm-class
    flow monitor vqm-mon

Attaching the Monitoring Policy Map to Interfaces

The following example shows how to attach a monitoring policy map to an interface:

Interface serial 0/0:0
   service-policy type performance-monitoring input vqm-policy
   service-policy type performance-monitoring output vqm-policy
 
   

Troubleshooting Video Quality Monitoring

Table 1-8 lists the debug and show commands that you can use to troubleshoot VQM.

Table 1-8 Show and debug Commands for Troubleshooting VQM

Commands
Description
show video quality monitoring 

Displays the video quality monitoring status.

show video quality monitoring record

Displays all the active video quality monitoring sessions.

show video quality monitoring record <num>

Displays one particular session's record.

debug video monitoring error

Turns on VQM error debugging.

debug video monitoring event

Turns on VQM event debugging.

debug video monitoring datapath

Turns on monitoring datapath debugging.


The following example shows the output of the show video quality monitoring record command:

router# show video quality monitoring record
 
   
----- record begin -----
vqm session id: 14, vqm_cpu_saving_mode: FALSE    time: 23:46:44
 
   
From: 20.20.20.1,   To: 20.20.20.2
    SrcPort: 16422, DstPort: 16420, Protocol: udp
most_recent_resolution_width: 1280, most_recent_resolution_height: 720
frame_rate: 30
video_payload_average_bit_rate: 1153, video_payload_bit_rate_fluctuation: 105
I_frame_count: 0, I_packet_count: 0, I_packet_byte_count: 0
STR_frame_count: 24, STR_packet_count: 99, STR_packet_byte_count: 1219
LTR_frame_count: 43, LTR_packet_count: 195, LTR_packet_byte_count: 1128
SuperP_frame_count: 41, SuperP_packet_count: 188, SuperP_packet_byte_count: 1130
NR_frame_count: 33, NR_packet_count: 154, NR_packet_byte_count: 984
quantization_level: I-slice 0, STR-slice 23, LTR-slice 23, SuperP-slice 23, NR-slice 23
I-pkt-lost: 9, STR-pkt-lost: 9, LTR-pktlost: 2, SuperP-pkt-lost: 3, NR-pkt-lost: 20
severely_damaged_frame_percentage: 91
emos_compression_bitstream: 76, emos_packet_loss_bitstream: 75, emos_bit(14): 1
emos_compression_network: 89, emos_packet_loss_network: 7, emos_net(14): 82
scene_complexity: 42, level_of_motion: 1
first_packet_rtp_sequence_num: 7907, last_packet_rtp_sequence_num: 8669
-----record end-----
Total active record number: 1