Guest

Cisco Videoscape Distribution Suite for Television

Release Notes for Cisco VDS-TV 2.5.2

  • Viewing Options

  • PDF (540.4 KB)
  • Feedback
Release Notes for Cisco TV CDS 2.5.2

Table Of Contents

Release Notes for Cisco TV CDS 2.5.2

Contents

New Features

All Environments

CDSM Features

Login Authentication and Password Control Feature

Popularity-Based Caching

Logging

SNMP Features

Trick-Mode Restriction

Bandwidth Manager for Thin Pipe

APIs

VOD Error Repair

ISA Environments

ISA Stream Extensions

Dynamic Modification of Playlists

ISA Regionalization

TV Playout

Content Chunking

Virtual Content Store

RTSP Environments

Ingest Steering

Streamer Cache-Fill from Multiple Cache Groups in RTSP NGOD

Asynchronous Stream Control Message Processing

NAT Support

Digital Video Watermarking

Multiple Vault Groups by Stream Service

Enhancements

New MIBs

New Traps and Objects

Platform Default Baud

Software Upgrade

New Installations

HTTP Live Streaming

Catch-Up to Live

Play While Ingesting the Same Content

H.264/AVC Ingest

Playlist Enhancements

Capacity Enhancements

Library Timeout

CDSM GUI Enhancements

Other Enhancements

Supported Environments

System Requirements

Limitations and Restrictions

Open Caveats

CServer

Configuration

FSI

Monitoring

Statsd

Resolved Caveats

CServer

Cache2App

Configuration

Database

FSI

Installation

ISA

Monitoring

MSA

Network

Reporting

RTSP

Statsd

Streamer

Syslog

Video Quality

Other

Upgrading to Release 2.5.2

Related Documentation

Obtaining Documentation and Submitting a Service Request


Release Notes for Cisco TV CDS 2.5.2


These release notes cover Cisco TV CDS Release 2.5.2.

Revised: February 2012 OL-24790-01

Contents

The following information is in the release notes:

New Features

Enhancements

Supported Environments

System Requirements

Open Caveats

Resolved Caveats

Upgrading to Release 2.5.2

Related Documentation

Obtaining Documentation and Submitting a Service Request

New Features

The features supported in Release 2.5.2 are grouped as follows:

All Environments

ISA Environments

RTSP Environments

All Environments

The following features are supported for all environments:

CDSM Features

Login Authentication and Password Control Feature

Popularity-Based Caching

Logging

SNMP Features

Trick-Mode Restriction

Bandwidth Manager for Thin Pipe

APIs

VOD Error Repair

CDSM Features

Release 2.5.2 introduces a number of enhancements and features to the CDSM. This section covers the following changes to the CDSM in Release 2.5.2:

HTTPS Support

CDSM Logs

CDSM Setup Settings

Capacity Planning Report

System Cleanup

Interface Setup and Server Setup Pages

HTTPS Support

The CDSM GUI now supports HTTPS as a secure way to access the browser-based interface. The cdsconfig script offers the following choices to access the CDSM GUI:

HTTP

HT TPS

HTTP and HTTPS

CDSM Logs

The CDSM has the following logs:

httpd.log.<yyyymmdd>—Apache error log

httpd_access.log.<yyyymmdd>—Apache access log

cdsm.log.<yyyymmdd>—CDSM GUI log

cdsm-ws.log.<yyyymmdd>—Web Services log

All log files use UTC for the log entry time stamps and filenames. All four files are located in /arroyo/log directory.

CDSM Setup Settings

The CDSM Setup page has the following changes:

Stream Destination Settings

RTSP and NAT

Fail Ingest Tuning

Content Storage

VVI

VOD Error Repair

Playout Scheduler

Other Changes

Stream Destination Settings

Added Mixed option for Stream Destination, which allows both cable and IPTV configuration. Previously, only one Stream Destination type was allowed. The Mixed option makes the QAM Gateway page and associated Headend Setup page available, along with the Stream Destination page.

An additional option of Auto was added for RTSP environments, where it typically is not necessary to explicitly configure QAM gateways or IPTV subnets. The Auto option removes these configuration pages from the CDSM GUI. The Auto option is not supported for ISA environments.

In ISA environments, the Mixed option is only available for gigabit Ethernet streaming. The Streaming Mode is set on the following configuration pages:

VVI with Content Storage set to Shared—Shared ISA Setup page

CDS (legacy)—Streamer BMS page

VVI with centralized management (combined VVIM and Stream Manager)— Streamer BMS page

VVI with Content Storage set to Distributed—CDSM Setup page under VVI section

RTSP and NAT

Previously, NAT was supported for Stream Destination Type of IPTV in an ISA environment.

In Release 2.5.2, NAT is supported for RTSP environments when Stream Destination Type is IPTV or Mixed. The NAT option adds the STUN Play Error Delay and STUN Play Timeout fields to Server Setup page for Streamers.

For more information, see the "NAT Support" section.

Fail Ingest Tuning

The Fail Ingest Tuning setting is enabled by default and is available for the CDSM, VVI with central management, and VVIM; it is not available for the Stream Manager. When enabled, the Fail Ingest Tuning fields are displayed on the Configure > System Level > Ingest Tuning page and provides the ability to configure the ingest error detection settings for all Vaults in the CDS.

Content Storage


Note The Content Storage feature is only displayed for ISA environments.


In Release 2.5.2, the "Shared Content Storage" section on the CDSM Setup page is changed to the "Content Storage" section.Content Storage has the following options:

Shared—Introduced in a previous release as the "Shared Content Store" (SCS) feature, the SCS option allows one instance of a Content Store to be shared with many instances of Stream Services, each located in its own video hub office (VHO) with its own video backoffice.

Distributed—Introduces in Release 2.5.2, the Distributed option allows for two configurations:

ISA Regionalization—Allows the use of a centralized storage facility containing both Vaults and Caching Nodes in a Virtual Video Infrastructure (VVI), while maintaining a localized or remote CDS at each headend. For more information, see the "ISA Regionalization" section. ISA Regionalization requires that Vault Groups be enabled on the Stream Manager CDSM.

Vault Virtualization—Replaces the SCS with the Virtual Content Store (VCS). No content is ingested at the local VHO. All ingests and deletions of content occur at the central location, and both ingests and deletions are initiated by the local BMS at each local VHO, just as they were in the SCS. However, the VHOs do not need to communicate with the super headend (SHE) as they did with the SCS feature. With VCS, communication of ingestions and deletions is handled by the Ingest Driver client residing on the master Streamer in each VHO and the Ingest Driver server residing on the master Vault in the SHE. Vault Virtualization requires that Vault Groups be disabled on the Stream Manager CDSM. For more information, see the "Virtual Content Store" section.

When VVIM or Stream Manager is the role for a Distributed Content Storage, then the Change Notification option is available. When Change Notifications is enabled, notifications are sent and received between the Stream Manager and the VVIM when Vault Groups and Cache Groups are changed.

VVI

Added Streaming Mode (ASI or Gige) when Content Storage is Distributed. Streaming Mode must be set to Gige for Content Storage feature.

VOD Error Repair

When enabled, the Error Repair page is displayed at the Configure > System Level, Configure > Array Level, and the Configure Server Level. For more on VOD Error Repair, see the "VOD Error Repair" section.

Playout Scheduler

Playout Scheduler is only available in an ISA environment on a VVI with central management or a legacy CDSM.

Playout Scheduler—On/Off

Localized EPG Extension—On/Off

When Playout Scheduler is enabled, the TV Playout Scheduler pages are displayed. To enable Localized EPG Extensions, the Playout Scheduler must be enabled. For more information on the Playout Scheduler, see the "TV Playout" section.

Other Changes

Other CDSM Setup page changes consist of the following:

Vault Redundancy section changed to Vault Groups section.

RS DVR section changed to nDVR section.

Bandwidth Manager removed from CDSM Setup page for RTSP environments. This feature was only used in internal labs and is replaced by the Bandwidth Manager for Thin Pipes. See the "Bandwidth Manager for Thin Pipe" section for more information.

Capacity Planning Report

Release 2.5.2 introduces the Capacity Planning report. This report shows high-watermark (peak) streams played in a given time period. The following views are supported:

Hourly

Daily view at 1 minute, 5 minute, 15 minute, and 1 hour intervals

Weekly view at daily intervals

Monthly

The Capacity Planning report is supported on an overall basis, as well as filtered by service group (if applicable) or by Streamer. The report is available through the CDSM GUI as well as through an HTTP API. For each time period, the peak total streams and the peak bandwidth within that period is displayed. Additionally, the peak total HD streams and peak total SD streams are displayed.


Note For streams to be included in the Capacity Planning report, the start and end time stamps should have a difference of at least one minute to account for processing and propagation delays. We recommend at least two to three minute difference between the start and end of a stream to account for one of the Streamers begin busy. If the streams follow this rule, then the difference between the peak stream count in the Capacity Planning Report and the actual stream count is less than 0.5 percent.


System Cleanup

As part of improving the ease of identifying and fixing common configuration errors, the System Cleanup feature has been added to Release 2.5.2. If encountered, these errors are added to the Alarms table, which is displayed in the CDSM banner. Clicking the alarm in the Alarms table takes you to the CDSM page where you can resolve the issue.

The following are monitored and registered in the Alarms table and linked to the associated CDSM page:

System clock is out of synchronization

MSA events exist for the current CDSM day

Incorrect IDs on the Stream Manager (for ISA environments only)

Missing or incorrect initial IDs (Group, Server, or Setup)

For more information, see the "System Cleanup" section in the "System Maintenance" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

In addition to the System Cleanup page links, the following situations are monitored and registered in the Alarms table:

System clock is out of synchronization

MSA events exist for the current CDSM day

Incorrect IDs on the Stream Manager (for ISA environments only)

Missing or incorrect initial IDs (Group, Server, and Setup)

Login Authentication and Password Control Feature

Release 2.5.2 supports the following features for login authentication and password control:

User Authentication

CDSM User Login Checks

System Authentication Settings

Add User—Force Password Change

Edit User—Manage User Account

User Authentication

Release 2.5.2 offers the following database options for maintaining user authentication data:

Local database (located on the CDSM)

RADIUS server (external database)

TACACS+ server (external database)

User authentication is configured through the cdsconfig script.The user authentication settings consist of choosing an external access server (TACACS+ or RADIUS) or the internal (local) CDSM authentication database for user access management, and setting the challenge key and timeout. The default is to use the local database for authentication. The cdsconfig script prompts you for the primary and backup external access server configuration. If the CDSM does not get a response from the primary server within the timeout period, the backup server is contacted.

Local Database User Password Encryption

Passwords are not stored as clear text in the local database, they are stored using Secure Hash Algorithm (SHA), which includes a salt that is randomly generated for increased security. When a user logs in to the CDSM, SHA-1 is used to generate the hashed version of the user password, including the randomly generated salt, which is then sent for authentication. If the hashed version stored in the database matches what the user entered, the user is allowed access to CDSM; otherwise, access is denied.

CDSM User Login Checks

Release 2.5.2 introduces system checks that are performed on the CDSM during the user login process and during access to the CDSM GUI. If any one of the checks does not pass, access to the CDSM is denied and an error message is displayed with information on which check failed.

Table 1 describes the system checks that are performed during the user login process and during user access to the CDSM.

Table 1 CDSM Checks for User Login 

Check
Description
Additional Information
Error Message

Disk Space

Verify that all drives have not exceeded 95 percent storage capacity.

Disk space is checked every time an HTTP request is received by the CDSM. If any drive exceeds the threshold, the CDSM access is denied and the user is navigated to the login window where an error message is displayed.

The drive names and threshold values can be configured in cdsm.ini file in the /arroyo/www/htdocs/cdsm/cdsTV/conf directory.

[disk-partition]
drive.names = /arroyo,/arroyo/db
drive.threshold = 95
 
        

CDSM is running out of disk space (/arroyo). Contact the System Administrator for further assistance.

User Account Locked

Verify that the user attempting to log in does not have this attribute enabled on the account.

The User Account Locked check box is checked on the Edit User page for the account. Only a user with Master-level access can check or uncheck the User Account Locked check box.

User account is locked. Contact the System Administrator for further assistance.

Concurrent User Sessions

Verify that the number of concurrent user sessions has not been exceeded.

The Concurrent User Sessions field is set on the Edit User page for the account. If the number of sessions the user is concurrently logged in to does not exceed the setting, access is allowed; otherwise, access is denied until the user logs out of one of the other sessions.

Maximum number of concurrent sessions reached. Try again later.

Password Expiration Interval

Verify that the password has not expired.

The Password Expiration Interval field is set on the System Authentication page. If this field is set, and the password has expired, the user is denied access to the CDSM.

Password has expired. Contact the System Administrator for further assistance.


If the checks described in Table 1 all pass, the user is authenticated and if authentication is successful, the following checks are performed:

1. If the Force Password Change check box is checked for the user account, then the user is navigated to the Edit User page and the user is forced to change the password.

2. If the Password Expiration Reminder interval has started, the user is navigated to the Edit User page and notified that the password is about to expire. The user can, however, ignore the reminder and continue without changing the password.

System Authentication Settings

The System Authentication page is only visible to users with Master-level access. The System Authentication fields apply system wide to all users of the CDSM GUI. Table 2 describes the System Authentication fields.

Table 2 System Authentication Fields 

Field
Description

Lock Account on Unsuccessful Login

If the Lock Account on Unsuccessful Login check box is checked, a user account is locked after the number of Unsuccessful Login Attempt Count has been reached within the Unsuccessful Login Attempt Period.

For example, if the Unsuccessful Login Attempt Count is set to 3, the Unsuccessful Login Attempt Period is set to 1 day, and Lock Account on Unsuccessful Login is checked; then after 3 unsuccessful attempts within 1 day, the user account is locked.

Unsuccessful Login Attempt Count

Number of login attempts to allow the user before the account is locked. If the account is locked, the master-level user can unlock the account by unchecking the User Account Locked check box on the Edit Users page.

Unsuccessful Login Attempt Period

Time interval for which the number of unsuccessful login attempt count is persisted. When the time interval lapses, and if the account is not locked, the Unsuccessful Login Attempt Count is reset to 0.

Enable Password History

The history of user passwords is stored in the database if the Enable Password History check box is checked.

During a password change, if the Enable Password History check box is checked, the new password is compared with the history of the user's passwords, and the password change is only successful if the new password is different than the passwords that were previously used.

Password History Size

Specify the number of old passwords to store for each user in the database. The default is 2.

Password Change Interval

Minimum interval between non-administrative password changes for a given user. The default is 24 hours.

Password Expiration Interval

Maximum lifetime of the password. If the password has not been changed within the Password Expiration Interval, then the user account is automatically disabled.

Password Expiration Reminder

Interval prior to the password expiration that the user is notified about the password expiration.

Idle Session Timeout Interval

Maximum time a session can be idle. If the time lapse between user requests exceeds the Idle Session Timeout Interval setting, the user is redirected to the Login page.


As an example, if the Password Expiration Interval is set to 6 months (180 days) and the Password Expiration Reminder is set to 15 days; then 15 days before the password expires, the user is taken to the Edit Users page where a message is displayed stating the password is soon to expire. The message also includes the number of days the current password is active before it expires. The user has the option to change the password or continue without changing the password.

If the password expires, the user cannot log in to the CDSM. A Master- level user can change the user password and unlock the user account. Anytime the user password is changed by the Master-level user, the Force Password Change check box is checked and the next time the user logs in to the CDSM, the user is taken to the Edit Users page and is forced to change the password. The user is not be able to access any of the other CDSM GUI pages until a password change has occurred.

Password Complexity Rules

Password Complexity Rules apply to any password change performed by the user. These rules can be overridden by Master-level users when the Override Password Check check box is checked on the Add Users page or the Edit Users page.

Add User—Force Password Change

When a new user is added, the Force Password Change attribute for the user is checked. When the user logs in to the CDSM for the first time, the Edit User page is displayed and the user is forced to change the password.

During a password change, the new password is validated for complexity based on the Password Complexity Rules set on the System Authentication page. The password complexity check can be overridden if the change password is performed by a user with Master-level access and the Override Password Check check box is checked. The Override Password Check check box is available on the Add Users page and the Edit Users page if the user has Master-level access.

Edit User—Manage User Account

Release 2.5.2 introduces a new action, Manage User Account. Table 3 describes the new Manage User Account fields, which are only available to Master-level users.

Table 3 Manage User Account Settings 

Field
Description

Lock Account on Failed Login

When the Lock Account on Failed Login check box is checked, the user is locked out of the CDSM GUI if the number of failed login attempts exceeds the allowed number of failed attempts configured in the System Authentication page.

This setting overrides the Lock Account on Unsuccessful Login setting on the System Authentication page.

User Account Locked

To lock a user out of the CDSM GUI, check the User Account Locked check box.

Force Password Change

To force a password change for the user. at the next login, check the Force Password Change check box. If this check box is checked, the user is taken to the Edit User page at the next CDSM GUI login and must initiate a password change.

Concurrent User Sessions

Maximum number of concurrent sessions allowed for this user.


Popularity-Based Caching

Popularity-based caching reduces the write rate to the storage devices on the Streamer and Caching Node while maintaining the best possible cache-hit rate on the available storage. .

To control peak and average write rates to cache (flash or disk storage), the algorithm that determines when content is written to cache is changed so that only content that is likely to be accessed most often is cached. Content is only cached if it is more popular than the least popular content that is currently cached. Otherwise, the content is transmitted from the Vaults to the end-users by way of the cut-through mode, where content is temporarily stored in the Streamer and Caching Node RAM without ever writing it to disk or flash storage, and then streamed directly from the Streamer's RAM to the end-user. When cache space is needed, the least popular content is evicted from cache first.

The write rate for caching content is determined by the rate at which previously popular content becomes less popular to the point where it no longer makes sense to keep it in cache, and previously unpopular content becomes more popular to the point where it does make sense to keep it in cache. Content popularity is measured by the time-decaying average of the number of play requests on each Global Object Identifier (GOID).

Previously, all content was written to cache (except when overloaded) and the Least Recently Used (LRU) content was evicted first.

With the Popularity-Based Caching feature, only popular content is written to cache and the least popular content is evicted first.

CDSM GUI

The popularity half-life value is configurable through the CDSM GUI Maintain > System Configs page (Popularity Half Life field), which is only accessible to users with Engineering-Level access. In most cases the default setting is sufficient, but in cases where a significant fraction of viewed content has a "flash" popularity pattern shorter than the popularity half-life value, changing the setting may result in a better cache-hit rate overall.

Logging

Release 2.5.2 introduces Logging enhancements. Previously, log files were dispersed among several directories and followed different filename formats and log entry formats.

In Release 2.5.2, all logs are located in the /arroyo/log directory. The log files are rotated at least once a day and time stamps are added to the filenames. Some log files that grow rapidly are rotated more frequently (determined by file size); this rotation may happen up to once an hour. Most log files have the following suffix: .log.<YYYYMMDD.> The time zone for log rotation and filename suffixes is coordinated universal time (UTC). As part of the new log entry format, the log level and facility are included.

All log entries have the following changes:

Stream handle is represented in decimal format

IP addresses are represented in dotted-decimal format

Clear identification of where a stream is going rather than a MAC address

Time is represented in UTC

Global Object ID (GOID) is represented in hexadecimal

Log messages currently in the streamevent.log file are converted to a structured message and assigned the "stream trace" facility number. Other messages that record stream creation, routing, or playout are converted to a structured message and assigned the "stream trace" facility number. This enhancement, along with configuring syslog-ng to direct all "stream trace" facility messages to a single, centralized log server, provides a coherent set of log messages that describe stream history.

Configuring Logging Levels

Previously, logging levels for RTSP, FSI, and Ingest Manager were configured on the respective configuration pages (for example, RTSP logging was set on the Server Level RTSP Setup page).

In Release 2.5.2, all logging is configured at the System Level or Server Level. The configuration of the logging levels at the Server Level overrides the System Level settings.

For information on configuring the logging levels, see the "Configuring the CDS" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

New Logging Level Mapping Between Release 2.5.2 on the CDSM and Previous Releases on the CDS Servers

This section describes the mapping between logging configuration in previous releases and logging configuration in Release 2.5.2.

There are a large number of changes in the log utility. The behaviors of rtsp.log, fsi.log and aim.log are different. The location of the log files is changed from /home/isa/bss/log to /arroyo/log, and the time stamp is changed to use UTC time in the all the CServer and application log files. Additionally, the rotation method is changed.

Table 4 describes the mapping between the logging level configuration for the RTSP Setup page in previous releases of the Cisco TV CDS software and the Logging page in Release 2.5.2.

Table 4 RTSP Server Logging Level Mapping

RTSP Setup Page
Logging Page (Choose Facility Name: rtsp)

OFF

Local Log Level: emergency

LOW

Local Log Level: informational

HIGH

Local Log Level: informational + Debug flag: general and messages

PERFORMANCE

Local Log Level: critical + Debug flag: messages


Table 5 describes the mapping between the logging level configuration for the FSI Setup page in previous releases of the Cisco TV CDS software and the Logging page in Release 2.5.2. The default log level for FSI is informational.

Table 5 FSI Server Logging Level Mapping

FSI Setup Page
Logging Page (Choose Facility Name: fsi)

OFF

Local Log Level: critical

LOW

Local Log Level: error

HIGH

Local Log Level: informational


Table 6 describes the mapping between the logging level configuration for the Ingest Manager page in previous releases of the Cisco TV CDS software and the Logging page in Release 2.5.2.

Table 6 Ingest Manager Logging Level Mapping 

Ingest Manager Page
Logging Page (Choose Facility Name: aim)

OFF

Local Log Level: error

LOW

Local Log Level: informational + Debug flag: general

FULL

Local Log Level: informational + Debug flag: general + Debug flag: verbose


Table 7 describes the mapping between the logging level configuration for the Authentication Manager page in previous releases of the Cisco TV CDS software and the Logging page in Release 2.5.2.

Table 7 Authentication Manager Logging Level Mapping

Authentication Manager Page
Logging Page (Choose Facility Name: ev_auth)

ERROR

Local Log Level: error

INFO

Local Log Level: informational

HIGH

Local Log Level: informational + Debug flag: general

HEX_DUMP

Local Log Level: informational + Debug flag: general and verbose


SNMP Features

Release 2.5.2 supports SNMPv3, which adds support for user-password-based authentication and access control. SNMPv3 also optionally allows encryption of all SNMP communications, including objects contained in a response to a GET or inside traps (notifications or INFORMs).

While SNMPv3 provides multiple ways of implementing authentication, access control, and encryption, Release 2.5.2 has the following implementation:

User-Based Security Model

View-Based Access Control Model

User-Based Security Model

The User-based Security Model (USM), which provides SNMP message-level security, is implemented in Release 2.5.2 as follows:

Users are created (configured) in the SNMP agent on a CDS server through the CDSM GUI, as well as on the Network Management Station (NMS).

Password-based authentication is optional, and if enabled, the user must have an associated authentication key (password) of a minimum length of eight characters, and an authentication protocol of either HMAC-MD5 or HMAC-SHA1.

Encryption may optionally be enabled, in which case an encryption key (a minimum of eight characters) is required, and an encryption protocol of DES or AES.

View-Based Access Control Model

The View-based Access Control Model (VACM) is used for controlling access to management information Release 2.5.2 implements VACM by allowing configuration of each management object (OID) or group of OIDs on a CDS server through the CDSM GUI to be exposed with read-only or read-write access to a configured user.


Note The SNMPv2c security model that uses community strings for read-only or read-write access are still supported in this release. SNMPv3 USM and VACM are optional.


Trap Community Enhancements

Release 2.5.2 supports the configuration of a per-trap-sink community string or a default community string. The supported notifications are: SNMPv1 TRAPs, SNMPv2 NOTIFICATIONS, and SNMPv2-inform INFORM. Each trap sink, associated with a different trap station, can have an optional default community strings to be used when sending traps. Alternatively, a default trap community string can be configured, which is used if the per-station community string is not configured.

CDSM GUI

For information on configuring SNMP, see the "Configuring the SNMP Agent" section in the "Configuring the CDS" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

New SNMP Agent Commands and Log Changes

Release 2.5.2 includes the following changes:

SNMP Agent Commands

SNMP Log Changes

SNMP Agent Commands

In Release 2.5.2, the SNMP agent is now controlled by the following commands:

# service snmpd start
# service snmpd stop
# service snmpd restart
 
   

The snmpd service rc script included in Release 2.5.2 automatically configures the snmpd service to be started in Linux run-levels 3 and 5. To make any changes to this behavior, the chkconfig or ntsysv commands can be used. The following command configures snmpd to be managed by using the chkconfig command:

# chkconfig --add snmpd 
 
   

The following command configures snmpd to be turned on in run levels 3 and 5:

# chkconfig --level 35 snmpd on
 
   

SNMP Log Changes

The SNMP log file, snmpd.log, is now located in the /arroyo/log directory. All log entries use UTC for the time stamp. All SNMP traps are now logged in the snmpd.log file.

Trick-Mode Restriction

Restriction of trick-mode controls (pause, rewind, fast-forward) per playlist segment is supported.

If a client issues a trick-mode command for a locked out playlist segment or attempts to bypass a trick-mode restricted segment by jumping to the next segment, an LSC_NOT_PERMITTED response is sent to the set-top box. If a client has sent a fast-forward trick-mode command and a restricted segment is reached, the stream continues a normal play speed and an LSC_DONE response is sent to the set-top box with the NPT of the beginning locked out segment. An LSC_NOT_PERMITTED response is also sent to indicate that the LSC_DONE is because of a locked out trick-mode segment.


Note Fast-forward trick-mode restriction is not available in RTSP environments.


The default setting for Fast-Forward Resume and Rewind Skip is disabled. For information on configuring the trick-mode restrictions, see the "Configuring MPEG Tuning" section in the "Configuring the CDS" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

Bandwidth Manager for Thin Pipe

Previously when configuring thin pipe mappings, the bandwidth is distributed equally among all the servers in the group.

In Release 2.5.2, one server in the site is elected as the bandwidth manager for all servers in the site. A site is defined by using a new GUI page (Configure > Array Level > Site Setup) that associates groups with a site. The bandwidth manager controls the traffic leaving the site to any other site and queries all the CDS servers in the site for the thin pipe mapping configuration of each CDS server. Initially, the bandwidth manager allocates bandwidths of whatever the CDS servers have already committed, provided the committed bandwidths are within the pipe bandwidth limits; otherwise, the bandwidth manager allocates a percentage of what is committed. After the initial allocation, the bandwidth manager distributes the bandwidth equally among all the remaining CDS servers in the site.

Each CDS server in each group reports the bandwidth each one is using to the bandwidth manager every ten seconds. The bandwidth threshold for each server has an upper limit of 90 percent and a lower limit of 5 percent. If a server reaches either limit, the server reports this to the bandwidth manager immediately, and does not wait for the ten-second report interval. For example, if the server is given 100 Mbps and the streams that were just started uses 90 Mbps, the upper threshold limit has been reached and the server asks the bandwidth manager for more bandwidth.

A separate entry is maintained for each thin pipe with a list of servers that have the same thin pipe configuration. Servers that belong to the same thin pipe are added and removed as they become reachable or unreachable.

The bandwidth manager service runs on each server in either the primary mode or the passive mode. The one server at the site that is running the primary mode is selected through a discovery mechanism. The primary bandwidth manager maintains all the thin pipes and associated server structures. If the server running the primary bandwidth manager fails or loses connectivity, the newly elected bandwidth manager takes over and when the old primary bandwidth manager becomes available again and connectivity is restored, the thin pipe information and structures are deleted from the old primary.

A new log file, bwm.log has been added to log all bandwidth manager messages. The following logging levels are defined (default level is Information):

Critical

Error

Warning

Information

Debug

Debug Verbose

For information on configuring the thin pipe mapping, see the "Configuring Cache-Fill Bandwidth Using Thin Pipe Mapping" section in the "Configuring the CDS" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

APIs

The following monitoring APIs have been added in Release 2.5.2:

ContentRange—Returns a list of content objects contained within the CDS network within a specified range.

CapacityPlanning—Returns capacity-related data from Streamers and ISVs (SSVs) to assist in monitoring the capacity of the CDS.

Trick Info Reconciliation Request—Returns a list of trick-mode events for a specified stream ID

The following TV Playout APIs have been added in Release 2.5.2:

TV Playout Content Currently Playing—Returns a list of content currently being played by TV playout.

Barker Streams Currently Playing—Returns a list of barker streams currently being played.

TV Playout Channels—Returns a list of channels configured in the TV Playout.

TV Playout Stream Report—Returns a report of TV Playout streams for a specified time interval.

TV Playout Schedule Exporter—Retrieves a TV Playout schedule.

TV Playout Schedule Importer—Imports a TV playout schedule into the TV CDS.

Create Barker Playlist—Creates a barker playlist.

VOD Error Repair

The VOD Error Repair feature retransmits lost packets to improve the quality of the end-user video experience. The VOD Error Repair feature uses negative acknowledgement (NACK) retransmission methods to implement retransmission-based error repair.


Note VOD Error Repair is supported on ISA environments that use the Cisco (RTSP) setting as the LSCP Client Protocol, and RTSP environments that use the Cisco RTSP deployment type.


Previously, the Cisco TV CDS software supported User Datagram Protocol (UDP) for RTSP environments.

In Release 2.5.2, in addition to UDP streaming, unicast Realtime Transport Protocol (RTP) with Realtime Transport Control Protocol (RTCP) streaming, as well as Error Repair (ER) are supported.

The client dictates which streaming protocol is used by way of the RTSP SETUP message. The following streaming protocols are supported in the same system with simultaneous streams of each type:

UDP

RTP

UDP with NAT traversal (Interactive Connectivity Establishment [ICE])

RTP with NAT traversal (ICE)

RTP with retransmission-based error repair

RTP with NAT traversal (ICE) and retransmission-based error repair

For more information on VOD Error Repair, see the "VOD Error Repair" section in the "Product Overview" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide.


Note Retransmission-based Error Repair is only available with RTP streaming.


Error Repair Client on STB

VOD Error Repair feature requires that the STB have the Cisco Visual Quality Experience Client (VQE-C) software running on it. The VQE-C is the error-repair client software, which has the following capabilities:

Receives RTP video packets

Detects missing packets

Requests retransmission of missing packets

Merges retransmitted packets with original stream

Collects statistics and counters for monitoring

The VQE-C is a software development kit (SDK) that is available for download through the open-source program. Additionally, the STB must comply with the Cisco RTSP syntax for VOD Error Repair.

CDSM GUI

For information on configuring VOD Error Repair in the CDS, see the "VOD Error Repair" section in the "Engineering Access Level Pages" appendix of the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

Monitoring

The play management application (PMA) log file, vqe.log, is located in the /arroyo/log directory. To check for PMA errors, enable the PMA debug flag on the Logging page in the CDSM.

AMT

Application Monitoring Tool (AMT) runs a web application on each Streamer and provides several troubleshooting tools. For more information, see the "Using the TV CDS Streamer Application Monitoring Tool" appendix in the Cisco TV CDS 2.5 ISA Software Configuration Guide or the Cisco TV CDS 2.5 RTSP Software Configuration Guide.

ISA Environments

The following features are supported for ISA environments:

ISA Stream Extensions

Dynamic Modification of Playlists

ISA Regionalization

TV Playout

Content Chunking

Virtual Content Store

ISA Stream Extensions

The ISA Stream Extensions feature allows real-time splicing of MPEG-2 transport streams as identified by the Society of Telecommunications Engineers (SCTE) 35 standard. The embedded SCTE 35 cue messages contain information for digital program insertion (including advertisement insertion) in live content as well as content recorded for the purpose of enabling time-shifted on-demand services.

Release 2.5.2 supports pre-roll, post-roll, and mid-roll placements of digital program insertion on a CDS in an ISA environment that is based on a playlist structure. The Vault detects the SCTE-35 cures and process them at the time of ingest. The StreamExtChannel event channel on the CORBA NotificationService is used to send ContentSignalingEvents that contain the SCTE-35 cue information to the backoffice.


Note The SCTE-35 cue message cannot be greater than 400 bytes.


CDSM Configuration

To configure this feature set the TME/SCE field to Enable for MystroMDN.

The TME/SCE field is located on different CDSM pages, depending on the type of system configured (VVI or CDS).

In a VVI with split-domain management, on the VVIM GUI, choose Configure > System Level > Shared/Distributed ISA Setup

In a VVI with split-domain management, on the Stream Manager, choose Configure > Array Level > VHO ISA Setup

In a VVI with central management or a legacy CDS, choose the Configure > Array Level > Streamer BMS


Note The configuration change for the ISA Stream Extensions feature requires that the ISA service is restarted on both the VVIM and the Stream Manager (or the legacy CDS). To restart the ISA service, choose Maintain > Services, select the check box for ISA, and click Submit.


Dynamic Modification of Playlists

In addition to supporting digital program insertion, the ISA Stream Extensions feature supports dynamic playlists modification. The following modifications can be performed on playlists that have been defined and created:

Delete_Segment—Remove a segment from the playlist

Replace_Segment—Replace a segment in the playlist with one or more segments

Splice_Segment—Insert one or more segments at a specified NPT start value or NPT end value within an existing playlist segment

Add_Segment—Add one or more segments after a segment in the playlist

CDSM

The CDSM GUI provides augmentations to the Stream Play History. The Stream Play History report first displays the Session ID Summary. When a session ID is clicked, if a playlist was streamed for the session, the Session Playlist History report is displayed.

The Session Playlist History report shows the playlists that occurred for the selected session ID, with each playlist segment represented by a different color. Clicking a playlist segment displays the trick-mode history of that playlist. Hovering the mouse over a trick mode brings up a pop-up text box that shows the playlist segment that was playing during the trick-mode action that occurred. When the Show Stream Data option is clicked, additional details about the stream and content associated with the playlist are displayed.


Note The Trick Mode Capture feature must be enabled to access the Stream Play History reports.


ISA Regionalization

The ISA Regionalization feature is a combination of the Virtual Video Infrastructure (VVI) and legacy Content Delivery System (CDS). This feature provides the ability to centrally store content on Vaults located in a centralized storage facility and allow remote sites to have a record of inventory of this content and access it by way of the Caching Nodes or directly on the central Vaults. The remote sites still operate as independent entities with their own local Vault Group, local Content Store, and local Streamers; managed by their own CDSMs and possibly accessing their own local BMS and AMS. The Streamers at each remote site can stream both locally stored content and centrally stored content.

The ISA Regionalization feature allows the use of a centralized storage facility containing both Vaults and Caching Nodes in a Virtual Video Infrastructure (VVI), while maintaining a localized or remote CDS at each headend.

For more information on ISA Regionalization, see the "Vault Virtualization" section in the "Network Design" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide.

TV Playout

Release 2.5.2 incorporates the TV Playout features available in Release 1.5.4.6 and adds enhancements to these features. The TV Playout feature is for ISA environments and includes Public, Education, and Government (PEG) channels and Barker Streams. PEG channels differ from traditional broadcast channels in that the service provider itself must ingest and stream the content rather than receiving and forwarding a satellite feed.

For information on TV Playout, see the Cisco TV CDS 2.5 ISA Software Configuration Guide.

New Release 2.5.2 Features for TV Playout

The following enhancements have been added to the TV Playout feature in Release 2.5.2:

SSV Groups—SSV Groups are required for TV Playout in Release 2.5.2

Playout Upgrade Status—Upgrading from Release 1.5.4.6 status report and step-through process

Barker Stream/Playlists—Allows the creation of barker playlists. The Barker Stream page from Release 1.5.4.6 has been renamed the Barker Stream /Playlist page to encompass the playlist feature. A playlist is a list of content objects with the number of loops set for each content. The playlist can be scheduled by selecting the timeslot in the Playout Scheduler and selecting Playlist as the Content Type in the Playout Creator window.

The Type field is added, which offers the choice of Barker or Playlist. If Playlist is selected, the Output Channel field and the Start Barker (Stop Barker) button are removed.

Playout Scheduler User Preferences—Content Selection setting is now available both on this page and the User Default Settings page. Action on Recurring Schedules has been added.

Added Action on Recurring Schedule field (Preserve Existing Schedules or Overwrite Existing Schedules) to Playout Scheduler Preferences.

Added Content Selection option. Choose either Use Suggester or Use Select Box. Use Suggester displays a text box for selecting content; Use Select Box displays a drop-down list. If Use Suggester is selected, as the user types in the text box, content matching the text is displayed. If Use Select Box is selected, a content list is displayed and the user can type the first few letters of the content and is taken to that part of the list.

Playout Creator—Content search capability has been enhanced (Use Suggester option). By default, the 25 most recently ingested content objects are displayed. The Playout Creator has also been enhanced to configuring a playlist, as well as a single content object.

If Use Suggester is selected, as you type in the text box, content matching the text is displayed in a list. If you click Search, The Content List window is displayed with the following options:

Quick Lists—Click Most Recent Ingests, and the 25 most recently ingested content objects are listed.

Browse Content—Click a character in the Browse Content section, and all content objects beginning with that letter are listed.

Content List—Displays the results of the Search, the Quick List, or the Browse Content selection. The content name and ingest time are listed.

You can select a content object from the Content List, or select Close in the upper-right corner of the window and start your search again

Recurring Schedules—Check the Recurring Schedule check box to display the Recurring Schedules fields. You can schedule the content to repeat daily, weekly, or monthly; and you can set the recurrence end time to a year from the start time, to end after so many occurrences, or to end on a specific date.

Playout Scheduler—Changes to page options:

Schedule/Unschedule button and Delete button have been added, and Select Multiple Channels button has been added to the Filler Content at the bottom of page.

To unschedule a content object or playlist, select all content objects to be unscheduled and click Schedule/Unschedule. The content is unscheduled but is not deleted.

To schedule an unscheduled content object or playlist, select all content objects to be scheduled and click Schedule/Unschedule.

To delete a content, whether scheduled or unscheduled, select the content and click Delete.

Select Multiple Channels—Filler Content area has a new option, the "Select Multiple Channel" button.

Calendar View—Displays days of the month that have events scheduled for a specified channel in the CDS. The Channel drop-down list was added for this purpose. If a channel is selected, only that channel is available in the Playout Scheduler. If "Select a Channel" is selected, and the User Preferences has all channels selected, then all the channels are displayed in the Playout Scheduler.

Playout Monitor—Previously, when a user had Read-only access, they could not view the Playout Schedule. Release 2.5.2 adds the Playout Monitor page to allow viewing of the Playout Schedule for read-only users (Monitor > Array Level > Playout Monitor).

Content Manager—Allows deletion of multiple content objects. The Content Manager page is accessible by going to Maintain > Services > Content Manager. The search mechanism is the same as the one for Completed Ingests. The list displays the content name, file size, duration, date of ingest, and delete check box. Check the delete check box for each content to be deleted and click Delete.

APIs—Pre-existing APIs have been converted to Representational State Transfer (REST)-based Web Services that use the curl utility and the following new APIs have been added:

Playout Schedule Exporter—Retrieves a Playout schedule

Playout Schedule Importer—Imports a Playout schedule

Create Barker Streams—Configures a barker stream

Create Barker Playlist—Configures a playlist for a barker stream

Start or Stop Barker Streams—Starts and stops a barker stream

Get all Barker Streams—Returns a list of all barker streams and the status of each

TV Playout Installation and Upgrade

The Release 2.5.2 ISO image file includes Public, Education, and Government (PEG) Assets. The cdsinstall script has been enhanced to include support for TV Playout (PEG/Barker).

Following are the snippets of the cdsinstall script pertaining to installing TV Playout:

[root@outkast ~]# ./cdsinstall ./CDS-TV.iso
Select Deployment Type (ctrl-c to quit):
   1) ISA
   2) RTSP/FSI
   3) PEG/BARKER
Choice :3
PEG/BARKER Selected
 
   
.
.
.
Calling inst.sh for peg
Killing running processes: statsd syslogd
Un-taring peg-base.tgz
Calling forprod.sh
Installing PEG-specific files (existing files backed up to .file)
PEG installation complete
 
   

There are no changes to the cdsconfig script for TV Playout.

After upgrading the TV CDS software from Release 1.5.4.6 to Release 2.5.2, there are some steps that must be followed before any configuration changes can occur. The Playout Upgrade Status page (Maintain > Software > Playout Upgrade Status) tracks the status of these steps. Clicking the Status of each step takes the user to the page that needs to be modified. (For the link to work on the first one, the user needs to have Engineering-level access.)

Additionally, the Alarms table displays an alarm stating the playout upgrade is incomplete.

Content Chunking

For DVD on Demand solutions and long recordings, Release 2.5.2 supports ingest and streaming of assets up to 120 GB in size and recordings that last longer than 12 hours. This is accomplished by dividing the asset into multiple chunks of approximately 16 GB each. If the asset bit rate is slower than 3.4 Mbps, the live recording length cannot be longer than 12 hours.


Note The Content Chunking feature is disabled by default. All the CDS servers in a deployment must be upgraded before enabling this feature. To enable, add the following line to the setupfile file on each CDS server:

content id type 2
 
   


Note Content that was ingested after upgrading to Release 2.5.2 but before downgrading is valid as long as the ingest resulted in a single chunk. Content that was ingested as multiple chunks is truncated to the first chunk and all the GOIDs in the other chunks are identified as orphans and deleted when the server reboots after the software downgrade.


Virtual Content Store

The Virtual Content Store feature in an ISA environment replaces the Shared Content Store feature introduced in Release 2.1. The Shared Content Store (SCS) feature is the ability of several local sites (video hub offices [VHOs]) to ingest content at a central location and share that content with the other VHOs. The SCS feature eliminated ingesting multiple copies of the same content.

Release 2.2 and later releases offered Multi-Screen Video (MSV) support for ISA environments configured with SCS. An MSV Asset Deletion script ran nightly by default. The script identified the VOD assets that were removed by the backoffice and deleted them from the Vaults. If the asset was found in any one of the VHOs, the asset was not deleted. Only when the asset was not found in all VHOs was the asset deleted from the Vaults.

In Release 2.5.2, Vault Virtualization replaces the SCS with the Virtual Content Store (VCS). No content is ingested at the local VHO. All ingests and deletions of content occur at the central location, and both ingests and deletions are initiated by the local BMS at each local VHO, just as they were in the SCS. However, the VHOs do not need to communicate with the super headend (SHE) as they did with the SCS feature. With VCS, communication of ingestions and deletions is handled by the Ingest Driver client residing on the master Streamer in each VHO and the Ingest Driver server residing on the master Vault in the SHE. Vault Virtualization requires that Vault Groups be disabled.

The Virtual Content Store (VCS) component runs on the Streamer designated as the Stream Service Master, and if a failover occurs, the VCS fails over with the Stream Service Master.

Only one copy of the centrally located asset is ingested and shared by the system, and the asset is only deleted when all VHOs have requested the deletion.

With the VCS, the MSV Asset Deletion script is no longer needed and is removed during the software upgrade to Release 2.5.2.

For more information on ISA Regionalization, see the "Vault Virtualization" section in the "Network Design" chapter of the Cisco TV CDS 2.5 ISA Software Configuration Guide.

RTSP Environments

The following features are supported for ISA environments:

Ingest Steering

Streamer Cache-Fill from Multiple Cache Groups in RTSP NGOD

Asynchronous Stream Control Message Processing

NAT Support

Digital Video Watermarking

Multiple Vault Groups by Stream Service

Ingest Steering

The Ingest Steering feature offers the ability to have one backoffice send ingest information to the master Vault, and depending on the product ID in the content name, the content is ingested by one of the Vaults in the specified Vault Group.

For VOD content, Ingest Steering depends on the product ID in ADI.XML. For Mediax recording, Ingest Steering depends on the product ID defined in Input Channel page. Currently, for live recording, Ingest Steering only supports Media Scheduler live capture.

The Ingest Steering feature requires that Vault Groups and Ingest Manager be enabled.

Streamer Cache-Fill from Multiple Cache Groups in RTSP NGOD

Currently, because the RTSP NGOD environments use the C2 protocol over HTTP for communication between Cache Groups and Stream Groups, Stream Groups can only be associated with one Cache Group.

In Release 2.5.2, multiple Cache Groups can be configured for each Stream Group for RTSP NGOD environments. The same failover capability that applied to systems that use Cache Control Protocol (CCP) applies to the NGOD environments that use C2 over HTTP. If all the Caching Nodes in a Cache Group fail or if there is a network connectivity problem between the Cache Group and Stream Group, the next preferred Cache Group takes over the cache-fill process.

The ability to load balance among Cache Groups with the same preference setting is supported, which is the same functionality that currently works for CCP. Each Stream Group can have up to five Cache Groups mapped to it.

CDSM GUI

Support configuration for more than one Cache Group for Stream to Cache Map for VVI-RTSP-HTTP. The Stream to Cache Map page supports up to five Cache Groups for each Stream Group.

Asynchronous Stream Control Message Processing

Previously, the RTSP server maintained a thread pool (default of 32) to process both control and setup messages and used the Cache2App synchronous API to place requests to the CServer. The processing thread had to wait for the CServer to respond before it could reply to the client request. In a situation where the CServer did not respond in a timely manner, the processing threads were used up and system responsiveness degraded.

In Release 2.5.2, the control message processing model processes stream control requests asynchronously, instead of synchronously. Requests are queued to the CServer input queue through the asynchronous Cache2App API. In addition, a set of response listening threads wait for responses coming back from the CServer and reply to the STB. Outstanding requests to the CServer are tracked.

NAT Support

Previously, the NAT Traversal feature was supported for ISA environments that use IPTV as the Stream Destination setting in the CDSM Setup page. Stream Destination offers the ability to map Stream Groups to subnets, which is useful in IPTV networks where each end-user has an IP address. The NAT Traversal feature allows streaming to client devices that are behind a NAT device. All session setup messages go through the backoffice before reaching the RTSP server, while all stream control messages go directly to the RTSP server from the STB for IPTV networks using NAT.

In Release 2.5.2, the NAT Traversal feature is supported for RTSP environments that use IPTV as the Stream Destination setting and Cisco as the RTSP Deployment Type.

CDSM GUI

To configure NAT for RTSP, do the following:


Step 1 Log in to the CDSM GUI as a user with Engineering access. The CDSM Setup page is displayed.

Step 2 For the Stream Destination Type, select IPTV and for Options select the NAT radio button.

Step 3 For the RTSP Deployment Type, select Cisco.

Step 4 Click Submit.



Note For NAT to function correctly, a STB client must implement the Cisco RTSP specification and support Session Traversal Utilities for NAT (STUN). Whether or not NAT traversal is used is dictated by the RTSP SETUP message.


Digital Video Watermarking

Release 2.5.2 supports digital video watermarking, also called digital video fingerprinting, of assets for the NGOD RTSP environment.


Note Digital Watermarking is only supported for NGOD deployments for RTSP environments.


The Digital Watermarking feature provides the ability to track the source of unauthorized content copying. A watermark is embedded into the content for each end-user. If a copy of the content is found, then the watermark can be retrieved from the copy and the source is identified. The watermark is undetectable by the person viewing the content.

The watermarking of an asset is performed by selecting a non-reference frame and either watermarking the entire frame or some portion of it. The portion of the MPEG2 Transport Stream containing the watermarked data is repeated back-to-back in the asset to be ingested. One ingested asset contains the original video, the other the watermarked version. When it is time to stream to the end-user, the Streamer streams either the watermarked version or the original, as directed by the CDS. If the watermarked version is streamed, it is unique to that session for that end-user and can be traced back to them.

Typically, watermarking assets is used for movies that have just been released.

When ingesting the asset into the Vault, the asset that has been watermarked has a special signature in the PMT, which identifies the PID that has the information about the watermark. The PID that has this information is removed from the asset, as is the PMT with the special signature, to further hide from the end-user that the asset has been watermarked.


Note Watermarked assets cannot be encrypted, and encrypted assets cannot be watermarked.



Note Release 2.5.2 supports watermarked assets automatically. If watermarked assets are to be streamed, all Streamers in a deployment must be upgraded to Release 2.5.2. Streamers that support watermark assets can stream non-watermarked assets. However, Streamers that do not support watermark assets stream them incorrectly.


Multiple Vault Groups by Stream Service

Release 2.5.2 supports associating specific Vault Groups with a specific stream service. Each stream service, high definition (HD), standard definition (SD), live ingest, and Video On Demand (VOD) can have a dedicated Vault Group. The ingest policy configured in the Ingest Steering page determines which Vault Group ingests the content for each stream service. The product ID field in the Ingest Steering page identifies what type of stream service the content is for, and the content type is mapped to the associated Vault Group.

CDSM Configuration

Enable both Ingest Steering and Ingest Manager in the CDSM Setup page. Use the product ID field in the Ingest Steering page to determine the stream service type and associate it with the appropriate Vault Group.

Enhancements

The following enhancements have been added in Release 2.5.2:

New MIBs

Platform Default Baud

HTTP Live Streaming

H.264/AVC Ingest

Playlist Enhancements

Capacity Enhancements

Library Timeout

CDSM GUI Enhancements

Other Enhancements

New MIBs

In previous releases, the CISCO-CDS-TV-MIB had all the configuration objects, monitoring objects, and notifications for the Cisco TV CDS software.

Release 2.5.2 introduces several new MIBs that either add to or replace some of the functionality provided in the CISCO-CDS-TV-MIB The CISCO-CDS-TV-MIB has been modified to remove any objects covered in the new MIBs. Additionally, several new traps have been added, and some objects such as bandwidth statistics have been added to the CISCO-CDS-TV-MIB in the cdstvBwReporting subtree.


Note The new MIBs do not completely replace CISCO-CDS-TV-MIB. There are still elements, such as DNS Setup, Disk Monitor, and MSA Event traps that still exist in CISCO-CDS-TV-MIB, and several new traps and new bandwidth statistics have been added to the CISCO-CDS-TV-MIB.

The latest version of CISCO-CDS-TV-MIB ships with the Release 2.5.2 TV CDS software and should be used instead of the old version.


Table 8 describes the new MIB files and the relationship to the old CISCO-CDS-TV-MIB file.

Table 8 MIB Changes 

New MIB File
New MIB Root
Root OID =
ciscoMgmt.X
Old MIB Subtree that is Replaced

CISCO-CDSTV-SERVICES-MIB

ciscoCdstvServicesMIB

729

cdstvServicesMonitor

CISCO-CDSTV-FSI-MIB

ciscoCdstvFsiMIB

735

cdstvFSISetup

CISCO-CDSTV-INGESTMGR-MIB

ciscoCdstvIngestmgrMIB

739

cdstvConfigIngestManager

CISCO-CDSTV-CS-STATS-MIB

ciscoCdstvCsStatsMIB

743

cdstvStatsMonitor

CISCO-CDSTV-BWMGR-MIB

ciscoCdstvBwmgrMIB

749

cdstvConfigBandwidthManager

CISCO-CDSTV-INGEST-TUNING-MIB

ciscoCdstvIngestTuningMIB

750

cdstvConfigIngestTuning

CISCO-CDSTV-AUTHMGR-MIB

ciscoCdstvAuthmgrMIB

751

cdstvConfigAuthenticationManager

CISCO-CDSTV-SERVER-MIB

ciscoCdstvServerMIB

754

cdstvServerSetup

CISCO-CDSTV-ISA-MIB

ciscoCdstvIsaMIB

755

cdstvISAConfig


CDSM

The new MIB files described in Table 8 are available for download on the CDSM GUI Configure > Server Level > SNMP Agent page.

Unsupported MIB Objects

The following new objects introduced in CISCO-CDSTV-CS-STATS-MIB are not supported in Release 2.5.2:

cdstvCacheCapacity—Use CISCO-CDS-TV-MIB::cdstvDiskTotal instead

cdstvCacheLevel—Use CISCO-CDS-TV-MIB::cdstvDiskUsed instead

cdstvCCPInfromVaultStreams

cdstvCCPInFromVaultStreamBW

cdstvCCPInFromCacheStreams

cdstvCCPInFromCacheStreamBW

cdstvCCPInFromStreamerStreamCount

cdstvCCPInFromStreamerStreamBW

cdstvHTTPInStreams

cdstvHTTPInStreamBW

cdstvCCPOutStreams

cdstvCCPOutStreamBW

cdstvHTTPOutStreams

cdstvHTTPOutStreamBW

cdstvSasBatteryFailed

cdstvSasBatteryNormal

New Traps and Objects

Release 2.5.2 introduces the following traps and objects:

Trap for broken asset

RTSP stream failure traps

Other new traps

Stream control message queue reporting

Session setup success and failure reporting

Thin pipe statistics

Bandwidth reporting statistics

Trap for Broken Asset

The cdstvBrokenAsset trap signifies that one or more assets on a Vault or ISV are broken. A trap is sent whenever the number of broken assets found changes, whether from 0 to n, n to m, or m to 0. The trap contains one object, cdstvBrokenAssets, which specifies the current number of broken assets.


Note The cdstvBrokenAssets value is only valid if the Vault is the master Vault, which can be verified by the cdstvVaultMasterSlaveStatus object.


The broken asset information stays in memory and is not persisted in the database.

Enhancements to the RTSP Stream Failure Traps

RTSP stream failures are reported using the cdstvMSAEvent traps. These traps now contain the PAID or GOID information in the cdstvMSAventErrorCode field.

Other Trap Enhancements

Table 9 describes the other new traps that were added to Release 2.5.2.

Table 9 Other Release 2.5.2 Traps 

Trap
Description
Trap Object
Object Description

cdstvVaultStatusSlave1

This Vault is now a slave Vault.

-

-

cdstvVaultStatusMaster1

This Vault is now a master Vault.

-

-

cdstvSetupIpChanged2

Setup IP address has changed (Streamer and ISV only).

cdstvSetupControlIp

New Setup IP

cdstvControlIpChanged2

Control IP address has changed (Streamer and ISV only).

cdstvSetupControlIp

New Control IP

cdstvDbConnectionFailed

Database synchronization connection from this CDS server to another CDS server has failed.

cdstvDBConnectionFailedIp

IP address of CDS server that failed to synchronize

cdstvLinuxFSReadOnly

Signifies that the Linux partition indicated by cdstvLinuxFSMountPoint is read-only.

-

-

cdstvLinuxFSReadWrite

Signifies that the Linux partition indicated by cdstvLinuxFSMountPoint is now back to normal (read-write).

-

-

1 The cdstvVaultMasterSlaveStatus object is set when the Vault status changes to master or slave; it has two possible values: master(1) and slave(2). A value of 0 means that the status is not yet available from statsd.

2 If Setup IP and Control IP are the same (Setup/Control IP) and both change simultaneously, both cdstvSetupIpChanged and cdstvControlIpChanged traps are sent.


Stream Control Message Queue Reporting

The following objects have been added to the CISCO-CDS-TV-CS-STATS-MIB and are available with an SNMP GET:

cdstvStreamControlMessageQueueMax—Maximum size of the stream control message queue used in CDS

cdstvStreamControlMessageQueueSize—Current size of the stream control message queue; that is, the number of elements in the queue

Session Setup Success and Failure Reporting

The following objects have been added to the CISCO-CDS-TV-CS-STATS-MIB:

cdstvSessionSetupSuccess—Number of successfully setup sessions since the counter was reset

cdstvSessionSetupFailures—Number of unsuccessfully setup (failed) sessions since the counter was reset

cdstvSecondsSinceReference—Seconds elapsed since the last time cdstvSessionSetupSucess and cdstvSessionSetupFailures counters were reset

Thin Pipe Statistics

The cdstvThinPipeConfigTable object added to the CISCO-CDS-TV-MIB provides the objects for thin-pipe statistics described in Table 10.

Table 10 Thin Pipe Statistics

Object
Description

cdstvThinPipeConfigIndex

Index into the table containing the thin-pipe configuration.

cdstvThinPipeName

Name of the configured thin pipe.

cdstvThinPipeEndPointType

Type of the configured thin pipe: CCP or HTTP.

cdstvThinPipeMaxTransmitBandwidth

Transmit bandwidth limit of the configured thin pipe.

cdstvThinPipeMaxReceiveBandwidth

Receive bandwidth limit of the configured thin pipe.

cdstvThinPipeNetPrefix

Subnet prefix of the configured thin pipe (HTTP endpoints only).

cdstvThinPipeNetMask

Subnet mask of the configured thin pipe (HTTP endpoints only).

cdstvThinPipeRemoteArrayId

Remote Array ID of the configured thin pipe (CCP endpoints only).

cdstvThinPipeRemoteServerId

Remote Server ID of the configured thin pipe (CCP endpoints only).


Bandwidth Reporting Statistics

The cdstvBwReportingArrayTable object added to the CISCO-CDS-TV-MIB provides the objects for bandwidth reporting statistics described in Table 11.

Table 11 Bandwidth Reporting Statistics 

Object
Description

cdstvBwReportingArrayId

Array ID of the device the bandwidth statistics are being reported on.

cdstvBwReportingActiveServerCount

Number of active servers in the array from which this device statistics are being reported.

cdstvTotalMaxReceiveStreams

Maximum total receive streams during the last ten seconds.

cdstvTotalPacketsPerSecondReceived

Ten-second average of total packets per second received.

cdstvTotalBwReceived

Ten-second average of total bandwidth received in kilobits.

cdstvCommittedMaxReceiveStreams

Maximum committed receive streams during the last ten seconds.

cdstvCommittedPacketsPerSecondReceived

Ten-second average of committed packets per second received.

cdstvCommittedBwReceived

Ten-second average of committed bandwidth received in kilobits.

cdstvLpMaxReceiveStreams

Maximum low-priority receive streams during the last ten seconds.

cdstvLpPacketsPerSecondReceived

Ten-second average of low priority packets per second received.

cdstvLpBwReceived

Ten-second average of low priority bandwidth received in kilobits.

cdstvTotalMaxSendStreams

Maximum total send streams during the last ten seconds.

cdstvTotalPacketsPerSecondSent

Ten-second average of total packets per second sent.

cdstvTotalBwSent

Ten-second average of total bandwidth sent in kilobits.

cdstvCommittedMaxSendStreams

Maximum committed send streams during the last ten seconds.

cdstvCommittedPacketsPerSecondSent

Ten-second average of committed packets per second sent.

cdstvCommittedBwSent

Ten-second average of committed bandwidth sent in kilobits.

cdstvLpMaxSendStreams

Maximum low-priority send streams during the last ten seconds

cdstvLpPacketsPerSecondSent

Ten-second average of low priority packets per second sent.

cdstvLpBwSent

Ten-seconds average of low priority bandwidth sent.


Platform Default Baud

Previously, the baud on all platforms was set to 115200 bits per second (bps). with the exception of the CDE250 that was introduced in Release 2.4. Most installations required the baud of 9600 bps, which required editing the grub.conf file for every TV CDS software upgrade.

In Release 2.5.2, the cdsinstall script and the cdsconfig script have been modified to support changing the baud without manually editing any files.

Software Upgrade

If the baud needs to be changed for a software upgrade, the user needs to create the file /etc/cdsbaud9600 before running the cdsinstall script. The cdsinstall script searches for the cdsbaud9600 file, and sets the baud to 9600 if the file is found; otherwise, the baud is set to 115200.

If the baud is changed to 9600 before the software upgrade, it will not be changed as part of the Release 2.5.2 upgrade; therefore, there is no need to create the cdsbaud9600 file.

New Installations

For a new installation, the cdsinstall script still requires the following settings on the terminal server serial port:

115200 baud

8 bits

No parity

After the cdsinstall script has completed and the cdsconfig script prompts the user as follows:

Serial Console BAUD speed is configured as '9600'. Do you wish to change it (yes/no) [n]: 
y
Please select the speed:
    1. 9600
    2. 115200
Choice: 
 
   

HTTP Live Streaming

In Release 2.5.2, HTTP Live Streaming is fully supported; similar to live streaming over Cache Control Protocol (CCP). The enhancements to HTTP Live Streaming consist of the following:

Catch-Up to Live

Play While Ingesting the Same Content

Catch-Up to Live

Release 2.5.2 allows a video player to play live content close to the live point, within 2.5 seconds of the live point, without macroblocking or leaving artifacts on the screen of the player.

If play starts at 0 or some point before the live point, then the Catch-up to Live feature allows the end-user to fast-forward to the live point and resume normal play at the live point. The play point will be within 2.5 seconds of the live point.

Play While Ingesting the Same Content

While ingesting the content, a STB can request the content play start at 0, at "play now," or at any specific normal play time (NPT) value between 0 and the live point; and the content will begin playing at the requested point of play.

When a set-top box (STB) sends a "play now" request, meaning the STB is requesting that the play begin at the live point, the "play now" point is within 2.5 seconds of the live point.

H.264/AVC Ingest

This enhancement adds the capability to ingest H.264 video files (in a CBR MPEG-2 transport stream wrapper) in an RTSP NGOD environment. This implementation is compliant with the NGOD index file specification, Comcast-SP-NGOD-CDN-OBJ-I02-101105 which includes updates to support H.264 video.

Playlist Enhancements

Release 2.5.2 supports the following playlist enhancements:

64 content elements (Previously, only 32 content items were supported)

RTSP environments—Failover all content elements in playlists to another Streamer if the Streamer fails (Previously only supported in ISA environments)


Note Interoperability with pre-Release 2.5.2 CDS servers is supported under the condition that no new features are enabled (or expected to work properly) until all CDS servers in a deployment have been upgraded.


Capacity Enhancements

The ISA environment supports 360,000 assets, which could be comprised of two formats for each asset for Dual Conditional Access Systems (CASs) and two preview assets.

The RTSP environment supports 150,000 assets and 200 live channels.

Library Timeout

In Release 2.5.2, the Cache to Application Library Timeout field has been added to the Configure > System Level > MPEG Tuning page.

A network partition could cause the Setup server to wait forever for the remote Stream Groups to respond to the application for setup requests. The Library Timeout sets the time interval (in microseconds) that the SetStreamInfo API should wait before considering the remote Stream Group unavailable. The range is from 1000 to 5000. The default is 2000 (2 seconds).

CDSM GUI Enhancements

The CDSM GUI has the following enhancements and changes in Release 2.5.2:

Reports and Monitoring Graphs

Audit Log FIltering

Completed Ingest Page

New Server Offload States

MPEG Tuning

Interface Setup and Server Setup Pages

Reports and Monitoring Graphs

The reports and monitoring graphs have been updated to offer graph and grid. The Disk Monitor graph and the NIC Monitor graph have this change and the following reports have this change:

Streams by Array

Streams by Time

Stream Failures

Content Popularity

Previously, the Stream Play History report displayed the content when the user clicked the Session ID; now if playlists are associated with the session, the playlists are displayed with details about the trick-mode history (If the Trick Mode feature is enabled).

Audit Log FIltering

Release 2.5.2 introduces two-tier filtering for the CDSM Audit Log reports that allows the display of a specific area of CDSM access activity. The top-level filter consists of the following options:

Configure

Monitor

Maintain

Auto System Cleanup

All Other

The sub-level filter offers the ability to further narrow the selection of the CDSM Audit Log and has the associated sub-levels of each top-level (for example, Configure has System Level, Array Level, and Server Level).

Completed Ingest Page

Content Status list option has been removed.

New Server Offload States

Release 2.5.2 introduces a new server offload state for Vaults and ISVs. The server offload state for Vaults and ISVs is set by choosing Maintain > Servers > Server Offload, and selecting one of the following options:

Online

Offline (No Ingest)

Offline (No Ingest & Fill)

The Offline (No Ingest) option enables the Vault or ISV to continue handling cache-fill requests and mirroring activities, but the server does not participate in any new content ingests. The Offline (No Ingest & Fill) option stops all cache-fill requests and any new content ingests, but the server still participates in mirroring activities.

MPEG Tuning

The MPEG Tuning page includes the Playlist Trickmode Restriction settings.

In Release 2.5.2, the Dual CAS field is added. Previously, dual CAS was enabled when Shared Content Store was enabled. This configuration setting is now added to the CDSM GUI. In addition to the system-level setting (Configure > System Level > MPEG Tuning), the Dual CAS field is also configurable at the server level (Configure > Server Level > Server Setup).


Note This field is only available in ISA environments with Stream Destination set to IPTV.


Cache to Application Settings section has the Library Timeout field, which has a range from 1000 to 5000 seconds, and a default of 2000 seconds.

Interface Setup and Server Setup Pages

The interface configurations are moved to the Interface Settings page in Release 2.5.2 to separate the interface configuration settings from the server configuration settings, and configure interface settings in one single step.

The Server Setup page in Release 2.5.2 has the configurations only for the server settings. All the interface configuration settings are now in the Interface Setup page. The FTP listener configuration is changed to a drop-down list from which an interface can be selected to be configured as the FTP Listener.

Other Enhancements

Table 12 lists the other enhancements that have been added in Release 2.5.2.

Table 12 Release 2.5.2 Additional Enhancements 

Enhancement
Description

Error message for setting tunable

The following error messages are returned when attempting to set the tunable to an incorrect value (CSCtr29217):

Return -EINVAL (Invalid argument) when writing to a writable proc file with an invalid value.

Return -EPERM (Operation not permitted) when writing to a read-only proc file.

Redirect Server Troubleshooting

The following enhancements have been added for troubleshooting the HTTP redirect service (CSCts08580):

Added a message to log the event in which a server stops being a backup server.

When a primary server attempts to select a backup server but fails, write to the log the reason why each candidate server is rejected.

Add two new files for troubleshooting:

/proc/calypso/status/resiliencystatus

/proc/calypso/status/resiliencyinfo

These files contain information about the service address manager's configured addresses:

For Streamers, the contents of these files is identical to those with the same name in the /proc/calypso/status/streamer/ directory.

For Caching Nodes, the contents shows information about the HTTP Redirect service address (including its state, such as Primary, Backup, Available, NotUsable, and so on).

CDSM GUI—DNS and NTP configuration

Added Push Config button to the following CDSM GUI pages, which forces the configuration to all servers (CSCtj80087):

Configure > System Level > System DNS

Configure > System Level > System NTP Server

Information on Control and Setup servers added to c2k.log

Each Streamer now writes the identifier of the primary and backup Control and Setup servers to the c2k log at the notice level. By examining the c2k log when troubleshooting, you can track which server was primary or backup during a given time period (CSCtj38978).

H.264 content

Support ingesting H.264 content that has the sequence scaling matrix flag on (CSCtt32347).

Cache-fill troubleshooting

Added "Tracking Cache-Fill Source" section to the Troubleshooting appendix. (CSCti08765)

FSI server

The FSI server returns "400 Bad Request" if malformed XML messages are received (CSCtq14392).

ISA ContentStoreMaster log

The ContentStoreMaster log has been enhanced to provide more detailed messages for failed ingests (CSCtj33310).

End of stream pause

Added pause at end-of-stream. which allows user to rewind after the end-of-stream (eos) has been reached (CSCto75302).

CDSM GUI—NIC Monitoring

Previously, the NICs listed on the Monitor > Server Level > NIC Monitor page were listed in the order they were brought up.

In Release 2.5.2, the NICs are listed in order from eth0 to the highest numbered configured Ethernet port on the CDE (CSCtr62978).

CDSM GUI—display SCTE-35 cue message

CDSM GUI displays the SCTE-35 cue messages with content information. There is a button on the Completed Ingest details page that shows the SCTE-35 cue messages (CSCts19229).

Multiple Master Vault Groups

Release 2.5.2 supports multiple Master Vault Groups.

ISA Range Play

Release 2.5.2 supports playing a range request in ISA environments.


Supported Environments

Release 2.5.2 of the Cisco TV CDS supports both the ISA and RTSP environments. Some RTSP environment features are only applicable to certain RTSP Deployment Types.

System Requirements

The Cisco TV CDS Release 2.5.2 runs on the CDE110, CDE220, CDE250, and CDE420. See the Cisco Content Delivery Engine110 Hardware Installation Guide, and the Cisco Content Delivery Engine 205/220/250/420 Hardware Installation Guide.

The Cisco TV CDS Release 2.5.2 also runs on the CDE100, CDE200, CDE300, and CDE400 hardware models that use the Lindenhurst chipset. See the Cisco Content Delivery Engine CDE100/200/300/400 Hardware Installation Guide for set up and installation procedures.

Release 2.5.2 does not support the CDEs with the ServerWorks chipset. All CDEs with the ServerWorks chipset need to be replaced with the CDEs with the Lindenhurst chipset or the Next Generation Appliances (CDE110, CDE220, CDE250, andCDE420) before upgrading to Release 2.5.2.


Note If you are using Internet Explorer 8 to access the CDSM GUI, for the online help to display correctly, you need to turn on Compatibility View. To turn on Compatibility View in Internet Explorer 8, choose Tools > Compatibility View.


Limitations and Restrictions

This release contains the following limitations and restrictions:

RTSP Redirect Server Limitations—The TV CDS Redirect Server provides a central entry-point for stream requests, which are then redirected based on internal load balancing policies to other Setup servers in the CDS deployment for session processing. The load balancing policies require that the CDS server running the Redirect Server have access to the database of each potential Setup server. The policies use key pieces of information from the database, such as the RTSP server status, the server load, and the Setup IP address to management IP address mappings. If the Redirect Server does not have access to this data, it cannot properly redirect the setup request to the "best" Setup server.

In the case of multiple Stream Groups, with the Redirect Server configured to run on Streamers within one of the Stream Groups, it is required that the database replication partners for the Streamers contain all of the Streamers from all Stream Groups, which basically constitutes a full-mesh database replication approach among all of the Streamers

If a full-mesh database replication for the Streamers is not desired, the Redirect Server can operate on the Vaults, which already have the necessary database replication partners defined to support required data.

Open Caveats

This release contains the following open caveats:

CServer

CSCtu37690

Symptom:

System crashes.

Conditions:

Longevity test.

Workaround:

Update to the maintenance release.

CSCtt22964

Symptom:

Transmit unit hang detected on e1000 interfaces during a server bring-up soon after upgrading from Release 2.3.3.

Conditions:

Happens only after upgrading the software from Release 2.3.3 during server bring-up on servers with e1000 adapters in quad formation and specific adapters on that quad configuration.

Workaround:

Rebooting a second time fixes the issue. This is a know issue with Intel drivers. Following is the excerpt from the driver README:

Detected Tx Unit Hang in Quad Port Adapters
-------------------------------------------
In some cases ports 3 and 4 don't pass traffic and report 'Detected Tx Unit
Hang' followed by 'NETDEV WATCHDOG: ethX: transmit timed out' errors. Ports 
1 and 2 don't show any errors and will pass traffic.
 
   
This issue MAY be resolved by updating to the latest kernel and BIOS. The 
user is encouraged to run an OS that fully supports MSI interrupts.
 
   

CSCtr46736

Symptom:

Primary-backup status conflict in some servers.

Conditions:

Network partition.

Workaround:

None.

CSCtt02130

Symptom:

Server goes into kernel debugger (KDB) mode after a "service network restart" is issued or an individual adapter is restarted.

Conditions:

It only happens when the adapters are shared with the Linux operating system.

Workaround:

None.

CSCtr15443

Symptom:

Evaluators shown as continuously active; they never seem to complete.

Conditions:

A disk drive that is bad or failing, yet not sufficiently bad to be declared "sick" by the disk driver continues to go in and out of "repair" mode, causing the evaluators to constantly restart and inhibiting the evaluators from completing.

In addition, the evaluators need to have a significant amount of local work that needs to be done (such as defragging or smoothing). If the amount of work is small, this symptom may not be exhibited because the evaluators are able to complete the task during the small windows of time that the disk drive is not in repair mode.

Workaround:

Remove and replace the bad disk drive.

CSCtu12699

Symptom:

When the primary Control server loses link state on all of the fill ports, but does not lose link state on the management network port, it will continue to hold the virtual IP address on the management network for the control service. This will allow other servers that may still have fill link-state to become the primary server because the virtual IP address for the control service is no longer in use by the original primary. After another server becomes primary it will make sure that all streams are playing on servers which still have fill link-state. Later when the original primary server gets link state again, the two primary servers detect that there is a conflict and the primary server that has most recently communicated with the backoffice will win and the other server will stop being primary. If the new primary server never communicated with the back end, then the original primary server will take over as primary again. However, because it does not know about the stream state changes that happened while it did not have fill link-state, some of the streams can be lost when the original primary takes over again.

Conditions:

The primary control server must lose link state on all of its fill ports, but not lose link state on its management network port. Another server must become primary, but never communicate with the back office. When the original primary gets fill link state again it will take over as the primary again.

Workaround:

If the primary server has lost link state on all of its fill ports and another server has become primary, you can prevent the original primary from taking over again by issuing the following command so that the server will stop being primary:

echo 1 >/proc/calypso/tunables/stop_being_primary
 
   

Starting or stopping a stream using the new primary server while the original primary server has no fill link-state will also prevent this issue from occurring.

CSCtu19076

Symptom:

System KDB may occur.

Conditions:

Removing a front-mounted device before the logical preparations for removal are complete.

Workaround:

Before removing a functioning storage device from the front of the server, ensure that it has first been logically removed by echoing the device name to /proc/cds/cdd/remove_device. For example,

echo csd3 > /proc/cds/cdd/remove_device prepares the device csd3 for removal from the server.

Ensure that the message "csdX can now be safely removed" (where X is the device number) appears on the console before pulling the device. Also, the red LED on the device will be flashing after it has been prepared to be removed.

Following these recommendations will avoid a possible system KDB event.

CSCts35941

Symptom:

During a failure of the Streamer hosting the primary Control server, occasionally part of the playing streams could be lost. This loss happens because of communication problems between the Streamers, causing confusion over which Streamer was the primary Control server, and also in some cases preventing the backup server from receiving the current state for all playing streams.

Conditions:

Loss of streams during a primary Control server failure are most likely to occur when a new Streamer has been brought online shortly before the primary Control server failed. Code that should have been sending switch forwarding probes on the ports for the new server to determine if the router forwarding table was sending and receiving traffic on the ports, before using the ports for live video and control traffic, was incorrectly making the ports available for use before the switch forwarding table was properly setup for sending and receiving traffic on these ports. This caused a very high percentage packet loss for communication traffic to the new server. The new server could then mistakenly believe there were no other primary Control server and could temporarily take over the role of primary Control server with no stream state. While in this mode, the new Streamer would issue commands to tear down all the streams that were playing on Streamer that it could communicate with.

A second problem could occur when the new Streamer did see the current primary server and the new server was the second Streamer to come up. There is a time window, during which the new Streamer has not finished bringing all of its ports online and had only one or two active ports, where the primary server could decide to use this new Streamer as the backup server. The primary would then send the new Streamer stream state information in parallel from multiple ports on the primary server exceeding the capacity to deliver the information being sent from many ports to the one or two active ports on the receiving Streamer. This would result in a high level of packet loss, and retransmitting the packets would continue to fail until the Streamers declared each other unreachable. The backup server would then see the primary as unreachable, but would only have state for part of the playing streams. If the primary server really failed so that the primary control IP address was no longer reachable over the management network, the backup server could become primary without having state for many of the stream.

These problems are much more likely to occur when the switch being used has a significant delay when a new port comes online, before the switch forwarding table is setup to forward traffic to and from the new port. The problem is also dependent on the timing between when a new Streamer comes online and when the existing primary server fails.

Workaround:

When possible, avoid shutting down the primary server if another Streamer has just come online. Wait for a couple of minutes after bringing the new Streamer online to make sure that it has had time for the switch to enable communication on all of the ports.

Some switches have configuration settings that can be modified to reduce the time delay between when a new port first detects link state and when the switch sets up the forwarding table information to send and receive traffic on the port. Reducing this time delay will help minimize the window in which this problem can occur.

CSCtt22591

Symptom:

Usually the first symptom is the presence of "WARNING: Stream transmit..." errors in the protocol timing log. This indicates that the LAN ports are getting behind.

Next some streams are dropped to reduce the load on the server, and some streams may end up getting changed to a NULL state-see the "Nulls:" part of the "Active streams:" lines in the protocol timing log to track this.

Conditions:

This appears to happen when the LAN channel has a total combined bandwidth (transmit and receive) of 41 to 46 Gbps.

Workaround:

There currently is no workaround other than to avoid having the server support both a maximum fill-bandwidth load and a maximum transmit-bandwidth load at the same time. It takes a well balanced load to do this; usually a server reaches a maximum on one or the other instead of both at the same time; so it is not likely you will actually encounter this problem.

Release 2.5.2 has the following LAN transmit and receive maximum limits:

Maximum transmit-bandwidth for a server with 4 10-gigabit Ethernet adapters is around 37.5 Gbps.

Maximum fill receive-bandwidth for a server with 4 10-gigabit Ethernet adapters is around 9.45 Gbps.

CSCtn28329

Symptom:

In the CDSM (or VVIM) GUI, on the Server Setup page (Configure > Server > Server Setup), when specifying an interface for the FTP Out Interface field under the FTP Out Setting section, the configuration setting does not take affect and the specified interface is not used for FTP out operations.

Conditions:

Using the CDSM (or VVIM) GUI, specify an FTP out interface and submit the Server Setup page. The setupfile on the server will be updated to reflect this configuration, as in following example:

ftpout if eth0 max utilization mbps 0 max sessions 0
 
   

Workaround:

The IP address of the interface to be used in FTP out operations can be specified on the ftp command line.

Configuration

CSCti78219

Symptom:

Not all packages are found in the drop-down list on the Monitor > System Level > Package Monitor page.

Conditions:

When the number of packages in the system reaches a level greater than 70,000.

Workaround:

If the number of packages in the system grows much beyond 70,000, then the drop-down list may not be able to display them all. The solution is to delete packages that are no longer needed. The maximum number of packages that will display in the drop-down list is variable, and is dependent on the system that is accessing the web page.

FSI

CSCtt46144

Symptom:

For a duplicated live future request, the state field in response XML body is empty instead of having the correct state of the scheduled live recording.

Conditions:

The client sends a request to schedule a live content with the same assetID, providerID, captureStart and captureEnd as an already scheduled future recording on the same channel.

Workaround:

None.

Monitoring

CSCtu23071

Symptom:

The following symptoms have been observed:

1. Monitor > Server Level > Disk Monitor shows a different failed disk than the Disk Activity graph.

2. When using some web browsers, the Disk Activity graph may reload repeatedly.

3. The Disk Activity Graph may show zero read/write activity if the disk activity is greater than100 Mbps.

Conditions:

Any CDE250.

Workaround:

For item one, there is no workaround. For item 2, using an older browser (IE6) should allow the user to see the Disk Activity graph. For item 3, when viewing the graph, the user can click on the icon to switch between the graph and a table display of the values to see if any disks are working at greater than 100 Mbps.

Statsd

CSCto43335

Symptom:

Caching Node remote servers file contains all servers from old group.

Conditions:

Moving Caching Node from group A to group B.

Workaround:

Restart statsd on the Caching Node that was moved and log in to the CDSM GUI and submit the Server Setup page for that Caching Node.

Resolved Caveats

This section contains he resolved caveats in Cisco TV CDS Release 2.5.2. Not all resolved issues are mentioned here. The following list highlights resolved caveats associated with customer deployment scenarios.

CServer

CSCto88453

Symptom:

Cserver trap inside method IOX::StreamEventLog::writeEntry()

Conditions:

There is a small timing window inside the slot code where a slot that is being deactivated may also be selected during a transition of the same stream. The slot is then left in an invalid state.

CSCtq12865

Symptom:

Divide by zero trap was executed inside method LOM::SmootherEvaluator::updateStatus.

Conditions:

This trap was caused by there being no (that is, zero) drives available in the server when the smoothing evaluator executes this routine. The drives were unavailable because the mptsas driver reported that it had lost the SSP device. This is actually a serial protocol engine on the SAS controller. It suggests that an event of some sort took place that put this controller into a bad state that subsequently caused the storage devices to become inaccessible.

CSCtq21474

Symptom:

Customers can report asset not found errors, or incorrect video playback for titles. This is because of avsdb on Vaults being put into a compromised operational state.

Conditions:

In NGOD environments, this condition can occur when an unvalidated C2 client sends an invalid locate request to the primary Caching Node that is hosting the C2 locate port. An invalid locate request can have missing or empty AssetID, ProviderID and SubType tags in the XML body of the request which can trigger the compromised avsdb state on the Vaults.

CSCtr13314

Symptom:

Trap indicating "Bug: soft lockup detected on CPU#2!" message.

Conditions:

There is a very small timing window that has only been seen at one CDS site. The issue is caused by a deadlock between processors 1 and 3.

CSCtr48734

Symptom:

The test_adapters.pl script used by network operations and Cisco Advanced Services for TV CDS network connectivity validation may give false positive results.

Conditions:

The test_adapters.pl script is used to validate basic connectivity with ping from the CServer cache-fill ports to the default gateway for those cache-fill ports. If a route exists over the management network to the default cache-fill gateway, the test_adapters.pl script can return a false positive indication of connectivity.

CSCts46747

Symptom:

Server goes into kernel debugger (KDB) mode at avs_cserver:Cif::ReadIndexHeader+0x44d/0x1df0

Conditions:

Cif::ReadIndexHeader() fails to check if the subFileIdx is out of bounds. A corrupt CIF file had a subFileIdx of 0x20000004 !! This was part of a network outage that occurred.

CSCtr46415

Symptom:

HTTP fill requests are being canceled, because a Caching Node believes it is experiencing lost data when filling content from the Vaults. The Caching Node prints the warnings "HTTP TRANSFER NO CAPACITY" and "HTTP TRANSFER: Lost packet pressure," and cancels the incoming HTTP fill requests even though the content might already be cached locally. The Caching Node thinks it was under lost packet pressure because it has to refetch more data than usual. Actually, the data is not actually lost, so you will also see "WARNING: Dup Packet Ratio" warnings in the protocol timing log that match very closely the packet loss warnings "WARNING: Packet Loss Ratio."

This could also be a problem with CCP Streamers filling from Caching Nodes or a Vaults.

Conditions:

Fill paths to the Vaults are not balanced, some paths take more time and others less time to reach the Vaults, the incoming data packets are coming in out-of-order, more than usually expected, causing the Caching Node to detect holes in the incoming data and to fetch the data again.

CSCtt26487

Symptom:

Low priority connections are not being tracked separately from normal priority connections, which rarely results in streaming being affected on the low priority server. This issue was aggravated by three things:

1. A separate issue in the ingest logic was errantly causing nearly all ingests to be assigned to a single Vault.

2. The excess ingest load because of item1 resulted in exhaustive pressure to the thin-pipe, causing the replication efforts to get far behind.

3. A large number of Vaults allowed the low-priority code to open many connections (up to 400 per remote Vault). The end result is that the system ran out of capacity because it was exceeding the 3000 outbound connection limit.

Conditions:

Excessive ingest load (compared to the available thin-pipe bandwidth) causes mirroring to get behind sufficiently that it consumes all available connections; thus starving the streaming traffic.

CSCtr15985

Symptom:

GroupID 0 and 65535 defaulted to group id 1.

Conditions:

User defined in setupfile.

CSCts56418

Symptom:

Highly intensive input-output job could stall the system and so other applications could suffer from iowait.

Conditions:

In most of the cases, the database transactions taking too long (10 seconds or more).

CSCtq86637

Symptom:

A Caching Node does not have free disk space for new content, and the available disk space on a Caching Node stays at 0 percent.

Conditions:

This condition is only encountered on Caching Nodes running Release 2.3.3-ES6.

CSCtj84853

Symptom:

The Stream Setup servers are failing intermittently because CServer is failing "SetStreamInfo" for a particular control, which is not reachable any more because of connectivity issues on the caching network.

Conditions:

The caching network between the primary Setup Streamers and the primary Control Streamer is partitioned.

CSCtq90254

Symptom:

In an NGOD environment, a Caching Node can crash and generate a core file.

Conditions:

When processing C2 locate requests, a long delay introduced in processing (such as a network that is loosing packets), or delay introduced by CSCtr13207 can cause the Caching Node to reference invalid memory and page fault.

CSCtr61761

Symptom:

When upgrading 2.3 servers that used a groupID of 201010 to Release 2.5.1, all the 2.3 servers were using an effective group ID of 4402, but the 2.5.1 servers used a group ID of 1. This caused problem during the upgrade because all the new servers could not communicate with the older servers. Because backward compatibility was restored in Release 2.5.2, there was the potential of a similar problem when upgrading from Release 2.5.1 to Release 2.5.2.

Conditions:

The conditions required to create the problem are following

Running a mixed version environment where one of the versions is specifically Release 2.5.1

Using a group ID greater than 65535

CSCtq32515

Symptom:

Server crashes when an invalid memory is referenced.

Conditions:

Possible race condition between physically removing a device and Linux proc file system cleanup.

CSCtj23849

Symptom:

Extra STUN handshakes after certain mid-stream events. This did not prevent the system from working, it just added extra overhead.

Conditions:

Occurs in NAT TCP/IP environments when a stream is started at a non-zero offset or at a speed other than 1x, or there is a switch to a trick mode during play.

CSCtr38460

Symptom:

When the streaming bandwidth is less than 10 percent of CDN volume bandwidth, RTSP server sends out an incorrect announce code of 771-Asset Not found.

Conditions:

When there are large number of streams running and the available volume bandwidth of CDN falls to 10 percent.

CSCtr38538

Symptom:

The RTSP client receives a "778 Session Setup Failed" error when the Streamer exceeds its bandwidth limit instead of a "453 No Bandwidth" error.

Conditions:

The Streamer is at or near its streaming bandwidth capacity.

CSCtr47527

Symptom:

The information displayed in the "/proc/calypso/status/streamer/resiliencyinfo" and "/proc/calypso/status/streamer/resiliencystatus" files may appear confusing, which may lead users into believing there is an operational problem when there is not.

Conditions:

The information displayed may be confusing if the configured service addresses have recently been reconfigured (added, modified, or deleted).

CSCtr60505

Symptom:

The trick play session history reported by the RTSP server does not match the actual sequence of play requests.

Conditions:

Play requests are issued close to content boundaries such that the result of the play request is to start playing the next content.

CSCts14875

Symptom:

When the sequence of play commands (rewind, pause, play) is entered quickly, stream playback fails. A large offset for the play command is observed in the log files.

Conditions:

When a sequence of three or more play commands is sent to the Streamer with less than 500 milliseconds between the first command and the last command, the commands between the first and the last are ignored and immediately canceled. The response and callback of ignored requests may be issued before the response/callback of the first request. If the first request is not complete, and an ignored request would have affected the current NPT (for example, a jump request), the active NPT is prematurely overwritten. Also, the NPT for an ignored pause is incorrectly treated as a jump command.

CSCts27549

Symptom:

When fast forwarding to the live point, the return NPT is unpredictable. It may return the NOW NPT or the current NPT. A subsequent play request using the returned NPT may result in the stream playing from the beginning of the recording.

Conditions:

Playing a live recording.

CSCtt06305

Symptom:

HTTP Streamer crashes trying to print a warning message "WARNING: Configuration error: Received standard sized network packets from server...when configured for jumbo frames." from the routine ROM::ReadRateControlBlock::deliverData().

Conditions:

Live streams are being sent and jumbo packets are enabled on the Streamer.

CSCtt23195

Symptom:

When a session is torn down the ODRM SessionHistory does not contain all of the required headers for the StreamResource and UserEvent elements; specifically, attributes associated with trick-mode restrictions.

Conditions:

Any TV CDS release prior to Release 2.5.2 that uses playlists and trick-mode restrictions.

CSCtr13207

Symptom:

Temporary processing backlog and high CPU utilization can be experienced when events occur that result in large amount of data being logged to the serial console.

Conditions:

This is primarily an issue when the serial console baud is changed from the default value of 115200. When the TV CDS services start on a system with a large number of servers, a message is printed to the server console for each remote server to which it connects, resulting in a large number of output lines being logged to the serial console. This large number of serial console output lines results in a significant processing delay, which can trigger the conditions described in CSCtq90254.

CSCtq30644

Symptom:

CServer fails if some content objects have big picture gaps.

Conditions:

If the input video content has big picture gap.

CSCtq98743

Symptom:

FSI returns 409 when ingesting a content that is already ingested and previous ingestion is not completed. This is the expected behavior from NGOD A3 specification.

Conditions:

Ingesting a content that is already ingested and previous ingestion is not completed.

CSCts16976

Symptom:

Live recording delayed one second from the scheduled time.

Conditions:

Live recording.

CSCtn83177

Symptom:

On the CDE220 and CDE420 (Generation Two hardware), the server hangs when four drives are pulled out and then reinserted.

Conditions:

There are two steps to see this issue:

1. Four drives must be removed from the system.

2. Those four drives are then reinserted without waiting for each drive to be recognized.

Once this is done the server crashes and fails to operate until rebooted.

CSCtq13470

Symptom:

In Release 2.5.1, real-time acquisition (RTA) ingest is limited to 120 channels per Vault.

Conditions:

When ingesting we are limited to 120 channels per Vault instead of 160 in previous releases.

CSCtl20318

Symptom:

When a Streamer requests a transfer of an object from a Cache Group by way of the C2 protocol, the Streamer may receive an error back from the Cache Group indicating that no server (or TransferPort) is available within the group to transfer the requested object.

This may happen if, for example, a Streamer experiences a connectivity issue (for example, TCP reset) during a Transfer request, and adds the affected IP address of the server's interface to an ExclusionList in a subsequent Locate request. If there are no other servers in the Cache Group to process the request, then the request will be rejected.

Conditions:

This may occur if the both following exists:

1. A Streamer includes one or more IP addresses in an ExclusionList element of its Locate request.

2. The excluded IP addresses map to all servers within the Cache Group; that is, at least one IP address belonging to each server in the Cache Group appears in the ExclusionList.

CSCth80826

Symptom:

No support for Live Ingest and Streaming for CDN.

Conditions:

Live Streaming of C2 Index formatted files.

CSCtk56148

Symptom:

No support for Live Ingest and Streaming for CDN.

Conditions:

Live Streaming of C2 Index formatted files.

CSCth80826

No support for live ingest and streaming for CDN.

Conditions:

Live streaming of C2 Index formatted files.

CSCtk56148

Symptom:

No support for Live Ingest and Streaming for CDN.

Conditions:

Live Streaming of C2 Index formatted files.

CSCtn99384

Symptom:

Vault crashes when number of content is above 300,000 titles.

Conditions:

When ingesting more than 300,000 titles, the system crashes.

CSCtl45933

Symptom:

The symptoms of this problem are that all fill streams are in cut-through mode (not being written to the local file system) and the disk channel is at a relatively low utilization. This results in a less than optimal caching efficiency since a local copy of the data will not be retained on disk.

Conditions:

This has only been seen when running a CDE 250 2G3 system as a streamer, and when running with a maximum fill load of around 5000 standard definition streams. If you are running with a lighter fill load or a different server, you should be able to avoid the problem.

Cache2App

CSCto75156

Symptom:

The ISA StreamService keeps losing threads on the cache2app APIs and eventually gets restarted. The network partitioning event could cause the Setup server to wait forever for remote Stream Groups and not respond to the application for setup requests.

The setup application blocks or loses a thread on each setup request and eventually after losing 16 threads, the monit script restarts the ISA StreamService.

Conditions:

Some network partitioning event could cause CCP packet loss, which could cause the setup application to lose all the threads or cause the Setup server to wait forever.

CSCti00738

Symptom:

CalypsoContentStoreServer process on the Vault restarts.

Conditions:

When an ingest provisioning request comes in and the backoffice is unreachable at the time the backoffice is queried for the pre-encryption details. This query is made within a second or two from the initial ingest request.

CSCtk64351

The ISA StreamService keeps getting restarted. The streams were getting stuck, so destroy stream API internally to the ISA/cache2app blocking a thread and eventually after losing 16 threads, the monitor script restarts the ISA StreamService.

Conditions:

There is an issue with the Primary or backup Control streamer that causes streams to get stuck.

Configuration

CSCtk98236

Symptom:

In the CDSM GUI, Configure > Array Level > Barker Setup page, the number of contents available for selection in the drop-down list is limited to 100.

Conditions:

This issue is encountered in both ASI and GigE streaming modes when the system has more than 100 completed ingests.

CSCtr00467

Symptom:

In the CDSM GUI, Configure > System Level > QAM Setup page, a multicast IP address is not accepted as QAM IP.

Conditions:

Add a new QAM Gateway in the CDSM GUI, enter a multicast IP address as the QAM IP, submit the page. The CDSM GUI displays an error message stating that the QAM IP should be a valid unicast IP. The page cannot be submitted.

CSCtt02077

Symptom:

The log priority set by the LogLevel directive in the httpd.conf file logs all information messages in the error log file.

Conditions:

Normal conditions.

CSCtr61086

Symptom:

There is no indication in the CDSM GUI whether a default log configuration is being displayed or whether a log configuration exists for the server.

Conditions:

After deleting the log configuration for a server in the Configure > Server Level > Logging page, when the user navigates to logging page to configure logging for the server, the default log configuration is displayed from the XML file.

CSCtr72190

Symptom:

On the Input Channels page (Configure > System Level > Input Channels), choose an existing channel or add a new one. Set Category ID different from Catalog ID, and click Submit.

The message shows "Channel update complete;" however, the Category ID value is always the same as Catalog ID instead of the value that was inputted.

Conditions:

On the Input Channels page (Configure > System Level > Input Channels) in the CDSM GUI, set the Category ID different from the Catalog ID, and click Submit.

CSCtr72090

Symptom:

On the Input Channels page (Configure > System Level > Input Channels) in the CDSM GUI, choose an existing channel or add a new one. From the Audio Type drop-down list, select an audio type other than Mono; for example, select Stereo, and click Submit.

A status message displays "Channel update complete;" however, the Audio Type is still displays Mono.

Conditions:

On the Input Channels page (Configure > System Level > Input Channels) in the CDSM GUI, choose an existing channel or add a new one. From the Audio Type drop-down list, select an audio type other than Mono; for example, select Stereo, and click Submit.

CSCto76316

Symptom:

When the CDSM is downgraded from 2.5 to any earlier releases, the servers' IP address and Interface types are missing in the Server Setup and Interface Setup page

Conditions:

1. Upgrade CDSM to Release 2.5.1.

2. Add a server and configure it.

3. Downgrade to an earlier release.

4. The IP address and Interface type configuration for the new server added in Release 2.5.1 are missing in the GUI.

CSCti70956

Symptom:

Cannot enter host names containing "." through the CDSM GUI. Error message is displayed preventing entry.

Conditions:

The form field validation does not allow for "." in Host Name field.

Database

CSCtq21453

Symptom:

C2 Locate requests begin failing at an alarming rate on Streamers with errors indicating Cisco Index files. Streamers are periodically encountering problems with data returned from Caching Nodes.

Examining the C2 requested and correlating the returned GOIDs with the Persistent Store Asset-to-Goid.py script found that the Caching Nodes were returning GOIDs and data that did not belong to the requested PAID.

At times, the Motorola streamers would reject the C2 Transfer responses from the Caching Nodes and internally fail the stream with INGEST_CDN_INDEX_FILE_CHECKSUM_ERROR.

Other times, the Motorola streamers would reject the C2 Transfer responses from the Caching Nodes and internally fail the stream with Content window disappeared- INGEST_CDN_IFRAME_MISMATCH.

In some instances, Cisco Streamers rejected returned data from the Caching Node with the following errors:

:crt:ERROR: IGate::GetMetaDataFormat: Unknown file format.

:err:ERROR: IGate::Read8k: Unrecognized metadata format: defaulting to 8K

:crt:ERROR: IGate::GetMetaDataFormat: Unknown file format.

In at least one instance, a Streamer requested an asset and was given MPEG data for a different asset. This session was not a Playlist session.

Conditions:

Invalid Caching Node PAID request made to AVSDB.

CSCtr82415

Symptom:

The Redirect server stops on the master Streamer and a core dump is generated. Cannot perform VOD setups.

Conditions:

Greater likelihood of occurring in high load situations.

CSCts56119

Symptom:

The Redirect server stops on the master Streamer and a core dump is generated. Cannot perform VOD setups.

Conditions:

Greater likelihood of occurring in high load situations.

CSCtg93593

Symptom:

Replay.db grows to excessively large size, eventually running the filesystem out of disk space.

Condition:

An extended outage event such as network partitioning betweenStreamers and Vaults or CDSM can cause a large number of error events (and SNMP traps), which are sent to the CDSM and they become queued on the source device, which causes this issue.

CSCtk84103

Symptom:

Database shutdown process took prolonged time for 73 minutes on the master Vault. During shutdown process, database is unavailable on the master Vault.

Conditions:

Database has a big number of assets.

CSCtl03347

Symptom:

When viewing the avsdb.log file, messages of the following sort may appear which may seem to indicate an error has occurred:

xx-xx-2010 01:32:50PM: CONTENT_OBJECT_GETALL_RATELIMIT for sockfd=195 starting
xx-xx-2010 01:32:50PM: CONTENT_OBJECT_GETALL_RATELIMIT for sockfd=195 exiting due to 
counter = 1 >= limit = 1
xx-xx-2010 01:32:50PM: CONTENT_OBJECT_GETALL_RATELIMIT for sockfd=195 send 9999 failed
xx-xx-2010 01:32:50PM: CONTENT_OBJECT_GETALL_RATELIMIT for sockfd=195 returning with 
error
 
   

A log of "send 9999 failed" appears after a mention of exiting due to a counter reaching its limit of 1, and then a "returning with error" message for the same sockfd appears.

Conditions:

These messages appear during content object resynchronization, which will also take place during node addition and patrol functionality. The messages would appear on the master Vault/SSV from which another server is performing the content object resynchronization.

FSI

CSCtr55999

Symptom:

FSI server crashes.

Conditions:

The A3 TransferContent request contains an ampersand (&) or other HTML entities; such as null quotes ('"'), escaped apostrophe (&apos;), the greater-than symbol (<), or the less-than symbol (>), in the source URL and the ingest request is dispatched to the remote FSI server.

CSCts19867

Symptom:

The GOIDs of the new ingest are deleted while ingest still succeeds. The content is not serviceable. As seen in the fsi.log, one thread is deleting the content while the other thread is ingesting the content.

Conditions:

The new ingest comes immediately after FSI restarts.

CSCtt23346

Symptom:

The FSI always selects the local server to ingest VOD if it failed to dispatch VOD request to remote server. This causes a high load stress on the master Vault.

Conditions:

This may occur under the following conditions:

Remote Vault is unreachable

FSI process on remote Vault is too busy to respond in time

FSI process on remote Vault is not running

CSCtr19755

Symptom:

The master Vault goes into kernel debugger (KDB) when ingesting live content, and the content still displays on the Monitor > System Level > Active Ingests page after one day.

Conditions:

The master Vault goes into kernel debugger (KDB) when ingesting live content.

CSCtk69310

Symptom:

The NGOD A3 live recording TransferStatus asynchronous notification message does not contain the correct bit rate value (for example, bitrate = 0).

Conditions:

For NGOD A3 live recording, all the status asynchronous notifications have this problem.

Installation

CSCtq97903

Symptom:

On the VVIM GUI, in the Configure > Array Level > Cache To Vault Map page, the preferences for the Cache Groups for filling from a Vault Group defaults to High for a newly created Vault Group.

Conditions:

Create a new Vault Group in the VVIM GUI, choose Configure > Array Level > Cache to Vault Map, the new Vault Group has High as the preference for all Cache Groups.

CSCtr52709

Symptom:

The cdsconfig script does not accept a server id greater than 65535.

Conditions:

Run cdsconfig script on a new server. Enter a value greater than 65535 at the server id prompt.

CSCtr74270

Symptom:

Streamer shows the following messages in C2K Logs:

Cost info from server 65535: streaming capacity 0. fill capacity 100. unique stream 
capacity 100.
Server 65535 out of capacity.
 
   

Conditions:

The server ID configured on the server is 65535.

ISA

CSCtj33259

Symptom:

If none of the Vaults are available to perform the ingest, the provision request fails but content is not marked as out-of-service. This causes reporting issues to the VVIM, the ingest failed but is not reported as failed.

Conditions:

There is some configuration or connectivity issues between Vaults causing none of the Vaults to be available to perform ingests.

CSCto99608

Symptom:

The ISA StreamService is getting abort signals similar to the following:

terminate called after throwing an instance of 'StreamModule::ContentNonExistent'
Abort
terminate called after throwing an instance of 'StreamModule::BadServiceGroup'
Abort
 
   

Conditions:

There are only two conditions identified that could cause this problem to occur.

1. The Content record is not in the InService state but StreamService is getting the "createStreamMultipleRanged" API invoked.

2. The SessionGateway is returning some CORBA exceptions causing the ISA StreamService to have the following message:

"BadServiceGroup" exception in the "createStreamMultipleRanged" API
 
   

CSCtq17022

Symptom:

The ISA StreamService is restarted because of failover, process abort, or manual restart while ContentStore is down or not reachable.

Conditions:

There are only two conditions identified that could cause this problem to occur.

1. The ContentStore is not reachable or down.

2. The StreamService is restarted while condition 1 is occurring.

CSCtt04229

Symptom:

Each LSCP request (including LSCP status) is creating an LSC.connect MSA event, and if the site has heartbeat messaged enabled between the STBs and the Control Service, then it is overwhelming the number of database replication records on the primary Setup Control servers.

Conditions:

MSA is enabled.

CSCth65408

Symptom:

The stream setup fails continuously if there is only one stream control and it gets stuck. The existing sessions also losses the trick-mode functionality.

Conditions:

The stream control network (QPSK) might have some issue so sending the LSCP response gets blocked.

CSCtj33259

Symptom:

If none of the Vaults are available to perform the ingest then provision request fails, but content is not marked as out of service. This causes reporting issues to the VVIM, the ingest fails, but not is not reported as failed.

Conditions:

There are some configuration or connectivity issues between Vaults causing none of the Vaults available to perform ingests.

Monitoring

CSCtq55990

Symptom:

Incorrect bandwidth and stream counts on the CDSM Monitor > System Level > System Snapshot page.

Conditions:

When a CDSM running 2.5.2 code is managing servers that have not been upgraded to 2.5.2 code, some of the new bandwidth and stream count features that are part of 2.5.2 will not function as expected.

CSCtq39815

Symptom:

Incorrect bandwidth and stream counts on the CDSM Monitor > System Level > System Snapshot page.

Conditions:

When a CDSM running 2.5.2 code is managing servers that have not been upgraded to 2.5.2 code some of the new bandwidth and stream count features that are part of 2.5.2 will not function as expected.

CSCtq83011

Symptom:

Drive ordering displayed on the CDSM GUI for the CDE250 servers does not match the physical drive ordering of the CDE250 servers.

Conditions:

In Release 2.5.1, the drive ordering of the CDE250 servers was incorrectly displayed as being from right (disk 1) to left (disk 24). The correct ordering is from left (disk 1) to right (disk 24). This only affects the CDSM GUI display.

CSCtq11474

Symptom:

This issue can potentially occur when Vaults are handling live ingests in an RTSP environment. If the primary copy of a live ingest session is failed, the backup copy is "swapped" in as the primary copy. It is possible that the content records in the database can go out of sync during this live ingest failover process, and as a result the content record would still show as an active ingest even after the ingest has completed.

Conditions:

This problem can occur in live ingest sessions with live ingest redundancy enabled, where ingest of the primary copy fails and the backup copy is used as the completed content.

CSCto13348

Symptom:

If Stream Destination on the CDSM Setup page is set to Mixed mode (IPTV+CableTV), then ServiceGroup field in the Stream Object is not displayed correctly if IPSubnet is used as service group. The IP Subnet as service group gets truncated in the stream reports.This problem does not exist in IPTV or Cable TV, only in MIXED mode.

Conditions:

The Stream Destination on the CDSM Setup page must be set to Mixed mode.

MSA

CSCts42500

Symptom:

The msa log files are created by msaLogger every few seconds or minutes rather one per day.

Conditions:

The Setup or Control server is running behind in time with CDSM, and msaLogger is up and running.

Network

CSCtj57299

Symptom:

When remote mirroring (within an array) or array mirroring (to another array or site) is enabled, the ingest code attempts to concurrently make a copy of the content to the remote Vaults. However, if the network is unable to keep up with the level of ingest traffic (especially when a thin-pipe is present), then the ingest mirroring logic sends different "chunks" of the ingested files to different remote servers. The side effect is that this causes the system to temporarily generate more than the desired number of copies of content. This is clearly inefficient. The fix is to make this ingest mirroring code more intelligent in how it mirrors content. Eventually, the mirroring logic cleans up these extra copies.

Several issues were discovered because of the fact that this code was generating an excessive number of copies. These defects have been resolved leaving this one as mostly an annoying inefficiency.

Conditions:

Mirroring is not able to keep up with the ingest or the "best" Vault changes while ingesting (this is a normal condition).

CSCtk66793

Symptom:

This problem occurs only if stream control interface is different than management interface. If the service is primary or back up control and this interface goes down then cserver does not perform the failover for the control role it has.

Conditions:

Switch or interface port could go down or pulled manually or cable is defective.

Reporting

CSCtq16136

Symptom:

If trick-mode restrictions are applied to a segment of a playlist, then the Stream Play History report in the CDSM GUI does not show the session transitions properly.

Conditions:

A trick-mode restriction has been applied to a playlist session.

RTSP

CSCtb38275

Symptom:

There is no way to auto-retort crashed applications in an RTSP environment.

Conditions:

Unlike the ISA environment, there is no provision to restart applications in the RTSP environment after they crash or quit on their own. This problem needs to be fixed to auto-restart applications that crash by having a monitoring process monitor these applications.

CSCtt23181

Symptom:

When a trick-mode play is issued, the streams transition to 1x play due to trick-mode restriction on that playlist item. As part of this transition from trick-mode scale to 1x play, an ANNOUNCE code to VOD backoffice is missing.

Conditions:

This condition can occur under the following conditions:

Playlist items that contain trick-mode restrictions

Trick-mode play request that would allow the Streamer to hit the restricted playlist item segment

Statsd

CSCtq05845

Symptom:

CDSM was in slave configuration, but the virtual IP address was still up.

Conditions:

With redundancy enabled, there should be only one master and one slave CDSM. After restarting statsd, slave CDSM should bring down its virtual IP address.

Streamer

CSCti79844

Symptom:

This defect typically should not have any adverse affect on a production system. If there are more than eight Setup and Control service address managers configured in /arroyo/test/streamer/setupfile, including the duplicate, there is the possibility that the final service address entries listed after the eight will be ignored if the are duplicate entries.

Conditions:

This defect is characterized by having duplicate service address entries in the /arroyo/test/setupfile. For example:

service address 10.94.153.130 setup 3300 control 9000
service address 10.94.153.130 setup 3300 control 9000
 
   

CSCti96225

Symptom:

Missing transition event in ODRMSessionHistory.

CSCtj66430

Symptom:

The Caching Node continues to serve cache-fill streams even when it loses connection to all the Vaults, which cause streaming errors.

Conditions:

Caching Node loses connection to all the Vaults.

Syslog

CSCts08880

Symptom:

Stream setup rate slows significantly at the top of each hour, especially at midnight UTC.

Conditions:

The logs for the previous day are large such that the log rotation and archiving script that runs each hours causes scheduling disruptions to other, more important, processes like the CalypsoStreamService. The issue is exacerbated on systems with fewer CPU resources, such as the CDE200.

CSCtr63333

Symptom:

Cannot log in to CDSM GUI. There is an error message on the login page that there is not enough disk space on the device.

Conditions:

The daily run of log rotate failed, and if the disk drive on the CDSM exceeds 95 percent usage, the user cannot log in.

Video Quality

CSCto65854

Symptom:

Audio glitch on fast-forward trick modes on STB.

Conditions:

Ghost audio sound when doing fast-forward trick modes.

CSCto89269

Symptom:

When resuming playback after a pause, macroblocking is observed.

Conditions:

When resuming a stream after a pause, STBs were specifying a starting NPT value, instead of using the value of NOW. This was causing the splice code to think that there was a content-to-content transition and to create a fade splice.

CSCtr21318

Symptom:

RTSP thread is stuck waiting for play response that never happens. The stuck play request becomes a resume play after a pause. The stream appears to never start.

Conditions:

The problem is usually caused by a corrupt or out-of-spec MPEG encoding. CServer restarts a paused stream at an MPEG I-frame boundary, and detours around any dangling B-frames that follow, replacing them with the null packets, until resuming again at a P-frame. CServer only buffers a few mega bytes of data (a couple seconds or less) when starting a stream. If the B-frames are very large (seconds of data) the detour is large and the resume point may never be reached in the stream buffer, causing CServer to not start the stream.

Other

CSCtn83921

Symptom:

Server ran out of memory and crashed.

Conditions:

Disk in repair mode was taking too long to cancel Input Output operations.

CSCtr58854

Symptom:

Setting RTSP log level to "emerg" does not turn "OFF" logging for Streamers running previous software releases (for example, Release 2.4.1).

Conditions:

Streamers running earlier Cisco TV CDS software release than Release 2.5.1.

CSCtj33903

Symptom:

Currently, the Steamer waits 500 milliseconds for the C2 interface to locate a response which could cause some "Content Not Found Error" messages.

Conditions:

The C2 interface locate response takes more than 500 milliseconds.

CSCtb61666

Symptom:

Ingest Manager fails to publish packages when the number of directories in the /arroyo/asmrpt/adi/packages directory on a CDSM or on an FTP server is the Linux maximum file counter of 32,000.

Conditions:

Number of directories exceeds 32,000, which is an ext3 limit.

Upgrading to Release 2.5.2

The following software upgrade paths are supported for Release 2.5.2:

Release 2.1 to Release 2.5.2

Release 2.2 to Release 2.5.2

Release 2.3 to Release 2.5.2

Release 2.4 to Release 2.5.2

If the CDS is running Release 2.0 or earlier, you must first upgrade to Release 2.1 before upgrading to Release 2.5.2.

For software upgrade procedures, see the Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide.

Related Documentation

Refer to the following documents for additional information about the Cisco TV CDS 2.5:

Cisco TV CDS 2.5 ISA Software Configuration Guide

http://www.cisco.com/en/US/docs/video/cds/cda/tv/2_5/configuration/isa_guide/tv_cds_2_5_isa_config.html

Cisco TV CDS 2.5 RTSP Software Configuration Guide

http://www.cisco.com/en/US/docs/video/cds/cda/tv/2_5/configuration/rtsp_guide/tv_cds_2_5_rtsp_config.html

Cisco TV CDS 2.5 API Guide

http://www.cisco.com/en/US/docs/video/cds/cda/tv/2_5/developer_guide/tv_cds_2_5_apiguide.html

Cisco TV CDS 2.5 Installation, Upgrade, and Maintenance Guide

http://www.cisco.com/en/US/docs/video/cds/cda/tv/2_5/installation_guide/tv_cds_2.5_maint_guide.html

Cisco Content Delivery System 2.x Documentation Roadmap

http://www.cisco.com/en/US/docs/video/cds/overview/CDS_Roadmap.html

Cisco Content Delivery Engine 205/220/250/420 Hardware Installation Guide

http://www.cisco.com/en/US/docs/video/cds/cde/cde205_220_420/installation/guide/cde205_220_420_hig.html

Cisco Content Delivery Engine 110 Hardware Installation Guide

http://www.cisco.com/en/US/docs/video/cds/cde/cde110/installation/guide/cde110_install.html

Cisco Content Delivery Engine 100/200/300/400 Hardware Installation Guide

http://www.cisco.com/en/US/docs/video/cds/cde/installation/guide/CDE_Install_Book.html

Regulatory Compliance and Safety Information for Cisco Content Delivery Engines

http://www.cisco.com/en/US/docs/video/cds/cde/regulatory/compliance/CDE_RCSI.html

Open Source Used in TV CDS 2.5

http://www.cisco.com/en/US/products/ps7127/products_licensing_information_listing.html

The entire CDS software documentation suite is available on Cisco.com at:

http://www.cisco.com/en/US/products/ps7127/tsd_products_support_series_home.html

The entire CDS hardware documentation suite is available on Cisco.com at:

http://www.cisco.com/en/US/products/ps7126/tsd_products_support_series_home.html

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.