Guest

Cisco Videoscape Distribution Suite for Internet Streaming

Release Notes for Cisco Internet Streamer CDS 3.1.2

  • Viewing Options

  • PDF (627.1 KB)
  • Feedback
Release Notes for Cisco Internet Streamer CDS 3.1.2

Table Of Contents

Release Notes for Cisco Internet Streamer CDS 3.1.2

Contents

Enhancements in Release 3.1.2-b60

Enhancements in Release 3.1.2-b54

Enhancements in Release 3.1.2-b34

Enhancements in Release 3.1.2-b26

Enhancements in Release 3.1.2-b17

Enhancements in Release 3.1.2-b11

New Features in Release 3.1.2-b60

New Features in Release 3.1.2-b54

New Features in Release 3.1.2-b34

New Features in Release 3.1.2-b26

New Features in Release 3.1.2-b17

New Features in Release 3.1.2-b11

System Requirements

Limitations and Restrictions

System Limits and Thresholds

Important Notes

Open Caveats

Open Caveats in Release 3.1.2-b60

Open Caveats in Release 3.1.2-b54

Open Caveats in Release 3.1.2-b34

Open Caveats in Release 3.1.2-b26

CDSM

Platform

Service Routing

Open Caveats in Release 3.1.2-b17

Multicast Distribution

Web Engine

WMT

Adobe-Fms

SPCDN

Open Caveats in Release 3.1.2-b11

Multicast Distribution

Web Engine

Data Server

CDSM

Platform

Service Routing

Resolved Caveats

Resolved Caveats in Release 3.1.2-b60

Resolved Caveats in Release 3.1.2-b54

Resolved Caveats in Release 3.1.2-b34

Resolved Caveats in Release 3.1.2-b26

Web Engine

CDSM

Transaction Logs

WMT

Mp3-Live

Multicast Distribution

Resolved Caveats in Release 3.1.2-b17

Web Engine

CDSM

Authorization Server

Service Routing

Platform

DSS

Resolved Caveats in Release 3.1.2-b11

Web Engine

Windows Media Streaming

Content Manager

CLI

Movie Streamer

CDSM

Authorization Server

Content Abstraction Layer (CAL)

Web Services

Syslog

Service Monitor

Service Routing

Acquisition and Distribution

Platform

Transaction Logs

RTSP Gateway

MP3 Live

Cache Router

Data Server

Accessing Bug Tool kit

Upgrading to Release 3.1.2

Downgrading from Release 3.1.2

Documentation Updates

Related Documentation

Obtaining Documentation and Submitting a Service Request


Release Notes for Cisco Internet Streamer CDS 3.1.2


These release notes cover Cisco Internet Streamer CDS Release 3.1.2-b60.

Last Updated: September 27, 2013
OL-28262-05

Contents

The following information is included in these release notes:

Enhancements in Release 3.1.2-b60

Enhancements in Release 3.1.2-b60

Enhancements in Release 3.1.2-b34

Enhancements in Release 3.1.2-b26

Enhancements in Release 3.1.2-b17

Enhancements in Release 3.1.2-b11

New Features in Release 3.1.2-b60

New Features in Release 3.1.2-b54

New Features in Release 3.1.2-b34

New Features in Release 3.1.2-b26

New Features in Release 3.1.2-b17

New Features in Release 3.1.2-b11

System Requirements

Limitations and Restrictions

System Limits and Thresholds

Important Notes

Open Caveats

Resolved Caveats

Accessing Bug Tool kit

Upgrading to Release 3.1.2

Documentation Updates

Related Documentation

Obtaining Documentation and Submitting a Service Request


Note The HTTPS feature and Session-Based Encryption feature are released as general availability (GA) features in Release 3.1.2. For more information on these features, see the Release Notes for 3.0.0 (http://www.cisco.com/en/US/docs/video/cds/cda/is/3_0/release_notes/CDS_RelNotes3_0_0.html).


Enhancements in Release 3.1.2-b60

Table 1 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b60 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 1 Enhancements in Cisco Internet Streamer CDS 3.1.2-b60 Release 

Bug ID
Description

CSCui52411

An option to schedule slow scan


Enhancements in Release 3.1.2-b54

Table 2 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b54 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 2 Enhancements in Cisco Internet Streamer CDS 3.1.2-b54 Release 

Bug ID
Description

CSCue32700

Add MIME type for nff extension

CSCug19342

Multicast Distribution code binds to INADDR_ANY


Enhancements in Release 3.1.2-b34

There are no new enhancements in Release 3.1.2-b34.

Enhancements in Release 3.1.2-b26

There are no new enhancements in Release 3.1.2-b26.

Enhancements in Release 3.1.2-b17

There are no new enhancements in Release 3.1.2-b17.

Enhancements in Release 3.1.2-b11

The following enhancements have been added in Release 3.1.2:

The curl utility has been upgraded to version 7.11.2. (CSCua55034)

MegaRAID BBU alarms are enabled for Generation 2 MegaRAID controller cards (CSCub63325).

Enhanced RTSPG logs to show client's IP address request and response (CSCuc05444).

Telenet button removed from the Device home menu (CSCuc05438).

New Features in Release 3.1.2-b60

VDS-IS 3.1.2-b60 introduces a feature to protect small cache files from eviction. When the cache file size is smaller than the configured size, this feature prevents files from being evicted for a period regardless of priority. The period is to be set on the Content Management screen. To configure this feature navigate to Devices>General Setting>Content Management.


Note The max protection file number is 300000 for small files protection.


New Features in Release 3.1.2-b54

There are no new features in release 3.1.2-b54.

New Features in Release 3.1.2-b34

There are no new features in release 3.1.2-b34.

New Features in Release 3.1.2-b26

There are no new features in release 3.1.2-b26.

New Features in Release 3.1.2-b17

There are no new features in release 3.1.2-b17.

New Features in Release 3.1.2-b11

The Web Engine supports Adobe Zeri content in cache mode. In cache mode, the SE gets the content fragments from the Origin server and streams them; similar to any other web objects that the Web Engine caches and delivers. The Web Engine does not support acting as the Origin server for Adobe Zeri files, nor is the Web Engine capable of generating content fragments from a large Zeri file.


Note The Service Router cannot be used for HDS requests because Adobe Zeri client does not support HTTP redirect. DNS-based redirection must be used for HDS requests.


System Requirements

The Internet Streamer CDS runs on the CDE205, CDE220, and the CDE250 hardware models, as well as two UCS models.

Table 3 lists the different device modes for the Cisco Internet Streamer CDS software, and which CDEs support them.

Table 3 Supported CDEs

Device Mode
CDE205
CDE220-2G2
CDE220-2S3i
CDE250 (all models)
UCS C200
UCS C210

CDSM

Yes

No

No

No

Yes

No

SR

Yes

Yes

No

No

Yes

No

SE

Yes

Yes

Yes

Yes

No

Yes

SR—Proximity Engine standalone

Yes

Yes

No

No

No

No


The new CDE250 models (CDE250-2S8, CDE250-2S9, and CDE250-2S10) have four interfaces at 10 gigabit Ethernet speeds and four interfaces at gigabit Ethernet speeds (plus two additional gigabit ethernet interfaces for management).

The new CDE250 models only support the SE device mode and have the following storage capacities:

CDE250-2S8—24 x 300 GB 2.5 SSD

CDE250-2S9—12 x 600 GB 2.5 SSD

CDE250-2S10—24 x 600 GB 2.5 SSD

The Cisco UCS models (UCS C200 and UCS C210) and the Cisco Internet Streamer Release 3.1 software are sold separately and ship independently of each other.

CDE250-2S6 and CDE250-2M0 platforms have four interfaces at 10 gigabit Ethernet speeds and four interfaces at gigabit Ethernet speeds (plus two additional gigabit ethernet interfaces for management).

The CDE220-2S3i platform has a total of 14 gigabit Ethernet ports in this CDE. The first two ports (1/0 and 2/0) are management ports. The remaining 12 gigabit Ethernet ports can be configured as two port channels. See the Cisco Content Delivery Engine CDE205/220/250/420 Hardware Installation Guide for set up and installation procedures for the CDE220-2S3i and the Cisco Internet Streamer CDS 2.6 Software Configuration Guide for information on configuring the Multi Port Support feature.

The CDE220-2G2 platform has a total of ten gigabit Ethernet ports. The first two ports (1/0 and 2/0) are management ports. The remaining eight gigabit Ethernet ports can be configured as one port channel. See the Cisco Content Delivery Engine CDE205/220/250/420 Hardware Installation Guide for set-up and installation procedures for the CDE220-2G2.

The CDE205 can run as the CDSM, SR or SE. See the Cisco Content Delivery Engine CDE205/220/250/420 Hardware Installation Guide for set-up and installation procedures for the CDE205.


Note For performance information, see the release-specific performance bulletin.


Limitations and Restrictions

This release contains the following limitations and restrictions:

There is a 4 KB maximum limit for HTTP request headers. This has been added to prevent client-side attacks, including overflowing buffers in the Web Engine.

Standby interface is not supported for Proximity Engine. Use port channel configuration instead.

There is no network address translation (NAT) device separating the CDEs from one another.

Do not run the CDE with the cover off. This disrupts the fan air flow and causes overheating.


Note The CDS does not support network address translation (NAT) configuration, where one or more CDEs are behind the NAT device or firewall. The workaround for this, if your CDS network is behind a firewall, is to configure each internal and external IP address pair with the same IP address.

The CDS does support clients that are behind a NAT device or firewall that have shared external IP addresses. In other words, there could be a firewall between the CDS network and the client device. However, the NAT device or firewall must support RTP/RTSP.

System Limits and Thresholds

This release has the following limits and thresholds:

Service Router Limits and Thresholds

Service Monitor Limits and Thresholds

Web Engine Limits and Thresholds

CDSM Limits and Thresholds

RTSP Gateway and Movie Streamer

Windows Media Streaming

Flash Media Streaming

Service Router Limits and Thresholds

The Service Router has memory-related limits and thresholds. Memory usage of the Service Router depends on the number of coverage zone entries, the number of Content Origin servers, the distribution of subnets in the Coverage Zone file, and the number of Service Engines in the CDS. From our tests using a sample Coverage Zone file, we have observed that we can support 20,000 Coverage Zone entries with 26 SEs, and 40 Content Origins servers.


Note The number of Coverage Zone entries, SEs, and Content Origin servers are subject to change depending on the Coverage Zone configured.

We recommend keeping the memory usage (both virtual and resident) below 1.5 GB.


Frequent configuration updates could cause memory fragmentation, which raises the memory usage.

Service Monitor Limits and Thresholds

When the Service Monitor thresholds are exceeded, an alarm is raised on the respective device and an SNMP trap is sent to the CDSM. The parameters monitored and thresholds for each component or protocol engine can be modified. The default thresholds are as outlined below.

Following are the parameters that are monitored on each device (SE, SR, and CDSM) and the default threshold setting of each parameter:

CPU—80 percent

Memory—80 percent

Kernel memory—50 percent

Disk usage—80 percent

Disk failures—75 percent

Augmentation alarms—80 percent

Following are the parameters that are monitored only on the SE, along with default threshold setting of each parameter:

Windows Media Streaming thresholds—90 percent

Flash Media Streaming thresholds—90 percent

Movie Streamer—90 percent%

Maximum number of concurrent sessions—200

Maximum Bandwidth—200,000 kbps

NIC bandwidth—90 percent

Burst Count—1

Web Engine Limits and Thresholds

The Web Engine has the following limits and thresholds:

Memory Usage

Session Limits

CAL Limits

Memory Usage

In Release 2.5.9, the memory threshold on each SE is 3.2 GB. If the threshold is exceeded, the memory_exceeded alarm is raised and trickle mode is enabled. In Release 2.5.9, the admission control is based on 30,000 session and 3.2 GB of memory.

In Release 2.6.1, the memory threshold on each SE is 3.2 GB. If the threshold is exceeded, the memory_exceeded alarm is raised. In cases where the memory reaches 3.7 GB, trickle mode is enabled and eventually the Web Engine is restarted. The above memory values, and the 20,000-60,000 sessions and 100,000 open file/socket descriptor (FD) limit are used for admission control in Release 2.6.1.

Session Limits

Web Engine supports the following session-threshold limits:

49,800 session count for the CDE250

15,000 session count for all other CDEs

The max_session_exceeded alarm is raised if the session-threshold limit is reached. If further requests are sent to the SE even when the session threshold is reached, the Web Engine attempts to process the requests but does not accept any more requests when the request count reaches 60,000 on a CDE250, and 20,000 on all other CDEs.

CAL Limits

Outstanding CAL Lookup threshold is 25,000 on the CDE250 and 15,000 on all other CDEs. The WebCalLookupThreshold alarm is raised on reaching this threshold limit.

Outstanding CAL disk Write threshold is 3,000 CAL requests (create, update, delete, popularity update) on the CDE250, and 1,500 on all other CDEs. The WebCalDiskWriteThreshold alarm is raised on reaching this threshold.

Other CAL thresholds are as follows:

File Descriptor usage threshold is 85 percent

TEMPFS usage threshold is 80 percent

Active datasource threshold is 2,000


Note CAL-related thresholds and the File Descriptor-related thresholds are introduced in Release 2.6.1.

Web Engine thresholds are also applicable to adaptive bit rate (ABR) streaming.


CDSM Limits and Thresholds

The CDSM has the following limits and thresholds:

RPC Connections

File Synchronization

CDSM Availability (primary and standby)

SE Configuration Change Synchronization

RPC Connections

A maximum of 40 RPC connections are supported among the managed devices (SE, SR, standby CDSM, and primary CDSM). The RPC connection maximum is defined in the httpd.conf.rpc configuration file located in the /state directory.

File Synchronization

The primary CDSM checks for file updates and synchronization with the managed devices (SE, SR, and standby CDSM) every ten minutes.

CDSM Availability (primary and standby)

The SE and SR check for the availability of the primary and standby CDSM on a regular interval; however, if the CDSM does not respond, the SE and SR use an exponential-backoff call for retrying the connection.

The exponential backoff call means that if the CDSM does respond to the first attempt, the SE or SR sleep for ten seconds before trying again. If the second attempt does not succeed, the wait time doubles (20 seconds), if that attempt does not succeed, the wait time doubles again (40 seconds). The wait time doubles every attempt (10, 20, 40, 80, and so on) until the maxWaitingTime of 320 seconds.

SE Configuration Change Synchronization

The period of time before the local configuration manager (LCM) on an SE sends a configuration change to the primary CDSM is a maximum of 2.25 times the polling rate. The polling rate is configurable through the CDSM GUI (System > Configuration > System Properties, System.datafeed.pollRate).

RTSP Gateway and Movie Streamer

The default RTSP Gateway transactions per second (tps) is 40. There are no other limits to the RTSP Gateway.

The Movie Streamer default maximum concurrent session is 200 and the default maximum bandwidth is 200 Mbps.

Windows Media Streaming

Windows Media Streaming has the following limits and thresholds:

Windows Media Streaming recommended concurrent remote server sessions 300


Note Regarding concurrent remote server sessions, if all requests are unique cache-miss cases, Windows Media Streaming can reach up to 1000 sessions of 1 Mbps file each. Windows Media Streaming can sustain 1000 remote server sessions at most if the Content Origin server can respond, but the recommended value is 300.


Windows Media Streaming transactions per second is 40 (because of the RTSP Gateway limitation).

Memory threshold 3 GB

CPU threshold is 80 percent

Flash Media Streaming

With the basic license, Flash Media Streaming the default maximum concurrent sessions is 200 and the default maximum bandwidth is 200 Mbps.

Buying more licenses can increase the concurrent sessions and maximum bandwidth as follows:

CDE220-2G2 and CDE220-2S3—15,000 concurrent sessions and 8 Gbps maximum bandwidth

CDE250-2M0—40,000 concurrent sessions and 40 Gbps maximum bandwidth

We recommend that the Flash Media Streaming process memory usage not exceed 3 GB resident set size (RSS). If the memory usage for Flash Media Streaming exceeds 3 GB RSS, a threshold exceeded alarm is raised.


Note RSS is the portion of a process that exists in physical memory (RAM), as opposed to virtual memory size (VSIZE), which includes both RAM and the amount in swap. If the device has not used swap, the RSS number is equal to VSIZE.


Important Notes

To maximize the content delivery performance of a CDE205, CDE220, or CDE250, we recommend you do the following:

1. Use port channel for all client-facing traffic.

Configure interfaces on the quad-port gigabit Ethernet cards into a single port-bonding interface. Use this bonding channel, which provides instantaneous failover between ports, for all client-facing traffic. Use interfaces number 1 and 2 (the two on-board Ethernet ports) for intra-CDS traffic, such as management traffic, and configure these two interfaces either as standby or port-channel mode. Refer to the Cisco Internet Streamer CDS 2.6 Software Configuration Guide for detailed instruction.

2. Use the client IP address as the load balancing algorithm.

Assuming ether-channel (also known as port-channel) is used between the upstream router/switch and the SE for streaming real-time data, the ether-channel load balance algorithms on the upstream switch/router and the SE should be configured as "Src-ip" and "Destination IP" respectively. Using this configuration ensures session stickiness and general balanced load distribution based on clients' IP addresses. Also, distribute your client IP address space across multiple subnets so that the load balancing algorithm is effective in spreading the traffic among multiple ports.


Note The optimal load-balance setting on the switch for traffic between the Content Acquirer and the edge Service Engine is dst-port, which is not available on the 3750, but is available on the Catalyst 6000 series.


3. For high-volume traffic, separate HTTP and WMT.

The CDE205, or CDE220 performance has been optimized for HTTP and WMT bulk traffic, individually. While it is entirely workable to have mixed HTTP and WMT traffic flowing through a single server simultaneously, the aggregate performance may not be as optimal as the case where the two traffic types are separate, especially when the traffic volume is high. So, if you have enough client WMT traffic to saturate the full capacity of a server, we recommend that you provision a dedicated server to handle WMT; and likewise for HTTP. In such cases, we do not recommended that you mix the two traffic types on all CDE servers which could result in suboptimal aggregate performance and require more servers than usual.

4. For mixed traffic, turn on the HTTP bitrate pacing feature.

If your deployment must have Streamers handle HTTP and WMT traffic simultaneously, it is best that you configure the Streamer to limit each of its HTTP sessions below a certain bitrate (for example, 1Mbps, 5Mbps, or the typical speed of your client population). This prevents HTTP sessions from running at higher throughput than necessary, and disrupting the concurrent WMT streaming sessions on that Streamer. To turn on this pacing feature, use the HTTP bitrate field in the CDSM Delivery Service GUI page.

Please be aware of the side effects of using the following commands for Movie Streamer:

Config# movie-streamer advanced client idle-timeout <30-1800>
Config# movie-streamer advanced client rtp-timeout <30-1800>
 
   

These commands are only intended for performance testing when using certain testing tools that do not have full support of the RTCP receiver report. Setting these timeouts to high values causes inefficient tear down of client connections when the streaming sessions have ended.

For typical deployments, it is preferable to leave these parameters set to their defaults.

5. For ASX requests, when the Service Router redirects the request to an alternate domain or to the origin server, the Service Router does not strip the .asx extension, this is because the .asx extension is part of the original request. If an alternate domain or origin server does not have the requested file, the request fails. To ensure requests for asx files do not fail, make sure the .asx files are stored on the alternate domain and origin server.

Open Caveats

The open caveat section has the following subsections:

Open Caveats in Release 3.1.2-b60

Open Caveats in Release 3.1.2-b54

Open Caveats in Release 3.1.2-b34

Open Caveats in Release 3.1.2-b26

Open Caveats in Release 3.1.2-b17

Open Caveats in Release 3.1.2-b11

Open Caveats in Release 3.1.2-b60

Table 4 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b60 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 4 Open Caveats in Cisco Internet Streamer CDS 3.1.2-b60 Release 

Bug ID
Description

CSCuj18923

SNMP cdsHttpPerfReqPerSec is not working

CSCuj38303

Web Engine transaction logs write stops after disable/enable translog


Open Caveats in Release 3.1.2-b54

Table 5 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b54 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 5 Open Caveats in Cisco Internet Streamer CDS 3.1.2-b54 Release 

Bug ID
Description

CSCuh49257

M3u8 does not refresh when HLS session tracking is enabled

CSCuh78209

Active session hangs after stopping spiren with Https DS, HDS, revalidation

CSCui01821

Active sessions even after stopping stress

CSCui05393

UCS install fails

CSCui16663

Corrupt content found when OS returns 200 OK for range request


Open Caveats in Release 3.1.2-b34

There are no new open caveats for 3.1.2-b34.

Open Caveats in Release 3.1.2-b26

Release 3.1.2-b26 contains the following open caveats:

CDSM

CSCue18749

Symptom:

When configuring a URL signature in GUI: Devices (choose a Service Engine) -> Service Control -> URL Signing, if the "Public Key URL" or "Private Key URL" is given an URL, which is invalid, the GUI accepts the configuration, but on the change, it doesn't reflect in the running-config in CLI on the SE side.

Conditions:

The "Public Key URL" or "Private Key URL" is given an URL which is invalid, either it doesn't exist or it's not valid key.

Workaround:

Specify valid "Public Key URL" and "Private Key URL".

CSCue17391

Symptom:

In page Devices -> Application Control -> Windows Media Streaming -> General Settings ->Windows Media Cache Settings, change Maximum TTL and Minimum TTL. The new value of Maximum TTL may not take effect and "Cli command failed" appears in cms log.

Conditions:

This only happens when all following conditions are met:

1. Both Maximum TTL and Minimum TTL are modified at the same time.

2. Maximum TTL is changed to be lower than or equal to 60 minutes.

Note: 60 minutes is the default value of Minimum TTL.

Workaround:

Avoid modifying Maximum TTL and Minimum TTL at the same time.

CSCue23277

Symptom:

Sometimes editing, Devices->Application Control ->Bandwidth Schedules, may result in an error message like "bandwith.obj.get failed 39838".

Conditions:

This happens only when all following conditions are met:

1. Modification is made at the time of SE syncing data to GUI.

2. The schedule rule of bandwidth is "specify day range: " and start/end day are configured with same value.

Workaround:

Open bandwidth edit page and do the modification again.

or

Avoid configuring start/end day of "Specify day range: " with same value. If user wants to set that kind of schedule, the right way is to use " Use specific days: " rule instead.

For example, instead of configuring both start and end day of "Specify day range" with Sunday, user should select "Use specific days:" rule and check "Sunday".

Platform

CSCue20828

Symptom:

During upgrade, the system may go into KDB.

Conditions:

This occurs very rarely and it happened when there is streaming traffic running during upgrade.

Workaround:

1. Wait until traffic is completely off before upgrade.

If the box does go into KDB:

2. If the box has KDB enabled at CLI, reboot it from Console, or power cycle.

3. If the box doesn't have KDB enabled at CLI, the box would reboot by itself, and this failure will be no harm to customer.

Service Routing

CSCue20112

Symptom:

When delivery service session exceeds quota and sends a rtsp request directly to SE. The content is not playing as expected and we receive 499 response in SE translog. But session is established with OS and wmt hdr file is cached in SE. The transaction log is as follows:

5.50.0.11 2013-01-23 05:43:11 - rtsp://V9-2S3-Fiber.se.spcdn.com/vod/test.wmv 0 0 0 499 {3300AD50-2C39-46C0-AE0A-AB5E3960BA83} - - "WMPlayer/11.0.5721.5251" - - - - - - 0 0 - rtsp - - - - 0 - 0 - - - - - - - - - - 4.25.1.14 - 1 -

Conditions:

When the delivery service session exceeds quota, and the request for a cache miss content.

Workaround:

None.

Open Caveats in Release 3.1.2-b17

Release 3.1.2-b17 contains the following open caveats:

Multicast Distribution

CSCuc14038

Symptom:

Sometimes the files are transferred by way of unicast when the delivery service is of the type Multicast Unicast.

Conditions:

When the multicast receiver has an alternate forwarder path to receive the content via unicast distribution.

Workaround:

Please note that

- the content is pre-positioned successfully

- the issue is observed rarely.

- and failover from multicast to unicast is expected in case of an error in multicast distribution

so there is no impact of this issue on the feature as far as the customer is concerned. No workaround is needed.

Web Engine

CSCud76729

Symptom:

Web Engine core dumps and hence all the active sessions will be lost, leading to service disruption.

Conditions:

This occurs in scenarios when the following conditions are met for HLS playout-

- HLS Session Tracking feature is enabled

- Cache Bypass (no-store option in the response header)

- The .m3u8 file requested is an empty file (0 bytes)

Workaround:

Avoid zero byte files OR Disable Session Tracking.

CSCud96407

Symptom:

The CLI "web-engine range-cache-fill enable" was hidden in Release 3.0 since KCF was introduced and this CLI is not used anymore. However, HTTPS does not use the KCF code and still needs this CLI to be executed to do a range-cache-fill.

Conditions:

When HTTPS is used.

Workaround:

This CLI still works as expected and can be executed. It's just hidden. This will not affect regular HTTP (KCF enabled) functioning.

CSCud52911

Symptom:

If the primary and streaming interface are configured to be the same physical interface and then separated out by creating a new primary interface (no change to streaming), the web-engine process does not run on port 443. This disables HTTPS functionality.

eg: Initial Config -

primary-interface PortChannel 1

streaming-interface PortChannel 1

Now, Web-Engine process runs on port 80 and rpc_httpd process runs on port 443 on interface PortChannel 1

Change to -

no primary-interface PortChannel 1

primary-interface PortChannel 2

Now, the rpc_httpd process gets restarted and is running on port 443 on PortChannel 2.

However, web-engine process is not restarted and thus runs only on port 80. There is no trigger for it to run on port 443 on the Streaming interface - PortChannel 1

Conditions:

See Symptoms for details.

Workaround:

In the above scenario, execute the following command -

no streaming-interface PortChannel 1

streaming-interface PortChannel 1

This will restart the web-engine process and will now run on port 443 on the streaming interface - PortChannel 1 and HTTPS should work fine.

WMT

CSCud92507

Symptom:

SE admin shell responds with latency (around 50 seconds).

Conditions:

The common_edm process hangs during unregistering from the data server.

Workaround:

None.

Adobe-Fms

CSCtr61896

Symptom:

fmscore process coredumps seen under very high stress.

Conditions:

Under too much of stress (alarms generated before), fmscore process coredumps randomly.

Workaround:

Nothing found at this point. But if SR is there in the deployment, then this issue should not be seen as there were alarms generated before this coredump.

SPCDN

CSCud79607

Symptom:

M3U playlists are returned empty when requested through CDS-IS.

Conditions:

Response from Origin Server doesn't consist "Content-Length" field.

Workaround:

Configure Origin Server to include "Content-Length" field if possible.

CSCud79731

Symptom:

CDS-CLIENT-INFO not printed in custom transaction log for cache hit case.

Conditions:

When enabling "content flow trace" + "filter trace flow to client" with custom transaction log including "%{CDS-CLIENT-INFO}i" field, it's not printed correctly in cache hit scenario.

Workaround:

None.

CSCud97492

Symptom:

Webengine session limit alarm is raised when there are more than 20000 webengine sessions, even though user can set maximum session to 60000 via CLI.

Conditions:

When there are more than 20000 webengine sessions.

Workaround:

None.

Open Caveats in Release 3.1.2-b11

Release 3.1.2-b11 contains the following open caveats:

Multicast Distribution

CSCuc14038

Symptom:

Sometimes the files are transferred by way of unicast when the delivery service is of the type Multicast Unicast.

Conditions:

When the backup Sender is in the same location as that of primary Sender. When an SE that is not assigned to the multicast cloud resides in any of the location as the Content Acquirer, primary Sender, or a middle-tier Receiver.

Workaround:

Assign the backup Sender to a new location. Assign the SE that is not part of the multicast cloud to a location that is not in the direct upstream path of the edge location. Content is distributed properly and no issues are observed.

Web Engine

CSCuc65092

Symptom:

On very rare instances, the Web Engine crashes and restarts.

Conditions:

During Web Engine startup if there are any configuration changes occurring simultaneously, corruption to the Web Engine data structures may occur and cause a crash of the Web Engine.

Workaround:

After the SE or Web-Engine process restarts, wait for 30 seconds before making any configuration changes using the CLI or CDSM GUI.

Data Server

CSCud16272

Symptom:

SE admin shell responds with latency (around 50 seconds).

Conditions:

The common_edm process hangs during unregistering from the data server.

Workaround:

None.

CDSM

CSCud16975

Symptom:

On the CDSM GUI device activation page, when the streaming interface IP address is set as the management IP address (no https-enabled delivery service exists), rpc_httpd fails to start up due to port 443 binding issue

Conditions:

In Release 3.0.0 and later releases, when the streaming interface IP address is set as the management IP address.

Upgrading the SE and CDSM from Release 2.x.x to Release 3.x.x with this GUI setting can also cause this issue.

Workaround:

Before upgrading the software, either check the Use SE's primary IP address for management communication checkbox on the CDSM GUI device activation page or use another IP address that is different than any streaming interface IP address (and assigned the IP address) if this device has more NICs.

CSCud15936

Symptom:

CDSM may run slow when there are a lot of alarms.

Conditions:

This happens when large number of alarms are generated back to back. This problem was hit in 3.1.1 due to other bugs such as CSCuc92136 and CSCuc03325. Because CSCuc92136 and CSCuc03325 are both fixed this release, this problem is unlikely to cause any visible impact.

Workaround:

None.

Platform

CSCuc62645

Symptom:

The no shutdown command does not work with standby interfaces.

Conditions:

Configure two interfaces for standby interface; the no shutdown command does not work on the non-active interface.

Workaround:

Reboot the device.

Service Routing

CSCud31565

Symptom:

SR does not redirect .m3u8 requests to SE unless the Movie Streamer service is enabled on the SE.

Conditions:

URL contains string ".sdp."

Workaround:

Enable Movie Streamer service on SEs.

CSCud34194

Symptom:

The Service Router should limit simultaneous RPC calls to 20, but this limit is not applied.

Conditions:

The Service Router should limit simultaneous RPC calls to 20, but this limit is not applied.

Workaround:

None.

Resolved Caveats

The following caveats have been resolved since Cisco Internet Streamer CDS Release 3.1.2. Not all the resolved issues are mentioned here. The following list highlights the resolved caveats associated with customer deployment scenarios.

The resolved caveats section has the following sub sections:

Resolved Caveats in Release 3.1.2-b60

Resolved Caveats in Release 3.1.2-b54

Resolved Caveats in Release 3.1.2-b34

Resolved Caveats in Release 3.1.2-b26

Resolved Caveats in Release 3.1.2-b17

Resolved Caveats in Release 3.1.2-b11

Resolved Caveats in Release 3.1.2-b60

Table 6 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b60 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 6 Resolved Caveats in Cisco Internet Streamer CDS 3.1.2-b60 Release 

Bug ID
Description

CSCui16663

Corrupt content found when OS returns 200 OK for range request

CSCui01010

Lots of connections with Origin Server established for m3u8 content

CSCug92064

Content get cached with SE name instead of OS IP

CSCui41796

SNMP core-dump - wrong pointer from storage array made SNMP crash

CSCui26945

Authsvr coredump when parse config

CSCui75646

CA does not rewrite host header on OS 301 redirect

CSCui92443

If OS redirect URL point to non-80 port, request failed

CSCui64457

DNS based routing issue affecting off-loaded SE


Resolved Caveats in Release 3.1.2-b54

Table 7 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b54 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit

Table 7 Resolved Caveats in Cisco Internet Streamer CDS 3.1.2-b54 Release 

Bug ID
Description

CSCuh26825

Web Engine should reconnect unify_log process to send transaction log entries

CSCuh37370

Service Router sends its own IP address in response to AAA request

CSCuh74634

High age value in Web Engine response

CSCuh43886

Streaming interface configured as standby not supported

CSCud16272

Common_edm hang causes dataserver to hang for 50 sec

CSCuh90716

When CA receives response from OS without content-length header Web Engine coredump happens

CSCug91355

Corrupted files in cache

CSCue72920

RPC exceptions were caused by receiver process

CSCue86012

Web Engine core during ABR traffic

CSCue91443

High CPU from Web Engine thread

CSCuf20401

Excessive revalidation requests from SE/SA to Origin servers

CSCuf21251

Fix interface selection logic for multicast distribution

CSCuf56037

core.web-engine.3.2.0.b26.ipvbuild.18148 found running HLS stress traffic

CSCuf56651

Memory leak after long time running

CSCuf68410

Delete failed when small file revalidation returned error response

CSCuf82185

fms_stats_daemon causing slow cli

CSCuf86453

Session ID not being assigned to HSS Live fragments

CSCug25909

HSS Session ID not assigned for HTTP 304 revalidation

CSCug28372

PxE pickup wrong location community from different subnet

CSCug47521

Cache meta data not updated before response was sent to SE and Client

CSCug52546

Maximum sessions exceeded

CSCug52945

SR transaction log has duplicate entries

CSCug63257

Static route to redirect multicast traffic doesn't work

CSCug71848

Web Engine memory_exceeded alarms on the SE

CSCug76826

rpc_httpd is disabled alarm

CSCug88872

CA sends http request on port 80 while CDSM set OS port on 8080

CSCug90600

Core in Web Engine - core.web-engine.3.2.0.b26

CSCug97540

"copy cdnfs disk" command deletes cached object

CSCuh26825

Web Engine should reconnect unify_log process to send transaction log entries

CSCuh32178

UKSE and Web Engine are in "D" status after Range-Request stress test

CSCug47585

Web Engine: DataSource hang after stress test

CSCug58695

Web Engine: range request failure after stress test

CSCug58187

Web Engine: HTTPS connection to OS will fail for certificate verify failed

CSCug65685

core-dump backtrace core.service_router.3.2.0.b26.ipvbuild.8750

CSCuh16166

memory leak in HLS vod stress when HLS session tracking enabled

CSCuf82490

Primary/Standby CDSM have stale entry issues

CSCuh29106

find core file after clear cache content

CSCug03107

local/local1 gets 100% full under stress

CSCug84379

Node-mgr doesn't restart once the Web Engine process get killed

CSCuf73571

Web Engine hangs while running stress with corrupted cached content

CSCuh27543

Web Engine crash while running Https, ABR profile

CSCuh08265

Authsvr coredump during HLS and HSS session tracking stress

CSCuh09175

Splunk monitor not enabled after restore factory-default preserve basic

CSCug55951

url signing for https doesn't work if set protocol=all in rule file

CSCuh15122

Web Engine getting killed itself even when SE is in idle state

CSCuc65092

Web Engine core-dump observed in SE

CSCue48422

Web Engine core dump while assigning rule file on running codenomicon

CSCug86267

SR/SE Keepalive messages should be handled by the management interface


Resolved Caveats in Release 3.1.2-b34

Table 8 lists the issues resolved in the Cisco Internet Streamer CDS 3.1.2-b34 release.

Click on the bug ID to view the bug details. This information is displayed in the Bug Toolkit.

Table 8 Resolved Caveats in Cisco Internet Streamer CDS 3.1.2-b34 Release 

Bug ID
Description

CSCuc98482

Misleading response when deleting cached object from CDSM

CSCud76022

Netlink report delivery service data not correct

CSCud85292

Uns-server core on a VDS-IS 3.1.2 SE

CSCue32700

Add MIME type for nff extension

CSCue34975

Device hangs at 100% CPU utilization, not releasing after test stops

CSCue39358

Power supply units failed after upgrade to 3.1.2b11

CSCue47536

Web Engine coredump on shutting down the interface with traffic

CSCue50340

Transaction-log issue after downgrade from 3.2 to 3.1.2

CSCue53756

Configure OSPF Proximity Routing fails on CDSM

CSCue66028

Web Engine gets disabled when session tracking is enabled

CSCue79169

Web Engine gets disabled when the box is idle

CSCue80090

Delivery Services are showing "in process" after upgrade to 3.1.2b26

CSCue91443

100% high CPU occupied by Web Engine thread

CSCue51634

Core dump in HttpResponseHelper::SetCacheStatusString( )

CSCue72920

TI upgrade: RPC exceptions were caused by the issue of receiver process

CSCuf20401

Too many revalidation requests from SE/CA to Origin servers

CSCuf56651

Memory leak after long time running

CSCuf56037

core.webengine.3.2.0.b26.ipvbuild.18148 found during HLS stress

CSCuf93475

Web Engine service dead core


Resolved Caveats in Release 3.1.2-b26

The following caveats are resolved in 3.1.2-b26:

Web Engine

CSCud76729

Symptom:

Web Engine core dumps and hence all the active sessions will be lost, leading to service disruption.

Conditions:

This is seen in error scenarios when the following conditions are met for HLS playout-

- HLS Session Tracking feature is enabled

- Cache Bypass (no-store option in the response header)

- The .m3u8 file requested is an empty file (0 bytes)

CSCud62223

Symptom:

WebEngine crashes after 3.1.2-b11 upgrade.

Conditions:

Player sends HTTP range request to SE and Windows Media Server is configured as the HTTP Origin Server.

Update to the problem scenarios, this problem happens in two cases

1) When a range request is sent (either WMT or non-WMT request) and OS replies with a 200 (ignoring range request) instead of 206.

2) When a small file request is sent and OS responds with HTTP header and complete body in one packet, with "Cache-Control: no-store" HTTP header option.

CSCud91498

Symptom:

An incorrect webengine error log message indicating that Internal Server Error (HTTP code 500) has occurred, although the actual error is different from 500.

Conditions:

Certain error scenarios - 504 gateway timeout, Zero byte file request, Server not found (403).

CSCud97492

Symptom:

Webengine session limit alarm is raised when there are more than 20000 webengine sessions, even though user can set max session to 60000 via CLI.

Conditions:

When there are more than 20000 webengine sessions.

CDSM

CSCud42568

Symptom:

1) System -> Configuration -> HTTPS -> Root CA File Registration

a) Upload CA.crt - accepted

b) Upload test.txt - not accepted

c) Upload test.xlsx - not accepted

2) System -> Configuration -> HTTPS -> HTTPS Certification Files Registration

a) Upload Certificate File CA.crt - accepted

b) Upload Certificate File test.txt - accepted

c) Upload Certificate File test.xlsx - accepted

d) Upload Key File CA.crt - accepted

e) Upload Key File test.txt - accepted

f) Upload Key File test.xlsx - accepted

Conditions:

3.0 or later releases.

Transaction Logs

CSCud79731

Symptom:

CDS-CLIENT-INFO not printed in custom transaction log for cache hit case.

Conditions:

When enabling "content flow trace" + "filter trace flow to client" with custom transaction log including "%{CDS-CLIENT-INFO}i" field, it is not printed correctly in cache hit scenario.

WMT

CSCud92507

Symptom:

In syslog.txt, there are lines as

Nov 5 14:12:29 edgecache501-cardiff wmt_be: %SE-SYSLOGD-4-581002: pid-2426: HTEndLoop=0, should set timeout!

Conditions:

In wmt http vod playback, there is possiblity that full-cached content refreshes incorrectly.

Mp3-Live

CSCud79607

Symptom:

M3U playlists are returned as empty when requested through CDS-IS.

Conditions:

Response from Origin Server does not consist "Content-Length" field.

Multicast Distribution

CSCuc14038

Symptom:

When the replication delivery type is set as Multicast or Unicast, we expect the distribution to all multicast receivers to occur over Multicast. But in some rare occasions the distribution occurs over Unicast.

Conditions:

The issue specifically occurs, if the multicast receiver has an alternate forwarder path to receive the content via unicast distribution. In cases where the notification to pre-position the content reaches ahead of the notification from the CDSM to the SE, saying that it is now a part of a multi-cast cloud for that DeliveryService this issue occurs.

Workaround:

Please note

- The content is pre-positioned successfully.

- The issue is observed rarely.

- And failover from multicast to unicast is expected in case of an error in multicast distribution.

So there is no impact of this issue on the feature as far as the customer is concerned. No workaround is needed.

Resolved Caveats in Release 3.1.2-b17

The following caveats are resolved in 3.1.2-b17:

Web Engine

CSCty62095

Symptom:

If ACQ path is down (tier-1), the edge SE returns 500 (internal error) to client instead of 504(gateway-timeout) or 503(DNS failure).

Conditions:

The SE returns 500 to the client when its tier-1 nodes are down.

CSCud51995

Symptom:

When client plays HDS stream, intermittently it gets 504 Gateway Timeout.

Conditions:

Race condition while OS timeout one TCP connection and SE reuse this connection.

CSCud62223

Symptom:

WebEngine crashed after 3.1.2-b11 upgrade.

Conditions:

Player sends HTTP range request to SE and Windows Media Server is configured as the HTTP Origin Server.

CDSM

CSCud63664

Symptom:

Flash Player(HDS) receives 504 error when admin adds new dummy Origin Server or Delivery Service into system.

Conditions:

Operator add new OS or DS from GUI.

CSCuc68365

Symptom:

Adobe HDS client received 504 errors while Admin creates a non-related delivery service on CDSM.

Conditions:

Creation of delivery service will impact other delivery service on 3.1.2.

Authorization Server

CSCud57879

Symptom:

Bad usage of CdsRegex regular expression leads to memory leak, it doesn't free the momory pointed to by "real_preg".

Conditions:

Valgrind checks if Auther server releases rule file in memory when deliver Service changes rule file from one file to antoher file(none).

Service Routing

CSCud31565

Symptom:

SR does not redirect .m3u8 requests to SE unless the MovieStreamer service is enabled on the SE.

Conditions:

URL contains string ".sdp"

CSCud34194

Symptom:

The service router should limit simultaneous RPC calls to 20, but this limit is not applied.

Conditions:

The limit is always not working.

CSCud85861

Symptom:

Service router core dump.

Conditions:

When client sends http/rtsp request with no '.' in its url.

Platform

CSCud29094

Symptom:

Service engine, service router does not reply with an ACK on a TCP syn.

Conditions:

Execute TAC supplied script to disable/enable tcp_tw_recycle or tcp_tw_reuse --or-- execute script that ignores the TCP timestamp. Use persistent connections on the clients.

CSCud29116

Symptom:

Add a show command to know the tcp settings after addition of these CLIs.

Conditions:

CSCud57383

Symptom:

The below alarm condition may be seen:

-Rootfs partition out of free inodes

In addition, the following error messages may be seen in the syslog:

... hil-cds-se1 kernel: %SE-SYS-3-900000: IPMI Watchdog: response: Error ff on cmd 22

Conditions:

This condition only occurs on CDE250 platforms.

CSCud56899

Symptom:

Upgrade from 2.5.X based releases to 2.X and/or 3.X releases results in intermittent premature aborts.Subsequently, rebooting / power-cycling the system results in the original 2.5.X release coming up.

Conditions:

Under very rare circumstances, pending access(es) to CDNFS volumes (even after offloading the device),may cause this upgrade issue.

CSCud70600

Symptom:

After upgrading all SEs from 2.6.3 to 3.1.2.7, it was found that that 15 SEs out of 50 SEs have their CPU threshold exceeded. The processor list displays 10 to 20 WMT Backend processes indicating abnormally high CPU usage, some at 100%. The aggregate of these WMT processes uses up all available processor availability and causes the service monitor CPU threshold exceeded to be raised, which in turn causes the service router to stop sending new service request to these SEs.

Conditions:

upgrading all SE's from 2.6.3 to 3.1.2.7.

DSS

CSCud61831

Symptom:

Signed URLs in movie-streamer do not log correctly in translog.

Conditions:

When URL signing is applied for movie-streamer request;

and S * 2 + 10 + R + E > 255 (S: length of query string excluding question mark in a signed URL, R: length of request URL, E: length of the string "sename.se." in RFQDN).

Resolved Caveats in Release 3.1.2-b11

The following caveats are resolved in 3.1.2-b11:

Web Engine

CSCud13712

Symptom:

During mixed profile (cache miss and cache hit requests) HTTPS stress test, some sessions were left hanging after a 15 hour run. There were associated Data sources also leftover. These hanging sessions look to be a cache hit scenario as we saw Managed Cache Data Source being left over.

Conditions:

This happens only under stress. Not easily reproducible. Some corner case is resulting in this behavior. The exact scenario is yet to be identified. This happens when there is a disk file create failure.

CSCud01575

Symptom:

The Web Engine goes into a core dump on long URLs.

Conditions:

The request is a POST message that is posted by the client for the WMT translog.

CSCuc92136

SE goes offline during upgrade from 2.6.3 to 3.1.0/3.1.1.

Conditions:

Upgrade from 2.6.3 to 3.1.x when there are more than 25 delivery services.

CSCub58009

Symptom:

Additional requests for revalidation of content despite 304 response within cache timeout.

Conditions:

Additional requests for revalidation of content despite 304 response within cache timeout.

CSCts99053

Symptom:

1. All the 4 SRs and 1 backup CDSM were reported to be down on the primary CDSM. Only these 5 devices were observed to flip between online and offline modes while the SE's status seem to be ok.

2. There were no reported interruption to the end user. But the CDN monitoring system (CDSM) is reported to be unreliable.

3. Huge /local/local1/logs/rpc_httpd/ssl_scache.pag file size (~44GB) on the primary CDSM.

4. No core files observed.

Conditions:

SRs/SEs/backup CDSM send http[s] messages to the primary CDSM which is handled by the rpc_httpd process on the CDSM. These requests are the http[s] messages that report the health of the various nodes to CDSM.

Apache(rpc_httpd) uses ssl_scache.pag file to speed up parallel request processing by avoiding unnecessary session handshakes. At every SSLSessionCacheTimeout interval the global/inter-process SSL Session Cache information is timed out, with the httpd process acquiring a lock and traversing the records. Due to the size of the file (44gb) this operation is taking excessively long time thereby blocking other processes from reading the file for session information.

Since the customer enabled "Fast SE Offline Detection mode", the SEs health is communicated to the CDSM using UDP messages (and not the http[s] mechanism). This corresponds to what was observed, where only the backup CDSM and the SRs were offline, while the SEs were reported to be online.

Happens once every 6 months (with 50+ SEs, 4SRs and 1 backup CDSM communicating with the primary CDSM using SSL.

CSCuc99872

Symptom:

System is not functioning well, such as wget to SE may hang, SE may become very slow, SR may see SE offline, and so on.

Conditions:

Installing the software by using the installation CD or rescue CD may result in this problem.

CSCuc37840

Symptom:

The files delivered through cache bypass flow may have a corrupted body with contents beginning at offset 0 repeats later in the body.

Conditions:

This problem happens in following scenarios:

If "no cache on disk" is enabled and the file requested is a large file (greater than 2 MB default value or configured value), then cache bypass flow is invoked causing the body not drained from the socket.

Second scenario is when "no-cache" or "no-store" is in the HTTP header from Origin server, then cache bypass flow is invoked causing this issue.

CSCuc80533

Symptom:

HLS streaming stops when HLS session tracking is enabled.

Conditions:

The Origin server configures "private" or "no-store" directive in HTTP response Cache-Control header.

CSCuc49835

Symptom:

WebEngine in edge SE bypasses Content Acquirer and reaches Origin server directly.

Conditions:

1. Edge SE, which means serving client directly

2. HTTP head request

3. The requesting content is cache-hit and expired and need revalidation

4. There is an existing Datasource for the requesting url but *no* existing DataSourceFinder for the request url.

CSCua71204

Symptom:

Users may get HTTP 404 errors when the new "Follow Origin Server Redirect" feature is enabled on CDS delivery service. This occurs because the hostname is not being re-written to what is listed in the 302 redirect location.

Conditions:

Enable the new "Follow Origin Server Redirect" feature on a delivery service. Redirected web server needs proper hostname to route traffic to proper application pool.

CSCtz50130

Symptom:

Custom access logs (transaction logs) shows large value in the column which corresponds to the "Bytes-Transferred-Excluding-Header" when no data is sent out to the downstream SE, for example:

x.x.x.x y.y.y.y 18446744073709551288 ....

(ClientIP) (SEIP) (BytesTransferred Excluding Header) ...

Conditions:

Only during overloaded conditions when the SE or Content Acquirer takes too long to send the data out to the downstream SE. But the downstream SE (after waiting for a certain time; default= 5secs), disconnects from the upstream SE and sends a 504-Gateway timeout to the client.

Windows Media Streaming

CSCuc72511

Symptom:

WMT process generate coredumps frequently.

Conditions:

WMT received a specific transaction log POST message from client.

CSCub98887

Symptom:

Whenever the end user changes the resolution of the video, a garbled video image is seen and the CDS operator has to restart the live program through CDSM GUI.

Conditions:

End user changes video resolution.

Content Manager

CSCua08680

Symptom:

When the clear-cache-all command is entered, sometimes the content is not completely deleted. This is because the Content Manager is not aware of all the content cached by the Web Engine.

Conditions:

When the Web Engine creates more than three million objects and the slow scan (slowscan) process has just finished running, some content is not known by the Content Manager.

CLI

CSCub25508

Symptom:

NTP server is accepting an invalid IP address range.

Conditions:

While configuring the NTP server IP, the IP address should validate.

CSCuc31049

Symptom:

In CLI prompt, RTSP max-request-rate shows a range of <1-2147483647>.

Conditions:

When the rtsp advanced max-request-rate ? command is entered.

CSCuc02673

Symptom:

Upgrade failure from Release 2.6.3 to 3.1.

Conditions:

System has just been upgraded from older release to Release 2.6.3, Disk RAID is still sync pending.

CSCuc00495

Symptom:

Cannot access server via console. Keep getting "Username" prompt and "System Initializing. Please wait....."

Conditions:

Upgrade "failure" from Release 2.6.3 to Release 3.1.

Movie Streamer

CSCuc01228

Symptom:

Movie Streamer does not return error code for invalid URL signature.

Conditions:

Service rule is enabled and a request with signed URL is sent to the Movie Streamer engine.

CSCua57200

Symptom:

When show programs, a DBAccess error appears

Conditions:

200 programs are created.

CDSM

CSCud24328

Symptom:

After downgrading from 3.1.2 to 2.6.3, can not log in to the new primary CDSM, and communication is lost between primary CDSM and standby CDSM.

Conditions:

This may happen when database was manually edited and had some old entries in there. This should rarely happen in production network.

CSCuc70165

Symptom:

Device states turn yellow and red when lowering the quota for content services

Conditions:

1. Lowered the quota by 100MB on all Content services (as opposed to just 1 service last week in my previous testing).

2. Straight away they all turn yellow.

3. After about 3min they start turning red.

4. Then about 2min later they become green.

CSCuc67508

Symptom:

The rpc_httpd process (the Apache process) goes in to a core dump when reloading.

Conditions:

When the device is reloading, the rpc_httpd process is exiting.

CSCuc32754

Symptom:

Unable to enter in Management Communication Address in Device Activation Screen when it's register the first time and inactive.

The reason is that customer need to do more steps to activate and deactivate to enter the Management Communication Address in Device Activation Screen. As production have around 90 plus devices to upgrade & activate, they looking for the quickest way possible to do this to reduce the workload.

Conditions:

Unable to enter in Management Communication Address in Device Activation Screen when it's register the first time and inactive.

CSCuc27344

Symptom:

When CDSM configuration for NTP apply to CDSM. The previous configuration still persist.

Conditions:

Applying the CDSM configuration for NTP applied to CDSM.

Steps To Reproduce the issue:

1. In the CDSM GUI, choose Devices > Devices (CDSM) > General Settings > Network > NTP

2. Enter the IP Address of NTP, example:

    1.1.1.1     2.2.2.2     3.3.3.3        4.4.4.4
 
   

3. Click Submit.

4. SSH to CDSM device, enter the show running-config| in ntp command.

5. This will list the ntp server

    cde-205-b-cdsm1# show running-config | in ntp 
    ntp server 1.1.1.1 
    ntp server 2.2.2.2
    ntp server 3.3.3.3
    ntp server 4.4.4.4
 
   

6. Change the NTP Configuration, remove 1.1.1.1, so below:

     2.2.2.2     3.3.3.3     4.4.4.4
 
   

7. However the list of the NTP server in the CDSM

    cde-205-b-cdsm1# show running-config | in ntp 
    ntp server 1.1.1.1 
    ntp server 2.2.2.2
    ntp server 3.3.3.3
    ntp server 4.4.4.4
 
   

8. After 5 minutes or so, the CDSM will have the previous configuration with all 4 IP addresses configured.

CSCua58973

Symptom:

After reload, rules do not apply immediately.

Conditions:

It happens only for a short period after reloading the device.

CSCuc03325

Symptom:

CDSM GUI may become slow or unresponsive.

Conditions:

When devices send alarms to CDSM.

CSCub42692

Symptom:

SE cannot be unassigned from the delivery service after downgraded the CDSM from Release 3.1 to Release 3.0. When unassigning an SE from a delivery service, the operation is rejected, with an error message saying this SE cannot be unassigned, because it is the Sender of a Multicast Cloud assigned to the delivery service.

Conditions:

Before downgrading the CDSM from Release 3.1, Multicast Clouds are configured and the SE is the Sender in the Multicast Cloud, and this Multicast Cloud has been assigned to the delivery service. And, the downgrade script "cms database downgrade script Downgrade3_1_to_3_0"(documented in the Release Notes for 3.1 was not run before running "cms database downgrade" on the CDSM.

CSCuc77655

Symptom:

When multiple delivery services are created and a new delivery service is added, the keep alive message (heartbeat) may not sent out in a timely manner. Use the tcpdump udp port 2000 command to monitor the packets sent out from the SE; some delay can be seen.

Conditions:

Add or delete a delivery service, the keep live message is missed or delayed.

CSCuc40705

Symptom:

A delivery service is showing a replication status of yellow and "No status reported."

Conditions:

When the content type of a delivery service has zero pre-position storage quota.

CSCuc38057

Symptom:

Cannot configure single word domain name in CDSM. Following error is seen:

------------------------------------------------
Transaction not completed.
Domain Names must match following rules:
1. Each label between dots must be no less than 1 character and no more than 63 
characters. 
2. Each label between dots must start with letters and end with letters or numbers. 
3. Only letters, numbers and "-" are valid for labels between dots.
------------------------------------------------
 
   

Conditions:

Configure DNS in CDSM under General Settings > Network > DNS

CSCua58973

Symptom:

After reload, rules do not apply immediately.

Conditions:

It happens only for a short period after reloading the device.

Authorization Server

CSCua65921

Symptom:

The Service Rule does not include a left parenthesis "("when the NOT ("!") operator is used.

Conditions:

When the Internet Streamer CDS is configured with service rules by using the Service Rule XML file and if the rule has !(<some regex>), then the left parenthesis after the NOT operator is not considered.

CSCuc93205

Symptom:

The Authorization Server returns a 302 message in release 2.6.3 compared to a 403 message in release 2.5.11. This happens when the Geo Server fails during a URL signing pass.

Conditions:

URL signing is applied and the error-redirect-url is configured.

Content Abstraction Layer (CAL)

CSCub06673

Symptom:

HIgh disk usage.

Conditions:

Slow scan in progress.

Web Services

CSCub33092

Symptom:

Based on the investigation over the error logs and the web-engine code, the root cause is:

1. Some existing data sources in the memory ended up with a bad status due to content downloading being terminated by OS HTTP response body read timeout 504 (due to OS being overloaded)

2. At the same time, many new requests kept hitting the existing data sources

3. Before using those data sources, there is not status check. Therefore, 500 was returned to client

4. Since new requests kept hitting the data sources, the data sources could not be evicted from the memory

In other words,

1. OS 504 causes data source in bad status;

2. Heavy traffic causes data sources hanging in the memory

3. Bad status causes 500 returned to client

Conditions:

Origin server is under heavy load.

Syslog

CSCud09062

Symptom:

In syslog.txt, the following lines appear:

Nov 5 14:12:29 edgecache501-cardiff wmt_be: %SE-SYSLOGD-4-581002: pid-2426: 
HTEndLoop=0, should set timeout! 
 
   

Conditions:

WMT HTTP and acquirer.

Service Monitor

CSCub41468

Symptom:

When service_monitor process is hung, SE state inside SR is flapping. The interval of UDP packet for keepalive is increased from 2 seconds to 8 seconds. It is because there are multiple SR devices in production network. Each query will add extra delay.

Conditions:

SE service_monitor process is hung.

Service Routing

CSCud33043

Symptom:

SR is using streaming interface's ip address for its PRC call back to SE. And RPC call is not successful because SE's streaming interface is not reachable from SR.


Note The PRC call is used between SR and SE to synchronize delivery service's flag in wholesale feature.


Conditions:

1. SE is using primary interface to send KAL message to SR

2. The streaming interface's ip address is not reachable from SR

CSCud11578

Symptom:

There is no external visible symptom. This is an internal code enhancement for reliability.

Conditions:

This is related to whole-sale feature.

CSCuc45864

Symptom:

Lots of established TCP connections do not timeout.

Conditions:

Client connects to SR (TCP port 80/554) and does not send any requests.

CSCuc12956

Symptom:

The Service Router has a core dump while checking the SR statistics for the API request and 302 redirect.

Conditions:

1. Send SR API request from browser. The syntax of the API is as follows\:

http://ServicerouterIP/routeURL?CDNURL=<RequestedURL>&ClientIP=<IPOfTheClient>;
 
   

2. Specify the SR 302 redirect request from the client in the same network as given in API request.

3. Check the SR statistics for the API request and 302 redirect.

CSCub41474

Symptom:

The ability of the content based routing feature is limited when URL signing is used. Content based routing generates a hash on the whole URL which includes the signed part and therefore is always unique. So all traffic go to origin server.

Conditions:

When URL signing feature is used.

Acquisition and Distribution

CSCub52883

Symptom:

The correct play length is not displayed for .wmv files.

Conditions:

When WMV files are prefetched, the play length is displayed as 00:00:00.

Platform

CSCty93920

Symptom:

False Power Supply Temperature alarm(s) seen from the show alarms command.

Major Alarms:
-------------
     Alarm ID             Module/Submodule     Instance
     -------------------- -------------------- -------------------------
   1 temp                 sysmon               System Temperature                      
     Mar 26 20:26:07.828 UTC, Equipment Alarm, #000002, 1000:445009
     System temperature threshold(s) exceeded.
     Please run 'show tech-support';
     check the System Health section for details.
 
   

Conditions:

Enter the show tech command, and look for the "-- system health --"section.

---------------------- system health -----------------------
 
   
 
   
*****************************************************************************
 Supero Doctor II - Linux version 2.79(110504)
 Copyright(c) 1993-2010 by Super Micro Computer, Inc. http://supermicro.com/
*****************************************************************************
Monitored Item            High Limit  Low Limit     Status
----------------------------------------------------------------------
Fan2 Fan Speed                              712       6221
Fan3 Fan Speed                              712       6459
Fan8/CPU2 Fan Speed                         712       6279
Power1 Fan 1 Speed                         4007       5267
Power1 Fan 2 Speed                         4007       7099
CPU1 Vcore Voltage              1.48       0.92       1.09
CPU2 Vcore Voltage              1.48       0.92       1.08
-12V Voltage                  -10.58     -13.40     -12.21
+1.5V Voltage                   1.98       1.34       1.52
+3.3V Voltage                   3.63       2.96       3.33
+12V Voltage                   13.25      10.75      11.81
+5V Voltage                     5.50       4.49       4.92
+5Vsb Voltage                   5.50       4.49       4.87
VBAT Voltage                    3.66       2.99       3.17
PECI Agent1 Temperature       93/199                   Low
PECI Agent2 Temperature       93/199                   Low
System Temperature            75/167                 25/77
Power1 Temperature            0/32                 27/80     Warning!    <-- Invalid 
High Limit  (0/32)
Power2 Temperature            0/32                 110/230   Warning!    <-- Invalid 
High Limit  (0/32)
Chassis Intrusion                                      Bad   Warning!
Power Supply Failure                                   Bad   Warning!
Power1 Supply Failure                                 Good
Power2 Supply Failure                                  Bad   Warning!
--------------------------------------------- Mon Apr  9 09:44:50 2012
 
   

If you see "0/32" for either Power1 / Power2 Temperature High Limits,

then the Power Supply Temperature alarm(s) is / are false and can be ignored.

CSCub85851

Symptom:

PSU alarm is raised when CDE205 has only a single PSU.

Conditions:

If CDE205 has only a single PSU, an alarm will be raised.

CSCub63675

Symptom:

Write Memory Errors displayed; "No space left on device" /state partition not mounted to the appropriate internal SSD partition.

Conditions:

Trying to save running config to startup flash, this is seen on new CDE250 boxes.

Transaction Logs

CSCuc31017

Symptom:

Alarms from ftp_export does not get cleared.

Conditions:

The instance of the alarm (address of ftp-server) is stuck. The configuration of the address has been removed.

CSCuc59183

Symptom:

The ftp_export process was stuck.

Conditions:

The sftp-server (SSH) process does not respond to the ftp_export and does not break the connection.

RTSP Gateway

CSCuc32752

Symptom:

Alarm is not clearing despite rtspg logs showing that there is no traffic hitting the device, at the same time the service_monitor_errorlog is filled up with threshold exceeded alarm.

Major Alarms:
-------------
     Alarm ID             Module/Submodule     Instance
     -------------------- -------------------- -------------------------
   1 tpsquotaexceed       streaming/rtspg      rtspg_tps_40  
 
   

Conditions:

These were generated during a change where 11 SEs were removed from a custom Device Group and assigned to "BASELINE" Device Group.

MP3 Live

CSCuc41744

Symptom:

When the CDS serves an AAC-encoded shoutcast stream (Content-type: audio/aacp), it also adds the HTTP header "Content-length: 0". This header is normally used to indicate there is no data to be transferred.

Some clients like VLC ignore this header and play the stream properly, but, some other clients such as Iphone/Ipad honor the header and immediately close the connection.

Conditions:

When the CDS serves an AAC-encoded shoutcast stream.

Cache Router

CSCuc70270

Symptom:

Cache router hash level calculation goes wrong if the URL used for calculating starts with "xxx.se."

Conditions:

If the OFQDN has the format "xxx.se.yyy," it is stripped by the Web Engine, and makes the hash level use the default value 0.

Data Server

CSCud15708

Symptom:

Service Router does not serve new added delivery service\

Conditions:

There are some alarm need to raise simultaneously on the SR, a race condition could happen.

Accessing Bug Tool kit

This section explains how to use the Bug Toolkit to search for a specific bug or to search for all bugs in a release.


Step 1 Go to http://tools.cisco.com/Support/BugToolKit.

Step 2 At the Log In screen, enter your registered Cisco.com username and password; then, click Log In. The Bug Toolkit page opens.


Note If you do not have a Cisco.com username and password, you can register for them at http://tools.cisco.com/RPF/register/register.do.


Step 3 To search for a specific bug, click the Search Bugs tab, enter the bug ID in the Search for Bug ID field, and click Go.

Step 4 To search for bugs in the current release, click the Search Bugs tab and specify the following criteria:

Select Product Category—Video.

Select Products—Cisco Content Delivery Engine Series.

Software Version—[3.1].

Search for Keyword(s)—Separate search phrases with boolean expressions (AND, NOT, OR) to search within the bug title and details.

Advanced Options—You can either perform a search using the default search criteria or define custom criteria for an advanced search. To customize the advanced search, click Use custom settings for severity, status, and others and specify the following information:

Severity—Choose the severity level.

Status—Choose Terminated, Open, or Fixed.

Choose Terminated to view terminated bugs. To filter terminated bugs, uncheck the Terminated check box and select the appropriate suboption (Closed, Junked, or Unreproducible) that appears below the Terminated check box. Select multiple options as required.

Choose Open to view all open bugs. To filter the open bugs, uncheck the Open check box and select the appropriate suboptions that appear below the Open check box. For example, if you want to view only new bugs in Prime Optical 9.5, choose only New.

Choose Fixed to view fixed bugs. To filter fixed bugs, uncheck the Fixed check box and select the appropriate suboption (Resolved or Verified) that appears below the Fixed check box.

Advanced—Check the Show only bugs containing bug details check box to view only those bugs that contain detailed information, such as symptoms and workarounds.

Modified Date—Choose this option to filter bugs based on the date when the bugs were last modified.

Results Displayed Per Page—Specify the number of bugs to display per page.

Step 5 Click Search. The Bug Toolkit displays the list of bugs based on the specified search criteria.

Step 6 To export the results to a spreadsheet:

a. In the Search Bugs tab, click Export All to Spreadsheet.

b. Specify the filename and location at which to save the spreadsheet.

c. Click Save. All bugs retrieved by the search are exported.

If you cannot export the spreadsheet, log into the Technical Support website at http://www.cisco.com/cisco/web/support/index.html or contact the Cisco Technical Assistance Center (TAC).

Upgrading to Release 3.1.2

Release 3.1.2 supports upgrades from Release 2.5.9, Release 2.5.11, Release 2.6.x, Release 3.0.x, and Release 3.1.x.


Note When upgrading from Release 2.x to Release 3.x, and the streaming interface IP address is set as the management IP address on the SE, the rpc_httpd process fails to start because of a port 443 binding issue. Before upgrading to Release 3.x., either check the Use SE's primary IP address for management communication check box on the Device Activation page for the SE, or use a separate IP addresses for the streaming traffic and the management traffic (primary interface).



Note If your CDS software is older than Release 2.6.1 and you have CDE205 and CDE220 platforms in your system, you must check that the partition size (specifically, disk 00/02), on each CDE205 and CDE220 in your system is larger than 0.5 GB. To check the partition size, enter the show disks detail command. If the disk00/02 partition is not larger than 0.5 GB, you must upgrade the CDE to Release 2.6.1 before upgrading to Release 3.x.


If your CDS is running an older release than Release 2.5.9, you need to upgrade to Release 2.5.9 or 2.5.11 before upgrading to Release 3.1.2.

When upgrading from Release 2.5.9 or 2.5.11, all content is erased. For Service Engines, this means that prefetched metadata and content need to be redistributed from upstream SEs after the upgrade, and that cached content is not preserved. Additionally, Flash Media Streaming service rules must be converted from device-based service rules to the Service Rule XML file. For more information on upgrading from Release 2.5.9 and 2.5.11, see the Cisco Internet Streamer CDS 2.6 Software Upgrade Guide (http://www.cisco.com/en/US/docs/video/cds/cda/is/2_6/upgrade_guide/upgrade.html).

We strongly recommend that you upgrade your CDS network devices in the following order:

1. Multicast sender Service Engines

2. Multicast receiver Service Engines

3. Edge Service Engines

4. Middle-tier Service Engines

5. Content Acquirers

6. Service Routers

7. Standby CDSMs (Upgrade before primary when using the GUI only.)

8. Primary CDSM


Note When using the CDSM GUI to upgrade from Release 2.5.9, 2.5.11, or 2.6.1 to Release 3.1.2, after you upgrade the standby CDSM, if you switch roles of the standby CDSM and primary CDSM to maintain an active CDSM, the old primary CDSM is now the standby CDSM, and the old standby CDSM is now the primary CDSM. At this point, you must use the CLI to upgrade the new standby CDSM. The primary CDSM GUI cannot upgrade the standby CDSM.

Alternatively, if you do not switch roles of the standby CDSM and primary CDSM, you can use the CDSM GUI to upgrade the primary CDSM. The primary CDSM loses connectivity with the CDS devices for a short time during the upgrade, but this is not service affecting.

When using the CDSM GUI to upgrade from Release 2.6.3 and later releases to Release 3.11, after you upgrade the standby CDSM, if you switch roles of the standby CDSM and primary CDSM to maintain an active CDSM at all times, the new primary CDSM GUI can be used to upgrade the new standby CDSM.


For more information on the upgrade procedure, see the Cisco Internet Streamer CDS 3.1 Software Configuration Guide.

After the upgrade procedure starts, do not make any configuration changes until all the devices have been upgraded.

Downgrading from Release 3.1.2


Note When downgrading from Release 3.1.2 to Release 2.5.x, make sure the Coverage Zone file is less than 7500 lines. If the Coverage Zone file is 7500 lines or greater, reduce the number of lines before downgrading to Release 2.5.x. This is not an issue when downgrading to Release 2.6.x.


For software downgrades from Release 3.1.2 on systems with primary and standby CDSMs, you need to do the following:


Step 1 If you are using the CDSM GUI, downgrade the standby CDSM first, followed by the primary CDSM.

If you are using the CLI, downgrade the primary CDSM first, followed by the standby CDSM.

Step 2 After downgrading the primary and standby CDSMs, using the CLI, log in to each CDSM and run the following commands:

To downgrade from 3.1.2 to 2.5.9 or 2.5.11

cms database downgrade script downgrade/Downgrade3_1_1_to_3_1
cms database downgrade script downgrade/Downgrade3_1_to_3_0
cms database downgrade script downgrade/Downgrade3_0_to_2_6
cms database downgrade
cms enable
 
   

Then, consult the "Downgrading the Internet Streamer CDS Software" chapter in the Cisco Internet Streamer CDS 2.6 Software Upgrade Guide for downgrading from Release 2.6.x to Release 2.5.9 or 2.5.11.

To downgrade from 3.1.2 to 2.6.x

cms database downgrade script downgrade/Downgrade3_1_1_to_3_1
cms database downgrade script downgrade/Downgrade3_1_to_3_0
cms database downgrade script downgrade/Downgrade3_0_to_2_6
cms database downgrade
cms enable
 
   

To downgrade from 3.1.2 to 3.0.x

cms database downgrade script downgrade/Downgrade3_1_1_to_3_1
cms database downgrade script downgrade/Downgrade3_1_to_3_0
cms database downgrade
cms enable
 
   

To downgrade from 3.1.2 to 3.1.0

cms database downgrade script downgrade/Downgrade3_1_1_to_3_1
cms database downgrade
cms enable
 
   

Step 3 Downgrade the software on the Service Routers, followed by the Service Engines.


Note If you are downgrading the CDSM from Release 3.0.0 to Release 2.6.x, after running the cms database downgrade command, run the downgrade/Downgrade3_0_to_2_6 command.


Documentation Updates

The following document has been added for this release:

Release Notes for Cisco Internet Streamer CDS 3.1.2

Related Documentation

Refer to the following documents for additional information about Cisco Internet Streamer CDS 3.1:

Cisco Internet Streamer CDS 3.1 Software Configuration Guide

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_1/configuration_guide/icds31confg.html

Cisco Internet Streamer CDS 3.0-3.1 Quick Start Guide

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_0/quick_guide/ISCDSQuickStart.html

Cisco Internet Streamer CDS 3.1 API Guide

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_1/developer_guide/iscds31APIGuide.html

Cisco Internet Streamer CDS 3.1Command Reference Guide

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_1/command_reference/Command_Ref.html

Cisco Internet Streamer CDS 3.1 Alarms and Error Messages Guide

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_1/message_guide/message_guide.html

Cisco Internet Streamer CDS 3.0--3.1 Software Installation Guide for non-CDEs

http://www.cisco.com/en/US/docs/video/cds/cda/is/3_0/install_guide/Non_CDE_IS_3_0_Software_Install.html

Cisco Content Delivery System 3.x Documentation Roadmap

http://www.cisco.com/en/US/docs/video/cds/overview/CDS_Roadmap3.x.html

Open Source Used in Cisco Internet Streamer CDS 3.1

http://www.cisco.com/en/US/docs/video/videoscape/Media_Processor_Mgmt_Console/OL-27694-01_CDS-IS_3.1_Open_Source.pdf

Cisco Content Delivery Engine 205/220/250/420 Hardware Installation Guide

http://www.cisco.com/en/US/docs/video/cds/cde/cde205_220_420/installation/guide/cde205_220_420_hig.html

Regulatory Compliance and Safety Information for Cisco Content Delivery Engines

http://www.cisco.com/en/US/docs/video/cds/cde/regulatory/compliance/CDE_RCSI.html

Cisco UCS C200 Installation and Service Guide

http://www.cisco.com/en/US/docs/unified_computing/ucs/c/hw/C200M1/install/c200M1.html

Cisco UCS C210 Installation and Service Guide

http://www.cisco.com/en/US/docs/unified_computing/ucs/c/hw/C210M1/install/C210M1.html

The entire CDS software documentation suite is available on Cisco.com at:

http://www.cisco.com/en/US/products/ps7127/tsd_products_support_series_home.html

The entire CDS hardware documentation suite is available on Cisco.com at:

http://www.cisco.com/en/US/products/ps7126/tsd_products_support_series_home.html

The Cisco UCS C-Series Rack Servers documentation is available on Cisco.com at:

http://www.cisco.com/en/US/products/ps10493/prod_installation_guides_list.html

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, submitting a service request, and gathering additional information, see the monthly What's New in Cisco Product Documentation, which also lists all new and revised Cisco technical documentation, at:

http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html

Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.

This document is to be used in conjunction with the documents listed in the "Related Documentation" section.