-
Cisco ACNS Software Configuration Guide for Centrally Managed Deployments, Release 5.2
-
Index
-
Title Page
-
Preface
-
Chapter 1: Understanding the ACNS 5.2 Network
-
Chapter 2: Using the Content Distribution Manager GUI
-
Chapter 3: Getting Started
-
Chapter 4: Setting Up Content Request Routing in the ACNS Network
-
Chapter 5: Configuring the ACNS Network for Content Distribution
-
Chapter 6: Configuring the ACNS Network for Content Acquisition
-
Chapter 7: Creating and Managing Programs
-
Chapter 8: Configuring Caching Services
-
Chapter 9: Configuring Streaming Media Services
-
Chapter 10: Working with Device Configurations
-
Chapter 11: Configuring Network Interfaces
-
Chapter 12: Configuring Request Processing Services
-
Chapter 13: Configuring Request Authentication and Authorization
-
Chapter 14: Configuring Login Authentication and Authorization
-
Chapter 15: Creating and Managing IP Access Control Lists
-
Chapter 16: Configuring Platform and System Settings
-
Chapter 17: Upgrading and Downgrading the Software
-
Chapter 18: Monitoring the ACNS Network
-
Chapter 19: Backup and Recovery Procedures
-
Appendix A: Creating Manifest Files
-
Appendix B: Configuring Disk Space
-
Appendix C: IP Multicast Addressing
-
Appendix D: Advanced Routing Configurations
-
Appendix E: Mapping ACNS Software CLI Commands to the Content Distribution Manager GUI
-
Table Of Contents
Configuring the ACNS Network for Content Acquisition
Preparing to Acquire Pre-positioned Content
Assigning a Backup Root Content Engine
Choosing a Content Acquisition Method for a Channel
Acquiring Pre-Positioned Content Using the Content Distribution Manager GUI
Adding Content Items to a Channel Using the Quick Crawl Utility
Adding a Crawl Task to a Channel
Configuring Content Acquisition Rules for a Crawl Task
Configuring Advanced Content Serving Settings
Modifying Multiple Content Items
Deleting Multiple Content Items
Acquiring Pre-Positioned Content Using an Externally Hosted Manifest File
Specify Channel Acquisition and Distribution Properties
Configure Manifest File and Proxy Information for the Channel
Configuring Host and Proxy Server Settings Using the Content Distribution Manager GUI
Configuring an HTTP Proxy Server Using the CLI
Configuring a Proxy Server Using the Manifest File
Configuring a Proxy Server and Port for the Manifest File
Acquiring Content that Requires NTLM Authentication
Configuring Authentication for a Proxy to Fetch the Manifest File
Configuring Authentication for a Proxy Using the Manifest File
Configuring Authentication for an HTTP Proxy Using the Content Distribution Manager GUI
Configuring Authentication for a Proxy Using the CLI
Configuring Authentication for a WCCP Proxy Using the Content Distribution Manager GUI
Modifying WCCP Proxy Authentication Settings
Configuring Authentication for a WCCP Proxy Using the CLI
Configuring Acquisition and Distribution Default Bandwidth Settings
Displaying a Graphical Representation of the Acquisition and Distribution Bandwidth Settings
Configuring Acquisition and Distribution Bandwidth Settings for Scheduled Times
Scheduling Bandwidth for Streaming Acquisition
ACNS Unified Name Space Errors
MMS-Specific Acquisition Error Codes
Configuring the ACNS Network for Content Acquisition
Content acquisition is an important part of content pre-positioning in the ACNS network. ACNS software uses the concept of a channel to map a set of content objects to a set of Content Engines. Before pre-positioned content can be acquired or distributed in the ACNS network, users must create a new channel, subscribe Content Engines to that channel, and designate one of the Content Engines as the root Content Engine. The source of the content might be stored on various file servers or web servers. The root Content Engine of the channel fetches the content objects from these origin servers and then replicates this content to all the Content Engines in the channel.
Note
Pre-positioned content is served only on ports that are standard for the protocol. If the incoming URL contains a port number other than the protocol's standard port (for example, HTTP uses port 80, and MMS uses port 1755), then the Content Engine does not attempt to serve the content from the pre-positioned file system (cdnfs). Instead, the Content Engine tries to serve the content from the cache file system (cfs) or tries to fetch the content from the origin server, depending on the existing configuration of the Content Engine. To server pre-positioned content from a nonstandard port, set the ignoreOriginPort attribute to true in the manifest file. For a description of this attribute, see the "item" section on page A-44.
This chapter outlines the procedures necessary for acquiring pre-positioned content in the ACNS network and contains information on the following topics:
•
•
•
•
Preparing to Acquire Pre-positioned Content
To prepare your ACNS network to acquire pre-positioned content, you must complete the following tasks:
3.
About the Acquirer
The root Content Engine uses a software agent, referred to as the acquirer, that gathers channel content before it is distributed to the receiver Content Engines in the ACNS network. The acquirer maintains a task list, which it updates after receiving a notification of changes in its channel configuration. Note the following items that are supported or not supported by the acquirer:
File Acquisition from SMB Servers
ACNS 5.2 software supports file acquisition from Windows file servers with shared folders and Unix servers running the SMB protocol. The acquirer first mounts the share folder to the root Content Engine. This mount point then acts as the origin server from which the content is to be fetched. The acquirer fetches the content and pre-positions it in the cdnfs storage space. (Note that files greater than 2 GB cannot be acquired using SMB.)
SMB is supported in the Content Distribution Manager GUI for fetching an externally hosted manifest file. The Content Distribution Manager GUI content import feature also supports SMB file acquisition for content items and crawl tasks.
You can pre-position content items and crawl tasks from SMB servers by using the following URL format: \\SMBserver\sharefolder\filepath. The full URL can be specified in the manifest file <item> or <crawler> tags. For example:
<CdnManifest> <server name="myserver1"> <host name="\\<SMB-Server>" /> </server> <item src = "<Share Folder>\<File>" server="myserver1"/> <item src="\\<SMB-Server>\<Share Folder>\File" /> </CdnManifestFile acquisition from an SMB server for crawl jobs is similar to FTP acquisition in which the crawler crawls the folder hierarchy rather than parsing the HTML file.
You can specify the port to be used for the SMB protocol in the URL or in the port attribute. If the port is not specified, the default port is 139.
Authentication for SMB server file acquisition is supported by means of the user, password, and userDomainName attributes in the <host> and <item> tags. The userDomainName attribute is new in ACNS 5.2 (see the "item" section on page A-44 for a description of this attribute.) These attributes must be specified separately rather than as part of the full URL. Proxy and proxy authentication support are not available for this feature.
Note
EMBED and OBJECT ID Tag Support
If an HTML file includes an EMBED tag or an OBJECT ID tag, the acquirer parses the file, obtains the file link from the src and filename attributes, and fetches these files.
Crawl .asx File
The acquirer can parse an .asx file and fetch its contents.
expires Attribute Specified in the Manifest File
The expires attribute, specified as part of the <item> tag or <crawler> tag in the manifest file, designates a time in yyyy-mm-dd hh:mm:ss format for content to be removed.
If you specify the expires attribute in the manifest file, the root Content Engine deletes the content at the time specified. The root Content Engine then replicates the deletion event to receiver Content Engines, which also delete the content from the channel.
Parsing Scripts in an HTML File
If an HTML file contains a JavaScript or a VBScript (Microsoft's Visual Basic Scripting Edition is a scaled-down version of Visual Basic), and the script codes have links, the acquirer does not parse the script code to get the links.
Cookie Support
Cookies are a general mechanism that server-side connections (such as CGI scripts) use to both store and retrieve information on the client side of the connection. The addition of a simple, persistent, client-side state significantly extends the capabilities of web-based client/server applications. Although the acquirer is also a client, it does not support cookies.
Set Up the Origin Server
Content is usually stored on file servers or web servers that are not part of the ACNS network of managed devices. In order to pre-position this content for ACNS network acquisition and distribution, the origin file server or web server must support at least one of the following protocols that are used by ACNS software to acquire content:
•
•
•
•
•
Using HTTP and HTTPS
Any standard web server supports the HTTP and HTTPS protocols. You can set up your web server as an origin server for pre-positioned content intended for the ACNS network by moving the content over to the web server or by configuring the web server to access the desired content. The following two web servers are the most popular:
•
•
For the HTTP and HTTPS protocols, content can be fetched as single content items by using the <item> tag in the manifest file, or content can be fetched by using the crawling feature to crawl web server directories. The crawler crawls the folder hierarchy rather than parsing the HTML file. Therefore, if you want to use the crawl feature, you must enable directory indexing and make sure that the directory does not contain index.html, default.html, or home.html files.
Tip
Redirection Support
For a single-item HTTP redirection, the acquirer follows the redirect to the new link to obtain the file. If the HTTP redirect is a crawl job, the acquirer checks the new link against the crawl parameters. For example, if a start-url http://www.ibm.com/ is redirected to http://www.cnn.com/, the acquirer checks www.cnn.com to see if the start-url matches the crawl parameters. Unless the externalPrefixes attribute is specified, the redirection is usually rejected because it does not match the default prefix.
HTTP Response no-cache Directive
The no-cache directive is related to the HTTP protocol. The HTTP protocol uses headers to communicate between the client and the server. The client sends a request header, and the server responds with a response header. Headers specify the various attributes for the resource being requested. For example, the client sends the request:
GET /index.html HTTP/1.0Host:www.google.comThe server might send a response, as follows:
HTTP/1.0 200 OKContent-Length:10pragma:no-cacheCache-control:no-cacheThe no-cache directive in the response header tells the client that the content being requested is not cacheable. When an HTTP server responds to a request with a no-cache directive in the header, the acquirer behaves as follows:
•
•
HTTP Response expires Directive
An expires directive in the HTTP response header indicates that the content being served is not valid after the time specified in the response. For example, if the response header contains
HTTP/1.0 200 OKexpires:<some time>the client (the acquirer in this case) is directed not to use the content after the time specified. The expires directive is honored only for content that is specified in the <crawler> tag. Single items configured using the <item> tag in the manifest file do not honor the expires directive sent by the server.
Using FTP
The root Content Engine acquirer supports acquiring files from FTP servers. When you use FTP, content can be acquired as single content items by using the <item> tag in the manifest file, or content can be fetched by using the crawling feature to crawl the FTP server directories. In FTP acquisition, the crawler crawls the folder hierarchy rather than parsing the HTML file. The following popular FTP servers are supported:
•
•
•
•
Other supported Windows FTP servers are:
•
•
•
•
You can use other FTP servers, as long as the following FTP commands are supported:
•
•
•
•
•
Secure FTP
The acquirer currently does not support secure FTP.
Using MMS and MMS-over-HTTP
The root Content Engine supports acquiring streaming media files using either the MMS protocol or MMS-over-HTTP. In the manifest file, MMS-over-HTTP can be specified as MMS-HTTP or simply as HTTP.
When the MMS protocol is specified, MMST is used to fetch the stream over TCP. When MMS-HTTP is specified, the software uses the MMS-over-HTTP protocol to download the stream. When HTTP is specified, the software automatically detects whether the download is a regular HTTP download or a streaming MMS over HTTP download, and uses the corresponding protocol.
Users sometimes need to pre-position streaming content that is hosted on nonstandard non-Microsoft Windows Media Technologies (WMT) servers. Using the MMS-HTTP protocol addresses this situation by allowing HTTP streaming acquisition from nonstandard origin servers that do not allow automatic switch-over from HTTP download to HTTP streaming download.
Note
To acquire content using the MMST protocol, make sure that port 1755 is open on the firewall. For MMS-HTTP, make sure that the HTTP port, which is typically port 80, is open.
Create a Channel
Channels map a set of content objects to a set of Content Engines. You must have channels configured before you can acquire pre-positioned content or distribute it. To create channels, see Chapter 5, "Configuring the ACNS Network for Content Distribution." Chapter 5 discusses how to create channels for content acquisition and distribution by configuring the following network elements:
1.
2.
3.
4.
Assign a Root Content Engine
A root Content Engine is used to acquire content for a channel. A channel can have only one root Content Engine. We recommend that you choose a root Content Engine that has enough bandwidth to access the content at the origin server.
For information on how to assign a root Content Engine, see the "Designating the Root Content Engine" section on page 5-14.
Assigning a Backup Root Content Engine
You do not need to specify a backup root Content Engine; however, for backup purposes, you must have a second Content Engine in the same location as the primary Content Engine that is subscribed to the channel. If the designated root Content Engine becomes inactive, the other Content Engine in the location automatically becomes a temporary root Content Engine. If the designated root Content Engine comes back online, it takes over as the root and the temporary root Content Engine becomes a regular Content Engine.
During content acquisition, the acquirer uses a lot of CPU power. We recommend that you set up a dedicated high-end root Content Engine for content acquisition. The root Content Engine can be used for streaming video files or serving pre-positioned content; however, streaming video quality might suffer during periods of heavy content acquisition.
Choosing a Content Acquisition Method for a Channel
When you configure a channel to acquire content, you must choose one of the following content acquisition methods:
•
Manifest files contain the XML tags, subtags, and attributes that you wish to use to define the parameters for content acquisition. You must be familiar with the structure of the XML-based manifest file and be sure that XML tags are properly formatted and syntactically correct before you can effectively create and use manifest files. (See the "Acquiring Pre-Positioned Content Using an Externally Hosted Manifest File" section and Appendix A, "Creating Manifest Files.")
•
The Content Distribution Manager GUI provides a user-friendly interface that you can use to easily add content items and specify crawl tasks without having to create and update a manifest file. The Content Distribution Manager GUI automatically validates all user input and generates an XML-formatted manifest file in the background that is free of syntax errors. (See the "Acquiring Pre-Positioned Content Using the Content Distribution Manager GUI" section.)
Only one manifest file is generated per channel for all content items. You can save your GUI-generated manifest file to any desired location.
To configure the content acquisition method for a channel, follow these steps:
Step 1
Step 2
Note
Step 3
Figure 6-1 Content Acquisition Method for Channel—Use Content Distribution Manager GUI
Use this window to add content items and specify crawl tasks for the channel. (See the "Acquiring Pre-Positioned Content Using the Content Distribution Manager GUI" section.)
Step 4
a.
b.
Note
c.
d.
Figure 6-2 Content Acquisition Method for Channel—Use External Manifest File
Use this window to define basic manifest settings and proxy information. (See the "Configure Manifest File and Proxy Information for the Channel" section.)
Acquiring Pre-Positioned Content Using the Content Distribution Manager GUI
ACNS 5.2 software allows you to pre-position content items and crawl tasks directly in the Content Distribution Manager GUI without having to create a manifest file. Use this method to set up simple demonstrations or systems, or to add a few items that do not require advanced acquisition and distribution features.
To pre-position content using the Content Distribution Manager GUI, follow these steps:
Step 1
Step 2
Step 3
Step 4
a.
b.
c.
See the"Choosing a Content Acquisition Method for a Channel" section.
Step 5
Figure 6-3 Content Manager Window
Step 6
a.
b.
c.
Step 7
a.
b.
Step 8
Step 9
Adding Content Items to a Channel Using the Quick Crawl Utility
Quick Crawl is a utility that automatically crawls websites starting from the specified source URL. You can use this utility when you know only the domain name and not the exact location of the content item. You can set a rule-based quick crawl filter so that only content with predefined characteristics will be matched for crawling. The results of the filtering are displayed, and you can select the content items that you want to be acquired. Quick Crawl supports crawling only for websites that use HTTP and HTTPS.
To add individual content items using the Quick Crawl utility, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Note
Step 7
Step 8
If depth = -1, there is no depth constraint.
If depth = 0, acquire only the starting URL.
If depth = 1, acquire the starting URL and all the content it references.Alternatively, you can choose to specify the link depth after you launch Quick Crawl.
Step 9
Note
Step 10
a.
b.
c.
d.
Alternatively, click the Calendar icon next to the Modified After and Modified Before fields to display the Date Time Picker popup window. In the Date Time Picker popup window, use the left or right arrow icons to choose previous and following years if required. Choose a month from the drop-down list. Click a day of the month. The chosen date is highlighted in blue. Click Apply. Alternatively, click Set Today to revert to the current day. The chosen date will be displayed in the Modified After or Modified Before field.
e.
f.
g.
If you have already entered the depth level in the Link Depth field of the Content Manager window, it is displayed here.
h.
i.
j.
Step 11
Once the specified number of items has been crawled, the window refreshes itself, and the Content Importer window appears. (See Figure 6-4.)
Figure 6-4 Content Importer Window
This window displays the associated MIME type, size, date modified, and the URL.
Step 12
Step 13
Step 14
Alternatively, click the Show Filter button to return to the Quick Crawl Filter window to change the filter settings.
Adding a Crawl Task to a Channel
The crawler application methodically and automatically searches websites and FTP sites that have met the specified crawl criteria and makes a copy of the visited pages for later processing. The crawler application starts with a list of URLs to visit and identifies every link on the page associated with that URL and adds these links to the list of URLs to visit.
The process ends after one or more of the following conditions are met:
•
•
•
If you use a manifest file, you can specify both the max-number and max-size attributes in the <crawler> tag as the criteria to stop a crawler job, whichever condition occurs first stops the job. For example, if the crawler has acquired the maximum number of objects specified in the manifest file, the crawler stops, even if it has not yet acquired the maximum content size.
If you use the Content Distribution Manager GUI, you cannot specify both attributes for crawl tasks. However, after you save the GUI-generated manifest file (click the Save settings locally icon), you can modify it and add additional XML tags manually using any text or XML editor.
Note
To add a crawl task to a channel using the Content Distribution Manager GUI, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
The value entered here is associated with the full URL of the start-url attribute of the <crawler> element. This value is also associated with the proto attribute of the <host> element. The proto attribute can be empty if the name attribute of the <host> element is a fully qualified domain name (FQDN).
Step 7
If depth = -1, there is no depth constraint.
If depth = 0, acquire only the starting URL.
If depth = 1, acquire the starting URL and all the content it references.This value is associated with the depth attribute of the <crawler> element.
Step 8
Step 9
To define optional content acquisition rules for the crawl task, see the "Configuring Content Acquisition Rules for a Crawl Task" section.
Configuring Content Acquisition Rules for a Crawl Task
To configure content acquisition rules, follow these steps:
Step 1
a.
b.
c.
d.
e.
Figure 6-5 Configuring Content Acquisition Rules for a Crawl Task
Step 2
Step 3
Step 4
Alternatively, click the Calendar icon next to the Time before and Time after fields to display the Date Time Picker popup window.
In the Date Time Picker popup window, use the left or right arrow icons to choose previous and following years, if required. Choose a month from the drop-down list. Click a day of the month. The chosen date is highlighted in blue. Alternatively, click Set Today to revert to the current day. The Time field displays the current system time. Edit the fields to configure a different time. Click Apply.
Step 5
Step 6
Step 7
Step 8
Note
Step 9
Step 10
Step 11
Configuring Advanced Content Serving Settings
In the Content Distribution Manager GUI under the Show advanced settings heading in the Content Manager window, you can configure the following settings:
•
•
•
•
By configuring these advanced settings, you can control the manner in which content is served by the Content Engines. These settings correspond to attributes in the manifest file and are associated with the <item> and <crawler> tags. These same attributes can also be specified in the <item-group> or <options> tags in a manifest file, so they can be shared by their <item> and <crawler> subtags.
To configure advanced settings for serving content, follow these steps:
Step 1
a.
b.
c.
d.
Alternatively, click the Add Content icon and configure the optional advanced settings along with the required settings for a new content item or crawl task.
e.
Figure 6-6 Advanced Settings for Serving Content—Top of Window
Figure 6-7 Advanced Settings for Serving Content—Bottom of Window
Step 2
a.
b.
c.
Alternatively, click the Calendar icon next to the Start serving time and Stop serving time fields to display the Date Time Picker popup window. In the Date Time Picker popup window the current date is highlighted in yellow by default. Use the left or right arrow icons to choose the previous or following years if required. Choose a month from the drop-down list. Click a day of the month. The chosen date is highlighted in blue. The Time field displays the current system time. Edit the fields to configure a different time. Click Apply. Alternatively, click Set Today to revert to the current day. The date and time chosen are displayed in the Start serving time or Stop serving time fields.
Step 3
a.
b.
If you leave this check box unchecked, NTLM authentication headers can be stripped off and fallback to the basic authentication method occurs. The username and password information are passed to the origin server in clear text with a basic authentication header.
c.
d.
e.
Step 4
a.
b.
c.
Step 5
a.
b.
c.
d.
Modifying Multiple Content Items
If multiple content items (single items and crawl tasks) need to be configured with the same content acquisition and distribution properties, you can select them and modify their link depth and advanced content serving settings in one instance. However, source URLs cannot be modified while you edit multiple content items and crawl tasks. Similarly, content acquisition rules, applicable for crawl tasks, cannot be modified for multiple crawl tasks. If you have selected both single items and crawl tasks for modification, even if you specify a common link depth for all crawl tasks, this value is not applied to single items.
To modify multiple content items, follow these steps:
Step 1
Step 2
Step 3
Step 4
•
•
Step 5
Step 6
Step 7
Step 8
Deleting Multiple Content Items
To delete multiple content items, follow these steps:
Step 1
Step 2
Step 3
Step 4
•
•
Step 5
Acquiring Pre-Positioned Content Using an Externally Hosted Manifest File
To configure your ACNS network to acquire content using a manifest file, you must complete the following tasks:
2.
3.
4.
Create a Manifest File
See Appendix A, "Creating Manifest Files," for details on creating manifest files.
After you create the manifest file, use the Manifest Validator utility to verify the syntax (see the "Manifest Validator Utility" section on page A-27). Next, specify the manifest URL in the Content Distribution Manager GUI. If authentication is required to fetch the manifest file, specify a username and password as well. (See the "Configure Manifest File and Proxy Information for the Channel" section.)
Specify Channel Acquisition and Distribution Properties
This section discusses four important content acquisition and distribution properties that you need to define:
•
•
•
•
Some of these properties are configured in the Content > Channels section of the Content Distribution Manager GUI and some are defined in the manifest file.
Distribution Priority
The distribution priority setting determines the priority of content acquisition and distribution. You configure this setting from the Distribution Priority drop-down list in the Content Distribution Manager GUI. The distribution priority values are High (750), Normal (500), or Low (250). Figure 6-8 shows the acquisition and distribution properties and the manifest properties that you can configure in the Content Distribution Manager GUI. See the "Creating a Channel" section on page 5-11 for channel configuration procedures and field descriptions.
Figure 6-8 Acquisition and Distribution Properties for a Channel
The priority of content acquisition also depends on the origin server. Requests from different origin servers are processed in parallel. Requests from the same origin server are processed sequentially by their overall priority. However, this does not hold true for Microsoft Media Server (MMS) and MMS-over-HTTP requests. All MMS and MMS-over-HTTP requests are sequentially processed by their overall priority, where overall priority = distribution priority * 10000 + item priority.
Item Priority
The item priority is determined by the manifest file. If the priority attribute is specified in the manifest file under the <item> tag, it is used as the item priority; if it is not specified, the order in which the item is listed in the manifest file (index number) is used as the item priority. (All crawled pages have the same item priority; therefore, you do not need to specify the priority attribute under the <crawler> tag.) See the "Specifying Content Priority" section on page A-12 for more information.
Channel Quota
The channel quota is the disk space allowed for the channel. You configure the channel quota in the Content Distribution Manager GUI (see Figure 6-8). See the "Configure Manifest File and Proxy Information for the Channel" section for configuration information.
When configuring the channel quota, keep in mind the following:
•
•
Because of overhead, the amount of disk space used by a file is always larger than the size of the file itself. To figure the amount of disk space needed for a file, follow these steps:
a.
(File size in KB / 4096) rounded up to the next integer value = Total number of blocks per file
b.
Total blocks per file * 4096 = Total disk usage in bytes
c.
Total disk usage in bytes + (4096 bytes * 4) = Disk usage per file
Also, because the software attempts to reserve enough space for other minor internal system functions, it is helpful to configure your channel quotas (and pre-positioned disk space) with a modest amount (perhaps 10 percent) of extra space beyond the total disk space consumed.
Channel quota in kilobytes = (Total disk usage in kilobytes) + (0.1 * Total disk usage in kilobytes)
Manifest File Update Interval
The update interval is the interval for the root Content Engine to check the manifest file itself. (This is not the update interval for checking content.) You configure the update interval in the Content Distribution Manager GUI (see the Check Manifest field in Figure 6-9). For configuration information, see the next section.
Configure Manifest File and Proxy Information for the Channel
The manifest file provides information about the content to be pre-positioned through the channel or information about the live and video-on-demand (VOD) content served through the channel.
To configure manifest file and proxy information for the channel, follow these steps:
Step 1
Step 2
Step 3
Step 4
Note
Step 5
Step 6
See Figure 6-9. Table 6-3 describes the manifest file fields in this window. Required fields are indicated by an asterisk in the GUI and in the table.
(See also the "Configuring a Proxy Server and Port for the Manifest File" section and the "Configuring Authentication for a Proxy to Fetch the Manifest File" section.)
Figure 6-9 Manifest File and Proxy Information for a Channel
Step 7
Generate the Publishing URL
A publishing URL is the URL that plays back pre-positioned content in the ACNS network. A complete publishing URL consists of three parts:
•
•
•
The path includes both the file directory path and the file name. The playserver list determines the publishing URL for the ACNS network. The playserver list is generated directly through the manifest file, through the <playServerTable> tag in the manifest file, or through the default playserver table.
Scheme
The scheme of the publishing URL is the protocol used to play the content type. For example, if an .asf video file can be played by both an HTTP and a WMT playserver, two URL schemes can be used to access this content: HTTP and MMS.
The scheme is determined by the type of playserver. The direct mapping between playserver and scheme is:
Domain Name
The domain name of the publishing URL is determined by the configuration of the ACNS network. If WCCP is used to redirect content requests to a Content Engine, the domain name in the request URL is the origin server FQDN (fully qualified domain name) in the website or channel. If content routing is used, the content routing FQDN (the FQDN of the website) becomes the domain name.
All content acquired through the manifest file is published under the domain name that is entered in the Content Distribution Manager GUI (Content > Channels > Edit channel > Edit Website) in the website Origin Server field (in the case of WCCP routing) or in the Request Routed FQDN field (in the case of a Content Router).
The content must be accessible from the website origin server FQDN, even if it is acquired from a different server.
Note
Manifest file attributes such as requireAuth and noRedirectToOrigin refer to the website origin server. Attributes such as ttl and user and password are related to the acquisition server. Thus, if you specify the requireAuth attribute for any content item, make sure that the origin server FQDN you entered in the Content Distribution Manager GUI Create New Web Site window or Modifying Web Site window (accessed through Content > Channels > Edit channel > Edit Website) is accurate and can do the following:
1.
2.
This reminder is applicable even when you have multiple acquisition servers specified in the manifest file. Because the "real" origin server is still the website origin server FQDN, you need to make sure that content is accessible from the website origin server FQDN and that the website origin server can accept authentication requests.
Path
In most cases, the path of the publishing URL is the relative source URL, or the src attribute in the <item> tags. For content crawling, it is a relative URL, relative to the host name of the origin server.
Certain attributes in the manifest file allow you to alter the publishing URL path. These attributes are cdn-url in the <item> tag, and srcPrefix or cdnPrefix in the <crawler> and <item-group> tags. These attributes convert a relative source URL into a completely new relative ACNS network URL.
For the content in the following example, the path uses default.html instead of index.html.
<item src="index.html" cdn-url="default.html" />The relative URL is always relative to the host name. In the following example, the relative URL is index.html, not sport/index.html.
<server><host name="http://www.cnn.com/sport/" /></server><item src="index.html" />In the following example, the srcPrefix and cdnPrefix attributes convert the prefix of every crawled content object from NBA/ to ABC/. The relative cdn-url is ABC/*. The path for the start-url attribute is ABC/index.html.
<crawlerstart-url="NBA/index.html"srcPrefix="NBA/"cdnPrefix="ABC/"/>Special Characters in URLs
Certain special characters are not allowed in URLs, according to the RFC 2396 standard. If the URL of the content to be acquired contains illegal characters, you must rewrite the URL using the ASCII code equivalent escape characters to represent the illegal special characters.
For example, the escape characters for a blank space are %20. An illegal URL that uses blank spaces in the filename, such as:
http://www.cnn.com/file with space
Can be rewritten as:
http://www.cnn.com/file%20with%20space
You might have to rewrite the URL using the ASCII equivalent escape characters when you specify a URL in the Manifest URL field in the Channel definition window of the Content Distribution Manager GUI (Content > Channels > Definition) or when you specify a URL in the manifest file within the following tags:
<host name=>
<item src=>
<crawler start-url=>
<crawler externalPrefixes=>
<match prefix=>
ACNS software enforces the following rules for escaping a URL:
•
In addition to the above special characters, you must also avoid using the following characters in filenames and folder names and substitute the ASCII equivalent shown:
space ==> %20
" ==> %22
% ==> %25
< ==> %3c
> ==> %3e
[ => %5b
\ ==> %5c
] ==> %5d
^ ==> %5e
{ ==> %7b
| ==> %7c
} ==> %7d
~ ==> %7e
•
•
Assign a Playserver
The playserver is assigned in the manifest file. See the "Generating a Playserver List" section on page A-13 for more details.
The <playServer> tag is very important for playing back pre-positioned content; it contains a list of playservers to play back the content. All ACNS pre-positioned content needs to use one of the ACNS software-supported playservers to play back content to end users.
ACNS 5.x software supports playservers that play back the following pre-positioned content types on the ACNS network: HTTP, HTTPS, WMT, and RTSP (RealMedia and QuickTime Streaming Server [QTSS]).
You can use any protocol to request content. Actually, the protocol information implies which playserver is needed to play the content. The ACNS software checks whether the requested protocol matches the list in the playserver table. If it matches, the request is delivered. If it does not match, the request is rejected.
You can generate a playserver list through:
•
•
To create the playserver list directly through the manifest file, configure playServer attributes of the playserver list in an <item> tag. If an <item> tag does not have a playServer attribute, its playserver list is generated through the <playServerTable> tag. If the <playServerTable> tag is omitted in the manifest file, a built-in default <playServerTable> tag is used to generate the playserver list. Multiple servers are separated by commas, as shown in the following example:
<item src="video.mpg" playServer="real,wmt" />You can also generate the playserver list that supports these streaming media types through the <playServerTable> tag. The <playServerTable> tag maps content into a playserver list based on the MIME-type extension name. If there is a <playServerTable> tag in the manifest file, use it to generate the playserver list.
To generate the playserver list though the <playServerTable> tag, use MIME-type extension names to configure which playserver can play the particular pre-positioned content, as shown in the following example:
<playServerTable><playServer name="real"><contentType name="application/x-pn-realaudio" /><contentType name="application/vnd.rn-rmadriver" /><extension name="rm" /><extension name="ra" /><extension name="rp" /><extension name="rt" /><extension name="smi" /></playServer><playServer name="wmt"><extension name="wmv" /><extension name="wma" /><extension name="wmx" /><extension name="asx" /><extension name="asf" /><extension name="avi" /></playServer><playServer name="http"><contentType name="application/pdf" /><contentType name="application/postscript" /><extension name="pdf" /><extension name="ps" /></playServer></playServerTable>The <playServerTable> tag is used to generate a playserver list for each content type. Note that in the preceding example, any file with a PDF or a PostScript extension uses HTTP to play the content.
HTTP and HTTPS are the default playservers. In other words, if you did not use the customized <playServerTable> tag or the playServer name field to specify playservers for content, HTTP and HTTPS are added to the playServer name fields to make sure the content can always be played back through HTTP and HTTPS.
If you do use a <playServerTable> tag or playServer attribute in the manifest file, HTTP and HTTPS are not automatically allowed for playback. In this case, if you want to use HTTP or HTTPS to play certain content, you must specify the protocol. For example:
<CdnManifest><playServerTable><playServer name="wmt"><extension name="asf" /></playServer><playServer name="tvout"><extension name="mpg" /></playServer><playServer name="https"><contentType name="text/html" /></playServer></playServerTable><item src="http://www.aaa.com/a.asf" /><item src="http://www.aaa.com/b.html" /><item src="http://www.aaa.com/c.mpg" /><item src="http://www.aaa.com/d.jpg" playServer="http,https" /><item src="http://www.aaa.com/e.rm" /></CdnManifest>In this example, the playserver list is generated as follows:
a.asf: wmt: from <playServerTable> by extensionb.html: https: from <playServerTable> by contentTypec.mpg: tvout: from <playServerTable> by extension.d.jpg : http + https: from "playServer" attributee.rm : real + http + https: from build-in <playServerTable>Because there is no playserver specified in the manifest file for the .rm extension, the built-in rule matches it to the RealMedia playserver. Also HTTP and HTTPS playservers are automatically added if the built-in default <playServerTable> tag is used.
The TV-out playserver is a special playserver because it allows all pre-positioned content to be played back. If you want to exclude content from being played back by other playservers, specify the TV-out playserver. ACNS software does not allow other playservers to play content once a playserver has been specified.
Proxy Support
Acquisition through a proxy server can be configured when the root Content Engine cannot directly access the origin server because the origin server is set up to allow access only by a specified proxy server. When a proxy server is configured for root Content Engine content acquisition, the acquirer contacts the proxy server instead of the origin server, and all requests to that origin server go through the proxy server.
Note
If a transparent WCCP proxy is used, you do not need to configure a proxy for content acquisition. In such cases, HTTP requests from the acquirer are redirected to the WCCP proxy by a router. However, when you do not have a router to redirect HTTP requests to the proxy server, you must configure the Content Engine to use the proxy server.
There are three ways to configure the proxy server: through the Content Distribution Manager GUI, through the Content Engine CLI, or through the manifest file. If you need to configure the Content Engine to use the proxy for both caching and pre-positioned content, use the CLI to configure the proxy. The CLI command is a global configuration command that configures the entire Content Engine to use the proxy. If only the acquirer portion of the Content Engine needs to use the proxy for acquiring pre-positioned content, use the manifest file or to specify the outgoing proxy. When you configure the proxy server in the manifest file, you are configuring the acquirer to use the proxy to fetch content for a particular channel.
Note
You can also configure a proxy for fetching the manifest file by using the Content Distribution Manager GUI (the Creating New Channel or Modifying Channel window). When you configure a proxy server in the Content Distribution Manager GUI, the proxy configuration is valid only for acquiring the manifest file itself and not for acquiring the channel content. Requests for the manifest file go through the proxy server, whereas requests for content go directly to the origin server.
Tip
Configuring Host and Proxy Server Settings Using the Content Distribution Manager GUI
To configure proxy server settings for content items defined using the Content Distribution Manager GUI, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
To return to the Content Acquisition Method for Channel-Use GUI to Specify Content Acquisition window, click the Return to Content Listing button.
Step 7
You can configure the proxy server for the acquirer by using the fields in this section. These fields are associated with the attributes specified in the <proxyServer> tag in the manifest file. The proxy server configured in the manifest file is valid for all the hosts named in the manifest file. If the specified proxy fails, the acquirer, by default, contacts the origin server directly and tries to fetch the object.
Step 8
Step 9
Step 10
Step 11
•
•
This option is associated with the noProxy attribute specified in the <host>, <item>, or <crawler> tags. The noProxy= "1" designation specifies that the item and the crawl task data are to be fetched directly from the origin server. (See the "About the noProxy Attribute" section for more information.)
•
To assign a common proxy server for all selected host URLs, choose the proxy host name (the selected proxy host is highlighted) and click the Save Assignment button. Otherwise, choose Do not change assignments to retain the previously defined proxy host assignments for the host URLs and click the Save Assignment button.
•
Configuring an HTTP Proxy Server Using the CLI
To configure a proxy server using the CLI, use the http proxy outgoing global configuration command. This is a global proxy configuration for caching and pre-positioned content acquisition. You can use this command to configure a list of outgoing proxy servers through the CLI, and when configured, all the requests (for any host) will go through the proxy server. The global proxy configuration can also be reset using the no form of the command. For example:
ContentEngine(config)# http proxy outgoing ?connection-timeout Timeout period used for probing outgoing proxy servers in microsecondshost Use Outgoing HTTP Proxymonitor Interval at which to monitor the outgoing proxy serversorigin-server Use Origin Server if all outgoing proxies failed.preserve-407 Preserve 407 HTTP authentication headerContentEngine(config)# http proxy outgoing host ?Hostname or A.B.C.D Hostname or IP address of outgoing proxyContentEngine(config)# http proxy outgoing host 128.107.192.24 ?<1-65535> Port of outgoing proxyContentEngine(config)# http proxy outgoing host 128.107.192.24 80If the acquirer is unable to contact a proxy server, the proxy is considered disabled. The acquirer retries a disabled proxy after the period specified in the monitor option of the http proxy outgoing command. If no monitor interval is configured, the retry interval defaults to every 30 minutes.
If all outgoing proxies fail and you want the acquirer to contact the origin server directly, use the origin-server key word. For example:
ContentEngine(config)# http proxy outgoing origin-serverIf a list of outgoing proxy servers is configured in the CLI and the first proxy server fails, the next proxy server in the list will be tried, and so on. If all the proxies fail, the acquirer will fail over to the origin server, depending on the configuration specified in the http proxy outgoing origin-server command.
To verify that the outgoing proxy is configured in the CLI, use the show http proxy EXEC command. For example:
CE-507# show http proxyIncoming Proxy-Mode:Not servicing incoming proxy mode connections.Outgoing Proxy-Mode:Primary Proxy Server: 128.107.192.190 port 9090Monitor Interval for Outgoing Proxy Servers is 60 secondsTimeout period for probing Outgoing Proxy Servers is 300000 microsecondsUse of Origin Server upon Proxy Failures is disabled.To confirm whether the content is being acquired through a proxy, use the acquisition and distribution transaction log, and check the proxyUsed line. This line shows the proxy server used for the request.
Configuring a Proxy Server Using the Manifest File
You can configure the proxy server for the acquirer by using the <proxyServer> tag in the manifest file. The proxy server configured in the manifest file is valid for all the hosts named in the manifest file. If the specified proxy fails, the acquirer, by default, contacts the origin server directly and tries to fetch the object.
If you have multiple proxy servers in the manifest file, and if you want to associate different proxy servers for different hosts, you must include the proxyServer attribute in the <host> tag. In this first example, the <proxyServer> tag is associated with the <host> tag through its proxyServer attribute (proxyServer="172.19.226.242"). All items that use this <host> tag will use this proxy server.
Note
<CdnManifest><proxyServerserverName="172.19.226.242"port="8080"/><server name="my-devbox"><host name="http://vista2.cisco.com"proxyServer="172.19.226.242"proto="http" /></server><item src="HR/salary.htlm" /><item src="HR/Holidays.html" /></CdnManifest>In the following example, the proxyServer attribute is associated with the <item> tag and the <crawler> tag. The proxy server is applied to the second <item> tag and the <crawler> tag, but not the first <item> tag.
<CdnManifest><item src="http://www.msnbc.com/computer/jokes.html" /><proxyServer serverName="128.107.192.24" port = "80" /><item src="http://www.cnn.com/War/World-war-two.jpg" /><crawler start-url="http://www.abc.com/War/World-war-two/index.html" /></CdnManifest>Instead of using an IP address for the serverName attribute in the <proxyServer> tag, you can also use a domain name. For example:
<proxyServerserverName="spachiap-uni1"port="8080"/><server name="my-devbox"><host name="http://128.107.150.26/nfs-obsidian/Unicorn"proxyServer="spachiap-uni1"proto="http" /></server>When you use a domain name instead of an IP address, make sure that the domain name can be resolved by the DNS server configured.
About the noProxy Attribute
You can configure the noProxy attribute in the manifest file to specify that all the requests for a particular host should go directly to the origin server and should not use the proxy specified in the http proxy outgoing host command, if any proxies were configured in the CLI.
The noProxy attribute can be specified in the <host>, <item>, or <crawler> tags. In the following example, if the root Content Engine has a proxy configured through the http proxy outgoing host command, the first item in the manifest file uses that proxy, but the rest of the items or crawl tasks in this manifest file do not. The noProxy="1" designation specifies that the second item and the crawl task data are to be fetched directly from the origin server.
<CdnManifest><item src="http://www.msnbc.com/computer/jokes.html" /><item src=" http://www.cnn.com/War/World-war-two.jpg"noProxy="1" /><crawler start-url="http://www.abc.com/War/World-war-two/index.html"noProxy="1" /><server name="cisco" ><host name="http://www.cisco.com" noProxy="1" /></server><crawler start-url="product/routers/" /></CdnManifest>Configuring a Proxy Server and Port for the Manifest File
A proxy server and port for fetching the manifest file can be set in the Channel Content window of the Content Distribution Manager GUI. If a proxy server is configured, requests for the manifest file go through the proxy server that is specified in the Content Distribution Manager GUI.
Note
If the manifest file resides on an origin server that requires a proxy, you need to specify the proxy name or IP address and the proxy port in the fields provided in the Define Manifest Proxy Information section of the Channel Content window in the Content Distribution Manager GUI. (See Figure 6-9. For a description of the fields in this window, see Table 6-3.)
If the manifest file resides on an origin server that cannot use the proxy configured from the http proxy outgoing host CLI command, you must check the Disable All Proxy check box.
The Disable All Proxy check box, when checked, overrides the values in the Proxy Hostname and Proxy Port fields.
Note
Authentication Support
The acquirer supports two types of authentication schemes: basic and NTLM authentication. Authentication information is configured either in the Content Distribution Manager GUI or in the manifest file, depending on what is being acquired.
If authentication is required to fetch the manifest file from the origin server or from a proxy server, you can specify the authentication information in the Content Distribution Manager GUI. From the Content > Channels Channel Content window, you can disable basic authentication, enter a username and password, and specify an NTLM user domain name. (Figure 6-9 shows the fields for entering authentication information, and Table 6-3 describes these fields.)
Note
If authentication is required to fetch pre-positioned content from the origin server or from a proxy server, you must specify the authentication information in the manifest file. (See the next section, "Acquiring Content that Requires NTLM Authentication.")
Acquiring Content that Requires NTLM Authentication
In ACNS 5.x software, a root Content Engine can acquire content from HTTP origin servers or proxy servers after performing basic authentication. However, origin servers often support NTLM authentication only. In ACNS 5.2 software, the acquirer can act as an NTLM client to talk to NTLM-capable origin servers.
Note
If authentication is required to fetch content, you must specify the authentication information in the manifest file by using following manifest file attributes:
•
•
If NTLM authentication is required, you can specify the following two additional manifest file attributes:
•
•
These attributes can be specified in the <host>, <item>, <item-group>, and <crawler> tags. See Appendix A, "Creating Manifest Files," for more information.
If an origin server or proxy server supports both basic authentication and NTLM authentication, the acquirer chooses the first supported authentication scheme from the response challenge list. The authentication scheme used by the acquirer can also depend on the specified authentication information. For example, an IIS server responds with NTLM as the first scheme and basic authentication as the second. In this case the acquirer chooses NTLM. However, if the ntlmUserDomain attribute is not specified in the manifest file, the acquirer chooses basic authentication. If the disableBasicAuth attribute is set to true, the acquirer chooses NTLM authentication.
You can view the acquisition and distribution transaction log from the root Content Engine CLI to determine which authentication scheme the acquirer has used.
Proxy Authentication Support
If the proxy for the root Content Engine was configured in the root Content Engine CLI (http proxy outgoing host command), you can set the authentication information for the proxy from either the Content Distribution Manager GUI (see the "Configuring Authentication for an HTTP Proxy Using the Content Distribution Manager GUI" section) or from the root Content Engine CLI (see the next section, "Configuring Authentication for a Proxy Using the CLI").
Configuring Authentication for a Proxy to Fetch the Manifest File
To configure the authentication information for fetching the manifest file from a proxy server, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Configuring Authentication for a Proxy Using the Manifest File
The following example specifies authentication information for the proxy server. See Appendix A, "Creating Manifest Files," for more information.
<CdnManifest><!-- specify a proxy server/port and its authentication infothis proxy supports Basic auth so only user/password is needed --><proxyServer serverName="128.107.192.24" port = "80"user="johnz" password="xxx123yyy"/><!-- this item is below the proxyServer tag; it is using the above proxy --><item src="http://www.cnn.com/War/World-war-two.jpg" /><!-- specify a proxy server/port and its authentication infothis proxy requires NTLM auth, so ntlmDomainName is neededfor extra password security, users want to disable basic authso their user/password is not sent over the wire --><proxyServer serverName="company-proxy" port = "80"user="johnz" password="xxx123yyy"ntlmUserDomain="cisco-eng"disableBasicAuth="1" /><!-- this crawler is below the proxyServer tag; it is using the proxy --><crawler start-url="http://www.abc.com/War/World-war-two/index.html"/></CdnManifest>Configuring Authentication for an HTTP Proxy Using the Content Distribution Manager GUI
If a root Content Engine is configured to receive content through a proxy server, the acquirer running on the root Content Engine must be authenticated by the proxy server before it can obtain content from the origin server.
To configure authentication information for a proxy that was specified using the http proxy outgoing host command, you can use the Acquirer Outgoing Proxy Authentication section in the Content Distribution Manager GUI (Devices > Devices > Streaming/Caching > HTTP > HTTP Connections) to set the authentication information for the proxy. ACNS software supports multiple proxies; therefore, you can set the proxy authentication information for multiple proxies as well.
Note
To configure the acquirer outgoing proxy authentication information, follow these steps:
Step 1
Step 2
Step 3
Step 4
a.
b.
c.
d.
If you leave this check box unchecked, NTLM authentication headers can be stripped to allow fallback to the basic authentication method against Microsoft Internet Information Services (IIS) servers; the username and password information can be passed to the origin server in clear text with a basic authentication header.
Step 5
Configuring Authentication for a Proxy Using the CLI
To configure the authentication information for a proxy that was specified using the http proxy outgoing host command, you can also use the acquirer proxy authentication outgoing command from the root Content Engine CLI. The following example shows the authentication configuration for a nontransparent proxy server (IP address 192.168.1.1, port 8080) with NTLM authentication:
CE(config)# acquirer proxy authentication outgoing 192.168.1.1 8080 myname password password ntlm mydomain basic-auth-disable
Note
To verify that the authentication information has been set up correctly, use the show acquirer proxy authentication EXEC command.
The acquirer supports proxy chaining as long as there is only one proxy in the chain that requires authentication. The acquirer will fail if more than one proxy in the chain requires authentication.
Configuring Authentication for a WCCP Proxy Using the Content Distribution Manager GUI
In a transparent caching environment using a WCCP proxy, requests for content are redirected to a Content Engine by a WCCP-enabled router. When an HTTP proxy-style request from the root Content Engine to the origin server is intercepted by a WCCP proxy that requires authentication, you can configure authentication settings for the WCCP proxy.
To configure acquirer WCCP proxy authentication settings in a transparent proxy environment, follow these steps:
Step 1
Step 2
Step 3
Figure 6-10 Acquirer WCCP Proxy Authentication
Step 4
Step 5
Step 6
Step 7
Step 8
If you leave this check box unchecked, NTLM authentication headers can be stripped to allow fallback to the basic authentication method against Microsoft Internet Information Services (IIS) servers; the username and password information can be passed to the origin server in clear text with a basic authentication header.
Step 9
A "Click Submit to Save" message appears in red next to the Current Settings line when there are pending changes to be saved after you have applied default or device group settings. You can also revert to the previously configured window settings by clicking Reset. The Reset button is visible only when you apply default or device group settings to change the current device settings but the settings have not yet been submitted.
Modifying WCCP Proxy Authentication Settings
You can modify the following WCCP proxy authentication settings in any order:
•
•
•
•
•
Configuring Authentication for a WCCP Proxy Using the CLI
For a transparent WCCP proxy, if authentication is required, you can specify the authentication information by using the acquirer proxy authentication transparent global configuration command in the root Content Engine CLI. The following example shows the authentication configuration for a transparent WCCP proxy server with basic authentication. The username is admin, and the password is default.
CE(config)# acquirer proxy authentication transparentCE(config)# acquirer proxy authentication transparent adminCE(config)# acquirer proxy authentication transparent admin passwordCE(config)# acquirer proxy authentication transparent admin password defaultVerifying the Results
Use the following show CLI commands to verify acquisition results.
For a Content Engine that is the root Content Engine for a channel, use the following command to monitor acquisition progress and to troubleshoot.
Use the show acquirer EXEC command to make sure that the acquirer process on the root Content Engine is working correctly, and that the device is using the expected amount of bandwidth for acquisition. The following example shows that the acquirer is running properly and that the device is configured with unlimited bandwidth for acquisition of content.
Content Engine# show acquirerAcquirer is running OKCurrent Acquisition Bandwidth:Not LimitedUse the show acquirer progress EXEC command to check how far the acquisition of content has progressed. A specific channel ID or channel name can be specified to obtain the progress for a specific channel. In the example below, the acquirer has already acquired 2237 items.
ContentEngine# show acquirer progress channel-id 639Querying Database.......Acquirer progress information for channel ID:639 Channel-Name:external-----------------------------------------------------------------Acquired Single Items : 0 / 0Acquired Crawl Items : 2237 / 2500 -- start-url=www.mtv.com//Use the show statistics acquirer channel-id or show statistics acquirer channel-name EXEC command to obtain the detailed acquisition statistics for a given channel. In the example below, there was an error acquiring two items.
ContentEngine# show statistics acquirer channel-id 639Querying Database.......Statistics for Channel Channel-id :639 Channel-Name :external---------------------------------------------------------Manifest:---------Fetch Errors :0Parsing Errors :0Parsing Warnings:0Acquisition:-------------Total Number of Acquired Objects :2237Total Disk Used for Acquired Objects :981511280 BytesTotal Number of Failed Objects :2Total Number of Re-Check Failed Objects :0Use the show statistics acquirer errors channel-id or show statistics acquirer errors channel-name EXEC command to see the reasons why the errors occurred. In the example below, one error occurred because there was a problem acquiring the URL. The other error occurred because the disk quota for the channel configured in the Content Distribution Manager GUI would have been exceeded if the specified URL had been acquired. You can increase the channel disk quota to correct this error.
Content Engine# show statistics acquirer errors channel-id 639Querying Database.......Acquisition Errors for the Channel ID:639-------------------------------------Crawl job:start-url http://www.mtv.com//Crawl Errors---------------Internal Server Error(500):http://cgi.cnn.com/entries/intl-emailsubs-confirmExceeded Disk Quota(703):http://www.cdt.org/copyright/backgroundchart.pdfUse the acquirer test-url global configuration command to use WGET or an MMS test program for a Microsoft Media Server URL to see whether the problem URLs can be acquired by the Content Engine. This command can be used to detect network or server-side problems in individual URLs so that they can be corrected.
Content Engine# acquirer test-url http://cgi.cnn.com/entries/intl-emailsubs-confirm--19:13:14-- http://cgi.cnn.com/entries/intl-emailsubs-confirm=> `/dev/null'Len - 50 , Restval - 0 , contlen - 0 , Res - 134727928Resolving cgi.cnn.com...done.Connecting to cgi.cnn.com[207.25.71.15]:80... connected.HTTP request sent, awaiting response... 500 An error occurred processing your request.(365 to go)ERROR 500:An error occurred processing your request..Use the show acquirer progress streams EXEC command to view the progress on the acquisition of streaming media. This command provides useful information for a channel that is acquiring streaming content. The example below shows that acquisition of the first URL is 43 percent complete and acquisition of the second one is 98 percent complete.
Content Engine# show acquirer progress streamsQuerying Database.......MMS URL = mms://172.19.224.235/MBR_Consumer_Med.wmvMMS Bandwidth ID = 0Stream Start Time = Mon Sep 22 19:29:44 2003Duration Requested = 258 secDuration Passed = 112 secPercentage Complete = 43%Requested Bandwidth = 1067 kbpsMinimum Bandwidth Required = 361 kbpsReserved Bandwidth = 1067 kbpsSize To Be Downloaded = 34418305 bytesSize Downloaded = 14881905 bytesMMS URL = http://172.19.224.235:8080/BillGSpeech_32k.wmaMMS Bandwidth ID = 1Stream Start Time = Mon Sep 22 19:29:44 2003Duration Requested = 111 secDuration Passed = 112 secPercentage Complete = 98%Requested Bandwidth = 32 kbpsMinimum Bandwidth Required = 24 kbpsReserved Bandwidth = 32 kbpsSize To Be Downloaded = 456245 bytesSize Downloaded = 450720 bytesIf more detailed troubleshooting of content acquisition is required, you can increase the debug level of the acquirer using the debug acquirer trace EXEC command. The logs are written to local1/errorlog/acquirer-errorlog.current. The logs for acquisition through the MMS protocol are written to local1/errorlog/acquirer-mms-errorlog.current.
Retry and Refresh Mechanisms
ACNS 5.x software supports various retry and refresh mechanisms in the manifest file. Following are some things to note.
Using the <schedule> and <repeat> Subelements
ACNS 5.2 software supports two new subelements in the manifest file: <schedule> and <repeat>. These tags allow you to specify a time to begin a recrawl or refetch and are more powerful than the ttl schema, which only allows you to specify the interval between refetch or recrawl occurrences. The <schedule> element can be a subtag of <options>, <item-group>, <item> or <crawler>. If both the <schedule> tag and the ttl attribute are specified in the manifest file, the <schedule> designation takes precedence over ttl. (See the "item" section on page A-44 for a detailed description of these subelements.)
Using the faiRetryInterval Attribute
When the acquirer tries to acquire a content item specified in the <item> or <crawler> tags, and the acquisition fails (for example, because of some intermittent error or for some other reason), the acquisition task (corresponding to the <item> or <crawler> tag) is retried after the interval specified in the failRetryInterval attribute.
If the failRetryInterval attribute is not specified, the default interval for retrying the task is 5 minutes. (For a crawl job, the task is considered to be a failure only when crawling fails to occur at all. If only some of the crawled pages fail to be acquired, the task is still considered a success.) The following rules apply to the retry mechanism:
•
•
•
Using the ttl Attribute or the Fetch Manifest Now Button
When an item is acquired successfully, and if a positive value is specified in the ttl attribute, the acquirer rechecks the content for freshness at the interval specified by the ttl attribute. A ttl value of 0 (zero) means that the acquirer will not recheck the item unless the manifest file is updated. A negative ttl value means that the acquirer will never recheck the item. The following rules apply to the refresh mechanism:
•
•
•
Reparsing the Local Manifest File From Content Distribution Manager GUI
When you add content to a channel using the Content Distribution Manager GUI to define the content, a manifest file is automatically generated and stored locally in the Content Distribution Manager. When you make any changes such as adding, removing, or modifying content definitions in the GUI and click the Submit or Update button, the local manifest file is automatically reparsed, changes are detected, and the corresponding items are acquired or removed. Note that when you click the Submit or Update button, it does not trigger a recheck of all the content in the channel.
Updating Channel Content
At any point after you have replicated content to the Content Engines that are associated with your channel, you can update that content using the fetch manifest feature. For example, if you modify your manifest file to point to new content or remove references to content that you want to make obsolete, you must fetch the manifest file to begin replication of any new channel content, and to sever connections to content that you want to make obsolete.
Note
To fetch a new or updated manifest file, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
When you click this button, the software checks to see if the manifest file has been updated, and the updated manifest file is downloaded and reparsed. Also, regardless whether the manifest file has been updated, all content in the channel is rechecked and the updated content is downloaded.
Step 6
To force the replication of channel content and refresh the information, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
For a description of the channel replication status data, see the "Monitoring Content Replication Status" section on page 18-11.
Bandwidth Control
The bandwidth control feature allows you to specify how much bandwidth in the network is consumed by data replication from the root Content Engine to the edge Content Engines. Bandwidth controls allow you to specify the amount of bandwidth to be used at various times during the day and also to set up a weekly schedule that repeats week after week. This section describes the bandwidth controls as they relate to content acquisition and distribution. Bandwidth controls are available for the acquisition process, replication process, and multicast sender.
Configuring Acquisition and Distribution Default Bandwidth Settings
Default bandwidth settings can be configured for acquisition and distribution of content. Default bandwidth is the amount of bandwidth allocated for content acquisition and distribution when there is no scheduled bandwidth.
If a Content Engine is assigned to a device group and no default bandwidth has been set for the device, the device group default bandwidth settings are applied. If the Content Engine is part of multiple device groups, the most recently updated default bandwidth settings are applied.
However, if default bandwidth is specified for a device, it will override the settings at the device group level. This occurs when a Content Engine is a member of a device group.
To configure the default bandwidth settings for acquisition and distribution for the Content Engine, follow these steps:
Step 1
Step 2
Step 3
Figure 6-11 Acquisition and Distribution Default Bandwidth Window
Step 4
Step 5
Step 6
Minimum nonstreaming bandwidth is the bandwidth allocated for acquiring content using nonstreaming protocols such as HTTP, HTTPS, and FTP. This setting is useful when streaming protocols, such as MMS, take up all the available bandwidth in certain cases, and bandwidth needs to be reserved for acquiring manifest files and other content items from origin servers using nonstreaming protocols. We recommend that you retain the minimum nonstreaming bandwidth default setting, and adjust it only when it might be necessary to increase the value, such as when certain large files fail to be acquired because of many stream acquisitions taking place simultaneously.
Step 7
Note
Step 8
Step 9
Displaying a Graphical Representation of the Acquisition and Distribution Bandwidth Settings
You can view a graphical representation of the bandwidth settings configured on a Content Engine for the acquisition and distribution of files. The vertical axis of the graph represents the amount of bandwidth in kbps and the horizontal axis represents the days of the week. The scale shown on the vertical axis is determined dynamically based on the bandwidth rate for a particular type of bandwidth and is incremented appropriately. The scale shown on the horizontal axis for each day is incremented for each hour. Each type of bandwidth is represented by a unique color. A legend at the bottom of the graph maps the colors to the corresponding bandwidths.
To view the graph that displays the acquisition and distribution bandwidth, follow these steps:
Step 1
Step 2
Step 3
Step 4
You can choose the view that you wish to apply to the bandwidth graph by clicking the different viewing options. See Table 6-6 for a description of the viewing options and their descriptions.
Step 5
Configuring Acquisition and Distribution Bandwidth Settings for Scheduled Times
In addition to being able to set the default bandwidth limits for incoming and outgoing traffic on the Content Engine, you can use the Content Distribution Manager GUI to configure different limits for different time segments that form a week-long cycle. For example, you can configure the acquisition-in limit at a maximum of 100 kbps from 8:00 a.m. to 8:00 p.m., Monday through Friday, and extend that limit to as high as 10 Mbps during the nights from 8:00 p.m. to 8:00 a.m., Monday through Friday and all day Saturday and Sunday. These settings override the default value for the time and period that you specify.
Note
Note
Note
To configure distribution bandwidth settings for specific days and times, follow these steps:
Step 1
Step 2
Step 3
Step 4
Step 5
Figure 6-12 Creating New Acquisition and Distribution Bandwidth Schedule Window
Step 6
Step 7
Step 8
Scheduling Bandwidth for Streaming Acquisition
When MMS acquisition is configured, the streaming content consumes a large amount of bandwidth over the playtime duration of the file. You must ensure that sufficient bandwidth is available for sufficiently large blocks of time. For multi-bit-rate streams (MBR), the required bandwidth is the sum total of all the bit rates encoded in the stream.
Error Codes
This section describes the error codes for content replication. The following error codes are shown in the Content Distribution Manager GUI Replication Status window:
ACNS Unified Name Space Errors
1401: Bad magic number in unified name space (UNS) meta file
1402: Unknown version in UNS meta file
1403: Bad checksum on UNS meta file
1404: Internal error - URL mismatch between caller specified URL and URL in meta file
1405: Invalid URL syntax
1406: Attempt to create URL already in UNS
1407: Insufficient space to store requested object
1408: Internal logic error in UNS server code
1409: Requested object not found in UNS
1410: Requested UNS operation not implemented
1411: Failure in underlying RPC transport
1412: Destination URL already exists
1413: Channel does not exist
1414: Writes failed to all metadata files
1415: Object is not servable
1416: Object is out of presentation time
1417: Object playback not allowed by this playback server
1418: Live object, but attributes invalid
1419: Alternate media attributes invalid
1420: Alternate media is not servable
1421: Channel would be over disk quota
1422: Cannot change channel ID
1423: Object already in specified channel
1424: Object not in specified channel
1425: Metadata operations disabled (no file systems)
1426: Channel operations disabled (no file systems)
1427: Nonobject metadata services not initialized
1428: Out of handles for nonobject metadata files
1429: Internal error loading metadata file
1430: (No error text available)
1431: Too many CDN files (cannot add to URL for file system map [UFM])
1432: Specified legacy ECDNFS file not found
1433: Bad MD5 checksum passed by caller
1434: Bad tags present in supplied attributes (OBSOLETE)
1435: Cannot resize content file migrated from ECDN
1436: UNS symlink references nonexistent URL
1437: URL is not a UNS symlink
1438: Too many levels of UNS symlinks
1439: UNS entry metafile truncated
1440: Output to be returned is too big in size
1441: URL request was made on a nondefault service port
1442: Specified legacy file not on any UNS filesystem
1443: Specified legacy file is not in the `cache' directory
1444: Specified legacy file is referenced by a UNS entry
1445: Specified legacy file is not a .data file
1446: File open failed during ASX/SMIL rewrite operation
1447: Parse error during ASX/SMIL rewrite operation
1448: Initialization error during ASX/SMIL rewrite operation
1449: Actual Rewriting failed during ASX/SMIL rewrite operation
1450: No more file system slots
1451: Specified file system is already in use
1452: Specified file system not known to UNS
1453: Specified file system bytes in use exceeds target
1454: Cannot unuse the file system containing the symlink tree
1455: Cannot add file system because there is no local disk-based CDNFS storage
Acquirer Error Codes
The following error codes are common for HTTP, FTP, HTTPS, and MMS content acquisition:
700: Acquirer internal error
701: Manifest parser error
702: Manifest parse warning
703: Exceeded disk quota
704: No space in UNS
705: It is a folder
706: ACCEPT failed
707: Connection refused
708: Listen failed
709: Mismatched in crawling
710: No-cache instructed from server
711: Disabled by the user
712: Downloaded size mismatched with content length
713: Invalid response received from server
714: Database access error
715: Not to acquire URL with ?
716: Invalid redirect foo to foo/
717: Illegal folder for file import
718: Unable to connect to proxy server
719: URL is too long
720: HTTP metadata is too long
721: Connection closed by peer
722: Invalid content length
905: Socket timeout
906: No host
907: Zero bandwidth
908: File download aborted
909: Content expired before fetch
910: Insufficient bandwidth to download the stream
911: Live stream not supported
912: Request timed out
1000: UNS error
MMS-Specific Acquisition Error Codes
2002: Cannot connect to remote server
2003: Remote cannot connect
2004: Requested file not found
2005: Requested remote file not found
2006: Max number of connections is reached
2007: Remote server reached max number of connections
2008: Max bandwidth limit is reached
2009: Remote server reached max bandwidth limit
2010: Max bit rate limit reached
2011: Remote server reached max bit rate limit
2012: Illegal memory address encountered
2013: Illegal memory address encountered at remote server
2015: Error creating a socket
2016: Error creating a socket at remote server
2017: Internal system error
2018: Error receiving data from server
2019: Server error at remote server
2020: Authentication failed
2021: Remote server timed out
2022: Error in stream type
2023: Remote proxy error
2024: Invalid request at remote server
2025: Stream file is corrupt
2026: Stream file is corrupt at remote server
2027: Data received is corrupt
2028: Data received is corrupt at remote server
2029: Remote access denied
2030: Remote connection refused
2031: MMS over UDP not allowed in WCCP mode
2032: MMS over UDP has been disabled
2033: MMS over TCP has been disabled
2034: MMS over TCP and UDP has been disabled
2035: MMS over HTTP has been disabled
2036: Client error at first round of handshake over MMS
2037: Client error
2038: Client request blocked
2039: Client request filtered
2041: Unknown MMS error
2042: Unknown error from remote server
2043: Max incoming bandwidth limit is reached
2044: Max incoming bit rate limit reached
2100: Generic MMS acquisition error
2101: Acquirer could not be contacted to check header
2102: Check of head result failed to match criteria
2103: Could not write stream to disk
2104: Could not open file to write
2105: Stream header could not be retrieved
2106: Remote stream was closed abnormally
2107: Header obtained from remote stream was in invalid format
2108: Remote server closed the connection
2109: Connection to remote server timed out
2110: Remote file contains no streams
2111: Could not load ASF block for indexing
2112: Specified URL could not be resolved to a remote host
2113: File size is bigger than that supported
2114: Acquirer could not be contacted to send status of download
2115: Error in forking a process
2116: Stream acquisition terminated
2117: MMS acquisition was stopped
2118: Bandwidth not available for streaming in next packet
2119: RPC exception thrown when contacting main acquirer
Where to Go Next
In this chapter you accomplished the following tasks:
•
•
•
•
•
Now you need to consider how the content is going to be played back to the end user. If the content is to be scheduled or rebroadcast, proceed to Chapter 7, "Creating and Managing Programs."
