Cisco TV CDS 2.4 Installation, Upgrade, and Maintenance Guide for the Cisco ISM (Integrated Service Module) Line Card
Overview of Upgrading and Downgrading the TV CDS Software
Downloads: This chapterpdf (PDF - 153.0KB) The complete bookPDF (PDF - 889.0KB) | Feedback

Overview of Upgrading and Downgrading the TV CDS Software

Table Of Contents

Overview of Upgrading and Downgrading the TV CDS Software

Introduction

Prerequisites for Upgrading or Downgrading the TV CDS Software

General Software Upgrade and Downgrade Information for CDS Servers

Getting the Cisco TV CDS Software Files for Release 2.4.1

Getting a Software File from Cisco.com

Upgrade Sequence for Different Deployments

Upgrading a VVI

VVI Upgrade Sequence

Upgrading a CDS

CDS Upgrade Sequence

Downgrade Sequence for Different Deployments

Upgrade and Downgrade Workflow for a CDS Server

Software Upgrade Workflow for a CDS Server

Software Downgrade Workflow for a CDS Server


Overview of Upgrading and Downgrading the TV CDS Software


This chapter provides an overview of upgrading and downgrading the CDSM, VVIM, and CDS servers. The chapter covers the following topics:

Introduction

Getting the Cisco TV CDS Software Files for Release 2.4.1

Upgrade Sequence for Different Deployments

Downgrade Sequence for Different Deployments

Upgrade and Downgrade Workflow for a CDS Server

Upgrade and Downgrade Kits for CDS Servers, page 2-10

Introduction

The Release 2.4.1 software upgrade for non- Cisco ASR 9000 Series Aggregation Services Router ISM (Integrated Service Module) line card based CDS servers (Streamer, Vault, Caching Node, and ISV) may need to include an upgrade from the 32-bit operating system (OS) to a 64-bit OS. The CDSM and VVIM and any Cisco ASR 9000 Series Aggregation Services Router ISM (Integrated Service Module) line cards are already running the 64-bit OS. When a CDS server is being upgraded or downgraded, the OS is completely reinstalled. The CDS server must be taken out of service.

Refer to the Cisco TV CDS 2.3 Installation, Upgrade, and Maintenance Guide on-line for instructions on upgrading non- Cisco ASR 9000 Series Aggregation Services Router ISM (Integrated Service Module) line card based CDS servers .


Note Release 2.4.1supports upgrades from Release 2.0.x and Release 2.1.x. If your CDS is running an older version, you need to upgrade to Release 2.0.x or 2.1.x, before upgrading to Release 2.4.1.



Note The following TV CDS software releases can coexist during an upgrade process, allowing for long upgrade windows if necessary:

For VVI in an ISA environment with SCS, the Release 2.1.3 software can coexist with the Release 2.4.1 software.

For VVI in an RTSP environment with HTTP Streamers, the Release 2.1.3 ES3 software can coexist with the Release 2.4.1 software.

For CDS (non-VVI) in both an ISA and RTSP environment, the Release 2.0.x and 2.1.x software can coexist with the Release 2.4.1 software.

While TV CDS software releases can coexist during an upgrade process, the CDSM and ISM MUST be installed with the same version of software.



Note Software upgrades and downgrades should be performed during maintenance windows; that is, during off-peak hours when no new content is ingested into the CDS and stream demands are the lowest.



Caution Basic understanding of the Linux command line and the vi editor are required for the completion of the software upgrade. Do not attempt the software upgrade if you are unfamiliar with the Cisco CDS products and the Linux command line and vi editor.

We recommend that an experienced Linux system administrator perform the software upgrade and downgrade. The TV CDS Release 2.4.1software upgrade and downgrade require disk space and network connectivity verification, knowledge of the VNC application (if applicable), and general administration of the remote backup server for archive storage.

Prerequisites for Upgrading or Downgrading the TV CDS Software

Upgrading or downgrading a CDS server has the following prerequisites:

At least 1.8 GB of unused disk space on the / partition for storing and running the upgrade kit and downgrade kit.

At least 20 percent unused disk space on the file system for the /boot directory.

If the /boot_cds directory exists, at least 4.8 GB partition size for the file system and at least 1 GB unused disk space.

VNC listener setup defined and operational (We highly recommend using a VNC listener. See the "VNC Listener" section for more information.)

Server is operational, which means connected to the network, boot up has completed, all file systems are mounted, and all content drives are operational.

Any failed content drives should be removed from the server. Upgrading a server with failed content drives may result in a stalled upgrade process, which dramatically increases the amount of time required to perform the upgrade.

Serial console connected TTYS0 (Not mandatory, but we highly recommend a serial console for monitoring the process).

Direct physical access in the event of a major failure (for example, power outage) during upgrade.

Access to a remote server used for backing up each CDS server. The remote Linux server should have enough space to store multiple backups. (The log files that are backed up are not restored.)

All non-essential files should be archived to a remote location and then removed if the disk space usage is high on all partitions. Any file system at 90 percent capacity should be cleaned up.

Before upgrading the CDSM or VVIM, all settings on the CDSM Setup page should be recorded. After the VVIM or CDSM is upgraded, all setting on the CDSM Setup page should be verified with the settings that were recorded and resubmitted.


Note After the upgrade procedure starts, do not make any configuration changes until all the servers have been upgraded. The only exceptions to this are submitting the CDSM Setup page after a CDSM or VVIM upgrade, and submitting the Route Table page after upgrading a CDS server from Release 2.0.



Note The /arroyo/log/archive directory is not preserved. If you want to save the archive, copy it to another server before upgrading the software.



Note During the initialization process of a CDS server or after recovering a CDS server that has been down for less than an hour, the CDS database performs a complete synchronization. The database synchronization takes about five minutes before the server becomes ready for service. If the CDS server is down for a much longer time than an hour, the database synchronization takes longer than five minutes. The netstat command will not show the interfaces as up until the synchronization has completed.


General Software Upgrade and Downgrade Information for CDS Servers

The following list provides information about the TV CDS Release 2.4.1software upgrade:

Upgrade kit and the TV CDS ISO image file must be copied to the CDS server. The TV CDS ISO image file is copied after the upgrade is performed.

Upgrade and downgrade procedures are not for imaging a server to the same state as a brand new system.

Upgrade can take approximately one hour. The minimum time to perform an upgrade has been 25 minutes, however, servers in different network deployments may require additional time. This time is also based on no failures during the upgrade (that is, power loss or other major failures).

All configuration information concerning the network should be recorded before upgrading the software.

Upgrade can be monitored (recommended) from a VNC listener. During the stage 2 boot process of the upgrade, a window is provided to the user to see the operations taking place on the server. Using the VNC listener provides the additional ability to triage issues through the serial console if there is a failure during the upgrade.

The previous "preupgrade" and "upgrade" scripts provided with the TV CDS software should not be used in conjunction with the upgrade and downgrade kits.

Log files are backed up during the upgrade and downgrade procedures. However, if the log file backup process fails, it is not considered a fatal error and the procedure continues.

File systems preserved across the OS upgrade are those associated with the following directories: /boot, /arroyo/db, and /boot_cds (if it exists). The upgrade procedure creates the /boot_cds directory if it does not exist.

Backups are created in /arroyo/db directory. After the upgrade, the administrator performing the upgrade is responsible for removing these files when they are no longer needed.

CDS servers received from Cisco that will be installed into a Release 2.4.x deployment need to be upgraded with this upgrade kit. Review the file /arroyo/image/tags to see if the installed version on the server is Release 2.4 or higher. See the "Upgrading a New CDS Server to Release 2.4.1" section for more information.

To perform a downgrade, the ISO image file for the TV CDS release and the backup files created during the upgrade (backup.tgz, and backup_db.tgz) are required. To downgrade the CDS server, these files must be copied to an accessible server.

Getting the Cisco TV CDS Software Files for Release 2.4.1

Table 2-1 lists the different files for upgrading and downgrading the CDSM and the CDS servers (Streamers, Caching Nodes, Vaults, and ISVs) to Release 2.4.

Table 2-1 Files for Release 2.4 Software Upgrade

Server
Operating System Upgrade Package
TV CDS Software Upgrade

CDSM

(64-bit OS already installed)

cdsinstall
CDS-TV-2.4.1.iso

Vault, Caching Node, Streamer, or ISV

cdstv-2.4.1-upgrade-kit.sh
cdstv-2.4.1-downgrade-kit.sh
CDS-TV-2.4.1-DVD-OS.iso

CDS-TV-2.4.1.iso


The CDS-TV-2.4.1.iso file is the ISO image file of the Release 2.4.1 software. This file is used for the CDS servers and the CDSM and VVIM.

The cdsinstall script must be downloaded and copied to the CDSMs and VVIMs for upgrading to Release 2.4.1.

The cdstv-2.4.1-upgrade-kit.sh and cdstv-2.4.1-downgrade-kit.sh files are the self-extracting upgrade and downgrade kits only for the CDS servers. For more information, see the "Upgrade and Downgrade Kits for CDS Servers" section on page 2-10.

The CDS-TV-2.4.1-DVD-OS.iso file is an ISO image file that can be burned to a DVD for recovering from an upgrade or used for a clean install of the CDS servers. For more information, see the "Imaging a CDS Server with 64-Bit OS using a DVD" section on page 3-16.

Getting a Software File from Cisco.com

To get a software file from Cisco.com, do the following:


Step 1 Launch your web browser and enter the following URL:

www.cisco.com/cisco/web/download/index.html

The Log In page is displayed.

Step 2 Log in to Cisco.com using your designated username and password. The Video and Content Delivery page is displayed, listing the available software products.

Step 3 Click Content Delivery Systems. The Downloads page is displayed.

Step 4 Click the Cisco Content Delivery Applications folder to expand it, and click the Cisco TV Application. The page refreshes and the software releases are displayed.

Step 5 Click the software release you want. The page refreshes and the software image files are displayed.

Step 6 Click the link for the software image file you want.

If this is the first time you have downloaded a file from Cisco.com, the Cisco Systems Inc., Encryption Software Usage Handling and Distribution Policy is displayed. Read the policy, fill in the unfilled fields, and click Accept.

If you previously filled out the Encryption Software Usage and Handling and Distribution form, the form does not display again.

The Download page is displayed with the information about the software image file.

Step 7 Click Download Now to download the file, or click Add to cart to select more image files before downloading them. The Download Cart page is displayed.


Note Make note of the MD5 checksum value to verify the MD5 checksum after download. You can copy and paste the value into a text file for easy reference.


Step 8 Click Proceed With Download. The Cisco End User Software License Agreement is displayed.

Step 9 Read the agreement and click Agree. The Download Software page is displayed.

Step 10 Choose a download option, either Download Manager Option or Non Java Download Option. A new window displays the filename of the ISO image file.

Step 11 Click Download. the File Download dialog box is displayed.

Step 12 Click Save. The Save As dialog box is displayed.

Step 13 Navigate to the location where you want to save the file and click Save. The file downloads.



Note If you are using a Windows system to get the upgrade kit from www.cisco.com, and you plan to use the scp command to copy it to the CDS server, make sure the Windows system does not corrupt the kit when copying it to the CDS server.


Upgrade Sequence for Different Deployments

This section describes the upgrade sequence for a Virtual Video Infrastructure (VVI) and a Content Delivery System (CDS). A VVI includes of Caching Nodes and split-domain management. A CDS consists of Streamers and Vaults, or ISVs.

Upgrading a VVI

This section describes the software upgrade sequence for a Virtual Video Infrastructure (VVI). The upgrade sequence for a VVI in an ISA environment and a VVI in an RTSP environment are the same, except the Caching Nodes are upgraded in a specific order in the RTSP environment.

ISA Environment

A VVI in an ISA environment has the following network design:

Multiple video hub offices (VHOs) and multiple sites per VHO

Shared Content Store

ISA with Stream Destination

Vault Group Redundancy

Caching Nodes (VVI)

Split-domain management (VVIM and Stream Manager [CDSM])

CDSM Redundancy

RTSP Environment

A VVI in an RTSP environment with NGOD deployment and HTTP Streamers has the following network design:

Multiple Stream Groups

Multiple Source Output Ports

Vault Group Redundancy

Caching Nodes (VVI)

Split-domain management (VVIM and Stream Manager [CDSM])

CDSM Redundancy

VVI Upgrade Sequence

The VVI must be upgraded in the following order:

1. Stream Manager (CDSM) of the first Stream Domain (VHO1 in ISA environment). Upgrade the secondary CDSM, then upgrade the primary CDSM.

The primary and secondary CDSM (or VVIM) can be determined by entering the ifconfig -a | more command. The primary has the following output:

eth0:1    Link encap:Ethernet HWaddr 00:11:00:00:00:00 
          inet addr:172.22.98.54 Bcast:172.22.99.255 Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          Memory:b8820000-b8840000 
 
   

The primary CDSM has device eth0:1. The secondary CDSM does not have the virtual IP address as up.

2. Streamers in the first Stream Domain. If the Streamers are in multiple Stream Groups (sites), upgrade the Streamers in the "Control" sites first (sites that have Stream Groups with only a Control server), followed by the "Setup/Control" sites (sites that have Stream Groups with a Setup/Control server). In each Stream Group, upgrade the Streamers in the following order:

a. Available Streamers

b. Backup Streamer

c. Primary Streamer

To identify the Streamers, use the following command:

# cat /proc/calypso/status/streamer/resiliencyinfo 
	Streamer Resiliency Info:
		Service Address: 172.22.98.50
		Control Service: Primary
 
   

3. Repeat tasks 1 and 2 for each Stream Domain in the VVI. Upgrade the secondary CDSM, followed by the primary CDSM, then upgrade the Streamers in the Control site (available Streamers first, backup Streamer second, and primary Streamer last), followed by the Streamers in the Setup/Control site.


Note A Stream Manager can manage multiple VHOs (ISA) or Stream Groups (RTSP). Always upgrade the Stream Manager first, then upgrade the Streamers in each VHO (or Stream Group) managed by this CDSM.


4. Virtual Video Infrastructure Manager (VVIM), which manages Vaults and Caching Nodes. Upgrade the secondary VVIM first, then upgraded the primary VVIM.

5. Caching Nodes. There is no specific order for upgrading the Caching Node in an ISA environment, but to guarantee nonstop services, keep at least one Caching Node online at all times at each site.

Upgrade the Caching Nodes in an RTSP environment in the following order:

a. Available Caching Nodes

b. Backup Caching Node

c. Primary Caching Node

To identify the Caching Nodes in Release 2.1.x, view the /var/log/debugmessages file. The following messages indicate the Caching Node was assigned the role of backup:

Mar 17 14:53:50 cc_c36 kernel: Backup HTTP Locate Port on service address 192.169.87.100
Mar 17 14:53:50 cc_c36 kernel: Starting backup service for HTTP Locate Port address 
192.169.87.100
Mar 17 14:53:50 cc_c36 kernel: Stopped being primary during synchronization to backup for 
service address 192.169.87.100
 
   

To identify the Caching Nodes in Release 2.4.x, use the cat httpinfo command to view the /proc/calypso/status/httpinfo file. The following example indicates that Caching Node 35 is the primary and Caching Node 36 is the backup:

# cat httpinfo
C2 Protocol Info:
 Locate Port Service:
    IPv4 Address: 192.169.87.100
    Primary Server: 35 local
    Backup Server: 36
    Time Offset: 0 usec
 Local Transfer Ports:
    192.169.87.10: Up: Allocated 0bps
    192.169.87.11: Up: Allocated 0bps
    192.169.87.12: Up: Allocated 0bps
 
   

6. Vaults. Upgrade all slave Vaults first, then upgrade the master Vault.

The master and slave Vault can be determined by entering the ifconfig -a | more command. The master has the following output:

eth0:1    Link encap:Ethernet HWaddr 00:11:00:00:00:00 
          inet addr:172.22.98.54 Bcast:172.22.99.255 Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          Memory:b8820000-b8840000 
 
   

The master Vault has device eth0:1. The slave Vault does not have the virtual IP address as up.

Upgrading a CDS

This section describes the software upgrade sequence for a Content Delivery System (CDS) as opposed to a VVI. The upgrade sequence for a CDS in an ISA environment and a CDS in an RTSP environment are the same.

CDS Upgrade Sequence

The CDS must be upgraded in the following order:

1. CDSM. Upgrade the secondary CDSM, then upgrade the primary CDSM.

2. Streamers. If the Streamers are in multiple Stream Groups (sites), upgrade the Streamers in the "Control" sites first (sites that have Stream Groups with only a Control server), followed by the "Setup/Control" sites (sites that have Stream Groups with a Setup/Control server). In each Stream Group, upgrade the Streamers in the following order:

a. Available Streamers

b. Backup Streamer

c. Primary Streamer

To identify the Streamers, use the following command:

# cat /proc/calypso/status/streamer/resiliencyinfo 
	Streamer Resiliency Info:
		Service Address: 172.22.98.50
		Control Service: Primary
 
   

3. Vaults. Upgrade all slave Vaults first, then upgrade the master Vault.


Note If the CDS consists of a CDSM and ISVs, upgrade the CDSM first followed by the ISVs.


Downgrade Sequence for Different Deployments

This section describes the downgrade sequence for a Virtual Video Infrastructure (VVI) and a Content Delivery System (CDS). The software downgrade should be performed on the CDS server types in the reverse order of the upgrade sequence—that is, Vaults, then Caching Nodes, then Streamers, and lastly CDSMs.

If all Streamers have been upgraded, we recommend not downgrading any of the Streamers. This also applies to Caching Nodes and Vaults. If all CDS servers of a specific type (Streamer, Caching Node, Vault, or ISV) have all been upgraded, we recommend not downgrading the software.


Note Before downgrading the software, any problems encountered as a result of the upgrade should be understood first. Downgrading a system (VVI or CDS) may result in loss of configuration changes and loss of content that was ingested since the upgrade. Contact Cisco support before downgrading your system.


Downgrade the slave Vaults first, followed by the master Vault. If a system has redundant CDSMs and multiple Streamer sites, downgrade the system in the following order:

1. Vaults. Downgrade all slave Vaults first, then downgrade the master Vault.

2. Streamers. Downgrade the Control site first, followed by the Setup/Control site. In each Stream Group, downgrade the Streamers in the following order:

a. Available Streamers

b. Backup Streamer

c. Primary Streamer

3. CDSM. Downgrade the secondary CDSM before the primary CDSM.

If the Streamers at a Setup/Control site have not been upgraded, just downgrade the Streamers at a Control site. If the Streamers at a Setup/Control site have been upgraded, downgrade these Streamers first, then downgrade the Control site Streamers.

If the system consists of ISVs and CDSMs, downgrade the ISVs first followed by the CDSMs.

Upgrade and Downgrade Workflow for a CDS Server

The procedure to upgrade or downgrade a CDS server is more complicated than for a CDSM or VVIM. This section covers the following topics:

Software Upgrade Workflow for a CDS Server

Software Downgrade Workflow for a CDS Server

Software Upgrade Workflow for a CDS Server

A high-level view of the software upgrade workflow is as follows:

1. Get the upgrade kit from www.cisco.com and copy it to the target CDS server.

2. Compare the MD5 checksum from www.cisco.com to the file on the target CDS server.

3. Extract the files from the upgrade kit.

4. Offload the CDS server.

5. Copy the TV CDS ISO image file for Release 2.4.1 and run the cdsinstall script.

6. Update (move) the rc.local file and reboot the server.

7. Enable the CDS server by disabling Server Offload.

Software Downgrade Workflow for a CDS Server

A high-level view of the software downgrade workflow is as follows:

1. Make original backup of CDS server available.

2. Compare the MD5 checksum from www.cisco.com to the file on the target CDS server.

3. Extract the files from the downgrade kit.


Note Step 4 is only applicable if you are performing an downgrade from 2.4.1 to 2.0 or 2.1.


4. Offload the CDS server.

5. Restore the original backup of the configuration (this is not the backup from the previous step).

6. Copy the TV CDS ISO image file for the Release you are downgrading to and run the cdsinstall script.

7. Update (move) the rc.local file and reboot the server.

8. Enable the CDS server by disabling Server Offload.