When Cisco® started selling the Cisco IP Phone 7940 and 7960 models, the firmware needed to run the phones was relatively small (less than 1 MB). Because updated phone models with enhanced features are now available and it is possible to centralize the call-processing servers, the method by which firmware is upgraded on a phone has become increasingly more important. Over a high-bandwidth LAN environment, the download process and duration are limited by how quickly the phone can consume the downloaded file. But when downloading new firmware over a low-bandwidth link, the process is limited by the size and latency of the data pipe between the firmware server and the upgrading phone.
Cisco Unified Communications Manager now has two additional methods for updating phone firmware to improve the speed and efficiency of updates on both LAN and WAN environments. This document discusses the three different methods for firmware deployment offered by the Cisco Unified Communications Manager environment.
Traditional TFTP Firmware Download
The original voice-over-IP (VoIP) phones that shipped with Cisco Unified CallManager 2.2 had a firmware image size of 145 Kb. Even when the Cisco IP Phone 7940G and 7960G models were released, the firmware image was just slightly larger at 245 Kb. So when a firmware upgrade for a phone was needed, the amount of data that needed to be transported over the network was relatively small. When downloading new firmware over a low-bandwidth link (384 kbps or less), sending a 245-Kb firmware image did not result in extended downtime for the phones during the upgrade.
When Cisco released the Cisco Unified IP Phone 7941G and 7961G models, the firmware needed to run the phones increased to 7.1 Mb. This size increase had a negligible effect on LAN-based upgrades because the download speed was still limited by the ability of the phone to consume the data, but the increase in file size for a download over a WAN connection significantly affected the duration to complete the upgrade. And as the number of phones at a WAN-connected remote site increases, the phone upgrade time increases at a slightly less than linear rate.
One of the challenges with the traditional firmware upgrade strategy (the default image-distribution method used in all versions up to and including Cisco Unified Communications Manager 8.0) is illustrated when a phone has to download a new firmware image. Each phone initiates a TFTP session to the Cisco Unified Communications Manager TFTP server. The traditional TFTP upgrade process is an "every man for himself" strategy and the download process of each phone occurs independent of any other phone near it. The primary advantage to this method of image distribution is that it is a proven model that has been used for more than 10 years, and every phone eventually completes an upgrade. But when this download strategy is coupled with the fact that the transfer protocol is TFTP and each packet needs to be acknowledged before the next packet is sent, the likelihood of overwhelming the data queue on a low-bandwidth WAN interface is high.
To illustrate how the traditional TFTP download method works, consider the example of upgrading 40 Cisco Unified IP Phone 7965 phones. If the 40 phones are connected to the TFTP server through a LAN connection, all phones will request the new firmware (about 7.1 MB) around the same time, and the TFTP server will deliver ~284 Mb of data faster than the phones can consume the data. The upgrade time for 1 phone or 40 phones in the LAN environment will be nearly the same. But if those same 40 phones attempt to upgrade their firmware over a 384-kbps WAN link, the process will take much longer because the phones will be requesting 284 Mb of data over a relatively small 384-kbps link.
Even in an ideal situation with no other traffic and near zero latency, it would take about 13 minutes for the data to be delivered. But in this case, the download conditions are not ideal and the round-trip time will typically run 130-150 ms. And because of congestion at the WAN data queue caused by the large amount of TFTP data traffic, the phones will be very slow in completing the upgrade. Adding more phones to a remote location would only make the upgrade time longer.
Figure 1. Cisco Unified IP Phone 7962 and 7965 Models Upgrade Time over the WAN
Alternative Image-Distribution Solutions
When unified communications customers began to take advantage of the ability to locate phones remote to the call-processing servers and the firmware image size increased, Cisco added support to the Cisco Unified Communications Manager for two additional methods for distributing phone firmware.
One of the alternate methods of image distribution is with the load server, a customer-provided TFTP server that contains the image files the phone needs to complete an upgrade. A TFTP server can be a Cisco IOS® Software Gateway or a PC-based third-party TFTP server. The advantage to using the load server to deliver firmware is that you can direct phones to retrieve firmware files from a LAN-connected device instead of the default Cisco Unified Communications Manager TFTP server. If the TFTP load server is connected through a LAN that is local to the phone, the upgrade process will complete in approximately the same time as if the phones were local to the Cisco Unified Communications Manager TFTP server.
When configured correctly, the load-server image distribution works very well. But because image files must be manually moved to the load server, there is room for error when using this firmware deployment model. The Cisco Unified Communications Manager system administrator is responsible for identifying the required images, downloading the images, and then verifying the integrity of the images. Additionally, when a load server is configured on a phone, if the files are not available or invalid on the load server, the phone will never complete the upgrade.
Peer Firmware Sharing
The other image-distribution method that Cisco offers is called Peer Firmware Sharing (PFS), which works by setting up a parent-child hierarchy of the phones in which a firmware image is downloaded by the parent phone to a child phone. The advantage of using Peer Firmware Sharing is that instead of all phones individually retrieving a software image, they pass the image along from one phone to another phone on the same subnet. Figure 2 shows an established hierarchy of phones for image distribution.
Figure 2. Established Peer Firmware Sharing distribution hierarchy
As the phone at the root of the tree is downloading a firmware image, the other phones wait for the download to complete. As long as the parent is making progress on the file download, a child phone continues to wait for the download to finish. When the download is complete, the image is then transferred through a TCP connection to the child phones. Because the image transfer is device-to-device transfer using a TCP connection on the same subnet, this file transfer does not cross a router or firewall, so the file transfer is efficient and fast. After an image is transferred from a parent phone to all child phones, the TCP connection closes and the phone restarts to complete the firmware upgrade. To prevent any one phone from getting overloaded, the hierarchy establishes in a semi-balanced tree structure, with no single parent having more than two children associated to it. The primary benefit of the Peer Firmware Sharing model is that it dramatically reduces the amount of time to complete an upgrade over a low-bandwidth connection. The more phones that need to be upgraded at a given site the better the performance increase.
Returning to the example of the remote site connected through a 384-kbps link with 40 Cisco Unified IP Phone 7965 phones deployed: If a firmware upgrade is started, the phones will take 1 hour and 47 minutes to upgrade using the traditional TFTP download method. The same upgrade scenario using the Peer Firmware Sharing method will take only 9 minutes and 20 seconds-the installation time is reduced by 92 percent.
Figures 3 and 4 show a dramatic reduction in the upgrade time with the performance improvements based on the number of phones and download times for each firmware distribution method.
Figure 3. Firmware Download Comparison for Cisco Unified IP Phone 7942 and 7965 Models
Figure 4. Firmware Download Comparison for Cisco Unified IP Phone 8961 and 9971 Models
Choosing the Right Distribution Method
Which of the three different image-distribution methods discussed so far is the best for a customer deployment? Each method has advantages and disadvantages, and they are summarized in Table 1.
Table 1. Summary of Distribution Models
Peer Firmware Sharing
• Hierarchy is automatic
• One download per phone model on a subnet
• Uses TCP
• Fails back to TFTP
• Speeds up LAN upgrades
• Reduces TFTP CPU load during upgrade
• Has same download time as LAN image distribution
• Distributes TFTP load over multiple TFTP servers
• Proven distribution
• Default behavior
• Must be enabled on each phone
• Hierarchy is formed for each phone model
• Hierarchy is limited to subnet
• IP must be set on each phone
• Administrator must manually file copy to load server
• No failback to TFTP on error
• More prone to user error
• High-bandwidth requirements
• Multiple requests for same file
• High CPU usage on TFTP server
Although the load-server model has the best performance numbers for image distribution, because the load server is manually defined for each device and all files required for an upgrade must be present on the load server, this solution is undesirable.
The Peer Firmware Sharing model is enabled by selecting a checkbox on the device-specific configuration section of the phone configuration. But an administrator can set the firmware sharing parameter using the Bulk Administration Tool features to enable all phones for Peer Firmware Sharing very quickly. Additionally, the ability for the phones to create a semi-balanced tree hierarchy based on phone model means an administrator can greatly increase image-distribution efficiency without having to manage files or other load servers. Thus this model is more efficient.
The Cisco Unified IP Phones 7900 Series is unusable during an upgrade. This fact makes reducing the upgrade time even more important for providing voice services to users. With the recent release of the Cisco Unified IP Phones 8900 and 9900 models, the firmware update process has been changed to minimize the downtime of a phone during upgrade. The newer Cisco Unified IP Phones 8900 and 9900 models perform a firmware upgrade as a background process and the phone continues normal operations. The new firmware is stored as an inactive image on the phone, and then when validated, the phone restarts and switches to the new firmware image. This process limits the service outage of the phone to 1-2 minutes (while restarting). The background downloading of a new firmware does not exclude the newer phones from taking advantage of the Peer Firmware Sharing feature. You can use both to greatly reduce the total time for all phones to upgrade.
Note: For the Cisco Unified IP Phone 7900 Series, the Peer Firmware Sharing feature is not applicable to the Cisco Unified IP Phone 7985, IP Conference Stations 7936 and 7937G and the wireless handset portfolio (i.e., 7921G, 7925G).The Cisco Unified IP Phone 6900 Series is also not eligible for the Peer Firmware Sharing as of time of publication of this document. The IP Communications Business Unit is investigating support of Peer Firmware Sharing for the 6900 Series portfolio in a future firmware release.
Cisco will continue to release new phone firmware for use with Cisco Unified Communications Manager as new features are added and defects are fixed. The ability for a customer to efficiently deploy new firmware has become more of a concern with the proliferation of centralized call processing and phones that are remote from the Cisco Unified Communications Manager servers. The use of Peer Firmware Sharing can greatly reduce the amount of time spent waiting for a phone upgrade to complete. By implementing an alternate download strategy, system administrators can both reduce the amount of redundant traffic moved around their network and reduce how much time they spend waiting for the phones to complete the upgrade and return to service.