Document Number Created By Keith Dsouza EHWIC-4G-LTE-G and C819(H)G-4G-G-K9 MC7710 Modem Firmware Upgrade Procedure Reviewers Department Name/Title Development Engineering Rong Wang, Rahul Dasgupta, Shankar Garikapati DevTest Engineering Shivaji Diwane, Hassan Naseh Modification History Revision Date Originator Comments 1 03/18/13 Keith Dsouza Initial Version 2 03/25/13 Keith Dsouza Included changes suggested by Rahul 04/29/13 Rahul Dasgupta Added description and workaround for call setup issue. Table of Contents 1 Introduction 3 2 Instructions for downloading the Modem Firmware and IOS images 4 3 Procedure for manual upgrade of modem firmware and IOS image 5 3.1 Sample modem firmware upgrade and messages 7 4 Procedure for modem firmware and IOS image upgrade using the EEM script 9 4.1 Pre-requisites before beginning the modem firmware upgrade 9 4.2 Running the EEM script 10 4.3 EEM script 11 4.4 Post Upgrade Procedure 14 4.5 Error scenarios 14 4.6 Workaround for call setup failure after router reload 1 Introduction The Cisco LTE eHWIC and 819 with integrated LTE (EHWIC-4G-LTE-G, C819G-4G-G-K9 and C819HG-4G-G-K9) use the MC7710 LTE modem. A new modem firmware version is now available and recommended for the modem, which also provides improved stability for the LTE connection. For these reasons, IOS release 15.2(4)M3 and subsequent releases will be compatible with the new LTE modem firmware. This means that any ISR model 1900, 2900, 3900 or 819 with LTE must upgrade the firmware on the LTE eHWIC or 819 embedded modem. The current modem firmware version on the MC7710 LTE modem is 03.00.11.00 and should be upgraded to 03.05.19.04 for compatibility with IOS 15.2(4)M3 and later. This document highlights the different procedures to upgrade the LTE modem firmware and IOS depending on the deployment model. There are two methods to upgrade the LTE modem firmware and IOS image. For existing field deployments where LTE is not the primary WAN interface or not the only ISR WAN interface, the manual upgrade process is recommended. * For existing field deployments where a network administrator has out-of-band or local access to the ISR, the manual upgrade process is recommended. * For cases where an ISR has yet to be deployed, the manual upgrade process is recommended. * For existing field deployments where LTE is the only WAN interface, and there is no local or out-of-band administrative access to the ISR, an automated upgrade method using an IOS EEM script is recommended. The EEM script will upgrade the modem firmware and reload the ISR with the IOS image compatible with the new firmware release. 2 Instructions for downloading the Modem Firmware and IOS images 3 Procedure for manual upgrade of modem firmware and IOS image Below are the steps to manually upgrade the MC7710 LTE modem firmware from 03.00.11.00 to 03.05.19.04 A) Boot router with IOS 15.2(4)M2 or the IOS interim image (universalk9-mz.SSA.V152_4_M_LTE available via email request to interim_lte_image@cisco.com along with a cisco.com ID. B) Download the following files onto the router flash: New Modem firmware: MC7710_Global_03.05.19.04_00.cwe New IOS image: cxxx-universalk9-mz.SPA.152-4.M3 (xxx vary by ISR model) C) Verify that both the files are present in the router flash before beginning the modem firmware upgrade (via IOS enable-mode ìshow flashî command). D) Enable logging console (IOS enable mode command ìterm monî) in order to view the status of the upgrade. Initiate the modem firmware upgrade process using the following IOS enable-mode command: For LTE eHWIC Router#microcode reload cellular 0 modem-provision flash:MC7710_Global_03.05.19.04_00.cwe For 819 with embedded LTE Router#microcode reload cellular 0 0 modem-provision flash:MC7710_Global_03.05.19.04_00.cwe The LTE modem firmware upgrade procedure takes a maximum of 15 minutes, on average 5 minutes. E. Once the upgrade is completed, you will see the following message on the ISR console: F/W Upgrade: Complete Successfully Once you see this message, verify that the modem is now on the new firmware by using the appropriate enable-mode IOS command: For the LTE EHWIC Router#show cellular 0//0 hardware For the 819 with embedded LTE Router#show cellular 0 hardware e.g: Router#show cellular 0 hardware Modem Firmware Version = SWI9200X_03.05.19.04 Modem Firmware built = 2012/02/25 11:58:38 Ö If the firmware upgrade does not succeed, the following output would be seen from the previous command: Router#show cellular 0 hardware Modem Firmware Version = SWI9200X_03.00.11.00 Ö Reload the ISR and start the process again from step A. If the firmware upgrade fails 3 times, contact Cisco TAC. F. Once the modem firmware upgrade has been verified, the ISR must be reloaded with the IOS image cxxx-universalk9-mz.SPA.152-4.M3 or later After ISR reload, verify that the new IOS is running via the IOS command ìshow versionî e.g. Router1#show version Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.2(4)M3 Ö 3.1 Sample modem firmware upgrade and messages Below is an example of an upgrade procedure on an 819 platform. It is important to enable console logging to view these messages. Router#microcode reload cellular 0 0 modem-provision flash:MC7710_Global_03.05.19.04_00.cwe Reload microcode? [confirm] Log status of firmware download in router flash?[confirm] Firmware download status will be logged in flash:fwlogfile Microcode Reload Process launched for Cellular 37946756; hw type = 0x6F3 Router# ***************************************************** The interface will be Shut Down for Firmware Upgrade This will terminate any active data connections. ***************************************************** Modem radio has been turned off ***************************************************** Modem will be upgraded! Upgrade process will take up to 15 minutes. During this time the modem will be unusable. Please do not remove power or reload the router during the upgrade process. ***************************************************** Sending F/W[MC7710_Global_03.05.19.04_00.cwe] to the card [41569157 bytes]: Firmware file: MC7710_Global_03.05.19.04_00.cwe sent to the card The current modem F/W App Version: SWI9200X_03.00.11.00AP R2492 CARMD-EN-10526 2011/07/01 19:31:09 The current modem F/W Boot Version: SWI9200X_03.00.11.00BT R2492 CARMD-EN-10526 2011/07/01 19:28:52 The current modem Carrier String: 5 The current modem Device ID: MC7710 The current modem Package Identifier: MC7710_03.00.11.00_00_vzw_020.006_001 The current modem SKU ID: 1584083 FW UPgrade: In the progress. *Feb 21 23:39:35.407: %CISCO800-2-MODEM_DOWN: Cellular0 modem is now DOWN. F/W Upgrade: Complete Successfully *Feb 21 23:42:00.475: %CISCO800-2-MODEM_UP: Cellular0 modem is now UP. *Feb 21 23:42:00.475: %CISCO800-2-MODEM_DOWN: Cellular0 modem is now DOWN. *Feb 21 23:42:05.475: %CISCO800-2-MODEM_UP: Cellular0 modem is now UP. Modem radio has been turned on Router#show cellular 0 hardware | incl Modem Firmware Version Modem Firmware Version = SWI9200X_03.05.19.04 4 Procedure for modem firmware and IOS image upgrade using the EEM script 4.1 Pre-requisites before beginning the modem firmware upgrade Below is a list of steps that need to be taken before beginning the modem firmware upgrade: A. The ISR first needs to be upgraded to IOS 15.2(4)M2 or the IOS interim image (universalk9-mz.SSA.V152_4_M_LTE available via email request to interim_lte_image@cisco.com along with a cisco.com ID). B. Download the following files onto the router flash: New Modem firmware: MC7710_Global_03.05.19.04_00.cwe New IOS image: cxxx-universalk9-mz.SPA.152-4.M3 Verify that both the files are present in the router flash before beginning the modem firmware upgrade. C. Delete any ìboot system flash:î configuration commands from the running configuration (IOS config-mode command ìno boot system flashÖî) D. Copy the EEM scripts from section 4.3 into the ISR running configuration (from IOS configuration mode). E. Verify that the policy is registered by using the enable-mode command: Router# show event manager policy registered F. Verify that every line of the script has been registered since sometimes The editor used to access the router console, truncates the actions of the script entered. G. The EEM script will generate syslog messages. Please enable console logging to view the same. 4.2 Running the EEM script Once steps A through C of the pre-requisites are completed, the modem firmware upgrade procedure is started using the following enable-mode command: Router#event manager run FW For 819 platforms, the slot number is ì0î. For 1900, 2900, 3900 platforms with EHWICs, the slot number indicates the ISR slot where EHWIC-4G-LTE-G is inserted (seen via IOS command ìshow ip interface brief | incl Cellî) e.g For a 1941 ISR with EHWIC-4G-LTE-G in slot 1 which needs to be upgraded, the cli will be Router#event manager run FW 1 For an 819 ISR LTE modem firmware upgrade, the enable-mode command is Router#event manager run FW 0 The EEM script will upgrade the modem firmware from 03.00.11.00 to 03.05.19.04. Following the modem upgrade, the script will load the corresponding IOS image and reload the router. The total upgrade time including router reload is approximately 25 minutes. During this time, the corresponding cellular interface will not be accessible. After the upgrade, log back into the router and verify that the upgrade was successful. If the upgrade was successful, you should see the following output: Router#show cellular 0 hardware Modem Firmware Version = SWI9200X_03.05.19.04 Modem Firmware built = 2012/02/25 11:58:38 Router1#show version Cisco IOS Software, C800 Software (C800-UNIVERSALK9-M), Version 15.2(4)M3 Ö 4.3 EEM script Copy the EEM scripts below into the ISR running configuration as indicated in step 4.1 step D with the following edit (if needed). The EEM script is written assuming that the ISR is initially running the IOS interim image for LTE. If the router is running IOS 15.2(4)M2, replace the following line in the script before executing: action 1.3.4 set old_IOS "c$platform-universalk9-mz.SSA.V152_4_M_LTE" with action 1.3.4 set old_IOS "c$platform-universalk9-mz.SPA.152-4.M2" Script1: event manager applet FW authorization bypass event none maxrun 1200 action 1.0 if $_none_argc ne "1" action 1.0.1 syslog msg "Incorrect number of arguments passed. Please check and try again" action 1.0.2 exit action 1.0.3 end action 1.1 cli command "enable" action 1.2 set slot_number "$_none_arg1" action 1.3 cli command "show version | incl System image file" action 1.3.1 regexp "(.*)c(.*)-universalk9-(.*)\"" "$_cli_result" _match _sub1 _sub2 _sub3 action 1.3.2 set platform "$_sub2" action 1.3.3 set current_IOS "c$_sub2-universalk9-$_sub3" action 1.3.4 set old_IOS "c$platform-universalk9-mz.SPA.152-4.M2" action 1.3.5 set new_IOS "c$platform-universalk9-mz.SPA.152-4.M3" action 1.3.6 set firmware "MC7710_Global_03.05.19.04_00.cwe" action 1.3.7 set old_firmware "SWI9200X_03.00.11.00" action 1.3.8 set new_firmware "SWI9200X_03.05.19.04" action 1.4 if $platform eq "800" action 1.4.1 set cellular_interface 0 action 1.5 else action 1.5.1 set cellular_interface "0/$slot_number/0" action 1.5.2 end action 1.6 cli command "show cellular $cellular_interface hardware | incl Modem Firmware Version" action 1.7 string first "$new_firmware" "$_cli_result" action 1.8 if $_string_result ge 0 action 1.8.1 syslog msg "Modem is already on new firmware $new_firmware. Exiting upgrade!!" action 1.8.2 exit action 1.8.3 end action 2.1 if $current_IOS ne $old_IOS action 2.1.1 syslog msg "Current IOS version is incorrect. Please run $old_IOS before starting upgrade. Exiting upgrade!!" action 2.1.2 exit action 2.2 end action 2.3 cli command "show flash: | incl $new_IOS" action 3.0 string first "$new_IOS" "$_cli_result" action 3.1 if $_string_result lt 0 action 3.1.1 syslog msg "$new_IOS is not present in flash. Exiting upgrade!!" action 3.1.2 exit action 3.2 end action 3.3 cli command "show flash: | incl $firmware" action 5.0 string first "$firmware" "$_cli_result" action 5.1 if $_string_result lt 0 action 5.1.1 syslog msg "$firmware is not present in flash. Exiting upgrade!!" action 5.1.2 exit action 5.2 end action 5.3 cli command "configure terminal" action 5.4 cli command "no boot system" action 5.5 cli command "end" action 6.1 cli command "microcode reload cellular 0 $slot_number modem-provision flash:$firmware" pattern "confirm" action 6.2 cli command "y" action 6.3 wait 400 action 6.4 cli command "event manager run router_reload $old_IOS $new_IOS $old_firmware $cellular_interface" action 6.5 wait 120 action 6.6 exit Script2: event manager applet router_reload authorization bypass event none maxrun 120 action 1.0 set old_IOS "$_none_arg1" action 1.1 set new_IOS "$_none_arg2" action 1.2 set old_firmware "$_none_arg3" action 1.3 set cellular_interface "$_none_arg4" action 1.4 cli command "enable" action 2.0 cli command "show cellular $cellular_interface hardware | inc Modem Firmware Version" action 2.1 set _string_result "0" action 2.2 string first "$old_firmware" "$_cli_result" action 2.3 if $_string_result ge "0" action 2.3.1 set boot_IOS "$old_IOS" action 2.3.2 syslog msg "Firmware did not Upgrade successfully. Please try again after reload" action 2.4 else action 2.4.1 set boot_IOS "$new_IOS" action 2.4.2 syslog msg "Firmware upgraded successfully. value= $_string_result" action 2.4.3 end action 2.5 cli command "configure terminal" action 2.5.1 cli command "boot system flash:$boot_IOS" action 2.5.2 cli command "config-register 0x2102" action 2.5.3 cli command "interface cellular $cellular_interface" action 2.5.4 cli command "no shut" action 2.5.5 cli command "end" action 2.5.6 cli command "write memory" action 2.5.7 reload 4.4 Post Upgrade Procedure The EEM script makes configuration changes as part of the upgrade procedure. These changes are as follows: * The EEM script configures the ISR to boot with 15.2(4)M3 after the firmware upgrade has completed. This change should remain. For example with a 2901/2911/2921 ISR, it would be boot system flash:c2900-universalk9-mz.SPA.V152-4.M3 * The EEM script changes the config register to 0x2102. This change should remain as this is the appropriate value for an ISR to load the startup-config. * Once the upgrade is successful, the EEM scripts should be REMOVED via the following IOS config-mode commands: Router#config term Router(config)#no event manager applet FW Router(config)#no event manager applet router_reload Router(config)#end Router#write memory 4.5 Error scenarios If the LTE modem firmware upgrade did not complete successfully, perform the following based on the appropriate case: Case 1: Check if the following output is observed after the upgrade procedure Router#show cellular 0/1/0 hardware | incl Modem Firmware Version Modem Firmware Version = SWI9200X_03.00.11.00 Router#show version | incl System image System image file is "flash:c2900-universalk9-mz.SSA.152-4.M3" If the above outputs are observed, it implies that the modem firmware upgrade failed. In this case: * Reload the ISR with the previous IOS image: cxxx-universalk9-mz.SSA.V152_4_M_LTE or cxxx-universalk9-mz. SPA.152-4.M2.bin * For manual upgrade, repeat the procedure beginning from section 3 or * For EEM upgrade, remove the EEM script (see step 4.4), and repeat the procedure beginning from section 4 Case 2: If the following output is observed after the upgrade procedure Router#show cellular 0/1/0 hardware | incl Modem Firmware Version Modem Firmware Version = SWI9200X_03.00.11.00 Router#show version | incl System image System image file is "flash:c2900-universalk9-mz.SSA.V152_4_M_LTE" or System image file is "flash:c2900-universalk9-mz.SPA.V152_4_M2" The modem firmware upgrade failed. In this case: * For manual upgrade, repeat the procedure beginning from section 3 or * For EEM upgrade, remove the EEM script (see step 4.4), and repeat the procedure beginning from section 4 Case 3: If the following output is observed after the upgrade procedure Router#show cellular 0/1/0 hardware | incl Modem Firmware Version Modem Firmware Version = SWI9200X_03.05.19.04 Router#show version | incl System image System image file is "flash:c2900-universalk9-mz.SSA.V152_4_M_LTE" or System image file is "flash:c2900-universalk9-mz.SPA.152-4.M2.bin" The modem firmware upgrade succeeded, but the wrong IOS image was loaded onto the router. In this case: * Ensure a single boot system flash: command is configured, pointing to the IOS image cxxx-universalk9-mz.SPA.152-4.M3, by deleting any other boot system: commands (IOS config-mode command no boot system flash) * Perform a write memory enable-mode command, and reload the router with new IOS image (enable-mode command reload). 4.6 Workaround for call setup failure after router reload There is a known issue with the vendor provided 3.5.19.4 firmware that causes call setup to fail on EHWIC-G SKU only, after a router reload. The work around is to reset the modem by using the test cellular modem-power-cycle or the test cellular modem-reset CLI.  Alternatively, user can create a custom EEM applet to reset the modem automatically after router reload. Following is the sample applet: event manager environment cell_int 0/1/0 <= Please change  value depending on the slot of EHWIC event manager applet modem trap authorization bypass event timer watchdog time 50 action 100 cli command "enable" action 105 wait 15 action 200 cli command "show controllers cellular $cell_int | include Modem resets" action 300 set value "$_cli_result" action 400 regexp "Modem resets = ([0-9]*)" "$value" _match action 500 puts "****The result $value, $_match, ****" action 600 if $_match eq "Modem resets = 1" action 800  cli command "configure terminal" action 900  cli command "service internal" action 901  cli command "exit" action 902  cli command "test cellular $cell_int modem-power-cycle" action 903  syslog msg "***** Modem has been power-cycled successfully ******" action 904 else action 905  syslog msg "***** Modem does not require a power-cycle *******" action 906 end ! end Upgrade Instructions: LTE Modem Firmware and IOS for EHWIC-4G-LTE-G, C819(H)G-4G-G-K9 Copyright Cisco Systems 1 4/29/2013