THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
|Affected Product ID
||HS1: Device stuck with rommon upgrade from 29r to 38r|
||ASR920/RSP2: Bundle 43r ROMMON Changes to 17.3.1|
The routers listed in the Products Affected section will fail to boot during the upgrade to IOS XE Release 17.3.1 because of the image size increase.
The releases bundled with FPGA (0x00010039 or 0x10000007) will suspend the router during the ROMMON upgrade process.
The upgrade process will succeed if the upgrade is performed with the router in sub package mode. In order to perform a clean upgrade, follow the upgrade instructions that are provided in the Workaround/Solution section of this Field Notice.
The upgrade process will fail if the upgrade is performed with the router in consolidated binary mode. Recovery instructions are also provided in the Workaround/Solution section.
Because of image size increase, the router fails to boot during upgrade and it will reboot continuously with this console message: mount package service fail.
In order to boot the IOS XE image 17.3.1 successfully, the ROMMON version 43r is needed. The latest FPGA version (0x00010040 or 0x10000008) is also needed in order to ensure that the router does not become suspended during the upgrade.
Sample error logs are provided below.
Mount packages service fail log error
File reception completed.
Boot image size = 511841050 (0x1e82131a) bytes
[103.397489] localhost /usr/sbin/boot_verify_package: /tmp/sw/isos/asr920igp-sipspa.BLD_V173_THROTTLE_LATEST_20200502_210845_V17_3_0_149.SSA.pkg: Digital Signature Verification Failed, error=No keys for the specific key type found
Apr 10 03:04:28.858: %BOOT-3-SYSD_STARTFAIL: R0/0: Failed to launch boot task mount_packages.service ( exit-code )
Apr 10 03:04:37.454: %BOOT-0-BOOT_COMPLETE_FAIL: R0/0: Critical boot tasks failed: droputil.service iptables.service mount_packages.service oom.service xinetd@-etc-xinetd.conf.service xinetd@-etc-xinetd_telnetd.conf.service
Failed to set wall message, ignoring: The name org.freedesktop.login1 was not provided by any .service files
Failed to reboot system via logind: The name org.freedesktop.login1 was not provided by any .service files
Router suspend during ROMMON upgrade error
System Bootstrap, Version 15.6(29r)S, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 2018 by cisco Systems, Inc.
Compiled Mon 16-Jul-18 23:28 by
Starting Initialization of FMAN0
Loading ucode for FMAN0, size: 31424, ver: 106.04.14
Silicon Rev Major:Minor [1:1]
Initializing the pci..
IOFPGA version …
... Checking 1G Network PLL LOCK Status ...
PASS: 1G HSS0 PLL LOCKED HSS0_STATUS_REG Address 0x14300040 Data 0xf080002 ...
PASS: 1G HSS1 PLL LOCKED HSS1_STATUS_REG Address 0x14300044 Data 0xf010002 ...
... Checking 10G Network PLL LOCK Status ...
PASS: 10G HSS0 PLL LOCKED HSS0_STATUS_REG Address 0x14400040 Data 0x5010002 ...
PASS: 10G HSS0 PLL LOCKED HSS1_STATUS_REG Address 0x14400044 Data 0x1080002
... UEA platform with 1048576 Kbytes of main memory
Rommon upgrade requested
Flash upgrade reset 1 in progress
In order to perform the upgrade to IOS-XE Release 17.3.1, you must boot the software in sub package mode. In sub package mode, the router boots without any issues, and auto-ROMMON upgrade to version 43r happens on boot.
Follow these steps in order to boot the router in sub package mode:
- Copy the 17.3.1 release binary and expand it to configure packages.conf.
- Set the boot variable to packages.conf and reload the router.
- The auto-ROMMON upgrade to version 43r and the FPGA upgrade to latest version are completed in parallel and result in a successful upgrade to IOS XE 17.3.1
NOTE: Detailed instructions to boot the device in sub package mode are available here:
If the upgrade failed because the router was not in sub package mode, the mount boot package error might cause the router to enter a contnuous system reboot state, or the ROMMON upgrade issue might cause the router to enter a suspend state. This section provides instructions to recover the router from each of these states.
Follow these steps in order to recover the router from a continuous system reboot state because of a mount boot package error:
- Ensure that you are connected to the router in console mode.
- After you log in, press CTRL+C in order to prevent autoboot and enter ROMMON mode.
- Run the command “unset BOOT” in order to remove the existing boot variable.
- Run the command “BOOT= bootflash:<Your earlier release>” [Example: asr920igp-universalk9_npe.16.09.05.SPA.bin]
- Run the command “sync” in order to sync the configuration.
- Run the command “reset” in order to load the earlier image.
- When the system is back to the console prompt, follow the upgrade instructions above.
Follow these steps in order to recover the router from suspend state because of the ROMMON upgrade issue:
- Power cycle the router.
- Run the command #show platform on the router in order to see if the ROMMON upgrade was successful.
Chassis type: ASR-920-12SZ-D
Slot Type State Insert time (ago)
--------- ------------------- --------------------- -----------------
0/0 12xGE-10GE-FIXED ok 00:04:59
R0 ASR-920-12SZ-D ok, active 00:09:35
F0 ok, active 00:09:35
P0 ASR920-PSU0 ok never
P1 ASR920-PSU1 ps, fail never
P2 ASR920-FAN ok never
Slot CPLD Version Firmware Version
--------- ------------------- ---------------------------------------
R0 20050708 15.6(43r)S
F0 20050708 15.6(43r)S
- Follow the upgrade instructions above.
For More Information
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Receive Email Notification For New Field Notices
My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.