Install and Troubleshoot SMUs and Feature Packages in IOS eXR

Introduction

This document describes installation of Software Maintenance Updates (SMUs) and packages for routers that run Cisco IOS^® XR software in eXR versions.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Cisco IOS XR software
• Cisco IOS XR software installation and upgrade procedures

Components Used

This document is not restricted to specific hardware versions, this document apply for all routers running IOS XR 64 Bit.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Related Products

This document can also be used with these hardware versions:

• Cisco 9000 Series Routers
• Cisco 8000 Series Routers
• Cisco NCS 5700 Series Routers
• Cisco NCS 5500 Series Routers
• NCS 540 & 560 Routers

Upgrading Features

Upgrading features is the process of deploying new features and software patches on the router.

• Perform a feature upgrade by installing packages.
• Perform a software patch installation by installing Software Maintenance Upgrade (SMU) files.

Installing a package on the router installs specific features that are part of that package. Cisco IOS XR Software is divided into various software packages; this enables you to select the features to run on your router. Each package contains components that perform a specific set of router functions, such as routing, security, and so on.

For example, the components of the routing package are split into individual RPMs such as BGP and OSPF. BGP is a part of the base software version and is a mandatory RPM, and hence, cannot be removed. However, you can add and remove optional RPMs such as OSPF as required.

Cisco IOS XR SMU Overview

An SMU is a software patch that is installed on the Cisco IOS XR device. The concept of an SMU applies to all Cisco IOS XR hardware platforms.

A Cisco IOS XR SMU is an emergency point fix, which is positioned for expedited delivery and which addresses a network that is down or a problem that affects revenue.

When the system runs into a defect, Cisco can provide a fix for the particular problem in the base current Cisco IOS XR release.

Preparation

Is imperative to first confirm what packages we need and why. Most of the commond issues are related to not necessary instalations using disk space in the routers.

• Once we know what we need to install in the router, we have to download the package from he official site:

Cisco Software Dowload

For this document, the example includes the next 2 SMUs for ASR9K in 7.5.2

SMUs in Cisco Software Download site

• Now, copy these files to the harddisk: with a simple copy operation.

Note: If needed please review the File System Commands on Cisco IOS XR Software

File System Commands

Copy operation for these files, dir harddisk is the command that confirms the files are in the harddisk.

RP/0/RSP0/CPU0:ASR-9006#copy ftp: harddisk: 
Wed Jan 31 00:48:00.784 UTC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
1730560 bytes copied in      9 sec (   192284)bytes/sec

Copy operation success
RP/0/RSP0/CPU0:ASR-9006#copy ftp: harddisk: 
Wed Jan 31 00:49:10.754 UTC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
5980160 bytes copied in     30 sec (   199338)bytes/sec

Copy operation success
RP/0/RSP0/CPU0:ASR-9006#dir harddisk:
Wed Jan 31 00:50:01.732 UTC

Directory of harddisk:
 783363 drwxr-xr-x. 6      4096 Jan 24 10:32 showtech
     15 drwxr-xr-x. 2      4096 Oct 19 05:31 dumper
     31 -rwxr--r--. 1   5980160 Jan 31 00:49 asr9k-sysadmin-7.5.2.CSCwf39757                .tar
     30 -rwxr--r--. 1   1730560 Jan 31 00:48 asr9k-x64-7.5.2.CSCwd75868                .tar
      <SNIP>

23099260 kbytes total (21328112 kbytes free)
RP/0/RSP0/CPU0:ASR-9006#

Before proceeding with the installation, is suggested to do a basic check in the current active packages in XR and Admin.

RP/0/RSP0/CPU0:ASR-9006#sh install active summary 
Wed Jan 31 03:21:02.213 UTC
Label : 7.5.2

    Active Packages: 16
        asr9k-xr-7.5.2 version=7.5.2 [Boot image]
        asr9k-k9sec-x64-2.2-r752
---------------------- Truncated for Brevity -------------------------

RP/0/RSP0/CPU0:ASR-9006#admin show install active summary
Wed Jan 31 03:21:14.778 UTC
     Active Packages: 1
        asr9k-sysadmin-7.5.2 version=7.5.2 [Boot image]

RP/0/RSP0/CPU0:ASR-9006#

Installation

For this example, there are 2 packages. One for the sysadmin and the second for XR VM.

SMUs can distinguish from the XR VM and the System Admin VM. They can be identified by their file names.

For example, asr9k-px-7.9.1.CSCvu599**.pie  is an example of a package for the XR VM. asr9k-sysadmin-7.9.1.pie is associated with the system admin VM.

The XR packages or SMUs are activated from the XR VM, whereas the System Admin packages or SMUs are activated from the System Admin VM.

You can alternatively perform a cross VM operation, by activating or deactivating the System Admin packages and SMUs from XR.

Herein lies the beauty of the installations in eXR; everything is reduced to the correct completion of 3 commands:

1. install add source package path filename
2. install activate package name
3. install commit

• Example of install add operation once we have our desired packages in the harddisk, we can use the extension .tar without a problem.

RP/0/RSP0/CPU0:ASR-9006#install add source harddisk:/ asr9k-x64-7.5.2.CSCwd758**.tar asr9k-sysadmin-7.5.2.CSCwf397**.tar

Note: This command displays the operation ID of the add operation and its status. The operation ID can be later used to execute the activate command.

• Packages now are added, but activation is needed. The operation ID that the router created in the previous step can be used .

RP/0/RSP0/CPU0:ASR-9006#install activate id #
Wed Jan 31 04:00:27.299 UTC
2024-01-31 04:00:29 Install operation # started by *:
  install activate id 172 
2024-01-31 04:00:29 Package list:
2024-01-31 04:00:29     asr9k-iosxr-ce-64-1.1.0.1-r752.CSCwd75868            .x86_64
2024-01-31 04:00:29     asr9k-sysadmin-asr9k-7.5.2.8-r752.CSCwf39757            .x86_64

We can always execute one of these:

install activate package_name

install activate id operation_id

• SMU or package installation can require a router reload, in every succesful scenario finishing the installation with install commit is needed.
  

Verify

Run again the show install active commands in XR and Admin VM, for this example, two more packages to be active, one in each VM.

RP/0/RSP0/CPU0:ASR-9006#sh install active summary 
Wed Jan 31 04:20:44.061 UTC
Label : 7.5.2

    Active Packages: 17
        asr9k-xr-7.5.2 version=7.5.2 [Boot image]
        asr9k-k9sec-x64-2.2-r752
---------------------- Truncated for Brevity -------------------------
        asr9k-iosxr-ce-64-1.1.0.1-r752.CSCwd75868            <<<<<

RP/0/RSP0/CPU0:ASR-9006#admin show install active summary
Wed Jan 31 04:20:53.609 UTC
     Active Packages: 2
        asr9k-sysadmin-7.5.2 version=7.5.2 [Boot image]
        asr9k-sysadmin-asr9k-7.5.2.8-r752.CSCwf39757           .x86_64 <<<<

RP/0/RSP0/CPU0:ASR-9006#

Troubleshoot

There are some commands that always can help us to confirm the logs during a install operation failing and the reason of the failure.

insufficient space in harddisk:

LC/0/0/CPU0:Jan 26 18:51:03.152 MET: resmon[228]: %HA-HA_WD-3-DISK_ALARM_ALERT : A monitored device /misc/disk1 ( harddisk: ) is over 80% utilization. Current utilization = 80. Please remove unwanted user files and configuration rollback points.

The most common scenario is where there is not enough space in the router to copy and handle the installation. This usually happens in big installations; for a huge individual package or several packages to be installed at the same time.

Workaround:

Removing content of harddisk:/dumper

Removing content of harddisk:/showtech

NCS5500 rootfs: above 80% utilization.

LC/0/1/CPU0:Jan 26 18:46:02.245 MET: resmon[218]: %HA-HA_WD-3-DISK_ALARM_ALERT : A monitored device / ( rootfs:/ ) is over 80% utilization. Current utilization = 80. Please remove unwanted user files and configuration rollback points.

Few SMUs are bloated in size (huge size for each RPM), resulting in install failure due to "root-fs" memory limit.

Because of the huge size, we are unable to activate the SMU onto the router (NCS5500)

The issue has been observed when:

• The package component fan out is too large making the SMU size bigger than 100MB
• Dependency (pre-requisite) list has a large number of big packages

As a result, installing the SMU fails due to "root-fs" limit (500MB) reached.

Workaround:

Check the utilization using the command:

show media

Remove superseding packets installed on this router that are not required.

You can delete them from the admin plane using these commands. These are NOT service impacting:

admin

install deactivate superseded

install commit

install remove inactive all

If after removing all the inactive packages, the logs still appear and rootfs still shows the same usage, set the next threshold values to stop seeing a lot of alarms when rootfs reaches 80% of utilization:

Adjust watchdog memory thresholds, with rootfs and also this is also not impacting.

config

watchdog threshold disk minor 90 severe 95 critical 98

commit