Upgrade 9800 in N+1 Setup with AP Site Tag-based Rolling Upgrade

Introduction

This document describes the site-based Rolling AP Upgrade in an N+1 Network feature that enables a staggered upgrade of APs in an N+1 deployment.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• Catalyst 9800 Wireless LAN Controllers and AP (Cisco IOS®) platforms
• Catalyst 9800 Wireless LAN Controllers software feature sets

Components Used

The information provided in this document is based on the these software and hardware components.

• Two 9800-40 running 17.9.6
• Four 9136 APs

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

This feature helps you to effectively achieve a close to zero-downtime network upgrade in an N+1 network. The existing site filter functionality allows you to perform a software upgrade of a site or all the sites managed by the controller. The APs associated to a specific site tag are upgraded/moved and is monitored for the stability before moving the other site-tag associated APs.

Configure

GUI

Step 1: Establish mobility tunnels between the controllers running the same version.

This link describes the process and steps involved in establishing mobility tunnel between the controllers.

Configure Mobility Topologies on Catalyst 9800 WLCs

Step 2: Go to controller GUI > Administration > Software management.

Step 3: Verify both the controllers are running in INSTALL mode as N+1 hitless is not supported in Bundle mode.’

Step 4: Choose the Transport type from the drop down.

• If you choose My Desktop as the transport type, click Select File to navigate to the file from the Source File Path field.
• If you choose SFTP as the transport type, enter the source IP address, SFTP username, SFTP password, file path, and select the destination.
• If you choose FTP as the transport type, enter the source IP address, FTP username, FTP password, file path, and select the destination.
• If you choose TFTP as the transport type, enter the source IP address, file path, and select the destination.
• If you choose Device as the transport type, choose the file system and file path.

Step 5: Click the Enable Hitless Upgrade option which enables us to select site tag based upgrade.

Step 6: Setting the Site Filter to All Sites provides us the option to use Fallback after upgrade option. Changing the site filter to Custom site prompts us to select the site tags.

Select the site tags to check with first.

Step 7: Enter the destination(secondary) Controller IP and Controller Name.

Step 8: In the AP Upgrade Configuration section, use the AP Upgrade per Iteration drop-down list to select the percentage of APs to be upgraded per iteration. This configures the minimum percentage of APs that must join the destination controller to signal completion of iteration.

Step 9: (Optional) Check the Client Steering.

Step 10: (Optional) In the Accounting Percentage field, choose the percentage of APs that must join the destination controller after each iteration (of the staggered AP upgrade) to consider the iteration as successful. The default value is 50%.

Site Tag Based Upgrade

Step 11: Click Download and Install. This starts the upgrade process and the APs which are mapped to the custom site tags predownloads the image and move to the destination controller.

Once the APs with those mentioned site-tags are moved, before performing Save configuration and Activate,  an option Update Site Filter appears. Add more site tags to the existing list and click that option to upgrade and move those APs in the additionally added site-tag.

Adding Additional Site Tags

Step 12: Once the APs are moved successfully to the destination controller, Click the Save Configuration and Activate which activates the image in the primary controller.

After successful completion of the activation and reload, navigate to the same page and commit the upgrade.

Remarks

• If the secondary controller is already upgraded to the required version, the APs go on a momentary reload to swap the image before joining the secondary one. If you move back the APs to the primary after the upgrade completion, the APs reinitiate the CAPWAP connection to join the primary.
• If the secondary controller is not upgraded to the required version and stays in the same older version as the primary before upgrade, the APs reinitiate the CAPWAP connection to join the controller. If you move back the APs to the primary after it’s upgrade, the APs go on a momentary reload to swap the images before joining the primary one.

CLI

Step 1: Establish mobility tunnels between the controllers running the same version.

This link describes the process and steps involved in establishing mobility tunnel between the controllers.

Configure Mobility Topologies on Catalyst 9800 WLCs

Step 2: Go to enable mode and ensure that both the controllers are in INSTALL mode.

wlc2#show version | i Installation

Installation mode is INSTALL

Step 3: Copy the new image to flash using the command: 

copy tftp:image flash:

Step 4: Add the image package for the installation process using the command:

install add file flash:<package_name>

Step 5: (Optional) Disable client steering using the command:

Source_WLC# no ap upgrade staggered client-steering

Step 6: (Optional) Configure the minimum percentage of APs that must join the destination controller to signal iteration completion using the command:

Source_WLC (config)# ap upgrade staggered iteration completion min-percent

Step 7: (Optional) Configure the action to be taken when APs are missing after an iteration during AP upgrade using the command:

Source_WLC (config)# ap upgrade staggered iteration error action stop

Step 8: (Optional) Configures the maximum time allowed per iteration during AP upgrade. Valid values range from 9 to 60.

Source_WLC (config)# ap upgrade staggered iteration timeout timeout-duration

Source_WLC (config)# exit

Step 9: Predownload the latest image to the APs:

Source_WLC# ap image predownload

Step 9: Adds a site tag to a site filter. Repeat this command again to add more site-tag to the filter:

Source_WLC# ap image site-filter any-image add site-tag

Step 10: This command upgrades and moves the APs of the applied site tags to the destination controller:

Source_WLC# ap image upgrade destination dest_wlc_name dest_wlc_IP

Check if the APs are moved to the destination controller with the command show ap image or show ap summary

Step 11: If needed to add more site-tag to upgrade and move those APs, run this command:

Source_WLC# ap image site-filter any-image add site-tag

Source_WLC# ap image site-filter any-image apply

If upgrade is not completed successfully, use the ap image upgrade destination or ap image move destination command to restart the upgrade process.

Step 12: Verify all the APs have moved to the destination controller. Once verified, activate the image on the source controller.

Source_WLC# install active

Step 13: Commit the changes post the upgrade:

Source_WLC# install commit

Verify

• Verify that the controller is running in INSTALL mode

Source_WLC# show version | i mode  

Installation mode is INSTALL

• Ensure that the mobility tunnel is UP between the controllers

Source_WLC# show wireless mobility summary

Mobility Summary

Wireless Management VLAN: 10

Wireless Management IP Address: 10.107.70.177

Wireless Management IPv6 Address:

Mobility Control Message DSCP Value: 48

Mobility High Cipher : False

Mobility DTLS Supported Ciphers: TLS_ECDHE_RSA_AES128_GCM_SHA256, TLS_RSA_AES256_GCM_SHA384, TLS_RSA_AES128_CBC_SHA

Mobility Keepalive Interval/Count: 10/3

Mobility Group Name: default

Mobility Multicast Ipv4 address: 0.0.0.0

Mobility Multicast Ipv6 address: ::

Mobility MAC Address: 648f.3ebe.bb00

Mobility Domain Identifier: 0x34ac

Controllers configured in the Mobility Domain:

 IP                                        Public Ip                                  MAC Address         Group Name                       Multicast IPv4    Multicast IPv6                              Status                       PMTU

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

10.107.70.177                              N/A                                        648f.3ebe.bb00      default                          0.0.0.0           ::                                          N/A                          N/A

10.107.70.175                              10.107.70.175                              5856.9fe8.ac00      default                          0.0.0.0           ::                                          Up                           1385

• Run show ap upgrade on both the controllers to check where the APs are connected.
• Run show ap upgrade summary to see the upgrade reports.

wlc1# show ap upgrade summary

Report Name                                                      Start time

------------------------------------------------------------------------------------------

AP_upgrade_to_wlc2_822025155858                                  03/08/2025 15:58:58 Austral            

AP_upgrade_from_wlc2_82202516200                                 03/08/2025 16:20:00 Austral            

AP_upgrade_from_wlc2_822025163043                                03/08/2025 16:30:43 Austral            

AP_upgrade_from_wlc2_822025163110                                03/08/2025 16:31:10 Austral           

• Run show ap upgrade name <report_name> to see the progress report and AP status of that timestamp.

wlc1#sh ap upgrade name AP_upgrade_from_wlc2_822025163110

Status: Complete

From version: 17.15.1.6

To version: 17.12.4.22

Started at: 03/08/2025 16:31:10 Austral

Configured percentage: N/A

Percentage complete: 100

End time: 03/08/2025 16:40:53 Austral

Source controller: wlc2

Destination controller: wlc1

Progress Report

---------------

Iterations

----------

Iteration              Start time                          End time                    AP count

------------------------------------------------------------------------------------------------

0                      03/08/2025 16:31:10 Austral         03/08/2025 16:31:10 Austral 0      

1                      03/08/2025 16:31:10 Austral         03/08/2025 16:35:48 Austral 1      

2                      03/08/2025 16:35:48 Austral         03/08/2025 16:40:53 Austral 1      

Upgraded

--------

Number of APs: 2

AP Name                          Radio MAC               Iteration       Status               Site

----------------------------------------------------------------------------------------------------

AP4891.D5EE.7A94                 4891.d5f3.c890          1               Joined               default-site-tag   

AP4891.D5EF.35B8                 6cd6.e304.8ee0          2               Joined               test2              

In Progress

-----------

Number of APs: 0

AP Name                          Radio MAC  

-------------------------------------------------

Remaining

---------

Number of APs: 0

AP Name                          Radio MAC  

-------------------------------------------------

APs not handled by Rolling AP Upgrade

-------------------------------------

AP Name                          Radio MAC               Status          Reason for not handling by Rolling AP Upgrade
--------------------------------------------------------------------------------------------

Troubleshoot

• Post running the ap image site-filter any-image apply command. Wait for the upgrade to complete. If the upgrade is not successful, use the ap image upgrade destination or ap image move destination command to restart the upgrade process.

• The fallback option is not available in both GUI and CLI if you are using custom site-tag option. If needed, the fallback has to be done manually via the CLI from the destination controller using the ap image move destination command. With fallback, use reset or swap command.
  • Swap command interchanges the AP image so that the target code is marked as the primary image for the APs.
  • Reset command reloads the AP. It is assumed that the destination WLC is on the same version as the APs backup image.