Site-Based Rolling AP Upgrade in N+1 Networks

Site-based rolling AP upgrade in N+1 network

The site-based rolling AP upgrade in an N+1 network is a feature that

  • effectively achieves a zero-downtime network upgrade in an N+1 network

  • allows you to perform a software upgrade of a site or all the sites managed by the controller, and

  • upgrades the software of the APs belonging to a site and monitors the network to see whether it is functioning as intended before adding more sites to the site filter.

Feature history for site-based rolling AP upgrade in N+1 networks

This table provides release and related information for the features explained in this module.

These features are available in all releases subsequent to the one they were introduced in, unless noted otherwise.

Table 1. Feature history

Release

Feature

Feature Information

Cisco IOS XE 17.9.1

Site-Based Rolling AP Upgrade in N+1 Network

This feature helps to achieve a zero downtime network upgrade in N+1 networks.

Note

If the upgrade fails to meet the objectives, all the sites in the site filter can be removed using the ap image site-filter file any-image remove-all command.

The ap image site-filter command is modified to include the any-image keyword that is a substitute for the image file name to support the N+1 AP move site filter.

Prerequisites for site-based rolling AP upgrade in N+1 networks

Ensure these conditions are met for site-based rolling AP upgrade in N+1 networks

  • The source and destination controllers should be in the same mobility group (preferably running the latest image) but with different AP image versions.

  • Image of the destination controller should be available on the source controller.

  • Both the source and destination controllers should be in INSTALL mode.

  • If you reboot the source or destination controller during the N+1 upgrade, you must re-execute the procedure.

Restrictions for site-based rolling AP upgrade in N+1 networks

  • Site filter operations are supported only for N+1 upgrade and N+1 move; fallback and reset options of the ap image upgrade destination command are not supported.

  • You can move APs between controllers that have the same software.

  • The keywords any and remove-all of the ap image site-filter command work only for the N+1 procedures: AP upgrade or move. It will not work for other site filter operations such as AP Model Service Pack (APSP) or AP Device Package (APDP).

Best practices for N+1 redundancy deployments

Use spare controllers in N+1 redundancy deployments to allow APs to fail over when the primary controller becomes non-operational. This approach ensures minimal network downtime, typically lasting 30 to 40 seconds, as APs discover and join the network in local mode.. Ensure all devices reboot and converge efficiently within the N+1 deployment framework to achieve effective zero-downtime during network upgrades.

Configure site-based N+1 uprade (CLI)

This document provides a reference on configuring site-based N+1 upgrades using CLI commands. The configuration involves several critical settings and actions which ensure seamless transitions and effective management of access points across controllers.

These are the key commands and configuration steps

  • Configure Iteration settings for N+1 Upgrade on Primary Controller

  • Configure Site Filters for Predownload and Move APs to Spare Controller

  • Return APs (Per-Site) to Primary Controller using Site Filters

Configure iteration settings for N+1 upgrade on source controller (CLI)

This task involves executing a sequence of CLI commands to set iteration parameters for conducting an N+1 upgrade on the source controller.

Before you begin

Procedure

Step 1

Enter global configuration mode

Example:
Device# configure terminal

Step 2

(Optional) Disable client steering

Example:
Device# no ap upgrade staggered client-steering

Step 3

(Optional) Configure the minimum percentage of APs that must join the destination controller to signal iteration completion.

Example:
Device(config)# ap upgrade staggered iteration completion 50

Step 4

(Optional) Configure the action to be taken when APs are missing after an iteration

Example:
Device(config)# ap upgrade staggered iteration error action stop

Step 5

(Optional) Configure the maximum time allowed per iteration during AP upgrade.

Example:
Device(config)# ap upgrade staggered iteration timeout 18

Valid values range from 9 to 60.

Step 6

Return to privileged EXEC mode

Example:
Device(config)# exit

The APs upgrade and move to the designated destination controller within the mobility group. Any configured site filters and settings will be applied, ensuring that the APs operate under optimal conditions as defined during the configuration process.

Configure site filters for predownload and move APs to destination controller (CLI)

The purpose of this task is to guide users through configuring site filters and performing a controlled predownload of AP images, ensuring a seamless upgrade and transition to the designated destination controller within an N+1 network environment.

Before you begin

See the Prerequisites for Site-based Rolling AP Upgrade in an N+1 Network section.

  • Run all the commands only on the source controller.

Procedure

Step 1

Add a site tag to a site filter

Example:
Device# ap image site-filter any-image add site1

You can repeat this step to set up a multisite filter.

Step 2

Move the APs to a different controller in the mobility group.

Example:
Device# ap image move destination controller2 10.9.34.4

Note

 

It is preferable to move the APs to a different controller running the same image.

Wait for the upgrade to complete.

If upgrade is not completed successfully, you can use the ap image upgrade destination or ap image move destination commands to restart the upgrade process.

Step 3

Add additional site tag to a site filter

Example:
Device# ap image site-filter file any-image add site2

Step 4

Predownloads the image and upgrades the APs based on the site filter.

Example:
Device# ap image site-filter file any-image apply

Note

 

Wait for the upgrade to complete.

Step 5

(Optional) Clear the site filter table and predownloads the image and does a rolling AP upgrade to all the sites.

Example:
Device# ap image site-filter file any-image clear

Step 6

(Optional) Remove all the site filters

Example:
Device# ap image site-filter file any-image remove-all

Upon completion of this task, APs will be upgraded and transitioned to the selected destination controller, operating under optimized conditions with all site filter configurations applied. This process ensures robust performance and continuity within the mobility group.

Return APs (per-site) to source controller using site filters

This task will help to manage and transition APs in an N+1 network environment by adding site tags, moving APs to controllers, and upgrading the APs based on defined site filters.

Run all the commands only on the source controller.

Before you begin

See the Prerequisites for Site-based Rolling AP Upgrade in an N+1 Network section.

Procedure

Step 1

Add a site tag to a site filter.

Example:
Device# ap image site-filter any-image add site1

Step 2

Move the APs back to the parent controller.

Example:
Device# ap image move destination controller2 10.9.34.2

Note

 

Wait for the upgrade to complete.

Step 3

Add an additional site tag to a site filter.

Example:
Device# ap image site-filter any-image add site2

Step 4

Upgrade the APs based on the site filter.

Example:
Device# ap image site-filter any-image apply

Note

 

Wait for the upgrade to complete.

If upgrade is not completed successfully, use the ap image upgrade destination or ap image move destination command to restart the upgrade process.

Step 5

(Optional) Clear the site filter table and predownloads the image, and performs a rolling AP upgrade to all sites where it is not active.

Example:
Device# ap image site-filter any-image clear

Completing this task successfully ensures efficient management of APs within the N+1 network environment. All APs are upgraded and operate with the correct site tags and configurations.

Upgrade workflow for site-based N+1 upgrade

The site-based N+1 upgrade process enhances network reliability by utilizing spare controllers, ensuring seamless software updates without disrupting ongoing service and supporting continuous uptime in enterprise environments.

Summary

Hitless software upgrade uses the concept of N+1 high availability using a spare controller to upgrade the CAPWAP infrastructure comprising controllers and APs. Depending on your choice, the APs are upgraded in a staggered fashion either per site or on all sites using the rolling AP upgrade feature, thereby avoiding network disruption. This ensures that the clients are serviced by neighboring APs while one or selected APs undergo the upgrade process.

Workflow

  1. Initiate upgrade on the source controller. You can choose to upgrade all sites or per site based on your preference.
  2. Move the APs to the destination controller. APs are upgraded in a staggered fashion using the rolling AP upgrade algorithm.
  3. After all the APs move to the destination controller through multiple iterations, activate the target image on the source controller.
  4. The source controller reloads for the new image to take effect.
  5. (Optional) Move the APs back to the source controller using the cli commands.

Result

The process ensures N+1 availability with uninterrupted service to clients while successfully updating AP infrastructure.

Configure site-based N+1 upgrade (GUI)

This task provides instructions to configure a site-based N+1 upgrade using the GUI on a controller. This ensures that the upgrade process is seamless and minimizes downtime for APs and controllers.

Before you begin

  • The controller should be in INSTALL mode.

  • The controller should be paired with another controller and both should be part of the same mobility group.

    The spare controller should be upgraded with the target image.

Procedure

Step 1

Choose Administration > Software Management .

Step 2

From the Transport Type drop-down list, choose an option.

Note

 

In the File Path field, enter the complete path from where you want to download the software image file, including the name of the file.

  • If you choose My Desktop as the transport type, click Select File to navigate to the file from the Source File Path field.
  • If you choose SFTP as the transport type, enter the source IP address, SFTP username, SFTP password, file path, and select the destination.
  • If you choose FTP as the transport type, enter the source IP address, FTP username, FTP password, file path, and select the destination.
  • If you choose TFTP as the transport type, enter the source IP address, file path, and select the destination.

    Note

     
    In controllers, the IP TFTP source is mapped to the service port by default.
  • If you choose Device as the transport type, choose the file system and file path.

Step 3

Check the Enable Hitless Upgrade check box to allow the APs and the controller to be upgraded.

Step 4

From the Site Filter drop-down list, choose All Sites or one or more Custom Sites.

In case you choose to upgrade for All Sites, you can optionally enable Fallback after Upgrade so that the APs move back to the parent controller after the new image has been activated and the parent controller has reloaded.

In case you choose a Custom Site, select the site from the Site Tags drop-down list. In this case, the APs do not move back to the parent controller automatically and you will have to manually move them using CLIs.

Step 5

In the Controller IP Address (IPv4/IPv6) field, enter the source controller's IPv4/IPv6 address.

Step 6

In the Controller Name field, enter the source controller's name.

Step 7

In the AP Upgrade Configuration section, use the AP Upgrade per Iteration drop-down list to select the percentage of APs to be upgraded per iteration. This configures the minimum percentage of APs that must join the destination controller to signal completion of iteration.

Step 8

Check the Client Steering check box to move clients attached to APs undergoing an upgrade to other APs. If the clients still persist on the candidate APs, they are disconnected and the APs will reload with the new image.

Step 9

In the Accounting Percentage field, choose the percentage of APs that should join the destination controller after each iteration (of the staggered AP upgrade) to consider the iteration as successful. The default value is 90%.

Step 10

Tap to select the type of Accounting Action to configure for the APs. If you enable Terminate, the upgrade is terminated if the configured percentage of APs does not join the mobility peer, and a notification is sent via Syslog message. If you choose Ignore, the upgrade continues irrespective of whether the configured percentage of APs are joining the controller or not.

Step 11

In the Iteration Expiry field, select the number of minutes from the drop-down list to configure the expiry time for each iteration.

Step 12

Click Download & Install.

Step 13

Click Save Configuration & Activate.

Step 14

Click Commit to make the activation changes persistent across reloads.

Upon successful completion of the upgrade process, the controllers and APs should operate with the new software image with minimal disruption to network operations. The reliability and performance of the network are maintained during the transition.

Verify site-based rolling AP upgrade in a N+1 network

The commands ensure the successful implementation and seamless operation of site-based rolling AP upgrades within an N+1 network. By utilizing these commands, you will be able to monitor the progress, evaluate the status, validate updates, and troubleshoot potential issues related to AP upgrades.

Use these AP commands to check the progress of the upgrade and debugging

  • show ap summary

  • show ap tag summary

  • show ap status

  • show wireless mobility summary

  • show ap image

  • show ap upgrade

  • show ap upgrade site

  • show ap upgrade site summary

  • show ap upgrade name report-name

  • show wireless mobility ap-list

To view the summary of all the connected APs, use this command


Device# show ap summary

Number of APs: 8

AP Name             Slots  AP Model           Ethernet MAC    Radio MAC       Location           Country  IP Address    State
-------------------------------------------------------------------------------------------------------------------------------------
AP00D7.8F9A.43DE    2      AIR-AP2802I-D-K9   00d7.8f9a.43de  002c.c8df.3ca0  default location    IN       10.9.48.254  Registered
AP4C77.6D21.9098    2      AIR-AP2802E-N-K9   4c77.6d21.9098  00be.7573.b340  default location    IN       10.10.10.52  Registered
AP00F2.8B27.BB2C    2      AIR-AP2802I-D-K9   00f2.8b27.bb2c  0896.ad9b.f9e0  default location    IN       10.9.44.51   Registered
APA023.9F41.5A38    2      AIR-AP2802I-D-K9   a023.9f41.5a38  1880.90f4.7b00  default location    IN       10.10.10.51  Registered
AP00A3.8E4A.762C    2      AIR-AP2802I-D-K9   00a3.8e4a.762c  1880.90f5.14e0  default location    IN       10.9.48.54   Registered
AP40CE.2485.D616    2      AIR-AP3802I-D-K9   40ce.2485.d616  4001.7aca.5960  default location    IN       10.9.50.42   Registered
AP40CE.2485.D62C    2      AIR-AP3802I-D-K9   40ce.2485.d62c  4001.7aca.5aa0  default location    IN       10.10.10.53  Registered
AP2C57.4188.4BC4    3      C9130AXE-D         2c57.4188.4bc4  cc7f.75a8.78e0  default location    IN       10.9.34.207  Registered

To view the summary of all the access points with policy tags, use this command


Device# show ap tag summary
Number of APs: 8

AP Name             AP Mac           Site Tag Name     Policy Tag Name       RF Tag Name       Misconfigured    Tag Source
----------------------------------------------------------------------------------------------------------------------------
AP00D7.8F9A.43DE    00d7.8f9a.43de   site3              default-policy-tag   default-rf-tag    No               Static
AP4C77.6D21.9098    4c77.6d21.9098   site3              default-policy-tag   default-rf-tag    No               Static
AP00F2.8B27.BB2C    00f2.8b27.bb2c   site3              default-policy-tag   default-rf-tag    No               Static
APA023.9F41.5A38    a023.9f41.5a38   default-site-tag   default-policy-tag   default-rf-tag    No               Default
AP00A3.8E4A.762C    00a3.8e4a.762c   site1              default-policy-tag   default-rf-tag    No               Static
AP40CE.2485.D616    40ce.2485.d616   site2              default-policy-tag   default-rf-tag    No               Static
AP40CE.2485.D62C    40ce.2485.d62c   site2              default-policy-tag   default-rf-tag    No               Static
AP2C57.4188.4BC4    2c57.4188.4bc4   default-site-tag   default-policy-tag   default-rf-tag    No               Default

To view the status of the access points, use this command


Device# show ap status
AP Name                             Status     Mode              Country
-------------------------------------------------------------------------
AP00A3.8E4A.762C                    Enabled    Local             IN
AP00D7.8F9A.43DE                    Enabled    Monitor           IN
AP00F2.8B27.BB2C                    Enabled    Local             IN
AP2C57.4188.4BC4                    Enabled    Local             IN
AP40CE.2485.D616                    Enabled    Local             IN
AP40CE.2485.D62C                    Enabled    Local             IN
AP4C77.6D21.9098                    Enabled    Local             IN
APA023.9F41.5A38                    Enabled    Local             IN

To display the summary of the mobility manager, use this command


Device# show wireless mobility summary

Mobility Summary

Wireless Management VLAN: 34
Wireless Management IP Address: 10.9.34.5
Wireless Management IPv6 Address:
Mobility Control Message DSCP Value: 48
Mobility High Cipher : False
Mobility DTLS Supported Ciphers: TLS_ECDHE_RSA_AES128_GCM_SHA256, TLS_RSA_AES256_GCM_SHA384, TLS_RSA_AES128_CBC_SHA
Mobility Keepalive Interval/Count: 10/3
Mobility Group Name: mobility-1
Mobility Multicast Ipv4 address: 10.0.0.1
Mobility Multicast Ipv6 address: ::
Mobility MAC Address: 001e.14a5.b3ff
Mobility Domain Identifier: 0x39ab

Controllers configured in the Mobility Domain:

 IP        Public Ip  MAC Address    Group Name  Multicast IPv4 Multicast IPv6  Status  PMTU
---------------------------------------------------------------------------------------------
10.9.34.5  N/A        001e.14a5.b3ff mobility-1  0.0.0.0        ::              N/A     N/A
10.9.34.2  10.9.34.2   001e.bd2d.f2ff mobility-1  0.0.0.0        ::              Up      1385
10.9.34.3  10.9.34.3   001e.14c1.cbff mobility-1  0.0.0.0        ::              Up      1385
10.9.34.4  10.9.34.4   001e.140e.4bff mobility-1  0.0.0.0        ::              Up      1385

To view the cumulative statistics regarding the AP images in the controller, use this command


Device# show ap image

Total number of APs  : 8

Number of APs
        Initiated                  : 0
        Downloading                : 0
        Predownloading             : 0
        Completed downloading      : 0
        Completed predownloading   : 0
        Not Supported              : 0
        Failed to Predownload      : 0
        Predownload in progress    : No
AP Name           Primary Image Backup Image Predownload Status Predownload Version  Next Retry Time Retry Count  Method
------------------------------------------------------------------------------------------------------------------------
AP00D7.8F9A.43DE  17.9.0.19     17.8.0.74    None                0.0.0.0             N/A              0           N/A
AP4C77.6D21.9098  17.9.0.19     17.8.0.74    None                0.0.0.0             N/A              0           N/A
AP00F2.8B27.BB2C  17.9.0.19     17.9.1.19    None                0.0.0.0             N/A              0           N/A
APA023.9F41.5A38  17.9.0.19     17.8.0.74    None                0.0.0.0             N/A              0           N/A
AP00A3.8E4A.762C  17.9.0.19     17.9.1.19    None                0.0.0.0             N/A              0           N/A
AP40CE.2485.D616  17.9.0.19     17.9.1.19    None                0.0.0.0             N/A              0           N/A
AP40CE.2485.D62C  17.9.0.19     17.8.0.82    None                0.0.0.0             N/A              0           N/A
AP2C57.4188.4BC4  17.9.0.19     17.9.1.19    None                0.0.0.0             N/A              0           N/A

To verify the AP upgrade on the controller, use this command


Device# show ap upgrade

AP upgrade is in progress

From version: 17.9.0.19
To version: 17.9.1.25

Started at: 01/28/2022 09:53:07 IST
Configured percentage: 5
Percentage complete: 0
Expected time of completion: 01/28/2022 13:33:07 IST

Client steering: Enabled
Iteration expiry time: 15 minutes
Accounting percentage: 95%
Accounting action: Abort

Rolling AP Upgrade Site Summary
-------------------------------
site3

Progress Report
---------------
Iterations
----------
Iteration       Start time                   End time                    AP count
-------------------------------------------------------------------------------------
0               01/28/2022 09:53:07 IST      01/28/2022 09:53:07 IST     1
1               01/28/2022 09:53:07 IST      ONGOING                     0

Upgraded
--------
Number of APs: 1
AP Name                Radio MAC       Iteration   Status     Site
------------------------------------------------------------------------------------
AP00D7.8F9A.43DE       002c.c8df.3ca0  0           Rebooted    site3

In Progress
-----------
Number of APs: 1
AP Name                          Radio MAC
-------------------------------------------------
AP00F2.8B27.BB2C                 0896.ad9b.f9e0

Remaining
---------
Number of APs: 1
AP Name                          Radio MAC
-------------------------------------------------
AP4C77.6D21.9098                 00be.7573.b340

APs not handled by Rolling AP Upgrade
-------------------------------------
AP Name      Radio MAC           Status          Reason for not handling by Rolling AP Upgrade
-----------------------------------------------------------------------------------------------------

To verify the AP upgrade information on the sites, use this command


Device# show ap upgrade site

Site-filtered AP upgrade report data
====================================
Source controller: Controller1
Destination controller: Controller2

From version: 17.9.0.19
To version: 17.9.1.25
Site-filters present: Yes

AP image upgrade site summary
-----------------------------
Operation: N+1 upgrade

Site Tag                                  Status
---------------------------------------------------------
site3                                     In Progress

AP upgrade reports linked to these site-filters
-----------------------------------------------

Start time               Operation type             Report name
------------------------------------------------------------------------
01/28/2022 09:53:07 IST  AP image upgrade/move CLI  AP_upgrade_to_DEvice2_28020229536

To verify the AP image upgrade site summary, use this command


Device# show ap upgrade site summary

AP image upgrade site summary
-----------------------------
Operation: N+1 upgrade

Site Tag                                  Status
---------------------------------------------------------
site3                                     In Progress

To view AP upgrade information based on the upgrade report name, use this command


Device# show ap upgrade name AP_upgrade_to_Device2

AP upgrade is complete

From version: 17.9.0.19
To version: 17.9.1.25

Started at: 01/28/2022 14:12:49 IST
Configured percentage: 5
Percentage complete: 100
End time: 01/28/2022 14:18:59 IST

Client steering: Enabled
Accounting percentage: 95%
Iteration expiry time: 15 minutes
Accounting action: Abort

Rolling AP Upgrade Site Summary
-------------------------------
site1
site2

Progress Report
---------------
Iterations
----------
Iteration     Start time                    End time                    AP count
-----------------------------------------------------------------------------------------
0             01/28/2022 14:12:49 IST       01/28/2022 14:12:49 IST     0
1             01/28/2022 14:12:49 IST       01/28/2022 14:15:54 IST     1
2             01/28/2022 14:15:54 IST       01/28/2022 14:18:59 IST     1

Upgraded
--------
Number of APs: 2
AP Name               Radio MAC         Iteration  Status          Site
--------------------------------------------------------------------------------------------
AP40CE.2485.D616      4001.7aca.5960    1          Joined Member   site2
AP40CE.2485.D62C      4001.7aca.5aa0    2          Joined Member   site2

In Progress
-----------
Number of APs: 0
AP Name                          Radio MAC
-------------------------------------------------

Remaining
---------
Number of APs: 0
AP Name                          Radio MAC
-------------------------------------------------

APs not handled by Rolling AP Upgrade
-------------------------------------
AP Name      Radio MAC      Status    Reason for not handling by Rolling AP Upgrade
------------------------------------------------------------------------------------------------

To display the list of access points known to the mobility group, use this command


Device# show wireless mobility ap-list

AP name                           AP radio MAC      Controller IP     Learnt from
--------------------------------------------------------------------------------------
Unknown                           002c.c8df.3ca0    10.9.34.5          Self
Unknown                           00be.7573.b340    10.9.34.5          Self
Unknown                           0896.ad9b.f9e0    10.9.34.5          Self
Unknown                           1880.90f4.7b00    10.9.34.5          Self
Unknown                           1880.90f5.14e0    10.9.34.5          Self
Unknown                           4001.7aca.5960    10.9.34.5          Self
Unknown                           4001.7aca.5aa0    10.9.34.5          Self
Unknown                           687d.b45e.4b60    10.9.34.3          Mobility Group
Unknown                           cc7f.75a8.78e0    10.9.34.5          Self