Cisco Catalyst SD-WAN Multitenancy Guide, Releases 26.x and Later

PDF

Cisco Catalyst SD-WAN Multitenancy Guide, Releases 26.x and Later

Disaster recovery in an overlay network with virtual routers

Want to summarize with AI?

Log in

Explains how to recover a multitenant SD-WAN Manager deployment with virtual routers by using a standby cluster, restoring configuration backups, and activating the standby cluster after a failure.


The following disaster recovery procedure applies to an overlay network in which Cisco vEdge Cloud routers are deployed at branch locations.

Summary

If a Multitenant SD-WAN Manager cluster or the data center hosting the SD-WAN Manager nodes in the cluster fail, you can recover from the failure by activating a standby SD-WAN Manager cluster. You can perform disaster recovery as follows:

Workflow

  1. Deploy and configure a standby SD-WAN Manager cluster. The standby SD-WAN Manager cluster is not part of the overlay network and is not active.
  2. Back up the configuration database of the active SD-WAN Manager cluster periodically. Choose a SD-WAN Manager node in the cluster that hosts the configuration database service and back up the configuration database.
  3. If the active SD-WAN Manager cluster fails, restore the most recent configuration database on the standby SD-WAN Manager cluster, activate the standby SD-WAN Manager cluster, and remove the previously active SD-WAN Manager cluster from the overlay network.
  4. Choose a SD-WAN Manager node in the cluster that hosts the configuration database service and restore the configuration database backed up from the previously active SD-WAN Manager cluster.
  5. To test disaster recovery, you can simulate a scenario in which the active SD-WAN Manager cluster fails. One way to simulate such a failure would be by disabling the tunnel interface as described in this document.


Restore SD-WAN Manager cluster using the configuration database backup

Restore the most recent backup of the configuration database from the active SD-WAN Manager cluster on the standby SD-WAN Manager node to which you copied this backup.

  • The restore operation does not restore all the information included in the configuration database. SD-WAN Manager configurations such as users and repositories must be configured on the standby SD-WAN Manager node after the configuration database is restored using the backup.

  • When you complete the steps that follow, the previously active SD-WAN Manager nodes cannot be reused. To reuse the nodes, you must perform additional steps that are beyond the scope of this document.

Procedure

1.

On the CLI of the standby SD-WAN Manager node, run the following command: request nms configuration-db restore path file-path

Example:

In the following example, the configuration database is restored using the backup file db_backup.tar.gz.



Standby-vManage# request nms configuration-db restore path /home/admin/db_backup.tar.gz
Configuration database is running in a standalone mode
Importing database...Successfully restored database
2.

Verify services and node list on the standby SD-WAN Manager nodes

  1. Verify that the appropriate services are running on the standby SD-WAN Manager nodes: On the CLI of each standby SD-WAN Manager node, run the request nms all status command.

    From the command output, verify the services running on the node.

  2. Verify that every standby SD-WAN Manager node has a list of all the active and standby SD-WAN Manager nodes.

    1. From the SD-WAN Manager menu, choose Configuration > Devices > Controllers.

    2. Verify that the page displays all active and standby SD-WAN Manager nodes.

3.

Run the these commands:

  1. Log in to the CLI of each Cisco SD-WAN Validator and run the show orchestrator valid-vmanage-id command.

  2. Log in to the CLI of Cisco vEdge Cloud router and run the show control valid-vmanage-id command.

    In the command output, verify that the chassis numbers of the active and previously active SD-WAN Manager nodes are listed.

4.

Enable the transport interface on VPN 0 on the standby SD-WAN Manager nodes using either of the following methods:

  1. Enable the transport interface in VPN 0: On the CLI of each standby SD-WAN Manager node, run the no shutdown command.

    Example:

    Active-vManage# config
    Active-vManage(config)# vpn 0 interface interface-name
    Active-vManage(config-interface)# no shutdown
    Active-vManage(config-interface)# commit and-quit
  2. Activate the tunnel interface in VPN 0: On the CLI of each standby SD-WAN Manager node, run the tunnel-interface command.

    Example:

    Active-vManage# config
    Active-vManage(config)# vpn 0 interface interface-name
    Active-vManage(config-interface)# tunnel-interface
    Active-vManage(config-interface)# commit and-quit
5.

Add each standby SD-WAN Manager node to the overlay network.

  1. From the SD-WAN Manager menu, choose Configuration > > Devices.

  2. Click Controllers.

  3. For a Cisco SD-WAN Validator, click ... and click Edit.

  1. In the Edit dialog box, enter the following details of the Cisco SD-WAN Validator: WAN transport IP address, username, and password.

  2. Repeat Step 5c and Step 5d for every Cisco SD-WAN Validator.

6.

Disconnect the active SD-WAN Manager nodes from the overlay network using one of the following two methods.

  1. Shut down the transport interface in VPN 0: On the CLI of each active SD-WAN Manager node, run the shutdown command.

    Example:

    Active-vManage# config
    Active-vManage(config)# vpn 0 interface interface-name
    Active-vManage(config-interface)# shutdown
    Active-vManage(config-interface)# commit and-quit
  2. Deactivate the tunnel interface in VPN 0: On the CLI of each active SD-WAN Manager node, run the no tunnel-interface command.

    Example:

    Active-vManage# config
    Active-vManage(config)# vpn 0 interface interface-name
    Active-vManage(config-interface)# no tunnel-interface
    Active-vManage(config-interface)# commit and-quit
7.

From the standby SD-WAN Manager, send the updated controller and device list to the Cisco SD-WAN Validator.

  1. From the SD-WAN Manager menu, choose Configuration > Certificates .

  2. Click Controllers.

  3. Click Send to Validator.

    Wait for the configuration task to complete. When the task is complete,

    • The standby SD-WAN Manager nodes become the active SD-WAN Manager nodes.

    • The previously active SD-WAN Manager nodes are no longer part of the overlay network.

    • The active SD-WAN Manager nodes have the configuration from the most recent configuration database backup.

    • Every controller establishes connection with the other controllers in the network.

  4. Click WAN Edge List.

  5. Click Send to Controllers.

8.

Verify configuration and connectivity.

  1. Verify that policies, templates, and the controller and WAN edge device lists are intact.

  2. Verify valid SD-WAN Manager nodes:

    1. Log in to the CLI of each Cisco SD-WAN Validator and run the show orchestrator valid-vmanage-id command.

    2. Log in to the CLI of a Cisco vEdge Cloud router and run the show control valid-vmanage-id command.

    3. In the command output, verify that the chassis numbers of the active and previously active SD-WAN Manager nodes are listed.

    4. Check whether the device is connected to the active SD-WAN Manager nodes and the Cisco Catalyst SD-WAN Controller.

9.

Invalidate the previously active SD-WAN Manager nodes.

  1. From the SD-WAN Manager menu, choose Configuration > Certificates.

  2. Click Controllers.

  3. For each previously active SD-WAN Manager node, click ... and click Invalidate.

  1. The previously active SD-WAN Manager is the certificate issuer for the cloud WAN edge devices. The active SD-WAN Manager issues certificates to the cloud WAN edge devices only after the previously active SD-WAN Manager nodes are invalidated.

  2. After you invalidate the SD-WAN Manager nodes, the nodes cannot be reused without performing additional steps that are beyond the scope of this document.

  3. When you invalidate the previously active SD-WAN Manager nodes, SD-WAN Manager marks the nodes as invalid and sends an update to all controllers. However, SD-WAN Manager does not send an updated list of valid SD-WAN Manager UUIDs to Cisco SD-WAN Validator immediately because the previously active SD-WAN Manager is the CA for the cloud WAN edge devices. So, the output of the show orchestrator valid-vmanage-id command on a Cisco SD-WAN Validator includes the UUIDs of the invalidated SD-WAN Manager nodes.

  4. SD-WAN Manager has a scheduled task that runs every 24 hours and checks to see if all the cloud WAN edges have been moved to the active SD-WAN Manager. SD-WAN Manager sends the updated list of valid SD-WAN Manager UUIDs to Cisco SD-WAN Validator only after the cloud WAN edge devices have been moved to the active SD-WAN Manager. After this list is received, the output of the show orchestrator valid-vmanage-id command on a Cisco SD-WAN Validator does not include the UUIDs of the invalidated SD-WAN Manager nodes.

What to do next

To verify SD-WAN Manager nodes, see Verify the valid SD-WAN Manager nodes.