Provisioning

Getting Access to Cloud Hosted Controllers

Cisco managed Cloud Hosted controllers are by default closed for management access. Cisco does not allow access to 0.0.0.0/0 to the Cloud Hosted Cisco Catalyst SD-WAN Controllers for security reasons. It is expected that you have specific public IP prefixes within your enterprise VPN that you access from and hence only those will be allowed to be opened for access. You can restrict access by requesting to allow only https and ssh to be on the allowed list, for your given source IP prefixes.

Cloud-hosted controllers have private IP addresses on their interfaces. Each private IP address has a 1:1 NAT mapped to a public IP address on the cloud. These IP addresses do not change irrespective of whether the interface is configured to use static IP or DHCP. The IP addresses only change when the instance is recovered or replaced.

The allowed-list is applied to all the network interfaces of all the controllers that have public IP addresses.

Update Inbound Rules

You can update the allowed-list applied to your cloud-hosted controller set based on the overlay type.

  1. Shared tenant overlay: To update or view the allowed-list applied to your cloud-hosted controller set, open a case with Cisco TAC support.

    You can request support for the following:

    • Provide upto 5 IP prefixes to be allowed on the access-list

    • Allow only https access to the IP prefixes for the web login to the Cisco SD-WAN Manager portal

  2. Dedicated Overlay: To enable Cisco-hosted, cloud-based, single tenant dedicated controllers to add, delete, or modify cloud security group allowed-lists, use one of the following options:

    • You can login into the Cisco Catalyst SD-WAN Portal at https://ssp.sdwan.cisco.com and manage the access-list. You need to be the Cisco PNP Smart Account admin for the Smart Account where the overlay controller profile is based.

    • You can provide up to 200 IP prefixes to be allowed on the access-list.

    • You can open a Cisco TAC support case and provide the following information:

      • Overlay/VA name

      • Cisco SD-WAN Manager IP/FQDN

      • IP address

      • Specify whether to mark an IP address as allowed for all traffic or selected traffic (for example https, SSH, and so on).

Only the Smart Account administrator can access the Cisco Catalyst SD-WAN Portal which is used to view and perform operational tasks related to a customer's hosted-controller infrastructure, such as viewing the controllers’ IP addresses and modifying the controllers' IP access lists. To disable SA administrator privileges for users, go to the Manage Smart Account section in Cisco Software Central, and remove the users as Smart Account administrators. Alternatively, use the IDP (identity provider) onboarding feature to grant trusted users access to the Cisco Catalyst SD-WAN Portal.

Cloud-hosted SD-WAN Control Components: Interfaces and access

Cloud-hosted SD-WAN Control Components: Interfaces

Network interface allocation and configuration for SD-WAN Control Components

For SD-WAN Manager instances, Cisco allocates three network interfaces:

  • eth0

  • eth1

  • eth2

For SD-WAN Validator and SD-WAN Controller instances, Cisco allocates two network interfaces:

  • eth0

  • eth1

The network interfaces are assigned with public and private IP addresses, which are static. However, they may change when an SD-WAN Control Component instance is replaced or moved to a new region.


Note


You can view the static IP address using the Catalyst SD-WAN Portal, from Overlay Details > Controller view > Private IP.


Configuring the interfaces

This table summarizes the recommended configurations for each interface.

Table 1. Interface configuration

Control Component

Network interface 1 (eth0)

Network interface 2 (eth1)

Network interface 3 (eth2)

Management access

Control access by nodes in fabric

Communication among SD-WAN Manager instances in a cluster; SD-AVC component functionality

SD-WAN Manager

Private IP:

Static, assigned by Cisco. (See note 1.)

Public IP:

Static, assigned by Cisco.

1-to-1 NAT mapping to private IP.

Configuration requirements:

  • VPN 512

  • Non-tunnel interface

  • Configure as a DHCP client (See note 2.)

Private IP:

Static, assigned by Cisco.

Public IP:

Static, assigned by Cisco.

1-to-1 NAT mapping to private IP.

Configuration requirements:

  • VPN 0

  • Tunnel interface

  • Configure as a DHCP client (See note 2.)

Private IP:

Static, assigned by Cisco.

Public IP:

None assigned.

Configuration requirements:

  • VPN 0

  • Non-tunnel interface

  • Configure with a static IP. Use private IP address that Cisco has assigned to the interface.

    View the static IP address using the Catalyst SD-WAN Portal, from Overlay Details > Controller view > Private IP.

SD-WAN Validator

Same as for SD-WAN Manager.

Same as for SD-WAN Manager.

Not applicable

SD-WAN Controller

Same as for SD-WAN Manager.

Same as for SD-WAN Manager.

Not applicable

Note 1: Accessible by a cloud gateway deployed using the Catalyst SD-WAN Portal. For details, see the section about custom IP prefixes for cloud-hosted SD-WAN Control Components in the Cisco Catalyst SD-WAN CloudOps guide.

Note 2: Even with DHCP configured for IP assignment, the interface always receives the same private IP on every DHCP renewal.

Cloud-hosted SD-WAN Control Components: Access

Edge device access to SD-WAN Control Components

WAN edge devices connect to SD-WAN Control Components only using the VPN 0 tunnel interface of the SD-WAN Control Components.

Edge device access to SD-WAN Control Components

Edge devices communicate with SD-WAN Control Components by transport layer security (TLS) or datagram transport layer security (DTLS) ports.

If you have an on-premises firewall, you can configure your firewall either

  • to enable traffic on any IP (using 0.0.0.0) for these specific TLS or DTLS ports, or

  • to enable traffic to the current public IP addresses of the cloud-based SD-WAN Controllers.


    Note


    You can find the assigned public IP on the Catalyst SD-WAN Portal, in Overlay Details > Controller view > Public IP.


For more information about TLS and DTLS ports, see Ports Used by Cisco Catalyst SD-WAN Devices Running Multiple vCPUs in the Cisco Catalyst SD-WAN Getting Started Guide.

Management access to SD-WAN Manager

Users connect to SD-WAN Manager, for management access, using fully qualified domain names (FQDNs) mapped to the VPN 512 public IP.


If the overlay is provisioned with a 3-node or 6-node SD-WAN Manager cluster, the FQDN resolves to the public IP addresses of all of the SD-WAN Manager instances.

HTTPS access for SD-WAN Manager

HTTP/HTTPs access is available only for SD-WAN Manager, not for other SD-WAN Control Components.

Domain names for SD-WAN Manager and SD-WAN Validator

In a Cloud-hosted SD-WAN environment, Cisco assigns a domain name only to SD-WAN Manager and SD-WAN Validator for cloud hosting.

Access SD-WAN Validator by domain name

When configuring nodes in the SD-WAN fabric, do not configure access to SD-WAN Validator by IP address. Use the FQDN of the SD-WAN Validator instead. This will ensure continued reliable operation in case the SD-WAN Validator IP addresses change or more SD-WAN Validators are added to the fabric.

DNS server

We recommend configuring a DNS server accessible in VPN 0 for each node in the fabric, including hardware edge devices, software edge devices, and the SD-WAN Control Components.

Example:

vpn 0 
   dns 208.67.222.222 primary
   dns 208.67.220.220 secondary

Examples of correct and incorrect configurations for access to SD-WAN Validator

These examples show how to configure access to SD-WAN Validator on other nodes in the network, such as SD-WAN Manager, SD-WAN Controllers, and edge devices.

Correct

This configuration is correct because it uses the SD-WAN Validator domain name:

system
   vbond validator-domain-name

Incorrect

This configuration is incorrect because it uses a static IP address rather than a domain name:

system
   vbond 10.1.1.1

Note


You can view the FQDN of the SD-WAN Validator for your fabric using the Catalyst SD-WAN Portal. Open Overlay Details > Description > vBond DNS. Note that vBond may be replaced by SD-WAN Validator.


Example of correct and incorrect configurations of VPN 0 for access to SD-WAN Validator

These examples show the VPN 0 configuration to enable access to SD-WAN Validator on other nodes in the network, such as SD-WAN Manager, SD-WAN Controllers, and edge devices.

Correct

This configuration is correct because it configures a DNS server that can resolve the SD-WAN Validator domain name.

vpn 0
   dns dns-server-ip primary

Incorrect

This configuration is incorrect because it uses a static host IP for SD-WAN Validator:

vpn 0
   ip host validator-domain-name 10.1.1.1

Note


You can view the FQDN of the SD-WAN Validator for your fabric using the Catalyst SD-WAN Portal. Open Overlay Details > Description > vBond DNS. Note that vBond may be replaced by SD-WAN Validator.


Custom IP Prefixes for Cloud Hosted Controllers


Note


Custom IP prefixes are applicable only if you use Cisco-hosted, cloud-based, dedicated single tenant controllers. These are not applicable for shared tenant overlays.


There are certain use-cases, where you may need custom network prefix-based IPs on the cloud controller interfaces for management access and control. For example,

  • To access the management VPN 512 of Cisco SD-WAN Manager and Cisco SD-WAN Validator or Cisco SD-WAN Controller devices over Cisco Catalyst SD-WAN tunnel with AAA or TACACS based authentication.

  • To send syslog from Cisco SD-WAN Manager over VPN 512 to a syslog server over Cisco Catalyst SD-WAN tunnel.

Figure 1. AAA TACAS

By default, the Cisco-managed cloud hosted controllers are deployed with 10.0.0.0/16 based subnets, including the VPN 512 subnet. If you add the cloud Cisco Catalyst SD-WAN and bring the VPN 512 subnet as a reachable subnet within your fabric, it might conflict with an existing subnet.

In such cases, you need to share a /24 prefix for each of the two regions of deployment of controllers. These IP prefixes are used to create the controllers, and the subnets are then configured to be available within the Cisco Catalyst SD-WAN fabric.

Request for Cloud Gateways for Post Overlay Provisioning:

Open a case for CloudOps at TAC-CSOne with the following details:

  1. To enable AAA or TACACs, you need to provide IP prefixes, unused within your existing fabric, that you can use to create the controllers (original controllers are shutdown, snapshotted, and cloned back).

    Each region in which the controllers are set up has one /24 Cisco Catalyst SD-WAN fabric wise unique custom subnet. Each overlay has two regions, so we need two subnets.

  2. Admin credentials to the Cisco SD-WAN Validator, Cisco SD-WAN Controller and Cisco SD-WAN Manager devices.

    You can provide credentials at the start of the actual change window.

  3. You can schedule eight-hour maintenance window after the preapproval and prechecks completed by CloudOps engineer.

  4. Enable DNS for Cisco SD-WAN Validator and configure all the controllers prior to start of the process.

  5. Ensure that GR is set to default of 12 hours or more on Cisco Catalyst SD-WAN or Cisco SD-WAN Controller devices.

  6. Reserve two available Cloud Cisco Catalyst SD-WAN UUIDs through PNP and attach to Cisco SD-WAN Manager.

  7. Supported only for Single Tenant Single Node Cisco SD-WAN Manager overlays and Single Tenant Cluster Node Cisco SD-WAN Manager overlays for provisioned controllers, and all new to-be-provisioned controller sets. This feature is not supported for Cisco Multi-tenant Cisco SD-WAN Manager cluster overlay.

  8. It is recommended that Cisco SD-WAN Manager have templates attached to Cisco SD-WAN Validator, Cisco SD-WAN Controller, and if existing cloud Cisco Catalyst SD-WAN devices from Cisco.

Configuration of Cloud Gateways Post Cisco Provisioning

  1. Once Cisco CloudOps has completed the provisioning of the cloud gateways next to the cloud hosted controllers, CloudOps shares the public & private IP assignments for each cloud gateway to the customer. They are in the format (VPN 512, VPN 0, VPN X).

    Cisco CloudOps will share the credentials for the newly provisioned cloud gateways.

  2. The cloud gateways have their VPN 512 & VPN X interfaces in the same subnet as the VPN 512 of the controllers in that region.

    The cloud gateways provisioned by the Cisco CloudOps are specifically for the AAA/TACACS purpose and always created in the above network layout format.

    If there are any reachability issues to the cloud gateway, the issue generally lies with the interface IP or route configurations in the cloud gateway.

  3. Also, note that the public & private IPs are 1:1 NAT'd and assigned to the cloud gateway interfaces. The gateway interface itself may be configured with dhcp, but it will always get the same IP from Cloud.

    For VPN X interfaces, you will need to configure the static IP, exactly as the one shared by Cisco CloudOps.

    Random IPs within the subnet cannot be used.

  4. The cloud gateways are subject to the same Inbound allowed access-list as the controllers, as they are provisioned in the same unique environment per overlay.

    You must login via SSH to the gateway public IPs and the credentials provided.

  5. You must now configure the new cloud gateways with the necessary configurations. For example, site-id, system IP, organization name, Cisco SD-WAN Validator DNS or IP, and so on.

  6. If you are using Enterprise root-ca, then you must upload and install the same on the cloud gateways as well.

  7. You may configure AAA/TACACS on the Cisco SD-WAN Manager with auth-fallback to local with local having the viptelatac/ciscotacro/ciscotacrw user enabled. This allows Cisco support to login and troubleshoot issues when required.

  8. You would need to acquire an unused cloud gateway UUID from the device list of the Cisco SD-WAN Manager, one per cloud gateway provisioned.

    If you don't have any cloud gateway UUID available in the WAN Edge Device list on your Cisco SD-WAN Manager, then you may need to login into the Cisco PNP portal, on the overlay's associated Smart Account and Virtual Account, and Add Software Devices (VEDGE-CLOUD-DNA) and then Sync Smart Account on the Cisco SD-WAN Manager.

  9. You must then activate the UUID on the cloud gateways to allow them to be authenticated by the Cisco SD-WAN Manager and join the Cisco Catalyst SD-WAN fabric.

  10. You must configure the controllers' (Cisco SD-WAN Manager, Cisco SD-WAN Validator, Cisco SD-WAN Controller) VPN 512 with a specific static route for customer's Enterprise subnets (from customers admin team intend to access the controllers for management) to point to the cloud gateway's VPN X static IP.

  11. For an overlay hosted by Cisco on Azure, please open a Cisco TAC case and provide the specific enterprise subnet prefixes, from where the connectivity to the VPN 512 of the controllers is required.

    The Azure subnet default gateway is the defacto gateway even if you configure the gateway service VPN IP to be the gateway for your enterprise subnets. Hence in addition to your configuration on VPN 512 on the controllers, there is additional configuration needed on the Azure side. Cisco will help apply an Azure Route Table (RT) entry for each of the necessary Enterprise subnets and also enable IP forwarding on the cloud gateway interfaces.