Navigator

The document covers the following topics:

Audience

The audience for this guide comprises, but is not limited to, system architects, network/compute/systems engineers, field consultants, Cisco Solution Support specialists, and customers.

Readers should be familiar with networking protocols and IP Routing, basic network security and QoS, and be exposed to server virtualization using hypervisor and the Cisco Connected Communities Infrastructure (CCI) Solution architecture, which is described in the Cisco Connected Communities Infrastructure CVD Solution Design Guide at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/solutions/Verticals/CCI/CCI/DG/cci-dg.html

Document Objective and Scope

This implementation guide provides a comprehensive details of Cisco Connected Communities Infrastructure (CCI) horizontal network infrastructure implementation leveraging the Cisco Digital Network Architecture Center (Cisco DNA Center) Software Defined Access (SD-Access) Fabric. The CCI solution horizontal access network infrastructure implementation is based on the Cisco Software Defined Access Deployment Guide that can be found at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/sda-sdg-2019oct.html

This document also provides details about implementing CCI vertical use cases such as cities safety and security and Cisco Smart Street Lighting and CCI overlay network use cases such as transportation/roadways intersection. While the implementation steps detailed in this document should be used as a reference for deploying other CCI vertical use cases, the detailed implementation of specific vertical use cases on the CCI network that are not validated in this solution is beyond the scope of this document.

This document covers example network underlay routing configurations and Multiprotocol Label Switching (MPLS) network backhaul configuration for the deployment models and network topologies validated in the solution. Detailed implementation of network routing protocols and configuring MPLS network backhaul is beyond the scope of this document.

New Capabilities in CCI Release 2.1

■Cisco Ultra Reliable Wireless Backhaul (CURWB) for CCI backhaul and wireless access networks

■Enhanced Ethernet Access Ring & Provisioning

–IE-3300 10G Access Ring in CCI PoPs

–Daisy Chaining Automation of Extended and Policy Extended Nodes using Cisco DNA Center

–REP Ring Automation using Cisco DNA Center

■Cisco CyberVision OT Device and Flow Detection

–CyberVision Sensor deployment on IE-3400, IE-3300 10G and IR-1101 Platform

–OT Device and Protocols (DNP3 and MODBUS) Flow Detection using Cisco Cyber Vision Center

■Enhanced End-to-End QoS design on IE3400 and IE3300 10G

■Enhanced Remote Point-of-Presence (RPoP) Management design

–IR-1800 as RPoP gateway with multi-service and macro-segmentation at RPoP

–RPoP Management Design using Cisco DNA Center and Cisco IoT Operations Dashboard (IoTOD)

This document also provides implementation details for overlaying CCI vertical use cases like Cities Safety and Security, Cisco Smart Street Lighting, SCADA use cases, LoRaWAN Lighting, and Rail and Roadways intersection on the CCI network. It is recommended to implement CCI network and vertical use cases, as depicted in Figure 1, which shows the flow of the material in this implementation guide:

Figure 1 CCI Solution Implementation Flow

The document addresses the implementation of the following CCI network horizontal and vertical use cases:

■CCI Underlay Network implementation for basic network (Layer 3) IP forwarding and connectivity.

■Implementation of shared services like Cisco DNA Center, Identity Service Engine (ISE), Cisco Wireless LAN Controller (WLC) and Cisco Prime Infrastructure (PI), Cisco CyberVision Center, DHCP, and DNS servers, as well as other shared IoT devices management applications such as Field Network Director (FND).

■Configuring Cisco SD-Access Fabric Site (Point-of-Presence aka PoP) as overlay network and Interconnection of the Fabric Sites leveraging Cisco DNA Center.

■Implementation of Cisco Industrial Ethernet switches—Cisco Industrial Ethernet (IE) 4000, Cisco Industrial Ethernet (IE) 5000, Cisco Catalyst Industrial Ethernet and Embedded Services 3300 and 3400 series switches—as fabric extended nodes, policy extended nodes, in Ethernet access network rings.

■Implementation of Cisco Unified Wireless Network (CUWN) Wi-Fi Mesh and SD Access Wireless Wi-Fi access networks.

■Implementation of headquarter site data center applications for vertical use cases and services on the fabric overlay network. It covers Cities Safety and Security, LoRaWAN, ThingPark Enterprise, and applications such as Certificate Authority (CA) services needed for Cisco Smart Street Lighting solution use cases along with Public Wi-Fi use cases.

■Deployment details for LoRaWAN in Remote PoP.

■Deployment details for Remote PoP over cellular network backhaul for multi-services and macro-segmentation.

■Deployment details for LoRaWAN in Remote PoP (optional).

■Implementation of end-to-end network security, which covers macro- and micro-segmentation of CCI networks using Virtual Networks (VNs) and Scalable Group Tags (SGT) and Scalable Group Access Control Lists (SGACL), network devices and endpoints security, and network firewall implementation in the DMZ and Stealthwatch.

■End-to-end network QoS implementation for traffic classification, prioritization, queuing, and policing.

■Implementation of multicast network forwarding in CCI. Enabling multicast in CCI is optional as it is needed if you want to implement any vertical use case which requires multicast traffic forwarding in CCI.

■Implementation of SCADA communication use cases with CR-Mesh network.

■Implementation of LoRaWAN-based FlashNet lighting use case.

■Axis camera Day 0 onboarding and Day N management in CCI.

Deployment Topology Diagrams

This section describes the different deployment network topologies that have been validated in the CCI Solution Implementation.

High Level Solution Validation Topology

Figure 2 depicts the CCI high level validation topology, including the endpoints for vertical use cases validated in this solution implementation:

Figure 2 CCI High Level Solution Validation Topology

The multiple layers of topology include:

1. Internet Cloud and Data Center layer, which includes:

–Network connectivity to Demilitarized Zone (DMZ) to access Internet/Applications on the cloud.

–A Headquarter or Data Center Site (HQ/DC Site) aka Application Servers Site consisting of:

• Application servers for hosting vertical specific applications needed on the CCI horizontal network
• Shared services like Cisco DNA Center, ISE, DHCP, DNS servers, Stealthwatch Management Console (SMC) and Flow Collector (SFC), Cisco Cybervision Center, CURWB Global gateway along with other applications that are common to all vertical use cases like FND, Axis Device Manager (ADM) for both Cities and Transportation/Roadways.

2. Network backhaul layer interconnects PoPs and the Internet Cloud/Data Center layer with either the private enterprise Ethernet network or private MPLS network backhaul. Remote PoPs connect to the CCI network via cellular or private/public network backhaul.

3. Aggregation layer aggregates all PoPs traffic to the upper layers.

4. Ethernet access ring provides network access to gateways/endpoints validated in the solution.

5. Internet of Things (IoT) gateways and endpoints layer includes Access Points (AP) for Wi-Fi access, Curwb Access Points for Rail, access gateways based on access technologies (such as DSRC, LoRaWAN, and CR-Mesh) and their endpoints validated in this solution.

Solution Validation Topologies

Two deployment models of the CCI solution have been validated during this implementation:

1. CCI network deployment topology with Cisco SD-Access Transit, henceforth referred to as SDA Transit interconnection of all sites. The validation is done over the Enterprise Ethernet Network backhaul (using Cisco Catalyst 9500 switches as the network core). This topology is depicted in Figure 3.

2. CCI network deployment topology with IP Transit interconnection of PoPs and headquarter sites with validation done over Private MPLS network backhaul, as shown in Figure 4.

Figure 3 SD-Access Transit with Enterprise Backhaul Network Topology

Note : In Figure 3, the PoP1 with C9500 SVL is also supported to connect IE switches to just the nearest Catalyst 9500 stack member. This could be likely when there is insufficient fiber pairs between the two physical locations where each stack member is housed, however in this case also a Port Channel is still used with single member link automated by DNA Center.

Figure 4 IP Transit with MPLS Backhaul Network Topology

Network topologies validated in this CVD include FlexVPN tunnels that are configured for securing the communication between the Cisco 1240 Connected Grid Router and the HER in Cisco Smart Street Lighting solution use cases implemented on the CCI network.

For more details about fabric device roles (B-Border, CP-Control Plane, E-Edge, T-Transit, X-Extended Node) in the network topology, refer to the Cisco Connected Communities Infrastructure CVD Solution Design Guide.

IP Addressing

This section captures the example IP addressing prefixes used in the solution lab topology, as shown in Figure 3.

Note: The IP addresses captured in this section are example IP addressing used only for the solution validation as internal sub-networks in the CVD lab. It provides a reference for selecting subnets for the solution implementation. It is recommended to choose private network prefixes/IP addressing scheme depending on the solution deployment and devices connected to the CCI network.

Addressing Convention followed in the IP Subnet Selection

Four prefixes are used in the network subnet for the network topology (where X is the site ID chosen for a PoP site/ transit site and the underlay network devices, if any).

■192.0.X. YY—Devices Loopback IP addresses prefix

■172.10.X. YY—Virtual Network (VN) subnets prefix

■192.168.X. YY—Fabric Overlay Border Handoff Network prefix

■192.100.X. YY—Fabric Extended Nodes IP Pool prefix

Note: Refer to IP Addressing of Solution Components for more details about IP addresses, including IP addresses used for underlay network connectivity for the network topologies, as shown in Figure 3, Figure 4, and Figure 19.

Solution Virtual Networks and Scalable Groups

In the CCI implementation, a Virtual Network (VN) is used for a vertical service. This macro-segmentation provides complete separation between services. One VN can communicate with another only by leaking routes between the VRF at the fusion router. Table 2 provides an example list of VNs used in the CCI solution validation.

Example VNs for the Cities and Roadways applications include Safety and Security, Cisco Smart Street Lighting, Iteris, Schneider, and LoRaWAN. Further micro-segmentation within a virtual network is possible by using Scalable Group Tags (SGT). Table 2 also provides an example list of SGTs for micro-segmentation of the VN.

Configuring Enterprise Ethernet Network Underlay

Ethernet as a backhaul, is one of the enterprise networks backhaul deployment methods that can be implemented in CCI horizontal network, as shown in Figure 3. The underlay network connectivity between shared services and all devices in each PoP site (including HQ/DC site) is provided through the backhaul network. The underlay network configuration is a basic network connectivity prerequisite for implementing the fabric overlay network for the CCI solution using the Cisco DNA Center.

Many protocols are available to configure IP routing, but in this implementation EIGRP is used as an example routing protocol for configuring underlay network connectivity and IP routing across PoP Sites and shared services. Cisco DNA Center uses Border Gateway Protocol (BGP) as the routing protocol when a border node connects to an IP transit, which means the configuration co-exists with the underlay configuration.

Configuring Cisco Catalyst 9300 Switch Stack for Fabric-in-a-Box

In the CCI Solution, all fabric/PoP sites leverage the Cisco Catalyst 9300 switch stack as an aggregation/distribution layer switch for aggregating traffic from access rings. Switch stack ensures redundancy. A stack of Cisco Catalyst 9300 switches appears to the operator and the rest of the network as one single switch, making it easier to manage and configure. Newer switch models add stateful failover capability, providing similar behavior as a chassis with dual supervisors in case of a failure or the need to update software on the stack.

Cisco Catalyst 9300 switch stack configuration is the initial step for provisioning a PoP site network (along with redundancy) for the access rings network and backhaul network connectivity network in the CCI topology. Refer to the following URL for configuring Cisco Catalyst 9300 switches in a stack:

■ https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/software/release/17-6/configuration_guide/stck_mgr_ha/b_176_stck_mgr_ha_9300_cg/managing_switch_stacks.html

Configuring Cisco Catalyst 9500 Switches StackWise Virtual for Fabric-in-a-Box

Alternatively, Cisco Catalyst 9500 Series switches can be used as PoP sites aggregation/distribution layer switch for aggregating traffic from access rings in CCI. Cisco Catalyst 9500 platform StackWise Virtual (SVL) technology allows the clustering of two physical switches, that are geographically separated, together into a single logical entity. The two switches operate as one; they share the same configuration and forwarding state. This technology allows for enhancements in all areas of network design, including high availability, scalability, management, and maintenance.

Cisco Catalyst 9500 switch SVL configuration is the initial step for provisioning a PoP site network (along with redundancy) for the access rings network and backhaul network connectivity network in the CCI topology.

StackWise Virtual domain is elected as the central management point for the entire system when accessed via management IP or console. The switch acting as the single management point is referred to as the SV active switch. The peer chassis is referred to as the SV standby switch. The SV standby switch is also considered a hot-standby switch, since it is ready to become the active switch and it takes over all functions of active switch when active switch fails.

When the Catalyst 9500 SVL is used in the role of the Fabric-in-a-Box (FiaB) (Border + Control Plane + Edge), the connection to a Transit Site (for example, SD Access Transit switches) must be done with interfaces configured as a switchport trunk. A Switched Virtual Interface (SVI) is used for the Layer 3 configuration.

Conversion to StackWise Virtual Mode

Refer to the section “How to Configure Cisco StackWise Virtual” for configuring Cisco Catalyst 9500 switches in a SVL Mode at the following URL:

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-6/configuration_guide/ha/b_176_ha_9500_cg/configuring_cisco_stackwise_virtual.html

Configure Layer 3 on Ethernet Backhaul Network

Cisco Catalyst 9500 switches are used to provide the Ethernet network backhaul for interconnecting PoP sites and Shared Services, and Data Center applications in the HQ PoP Site, as shown in Figure 3. The following configuration provides an example configuration to enable Cisco Catalyst 9500 switches for the underlay network routing (Layer 3) for the network topology, as shown in Figure 3.

Configure the Layer 3 interface for the underlay network on Cisco Catalyst 9500 switches:

Example Interfaces Configuration on Cisco Catalyst 9500-1 (Transit Site)

1. Loopback interface is configured on the device for Cisco DNA Center discovery:

2. Configure an interface as a trunk to a PoP Site Cisco Catalyst 9300 stack:

3. Configure an SVI interface (example: VLAN 200) for underlay reachability between Fusion Router 1 and Cisco Catalyst 9300 stack, which is FiaB:

4. Configure Layer 3 Port Channel between C9500 switches in Transit sites. On 9500-1:

5. EIGRP routing protocol is configured between fusion routers and Cisco Catalyst 9300 stack network devices to form neighbors:

Note : EIGRP is chosen as an example routing protocol for the underlay network routing configuration. Refer to the Cisco Connected Communities Infrastructure Design Guide for more details on recommended routing protocol for the underlay network routing configuration.

Example Interfaces Configuration on 9500-2 (Transit Site)

1. Loopback is configured on the device for Cisco DNA Center discovery:

2. Configure an interface on 9500-2 as a trunk to the Cisco Catalyst 9300 stack:

3. Configure an SVI interface (for example, VLAN201) for underlay reachability between Fusion Router 2 and the Cisco Catalyst 9300 stack which is FiaB:

4. Configure Layer 3 Port Channel between 9500 switches in Transit sites. On 9500-1:

5. EIGRP routing configuration between fusion routers and Cisco Catalyst 9300 stack network devices to form neighbors:

Example Interfaces Configuration on 9500-2 (Transit Site)

1. Loopback is configured on the device for Cisco DNA Center discovery:

2. Configure an interface on 9500-2 as a trunk to the Cisco Catalyst 9300 stack:

3. Configure an SVI interface (Example VLAN201) for underlay reachability between Fusion Router 2 and the Cisco Catalyst 9300 stack which is FiaB:

Configure Layer 3 Port Channel between C9500 switches in Transit sites. On C9500-2:

interface TenGigabitEthernet1/0/1
no switchport
no ip address
channel-group 13 mode active
end
!
interface TenGigabitEthernet1/0/2
no switchport
no ip address
channel-group 13 mode active
end

4. EIGRP routing configuration between fusion routers and Cisco Catalyst 9300 stack network devices to form neighbors:

Configure Layer 3 on Fabric-in-a-Box

An example Layer 3 routing configuration on the PoP site network device Cisco Catalyst 9300 stack or 9500 SVL to reach fusion routers and shared services network:

1. Loopback interface on the Cisco Catalyst 9300 stack for Cisco DNA Center discovery:

2. Configure interfaces on the Cisco Catalyst 9300 stack as trunk ports to fusion routers:

3. Configure an SVI interfaces (example: VLAN200 and VLAN201) on the Cisco Catalyst 9300 stack to reach fusion routers:

4. Configure EIGRP neighbors between Cisco Catalyst 9300 Stack and Cisco Catalyst 9500 switches (fusion routers):

Note: The above are the example configurations for the PoP1 site, as shown in Figure 3. The same has to be applied for all the PoP sites, including the HQ/DC site, to reach the shared services network so that devices can be successfully discovered in the Cisco DNA Center.

Configuring Shared Services Network Connectivity

For all the network devices in a PoP site and fusion routers to reach the shared services network, configure the basic underlay routing between the fusion routers and shared services network. Refer to Figure 3 for the physical topology between the fusion router, Nexus, and the shared services network.

1. A pair of Nexus 5672UP switches in the HQ/DC site connecting to application servers is used for connecting the Cisco DNA Center appliance and the Cisco UCS server where other shared services applications are hosted. The following configuration provides an example configuration (Layer 3) on the Nexus switches for configuring the shared services network to Cisco Catalyst 9500 switches as fusion routers, as shown in Figure 3.

Nexus Switch-1 Configurations

a. Configure an SVI interface (example: shared service VLAN1000) in Nexus-1 to reach the shared services network:

b. Configure interface for connectivity to Cisco DNA Center appliance enterprise network interface:

c. Configure interface for connectivity to CSR1KV:

Nexus Switch-2 Configurations

a. Configure an SVI interface (VLAN1000) in Nexus-2 to reach the shared services network:

b. Configure Nexus-2 interface for connectivity to the CSR1KV:

2. Configure Cisco CSR1000V (fusion routers) to reach the shared services network.

a. For the shared services network (10.10.100.X), configure sub interfaces to reach Cisco DNA Center, DHCP, DNS, and ISE.

On Fusion Router 1:

On Fusion Router 2:

b. Cisco CSR1000v routers are configured as default routers for the shared services subnet with Next Hop Redundancy using the HSRP protocol. Configure HSRP to create gateway redundancy between the fusion routers for the shared services subnet. Example HSRP configuration on fusion routers:

On Fusion Router 1:

On Fusion Router 2:

3. Add the shared services network in the underlaying EIGRP routing configuration on both fusion routers, as shown in the example below.

Once the underlay routing configuration is complete for the Catalyst 9300 FiaB and fusion routers, the connectivity to the shared services (Cisco DNA, ISE, DHCP, WLC, Prime, etc.) network must be verified.

Transit Control Plane (C9500-1) IP Routing Verification:

FiaB IP Routing Verification:

Ping Devices in Shared Services:

After successfully verifying the underlay connectivity from the Catalyst 9300 FiaB to the shared services, the edge fabric can start being provisioned.

Configuring Network Underlay for MPLS Backhaul Network

In addition to a Layer 3 enterprise network deployment, an edge fabric site can also be connected to the data center fabric site through an MPLS backhaul network. This network could be deployed by the city operator or a separate service provider. In either case, the fabric border device will act as a customer edge (CE) router and the connecting router in the MPLS core will act as the provider edge (PE) router. For this testing, a Layer 3 Virtual Private Network (L3VPN) was implemented. Explaining the differences in MPLS implementations is outside the scope of this document. This implementation is one of many ways a service provider can separate one customer’s traffic from another.

Many ways exist for configuring a VRF-aware routing protocol between a PE and CE, but, in this implementation, eBGP was used. Cisco DNA Center only supports BGP as the routing protocol when a border node connects to an IP transit, which means the configuration can be combined with the underlay configuration. When the Catalyst 9300 is used in the role of the FiaB (Border + Control Plane + Edge), the connection to the PE must be done with an interface configured as a switchport trunk. An SVI is used for the Layer 3 configuration. For resiliency, another port on a different stack member can be connected to a different PE router.

Example Catalyst 9300 Configuration:

BGP Configuration:

Example Provider Edge Configuration:

VRF Definition:

Physical Interface:

Bridge Domain Interface:

BGP Configuration:

Note : Example VRF configuration is shown above for one VN. The configuration must be repeated if you add more VNs in the network.

Once the routing configuration is on the Catalyst 9300 FiaB and provider edge, connectivity to the shared services (Cisco DNA, ISE, DHCP, etc.) must be verified.

Provider Edge Verification:

FiaB Verification:

■Check Routing:

■Ping devices in shared services:

After successfully verifying the underlay connectivity from the Catalyst 9300 FiaB to the shared services, the edge fabric can start being provisioned.

Backhaul

When using Cisco Ultra-Reliable Wireless Backhaul (CURWB) in the backhaul to connect edge PoPs to the headquarters, it will takes on the role of underlay. Because the links act as invisible wires between the PoPs and the headquarters, they can be used as an SD Transit. However, because they are wireless devices, additional consideration and configuration is needed for deployment. The inherent challenges in an RF environment necessitate, a complete site survey is required before deploying the CURWB radios. Details of the site survey are outside the scope of this document. Using two different RF paths to provide higher throughput and resiliency for each PoP site is recommended. Configuring the radios prior to the physical installation is also recommended.

An example testbed is depicted below.

Figure 5 Multiple Wireless Backhaul Paths

In this deployment, two wireless paths are used to provide higher throughput and resiliency. Each PoP uses a routing protocol supporting Equal Cost Multipath (ECMP) which enables load balancing between the links. The effectiveness of the load balancing is dependent on the type of traffic and the load balancing algorithm chosen in the PoP border switch.

Plug-ins

Plug-ins are the licenses installed on the radios that enable specific features. The plug-ins needed to enable the fixed infrastructure will depend on the model chosen, the throughput needed, and whether the radios are in bridge mode or point-to –multipoint mode. The radios will also require the VLAN plug-in to enable the correct VLAN processing and AES to secure the wireless traffic. Enable MPLS fast failover is enabled by installing the TITAN plug-in.

CURWB Configuration

The radios can be configured in three different ways: 1) through RACER, 2) using the built-in web configuration tool, and 3) using the CLI. RACER and the CLI permit full configuration of all the options compared to the web configuration tool. RACER is the preferred tool for configuration because of the ability to manage all the CURWB radios’ configurations in a single dashboard.

General and Wireless Settings

Each radio is configured to operate in a specific mode based on its role in the network. In this deployment, the radios at the headquarters are configured as Mesh Ends and the radios installed at the PoP sites are Mesh Points. The Mesh End radio is responsible for connecting the mesh network to the LAN connected backbone. Because the radios are configured as part of the network underlay, the management interface on all the Mesh Ends and Mesh Points must be configured in the same subnet. The configured passphrases must also match on the Mesh End and all its associated Mesh Points. This passphrase must be different from the other Mesh End and its Mesh Points, ensuring that the wireless networks are kept separate.

Figure 6 Mesh End Wireless Path A - General

Figure 7 Mesh End Wireless Path B - General

Figure 8 Mesh Point Wireless Path A – General

Figure 9 Mesh Point Wireless Path B – General

The wireless part of the radio is a separate configuration, and each path is configured on a separate non-overlapping frequency as determined by the site survey. Because the radios are operating in Point –to-Multipoint mode, there is the chance that Mesh Points could communicate at the same time causing a collision. The FM3200 can operate in Time Division Multiple Access (TDMA) mode which increases efficiency in the communication by reducing collisions, but the FM3500 can only operate in Carrier Sense Multiple Access/Collision Avoidance (CSMA/CA) mode. To reduce collisions, it is necessary to enable RTS/CTS on the FM3500 Mesh End radios.

Figure 10 Mesh End/Mesh Point – Wireless Radio

Because the Mesh Ends communicate with numerous PoP sites, they are also configured using FluidMAX. This allows the unit configured as “Master” (Mesh End in this case) to dictate the operating frequency to the radio units configured as “Slave” (Mesh Points).

Note: In the Advanced Radio Settings UI shown below, the Primary is called Master and the Secondary is called Slave. This feature cannot be configured in RACER for the FM3500, it can only be configured using the web Configurator, or the CLI.

Advanced Radio Settings

Figure 11 FluidMAX Primary/Master

Figure 12 FluidMAX Secondary/Slave

Multicast

For this deployment, EIGRP was used as the underlay routing protocol which uses the well-known standard reserved multicast address 224.0.0.10. To forward these messages to the other radios, the Mesh Ends must be configured with multicast routes. The below configuration will send below sends the EIGRP update messages to all units in the mesh network.

Figure 13 Multicast

VLAN

Because the radios are all in the underlay network, the management VLAN can be configured as common across all the radios. The other configurable option for VLANs in the radio is the native VLAN. The native VLAN should must be configured the same on the Mesh End and Mesh Points while using the PoP border node to set the desired native VLAN. This ensures that any untagged packets coming into the wireless network do not inadvertently leave the radio with a VLAN tag. In the below examples below, VLAN 145 is used for management and VLAN 555 is used as the native VLAN. Note, VLAN 555 is not being used elsewhere in the network. Note that if the native VLAN is set to 0, any untagged traffic will be dropped.

Figure 14 VLAN Configuration

QoS

QoS can only be enabled through the RACER configuration or CLI, not using the web Configurator. Enableing QoS on the radio and leaving the marking and queueing to the connected switch is recommended. Enable this configuration on all Mesh Ends and Mesh Points. When enabling 802.1P, the CURWB radio will inspects the COS value in the VLAN header as opposed to the DSCP value in the Layer 3 header.

Figure 15 QoS Configuration

Infrastructure Configuration

Headquarters

Using multiple parallel wireless paths will increases throughput and resiliency. Each radio network is therefore treated as a separate network path to a PoP site. In this deployment, wireless path A is assigned to VLAN 200 and wireless path B is assigned to VLAN 201. Each radio is connected to a trunk port that disallows the other PoP VLAN. The MTU is also configured for the maximum size that the radios can pass. The MTU is also required on the SVI because EIGRP sends updates up to the maximum size allowed on the link. VLAN 145 is included to enable management of the radios.

Wireless Path A

Wireless Path B

Each VLAN has an associated SVI for Layer3 reachability.

VLAN 200 and 201 are added to EIGRP to form neighbors with the other PoP sites connected wirelessly.

PoP Site

At each PoP site, the VLAN for each wireless path must be configured. For sites with dual paths, this is VLAN 200 and 201. The interfaces facing the radio must also be set as trunks. When using the 9x00 as the border node, the MTU can be configured system wide for 2044.

Cisco DNA-C also requires a loopback for onboarding and management.

The underlay subnets are then added to the EIGRP process.

Looking at the EIGRP neighbors will confirms the underlay is functioning correctly.

After the underlay network is functional and all required configuration for discovery is in place, the Discovery workflow can be used to onboard the device.

Cisco DNA-C Configuration

Onboarding and provisioning the newly-discovered switch is the same process as a wired switch and requires no special configuration to support the CURWB connection. After provisioned to the fabric site, the border interfaces must be configured if an IP Transit is used. Each interface facing the CURWB radio is used as the External Interface.

Figure 16 Border External Interfaces

Because the PoP switch is connected to the headquarters through Layer2, each VLAN configured for a VN must be unique at the headquarters site.

Figure 17 Border Interface-1 VN Configuration

Figure 18 Border Interface-2 VN Configuration

Through the use of multiple interfaces, the routing protocol can be configured for fast failover and load balancing. Bidirectional Failure Detection (BFD) is configured on the interfaces and within the BGP instance for the VRF associated with the VN. Load balancing is achieved using the maximum-paths command. The routing protocol is dependent on having multiple interfaces to achieve these additions.

Edge PoP External SVI:

Edge PoP BGP Address Family

BFD Neighbor table

IP Routing table with multiple paths

The headquarters core switch needs the complementary configuration on the interfaces and BGP address family configuration. Upon completion, multiple paths will be available for traffic between the Edge PoP and headquarterss which can be used for load balancing and failover.

Cisco DNA Center Installation and Initial Configuration

Cisco DNA Center offers centralized, intuitive management that makes it fast and easy to design, provision, and apply policies across your network environment. The Cisco DNA Center provides a centralized management dashboard for complete control of the CCI horizontal network.

Cisco DNA Center, which is a dedicated hardware appliance powered through a software collection of applications, processes, services, packages, and tools, is the centerpiece for Cisco® Digital Network Architecture (Cisco DNA™). This software provides full automation capabilities for provisioning and change management, reducing operations by minimizing the touch time required to maintain the network.

This section covers the installation and basic network configuration needed on the Cisco DNA Center for accessing its GUI in CCI deployment.

For step-by-step instructions for installing and configuring Cisco DNA Center, refer to the Cisco DNA Center Installation Guide, Release 2.2.3 at the following URLs:

Cisco DNA Center First Generation Appliance Installation Guide

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/install_guide/1stgen/b_cisco_dna_center_install_guide_2_2_3_1stGen.html

Cisco DNA Center Second Generation Appliance Installation Guide

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/install_guide/2ndgen/b_cisco_dna_center_install_guide_2_2_3_2ndGen.html

Preparing Cisco Identity Service Engine for SD-Access

Cisco Identity Services Engine (ISE) is a policy-based access control system that enables the enterprises, Smart Cities, and the like to enforce compliance and infrastructure security. ISE is an integral part of Cisco SD-Access acting as the authentication, authorization, and accounting (AAA) server for devices identity management, access control, and enforcement of access policies on fabric devices.

In the CCI solution, ISE is coupled with the Cisco DNA Center for dynamic mapping of users and devices to scalable groups, which simplifies end-to-end security policy management and enforcement at a greater scale than traditional network policy implementations relying on IP access lists.

ISE Installation and Initial Configuration

A centralized standalone deployment of ISE is configured with the Cisco DNA Center in the shared services network as shown in the network topology that is depicted in Figure 3. ISE can be installed in various ways; OVA deployment of ISE as a virtual machine is used in this implementation. Refer to the URL below for step-by-step instructions on installing ISE:

For ISE v2.4:

■ https://www.cisco.com/c/en/us/td/docs/security/ise/2-4/install_guide/b_ise_InstallationGuide24/b_ise_InstallationGuide24_chapter_011.html

If you prefer to deploy the latest compatible version of ISE, refer the following URL for ISE v3.0 Installation:

■ https://www.cisco.com/c/en/us/td/docs/security/ise/3-0/install_guide/b_ise_InstallationGuide30/b_ise_InstallationGuide30_chapter_3.html

Once the ISE installation is complete, update the Patch 13 on ISE v2.4, which is compatible with Cisco DNA Center SD-Access, by completing the following steps:

1. Download the ISE patch bundle ise-patchbundle-2.4.0.357-Patch13-20080314.SPA.x86_64.tar.gz.

Note: Software downloads from Cisco website requires a registered Cisco Account and Cisco software download access.

2. Log in to the ISE GUI and navigate to Administration-> Maintenance-> Patch Management.

3. Click Install, upload the patch file, and then click Install again. The installation will take about 1 hour and during the time ISE will not be available.

4. To verify the patch is installed successfully, check Patch Management in to see whether the Patch 13 is listed, as shown in Figure 19.

Figure 19 Cisco ISE Patch Installation View

This completes the installation and relevant patch upgrade of ISE compatible with Cisco DNA Center Release 2.3.2.

Note: Refer to the Cisco SD-Access 2.3.2.x Hardware and Software Compatibility Matrix at the following URL for more details: https://www.cisco.com/c/dam/en/us/td/docs/Website/enterprise/sda_compatibility_matrix/index.html

Integrating ISE with Cisco DNA Center

Once ISE installation and basic configuration is complete, it has to be integrated with the Cisco DNA Center. Refer to the section Integrate Cisco ISE with Cisco DNA Center in the Cisco DNA Center Installation Guide Release 2.2.3 at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/install_guide/2ndgen/b_cisco_dna_center_install_guide_2_2_3_2ndGen/m_complete_first_time_setup_2_2_3_2ndgen.html#task_ikj_pg3_sfb

Note: Before integrating ISE with the Cisco DNA Center, ensure that PxGrid services are online on ISE and that the cluster node is up in Cisco DNA Center.

Once integrated with Cisco DNA Center using PxGrid, information sharing between the two platforms is enabled, including device information and group information. This allows the Cisco DNA Center to define policies that are pushed to ISE and then rendered into the network infrastructure by the ISE Policy Service Nodes (PSNs). When integrating the two platforms, a trust is established through mutual certificate authentication. This authentication is completed seamlessly in the background during integration and requires both platforms to have accurate NTP time synchronization.

Configuring DHCP and DNS Services

DHCP Server

A DHCP Server is a network server that automatically provides and assigns IP addresses, default gateways, and other network parameters to client devices. It relies on the standard protocol known as Dynamic Host Configuration Protocol (DHCP) to respond to broadcast queries by clients.

DHCP services can be configured in the network in many ways. In this implementation, a centralized DHCP services in the CCI network shared services, running on a Windows 2016 server is used. This section covers the example DHCP scope and IP pools definition and discusses other scope options that are required for implementing SD-Access in the CCI network.

Refer to the step-by-step instructions on Microsoft Windows Server 2016: DHCP Server Installation & Configuration at the following URL:

■ https://social.technet.microsoft.com/wiki/contents/articles/51170.microsoft-windows-server-2016-dhcp-server-installation-configuration.aspx

After the DHCP server is successfully configured on a Windows 2016 server, create Scopes for all the IP pools configured on the Cisco DNA Center with options 43 (example pools are for extended node and host node pools) in the DHCP server, as shown in Figure 20:

Figure 20 Example IP Scope and Scope Options in CCI Network

For more information on DHCP option 43, refer to the section DHCP Controller Discovery in the Cisco Digital Network Architecture Center User Guide, Release 2.2.3 at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_01101.html?bookSearch=true#id_90877

Domain Name Server

In this implementation, Domain Name Servers (DNS) in the CCI network shared services, running on a Windows 2016 server (co-located on the DHCP server), are used.

Refer to the following URL for step-by-step instructions and configuration of the DNS on the Windows 2016 server for the CCI network:

■ https://www.microsoftpressstore.com/articles/article.aspx?p=2756482

Implementing Field Network Director for CCI

Cisco Field Network Director (FND) is an essential component for IoT solution deployments. FND in CCI provides easier deployment and management of devices such as Field Area Routers (CGR), Connected Grid End Points (CGEs) and IC3000 Industrial Compute Gateway. FND is the critical component of the FAN solution. FND is the one component that interacts with most of the components in the FAN solution.

For information about installing/configuring FND, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/routers/connectedgrid/iot_fnd/install/oracle/iot_fnd_oracle/installation_rpm_new_oracle.html

Note: FND with the Oracle database, which is used in this implementation, is needed for CGR mesh support.

Prerequisites for FND Installation

■In the CCI network, FND OVA (this OVA includes Oracle for mesh management (CGR, IR5x)), can be downloaded from the following link:

– https://software.cisco.com/download/home/286287993/type/286320249/release/4.5.1

Note: -v containing image should be used for mesh deployment.

Note: After download, use the iot-fnd-oracle-4.4.0-79.ova file to install the FND Application.

■FND is installed in the shared services network in CCI so that it can be accessible by FAR and other headend components. The installation steps can be found at the following link (refer to section “Prerequisites, Installing the OVA”).

– https://www.cisco.com/c/en/us/td/docs/routers/connectedgrid/iot_fnd/install/ova/installation-ova-fnd-4-3-1.html#pgfld-1544292

■RHEL needs an active account with access to subscription management, needed for performing yum updates, yum install, and so on. Addressing these prerequisites is beyond the scope of this document. Please refer to Red Hat documentation.

■IP address configuration on a couple of interfaces:

–a) One interface configured with the IP Address of the FND:

• The interface must be configured with an IPv4 address and an IPv6 address.

–b) Another temporary interface providing Internet connectivity.

Implementation of FND

■The section “Implementing Field Network Director” in the FND implementation guide has detailed implementation information (you can skip the sections “Integrating FND with TPS Proxy” and “Integrating FND with FND-DB”) at the following URL:

– https://salesconnect.cisco.com/#/content-detail/da249429-ec79-49fc-9471-0ec859e83872

■After successful implementation, you should check the status of FND in the CLI:

Implementation of Cisco IC3000 Industrial Compute Gateway

In CCI, the IC3000 Industrial Compute Gateway connected to the edge switch via the management port learns about the FND via the DHCP server through option 43 and connects to the FND. Registration succeeds assuming the CSV file is uploaded to the FND and connectivity exists between the FND and the IC3000 Industrial Compute Gateway. As part of registration, FND enables the data ports for data traffic if enabled from the IC3000 Industrial Compute Gateway template under the FND.

For information about managing/deploying IC3000 Industrial Compute Gateway, refer to the Cisco IC3000 Industrial Compute Gateway Deployment Guide at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/routers/ic3000/deployment/guide/DeploymentGuide.html

Implementing Centralized Wireless LAN Controller for Cisco Unified Wireless Network

In CCI, Cisco Catalyst 9800 Series Wireless Controller (C9800-40) is configured as a Centralized Wireless LAN Controller (WLC) with High Availability (HA) for managing Cisco Unified Wireless Network (CUWN) with Wi-Fi mesh deployments in PoPs. Refer to the “Cisco Unified Wireless Network (CUWN) with Mesh” section in the Connected Communities Infrastructure Design Guide at the following URL for more details on the design:

■ https://www.cisco.com/c/en/us/td/docs/solutions/Verticals/CCI/CCI/DG/cci-dg.html

This section covers the initial installation and HA configuration of C9800-40 WLC in CCI Shared Services network. This section applies to you if you are doing CUWN wireless and deploying WLC centrally in Shared Services.

Cisco WLC (C9800-40) Installation and Initial Configuration

The Cisco Catalyst 9800-40 Wireless Controller is a 40-G wireless controller that offers a compact form factor that consumes less rack space and power while offering 40 Gbps forwarding throughput. This section covers the installation and Day-0 Configuration required to setup the C9800 WLC.

Refer to the following URL for rack mounting and installing the C9800-40 hardware:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-40/installation-guide/b-wlc-ig-9800-40/installing-the-controller.html

Once the WLC is rack mounted, verify the following:

1. The network interface cable or the optional Management port cable is connected.

2. The chassis is securely mounted and grounded.

3. The power and interface cables are connected

4. Terminal server is connected to the console port.

There are two modes in which a IOS XE software image on a Catalyst 9800 WLC can run: Install mode and Bundle mode.

Install Mode

The install mode uses pre-extracted files from the binary file into the flash in order to boot the controller. The controller uses the ‘packages.conf’ file that was created during the extraction as boot variable.

Bundle Mode

The system works in bundle mode if the controller boots with the binary image (.bin) as boot variable. In this mode the controller extracts the.bin file into the RAM and runs from there. This mode uses more memory than install mode since the packages extracted during boot up are copied to the RAM.

Note: Install mode is the recommended mode to run the wireless controller.

Boot the Controller in Install Mode:

Step 1: Make sure to boot from flash:packages.conf (and we do not have other boot files specified in our configuration).

Step 2: Install software image to flash. The install add file bootflash:<image.bin> activate commit command moves the switch from bundle-mode to install-mode where image.bin is our base image.

Step 3: Type yes to all the prompts. Once the installation is completed the controller proceeds to reload.

Step 4: After the controller bootup, you can verify the current installation mode of the controller. Run the show version command to confirm the mode.

For more details on WLC power up and initial configuration, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-40/installation-guide/b-wlc-ig-9800-40/power-up-and-initial-configuration.html

Day-0 Manual Configuration Using the Cisco IOS-XE CLI:

C9800-40 WLC is connected to shared services network with 10G link. The steps to access WLC CLI to perform the initial configuration on the controller are provided below.

Step 1: Terminate the configuration wizard (this wizard is not specific for wireless controller):

Step 2: Press Return and continue with the manual configuration.

Step 3: Press Return to bring up the WLC> prompt and Type enable to enter privileged EXEC mode.

Step 4: Enter the config mode and set the hostname:

Step 5: Configure login credentials:

Step 6: Configure the VLAN for wireless management interface and shared services VLAN in CCI network.

Step 7: Configure the SVI for wireless management interface.

Step 8: Configure the interface TenGigabitEthernet0/0/1 as trunk:

Step 9: Configure a default route (or a more specific route) to reach the box:

Step 10: Disable the wireless network to configure the country code:

Step 11: Configure the AP country domain. This configuration is what will trigger the GUI to skip the DAY 0 flow as the C9800 needs a country code to be operational:

Step 12: Specify the interface to be the wireless management interface:

Step 13: For the Controller to be discovered by the Cisco DNA Center or Prime Infrastructure, CLI, SSH and SNMP credentials should be configured on the devices along with NETCONF:

Verify that you can ping the wireless management interface and then just https://<IP of the device wireless management interface>. Use the credentials you have entered earlier. Since the box has a country code configured, the GUI will skip DAY 0 page and you will get access to the main Dashboard for DAY 1 configuration.

Accessing C9800-40 WebUI:

Access the C9800 Web UI using https://<IP_addr_of_C9800-40-WLC>. The username and password configured during the Day-0 configuration of WLC must be used to log on to WLC Web UI. Figure 21 shows C9800-40 WLC Web UI dashboard view after successful login.

Figure 21 Cisco 9800-L WLC Web UI Dashboard View

Cisco WLC (C9800-40) High Availability Configuration

High availability (HA) has been a requirement on wireless controllers to minimize downtime in live networks. This section provides information on the theory of operation and configuration for the Catalyst 9800 Wireless Controller as it pertains to supporting stateful switchover of access points and clients (AP and Client SSO).

The redundancy explained on this document is 1:1, which means that one of the boxes will be in Active State while the other one will be in Hot Standby. If the active box is detected to be unreachable, the Hot Standby unit will become Active and all the APs and clients will keep its service through the new active box.

Once both boxes are synchronized with each other, the standby 9800 WLC will mimic its configuration with the primary box. Any configuration change is done on the active unit will be replicated to the standby unit via the Redundancy Port (RP). Configuration changes are no longer allowed to be performed on the standby 9800 WLC.

Besides the synchronization of the configuration between boxes, they also synchronize the APs in UP state (not APs in downloading state or APs in DTLS handshaking), clients in RUN state (this means that if there is a client in Web Authentication required state and a switchover occurs, that client will have to restart its association process), RRM configuration along other settings.

For more details on deployment and configuration, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-6/config-guide/b_wl_17_6_cg/m_vewlc_high_availability.html

High Availability Prerequisites:

■HA Pair can only be form between two wireless controllers of the same form factor

■Both controllers must be running the same software version in order to form the HA Pair

■Maximum RP link latency = 80ms RTT, minimum bandwidth = 60 Mbps and minimum MTU = 1500

Configure HA on 9800 WLC Hardware:

C9800-40-K9 Wireless controller has two RP Ports as shown in Figure 22.

Figure 22 C9800-40 WLC Front View

In Figure 22:

1. RJ-45 Ethernet Redundancy port

2. SFP Gigabit Redundancy port

The HA Pair always has one active controller and one standby controller. If the active controller becomes unavailable, the standby assumes the role of the active. The Active wireless controller creates and updates all the wireless information and constantly synchronizes that information with the standby controller. If the active wireless controller fails, the standby wireless controller assumes the role of the active wireless controller and continues to the keep the HA Pair operational. Access Points and clients continue to remain connected during an active-to-standby switchover.

Figure 23 C9800-40 WLC High Availability Network Topology

Redundancy SSO is enabled by default, but you still need to configure the communication between the boxes. Follow the step-by-step instructions below for deploying WLC in HA.

Step 1: Make sure both the C9800 WLCs are reachable to each other. Wireless management interface from both boxes must belong to the same VLAN and subnet (in our case connected to Nexus 5000).

Step 2: Connect both 9800 WLC to each other through its RP port.

There are two options to connect both 9800 WLCs to each other, choose the one that fits you more. In this example implementation, RJ45 Ethernet ports are connected.

1. Redundancy Port—RJ45 10/100/1000 redundancy Ethernet port, as shown in Figure 24.

Figure 24 C9800-40 WLC RJ45 Redundancy Ports Connection

2. Redundancy Port—10-GE SFP ports, as shown in Figure 25:

Figure 25 C9800-L WLC Redundancy Ports Connection

Step 3: Provide the required redundancy configurations to both 9800 WLCs.

Step 4: On WLC Web UI, navigate to Administration-> Device-> Redundancy. Enable “Redundancy Configuration”, check ‘RP’ for “Redundancy Pairing Type” and enter the desired IP address along with the Active and Standby Chassis Priorities. Each box should have its own IP address and they should both belong to the same subnet.

On the Active controller, the priority is set to a higher value than the standby controller. The wireless controller with the higher priority value is selected as the active during the active-standby election process. If we do not choose a specific box to be active, the boxes themselves will elect Active based on lowest MAC address. The Remote IP is the IP address of the standby controller’s redundancy port IP.

C9800-40 WLC1 and C9800-40 WLC2:

Figure 26 Redundancy Pairing on both C9800-40 WLCs

Step 4: Switch to C9800 WLC CLI and configure Chassis HA interface.

Step 5: Configure the priority of the specified device.

Cisco Prime Infrastructure Installation and Configuration

Cisco Prime Infrastructure (PI) will act as a dedicated Network Management Server (NMS) providing network device and client monitoring and reporting services. The solution will integrate WLCs and APs with the existing virtual PI deployment. All configuration for WLCs and APs can be deployed using PI with the aid of configuration templates.

This section describes how to configure and integrate Catalyst 9800 Series Wireless Controllers with Prime Infrastructure (3.7) which uses CLI, Simple Network Management Protocol (SNMP) and NETCONF. Configuration details for SNMPv2 and SNMPv3 are included.

PI 3.7 Installation

PI 3.7 Virtual Appliance (VA) is installed in Shared Services network. Refer to the installation guide at the following URL which describes how to install Cisco Prime Infrastructure 3.7 as an OVA on VMware. Download the OVA file PI-VA-3.7.0.0.159.ova from Cisco.com. Verify the integrity of the OVA file using its checksum listed on Cisco.com.

■https://www.cisco.com/c/en/us/td/docs/net_mgmt/prime/infrastructure/3-7/quickstart/guide/bk_Cisco_Prime_Infrastructure_3_7_0_Quick_Start_Guide.html

Figure 27 Prime Infrastructure 3.7 Verification

Access the PI WebUI with the IP address configure:

Figure 28 Cisco Prime Infrastructure Web UI—Dashboard View

Managing Catalyst 9800 WLC with Prime Infrastructure Using SNMP v3 and NETCONF

In order for Prime Infrastructure to configure, manage, and monitor Catalyst 9800 Series Wireless LAN Controllers, it needs to be able to access Catalyst 9800 via CLI, SNMP, and NETCONF. When adding Catalyst 9800 to Prime Infrastructure, telnet/SSH credentials as well as SNMP community string, version, etc. will need to be specified. PI uses this information to verify reachability and to inventory Catalyst 9800 WLC. It will also use SNMP to push configuration templates as well as support traps for AP and client events. However, in order for PI to gather Access Point (AP) and client statistics, NETCONF is leveraged. NETCONF is not enabled by default on Catalyst 9800 WLC and needs to be manually configured.

For more details, refer to the following URL:

■ https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/214286-managing-catalyst-9800-wireless-controll.html

Configuration on Catalyst 9800 WLC

SNMPv2 Configuration on Catalyst 9800 WLC

GUI:

Step 1. Navigate to Administration -> Management -> SNMP -> Slide to Enable SNMP.

Step 2. Click on Community Strings and create a Read-Only and a Read-Write community name.

CLI:

SNMPv3 Configuration on Catalyst 9800 WLC

GUI:

Note: As of 17.1 IOS-XE, the web UI will only allow to create read-only v3 users. Follow the CLI procedure to create a read-write v3 user.

Click on V3 Users. Create a user, choose AuthPriv, SHA, and AES protocols and chose long passwords as show in Figure 29.

Figure 29 Cisco 9800-40 WLC SNMP Configuration

CLI:

Note: SNMPv3 User Config is not reflected on running-configuration. Only SNMPv3 group configuration is seen

NETCONF Configuration on the Catalyst 9800 WLC:

GUI:

Navigate to Administration -> Management -> HTTP/HTTPS/NetConf.

CLI:

Note: If aaa new-model is enabled on Cat9800, then we will also need to configure

NETCONF on 9800 uses the default method (and we cannot change this) for both aaa authentication login as well as aaa authorization exec. In case we want to define a different method for SSH connections, we can do so under the "line vty" command line. NETCONF will keep using the default methods.

Cisco Prime Infrastructure 3.7 Configuration

GUI:

Navigate to Configuration -> Interface -> Wireless.

Step 1. Capture the Wireless Management IP address configured on the Catalyst 9800 WLC.

CLI:

Navigate to Administration -> User Administration.

Step 2. Capture the privilege 15 user credentials as well as enable password.

CLI:

Step 3. Get the SNMPv2 community strings and/or SNMPv3 user as applicable.

GUI:

For SNMPv2, Navigate to Administration-> Management-> SNMP-> Community Strings.

For SNMPv3, Navigate to Administration-> Management-> SNMP-> V3 Users.

CLI:

Step 4. On Prime Infrastructure GUI, navigate to click on Configuration-> Network : Network Devices-> Click on Drop-Down beside +-> Select Add Device.

Step 5. On the Add Device pop-up, enter the interface IP address on 9800 that will be used to establish communication with Prime Infrastructure.

Step 6. Navigate to SNMP tab and provide SNMPv3 details configured on Cat9800 WLC. From Auth-Type Drop-down match the previously configured authentication type and from Privacy Type Drop-Down select the encryption method configured on Cat9800 WLC.

Step 7. Navigate to Telnet/SSH tab of Add Device, provide the Privilege 15 Username and Password along with Enable Password. Click on Verify Credentials to ensure CLI, SNMP credentials work fine. Then click on Add, as shown in Figure 30.

Figure 30 Adding C9800 WLC to the PI

Verification:

Step 1. Verify that NETCONF is enabled on Cat9800:

Step 2. Verify the telemetry connection to Prime from the Cat9800

Step 3. On Prime Infrastructure, navigate to Inventory-> Network Devices-> Device Type and verify the status as shown in Figure 31.

Figure 31 C9800 WLC on the PI as a Managed Device

Cisco Secure Network Analytics Installation and Configuration

The Cisco Secure Network Analytics (Stealthwatch) system collects and analyses flow telemetry generated by the network infrastructure for the purposes of network and security visibility. The Flow Collector leverages enterprise telemetry such as NetFlow, IPFIX (Internet Protocol Flow Information Export), and other types of flow data from existing infrastructure such as routers, switches, firewalls, endpoints, and other network infrastructure devices, Using flow telemetry, host behavior is monitored using continuous automated behavioral analysis techniques. The intelligence generated by Stealthwatch can be reported both to security and network operations staff in order to provide quick access and detailed analysis of security and network events.

The main components of Cisco Stealthwatch system are:

■Stealthwatch Management Console (SMC)

■Stealthwatch Flow Collector (SFC)

For more information, see the Cisco Secure Network Analytics web page:
https://www.cisco.com/c/en/us/products/security/stealthwatch/index.html

Stealthwatch Management Console

The Stealthwatch Management Console (SMC) is an enterprise-level security management system that allows network administrators to define, configure, and monitor multiple distributed Stealthwatch Flow Collectors from a single location. This system provides flow-based security, network, and application performance monitoring across physical and virtual environments. With Stealthwatch, network operations and security teams can see who is using the network, what applications and services are in use, and how well they are performing. The SMC client software allows you to access the SMC’s user-friendly graphical user interface (GUI) from any local computer with access to a web browser.

Through the client GUI, you can easily access real-time security and network information about critical segments throughout your network.

Stealthwatch Flow Collector for NetFlow

The Stealthwatch Flow Collector (SFC) is responsible for collecting all NetFlow telemetry generated by a network’s flow-capable devices. This is the heart of the Stealthwatch system and where data normalization and analysis occurs.

SMC and SFC are deployed as an Virtual Appliances in CCI Shared services VLAN in underlay network on ESXI host. This section describes and explains how to initialize SMC and add Flow Collector to SMC.

SMC and SFC Installation

For installing a SMC and FC Virtual Appliance using VMware, refer to “Installing a Virtual Appliance using VMware” in:

■ https://www.cisco.com/c/dam/en/us/td/docs/security/stealthwatch/system_installation_configuration/SW_7_1_Installation_and_Configuration_Guide_DV_1_0.pdf

Step 1: Configuring IP addresses.

After you install the Stealthwatch VE appliances (both SMC and SFC) using VMware, you are ready to configure the basic virtual environment for them. In CCI network, we deployed the OVA file and powered up the VM. After the initial boot, it will ask you to enter the IP address, subnet, broadcast address, and gateway you would like to use. After you configure this, it will restart again.

For IP address configuration refer to “Configuring the IP Addresses” in:

■ https://www.cisco.com/c/dam/en/us/td/docs/security/stealthwatch/system_installation_configuration/SW_7_1_Installation_and_Configuration_Guide_DV_1_0.pdf

Figure 32 Stealthwatch System Configuration

After the VM restarts, you are shown a log in prompt. The default username/password is sysadmin/lan1cope. You can enter this and change the default password if you want.

Note: You'll have to do the following setup for both the SMC and the SFC.

Step 2: Configuring the appliances.

Open up a browser and navigate to https://<ip-addr-of-SMC>.

You will be able to login to this page with the default username/password of admin/lan411cope. After initially signing in, you are shown the welcome screen shown in Figure 33.

Figure 33 Stealthwatch Appliance Setup Tool

To configure the appliance, refer to “Configuring Your Appliances” in:

■ https://www.cisco.com/c/dam/en/us/td/docs/security/stealthwatch/system_installation_configuration/SW_7_1_Installation_and_Configuration_Guide_DV_1_0.pdf

Figure 34 Stealthwatch Management Console Appliance Configuration

Note: You will have to do the appliance configurations for both the SMC and the SFC.

Step 3: Configure your Flow Collectors for Central Management.

To configure your Flow Collector so it communicates with your primary SMC/Central Manager, refer to “Configure your Flow Collectors for Central Management” in:

■ https://www.cisco.com/c/dam/en/us/td/docs/security/stealthwatch/system_installation_configuration/SW_7_1_Installation_and_Configuration_Guide_DV_1_0.pdf

Figure 35 Stealthwatch Flow Collector Appliance Configuration

Verification:

After you configure an appliance in the Appliance Setup Tool and configure SFC for Central Management, confirm the appliance status in Central Management by navigating to log in to your primary SMC. Click the Global Settings icon and select Central Management.

Confirm the appliance is shown in the inventory and the status for the appliance is shown as Up.

Figure 36 Appliances on SMC

Netflow Configuration on Network Devices

In CCI network, NetFlow is enabled on Cisco IE switches (IE4000, IE5000, IE3400, and IE3300) in the ring to monitor the network flows. Using the DNA Center templates, Netflow can be enabled on the CCI devices.

Refer the following URL for details about the Cisco DNA Center Template Editor:

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_01000.html

Template:

Verification:

You can verify the traffic flow monitoring on the SMC dashboard.

Figure 37 Stealthwatch Management Console Dashboard

Stealthwatch and ISE Integration

This section describes the steps to configure Cisco Stealthwatch Management Centre (SMC) and Cisco Identity Services Engine (ISE) using pxGrid. Once integrated with ISE, the SMC will learn the user session information (IP address/username bindings), Static TrustSec mappings, and Adaptive Network Control (ANC) mitigation actions for quarantining endpoints.

Step 1: Generating certificates.

To connect Stealthwatch and Cisco ISE, certificates must be deployed correctly for trusted communication between the two systems. Deploying certificates requires that you use several different product or application interfaces: the SMC Web App, the Central Management interface, and the Cisco ISE Server management portal. Starting with v7.0, Stealthwatch only imports client certificates created with a Certificate Signing Request (CSR) generated from Stealthwatch Central Management to connect to ISE pxGrid node.

The recommended method of deploying certificates is to use the ISE internal Certificate Authority (CA). This option is only available with ISE 2.2 and above.

To deploy certificates using the ISE internal CA, refer to “Using ISE Internal CA” in:

■ https://community.cisco.com/t5/security-documents/deploying-cisco-stealthwatch-7-0-with-cisco-ise-2-4-using-pxgrid/ta-p/3793357?attachment-id=165804

Figure 38 Client Identity in SMC

Step 2: Configuring ISE pxGrid integration.

To configure Stealthwatch to successfully connect, register, and subscribe to the ISE pxGrid node, refer to “Configuring ISE pxGrid Integration” in:

■ https://community.cisco.com/t5/security-documents/deploying-cisco-stealthwatch-7-0-with-cisco-ise-2-4-using-pxgrid/ta-p/3793357?attachment-id=165804

Figure 39 Stealthwatch Integration with ISE

Step 3: Applying ISE Adaptive Network Control (ANC) policies.

ISE ANC policies align with organizations security policies. For example, when malware or breaches are detected, the organization may investigate further by providing segmented network access or, if the threat is more severe and capable of propagating through the network, the IT administrator may want to shut down the port.

Possible ANC actions are: quarantine (Change or Authorization), port-shut, and port bounce. These ANC policies will then be used as condition rules in ISE authorization policies to enforce the organizations security policy.

To create ISE ANC policies and associate to Stealthwatch, refer to “SE Adaptive Network Control (ANC) Policies” in:

■ https://community.cisco.com/t5/security-documents/deploying-cisco-stealthwatch-7-0-with-cisco-ise-2-4-using-pxgrid/ta-p/3793357?attachment-id=165804

Figure 40 ISE ANC Policy on Stealthwatch

Stealthwatch Use Cases

Cisco Stealthwatch provides comprehensive network visibility and threat detection for accelerated incident response. For more information, see:

■ https://community.cisco.com/t5/security-documents/stealthwatch-use-cases/ta-p/3611837

Licensing

Use the Stealthwatch Downloading and Licensing Guide to activate licenses on your appliances:

■ https://www.cisco.com/c/en/us/support/security/stealthwatch/products-licensing-information-listing.html

Preparing Cisco DNA Center for PoP Site Provisioning

In the Cisco DNA Center, the “Design” area is where you create the structure and framework of your network, including the physical topology, network settings, and device type profiles that you can apply to devices throughout your network. Create a network hierarchy of areas, buildings, and floors that reflect the physical deployment. In later steps, discovered devices are assigned to respective PoP sites in Cisco DNA Center GUI, so that they are displayed hierarchically in the topology maps.

To prepare to design your Cisco DNA Center for CCI network fabric implementation, refer to the chapter “Design Network Hierarchy & Settings” in the Cisco Digital Network Architecture Center User Guide, Release 2.2.3 at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html

1. Creating network hierarchy.

In this implementation, three fabric sites are created as shown in Figure 3 and Figure 4 for various deployment models of the network topologies with IP Transit and SD-Access Transit fabric interconnection. Example fabric sites with the names MGRoad, Hebbal, Elex City are configured for the fabric sites PoP1 Site, PoP2 Site, PoP3 Site respectively. In Figure 41, the sites with names Cessna and Koramangala are configured as HQ/DC site and SDA transit site respectively.

Note: In CCI deployment, a PoP site can be mapped to an area with a building under that area in Cisco DNA Center network hierarchy. By creating buildings, you can apply settings to a specific area or a PoP site.

For more details about Network Hierarchy and steps to configure the hierarchy of PoP sites and HQ/DC Site, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.htm

Figure 41 shows an example Network Hierarchy under the Design tab in the Cisco DNA Center user interface for the CCI network implementation:

Figure 41 Example CCI Network Hierarchy View in Cisco DNA Center

2. Configuring network settings.

Set up network properties such as AAA, DHCP, DNS, and NTP for the CCI network. Cisco DNA Center will configure the network settings on the devices while provisioning the discovered devices in the fabric.

Refer to the following sections to configure network settings:

–Manage Global Network Settings:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#concept_gvs_1rd_wy

–Configure Global Network Servers:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#task_bw1_vyb_yz

Figure 23 shows an example Network Settings configured in the Cisco DNA Center for the CCI network topology:

Figure 42 Example Global Network Settings View in Cisco DNA Center

3. Setting device credentials for device discovery.

–Device credentials refer to the CLI, SNMP, and HTTPS credentials that are configured on network devices. Cisco DNA Center uses these credentials to discover and collect information about the devices in your network. Configure global/site level device credentials to discover all the network devices in the CCI network for fabric/PoP site provisioning.

Refer to the following sections for configuring device credentials in the Cisco DNA Center:

–About Global Device Credentials:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#concept_zww_qtd_wy

–Configure Global CLI Credentials:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#task_a5t_5xg_2z

–Configure SNMPv3 Credentials:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#task_fgz_lyg_2z

4. Configuring IP address pools.

IP address pools that will used for fabric infrastructure provisioning, extended nodes in the CCI network, and data network are manually defined and configured on the Cisco DNA Center. It reserves pools as a visual reference for use in fabric sites (PoPs). In this implementation, a windows DHCP server is used.

Alternatively, you can integrate third party IP Address Manager (IPAM) servers to Cisco DNA Center in order reduce IP address management tasks. IPAM integration with Cisco DNA Center provides:

–Access to existing IP address scopes, referred to as IP address pools in Cisco DNA Center.

–When configuring new IP address pools in Cisco DNA Center, the pools populate to the IPAM server automatically.

To integrate IPAM server to Cisco DNA Center, refer to the “Configure an IP Address Manager” section at the following URL:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#task_umt_4kt_pcb

Refer to the following sections for adding and reserving IP address pools as needed in CCI network deployment:

–Configure IP Address Pool:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#task_znt_jdc_yz

–Reserve an IP Pool:

• https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_0110.html#id_105306

Figure 43 and Figure 44 show example IPv4 address pools with global network prefixes and reserved IP pools in a site for fabric border network handoff, extended node, and data networks on fabric overlay VNs.

Figure 43 Example Global IPv4 Address Pools View in Cisco DNA Center

Figure 44 Example IPv4 Address Pools Reserved in a Fabric/PoP Site

This completes the initial preparation of Cisco DNA Center for devices discovery and fabric site provisioning.

Discovering Devices in the Network

Cisco DNA Center is used to discover and manage the SD-Access underlay network devices that are compatible with the Cisco DNA Center. For the list of the devices supported by Cisco DNA Center, refer to the following URL:

■ https://www.cisco.com/c/en/us/solutions/enterprise-networks/software-defined-access/compatibility-matrix.html

To discover equipment in the network, the appliance must have IP reachability to these devices, and CLI and SNMP management credentials must be configured on the devices. Once discovered, the devices are added to Cisco DNA Center's inventory, allowing the controller to make configuration changes through provisioning.

1. For the Network devices to be discovered by the Cisco DNA Center, CLI and SNMP credentials should be configured on the devices as configured at the Cisco DNA Center in the previous section.

The example configuration used network devices in this implementation:

a. Configure CLI SSH user credentials on the network device. Example configuration on Cisco Catalyst 9300 Switch Stack:

b. Configure SNNMPv3 credentials on the network device. Example configuration on Cisco Catalyst 9300 Switch Stack:

c. Enable SSH Version 2 access on the network device. Example configuration on Cisco Catalyst 9300 Switch Stack:

Repeat the above configurations on all the network devices in the network to be discovered by the Cisco DNA Center.

2. For detailed step-by-step instructions on discovering all the device in the CCI network on Cisco DNA Center, refer to the chapter “Discover your Network” in the Cisco Digital Network Architecture Center User Guide, Release 2.2.3 at the following URL:

– https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_010.html

Once device discovery is successful, all the devices are added to Cisco DNA Center Inventory, as shown in the example in Figure 45:

Figure 45 Example List of Discovered Devices in Cisco DNA Center Inventory

Provisioning Devices in SD-Access

Once the devices are discovered and managed in the Cisco DNA Center inventory, devices have to be provisioned to the sites for SD-Access Deployment.

For more details and step-by-step instruction for provisioning devices in SD-Access site, refer to the following section in the Software-Defined Access for Distributed Campus Deployment Guide at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/SD-Access-Distributed-Campus-Deployment-Guide-2019JUL.html#_Toc13487388

For how to assign the device to sites and provisioning devices, click “Process 5: Deploying SD-Access with the Provision Application” and follow Procedures 1 and 2 at the following URL:

■ https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/SD-Access-Distributed-Campus-Deployment-Guide-2019JUL.html#_Toc13487382

Once the devices are provisioned to the sites, all the details added in the network settings like AAA, NTP, DHCP, and DNS are configured on the devices by Cisco DNA Center.

Provisioning Fabric Overlay Network

Once devices are provisioned to a site, the fabric overlay workflows can begin. This starts through the creation of transits, the formation of a fabric domain, and the assignment of sites, buildings, and/or floors to this fabric domain.

Fabric domain is configured in the Cisco DNA Center for a fabric overlay network. After adding the sites to the Network Hierarchy, the network sites have to be part of a fabric domain. Once the fabric domain is added, add the transit networks (either IP transit or SDA transit or both) for interconnecting multiple fabric sites (PoPs). In this implementation, both the transit network types (IP Transit and SD-Access Transit) are validated for the network topologies, as shown in Figure 3 and Figure 4.

Configuring Fabric Domain and Transit Network(s)

Depending on your network deployment and backhaul network for interconnecting fabric sites, you can choose to deploy either IP Transit or SD-Access Transit as applicable:

1. For provisioning the fabric domain and creating a IP-based transit network in the Cisco DNA Center, click "Process 6: Provisioning the Fabric overlay"and follow Procedures 1, 3, and 4 in the Software-Defined Access for Distributed Campus Deployment Guide at the following URL:

– https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/SD-Access-Distributed-Campus-Deployment-Guide-2019JUL.html#_Toc13487387

2. Optionally, follow Procedure 2 to create an SD-Access transit network for an example network topology, as shown in Figure 3.

Note: For the SD-Access transit network, it is required to assign the transit control plane nodes (example: Cisco Catalyst 9500 switches) to a site that will be provisioned as a SD-Access transit site, by completing the steps mentioned in Provisioning Devices in SD-Access.

Figure 46 shows an example fabric domain: IP-based Transit and SD-Access Transit networks created for the network topology in Figure 3 and Figure 4.

Figure 46 Example Fabric Domain, IP-based, and SD-Access Transit Networks

Assigning Fabric Role (Fabric-in-a-Box)

Once the sites are added to the fabric domain in the Cisco DNA Center, a Cisco Catalyst 9300 Stack that is added in a site is provisioned with fabric roles. A fabric overlay consists of three different fabric nodes: control plane node, border node, and edge node. To function, a fabric must have an edge node and control plane node. This allows endpoints to traverse their packets across the overlay to communicate with each other (policy dependent). The border node allows communication from endpoints inside the fabric to destinations outside of the fabric along with the reverse flow from outside to inside.

In the CCI network fabric site (PoP), a switch stack (Cisco Catalyst 9300) is configured with all the fabric roles (i.e., border, control plane, and edge called FiaB). Fabric is provisioned with an overlay VN; i.e., macro-segmentation for the overlay network is defined. (Note that the overlay network will only be fully created until the host onboarding stage). This process virtualizes the overlay network into multiple self-contained VNs.

1. Creating Overlay VN

In the CCI network, VNs and SGTs are created for each vertical use case overlaid on the CCI network. An example list of VNs created in this implementation are available in Table 2.

To create VNs in the fabric as needed, refer to Procedure 1 under section "Process 4: Creating Segmentation with the Cisco DNA Center Policy Application” in the Software-Defined Access for Distributed Campus Deployment Guide

https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/SD-Access-Distributed-Campus-Deployment-Guide-2019JUL.html#_Toc13487379

2. Associate VN to Fabric Site

IP address pools enable host devices to communicate within the fabric site. Associate IP addresses pools for end-points data traffic in the overlay VN.

Follow the steps in Virtual Network section under the chapter “Provision Fabric Networks” to create VNs and associate IP address pools to a VN:

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_01110.html#id_50854

Note: Select No Authentication for the default Authentication Template for a fabric site. The No Authentication template is selected as default template.

Figure 47 shows an example VN and overlay IP pools associated with a VN (SnS_VN) in the CCI network:

Figure 47 Example Virtual Network and IP Pools Association in CCI Network

–Similarly, associate an IP pool in the fabric default INFRA_VN for Extended Nodes IP addressing.

3. Provisioning Fabric-in-a-Box (FiaB)

Once the VNs are associated with the fabric sites, provision a Cisco Catalyst 9300 Switch stack as FiaB in the CCI fabric (PoP) site. A Layer 3 handoff that extends the fabric VNs to the next hop (i.e., fusion router, in the case of IP-based Transit or SDA Transit CP or intermediate network device, in the case of SDA Transit). This will allow the end-points in the fabric to access shared services once the fusion router configuration is completed.

Complete the following steps to provision a Cisco Catalyst 9300 Switch stack in a fabric/PoP site as FiaB for IP-based transit network topology, as shown in Figure 3:

a. In Cisco DNA Center, navigate to Provision-> Fabric.

b. Select the Fabric Enabled Site (Bangalore) that was created.

c. Select the PoP site (MGRoad) from the fabric-enabled sites in the left pane.

d. Select the device to be provisioned as a FiaB. A slide pane appears.

e. On the slide pane, select the roles Edge node, control plane, and border node, as shown in Figure 48.

Figure 48 Example FiaB Provisioning View for IP Transit

f. Click Configure next to Border Role, configure the local Autonomous number for the site, and select the Layer 3 handoff network pool associated with the site.

g. Under the Transit/Peer networks, enable the option Default to all Virtual networks and then select the transit site. In this case, we used SD Access Transit.

h. Select the Transit Control Plane devices and then click Add.

Example provisioning for IP transit external interfaces is shown in Figure 49.

Figure 49 Example FiaB Border Configuration for SD-Access Transit in CCI Network

i. Once done, click Add and Save to provision the FiaB and for the successful provisioning of the Fabric message in the Cisco DNA Center UI.

j. Verify the FiaB provisioning in Cisco DNA Center UI for the network site. No errors should be reported in the Fabric Infrastructure map view of the FiaB.

Alternatively, if you are deploying an IP Transit based network topology, as shown in Figure 4, you need to configure the FiaB Border to connect to an SD-Access Transit Network created in Step 2 of the Configuring Fabric Domain and Transit Network(s).

Refer to the following URL for steps to create the IP Transit network in Cisco DNA Center:

– https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_01110.html#id_75992

This completes the FiaB provisioning or fabric role assignment in the fabric overlay network for a PoP site connected to either the SD-Access Transit or IP-based Transit network.

Implementing Wireless LAN Controller in a PoP

In CCI, Cisco Catalyst 9800 Series Wireless Controller (C9800-L) or Cisco Catalyst 9300 Series Switch Stack with embedded Wireless Controller can be configured. C9800-L WLC manages Cisco Unified Wireless Network (CUWN) Wi-Fi access mesh and non-mesh deployments. Alternatively, an embedded WLC on C9300 switch stack can be deployed for managing SD Access Wireless (Wi-Fi) networks. Refer to the “CCI Wi-Fi Access Network Solution” section in the Connected Communities Infrastructure Design Guide at the following URL for more details on the CCI Wi-Fi design.

■ https://www.cisco.com/c/en/us/td/docs/solutions/Verticals/CCI/CCI/DG/cci-dg.html

Implementing Cisco Unified Wireless Network WLC (C9800-L)

Cisco Catalyst 9800-L Wireless Controller can be configured as a per PoP Wireless LAN Controller (WLC) with High Availability (HA) for managing CUWN Wi-Fi networks within a PoP. Cisco Catalyst 9800-L Wireless Controller is the first low-end controller that provides a significant boost in performance and features from Cisco 3504 Wireless Controller. This section covers the initial installation and HA configuration of C9800-L WLC in a CCI PoP.

For more details on C9800-L controller, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-L/installation-guide/b-wlc-ig-9800-L/overview.html

For rack mounting and installing the C9800-L hardware, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-L/installation-guide/b-wlc-ig-9800-L/Installing-the-Cisco-Catalyst-9800-L-Wireless-Controller.html

Once the WLC is rack mounted make sure the following:

1. The network interface cable or the optional Management port cable is connected.

2. The chassis is securely mounted and grounded.

3. The power and interface cables are connected

4. Terminal server is connected to the console port

Note: Install mode is the recommended mode to run the wireless controller.

Boot the controller in INSTALL mode:

Step1: Make sure to boot from flash:packages.conf (and you do not have other boot files specified in your config).

Step2: Software install image to flash. The install add file bootflash:<image.bin> activate commit command moves the switch from bundle-mode to install-mode where image.bin is our base image.

Step3: Type yes to all prompts. When the install is complete, the controller reloads.

Verify:

After the controller reboot we can verify the current installation mode of the controller. Run show version to confirm.

For more details on WLC power up and initial configuration, refer to the following URL:

■ https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/9800-L/installation-guide/b-wlc-ig-9800-L/Power-Up-and-Initial-Configuration.html

Day-0 Manual Configuration Using the Cisco IOS-XE CLI:

C9800-L WLC is connected to one of our CCI PoP sites MGRoad C9300 FiaB Switch with 10G link.

This section shows you how to access the CLI to perform the initial configuration on the controller.

Step 1: Terminate the configuration wizard (this wizard is not specific for wireless controller):

Step 2: Press Return and continue with the manual configuration.

Step 3: Press Return to bring up the WLC> prompt and Type enable to enter privileged EXEC mode.:

Step 4: Enter the config mode and set the hostname:

Step 5: Configure login credentials:

Step 6: Configure the underlay VLAN for wireless management interface. For example, vlan 199, IP: 10.10.199.199 with gateway 10.10.199.1 in underlay EIGRP 2000 AS.:

Step 7: Configure the SVI for wireless management interface:

Step 8: Configure the interface TenGigabitEthernet0/0/0 as trunk:

Step 9: Configure a default route (or a more specific route) to reach the box:

Step 10: Disable the wireless network to configure the country code:

Step 11: Configure the AP country domain. This configuration is what will trigger the GUI to skip the DAY 0 flow as the C9800 needs a country code to be operational:

Step 12: Specify the interface to be the wireless management interface:

Step 13: For the Controller to be discovered by the Cisco DNA Center or Prime Infrastructure, CLI,SSH, and SNMP credentials should be configured on the devices along with NETCONF.:

Verify that we can ping the wireless management interface and then just https://<IP of the device wireless management interface>. Use the credentials you have entered earlier. Since the box has a country code configured, the GUI will skip DAY 0 page and you will get access to the main Dashboard for DAY 1 configuration.

Accessing C9800-L Web UI

Access the C9800 Web UI using https://<C9800-L-WLC-IP>. The username and password configured during the Day-0 configuration is used to log on to WLC Web UI.

Figure 50 Cisco 9800-L WLC Web UI Dashboard View

Cisco WLC (C9800-L) High Availability (HA) Configuration

The HA Pair always has one active controller and one standby controller. If the active controller becomes unavailable, the standby assumes the role of the active. The Active wireless controller creates and updates all the wireless information and constantly synchronizes that information with the standby controller. If the active wireless controller fails, the standby wireless controller assumes the role of the active wireless controller and continues to the keep the HA Pair operational. Access Points and clients continue to remain connected during an active-to-standby switchover. Follow the steps below for configuring C9800-L WLC with HA in a PoP.

Note: Redundancy SSO is enabled by default but you still need to configure the communication between the boxes.

Step 1: Make sure both the C9800 WLCs are reachable to each other. Wireless management interface from both boxes must belong to the same VLAN and subnet. Connected to C9300 FiaB, one of our PoP sites in our case.

Step 2: Connect both 9800 WLC to each other through its RP port. Connecting C9800-L Wireless Controllers using RJ-45 RP Port for SSO:

Figure 51 C-9800-L WLC Redundancy Port Connections

Step 3: Provide the required redundancy configurations to both 9800 WLCs.

On WLC Web UI:

Navigate to Administration-> Device-> Redundancy. Enable “Redundancy Configuration”, check ‘ RP ’ for “ Redundancy Pairing Type ” and enter the desired IP address along with the Active and Standby Chassis Priorities. Both boxes should have its own IP address, and both should belong to the same subnet.

On the Active controller, the priority is set to a higher value than the standby controller. The wireless controller with the higher priority value is selected as the active during the active-standby election process. If we do not choose a specific box to be active, the boxes themselves will elect Active based on lowest MAC address. The Remote IP is the IP address of the standby controller’s redundancy port IP.

C9800-L WLC1 & C9800-L WLC2 :

Figure 52 Redundancy Pairing on both C9800-L WLCs

CLI:

Configuring Chassis HA interface:

Configure the priority of the specified device:

Configuring SD Access Wireless Embedded WLC on C9300 Stack

Integrating the Wireless with the SD Access brings the best of both the architectures like Simplifying the Control & Management Plane, optimizing the data plane and Integrating Policy & Segmentation end to end. This section covers the installation of Cisco Catalyst Embedded 9800 Wireless Controller (eWLC) on Catalyst 9K series Switches and bring up with Cisco DNA Center.

In CCI, Cisco Catalyst Embedded 9800 Wireless Controller (eWLC) is installed in PoP sites which require SD Access Wireless (Wi-Fi) on C9300 FiaB switch stack. Follow the steps below to configured eWLC on C9300 switch stack.

We will categorize this section into two parts:

■ Installation of eWLC (c9800-sw) on C9300 FiaB PoP Site

■Enable Embedded SDA-Wireless through DNA Center Provisioning & AP Onboarding:

Installation of eWLC (c9800-sw) on C9300 FiaB PoP Site:

The steps to install eWLC on the C9300 FiaB switch are the following:

1. Check that license is dna-advantage.

2. Boot the switch in install mode.

3. Install the eWLC package.

4. Enable netconf-yang.

Step1: Check that license is dna-advantage.

For eWLC package to install properly you need to have the dna-advantage active on the switch. You can check this through show version command.

Step2: Boot the switch in install mode.

When we boot the switch passing directly the.bin image, this is called "bundle mode". Packaging only works, when the switch is booted in "install mode". To verify the mode, run "show version":.

Make sure to boot from flash:packages.conf (there are no other boot files specified in our configuration):

a. Software install image to flash. The install add file bootflash:<image.bin> activate commit command moves the switch from bundle-mode to install-mode where image.bin is our base image.

b. Type yes to all the prompt. Once the install is completed the switch proceeds to reload.

After the controller reboot, we can verify the current installation mode of the controller. Run the show version command in order to confirm.

Step3: Install the eWLC package.

After downloading the eWLC image to the switch you can install the wireless package using one single command line.

In our CCI, eWLC version installed is C9800-SW-iosxe-wlc.17.01.01s.SPA.bin

Where "flash:ewlc_pkg.bin" is our ewlc package. Alternatively, we can also install it from tftp directly.

Say "yes" to all questions. The switch should then reload and come up with ewlc package installed.

Verification:

After reloading we can confirm the install with the install summary command.:

Step4: Enable netconf-yang.

To enable NETCONF on the switch these three commands needs to be on the switch.

Verification:

We can check NETCONF is running with the following command.

Enable Embedded SDA-Wireless through DNA Center Provisioning and AP Onboarding

Prerequisites:

–Make sure we don’t have a WLC in the site where you plan to enable Embedded SDA-Wireless.

–It is important that we do the discovery after we have installed the -wlc pkg otherwise DNACenter will not display the “embedded wireless” option in fabric view.

–Configure our AP IP pool and attach it to INFRA_VN. In that DHCP scope, point DHCP option 43 DNA Center with this PnP will discover the AP

Reserving IP Pool for Access Points:

Navigate to Design-> Network Settings-> IP Address Pools, for MG Road PoP Site reserve IP Pool for SDA Wireless Access Points, as shown in Figure 53.

Figure 53 AP IP Pool Reservation on Cisco DNA Center

Attaching AP IP Pool to INFRA_VN:

Attach an AP IP pool to MGRoad Fabric Site by following the steps under “Add a Gateway to a Layer 3 Virtual Network” section in the following URL.

https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-2-3/user_guide/b_cisco_dna_center_ug_2_2_3/b_cisco_dna_center_ug_2_2_3_chapter_01110.html#task_zm2_sfl_1qb

Figure 54 Attaching AP Pool to INFRA_VN on Cisco DNA Center

The workflow is as follows:

1. Discover device.

2. Assign to site.

3. Provision switch.

4. Add device as FiaB.

5. Enable Embedded wireless.

6. Provision AP (Will be added to DNAC automatically by joining the Catalyst 9800 WLC).

7. Configure onboarding SSID (refer to Implementing Wi-Fi Access Network).

a. When configuring the discovery properties, click the add credentials and configure the NETCONF port to 830.

Figure 55 eWLC Discovery on Cisco DNA Center

b. Assign the switch to MGRoad PoP Site.

c. Provision the device. Refer to Provisioning Devices in SD-Access for the device provisioning steps.

d. Add the device as Fabric in a Box (configure as Border, Control, and Edge Node) and Enable Embedded Wireless.

Figure 56 Enabling Embedded Wireless on FiaB Switch

e. Connect the SDA Wireless APs connect to either Fabric Edge (FE) ports, or Extended Node (EN) ports. It is recommended to resync of the switch for it to add the AP. Go to Provision-> Inventory, select the switch from the site, and resync the switch. The APs will be shown in the Devices tab for us to assign to our eWLC site and provision.

Note: Latency between AP and WLC needs to be < 20 ms.

Figure 57 AP IP Pool Reservation on Cisco DNA Center

Note: To assign the APs to the Site, Floors should be created under the Building under Network Hierarchy.

Figure 58 Attaching AP Pool to INFRA_VN on Cisco DNA Center

Note: By default, the RF profile that you is marked as default under Design > Network Settings > Wireless > Wireless Radio Frequency Profile is selected in the RF Profile drop-down list. You can change the default RF Profile value for an AP by selecting a value from the RF Profile drop-down list. The options are: High, Typical, and Low. The AP group is created based on the RF profile selected.

For verifying successful provisioning of SD Access Wireless on C9300 Stack in a PoP site, navigate to Provision -> SD Access-> Fabric Infrastructure view as shown in Figure 59.

Figure 59 SD Access Wireless AP View on Fabric Infrastructure

PoP Interconnection over Ethernet Network Backhaul

This section covers the example configuration of fabric interconnection for the SD-Access Transit-based network topology shown in Figure 3.

When configuring the interfaces on a fabric border to communicate with SD-Access transit, Cisco DNA will configure a VRF for each VN in the fabric site Border (i.e., FiaB and Transit Control Plane (T-CP) nodes). BGP peering is configured between the T-CP node and FiaB to enable overlay routing. In this implementation, two Cisco Catalyst 9500 switches as Ethernet network backhaul are provisioned as "SD-Access Transit" T-CP nodes, as shown in Figure 19. When connecting fabric sites to a SD-Access Transit network, each VN with subnets configured for data traffic is created as a VRF in FiaB and VN subnet(s) network prefixes for data traffic are registered with T-CP nodes in the SD-Access Transit site.

Example FiaB VRF Configuration:

Cisco DNA Center automatically configured the BGP peering between the FiaB Border and SD-Access Transit Control Plane nodes (i.e., Cisco Catalyst 9500 switches in this implementation) using lookback interfaces configured (routing enabled in the underlay network) on these devices. It leverages the existing underlay physical interfaces/network connectivity to backhaul network. Therefore, no separate physical interface selection is required.

Note: P subnet pools configured for extended nodes are added in the Global Routing Table (GRT) address family in the BGP routing configuration outside of the VRF address family.

Example FiaB Border BGP Routing Automatically Configured by Cisco DNA Center

Example SD-Access Transit Control Plane Node BGP Routing Automatically Configured by Cisco DNA Center

PoP Interconnection via IP Transit over MPLS Network Backhaul

When configuring the interfaces on a fabric border to communicate with an IP transit, the Cisco DNA Center will configure a VRF for each VN in the fabric site. This is known as VRF-lite because the VRFs are only locally significant. When connecting to an MPLS backhaul, the provider will use its own VRFs to keep different customers' traffic separated. Using a VRF-aware routing protocol within the service provider gives them the ability to keep the VRF configuration at the service provider edge instead of every single device in the core. These VRFs, however, are not related to the VRFs configured on the fabric border. To maintain the macro-segmentation provided by a VN's use of VRFs between fabric sites over an IP transit, the service provider must also provide a VRF for each VN configured at a fabric site.

Example Provider Edge VRF Configuration

When configuring the border services, Cisco DNA will automatically configure a VLAN interface on the border node. When configuring the provider edge node, there must be a matching VLAN configuration to enable connectivity. The border configuration is shown in Figure 60.

Figure 60 Border Node External Interface

On the provider edge interface facing the edge fabric border, the services are separated using a different service instance. Each service instance is then associated with a bridge domain interface. For ease of administration, the VLAN encapsulation and bridge-domain should match. If the IP transit is owned by a different operator, they will have to ensure the encapsulation matches the VLAN configured on the fabric border.

The VRF is also added to the service provider's BGP configuration:

A service provider interface will be connected to the data center (fusion router, in this implementation) and this must have all the VRFs configured to maintain segmentation end to end. Because these devices are not part of the fabric, the configuration must be done manually.

Provider Edge VRF facing the fusion router:

Since the VLAN encapsulation is not automatically generated by Cisco DNA for this connection, there are no mandates on the VLAN other than what the service provider may require:

The VRF is then added to the service provider's BGP configuration:

A complementary configuration also exists on the customer edge device (fusion router, in this implementation):

Because the VRF separation is maintained within the IP transit network, the VN will maintain its macro-segmentation from one fabric site to another.

Configuring a Fusion Router in IP-Based Transit Network

For the IP Transit scenario, a Cisco ASR 1000 Series Router was used as the fusion router, but the only requirement is that the router must support route leaking between VRFs. In this implementation, the shared services were part of the global routing table, but they could also be part of a separate shared services VRF.

1. The fusion router configuration is outside the scope of Cisco DNA and must therefore be done manually. The first step is to configure a VRF for every VN configured in Cisco DNA.

2. The fusion router must then have interfaces configured in the VRF, which can connect to a fabric border node or other non-fabric router. In the case of a fabric border node, Cisco DNA will configure the interface and BGP configuration as part of the border configuration. The fusion router side must be done manually. The following is an example of the automatically generated border node interface configuration:

3. The following is the complementary interface configuration manually entered on the fusion router:

4. Cisco DNA also automatically generates the BGP config for the VRF on the border node:

5. The fusion router must be manually configured to successfully neighbor with the border node:

6. Because the VRF creates a routing table separate from the GRT, routes must be shared between them for the VRF to have access to the shared services, and vice versa. One way to achieve this is with prefix lists and route maps.

Example from fusion router:

7. The route-map must then be imported into the target VRF:

8. Verifying the routes on a fabric site:

9. Additionally, routes from the VRF must be exported to the GRT so the shared services can reach interfaces in the VRF:

10. The route map must then be exported from the target VRF:

11. Verifying the routes on the fusion router:

Configuring a Fusion Router in SD-Access Transit Network

Implementation of fusion routers in SD-Access Transit-based network topology (Figure 3) is similar to IP-based Transit since both network topologies connect to fusion routers via the IP Transit network. In this implementation, an IP Transit network interconnects a HQ/DC site with an external network outside of fabric overlay in order to provide access to shared services. Therefore, steps to configure the fusion router is similar to what was described in the previous section.

This section discusses an example implementation of redundant fusion routers in HQ/DC site, as shown in Figure 19, for a CCI implementation (with an SD-Access Transit-based network topology). A couple of Cisco Cloud Services Routers 1000V are used as redundant fusion routers in this implementation.

1. Configure VRF for every VN configured in Cisco DNA Center on the fusion router. Example VRF configuration:

Figure 61 shows an example VLAN automatically created by the Cisco DNA Center Border while FiaB role provisioning.

Figure 61 Example Border Configuration for Connecting to IP Transit

2. In Figure 61, GigabitEthernet2/0/6 is a physical link connecting to a fusion router (CSR1000V-1 used as fusion router) and GigabitEthernet1/0/6 is a physical link to redundant (secondary) fusion router (CSR1000V-2). Example VLAN configurations automatically configured by the Cisco DNA Center on HQ/DC Site FiaB border:

3. Configure complementary interface configurations matching this VLAN interfaces on the fusion router: