Cisco Data Center Network Manager 11 (DCNM): PowerOn Auto Provisioning (POAP) Guide

 

Introduction

The Cisco^® Data Center Network Manager (DCNM) offers Network Management System (NMS) support for traditional or multiple-tenant LAN and SAN fabrics. Cisco DCNM uses PowerOn Auto Provisioning (POAP) to automate the process of upgrading software images and installing configuration files on Cisco Nexus^® switches that are being deployed in the network for the first time.

When a Cisco Nexus switch supporting the POAP feature boots for the first time and does not find a start-up configuration, the switch enters POAP mode to automate the configuration process. For a successful installation, POAP requires the following network infrastructure:

●   A Dynamic Host Configuration Protocol (DHCP) server to bootstrap the interface IP address, gateway address, and DNS server

●   A Trivial File Transfer Protocol (TFTP) or HTTP server containing the configuration script used to automate the software image installation and configuration process

●   One or more servers containing the desired software images and configuration files

The POAP network infrastructure is integrated with Cisco DCNM, and the entire process can be configured from the Cisco DCNM web interface or through REST APIs.

Cisco DCNM POAP deployment scenarios

Reference topology

The network topology is divided into a spine-and-leaf switch Clos architecture with each leaf switch connected to a host or a server. The switches are Cisco Nexus 9000 switches running software version 9.2.1, and the host server is running Cisco DCNM software version 11.0(1).

Note:    This white paper focuses solely on example Cisco DCNM POAP deployment scenarios and the POAP bootstrap process in DCNM 11 for LAN fabric management. A complete description of the Cisco DCNM functionality and capabilities is beyond the scope of this document. This white paper also does not cover classic LAN management.

In this document, we highlight scenarios showing how to deploy Cisco Nexus Switches using Cisco DCNM’s POAP process. Since Linux provides the underlying Cisco DCNM infrastructure, you have many options for using the POAP process in your environment.

Cisco DCNM deployments rely on three network interfaces: eth0, eth1, and eth2. The role of each network interface depends on how you will manage the network fabric. Table 1 summarizes the roles of the Cisco DCNM network interfaces.

Table 1.       Cisco DCNM network interfaces

Let’s take a look at deployment scenarios supported in Cisco DCNM 11 and how its network interfaces play different roles in different scenarios.

Table 2.       Cisco DCNM POAP use cases

Note:    The OOB POAP scenarios apply to both standalone and High Availability (HA) Cisco DCNM installation modes.

POAP scenario 1: Out-of-Band (OOB) fabric management

In this scenario, use the management port, or an Out-of-Band (OOB) network interface, for fabric management. OOB fabric management is the most commonly used Cisco DCNM deployment model and can be deployed using one of these network configurations:

●   Use two different sets of IP subnets. Use one set of IP subnets to access the Cisco DCNM from an external source or over the Internet. In this case, configure eth0 on Cisco DCNM for external access. Use another set of IP subnets to manage the fabric network only. Configure eth1 on the Cisco DCNM to manage the network through an OOB connection.

●   Use the same IP subnet for external access and for managing the fabric network. In this case, configure eth0 and eth1 to be on the same subnet (for example, eth0:192.168.1.1/24 and eth1:192.168.1.2/24).

Note:    We will use the first OOB network configuration scenario, as shown in Figure 2, as the POAP OOB example.

POAP scenario 2: OOB network management and Layer 3 network between DCNM and fabric

This scenario is similar to POAP scenario 1, and all the steps will remain the same related to POAP and DCNM, but additional DHCP relay configuration will be required on the first-hop Layer 3 network device to send DHCP requests across the Layer 3 network, as shown in Figure 3.

R2 (Spine) example configuration:

interface Ethernet1/1

  no switchport

  mtu 9216

  ip address 192.168.1.1/24

  ip ospf network point-to-point

  ip router ospf UNDERLAY area 0.0.0.0

  ip dhcp relay address 1.53.39.24

  ip dhcp relay address 1.53.39.23

  no shutdown

Note:    DCNM HA mode requires both (active and standby) addresses the DHCP relays on the first-hop L3 router.

Note:    Only the DCNM address will be required as the DHCP relay if DCNM is in standalone mode.

Cisco DCNM POAP bootstrap

The steps to bootstrap Cisco Nexus Switches using POAP are outlined in the steps below.

Step 1:

Create a new fabric

Create the fabric by navigating to Control -> Fabrics-> Fabric Builder as shown in Figure 4.

The Fabric Builder page comes up. When you log in for the first time, the Fabrics section will have no entries (as displayed in Figure 5). After you create a fabric, it is displayed on the Fabric Builder page, wherein a rectangular box represents each fabric (as displayed in Figure 6).

Click the Create fabric button. The Add Fabric screen comes up (Figure 7). The fields are explained below.

Fabric name: Enter the name of the fabric.

Fabric template: This field has template options for creating specific types of fabric. Choose Easy_Fabric. The fabric creation screen for creating a standalone fabric comes up.

Step 2:

Input fabric details

The tabs and their fields in the screen are explained below. Note that overlay and underlay network parameters are included in these tabs.

The General tab is displayed by default.

The fields in this tab are (as shown in Figure 8):

BGP ASN: Enter the BGP AS number the fabric is associated with.

Fabric interface numbering: Specifies whether you want to use point-to-point or unnumbered networks.

Link-State routing protocol: The IGP used in the fabric, OSPF or IS-IS.

Replication mode: The mode of multicast replication used in the fabric, Ingress Replication or Multicast.

Multicast group subnet: Multicast group address of the network.

Anycast gateway MAC: Anycast gateway MAC address.

NX-OS software image version: Select an image from the list.

If you upload Cisco NX-OS software images through the image upload option, the uploaded images are listed in this field. If you select an image, the system checks if the switch has the selected version. If not, an error message is displayed. You can resolve the error by clicking on Resolve. The image management screen comes up and you can proceed with the ISSU option. Alternatively, you can delete the release number if you do not want to enforce a single version.

Click the Bootstrap tab.

The fields in this tab are (as shown in Figure 9):

Enable DHCP: Click this check box to initiate enabling of automatic IP address assignment through DHCP. When you click the check box, the other fields become editable. They are:

DHCP scope start address and DHCP scope end address: Specifies the first and last IP addresses of the IP address range to be used for the Switch Out-of-Band POAP.

Switch management default gateway: Specifies the default gateway for the management VRF on the switch.

Switch Management Subnet Prefix: Specifies the prefix for the Mgmt0 interface on the switch. The prefix should be between 8 and 30.

DHCP scope and management default gateway IP address specification: If you specify the management default gateway IP address 10.1.1.0 and subnet mask 24, ensure that the DHCP scope is within the specified subnet, between 10.1.1.1 and 10.1.1.254.

Step 3:

Save fabric details

Click Save after filling and updating relevant information. A note appears briefly at the bottom right part of the screen, indicating that the fabric is created. When a fabric is created, the fabric page comes up. The fabric name appears at the top left-hand part of the screen (Figure 10 and 11)

Step 4:

POAP bootstrap

At this point, we have prepared the Cisco DCNM POAP configuration and are ready to initiate the zero-touch POAP process on the switch. We can do this in the following ways:

●   Install and turn on a new device

●   Write Erase and Reload an existing device

Click on add switches under the fabric to initiate discovery, using POAP on switches (Figure 12).

Navigate to the PowerON Auto Provisioning (POAP) tab under add switches (Figure 13)

In this use case, we will initiate the Leaf1, Leaf2, Spine1, Spine2 POAP process by sending the Write Erase and Reload command, as shown.

The switches will automatically show up once reloaded. This process can take 5 to 15 minutes.

Enter the password that should be configured to log in to the switches. The discovered switches will show up as seen in Figure 14.  Enter the IP address of the management port (eth0) to be configured on the switch, along with hostname.

Select the respective switches, and click on bootstrap to finish the POAP bootstrap process.

Close this window to navigate to the topology view of the fabric. The selected switches will show up in gray, indicating Unknown status (Figure 15)

Click on Tabular View to see the status of the bootstrap (Figure 16)

As seen in the tabular view (Figure 16), the discovery status of switches shows as discovering.

During this time, the switch console indicates the POAP configuration.

<Snipped console logs>

 

2018 Sep  7 12:43:50 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Start DHCP v4 session

2018 Sep  7 12:43:50 switch %$ VDC-1 %$ %POAP-2-POAP_DHCP_DISCOVER_START: [SAL1926J2J4-D8:B1:90:71:D6:E1] - POAP DHCP Discover phase started

2018 Sep  7 12:43:50 switch %$ VDC-1 %$ %POAP-2-POAP_INFO:   - Abort Power On Auto Provisioning [yes - continue with normal setup, skip - bypass password and basic configuration, no - continue with Power On Auto Provisioning] (yes/skip/no)[no]:

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_DHCP_DISCOVER_RECVD_NAK: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Received DHCP NAK

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ last message repeated 1 time

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Using DHCP, information received over mgmt0 from 10.126.216.248

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Assigned IP address: 10.126.216.167

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Netmask: 255.255.255.0

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - DNS Server: 171.70.168.183

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Default Gateway: 10.126.216.1

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Script Server: 10.126.216.248

2018 Sep  7 12:44:13 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Script Name: poap_dcnm.py

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Using DHCP, information received over mgmt0 from 10.126.216.249

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Assigned IP address: 10.126.216.167

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Netmask: 255.255.255.0

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - DNS Server: 171.70.168.183

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Default Gateway: 10.126.216.1

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Script Server: 10.126.216.248

2018 Sep  7 12:44:15 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Script Name: poap_dcnm.py

2018 Sep  7 12:44:25 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - The POAP Script download has started

2018 Sep  7 12:44:25 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - The POAP Script is being downloaded from [copy tftp://10.126.216.248/poap_dcnm.py bootflash:scripts/script.sh vrf management ]

2018 Sep  7 12:44:26 switch %$ VDC-1 %$ %POAP-2-POAP_SCRIPT_DOWNLOADED: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Successfully downloaded POAP script file

2018 Sep  7 12:44:26 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - Script file size 95089, MD5 checksum 72f5654fb8a49ee5015ceadaf27b6764

2018 Sep  7 12:44:26 switch %$ VDC-1 %$ %POAP-2-POAP_INFO: [SAL1926J2J4-D8:B1:90:71:D6:E1] - MD5 checksum received from the script file is 72f5654fb8a49ee5015ceadaf27b6764

2018 Sep  7 12:44:26 switch %$ VDC-1 %$ %POAP-2-POAP_SCRIPT_STARTED_MD5_VALIDATED: [SAL1926J2J4-D8:B1:90:71:D6:E1] - POAP script execution started(MD5 validated)

2018 Sep  7 12:44:33 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG:   - CLI : show license host-id - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG:   - INFO: Get serial number: SAL1926J2J4 - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO:device type is n9k - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO:device os version is 9.2(1) - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - WARN: https error: [Errno 8] _ssl.c:507: EOF occurred in violation of protocol to DCNM server - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ last message repeated 1 time

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: check free space - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: free space is 44756684 kB - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - Get and set interface default  - script.sh

2018 Sep  7 12:44:34 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : show run | inc breakout - script.sh

2018 Sep  7 12:44:35 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : show run int | inc Ethernet - script.sh

2018 Sep  7 12:44:36 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Ready to copy protocol scp, host 10.126.216.248, source /var/lib/dcnm/dcnm-server-list.cfg vrf management user root password ***** - script.sh

2018 Sep  7 12:44:36 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : terminal dont-ask ; terminal password ***** ; copy scp//root@10.126.216.248/var/lib/dcnm/dcnm-server-list.cfg dcnm-server-list.cfg vrf management - script.sh

2018 Sep  7 12:44:37 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Get Device Image Config File - script.sh

2018 Sep  7 12:44:37 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: removing tmp file /bootflash/dcnm-server-list.cfg - script.sh

2018 Sep  7 12:44:37 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Ready to copy protocol scp, host 10.126.216.248, source /var/lib/dcnm/SAL1926J2J4/device-recipe.cfg vrf management user root password ***** - script.sh

2018 Sep  7 12:44:37 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : terminal dont-ask ; terminal password ***** ; copy scp: //root@10.126.216.248/var/lib/dcnm/SAL1926J2J4/device-recipe.cfg device-recipe.cfg vrf management - script.sh

2018 Sep  7 12:44:38 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Get Device Recipe - script.sh

2018 Sep  7 12:44:38 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: removing tmp file /bootflash/device-recipe.cfg - script.sh

2018 Sep  7 12:44:38 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: create_image_conf - script.sh

2018 Sep  7 12:44:38 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Ready to copy protocol scp, host 10.126.216.248, source /var/lib/dcnm/licenses/device-license.idx vrf management user root password ***** - script.sh

2018 Sep  7 12:44:38 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : terminal dont-ask ; terminal password ***** ; copy scp: //root@10.126.216.248/var/lib/dcnm/licenses/device-license.idx device-license.idx vrf management - script.sh

2018 Sep  7 12:44:39 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: device license index does not exist, no device licenses will be downloaded - script.sh

2018 Sep  7 12:44:39 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Ready to copy protocol scp, host 10.126.216.248, source /var/lib/dcnm/SAL1926J2J4/device-config vrf management user root password ***** - script.sh

2018 Sep  7 12:44:39 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : terminal dont-ask ; terminal password ***** ; copy scp: //root@10.126.216.248/var/lib/dcnm/SAL1926J2J4/device-config device-config vrf management - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Completed Copy of Config File - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Split config invoked.... - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - Found an interface line in config:interfacemgmt0  - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - Adding interface defaults - no shut on all interfaces - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Split config is complete - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Setting the boot variables - script.sh

2018 Sep  7 12:44:41 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : copy running-config startup-config - script.sh

2018 Sep  7 12:44:44 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : copy poap_2.cfg scheduled-config - script.sh

2018 Sep  7 12:44:45 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Copying the scheduled cfg done - script.sh

2018 Sep  7 12:44:45 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - INFO: Configuration successful - script.sh

2018 Sep  7 12:44:45 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - FINISH: Clean up files. - script.sh

2018 Sep  7 12:44:45 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : delete device-config - script.sh

2018 Sep  7 12:44:45 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : delete poap_1.cfg - script.sh

2018 Sep  7 12:44:46 switch %$ VDC-1 %$ %USER-1-SYSTEM_MSG: S/N[SAL1926J2J4] - CLI : delete poap_2.cfg - script.sh

2018 Sep  7 12:44:49 switch %$ VDC-1 %$ %POAP-2-POAP_SCRIPT_EXEC_SUCCESS: [SAL1926J2J4-D8:B1:90:71:D6:E1] - POAP script execution success

Once the bootstrap is complete, the Discovery Status changes to ok and all fields are populated: IP address, Model, Serial Number, and Software version (Figure 17).

Note:    The default Role for switches in DCNM is leaf. The role can be changed from the topology view by right clicking on the switch (see Figure 18). This must be done for all of the switches in the fabric.

Step 5:

Save and deploy

This step is needed in order to complete the second phase of the fabric bring-up (post POAP). The base configuration for VXLAN (underlay) is pushed at this time. The initial phase of POAP (Steps 1 through 4) only pushes the basic manageability. Click on Save and deploy to analyse the configurations (see Figure 19) based on the roles defined in the previous step, and deploy these.

 

 

The configurations can be previewed to ensure that these are as intended. If yes, click on deploy to push the configurations to all of the switches (or per switch, by right-clicking and selecting deploy config). Once deployed, the history of each switch can be viewed by navigating to the topology view, right-clicking on the switch, and selecting history (Figure 20).