Step 1

Download the Cisco Nexus Dashboard image.

1. Browse to the Software Download page.

   https://software.cisco.com/download/home/286327743/type/286328258

2. Click Nexus Dashboard Software.

3. From the left sidebar, choose the Nexus Dashboard version you want to download.

4. Download the Cisco Nexus Dashboard image for Linux KVM (nd-dk9.<version>.qcow2).

Step 2

Copy the image to the Linux KVM servers where you will host the nodes.

# scp nd-dk9.<version>.qcow2 root@<kvm-host-ip>:/home/nd-base

Step 3

Make the following configurations on each KVM host:

1. Edit /etc/libvirt/qemu.conf and make sure the user and group is correctly configured based on the ownership of the storage that you plan to use for the Nexus Dashboard deployment.

   This is only required if you plan to use disk storage paths that are different from the default libvirtd.

2. Edit /etc/libvirt/libvirt.conf and uncomment uri_default.

3. Restart the libvirtd service after updating the configuration using the systemctl restart libvirtd command from root.

Step 4

Create the required disk images on each node.

Step 5

Log in to your KVM host as the root user and perform the following steps on each node.

1. Mount the storage disk (raw disk or LVM) to directory /opt/cisco/nd.

2. Create the following script as /root/create_vm.sh under the root directory.

   Note	If you manually type this information, verify that there are no empty spaces present after any of these lines.

   Create the script based on the information provided in Understanding system resources for your form factor:

   • For 1-node or 3-node KVM (app) form factors:

     #!/bin/bash -ex
     
     # Configuration
     # Name of Nexus Dashboard Virtual machine
     name=nd1
     
     # Path of Nexus Dashboard QCOW2 image.
     nd_qcow2=/home/nd-base/nd-dk9.4.1.1i.qcow2
     
     # Disk Path to storage Boot and Data Disks.
     data_disk=/opt/cisco/nd/data
     
     # Management Network Bridge
     mgmt_bridge=mgmt-bridge
     
     # Data Network bridge
     data_bridge=data-bridge
     
     # Data Disk Size
     data_size=500G
     
     # CPU Cores
     cpus=16
     
     # Memory in units of MB.
     memory=65536
     
     # actual script
     rm -rf $data_disk/boot.img
     /usr/bin/qemu-img convert -f qcow2 -O raw $nd_qcow2 $data_disk/boot.img
     rm -rf $data_disk/disk.img
     /usr/bin/qemu-img create -f raw $data_disk/disk.img $data_size
     virt-install \
     --import \
     --name $name \
     --memory $memory \
     --vcpus $cpus \
     --os-type generic \
     --osinfo detect=on,require=off \
     --check path_in_use=off \
     --disk path=${data_disk}/boot.img,format=raw,bus=virtio \
     --disk path=${data_disk}/disk.img,format=raw,bus=virtio \
     --network bridge=$mgmt_bridge,model=virtio \
     --network bridge=$data_bridge,model=virtio \
     --console pty,target_type=serial \
     --noautoconsole \
     --autostart

   • For 1-node or 3-node KVM (data) form factors:

     #!/bin/bash -ex
     
     # Configuration
     # Name of Nexus Dashboard Virtual machine
     name=nd1
     
     # Path of Nexus Dashboard QCOW2 image.
     nd_qcow2=/home/nd-base/nd-dk9.4.1.1i.qcow2
     
     # Disk Path to storage Boot and Data Disks.
     data_disk=/opt/cisco/nd/data
     
     # Management Network Bridge
     mgmt_bridge=mgmt-bridge
     
     # Data Network bridge
     data_bridge=data-bridge
     
     # Data Disk Size
     data_size=3072G
     
     # CPU Cores
     cpus=32
     
     # Memory in units of MB.
     memory=131072
     
     # actual script
     rm -rf $data_disk/boot.img
     /usr/bin/qemu-img convert -f qcow2 -O raw $nd_qcow2 $data_disk/boot.img
     rm -rf $data_disk/disk.img
     /usr/bin/qemu-img create -f raw $data_disk/disk.img $data_size
     virt-install \
     --import \
     --name $name \
     --memory $memory \
     --vcpus $cpus \
     --os-type generic \
     --osinfo detect=on,require=off \
     --check path_in_use=off \
     --disk path=${data_disk}/boot.img,format=raw,bus=virtio \
     --disk path=${data_disk}/disk.img,format=raw,bus=virtio \
     --network bridge=$mgmt_bridge,model=virtio \
     --network bridge=$data_bridge,model=virtio \
     --console pty,target_type=serial \
     --noautoconsole \
     --autostart

Step 6

Repeat previous steps to deploy the second and third nodes, then start all VMs.

Step 7

Open one of the node's console and configure the node's basic information.

1. Press any key to begin initial setup.

   You will be prompted to run the first-time setup utility:

   [ OK ] Started atomix-boot-setup.
          Starting Initial cloud-init job (pre-networking)...
          Starting logrotate...
          Starting logwatch...
          Starting keyhole...
   [ OK ] Started keyhole.
   [ OK ] Started logrotate.
   [ OK ] Started logwatch.
   
   Press any key to run first-boot setup on this console...

2. Enter and confirm the admin password

   This password will be used for the rescue-user SSH login as well as the initial GUI password.

   Note	You must provide the same password for all nodes or the cluster creation will fail.

   Admin Password:
   Reenter Admin Password:

3. Enter the management network information.

   Management Network:
     IP Address/Mask: 192.168.9.172/24
     Gateway: 192.168.9.1

4. For the first node only, designate it as the "Cluster Leader".

   You will log into the cluster leader node to finish configuration and complete cluster creation.

   Is this the cluster leader?: y

5. Review and confirm the entered information.

   You will be asked if you want to change the entered information. If all the fields are correct, choose n to proceed. If you want to change any of the entered information, enter y to re-start the basic configuration script.

   Please review the config
   Management network:
     Gateway: 192.168.9.1
     IP Address/Mask: 192.168.9.172/24
   Cluster leader: yes
   
   Re-enter config? (y/N): n

Step 8

Repeat previous step to configure the initial information for the second and third nodes.

Step 9

Wait for the initial bootstrap process to complete on all nodes.

Please wait for system to boot: [#########################] 100%
System up, please wait for UI to be online.

System UI online, please login to https://192.168.9.172 to continue.

Step 10

Open your browser and navigate to https://<node-mgmt-ip> to open the GUI.

Step 11

Enter the requested information in the Basic Information page of the Cluster Bringup wizard.

1. For Cluster Name, enter a name for this Nexus Dashboard cluster.

   The cluster name must follow the RFC-1123 requirements.

2. For Select the Nexus Dashboard Implementation type, choose either LAN or SAN then click Next.

Step 12

Enter the requested information in the Configuration page of the Cluster Bringup wizard.

1. (Optional) If you want to enable IPv6 functionality for the cluster, put a check in the Enable IPv6 checkbox.

2. Click +Add DNS provider to add one or more DNS servers, enter the DNS provider IP address, then click the checkmark icon.

3. (Optional) Click +Add DNS search domain to add a search domain, enter the DNS search domain IP address, then click the checkmark icon.

4. (Optional) If you want to enable NTP server authentication, put a check in the NTP Authentication checkbox.

5. If you enabled NTP authentication, click + Add Key, enter the required information, and click the checkmark icon to save the information.

   • Key–Enter the NTP authentication key, which is a cryptographic key that is used to authenticate the NTP traffic between the Nexus Dashboard and the NTP servers. You will define the NTP servers in the following step, and multiple NTP servers can use the same NTP authentication key.

   • ID–Enter a key ID for the NTP host. Each NTP key must be assigned a unique key ID, which is used to identify the appropriate key to use when verifying the NTP packet.

   • Authentication Type–Choose authentication type for the NTP key.

   • Put a check in the Trusted checkbox if you want this key to be trusted. Untrusted keys cannot be used for NTP authentication.

   For the complete list of NTP authentication requirements and guidelines, see General prerequisites and guidelines.

   If you want to enter additional NTP keys, click + Add Key again and enter the information.

6. If you enabled NTP authentication, click +Add NTP Host Name/IP Address, enter the required information, and click the checkmark icon to save the information.

   • NTP Host–Enter an IP address; fully qualified domain names (FQDN) are not supported.

   • Key ID–Enter the key ID of the NTP key you defined in the previous substep.

     If NTP authentication is disabled, this field is grayed out.

   • Put a check in the Preferred checkbox if you want this host to be preferred.

   Note

   If the node into which you are logged in is configured with only an IPv4 address, but you have checked Enable IPv6 in a previous step and entered an IPv6 address for an NTP server, you will get the following validation error: This is because the node does not have an IPv6 address yet and is unable to connect to an IPv6 address of the NTP server. You will enter IPv6 address in the next step. In this case, enter the other required information as described in the following steps and click Next to proceed to the next page where you will enter IPv6 addresses for the nodes.

   If you want to enter additional NTP servers, click +Add NTP Host Name/IP Address again and enter the information.

7. For Proxy Server, enter the URL or IP address of a proxy server.

   For clusters that do not have direct connectivity to Cisco cloud, we recommend configuring a proxy server to establish the connectivity. This allows you to mitigate risk from exposure to non-conformant hardware and software in your fabrics.

   You can click +Add Ignore Host to enter one or more destination IP addresses for which traffic will skip using the proxy.

   The proxy server must have these URLs enabled:

   dcappcenter.cisco.com
   svc.intersight.com
   svc.ucs-connect.com
   svc-static1.intersight.com
   svc-static1.ucs-connect.com

   If you do not want to configure a proxy, click Skip Proxy then click Confirm.

8. (Optional) If your proxy server requires authentication, put a check in the Authentication required for Proxy checkbox and enter the login credentials.

9. (Optional) Expand the Advanced Settings category and change the settings if required.

   Under advanced settings, you can configure these settings:

   • App Network–The address space used by the application's services running in the Nexus Dashboard. Enter the IP address and netmask.

   • Service Network–An internal network used by Nexus Dashboard and its processes. Enter the IP address and netmask.

   • App Network IPv6–If you put a check in the Enable IPv6 checkbox earlier, enter the IPv6 subnet for the app network.

   • Service Network IPv6–If you put a check in the Enable IPv6 checkbox earlier, enter the IPv6 subnet for the service network.

   For more information about the application and service networks, see General prerequisites and guidelines.

10. Click Next.

Step 13

In the Node Details page, update the first node's information.

1. For Cluster Connectivity, if your cluster is deployed in L3 HA mode, choose BGP. Otherwise, choose L2.

   BGP configuration is required for the persistent IP addresses feature used by telemetry. This feature is described in more detail in BGP configuration and persistent IP addresses and the "Persistent IP Addresses" sections of the Cisco Nexus Dashboard User Guide.

   Note	You can enable BGP at this time or in the Nexus Dashboard GUI after the cluster is deployed. All remaining nodes need to configure BGP if it is configured. You must enable BGP now if the data network of nodes have different subnets.

2. Click the Edit button next to the first node.

   The node's Serial Number, Management Network information, and Type are automatically populated, but you must enter the other information.

3. For Name, enter a name for the node.

   The node's Name will be set as its hostname, so it must follow the RFC-1123 requirements.

   Note	If you need to change the name but the Name field is not editable, run the CIMC validation again to fix this issue.

4. For Type, choose Primary.

   The first nodes of the cluster must be set to Primary. You will add the secondary nodes in a later step if required for higher scale.

5. In the Data Network area, enter the node's data network information.

   Enter the data network IP address, netmask, and gateway. Optionally, you can also enter the VLAN ID for the network. Leave the VLAN ID field blank if your configuration does not require VLAN. If you chose BGP for Cluster Connectivity, enter the ASN.

   If you enabled IPv6 functionality in a previous page, you must also enter the IPv6 address, netmask, and gateway.

   Note	If you want to enter IPv6 information, you must do so during the cluster bootstrap process. To change the IP address configuration later, you would need to redeploy the cluster. All nodes in the cluster must be configured with either only IPv4, only IPv6, or dual stack IPv4/IPv6.

6. If you chose BGP for Cluster Connectivity, then in the BGP peer details area, enter the peer's IPv4 address and ASN.

   You can click + Add IPv4 BGP peer to add addition peers.

   If you enabled IPv6 functionality in a previous page, you must also enter the peer's IPv6 address and ASN.

7. Click Save to save the changes.

Step 14

In the Node Details screen, click Add Node to add the second node to the cluster.

If you are deploying a single-node cluster, skip this step.

1. In the Deployment Details area, provide the Management IP Address and Password for the second node

   You defined the management network information and the password during the initial node configuration steps.

2. Click Validate to verify connectivity to the node.

   The node's Serial Number and the Management Network information are automatically populated after connectivity is validated.

3. Provide the Name for the node.

4. From the Type dropdown, select Primary.

   The first 3 nodes of the cluster must be set to Primary. You will add the secondary nodes in a later step if required for higher scale.

5. In the Data Network area, provide the node's Data Network information.

   You must provide the data network IP address, netmask, and gateway. Optionally, you can also provide the VLAN ID for the network. For most deployments, you can leave the VLAN ID field blank.

   If you had enabled IPv6 functionality in a previous screen, you must also provide the IPv6 address, netmask, and gateway.

   Note	If you want to provide IPv6 information, you must do it during cluster bootstrap process. To change IP configuration later, you would need to redeploy the cluster. All nodes in the cluster must be configured with either only IPv4, only IPv6, or dual stack IPv4/IPv6.

6. (Optional) If your cluster is deployed in L3 HA mode, Enable BGP for the data network.

   BGP configuration is required for the persistent IP addresses feature. This feature is described in more detail in BGP configuration and persistent IP addresses and the "Persistent IP Addresses" sections of the Cisco Nexus Dashboard User Guide.

   Note	You can enable BGP at this time or in the Nexus Dashboard GUI after the cluster is deployed.

   If you choose to enable BGP, you must also provide the following information:

   • ASN (BGP Autonomous System Number) of this node.

     You can configure the same ASN for all nodes or a different ASN per node.

   • For pure IPv6, the Router ID of this node.

     The router ID must be an IPv4 address, for example 1.1.1.1

   • BGP Peer Details, which includes the peer's IPv4 or IPv6 address and peer's ASN.

7. Click Save to save the changes.

8. Repeat this step for the final (third) primary node of the cluster.

Step 15

In the Node Details page, verify the information that you entered, then click Next.

Step 16

Choose the Deployment Mode for the cluster.

1. Click Add Persistent Service IPs/Pools to provide the required persistent IP addresses.

   For more information about persistent IP addresses, see the Nexus Dashboard persistent IP addresses section.

2. Click Next to proceed.

Step 17

In the Summary screen, review and verify the configuration information and click Save to build the cluster.

Step 18

Verify that the cluster is healthy.

After the cluster becomes available, you can access it by browsing to any one of your nodes' management IP addresses. The default password for the admin user is the same as the rescue-user password you chose for the first node. During this time, the UI will display a banner at the top stating "Service Installation is in progress, Nexus Dashboard configuration tasks are currently disabled".

After all the cluster is deployed and all services are started, you can look at the Anomaly Level on the Home > Overview page to ensure the cluster is healthy:

Alternatively, you can log in to any one node using SSH as the rescue-user using the password you entered during node deployment and using the acs health command to see the status:

• While the cluster is converging, you may see the following output:

  $ acs health
  k8s install is in-progress
  
  $ acs health
  k8s services not in desired state - [...]
  
  $ acs health
  k8s: Etcd cluster is not ready

• When the cluster is up and running, the following output will be displayed:

  $ acs health
  All components are healthy

Note

In some situations, you might power cycle a node (power it off and then back on) and find it stuck in this stage: deploy base system services This is due to an issue with etcd on the node after a reboot of the physical Nexus Dashboard cluster. To resolve the issue, enter the acs reboot clean command on the affected node.

Step 19

(Optional) Connect your Cisco Nexus Dashboard cluster to Cisco Intersight for added visibility and benefits. Refer to Working with Cisco Intersight for detailed steps.

Step 20

After you have deployed Nexus Dashboard, see the collections page for this release for configuration information.