Installing Cisco APIC-EM on Bare-Metal Hardware

About the Bare-Metal Hardware Installation

You can install the Cisco APIC-EM on a server (bare-metal hardware) and then deploy it within your network. The Cisco APIC-EM can be deployed as a single host (single server) or within a multi-host environment (multiple servers).


Important

We recommend that you install and deploy Cisco APIC-EM in a multi-host environment for enhanced scalability and redundancy. For information about multi-host support, see Multi-Host Support.

The following table lists the steps for installing the Cisco APIC-EM on a server (bare-metal hardware).

Table 1. Cisco APIC-EM Bare-Metal Hardware Installation

Step

Description

1

Review Cisco UCS server support for Cisco APIC-EM. See Cisco UCS Server Support for Cisco APIC-EM.

2

Review the listed considerations for a bare metal installation. See Before You Begin a Bare Metal Installation.

3

Review the system requirements for a bare-metal hardware installation. See System Requirements—Server (Bare-Metal Hardware).

4

Review the pre-install checklists for the installation (standalone and multi-host modes). See Pre-Install Checklists.

5

Download and verify the ISO image. See Verifying the Cisco ISO Image.

6

Install the ISO image. See Installing the Cisco ISO Image.

7

Proceed to configure the Cisco APIC-EM in standalone or multi-host mode. Refer to the following sections for information about the configuration wizard process:

Cisco UCS Server Support for Cisco APIC-EM

The Cisco APIC-EM is available as an ISO image that can be downloaded from Cisco.com and installed on any Cisco UCS server that meets the minimum server (bare-metal hardware) requirements as listed in the following section.

Cisco APIC-EM has been tested and qualified to run on the following Cisco UCS servers:

  • Cisco UCS C220 M5 Server

Before You Begin a Bare Metal Installation

Before you begin your bare-metal hardware installation, note the following:

  • You must configure RAID on your bare metal hardware before you begin the installation process.

  • If you're using the Cisco server UCS C220 M4S, then change the LOM from shared to dedicated, and change the port speed to Auto-negotiate or 1 Gbps.

  • After starting the installation of the ISO, the Cisco APIC-EM software packages will be installed. The bare metal hardware system will reset twice.


    Important

    Do not press any the Escape key or any other keys on the keyboard once the installation process has started. Pressing a key will cause the system logs to be displayed during the installation process.

  • If you are installing the Cisco APIC-EM ISO using a bootable disk, then you may encounter a known issue with mounting the media (bootable disk). If this occurs, you will receive the following error message: 

    There was a problem reading data from the CD-ROM. 
Please make sure it is in the drive. If retrying does not work, 
you should check the integrity of your CD-ROM.

Failed to copy file from CD-ROM. Retry?

    If this occurs, then you need to unmount the media and mount it as a CD-ROM. For example, the following Linux commands can be used to mount the media as a CD-ROM. Log onto the bare metal console to enter these commands.

    1. mount

    2. unmount /dev/<sda>/

    3. mount /dev/<sda> /cdrom

    The following is an example of entering these Linux commands: 

    
~ #
~ # mount

rootfs on / type rootfs (rw, size=32927728k,nr_inodes=823192)
none on /run type tmpfs (rw, nousid, relatime, size=6586808k,mode=755)
none on /proc type proc (rw, relatime)
none on /sys type sysfs (rw, relatime)
devtmpfs on /dev type devtmpfs (rw, relatime, size=32927744k, nr_inodes-8231936, mode=755)
devpts on /dev/pts type devpts (re, nosuid, noexec, relatime, gid=5, mode=620, ptmxmode=000)
/dev/sr0 on /media type iso9660 (ro, relatime)

~ #
~ #
~ # umount /dev/sr0
~ # mount /dev/sr0/cdrom
~ #
~ #
~ #
~ #
~ #
~ #

    Note

    For information about and an example of creating a bootable USB disk from a USB flash drive and attaching the ISO to it, see Creating a Bootable USB Disk and Attaching the ISO. For additional information about this issue, see https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1347726.

  • During the installation process, you may be prompted to install the Linux GNU GRUB boot loader package. If so prompted, select the option to install the GRUB boot loader package and proceed with the installation.

System Requirements—Server (Bare-Metal Hardware)

The following table lists the minimum system requirements for a successful Cisco APIC-EM server (bare-metal hardware) installation. The minimum system requirements for each server in a multi-host deployment are the same as in a single-host deployment, except that the multi-host deployment requires two or three servers.


Note

The three server, multi-host deployment provides both software and hardware high availability. The two server, multi-host deployment only provides software high availability and does not provide hardware high availability. For this reason, we strongly recommend that for a multi-host deployment three servers be used. With either two or three servers, all of the servers must reside in the same subnet.


Caution

You must dedicate the entire server for the Cisco APIC-EM. You cannot use the server for any other software programs, packages, or data. During the Cisco APIC-EM installation, any other software programs, packages or data on the server will be deleted.

Table 2. Minimum System Requirements—Server

Server Option

Image Format

Bare metal/ISO

Hardware Specifications

Processors

Up to two CPUs from the Intel Xeon Processor Scalable Family. This includes CPUs from the following series:

  • Intel Xeon Bronze 3XXX Processors

  • Intel Xeon Silver 4XXX Processors

  • Intel Xeon Gold 5XXX Processors

  • Intel Xeon Gold 6XXX Processors

  • Intel Xeon Platinum 8XXX Processors

Memory

24 slots for registered DIMMs (RDIMMs), load-reduced DIMMs (LRDIMMs), or through silicon via (TSV) DIMMs.

Storage

10 2.5-inch Small Form Factor (SFF) SAS and SATA hard drives and solid state drives (SSD).

Disk Management (RAID)

  • Internal controllers: Cisco 12-Gbps Modular RAID Controller (PCIe 3.0) with 1- or 2-GB Flash-Backed Write Cache (FBWC), providing enterprise-class data protection for up to 10 SAS and SATA Hard-Disk Drives (HDDs), SSDs, or NVMe PCIe SSDs; or Cisco 12-Gbps Modular SAS Host Bus Adapter (HBA)

  • External controller: Cisco 12-Gbps 9400-8e SAS HBA

Network and Management I/O

Supported connectors:

  • Two 10 Gbps Ethernet ports on the Intel X710-DA2 NIC

  • One 1 Gbps RJ-45 management port (Marvell 88E6176)

  • Two 10GBase-T LOM ports (Intel X550 controller embedded on the motherboard)

The following connectors are available but not typically used in day-to-day operation:

  • One RS-232 serial port (RJ-45 connector)

  • One VGA (DB-15) connector

  • Two USB 3.0 connectors

  • One front-panel KVM connector that is used with the KVM cable, which provides two USB 2.0, one VGA (DB-15), and one serial port (RS-232) RJ-45 connector.

Note that the Intel X710-DA4 NIC, which provides four 10 Gbps Ethernet ports, has been disabled.

Power

Dual AC power supplies, 770 W AC each.

Do not mix power supply types or watt ratings in the server.

Redundant as 1+1.

Cooling

Seven hot-swappable fan modules for front-to-rear cooling.

Video

VGA video resolution up to 1920 x 1200, 16 bpp at 60 Hz, and up to 512 MB of video memory (8 MB is allocated, by default).

Networking

Web Access

Required

Browser

The following browsers are supported when viewing and working with the Cisco APIC-EM:

  • Google Chrome, version 56.0 or later

  • Mozilla Firefox, version 51.0 or later

Pre-Install Checklists

Standalone Mode Checklists

Review the following checklists before beginning a single-host Cisco APIC-EM installation (standalone mode).


Note

A host is defined as an appliance, physical server, or virtual machine with instances of a Grapevine root and clients running. The Grapevine root is located in the host OS and the clients are located within Linux containers. The clients run the services within the Linux containers. You can set up either a single-host deployment or multi-host deployment (2 or 3 hosts) for your network. For high availability and scale, your multi-host deployment must contain three hosts. All inbound traffic to the controller in a single-host deployment is through the host IP address that you configure using the configuration wizard. All inbound traffic to the controller in a multi-host deployment is through a Virtual IP that you configure using the configuration wizard.

Networking Requirements

This Cisco APIC-EM installation requires that the network adapters (NICs) on the host (physical or virtual) are connected to the following networks:

  • Internet (network access required for Make A Wish requests and telemetry collection)

  • Network with NTP server(s)

  • Network with devices that are to be managed by the Cisco APIC-EM


Note

The Cisco APIC-EM should never be directly connected to the Internet. It should not be deployed outside of a NAT configured or protected datacenter environment.

IP Address Requirements

Ensure that you have available at least one IP address for the network adapter (NIC) on the host.

The IP address is used as follows:

  • Direct access to the Grapevine root

  • Direct access to the Cisco APIC-EM controller (for GUI access)


Note

If your host has 2 NICs, then you may want to have two IP addresses available and configure one IP address for each NIC.

Multi-Host Mode Checklists

Review the following checklist before beginning a multi-host Cisco APIC-EM installation (multi-host mode).

  • You must satisfy the requirements for the single-host installation as described in the previous section for each host.

  • Additionally, you must establish a network connection between each of the hosts using either a switch or a router. Each host must be routable with the other two hosts.

  • You must configure a virtual IP (VIP).

    You configure one or more NICs on each host using the configuration wizard. Each NIC that you configure must point to a non-routable network (if all your networks are routable, then you only need one NIC). A VIP is required per non-routable network. For example, if you configure 2 NICs on all 3 hosts in a multi-host cluster and each NIC points to a separate, non-routable network, then you need to configure 2 VIPs. The VIP provides an interface redundancy feature for your multi-host deployment. With a VIP, the IP address can float between the hosts.

    When deploying the controller in a multi-host configuration:

    • You provide a VIP address when configuring the controller using the wizard.

    • On startup, the controller will bring up the VIP on one of the hosts.

    • All inbound requests into controller from the external network are made via this VIP (instead of the host IP address), and the requests are routed to the services running on different hosts via the reverse-proxy service.

    • If the host on which has the VIP fails, then Grapevine will bring up the VIP on one of the remaining two hosts.

    • The VIP must reside in the same subnet as the three hosts.

    • If you are planning to obtain a certificate issued for a multi-host environment, then it is important to get the certificate issued against the virtual IP or the host name resolvable to the virtual IP.

    • For a multi-host configuration with Cisco APIC-EM located behind a NAT within your network, note the following information and requirement:

      • The Virtual IP address of the Cisco APIC-EM controller is intended as a destination address for HTTP(S) traffic such as Cisco PnP and PKI download requests.

      • Any outbound connections initiated from the Cisco APIC-EM controller, such as during a Discovery, Inventory Collection, etc., will use the host IP address of one of the three Cisco APIC-EM hosts.

      • Therefore, you need to PAT (Port Address Translation) the host IP addresses of the Cisco APIC-EM hosts to a global public facing IP address for outbound connections from Cisco APIC-EM controller.

Multi-Host Deployment Virtual IP

A multi-host deployment has three physical IP addresses and one virtual IP that floats across the IP addresses by design in order to provide high availability. This capability to float also means that any SSH client that wants to connect to the virtual IP address will see different host-identity public SSH keys each time the virtual IP moves its residence from one host to another host. Most SSH clients will complain that the new host is not trusted, since an entry already exists (as you might have accepted the key earlier for the older host which owned that virtual IP address before). To prevent this inconvenience, you may want to add the host keys of all the three hosts to your known hosts list as described below.

For example on a Linux or Apple Mac OS client machine, run the ssh-keyscan command on each of the three host physical IP addresses as follows: 


$ ssh-keyscan -t rsa 209.165.200.30
# 209.165.200.30 SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.3
209.165.200.30 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDA1B6/1JpKPFOmG3S82eE8OKZkGYmRd
SYnuCHfDiY5Pptt3BmaPgC6OlER4wwDL8VP2Rx2kxj3diIzFpUOyDqTbFxIRKVzlwtHHZdhO6G93MyLLGsWq
XSMWs4xVcqpembKeCrdjakPaPAXqiAeKW9oimdv.....



$ ssh-keyscan -t rsa 209.165.200.31
# 209.165.200.31 SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.3
209.165.200.31 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF57F90z2His86tEj4s75pTc7h0nfzF
2c3QweHCNN2ov474HJJcPrnWTw4DAoPpPCU6zWvR0QLxunURDb+pMeZrIIyd49xn9+OBSmBpzrnety7UB2uP
XzL1RvVxayw8mkXkj779LhFh9vkXR4DtX7XLjg.....



$ ssh-keyscan -t rsa 209.165.200.32
# 209.165.200.32 SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.3
209.165.200.32 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9kwzodGzGkh/UFXVa9fptGe+sa3CBR
6SNerXxpCmfT9AOXH8xuk3/CBX+DDUQgGJVmqw6maCYKOy0RtAhGxdsNdPL6ETTKzxYB5uzw3KhcDJ6D6ob6
jdzkR6yRuXVFi2OE+u1Aqs7J8GO66FfdavU8.....

Next, change the IP address in the SSH key line of each output to the virtual IP address of the following and append all three key lines to the ~/.ssh/known_hosts file and save it.

Assuming that 209.165.200.33 is the virtual IP address in the above multi-host example, you would add three lines in the ~/.ssh/known_hosts file of your client machine as follows: 


209.165.200.33 ssh-rsa 

AAAAB3NzaC1yc2EAAAADAQABAAABAQDA1B6/1JpKPFOmG3S82eE8OKZkGYmRdSYnuCHfDiY5Pptt3BmaPgC6OlER4
wwDL8VP2Rx2kxj3diIzFpUOyDqTbFxIRKVzlwtHHZdhO6G93MyLLGsWqXSMWs4xVcqpembKeCrdjakPaPAXqiAeKW9
oimdvPbrQPua7Zg9oblDxaBPn0Fqj00YDjKqTkp/IkZHEfHbDM996GLEbWlOvoHeCCqeZ1nWgFIqzAF+ty8+X5Z/fh
hmGe+w2tQlMfrs9pcZDaEEmq/w1W+uRohxLKs+OHnHYAbMzC6O+5fLEr2BwaZf8W016eo1WpPsxUVK6StbXBOQZrcH0
bPsUbIjKJkzafpft9Dp73pSd/vwaoB3DrvNec/PiEJYk+R.....

After the above change, the client will have no trouble performing uninterrupted SSH into the virtual IP address of the hosts even with the IP address floating.

Verifying the Cisco ISO Image

Prior to deploying the Cisco APIC-EM, verify that the ISO image that you downloaded is a genuine Cisco image.


Note

If you are deploying the Cisco APIC-EM from an ISO image that you downloaded, then perform this procedure. This procedure is not required, if deploying the controller with the Cisco APIC-EM Controller Appliance (Cisco APIC-EM ISO image pre-installed and tested).

Before you begin

You must have received notification of the location of the Cisco APIC-EM ISO image or contacted Cisco support for the location of the Cisco APIC-EM ISO image.

Procedure

Step 1

Download the ISO image from the location specified by Cisco.

Step 2

Download the Cisco public key for signature verification from the location specified by Cisco.

The Cisco public key is named:

cisco_image_verification_key.pub

Step 3

Obtain the secure hash algorithm (SHA512) checksum file for the ISO image from the location specified by Cisco.

Step 4

Obtain the specific release ISO image's signature file from Cisco support via email or by download from the secure Cisco website (if available).

For example, apic-em-CA-0.8.2.4704-0.1.0.15.dev1300-gaafbb68.sig.

Step 5

(Optional) Perform a SHA verification to determine whether the ISO image was corrupted due to a partial download.

For example, run one of the following commands (depending upon your operating system):

  • On a system running MAC OS X version: 

    shasum -a 512 apic-em-CA-0.8.2.4704-0.1.0.15.dev1300-gaafbb68.iso

  • On a Linux system: 

    sha512sum apic-em-CA-0.8.2.4704-0.1.0.15.dev1300-gaafbb68.iso

Microsoft Windows does not include a built-in checksum utility, but you can install a utility from Microsoft at this link: http://www.microsoft.com/en-us/download/details.aspx?id=11533

Compare the output of the above command (or Microsoft Windows utility) to the SHA512 checksum file obtained earlier in step 3. If the command output fails to match, download the ISO image again and run the appropriate command a second time. If the output still fails to match, contact Cisco support.

Step 6

Verify that the ISO image is genuine and from Cisco by verifying the signature. Run the following command on the ISO image: 

openssl dgst -sha512 -verify cisco_image_verification_key.pub -signature  apic-em-CA-0.8.2.4704-0.1.0.15.dev1300-gaafbb68.sig apic-em-CA-0.8.2.4704-0.1.0.15.dev1300-gaafbb68.iso

If the ISO image is genuine, then running this command should result in a Verified OK message. If this message fails to appear, then do not install the ISO image and contact Cisco support.

Note 

The image name and the signature names used here are only examples. Use the exact names of these files that you downloaded from the Cisco website.

This command will work in both MAC and Linux environments. For Windows, you need to download and implement OpenSSL from www.openssl.org, if you have not already done so.

What to do next

After you verify that the ISO image is genuine and from Cisco, install the Cisco ISO image.

Installing the Cisco ISO Image

Perform the steps in the following procedure to install the Cisco ISO image on the host (bare-metal hardware or server).


Note

If you are deploying the Cisco APIC-EM from an ISO image that you downloaded, then perform this procedure. This procedure is not required, if deploying the controller with the Cisco APIC-EM Controller Appliance (ISO image pre-installed and tested).

Before you begin

You must review the system requirements before beginning this procedure.

You must review the Cisco APIC-EM pre-deployment checklist before beginning this procedure.

You must have downloaded and verified the Cisco ISO image by performing the tasks in the previous procedure.

Procedure

Step 1

Burn the ISO image onto a DVD or copy it onto a bootable USB disk.

Step 2

If the ISO image was burned onto a DVD, then insert the DVD into the DVD drive of the server.

Note 

If your server does not come with a DVD drive, you can connect an external USB DVD drive to the server and insert the disk into that external drive.

Step 3

Alternatively, if the ISO image was copied onto a bootable USB disk, then insert this bootable USB disk into the server.

Important 

For information about and an example of creating a bootable USB disk from a USB flash drive and attaching the ISO to it, see Creating a Bootable USB Disk and Attaching the ISO.

Note 

Cisco UCS servers provide an additional method of installing a remote ISO using a Virtual KVM console. See your Cisco UCS server documentation for information about this procedure. Note that installing the ISO image using a Virtual KVM console may take longer than the above methods.

Step 4

Boot up the host (server) and start the configuration wizard.

What to do next

Proceed to configure Cisco APIC-EM to run on either a single or multiple hosts. Refer to the following sections for information about the configuration wizard process: