Overview

This topic explains how to establish a local serial connection to the Cisco APIC server for initial configuration, detailing methods for connecting via KVM cable, USB keyboard, and VGA monitor, and providing guidance on enabling Serial-over-LAN (SoL) for remote access.

This section describes how to establish a local serial connection to the Cisco APIC server to begin the initial basic configuration. For additional connection information, including instructions on connecting to the server remotely for setup, refer to "Initial Server Setup" in the Cisco APIC M4/L4 server installation and service guide. If you are on Cisco APIC release 6.1(4), you can refer the Cisco APIC G5 server installation and service guide.

Initial connection

The Cisco APIC server operates on a Cisco Integrated Management Controller (CIMC) platform. You can make an initial connection to the CIMC platform using one of these methods:

Use a KVM cable (Cisco PID N20-BKVM) to connect a keyboard and monitor to the KVM connector on the front panel of the server.

If you want to use vKVM, see Installing an OS Using the KVM Console in Cisco UCS C-Series Integrated Management Controller GUI Configuration Guide, Release 4.3.
Connect a USB keyboard and VGA monitor to the corresponding connectors on the rear panel of the server.

Note
You cannot use the front panel VGA and the rear panel VGA at the same time.

You can make a serial connection using one of the following methods. Two of these methods require a configuration change in the CIMC:

Note
You cannot use more than one of these methods simultaneously.

Use the DB9 connector of the KVM cable
Use the rear panel RJ-45 console port (after enabling in the CIMC)
Connect by Serial-over-LAN (SoL) (after enabling in the CIMC)

The default connection settings from the factory are:

The serial port baud rate is 115200
The RJ-45 console port located on the rear panel is disabled in the CIMC
SoL is disabled in the CIMC

The following are additional notes about serial access:

If you are using a Cisco Integrated Management Controller (CIMC) for your setup, setup the CIMC first, and then access the Cisco APIC through the CIMC KVM or continue to access the Cisco APIC locally through the rear panel USB/VGA port. If you choose the CIMC KVM access, you will have remote access available later which is required during operations.
If you are using the RJ-45 console port, connect to CIMC using SSH and enable the SoL port using the following commands:
```
scope sol
  set enabled yes
  set baud-rate 115200
  commit
  exit
```
After enabling SoL, enter the command connect host to access the APIC console.

Note

When using SoL, physically disconnect the rear panel RJ-45 console port.

Initial Cisco APIC Setup

When the Cisco APIC is launched for the first time, the APIC console presents a series of initial setup options. For many options, you can press Enter to choose the default setting that is displayed in brackets. At any point in the setup dialog, you can restart the dialog from the beginning by pressing Ctrl-C.

Important Notes

If the UNIX user ID is not explicitly specified in the response from the remote authentication server, then some Cisco APIC software releases assign a default ID of 23999 to all users. If the response from the remote authentication server fails to specify a UNIX ID, all users will share the same ID of 23999 and this can result in the users being granted higher or lower privileges than the configured privileges through the RBAC policies on the Cisco APIC.
Cisco recommends that you assign unique UNIX user IDs in the range of 16000 to 23999 for the AV Pairs that are assigned to the users when in Bash shell (using SSH, Telnet, or Serial/KVM consoles). If a situation arises where the Cisco AV Pair does not provide a UNIX user ID, the user is assigned a user ID of 23999 or similar number from the range that also enables the user's home directories, files, and processes accessible to the remote users with a UNIX ID of 23999.

To ensure that your remote authentication server does not explicitly assign a UNIX ID in its cisco-av-pair response, open an SSH session to the Cisco APIC and log in as an administrator (using a remote user account). Once logged in, run the following commands (replace userid with the username that you logged in with):
- admin@apic1: remoteuser-userid> cd /mit/uni/userext/remoteuser-userid
- admin@apic1: remoteuser-userid> cat summary
Cisco recommends against modifying any parameters using CIMC. If there are any issues, ensure that the default setting for CIMC management node is Dedicated mode and not Shared. If Dedicated mode is not used, it can prevent the discovery of fabric nodes.
Do not upgrade software or firmware using the CIMC user interface, XML, or SSH interfaces unless the modified property and software or firmware version are supported with your specific Cisco APIC version.

Set the NIC mode to Dedicated, when setting up the CIMC, in the CIMC Configuration Utility. After the CIMC is configured, in the CIMC GUI, verify that you have the following parameters set.


Parameters	Settings
LLDP	Disabled on the VIC
TPM Support	Enabled on the BIOS
TPM Enabled Status	Enabled
TPM Ownership	Owned

If you log in to your Cisco APIC using https, and then attempt to log in to the same Cisco APIC using http in the same browser window without first logging out of the Cisco APIC in the https window, you might see the following error message:
Need a valid webtoken cookie (named APIC-Cookie) or a signed request with signature in the cookie.
If this occurs, resolve the issue using either of the following methods:
- Log out of the Cisco APIC in the https window, or
- Delete the cookies in the browser window
You should be able to successfully log into the Cisco APIC using http after resolving the issue with either of the methods above.
During the initial setup, the system will prompt you to select IPv4, or IPv6, or dual stack configuration. Choosing dual stack will enable accessing the Cisco APIC and Cisco ACI fabric out-of-band management interfaces with either IPv4 or IPv6 addresses. While the examples in the table below use IPv4 addresses, you can use whatever IP address configuration options you chose to enable during the initial setup.

Note

A minimum subnet mask of /19 is recommended.
Connecting the Cisco APIC to the Cisco ACI fabric requires a 10G interface on the ACI-mode leaf switch. You cannot connect the Cisco APIC directly to the Cisco Nexus 9332PQ, Cisco Nexus 93180LC, or Cisco Nexus 9336C-FX2 ACI-mode leaf switches unless you use a 40G to 10G converter (part number CVR-QSFP-SFP10G), in which case the port on the leaf switches will auto-negotiate to 10G without requiring any manual configuration.
The fabric ID is set during the Cisco APIC setup and it cannot be changed unless you perform a clean reload of the fabric. To change the fabric ID, export the Cisco APIC configuration, change the sam.config file, and perform a clean reload of the Cisco APIC and leaf switches. Remove the "fvFabricExtConnP" setting from the exported configuration before importing the configuration into the Cisco APIC after the Cisco APIC comes up. All Cisco APICs in a cluster must have the same fabric ID.

Note

All logging is enabled by default.

For login and cluster operations, non-default HTTPS port (default is 443) is not supported for layer 3 physical and layer 3 virtual APICs (on ESXi and AWS). Virtual APICs on ESXi/ AWS are supported from release 6.0(2).

Bring up the APIC cluster using the GUI

This procedure provides details of the initial cluster set up and bootstrapping process. Enter the relevant details for each of the screens

The APIC Cluster Bringup GUI supports virtual and physical APIC platforms. The virtual APICs (deployed using ESXi or AWS), and physical APICs can be connected to the ACI fabric directly to the leaf switches or remotely attached through a layer 3 network. The GUI supports both the scenarios. A major advantage of using the APIC Cluster Bringup GUI is that, you do not need to enter the parameters for every APIC in a cluster. One APIC can relay the information to the other APICs of the cluster.

Beginning with Cisco APIC release 6.2(1), an extra layer of security has been added for clusters with virtual APICs. After you bring up a new virtual APIC to a cluster, you will be prompted to change the password. See Step 12, below, for details.

Note

: When deploying the virtual APIC (VMware or Nutanix), you should enter a temporary password. This is crucial especially when you are adding the virtual APIC to an existing cluster. If the cluster password is used during the VM deployment, you will need to change the existing cluster password after the APIC has joined the cluster.

Alternatively, you can perform the initial setup and cluster bringup using the REST APIs. See the Getting Started section of the APIC REST API Configuration Procedures guide.

Before you begin

For virtual APIC on ESXi, ensure to complete the deployment of the APIC VM using the OVF template on the VMware vCenter GUI. For a three-node cluster, configure three VMs with the management IP address, gateway, and admin passwords. The number of VMs is dependent on the size of the APIC cluster.
For virtual APIC on AWS, ensure to complete the deployment of the APIC VM using the cloud formation template (CFT) on the AWS GUI. AWS allocates IP addresses dynamically from the out-of-band (OOB)/infra/inband subnets accordingly, to correspond with the network adapters of the virtual APIC's EC2 instance.
For virtual APICs (deployed using AWS/ ESXi), ensure that the admin password(s) are the same for all the APICs in a cluster.
For the physical APIC cluster, configure the OOB address for APIC 1. Ensure that the CIMC addresses of APICs 2 to N (where N is the cluster size) are reachable via the OOB address of APIC 1.
Connectivity between out-of-band and the CIMC is mandatory.

Limitations:

No support for IPv6 addresses on virtual APICs deployed using AWS.
For login and cluster operations, non-default HTTPS port (default is 443) is not supported for remotely-attached APICs (physical and virtual).

Procedure

	1.	Log in to the APIC 1 using https://APIC1-IP. For virtual APICs: If you have completed the deployment of virtual APICs using ESXi (OVF template) or remote AWS (CFT), then you see output on the VM console similar to the following example: `System pre-configured successfully. Use: https://172.31.1.2 to complete the bootstrapping.` The IP address to access the bootstrapping GUI (APIC Cluster Bringup) is explicitly indicated, as shown in the example. You can proceed to step 2. After deploying APIC on AWS, keep the OOB management IP address handy to access the Cluster Bringup GUI. You can get the OOB management IP address from the Stacks Outputs tab on the AWS GUI. For physical APICs: Log in to the APIC 1 KVM console using the CIMC; you will see a screen as shown below: `APIC Version: 6.0(2a) Welcome to Cisco APIC Setup Utility Press Enter Or Input JSON string to bootstrap your APIC node.` If you see only a black screen on the KVM, connect to the CIMC using SSH and use serial over LAN (SoL) ("connect host") to connect to the console. On APIC 1, press Enter and provide the requested information. The IP address to access the bootstrapping GUI (APIC Cluster Bringup) is explicitly indicated. `admin user configuration ... Enter the password for admin [None]: Reenter the password for admin [None]: Out-of-band management configuration ... Enter the IP Address [192.168.10.1/24]: 172.20.7.79/23 Enter the IP Address of default gateway [192.168.10.254]: 172.20.6.1 Would you like to edit the configuration? (y/n) [n]: System pre-configured successfully. Use: https://172.20.7.79 to complete the bootstrapping` The IP addresses displayed above are examples. The IP addresses will vary based on your deployment.
	2.	Using the OOB address, log in to the APIC Cluster Bringup GUI.
	3.	In the Select Workflow screen, choose New cluster and click Next. Note The workflow selection is only from Release 6.1(3) and later. The GUI screen has four parts. Enter the details in the following screens: Connection Type Cluster Details Controller Registration Summary Each of the above screens are discussed in detail in the subsequent steps. The screens are marked as steps with sequential numbers: 1, 2, 3, and 4; after you have entered and saved the required details in each of these screens, the number is replaced with a tick-mark.
	4.	The first step is entering the Connection Type information. In the Connection Type screen, choose the type of connection between the APIC and the fabric. The options are: Directly connected to leaf switches (APIC fabric) Remotely attached through a Layer 3 network If it is virtual APIC using AWS, the system detects that the APIC is remotely-attached through a Layer 3 network and proceeds directly to the Cluster Details screen. Click Next.
	5.	The second step is entering the Cluster Details. Enter the fabric-level details in the Cluster Details screen. Fabric Name: Enter a name for the fabric. Cluster Size: The default cluster size displayed is "3", which is the recommended minimum cluster size. You can modify this value, based on your cluster size. The supported values are 1, 3, 4, 5, 6, 7, 8, and 9. GiPo Pool: Enter the IP address used for fabric multicast. The default address is 225.0.0.0/15. The range is from 225.0.0.0/15 to 231.254.0.0/15. The prefixlen must be 15 (128k IP addresses). You cannot change this value after you have completed the configuration. Having to modify this value requires a wipe of the fabric. Pod ID: (applicable only for directly connected APICs (virtual and physical)) the pod ID is displayed. If this is your first APIC, "1" is auto-populated. Subsequent APICs of the cluster can be associated with any pod number. For a remotely-attached APICs, pod is 0. TEP Pool: (applicable only for directly connected APICs (ESXi virtual APIC and physical APIC)) enter the subnet of addresses used for internal fabric communication. The size of the subnet used will impact the scale of your pod. You can not change this value after you have completed the configuration. Having to modify this value requires a wipe of the fabric. Infrastructure VLAN: Enter the VLAN ID for fabric connectivity (infra VLAN). This VLAN ID should be allocated solely to APIC, and not used by any other legacy device(s) in your network. Default value is 3914. The range is from 0 to 4093. You can not change this value after you have completed the configuration. Having to modify this value requires a wipe of the fabric. Enable IPv6 on APICs (not applicable for virtual APIC on AWS): Put a check in this box if you want to enable IPv6 addresses for out-of-band management. Click Next.
	6.	The third step is entering the Controller Registration details. Click Add Controller to add the first APIC (of the cluster). Enter the following details: Controller Type: The bootstrapping procedure auto-detects the deployment for which the configuration is being carried out. Based on that, either Virtual or Physical is chosen. The options displayed for the virtual and physical controller types are discussed in substeps (a) and (b), respectively. Follow either of these substeps based on the controller type. When the Controller Type is Virtual: Virtual Instance: The management IP used to access the APIC cluster bringup GUI. Only for the first APIC, this IP address is auto-populated. For the nodes that you subsequently add to the cluster, you will need to enter the management IP address and click Validate. The management IP addresses are defined during the deployment of the VMs using ESXi/AWS. As mentioned in the prerequisites, keep all the required IP addresses handy while bringing up the cluster. General pane Name: User-defined name for the controller. Controller ID: The ID is auto-populated. If this is the first APIC of the cluster, the ID is "1". If you are adding the second controller of the cluster, "2" is auto-populated (and so on). Pod ID: (Applicable only for directly connected virtual APIC on ESXi) The pod ID is auto-populated for APIC 1 of the cluster. For subsequent controllers of the cluster, enter a value. The range is from 1 to 128. Serial Number: The serial number of the virtual machine is auto-populated. Out of Band Network pane IPv4 Address: The IP address is displayed (as defined during the deployment). IPv4 Gateway: The IP address is displayed (as defined during the deployment). If you have enabled IPv6 addresses for OOB management earlier (step 5), enter the IPv6 address and gateway. Infra L3 Network pane (this pane is displayed only if the Connection Type that you chose earlier is Remotely attached through an L3 network. IPv4 Address: Enter the infra network address. IPv4 Gateway: Enter the IP address of the gateway. VLAN: (Applicable only for remotely attached virtual APIC- ESXi) Enter the interface VLAN ID to be used. The Infra L3 Network pane does not display when you deploy the virtual APIC using AWS. After you have entered and saved the first APIC details, click Add Controller on the Controller Registration screen to add another APIC to the cluster. When the Controller Type is Physical: CIMC Details pane IP Address: The CIMC IP address. Only for the first APIC, this IP address is auto-populated. When you add more controllers to the cluster, you need to enter the CIMC IP addresses. Username: The username to access the CIMC. The username is auto-populated (for the first controller and subsequent controllers). Password: Enter the password to access CIMC. For the first controller, the password is auto-populated. For the subsequent controllers, enter the password. Click Validate. Validation success is displayed on successful authentication. If the CIMC is unreachable from the APIC out of band management IP address due to the CIMC NIC mode settings, change the NIC mode or enter JSON strings to perform the bootstrap. General pane Name: Enter a name for the controller. Controller ID: If it is the first controller of the cluster, "1" is auto-populated. If it is the second controller, "2" is auto-populated, and so on (increasing order). Pod ID: (applicable only for a directly-connected APIC) the pod ID is auto-populated for APIC 1 of the cluster. For subsequent controllers of the cluster, enter a value. The range is from 1 to 128. Serial Number: The serial number is auto-populated (for APICs 1 to N, where N is the cluster size) after CIMC validation. APIC 1 verifies the reachability of the CIMC IP addreses and also captures the serial number of the new APICs. Out of Band Network pane IPv4 Address: For APIC 1, the address is auto-populated. For subsequent APICs, enter the IP address (as defined during the deployment). IPv4 Gateway: For APIC 1, the gateway address is auto-populated. For subsequent APICs, enter the IP address (as defined during the deployment). If you have enabled IPv6 addresses for OOB management earlier (step 5), enter the IPv6 address and gateway. Infra L3 Network pane (this pane is displayed only if the Connection Type that you chose earlier is remotely attached through a Layer 3 network). IPv4 Address: Enter the infra network IP address. IPv4 Gateway: Enter the infra network IP address of the gateway. VLAN: Enter a VLAN ID. On the Controller Registration screen, after you have entered and saved the first APIC details, click Add Controller to add another APIC to the cluster. (Optional, applicable only for virtual APICs) On the Controller Registration screen, put a check in the Import existing security certificates box to import existing security certificates for fabric recovery in virtual APICs. After putting a check in the box, enter the required details in the following fields: The Remote Server IP Address which contains the configuration file. The Remote Path which contains the configuration file. The configuration File Name. The AES Encryption Passphrase which was earlier used while backing up the configuration. The backup configuration file is linked to this key (passphrase). Choose the Protocol. The choices are: FTP SFTP SCP Remote Port (applicable only for SFTP and SCP Protocols) Choose the Authentication Type. The choices are: Use Password Use SSH Private Key Files The Username to access the remote server. The Password to authenticate access to the remote server. (applicable only for Use SSH Private Key Files Authentication Type) Enter the SSH Key Contents here. (applicable only for Use SSH Private Key Files Authentication Type) Specify the SSH Key Passphrase used for encrypting the private key. For details about the Import/Export procedure, see the Cisco ACI Configuration Files: Import and Export document. The Import existing security certificates is applicable only for virtual APICs (deployed using AWS/ ESXi). Physical APICs have in-built certificates. However, in case of virtual APICs, when you are restoring using backup configuration to recover the fabric, the existing security certificates can be re-used.
	7.	Click Next. The Next button is disabled until all the controllers for a cluster are added. This is defined by the value you have entered for Cluster Size in the Cluster Details screen. You can use the Back button to navigate to an earlier screen. After adding an APIC, click Edit Details to edit the information for an APIC. Except the first APIC, you can delete the other controllers, if required, by clicking the delete icon.
	8.	In the Summary screen, review the updates, and click Deploy. The Cluster Status page is displayed, which shows the current status of the cluster formation. Wait for a few minutes after which you will be automatically redirected to the standard Cisco APIC GUI.
	9.	(for virtual APICs on VMware and Nutanix only) The Change Password window is displayed. Enter the required details and click Submit.

IPv6 management addresses on Cisco APIC

IPv6 management addresses can be provisioned on the Cisco Application Policy Infrastructure Controller (APIC) at setup time or through a policy once the Cisco APIC is operational. Pure IPv4, pure IPv6, or dual stack (that is, both IPv6 and IPv4 addresses) are supported.

For the detailed cluster bring up procedure, see Bring up the APIC cluster using the GUI.

Local serial connection for basic configuration