Cisco ACI Installation Guide for Red Hat OpenStack Using the OpenStack Platform 16.1 Director
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
Beginning in the 5.0(1) OpenStack plug-in release for Cisco Application Policy Infrastructure
Controller (APIC), the use of Ironic provisioning of bare metal servers is supported.
Ironic, which is deployed on the OpenStack overcloud, integrates with OpenStack services, such as Compute (Nova) and Network
(Neutron). You can use Ironic on its own or as part of an OpenStack cloud. Using Ironic with OpenStack enables you to provision
both virtual machines and bare metal servers on the same OpenStack cluster.
This appendix provides information about changes for Ironic in the 5.0(1) release when using the Cisco Application Centric
Infrastructure (ACI) plug-in for OpenStack.
Ironic In Your Network
Neutron manages connectivity between bare metal hosts and switches. Ironic maintains an inventory of bare metal hosts, and
connections between the hosts’s NICs and ports on switches. Ironic provides this inventory information to Neutron when bare
metal host ports must be connected to Neutron networks. Mechanism drivers use this information to configure switch ports with
the appropriate VLANs for the Neutron networks.
The following illustration shows networks when you use Ironic.
Ironic defines three logical networks, collectively called bare metal networks, which are named for their purpose in a bare
metal instance’s lifecycle:
Provisioning: Configure instances for bringup
Cleaning: Erase hard disk and other security issues
Rescue: Attach instances for recovery
Bare metal networks are regular networks in Neutron created by administrators. However, they should be created so that they
are only visible to the admin project in OpenStack. (Create them using the --no-share option). All three logical networks can be implemented on the same Neutron network, separate Neutron networks, or any combination
of networks. Using the provision network provides more security than using a single provider network for both provisioning
and tenant network connectivity.
Workflow for Configuring and Deploying Ironic
This section provides a high-level overview of the tasks that you need to complete to configure and deploy OpenStack Ironic
bare metal instances.
Connect bare metal networks to Ironic services. The connection is required for contracts to be applied and for the addition
of Cisco Application Centric
Infrastructure (ACI) policies for extra protection of OpenStack services.
Enroll bare metal nodes, following instructions in the Red Hat OpenStack Platform 13 documentation on the Red Hat website.
Create bare metal ports and port groups; although the procedure is almost the same as the one in Red Hat OpenStack Platform
13 documentation, you must specify additional topology information.
Create the bare metal flavors and availability zones, following instructions in the Red Hat OpenStack Platform 13 documentation
on the Red Hat website.
Launch the bare metal instances, following instructions in the Red Hat OpenStack Platform 13 documentation on the Red Hat
website.
Prerequisites for Deploying OpenStack to Support Ironic
You must complete the following tasks before you deploy OpenStack to support Ironic:
Ensure that Red Hat OpenStack Platform (OSP)13 is installed with Cisco Application Centric
Infrastructure (ACI) ML2 plug-in version 5.0(1) or above.
Note
Ironic bare metal provisioning is supported only for Red Hat OSP16.1.
Ensure that all the OpenStack servers are connected only to Nexus 9000 EX, FX or GX switches.
Decide the location of the Ironic services.
Ironic adds the following services to the deployment:
ironic-api: Is the application programming interface for Ironic deployment.
ironic-conductor: Creates the conductor manager, which performs all actions on bare metal resources.
Preboot Execution (PXE) server: Allows networked computers that are not yet loaded with an operating system to be configured and booted remotely by an administrator.
For HTTP, TFTP or both.
The recommended approach is to use a custom composable network to host the Ironic services. Follow the guidelines in the OSP16.1
documentation for creating custom composable networks, as well as the Ironic section that describes how to target Ironic services
on this network.
Define interfaces on the controller and their associated bridge mappings.
The OpenStack Platform (OSP) ensures that each controller host has an interface on the undercloud control network. That means
that if the default configuration is used, Ironic services will use it for their network interface. However, if you use the
ServiceNetMap parameter to specify a different network for Ironic services, you will need to configure additional network
interfaces. See the page "OpenStack Provisioning" for OSP16.1 documentation on the Red Hat website.
Ironic services are typically configured to run on a controller node. Like other services, they need IP addresses, service
authentication in OpenStack Identity (Keystone), and configuration files with information on how to reach the message bus,
database, and other services in OpenStack.
Predeployment Configuration
The Red Hat OpenStack Platform13 Director documentation explains the additional template configuration needed to deploy Ironic
services in the overcloud. There is no new template configuration that is specific to the Cisco Application Centric
Infrastructure (ACI) ML2 plug-in integration for OpenStack, when used with Ironic services.
If you use virtual port channels (vPCs) to connect the bare metal servers, then the Port Channel Policy assigned to the Leaf
Interface Policy Group must have only the following control parameters:
Fast Select Hot Standby Ports
Graceful Convergence
The Port Channel Policy should have the mode configured for “LACP Active.” This configuration is needed because bare metal
instances only use a single NIC when Unified Extensible Firmware Interface (UEFI) boots and attaches to bare metal networks.
Both ports are used when connecting to tenant networks. Also, the "Suspend Individual Port" option should be disabled.
If a custom composable network is used for the Ironic services, then an endpoint group (EPG) and bridge domain must be created
in Cisco ACI to implement that network. This is the same for any undercloud network implemented in Cisco ACI, as described in the section “Setting Up the Cisco APIC and the Network” section in the Cisco ACI Installation Guide for Red Hat OpenStack Using the OpenStack Platform 13 Director guide.
Regardless of where the Ironic services are created—for example, on the undercloud control network or on a custom composable
network—a subnet must be configured in the bridge domain that implements the undercloud network hosting the Ironic services.
This subnet must have a CIDR IP address that matches the subnet used for the undercloud network.
For example, if the control plane network is used for the service (default), the default value for the subnet is 1.100.1.0/24.
Therefore, some IP address on this subnet should be used as the CIDR configured in the subnet in this bridge domain.
The following diagram shows the different paths that the controller should use to reach the bare metal networks. The red arrow
going from the Controller to the Bare Metal networks shows the path that needs to be established (as opposed to the default
route). The two-headed black arrows show connections: the controller is connected to the internal, control, and management
networks, and the control and storage networks are connected to the bare metal networks.
Note
When using the control network for Ironic services, the IP address should be something other than 1.100.1.1, because the OpenStack
Platform allocates this IP address for the undercloud virtual machine (VM).
This subnet configured in the bridge domain is used as the next-hop IP address for routes that are added after deployment
so that the Ironic services can reach the bare metal networks.
Create and upload images to the overcloud, following instructions in Red Hat OpenStack Platform 16.1 documentation on the
Red Hat website.
Creating Contracts to Allow Traffic Between EPGs
Policy is needed to allow traffic between the endpoint group (EPG) for the undercloud network that hosts Ironic services and
any EPGs for overcloud bare metal networks. Because Ironic services may be on the same network as other services, the policy
should be limited to only the traffic needed between Ironic services and bare metal instances. The contracts, subjects, filters,
and filter entries for these policies need to cover the protocols in this section.
Required Contracts
Contracts that must be provided by Ironic services, consumed by bare metal networks:
PXE-TFTP (UDP destination port 69)
PXE-HTTP (TCP, default destination port 8088)
Confirm the port in use by running the following script on the controller for OSP16.1 installations:
Contracts that must be provided by bare metal networks, consumed by Ironic services:
RAMDISK Server (TCP destination port 9999)
iSCSI for iSCSI deploy interfaces (TCP destination port 3260)
Note
Contracts for on-network traffic, such as DHCP and ARP aren’t needed, as the on-network traffic is intra-EPG traffic.
Commands for Creating Contracts
Contracts can be created using aimctl commands, or directly with the Cisco Application Centric
Infrastructure (APIC) GUI or CLI. See the following examples of XML commands that you can run on OpenStack Overcloud controller node in order
to create the contracts in Cisco Application Centric
Infrastructure (ACI):
The newly created contracts must be applied to the EPG used for the undercloud network hosting the Ironic services. This can
be done with the Cisco ACI GUI, or any other supported means.
Create the Bare Metal Networks
The Ironic cleaning, provisioning, and rescue networks are collectively known as the bare metal networks. You can use one
Neutron network for all three purposes, or you can use separate networks for each. Because the Ironic services must communicate
directly with the instances, you must take additional steps when creating these bare metal networks.
Create the bare metal networks using the apic:extra_consumed_contracts and apic:extra_provided_contracts extensions.
These extensions create lists of additional provided or consumed contracts that are applied once a subnet on the Neutron network
is connected to a Neutron router. The following example shows how the extensions are used when creating the network, using
a baremetal-contract that was previously created in the common tenant in Cisco Application Policy Infrastructure
Controller (APIC):
The contract should only allow the traffic types needed between the bare metal host and the Ironic services. The contract
should also already be provided and consumed by the endpoint group (EPG) used for the undercloud network that hosts the Ironic
services.
Step 2
Ensure that the networks belong to the same VRF in Cisco Application Centric
Infrastructure (ACI) as the EPG that is used by the Ironic services.
For example, if the Ironic services are placed on the undercloud control network (default), then the VRF used for the bridge
domain for the undercloud control network must also be used for the overcloud bare metal networks.
You can control the VRF that a network belongs to by using address scopes in OpenStack. The apic:distinguished_names extension allows you to create an address scope that maps to an existing VRF in Cisco ACI. The following example shows how to create in OpenStack an address scope that maps to the control VRF under the common tenant
in Cisco ACI.
Add routing table entries to the host running Ironic services.
The default route on a host is typically not through the interface used for Ironic services. Add subnet routes to ensure that
the Ironic services interface is used to reach the bare metal networks. A subnet must be configured in the bridge domain used
for the undercloud network that hosts the Ironic services, so that Cisco Application Policy Infrastructure
Controller (APIC) can provide a subnet gateway for the services. (If the control plane network is used for Ironic services, the .1 address
can’t be used for the CIDR, as that is allocated for use by the undercloud VM.)
The following is an example of a static route to reach a bare metal network with a 40.40.40.0/24 subnet, through the subnet
gateway 1.100.1.254 (CIDR used for the undercloud control plane network):
To ensure that the route persists across reboots, add it to network initialization scripts. For example, /etc/sysconfig/network-scripts/route-br-baremetal.
Follow the instructions in the section "Adding Physical Machines as Bare Metal Nodes" in Red Hat OpenStack Platform 13 Bare Metal Provisioning on the Red Hat website.
Create the Bare Metal Ports and Port Groups
This step is almost identical to the steps in the OpenStack Platform (OSP) 13 documentation. However, you must specify additional
topology information for the Ironic ports, so that the fabric can be reconfigured correctly whenever the bare metal instances
are connected to Neutron networks.
You specify the topology information using the switch_id parameter in the local-link-connection portion of the Ironic port. This parameter is a string, which consists of a set of
key:value pairs, separated by commas. Three parameters are currently supported:
apic_dn:
This parameter provides the distinguished name (DN) in Cisco Application Policy Infrastructure
Controller (APIC) that is used for the static path binding. The following is the format for the DN when a single interface is used:
topology/pod-pod number/paths-node/pathep-port or policy group name
For example:
topology/pod-1/paths-101/pathep-[eth1/2]
The following is the format for the DN when a virtual port channel (vPC) is used:
This parameter is the physical network used if a VLAN must be allocated for the static path.
physical_domain:
This parameter is the name of the PhysDom that will be associated with the endpoint group (EPG) that the static port is created
on. This parameter is optional.
This section shows how to set the topology information in the local_link_connection field in the bare metal ports. It is assumed
that the bare metal node ID and its associated bare metal port IDs are defined in their respective environment variables.