Reference Information

Configure Hierarchical Port Binding

This section describes configuring the Single Root I/O Virtualization (SR-IOV) and other VLAN-based ml2 mechanism agents to work with OpFlex plug-in. The configuration is accomplished by using Hierarchical Port Binding (HPB) and should work without any special modification to the configuration. Here are the basic steps that you need to configure OpFlex with SR-IOV.

When using HPB, data path connectivity in Cisco Application Centric Infrastructure (ACI) is accomplished by creating static VLAN bindings to the EPGs for networks created by OpenStack. There maybe other configuration required for data path, for example, setting up VLAN on SR-IOV NIC or configuring OVS (or a load balancer in case of LBaaS). This is done by the third-party agent or mechanism driver (for example, sriovnicswitch).

How to create these assets:

Before you begin

To configure the data path using static VLAN bindings, ensure that the plug-in requires following assets:

  • A physical domain (physdom) with the correct VLAN pool.

  • Host-link information (which compute node fabric Ethernet interface is connected to which leaf switch port)

  • Host-link-network-label information (describing which fabric Ethernet interface on compute node is used to serve which physnet)

  • You need this information only if the deployment uses multiple physnets.

Procedure

Before deploying OpenStack Platform Overcloud, make sure you have one Physical Domain (physdom) created per each physnet required. Add pdom_ prefix to the name of physical domain created. For example for physnet1 create pdom_physnet1, and attach the right VLAN pool.

You must also set NeutronNetworkVLANRanges and enable the third-party mechanism drivers using ACIMechanismDrivers parameter, make sure that the apic_aim is the last mechanism in the list.

Example:

NeutronPhysicalDevMappings: physnet1:ens11,physnet2:ens7,physnet3:ens9
NeutronNetworkVLANRanges:physnet1:1200:1250,physnet2:1251:1300,physnet3:1301:1350
ACIMechanismDrivers: 'sriovnicswitch,apic_aim'
ACIHostLinks: '{"101": [{"host01|ens11": "1/14"}], "102": [{"host02|ens9": "1/14"}]}'

Parameters for the Cisco ACI Environment

The following table provides information about parameters that are required to configure the Cisco Application Centric Infrastructure (ACI) environment.

Parameter

Details

NeutronCorePlugin

  • Value: 'ml2plus'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronServicePlugins

  • Value: 'group_policy,ncp,apic_aim_l3'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronEnableIsolatedMetadata

  • Value: true

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Must be set to true

NeutronEnableForceMetadata

  • Value: true

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Must be set to true

ACIYumRepo

  • Value:: http://undercloud pxe network ipaddress:8787/v2/__acirepo

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicHosts

  • Value: Cisco Application Policy Infrastructure Controller (APIC) name and addresses

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicUsername

  • Value: Username with administrative privileges

  • Default: admin

  • Mandatory or Optional: Optional

  • Comments: None

ACIApicPassword

  • Value: Password

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

Note

 
Do not provide this parameter if certificate-based authentication is used.

ACIMechanismDrivers

  • Value: 'apic_aim'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Add extra drivers—for example, for Open vSwitch when using neutron ovs agent or for sriovnicswitch when using sriov

ACIApicEntityProfile

  • Value: The Cisco ACI entity profile that has been preprovisioned on Cisco ACI

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicInfraVlan

  • Value: The Cisco ACI fabric infra VLAN

  • Default: 4093

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIApicInfraSubnetGateway

  • Value: The Cisco ACI infra subnet gateway

  • Default: 10.0.0.30

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIApicInfraAnycastAddr

  • Value: The Cisco ACI anycast address

  • Default: 10.0.0.32

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIUseLldp

  • Value: true or false

  • Default: true

  • Mandatory or Optional: Optional

  • Comments: If set to false, set CiscoAciLldp service to OS::Hat:None

ACIOpflexUplinkInterface

  • Value: Interface name that is connected to the Cisco ACI leaf switch

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Actual interface name—for example, enp8s0

ACIOpflexEncapMode

  • Value: vxlan or vlan

  • Default: vxlan

  • Mandatory or Optional: Optional

  • Comments: None

ACIOpflexVlanRange

  • Value: starting_vlan:ending_vlan

  • Default: None

  • Mandatory or Optional: Mandatory if ACIOpflexEncapMode is set to vlan

  • Comments: None

ACIOpflexInterfaceType

  • Value: linux or ovs

  • Default: linux

  • Mandatory or Optional: Optional

  • Comments: Set this value to 'ovs' when planning to deploy an "OpenShift on OpenStack" nested installation. The setting causes the OpFlex interface to be created on the ovs switch.

ACIOpflexInterfaceMTU

  • Value: Intended MTU size

  • Default: 1500

  • Mandatory or Optional: Optional

  • Comments:

    Use this parameter to set the MTU for the OpFlex interface. This must be set to 8000 for installing OpenShift on OpenStack.

NeutronPluginMl2PuppetTags

  • Value: 'neutron_plugin_ml2,neutron_plugin_cisco_aci,neutron_sfc_service_config'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronNetworkVLANRanges

  • Value: physnet:starting vlan:ending vlan (For example, physnet1:1100:1150,physnet2:1201:1211)

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: None

AciPhysDomMappings

  • Value: For example: 'physnet0:my_pdom0, physnet1:my_pdom1'

  • Default: None

  • Mandatory or Optional: Optional

  • Comments: List of physical_network:ACI Physdom. By default each physnet maps to a precreated ACI physdom with pdom_physnet_name . For example physnet0 will map to physdom named pdom_phynet0. This parameter allows the user to override the mapping.

NeutronBridgeMappings

  • Value: For example: 'physnet1:br-ex,physnet2:br-ex'

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: Physnets should match as provided in NeutronBridgeMappings

AciTenantNetworkType

  • Value: vlan

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: None

AciOpenvswitch

  • Value: true or false

  • Default: false

  • Mandatory or Optional: Set to true when using neutron ovs agent

  • Comments: None

NeutronOVSFirewallDriver

  • Value: 'neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Set to the value shown when using neutron ovs agent

ACIHostLinks

  • Value: For example: '{"101":{"ha.dom":"1/1", "hb.dom":"1/2"}, "102":{"hc.dom":"1/1"} }'

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent and not using lldp agent

  • Comments: Describes the host connections to switches in JSON format.

    In the example, The ha.dom host is connected to port 1/1 of switch ID 101, the hb.dom host is connected to port 1/2 of switch ID 101, and the hc.dom is connected to port 1/1 of switch ID 102.

NeutronPhysicalDevMappings

  • Value:

  • Default: None

  • Mandatory or Optional: Optional

  • Comments:

NeutronPhysicalDevMappings

  • Value: For example: physnet1:eth1,physnet2:eth2

  • Default: None

  • Mandatory or Optional: Optional

  • Comments: You must set this parameter when you want to map a particular interface to a specific physnet

ACIApicCertName

  • Value: Name of the Cisco APIC cert User (used for certificate-based authentication)

  • Type: String

  • Default: None

  • Mandatory or Optional: Optional

ACIApicPrivateKey

  • Value: Private key for the cert User

  • Type: String

  • Default: None

  • Mandatory or Optional: Optional

AciKeystoneNotificationPurge

  • Value: True or False

  • Type: Boolean

  • Default: False

  • Comment: Enables the automatic purge of Cisco APIC tenants when the project is deleted in OpenStack.

NeutronPluginExtensions

  • Value: Comma-separated list of enabled extension plugins.

  • Default: apic_aim, port_security

  • Mandatory or Optional: Optional

  • Comment:Recommended values when parameter is explicitly configured are - apic_aim, port_security, qos.

NeutronMechanismDrivers

  • Value: 'apic_aim'

  • Default: ovn

  • Mandatory or Optional: Mandatory (when Octavia is deployed)

  • Comment:When deploying Octavia, by default ovn provider is deployed. In Octavia deployment, NeutronMechanismDrivers is taken into consideration to define provider. In Amphora deployment, ovn must be explicitly set.

Example of Resources Declaration

The following is a full example of the Cisco Application Centric Infrastructure (ACI) resources declaration (ciscoaci-env.yaml):

The following is a full example of the Cisco Application Centric Infrastructure (ACI) resources declaration (ciscoaci-env.yaml). The example provided is for Cisco ACI Release 5.2(1) or later. You declare resources for the Cisco ACI environment when you install the Overcloud. See the procedure Install Overcloud in this guide. 

# A Heat environment file which can be used to enable a
# a Neutron Cisco Aci backend on the controller, configured via puppet
resource_registry:
 
  #controller
  OS::TripleO::ControllerExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::NeutronOvsAgent: /opt/ciscoaci-tripleo-heat-templates/deployment/neutron_opflex/neutron-opflex-agent-container-puppet.yaml
  OS::TripleO::Docker::NeutronMl2PluginBase: /opt/ciscoaci-tripleo-heat-templates/deployment/neutron/neutron-ml2-ciscoaci.yaml
  OS::TripleO::Services::CiscoAciAIM: /opt/ciscoaci-tripleo-heat-templates/deployment/aciaim/cisco-aciaim-container-puppet.yaml
  OS::TripleO::Services::NeutronMetadataAgent: /usr/share/openstack-tripleo-heat-templates/deployment/neutron/neutron-metadata-container-puppet.yaml
  #compute
  OS::TripleO::ComputeExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::ComputeNeutronOvsAgent: /opt/ciscoaci-tripleo-heat-templates/deployment/neutron_opflex/neutron-opflex-agent-container-puppet.yaml
  OS::TripleO::Services::ComputeNeutronMetadataAgent: /opt/ciscoaci-tripleo-heat-templates/deployment/compute_neutron_metadata/compute-neutron-metadata.yaml
 
 OS::TripleO::Services::CiscoAciLldp: /opt/ciscoaci-tripleo-heat-templates/deployment/lldp/cisco_lldp.yaml
  OS::TripleO::Services::CiscoAciOpflexAgent: /opt/ciscoaci-tripleo-heat-templates/deployment/opflex/opflex-agent-container-puppet.yaml
 
 
  OS::TripleO::Services::OVNDBs: OS::Heat::None
  OS::TripleO::Services::OVNController: OS::Heat::None
  OS::TripleO::Services::OVNMetadataAgent: OS::Heat::None
  OS::TripleO::Services::ComputeNeutronL3Agent: OS::Heat::None
  OS::TripleO::Services::NeutronL3Agent: OS::Heat::None
 
 
parameter_defaults:
  NeutronSfcDriver: 'aim'
  NeutronFcDriver: 'aim'
  NeutronCorePlugin: 'ml2plus'
  NeutronServicePlugins: 'group_policy,ncp,apic_aim_l3'
  NeutronPluginMl2PuppetTags: 'neutron_plugin_ml2,neutron_plugin_cisco_aci'
  NeutronEnableIsolatedMetadata: true
  EnablePackageInstall: true
  ACIYumRepo: http://10.10.250.67:8787/v2/__acirepo
  ACIApicHosts: 10.105.1.10
  ACIApicUsername: admin
  ACIApicPassword: password
  ACIApicSystemId: osp16.2
  ACIUseLLDPDiscovery: 'true'
  ACIApicEntityProfile: OSP16.2
  ACIApicInfraVlan: 4093
  ACIApicInfraSubnetGateway: 10.0.0.30
  ACIApicInfraAnycastAddr: 10.0.0.32
  ACIOpflexUplinkInterface: ens8
  ACIOpflexEncapMode: vxlan
  ACIOpflexVlanRange: 1200:1300
  ACIYumRepoMetadataExpiry: 90
  DockerInsecureRegistryAddress: ["director16.2.ctlplane.localdomain:8787", "10.10.250.67:8787"]

Note

If you are deploying a release prior to Cisco ACI Release 5.2(1), you need to make the following changes in the above example:

  • Remove the definition for OS::TripleO::Services::CiscoAciOpflexAgent.

  • Change the OS::TripleO::Services::NeutronOvsAgent and OS::TripleO::Services::ComputeNeutronOvsAgent to reference the /opt/ciscoaci-tripleo-heat-templates/deployment/opflex/opflex-agent-container-puppet.yaml template.

Examples of Creating Host Reports

Topic created from previous version for Judith. ~ catortiz 11/22/2020.

During troubleshooting, you might need to collect host reports from the OpenStack cluster. You do it using the provided playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml. This section provides example of using the host report playbook.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml

    This example collects data from all nodes and creates the file /home/stack/overcloud_aci_report.tgz.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml -e '{"limit_flavors":['control'], "dest_file":/tmp/abc}

    This example limits the repot to controllers and changes the default output file.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml -e '{"limit_hosts":[overcloud-controller-0, overcloud-controller-2]}'

    This example limits the report collection to the hosts specified. You can club "limit_flavors" and "limit_hosts" to further filter the nodes from which to collect data.

Deploying with TLS

Deploying Red Hat OpenStack 16.2 with Transport Layer Security (TLS) is a supported configuration. To enable TLS on OpenStack endpoints, follow the instructions in Advanced Overcloud Customization on the Red Hat website.

To enable TLS between AIM and Cisco Application Policy Infrastructure Controller (APIC), follow the cert base authentication procedure described in step 4 of Install Overcloud in this guide.

Cleaning up Cisco ACI Container Images

If you run the cisco container image generation /opt/ciscoaci-tripleo-heat-templates/tools/build_openstack_aci_containers.py script multiple times (even for minor updates), you will continue to have old Cisco ACI container images in your repository. Use this procedure to clear old container images.

Procedure

Step 1

Find the tag for the latest built images.

You can check the file ciscoaci_containers.yaml to find out the latest image tags. In the below example, the tag is 1614292118. 


[stack@director16 ~]$ cat templates/ciscoaci_containers.yaml
parameter_defaults:
ContainerHorizonImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-horizon-ciscoaci:1614292118
ContainerHeatEngineImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-heat-engine-ciscoaci:1614292118
ContainerNeutronApiImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-neutron-server-ciscoaci:1614292118
ContainerNeutronConfigImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-neutron-server-ciscoaci:1614292118
ContainerCiscoLldpImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-ciscoaci-lldp:1614292118
ContainerCiscoAciAimImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-ciscoaci-aim:1614292118
ContainerCiscoAciAimConfigImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-ciscoaci-aim:1614292118
ContainerOpflexAgentImage: director16.ctlplane.localdomain:8787/ciscoaci/openstack-ciscoaci-opflex:1614292118
[stack@director16 ~]$

Step 2

To identify old images in your repository, use the sudo openstack tripleo container image list command.

Example below displays the new and older images. With reference to the example in Step 1, the new images are indicated with 1614292118, the others are older images.

[stack@director16 ~]$ sudo openstack tripleo container image list|grep cisco
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-horizon-ciscoaci:1613593371 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-horizon-ciscoaci:1613614575 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-horizon-ciscoaci:1614292118 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-heat-engine-ciscoaci:1613593371 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-heat-engine-ciscoaci:1613614575 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-heat-engine-ciscoaci:1614292118 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-neutron-server-ciscoaci:1613593371 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-neutron-server-ciscoaci:1613614575 |
| docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-neutron-server-ciscoaci:1614292118
With reference to the example in Step 1, the new images are indicated with 1614292118, the others are older images.

Step 3

To delete old images from your repository, use the sudo openstack tripleo container image delete command. 


[stack@director16 ~]$ sudo openstack tripleo container image delete \
docker://director16.ctlplane.localdomain:8787/ciscoaci/openstack-horizon-ciscoaci:1613593371