Reference Information

This chapter contains the following sections:

Configuring Hierarchical Port Binding

This section describes configuring the Single Root I/O Virtualization (SR-IOV) and other VLAN-based ml2 mechanism agents to work with OpFlex plug-in. This is accomplished by using Hierarchical Port Binding (HPB) and should work without any special modification to the configuration. Here are the basic steps needed to configure OpFlex with SR-IOV.

When using HPB, datapath connectivity in Cisco Application Centric Infrastructure (ACI) is accomplished by creating static VLAN bindings to the EPGs for networks created by OpenStack. There maybe other configuration required for datapath, for example, setting up VLAN on SR-IOV NIC or configuring OVS (or a load balancer in case of LBaaS). This is done by the third party agent or mechanism driver (for example, sriovnicswitch).

How to create these assets:

Before you begin

In order to configure the datapath using static VLAN bindings. The plug-in requires following assets:

  • A physical domain (physdom) with the correct VLAN pool.

  • Host-link information (which compute node fabric Ethernet interface is connected to which leaf switch port)

  • Host-link-network-label information (describing which fabric Ethernet interface on compute node is used to serve which physnet)

    This information is only needed if the deployment uses multiple physnets.

Procedure

Step 1

Before deploying OpenStack Platform Overcloud, make sure you have one Physical Domain (physdom) created per each physnet required. Add pdom_ prefix to the name of physical domain created. For example for physnet1 create pdom_physnet1, and attach the right VLAN pool.

Step 2

The host-link information usually comes from LLDP auto discovery. It can also be provided statically using ACIHostLinks parameter in the Cisco ACI deployment template as shown in example below.

You also need to set NeutronNetworkVLANRanges and enable the third party mechanism drivers using ACIMechanismDrivers parameter, make sure that the apic_aim is the last mechanism in the list.

Example:

NeutronPhysicalDevMappings: physnet1:ens11,physnet2:ens7,physnet3:ens9
NeutronNetworkVLANRanges:physnet1:1200:1250,physnet2:1251:1300,physnet3:1301:1350
ACIMechanismDrivers: 'sriovnicswitch,apic_aim'
ACIHostLinks: '{"101": [{"host01|ens11": "1/14"}], "102": [{"host02|ens9": "1/14"}]}'

Parameters for the Cisco ACI Environment

The following table provides information about parameters that are required to configure the Cisco Application Centric Infrastructure (ACI) environment.

Parameter

Details

NeutronCorePlugin

  • Value: 'ml2plus'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronServicePlugins

  • Value: 'group_policy,ncp,apic_aim_l3'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronEnableIsolatedMetadata

  • Value: true

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Must be set to true

NeutronEnableForceMetadata

  • Value: true

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Must be set to true

ACIYumRepo

  • Value:: http://undercloud pxe network ipaddress

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicHosts

  • Value: Cisco APIC name and addresses

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicUsername

  • Value: Username with administrative privileges

  • Default: admin

  • Mandatory or Optional: Optional

  • Comments: None

ACIApicPassword

  • Value: Password

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

Note 
Do not provide this parameter if certificate-based authentication is used.

ACIMechanismDrivers

  • Value: 'apic_aim'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Add extra drivers—for example, for Open vSwitch when using neutron ovs agent or for sriovnicswitch when using sriov

ACIApicEntityProfile

  • Value: The Cisco ACI entity profile that has been preprovisioned on Cisco ACI

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

ACIApicInfraVlan

  • Value: The Cisco ACI fabric infra VLAN

  • Default: 4093

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIApicInfraSubnetGateway

  • Value: The Cisco ACI infra subnet gateway

  • Default: 10.0.0.30

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIApicInfraAnycastAddr

  • Value: The Cisco ACI anycast address

  • Default: 10.0.0.32

  • Mandatory or Optional: Optional

  • Comments: Contact the Cisco ACI administrator for the correct value

ACIUseLldp

  • Value: true or false

  • Default: true

  • Mandatory or Optional: Optional

  • Comments: If set to false, set CiscoAciLldp service to OS::Hat:None

ACIOpflexUplinkInterface

  • Value: Interface name that is connected to the Cisco ACI leaf switch

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Actual interface name—for example, enp8s0

ACIOpflexEncapMode

  • Value: vxlan or vlan

  • Default: vxlan

  • Mandatory or Optional: Optional

  • Comments: None

ACIOpflexVlanRange

  • Value: starting_vlan:ending_vlan

  • Default: None

  • Mandatory or Optional: Mandatory if ACIOpflexEncapMode is set to vlan

  • Comments: None

ACIOpflexInterfaceType

  • Value: linux or ovs

  • Default: linux

  • Mandatory or Optional: Optional

  • Comments: Set this value to 'ovs' when planning to deploy an "OpenShift on OpenStack" nested installation. The setting causes the OpFlex interface to be created on the ovs switch.

ACIOpflexInterfaceMTU

  • Value: Intended MTU size

  • Default: 1500

  • Mandatory or Optional: Optional

  • Comments:

    Use this parameter to set the MTU for the OpFlex interface. This must be set to 8000 for installing OpenShift on OpenStack.

NeutronPluginMl2PuppetTags

  • Value: 'neutron_plugin_ml2,neutron_plugin_cisco_aci,neutron_sfc_service_config'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: None

NeutronNetworkVLANRanges

  • Value: physnet:starting vlan:ending vlan (For example, physnet1:1100:1150,physnet2:1201:1211)

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: None

NeutronBridgeMappings

  • Value: For example: 'physnet1:br-ex,physnet2:br-ex'

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: Physnets should match as provided in NeutronBridgeMappings

AciTenantNetworkType

  • Value: vlan

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent

  • Comments: None

AciOpenvswitch

  • Value: true or false

  • Default: false

  • Mandatory or Optional: Set to true when using neutron ovs agent

  • Comments: None

NeutronOVSFirewallDriver

  • Value: 'neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver'

  • Default: None

  • Mandatory or Optional: Mandatory

  • Comments: Set to the value shown when using neutron ovs agent

ACIHostLinks

  • Value: For example: '{"101":{"ha.dom":"1/1", "hb.dom":"1/2"}, "102":{"hc.dom":"1/1"} }'

  • Default: None

  • Mandatory or Optional: Mandatory when using neutron ovs agent and not using lldp agent

  • Comments: Describes the host connections to switches in JSON format.

    In the example, The ha.dom host is connected to port 1/1 of switch ID 101, the hb.dom host is connected to port 1/2 of switch ID 101, and the hc.dom is connected to port 1/1 of switch ID 102.

NeutronPhysicalDevMappings

  • Value:

  • Default: None

  • Mandatory or Optional: Optional

  • Comments:

NeutronPhysicalDevMappings

  • Value: For example: physnet1:eth1,physnet2:eth2

  • Default: None

  • Mandatory or Optional: Optional

  • Comments: You must set this parameter when you want to map a particular interface to a specific physnet

ACIApicCertName

  • Value: Name of the Cisco APIC cert User (used for certificate-based authentication)

  • Type: String

  • Default: None

  • Mandatory or Optional: Optional

ACIApicPrivateKey

  • Value: Private key for the cert User

  • Type: String

  • Default: None

  • Mandatory or Optional: Optional

ACIEnableBondWatchService

  • Value: True or False

  • Type: Boolean

  • Default: False

  • Comment: Set this parameter to True if you use Cisco Unified Computing System (UCS) blade servers for OpenStack nodes.

AciKeystoneNotificationPurge

  • Value: True or False

  • Type: Boolean

  • Default: False

  • Comment: Enables the automatic purge of Cisco APIC tenants when the project is deleted in OpenStack.

NeutronPluginExtensions

  • Value: Comma-separated list of enabled extension plugins.

  • Default: apic_aim,port_security

  • Mandatory or Optional: Optional

  • Comment:Recommended values when parameter is explicitly configured are - apic_aim, port_security, qos.

Example of Resources Declaration

The following is a full example of the Cisco Application Centric Infrastructure (ACI) resources declaration (ciscoaci-env.yaml):

You declare resources for Cisco ACI environment when you install Overcloud. See the procedure Installing Overcloud in this guide.

The following example is applicable for Cisco ACI Release 5.2(1) and later: 


# A Heat environment file which can be used to enable a
# a Neutron Cisco Aci backend on the controller, configured via puppet
resource_registry:

  #controller
  OS::TripleO::ControllerExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::NeutronL3Agent: OS::Heat::None
  OS::TripleO::Services::NeutronOvsAgent: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_nuetron_opflex.yaml
  OS::TripleO::Docker::NeutronMl2PluginBase: /opt/ciscoaci-tripleo-heat-templates/puppet/services/ciscoaci-ml2.yaml
  OS::TripleO::Services::CiscoAciAIM: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_aciaim.yaml

  #compute
  OS::TripleO::ComputeExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::ComputeNeutronOvsAgent: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_neutron_opflex.yaml
  OS::TripleO::Services::ComputeNeutronCorePlugin: /opt/ciscoaci-tripleo-heat-templates/puppet/services/ciscoaci_compute.yaml
  OS::TripleO::Services::ComputeNeutronMetadataAgent: /opt/ciscoaci-tripleo-heat-templates/docker/services/compute-neutron-metadata.yaml


  OS::TripleO::Services::CiscoAciLldp: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_lldp.yaml
  OS::TripleO::Services::CiscoAciOpflexAgent: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_opflex.yaml


parameter_defaults:
  NeutronSfcDriver: 'aim'
  NeutronFcDriver: 'aim'
  AdminPassword: 'admin0123'
  EC2MetadataIp: 1.100.1.1
  ControlPlaneDefaultRoute: 1.100.1.1
  DockerInsecureRegistryAddress: 1.100.1.1:8787
  OvercloudControllerFlavor: control
  OvercloudComputeFlavor: compute
  ExternalNetworkVlanID: 1
  InternalApiNetworkVlanID: 301
  StorageNetworkVlanID: 302
  StorageMgmtNetworkVlanID: 303
  TenantNetworkVlanID: 304
  NeutronCorePlugin: 'ml2plus'
  NeutronPluginMl2PuppetTags: 'neutron_plugin_ml2,neutron_plugin_cisco_aci'
  NeutronServicePlugins: 'group_policy,ncp,apic_aim_l3'
  NeutronEnableIsolatedMetadata: true
  NeutronEnableForceMetadata: true
  EnablePackageInstall: true
  ACIScopeNames: true
  ACIYumRepo: http://1.100.1.1/acirepo
  ACIApicHosts: 172.171.170.169
  ACIApicUsername: admin
  ACIApicPassword: mypass
  ACIApicSystemId: osp13
  ACIMechanismDrivers: 'apic_aim'
  ACIApicEntityProfile: osp13-aep
  ACIApicInfraVlan: 4093
  ACIApicInfraSubnetGateway: 10.0.0.30
  ACIApicInfraAnycastAddr: 10.0.0.32
  ACIOpflexUplinkInterface: enp8s0
  ACIOpflexEncapMode: vxlan
  HeatEnginePluginDirs: /usr/lib64/heat,/usr/lib/heat,/usr/local/lib/heat,/usr/local/lib64/heat,/usr/lib/python2.7/site-packages/gbpautomation/heat
  NeutronGlobalPhysnetMtu: 8950
  ACIOpflexInterfaceMTU: 9000
  NtpServer: ['172.28.184.8']
  NeutronPluginMl2PuppetTags: 'neutron_plugin_ml2,neutron_plugin_cisco_aci,neutron_sfc_service_config'

Note

If you are deploying a release prior to Cisco ACI Release 5.2(1), you need to make the following changes in the above example:

  • Remove the definition for the OS::TripleO::Services::CiscoAciOpflexAgent service.

  • Change the OS::TripleO::Services::NeutronOvsAgent and OS::TripleO::Services::ComputeNeutronOvsAgent to reference the /opt/ciscoaci-tripleo-heat-templates/deployment/ opflex/copflex-agent-container-puppet.yaml template.

Example of Resources Declaration When Using the Neutron OVS Agent

When you use the Neutron OVS agent, Cisco-specific containers for Horizon and Heat may be optional. If that is the case, remove them from the ciscoaci_containers.yaml file that was generated in step 5 in the section Preparing Undercloud for Cisco ACI with OpFlex Orchestration in this guide.

You can use the following commands to remove Horizon and Heat from the ciscoaci_containers.yaml file: 
resource_registry:
#controller
  OS::TripleO::ControllerExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::NeutronL3Agent: OS::Heat::None
  OS::TripleO::Services::NeutronOvsAgent: /usr/share/openstack-tripleo-heat-templates/docker/services/neutron-ovs-agent.yaml
  OS::TripleO::Docker::NeutronMl2PluginBase: /opt/ciscoaci-tripleo-heat-templates/puppet/services/ciscoaci-ml2.yaml
  OS::TripleO::Services::CiscoAciAIM: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_aciaim.yaml
  OS::TripleO::Services::NeutronSfcApi: /usr/share/openstack-tripleo-heat-templates/puppet/services/neutron-sfc-api.yaml

  #compute
  OS::TripleO::ComputeExtraConfigPre: /opt/ciscoaci-tripleo-heat-templates//nodepre.yaml
  OS::TripleO::Services::ComputeNeutronOvsAgent: /usr/share/openstack-tripleo-heat-templates/docker/services/neutron-ovs-agent.yaml
  OS::TripleO::Services::ComputeNeutronCorePlugin: /opt/ciscoaci-tripleo-heat-templates/puppet/services/ciscoaci_compute.yaml
  OS::TripleO::Services::ComputeNeutronMetadataAgent: /opt/ciscoaci-tripleo-heat-templates/docker/services/compute-neutron-metadata.yaml

#if using LLDP, else set it to OS::Heat::None
  OS::TripleO::Services::CiscoAciLldp: /opt/ciscoaci-tripleo-heat-templates/docker/services/cisco_lldp.yaml

parameter_defaults:
  NeutronSfcDriver: 'aim'
  NeutronFcDriver: 'aim'
  AdminPassword: 'myPass123'
  EC2MetadataIp: 10.10.253.10
  ControlPlaneDefaultRoute: 10.10.253.10
  DockerInsecureRegistryAddress: 10.10.253.10:8787
  OvercloudControllerFlavor: control
  OvercloudComputeFlavor: compute
  NeutronCorePlugin: 'ml2plus'
  NeutronPluginMl2PuppetTags: 'neutron_plugin_ml2,neutron_plugin_cisco_aci'
  NeutronServicePlugins: 'group_policy,ncp,apic_aim_l3,flow_classifier,sfc,trunk'
  NeutronEnableIsolatedMetadata: true
  NeutronEnableForceMetadata: true
  EnablePackageInstall: true
  ACIScopeNames: true
  ACIYumRepo: http://10.10.253.10/acirepo
  ACIApicHosts: 171.36.17.106
  ACIApicUsername: admin
  ACIApicPassword: apicPass123
  ACIApicSystemId: osp13-ovs
  ACIApicEntityProfile: OSP-AEP
  ACIApicInfraVlan: 4093
  ACIApicInfraSubnetGateway: 10.0.0.30
  ACIApicInfraAnycastAddr: 10.0.0.32
  ACIOpflexUplinkInterface: ens9
  HeatEnginePluginDirs: /usr/lib64/heat,/usr/lib/heat,/usr/local/lib/heat,/usr/local/lib64/heat,/usr/lib/python2.7/site-packages/gbpautomation/heat
  NeutronGlobalPhysnetMtu: 8950
  NtpServer: ['172.28.184.8']
  NeutronPluginMl2PuppetTags: 'neutron_plugin_ml2,neutron_plugin_cisco_aci,neutron_sfc_service_config'
  AciExternalRoutedDomain: 'myERD'
  ACIMechanismDrivers: openvswitch,apic_aim
  NeutronNetworkVLANRanges: physnet1:1200:1250
  AciOpenvswitch: true
  NeutronBridgeMappings: 'physnet1:br-ex'
  NeutronTunnelTypes: ''
  NeutronOVSFirewallDriver: 'neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver'
  NeutronEnableDVR: false

Examples of Creating Host Reports

During troubleshooting, you might need to collect host reports from the OpenStack cluster. You do it using provided playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml. This section provides example of using the host report playbook.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml

    This example collects data from all nodes and creates the file /home/stack/overcloud_aci_report.tgz.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml -e '{"limit_flavors":['control'], "dest_file":/tmp/abc}

    This example limits the repot to controllers and changes the default output file.

  • ansible-playbook /opt/ciscoaci-tripleo-heat-templates/tools/report.yaml -e '{"limit_hosts":[overcloud-controller-0, overcloud-controller-2]}'

    This example limits the report collection to the hosts specified. You can club "limit_flavors" and "limit_hosts" to further filter the nodes from which to collect data.

Deploying with TLS

Deploying Red Hat OpenStack 13 with Transport Layer Security (TLS) is a supported configuration. To enable TLS on OpenStack endpoints, follow the instructions in Advanced Overcloud Customization on the Red Hat website.

To enable TLS between AIM and Cisco Application Policy Infrastructure Controller (APIC), follow the cert base authentication procedure described in step 4 of Installing Overcloud in this guide.