Chapter 1. New and Changed Information

This chapter contains the following sections:

- New and Changed Information

New and Changed Information

The following tables provide an overview of the significant changes to this guide for the Cisco Application Policy Infrastructure Controller APIC 4.2(x) family of releases. The tables do not provide an exhaustive list of all changes to the guide or of the new features.

Chapter 2. Cisco ACI Virtual Machine Networking

This chapter contains the following sections:

- Cisco ACI VM Networking Support for Virtual Machine Managers

- Mapping Cisco ACI and VMware Constructs

- Virtual Machine Manager Domain Main Components

- Virtual Machine Manager Domains

- VMM Domain VLAN Pool Association

- VMM Domain EPG Association

- About Trunk Port Group

- Attachable Entity Profile

- EPG Policy Resolution and Deployment Immediacy

- Guidelines for Deleting VMM Domains

- Troubleshooting VMM Connectivity

Cisco ACI VM Networking Support for Virtual Machine Managers

Benefits of ACI VM Networking

Cisco Application Centric Infrastructure(ACI) virtual machine (VM) networking supports hypervisors from multiple vendors. It provides the hypervisors programmable and automated access to high-performance scalable virtualized data center infrastructure.

Programmability and automation are critical features of scalable data center virtualization infrastructure. The Cisco ACI open REST API enables virtual machine integration with and orchestration of the policy model-based Cisco ACI fabric. Cisco ACI VM networking enables consistent enforcement of policies across both virtual and physical workloads that are managed by hypervisors from multiple vendors.

Attachable entity profiles easily enable VM mobility and placement of workloads anywhere in the Cisco ACI fabric. The Cisco Application Policy Infrastructure Controller (APIC) provides centralized troubleshooting, application health score, and virtualization monitoring. Cisco ACI multi-hypervisor VM automation reduces or eliminates manual configuration and manual errors. This enables virtualized data centers to support large numbers of VMs reliably and cost effectively.

Supported Products and Vendors

Cisco ACI supports virtual machine managers (VMMs) from the following products and vendors:

- Cisco Unified Computing System Manager (UCSM)

Integration of Cisco UCSM is supported beginning in Cisco Cisco APIC Release 4.1(1). For information, see the chapter "Cisco ACI with Cisco UCSM Integration in the Cisco ACI Virtualization Guide, Release 4.1(1) .

- Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod)

Cisco ACI vPod is in general availability beginning in Cisco APIC Release 4.0(2). For information, see the Cisco ACI vPod documentation on Cisco.com.

- Cisco Application Centric Infrastructure (ACI) Virtual Edge

For information, see the Cisco ACI Virtual Edge documentation on Cisco.com.

- Cisco Application Virtual Switch (AVS)

For information, see the chapter "Cisco ACI with Cisco AVS" in the Cisco ACI Virtualization Guide and Cisco AVS documentation on Cisco.com.

- Cloud Foundry

Cloud Foundry integration with Cisco ACI is supported beginning with Cisco APIC Release 3.1(2). For information, see the knowledge base article, Cisco ACI and Cloud Found Integration on Cisco.com.

- Pivotal Cloud Foundry

Pivotal Cloud Foundry integration with Cisco ACI is supported beginning with Cisco APIC Release 3.2(1). For information, see the knowledge base article, Cisco ACI and Cloud Foundry Integration on Cisco.com.

- Kubernetes

For information, see the knowledge base article, Cisco ACI and Kubernetes Integration on Cisco.com.

- Microsoft System Center Virtual Machine Manager (SCVMM)

For information, see the chapters "Cisco ACI with Microsoft SCVMM" and "Cisco ACI with Microsoft Windows Azure Pack" in the Cisco ACI Virtualization Guide on Cisco.com

- OpenShift

For information, see the OpenShift documentation on Cisco.com.

- OpenStack

For information, see the OpenStack documentation on Cisco.com.

- Red Hat Virtualization (RHV)

For information, see the knowledge base article, Cisco ACI and Red Hat Integration on Cisco.com.

- VMware Virtual Distributed Switch (VDS)

For information, see the chapter "Cisco "ACI with VMware VDS Integration" in the Cisco ACI Virtualization Guide.

See the Cisco ACI Virtualization Compatibility Matrix for the most current list of verified interoperable products.

Mapping Cisco ACI and VMware Constructs

Cisco Application Centric Infrastructure (ACI) and VMware use different terms to describe the same constructs. This section provides a table for mapping Cisco ACI and VMware terminology; the information is relevant to VMware vSphere Distributed Switch (VDS), Cisco ACI Virtual Edge, and Cisco Application Virtual Switch (AVS).

Cisco ACI Terms

VMware Terms

Endpoint group (EPG)

Port group, portgroup

LACP Active

- Route based on IP hash (downlink port group)

- LACP Enabled/Active (uplink port group)

LACP Passive

- Route based on IP hash (downlink port group)

- LACP Enabled/Active (uplink port group)

MAC Pinning

Note

This construct does not apply to Cisco AVS.

- Route based on originating virtual port

- LACP Disabled

MAC Pinning-Physical-NIC-Load

- Route based on physical NIC load

- LACP Disabled

Static Channel - Mode ON

- Route based on IP Hash (downlink port group)

- LACP Disabled

Virtual Machine Manager (VMM) domain

VDS

VM controller

vCenter (Datacenter)

Virtual Machine Manager Domain Main Components

ACI fabric virtual machine manager (VMM) domains enable an administrator to configure connectivity policies for virtual machine controllers. The essential components of an ACI VMM domain policy include the following:

- Virtual Machine Manager Domain Profile—Groups VM controllers with similar networking policy requirements. For example, VM controllers can share VLAN pools and application endpoint groups (EPGs). The APIC communicates with the controller to publish network configurations such as port groups that are then applied to the virtual workloads. The VMM domain profile includes the following essential components:

- Credential—Associates a valid VM controller user credential with an APIC VMM domain.

- Controller—Specifes how to connect to a VM controller that is part of a policy enforcement domain. For example, the controller specifies the connection to a VMware vCenter that is part a VMM domain.

Note

A single VMM domain can contain multiple instances of VM controllers, but they must be from the same vendor (for example, from VMware or from Microsoft.

- EPG Association—Endpoint groups regulate connectivity and visibility among the endpoints within the scope of the VMM domain policy. VMM domain EPGs behave as follows:

- The APIC pushes these EPGs as port groups into the VM controller.

- An EPG can span multiple VMM domains, and a VMM domain can contain multiple EPGs.

- Attachable Entity Profile Association—Associates a VMM domain with the physical network infrastructure. An attachable entity profile (AEP) is a network interface template that enables deploying VM controller policies on a large set of leaf switch ports. An AEP specifies which switches and ports are available, and how they are configured.

- VLAN Pool Association—A VLAN pool specifies the VLAN IDs or ranges used for VLAN encapsulation that the VMM domain consumes.

Virtual Machine Manager Domains

An APIC VMM domain profile is a policy that defines a VMM domain. The VMM domain policy is created in APIC and pushed into the leaf switches.

VMM domains provide the following:

- A common layer in the ACI fabric that enables scalable fault-tolerant support for multiple VM controller platforms.

- VMM support for multiple tenants within the ACI fabric.

VMM domains contain VM controllers such as VMware vCenter or Microsoft SCVMM Manager and the credential(s) required for the ACI API to interact with the VM controller. A VMM domain enables VM mobility within the domain but not across domains. A single VMM domain can contain multiple instances of VM controllers but they must be the same kind. For example, a VMM domain can contain many VMware vCenters managing multiple controllers each running multiple VMs but it may not also contain SCVMM Managers. A VMM domain inventories controller elements (such as pNICs, vNICs, VM names, and so forth) and pushes policies into the controller(s), creating port groups, and other necessary elements. The ACI VMM domain listens for controller events such as VM mobility and responds accordingly.

VMM Domain VLAN Pool Association

VLAN pools represent blocks of traffic VLAN identifiers. A VLAN pool is a shared resource and can be consumed by multiple domains such as VMM domains and Layer 4 to Layer 7 services.

Each pool has an allocation type (static or dynamic), defined at the time of its creation. The allocation type determines whether the identifiers contained in it will be used for automatic assignment by the Cisco APIC (dynamic) or set explicitly by the administrator (static). By default, all blocks contained within a VLAN pool have the same allocation type as the pool but users can change the allocation type for encapsulation blocks contained in dynamic pools to static. Doing so excludes them from dynamic allocation.

A VMM domain can associate with only one dynamic VLAN pool. By default, the assignment of VLAN identifiers to EPGs that are associated with VMM domains is done dynamically by the Cisco APIC. While dynamic allocation is the default and preferred configuration, an administrator can statically assign a VLAN identifier to an endpoint group (EPG) instead. In that case, the identifiers used must be selected from encapsulation blocks in the VLAN pool associated with the VMM domain, and their allocation type must be changed to static.

The Cisco APIC provisions VMM domain VLAN on leaf ports based on EPG events, either statically binding on leaf ports or based on VM events from controllers such as VMware vCenter or Microsoft SCVMM.

Note

In dynamic VLAN pools, if a VLAN is disassociated from an EPG, it is automatically reassociated with the EPG in five minutes.

Note

Dynamic VLAN association is not a part of configuration rollback, that is, in case an EPG or tenant was initially removed and then restored from the backup, a new VLAN is automatically allocated from the dynamic VLAN pools.

VMM Domain EPG Association

The Cisco Application Centric Infrastructure (ACI) fabric associates tenant application profile endpoint groups (EPGs) to virtual machine manager (VMM) domains, The Cisco ACI does so either automatically by an orchestration component such as Microsoft Azure, or by a Cisco Application Policy Infrastructure Controller (APIC) administrator creating such configurations. An EPG can span multiple VMM domains, and a VMM domain can contain multiple EPGs.

VMM Domain EPG Association

In the preceding illustration, end points (EPs) of the same color are part of the same EPG. For example, all the green EPs are in the same EPG although they are in two different VMM domains.

See the latest Verified Scalability Guide for Cisco ACI for virtual network and VMM domain EPG capacity information.

VMM Domain EPG VLAN Consumption

Note

When multiple VMM domains with an overlapping VLAN ID range are connected to the same leaf switch, those domains should use the same VLAN pool. With the same VLAN pool, Cisco APIC can make sure to pick a different VLAN ID for each domain-to-EPG association. Otherwise, Cisco APIC might pick a VLAN ID that is already used on the switch for another domain-to-EPG association, which causes the VLAN deployment fail.

When multiple VMM domains with an overlapping VLAN ID range are connected to the same leaf switch and those domains use the same VLAN pool, you can have multiple VMM domains associated with the same EPG. However, each domain-to-EPG association deploys a different VLAN ID, respectively, even though the VLANs are for the same EPG and potentially are on the same port. If using VLAN IDs in this manner is suboptimal to your requirements, you can use the same VMM domain with multiple VMM controllers instead of having multiple VMM domains.

EPGs can use multiple VMM domains in the following ways:

- An EPG within a VMM domain is identified by using an encapsulation identifier. Cisco APIC can manage the identifier automatically, or the administrator can statically select it. An example is a VLAN, a Virtual Network ID (VNID).

- An EPG can be mapped to multiple physical (for baremetal servers) or virtual domains. It can use different VLAN or VNID encapsulations in each domain.

Note

By default, the Cisco APIC dynamically manages the allocatiion of a VLAN for an EPG. VMware DVS administrators have the option to configure a specific VLAN for an EPG. In that case, the VLAN is chosen from a static allocation block within the pool that is associated with the VMM domain.

Applications can be deployed across VMM domains.

Multiple VMM Domains and Scaling of EPGs in the Fabric

While live migration of VMs within a VMM domain is supported, live migration of VMs across VMM domains is not supported.

Note

When you change the VRF on a bridge domain that is linked to an EPG with an associated VMM domain, the port-group is deleted and then added back on vCenter. This results in the EPG being undeployed from the VMM domain. This is expected behavior.

About Trunk Port Group

You use a trunk port group to aggregate the traffic of endpoint groups (EPGs) for VMware virtual machine manager (VMM) domains. Unlike regular port groups, which are configured under the Tenants tab in the Cisco Application Policy Infrastructure Controller (APIC) GUI, trunk port groups are configured under the VM Networking tab. Regular port groups follow an EPG’s T|A|E name format.

The aggregation of EPGs under the same domain is based on a VLAN range, which is specified as encapsulation blocks contained in the trunk port group. Whenever a EPG’s encapsulation is changed or a trunk port group’s encapsulation block is changed, the aggregation is be re-evaluated to determine if the EGP should be aggregated.

A trunk port group controls the leaf deployment of network resources, such as VLANs, that allocated to the EPGs being aggregated. The EPGs include both base EPG and microsegmented (uSeg) EPGs. In the case of a uSeg EPG, the trunk port group’s VLAN ranges need to include both the primary and secondary VLANs.

Note

Cisco ACI does not support IP fragmentation. Therefore, when you configure Layer 3 Outside (L3Out) connections to external routers, or Multi-Pod connections through an Inter-Pod Network (IPN), it is recommended that the interface MTU is set appropriately on both ends of a link. On some platforms, such as Cisco ACI, Cisco NX-OS, and Cisco IOS, the configurable MTU value does not take into account the Ethernet headers (matching IP MTU, and excluding the 14-18 Ethernet header size), while other platforms, such as IOS-XR, include the Ethernet header in the configured MTU value. A configured value of 9000 results in a max IP packet size of 9000 bytes in Cisco ACI, Cisco NX-OS, and Cisco IOS, but results in a max IP packet size of 8986 bytes for an IOS-XR untagged interface.

For the appropriate MTU values for each platform, see the relevant configuration guides.

We highly recommend that you test the MTU using CLI-based commands. For example, on the Cisco NX-OS CLI, use a command such as ping 1.1.1.1 df-bit packet-size 9000 source-interface ethernet 1/1.

Caution

If you install 1 Gigabit Ethernet (GE) or 10GE links between the leaf and spine switches in the fabric, there is risk of packets being dropped instead of forwarded, because of inadequate bandwidth. To avoid the risk, use 40GE or 100GE links between the leaf and spine switches.

Note

Multiple Spanning Tree (MST) is not supported on interfaces configured with the Per Port VLAN feature (configuring multiple EPGs on a leaf switch using the same VLAN ID with localPort scope).

Note

If you are using Cisco ACI Multi-Site with this Cisco APIC cluster/fabric, look for a cloud icon on the object names in the navigation bar. This indicates that the information is derived from Multi-Site. It is recommended to only make changes from the Multi-Site GUI. Please review the Multi-Site documentation before making changes here.

Note

For a Cisco APIC REST API query of event records, the Cisco APIC system limits the response to a maximum of 500,000 event records. If the response is more than 500,000 events, it returns an error. Use filters to refine your queries. For more information, see Composing Query Filter Expressions in the Cisco APIC REST API Configuration Guide.

For more information, see

- Creating a Trunk Port Group Using the GUI

- Creating a Trunk Port Group Using the NX-OS Style CLI

- Creating a Trunk Port Group Using the REST API

Attachable Entity Profile

The ACI fabric provides multiple attachment points that connect through leaf ports to various external entities such as bare metal servers, virtual machine hypervisors, Layer 2 switches (for example, the Cisco UCS fabric interconnect), or Layer 3 routers (for example Cisco Nexus 7000 Series switches). These attachment points can be physical ports, FEX ports, port channels, or a virtual port channel (vPC) on leaf switches.

Note

When creating a VPC domain between two leaf switches, both switches must be in the same switch generation, one of the following:

- Generation 1 - Cisco Nexus N9K switches without “EX” or "FX" on the end of the switch name; for example, N9K-9312TX

- Generation 2 – Cisco Nexus N9K switches with “EX” or "FX" on the end of the switch model name; for example, N9K-93108TC-EX

Switches such as these two are not compatible VPC peers. Instead, use switches of the same generation.

An Attachable Entity Profile (AEP) represents a group of external entities with similar infrastructure policy requirements. The infrastructure policies consist of physical interface policies that configure various protocol options, such as Cisco Discovery Protocol (CDP), Link Layer Discovery Protocol (LLDP), or Link Aggregation Control Protocol (LACP).

An AEP is required to deploy VLAN pools on leaf switches. Encapsulation blocks (and associated VLANs) are reusable across leaf switches. An AEP implicitly provides the scope of the VLAN pool to the physical infrastructure.

The following AEP requirements and dependencies must be accounted for in various configuration scenarios, including network connectivity, VMM domains, and multipod configuration:

- The AEP defines the range of allowed VLANS but it does not provision them. No traffic flows unless an EPG is deployed on the port. Without defining a VLAN pool in an AEP, a VLAN is not enabled on the leaf port even if an EPG is provisioned.

- A particular VLAN is provisioned or enabled on the leaf port that is based on EPG events either statically binding on a leaf port or based on VM events from external controllers such as VMware vCenter or Microsoft Azure Service Center Virtual Machine Manager (SCVMM).

- Attached entity profiles can be associated directly with application EPGs, which deploy the associated application EPGs to all those ports associated with the attached entity profile. The AEP has a configurable generic function (infraGeneric), which contains a relation to an EPG (infraRsFuncToEpg) that is deployed on all interfaces that are part of the selectors that are associated with the attachable entity profile.

A virtual machine manager (VMM) domain automatically derives physical interface policies from the interface policy groups of an AEP.

An override policy at the AEP can be used to specify a different physical interface policy for a VMM domain. This policy is useful in scenarios where a VM controller is connected to the leaf switch through an intermediate Layer 2 node, and a different policy is desired at the leaf switch and VM controller physical ports. For example, you can configure LACP between a leaf switch and a Layer 2 node. At the same time, you can disable LACP between the VM controller and the Layer 2 switch by disabling LACP under the AEP override policy.

EPG Policy Resolution and Deployment Immediacy

Whenever an endpoint group (EPG) associates to a virtual machine manager (VMM) domain, the administrator can choose the resolution and deployment preferences to specify when a policy should be pushed into leaf switches.

Resolution Immediacy

- Pre-provision: Specifies that a policy (for example, VLAN, VXLAN binding, contracts, or filters) is downloaded to a leaf switch even before a VM controller is attached to the virtual switch (for example, VMware vSphere Distributed Switch (VDS). This pre-provisions the configuration on the switch.

This helps the situation where management traffic for hypervisors/VM controllers is also using the virtual switch associated to the Cisco Application Policy Infrastructure Controller (APIC) VMM domain (VMM switch).

Deploying a VMM policy such as VLAN on Cisco Application Centric Infrastructure (ACI) leaf switch requires Cisco APICto collect CDP/LLDP information from both hypervisors through the VM controller and Cisco ACI leaf switch. However, if the VM controller is supposed to use the same VMM policy (VMM switch) to communicate with its hypervisors or even Cisco APIC, the CDP/LLDP information for hypervisors can never be collected because the policy that is required for VM controller/hypervisor management traffic is not deployed yet.

When using pre-provision immediacy, policy is downloaded to Cisco ACI leaf switch regardless of CDP/LLDP neighborship. Even without a hypervisor host that is connected to the VMM switch.

- Immediate: Specifies that EPG policies (including contracts and filters) are downloaded to the associated leaf switch software upon ESXi host attachment to a DVS. LLDP or OpFlex permissions are used to resolve the VM controller to leaf node attachments.

The policy will be downloaded to leaf when you add host to the VMM switch. CDP/LLDP neighborship from host to leaf is required.

- On Demand: Specifies that a policy (for example, VLAN, VXLAN bindings, contracts, or filters) is pushed to the leaf node only when an ESXi host is attached to a DVS and a VM is placed in the port group (EPG).

The policy will be downloaded to the leaf when host is added to the VMM switch. The VM needs to be placed into a port group (EPG). CDP/LLDP neighborship from host to leaf is required.

With both immediate and on demand, if host and leaf lose LLDP/CDP neighborship the policies are removed.

Note

In OpFlex-based VMM domains, an OpFlex agent on the hypervisor reports a VM/EP virtual network interface card (vNIC) attachment to an EPG to the leaf OpFlex process. When using On Demand Resolution Immediacy, the EPG VLAN/VXLAN is programmed on all leaf port channel ports, virtual port channel ports, or both when the following are true:

- Hypervisors are connected to leafs on port channel or virtual port channel attached directly or through blade switches.

- A VM or instance vNIC is attached to an EPG.

- Hypervisors are attached as part of the EPG or VMM domain.

Opflex-based VMM domains are Microsoft Security Center Virtual Machine Manager (SCVMM) and HyperV, Cisco ACI Virtual Edge, Cisco Application Virtual Switch (AVS), and VMware VDS domains.

Deployment Immediacy

Once the policies are downloaded to the leaf software, deployment immediacy can specify when the policy is pushed into the hardware policy content-addressable memory (CAM).

- Immediate: Specifies that the policy is programmed in the hardware policy CAM as soon as the policy is downloaded in the leaf software.

- On demand: Specifies that the policy is programmed in the hardware policy CAM only when the first packet is received through the data path. This process helps to optimize the hardware space.

Note

When you use on demand deployment immediacy with MAC-pinned VPCs, the EPG contracts are not pushed to the leaf ternary content-addressble memory (TCAM) until the first endpoint is learned in the EPG on each leaf. This can cause uneven TCAM utilization across VPC peers. (Normally, the contract would be pushed to both peers.)

Guidelines for Deleting VMM Domains

Follow the sequence below to assure that the APIC request to delete a VMM domain automatically triggers the associated VM controller (for example VMware vCenter or Microsoft SCVMM) to complete the process normally, and that no orphan EPGs are stranded in the ACI fabric.

1. The VM administrator must detach all the VMs from the port groups (in the case of VMware vCenter) or VM networks (in the case of SCVMM), created by the APIC.

In the case of Cisco AVS, the VM admin also needs to delete vmk interfaces associated with the Cisco AVS.

2. The ACI administrator deletes the VMM domain in the APIC. The APIC triggers deletion of VMware VDS or Cisco AVS or SCVMM logical switch and associated objects.

Note

The VM administrator should not delete the virtual switch or associated objects (such as port groups or VM networks); allow the APIC to trigger the virtual switch deletion upon completion of step 2 above. EPGs could be orphaned in the APIC if the VM administrator deletes the virtual switch from the VM controller before the VMM domain is deleted in the APIC.

If this sequence is not followed, the VM controller does delete the virtual switch associated with the APIC VMM domain. In this scenario, the VM administrator must manually remove the VM and vtep associations from the VM controller, then delete the virtual switch(es) previously associated with the APIC VMM domain.

About NetFlow with Virtual Machine Networking

The NetFlow technology provides the metering base for a key set of applications, including network traffic accounting, usage-based network billing, network planning, as well as denial of services monitoring, network monitoring, outbound marketing, and data mining for both service providers and enterprise customers. Cisco provides a set of NetFlow applications to collect NetFlow export data, perform data volume reduction, perform post-processing, and provide end-user applications with easy access to NetFlow data. If you have enabled NetFlow monitoring of the traffic flowing through your datacenters, this feature enables you to perform the same level of monitoring of the traffic flowing through the Cisco Application Centric Infrastructure (Cisco ACI) fabric.

Instead of hardware directly exporting the records to a collector, the records are processed in the supervisor engine and are exported to standard NetFlow collectors in the required format.

For more information about NetFlow, see the Cisco APIC and NetFlow knowledge base article.

About NetFlow Exporter Policies with Virtual Machine Networking

A virtual machine manager exporter policy (netflowVmmExporterPol) describes information about the data collected for a flow that is sent to the reporting server or NetFlow collector. A NetFlow collector is an external entity that supports the standard NetFlow protocol and accepts packets marked with valid NetFlow headers.

An exporter policy has the following properties:

- VmmExporterPol.dstAddr—This mandatory property specifies the IPv4 or IPv6 address of the NetFlow collector that accepts the NetFlow flow packets. This must be in the host format (that is, "/32" or "/128"). An IPv6 address is supported in vSphere Distributed Switch (vDS) version 6.0 and later.

- VmmExporterPol.dstPort—This mandatory property specifies the port on which the NetFlow collector application is listening on, which enables the collector to accept incoming connections.

- VmmExporterPol.srcAddr—This optional property specifies the IPv4 address that is used as the source address in the exported NetFlow flow packets.

NetFlow Support with VMware vSphere Distributed Switch

The VMware vSphere Distributed Switch (VDS) supports NetFlow with the following caveats:

- The external collector must be reachable through the ESX. ESX does not support virtual routing and forwardings (VRFs).

- A port group can enable or disable NetFlow.

- VDS does not support flow-level filtering.

Configure the following VDS parameters in VMware vCenter:

- Collector IP address and port. IPv6 is supported on VDS version 6.0 or later. These are mandatory.

- Source IP address. This is optional.

- Active flow timeout, idle flow timeout, and sampling rate. These are optional.

NetFlow Support with Cisco Application Virtual Switch

The Cisco Application Virtual Switch (AVS) supports NetFlow with the following caveats:

- The external collector must be reachable through the ESX. ESX does not support virtual routing and forwardings (VRFs).

- A port group can enable or disable NetFlow and specify the direction of the traffic to be collected.

- Cisco AVS does not support flow-level filtering.

Configuring a NetFlow Exporter Policy for VM Networking Using the GUI

The following procedure configures a NetFlow exporter policy for VM networking.

Procedure

Step 1

On the menu bar, choose Fabric > Access Policies.

Step 2

In the navigation pane, expand Policies > Interface > NetFlow.

Step 3

Right-click NetFlow Exporters for VM Networking and choose Create NetFlow Exporter for VM Networking.

Step 4

In the Create NetFlow Exporter for VM Networking dialog box, fill in the fields as required.

Step 5

Click Submit.

Consuming a NetFlow Exporter Policy Under a VMM Domain Using the GUI

The following procedure consumes a NetFlow exporter policy under a VMM domain using the GUI.

Procedure

Step 1

On the menu bar, choose Virtual Networking > Inventory.

Step 2

In the Navigation pane, expand the VMM Domainsfolder, right-click VMware, and choose Create vCenter Domain.

Step 3

In the Create vCenter Domain dialog box, fill in the fields as required, except as specified:

1. In the NetFlow Exporter Policy drop-down list, choose the desired exporter policy or create a new one.

2. In the Active Flow Timeout field, enter the desired active flow timeout, in seconds.

The Active Flow Timeout parameter specifies the delay that NetFlow waits after the active flow is initiated, after which NetFlow sends the collected data. The range is from 60 to 3600. The default value is 60.

3. In the Idle Flow Timeout field, enter the desired idle flow timeout, in seconds.

The Idle Flow Timeout parameter specifies the delay that NetFlow waits after the idle flow is initiated, after which NetFlow sends the collected data. The range is from 10 to 300. The default value is 15.

4. (VDS only) In the Sampling Rate field, enter the desired sampling rate.

The Sampling Rate parameter specifies how many packets that NetFlow will drop after every collected packet. If you specify a value of 0, then NetFlow does not drop any packets. The range is from 0 to 1000. The default value is 0.

Step 4

Click Submit.

Enabling NetFlow on an Endpoint Group to VMM Domain Association Using the GUI

The following procedure enables NetFlow on an endpoint group to VMM domain association.

Before you begin

You must have configured the following:

- An application profile

- An application endpoint group

Procedure

Step 1

On the menu bar, choose Tenants > All Tenants.

Step 2

In the Work pane, double-click the tenant's name.

Step 3

In the left navigation pane, expand tenant_name > Application Profiles > application_profile_name > Application EPGs > application_EPG_name

Step 4

Right-click Domains (VMs and Bare-Metals) and choose Add VMM Domain Association.

Step 5

In the Add VMM Domain Association dialog box, fill in the fields as required, except as specified below:

1. In the NetFlow are, choose Enable.

2. (Cisco AVS only) Click NetFlow Direction and choose ingress, egress, or both for the flows that need to be monitored and collected.

Step 6

Click Submit.

Troubleshooting VMM Connectivity

The following procedure resolves VMM connectivity issues:

Procedure

Step 1

Trigger inventory resync on the Application Policy Infrastructure Controller (APIC).

For more information about how to trigger an inventory resync on APIC, see the following knowledge base article:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_VMM_OnDemand_Inventory_in_APIC.html

Step 2

If step 1 does not fix the issue, for the impacted EPGs, set the resolution immediacy to use preprovisioning in the VMM domain.

"Pre-Provision” removes the need for neighbor adjacencies or OpFlex permissions and subsequently the dynamic nature of VMM Domain VLAN Programming. For more information about Resolution Immediacy types, see the following EPG Policy Resolution and Deployment Immediacy section:

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_01011.html#concept_EF87ADDAD4EF47BDA741EC6EFDAECBBD

Step 3

If steps 1 and 2 do not fix the issue and you see the issue on all of the VMs, then delete the VM controller policy and readd the policy.

Note

Deleting the controller policy impacts traffic for all VMs that are on that controller.

Chapter 3. Cisco ACI with VMware VDS Integration

This chapter contains the following sections:

- Configuring Virtual Machine Networking Policies

- Creating a VMM Domain Profile

- Creating VDS Uplink Port Groups

- Creating a Trunk Port Group Using the GUI

- Troubleshooting the Cisco ACI and VMware VMM System Integration

Configuring Virtual Machine Networking Policies

Cisco Application Policy Infrastructure Controller (APIC) integrates with third-party VM managers (VMMs)—such as VMware vCenter—to extend the benefits of Cisco Application Centric Infrastructure (ACI) to the virtualized infrastructure. Cisco APIC enables the administrator to use Cisco ACI policies inside the VMM system.

The following modes of Cisco ACI and VMware VMM integration are supported:

- VMware VDS: When integrated with Cisco ACI, the VMware vSphere Distributed Switch (VDS) enables you to configure VM networking in the Cisco ACI fabric.

- Cisco ACI Virtual Edge: For information about how to install and configure Cisco ACI Virtual Edge, see the Cisco ACI Virtual Edge Installation Guide and the Cisco ACI Virtual Edge Configuration Guide on Cisco.com.

- Cisco Application Virtual Switch (AVS): For information about how to install and configure Cisco AVS with Cisco ACI, see Cisco AVS documentation on Cisco.com.

Note

When a Cisco APIC is connected to a VMware vCenter with many folders, you may see a delay when pushing new port groups from the Cisco APIC to the VMWare vCenter.

- Cisco APIC Supported VMware VDS Versions

- Guidelines for Upgrading VMware DVS from 5.x to 6.x and VMM Integration

- Guidelines for VMware VDS Integration

- Mapping Cisco ACI and VMware Constructs

Cisco APIC Supported VMware VDS Versions

See the Cisco ACI Virtualization Compatibility Matrix for information about the compatibility of VMware components with Cisco APIC.

Note

VMware vSphere version 6.7 includes vCenter 6.7, ESXi 6.7, and DVS 6.6.

Note

When adding additional VMware ESXi hosts to the VMM domain with VMware vSphere Distributed Switch (VDS), ensure that the version of ESXi host is compatible with the Distributed Virtual Switch (DVS) version already deployed in the vCenter. For more information about VMware VDS compatibility requirements for ESXi hosts, see the VMware documentation.

If the ESXi host version is not compatible with the existing DVS version, vCenter will not be able to add the ESXi host to the DVS, and an incompatibility error will occur. Modification of the existing DVS version setting from the Cisco APIC is not possible. To lower the DVS version in the vCenter, you need to remove and reapply the VMM domain configuration with a lower setting.

Important

If you have ESXi 6.5 hosts running UCS B-Series or C-Series server with VIC cards, some of the vmnics may go down on a port state event, such as a link flap or a TOR reload. To prevent this problem, do not use the default eNIC driver but install it from the VMware website: https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI65-CISCO-NENIC-1020&productId=614.

Guidelines for Upgrading VMware DVS from 5.x to 6.x and VMM Integration

This section describes the guidelines for upgrading VMware Distributed Virtual Switch (DVS) from 5.x to 6.x and VMM integration.

- DVS versioning is only applicable to the VMware DVS and not the Cisco Application Virtual Switch (AVS). DVS upgrades are initiated from VMware vCenter, or the relevant orchestration tool and not ACI. The Upgrade Version option appears grayed out for AVS switches within vCenter.

- If you are upgrading the DVS from 5.x to 6.x, you must upgrade the vCenter Server to version 6.0 and all hosts connected to the distributed switch to ESXi 6.0. For full details on upgrading your vCenter and Hypervisor hosts, see VMware's upgrade documentation. To upgrade the DVS go to the Web Client: Home > Networking > DatacenterX > DVS-X > Actions Menu > Upgrade Distributed Switch.

- There is no functional impact on the DVS features, capability, performance and scale if the DVS version shown in vCenter does not match the VMM domain DVS version configured on the APIC. The APIC and VMM Domain DVS Version is only used for initial deployment.

Guidelines for VMware VDS Integration

Follow the guidelines in this section when integrating VMware vSphere Distributed Switch (VDS) into Cisco Application Centric Infrastructure (ACI).

- Do not change the following settings on a VMware VDS configured for VMM integration:

- VMware vCenter hostname (if you are using DNS).

- VMware vCenter IP address (if you are using IP).

- VMware vCenter credentials used by Cisco APIC.

- Data center name

- Folder, VDS, or portgroup name.

- Folder structure containing the VMware VDS.

For example, do not put the folder in another folder.

- Uplink port-channel configuration, including LACP/port channel, LLDP, and CDP configuration

- VLAN on a portgroup

- Active uplinks for portgroups pushed by Cisco APIC.

- Security parameters (promiscuous mode, MAC address changes, forged transmits) for portgroups pushed by Cisco APIC.

- Use supported versions of VMware vCenter/vSphere with the version of Cisco ACI that you are running.

- If you are adding or removing any portgroups, use Cisco APIC or the Cisco ACI vCenter plug-in in VMware vCenter.

- Know that Cisco APIC may overwrite some changes that are made in VMware vCenter.

For example, when Cisco APIC updates a portgroup, port binding, promiscuous mode, and load-balancing can be overwritten

Mapping Cisco ACI and VMware Constructs

Mapping of Cisco Application Centric Infrastructure (ACI) and VMware Constructs

Cisco ACI Terms

VMware Terms

Endpoint group (EPG)

Port group

LACP Active

- Route based on IP hash (downlink port group)

- LACP Enabled/Active (uplink port group)

LACP Passive

- Route based on IP hash (downlink port group)

- LACP Enabled/Active (uplink port group)

MAC Pinning

- Route based on originating virtual port

- LACP Disabled

MAC Pinning-Physical-NIC-Load

- Route based on physical NIC load

- LACP Disabled

Static Channel - Mode ON

- Route Based on IP Hash (downlink port group)

- LACP Disabled

Virtual Machine Manager (VMM) Domain

vSphere Distributed Switch (VDS)

VM controller

vCenter (Datacenter)

VDS Parameters Managed by APIC

See the section Mapping Cisco ACI and VMware Constructs in this guide for a table of corresponding Cisco Application Centric Infrastructure (ACI) and VMware terminology.

VMware VDS

Default Value

Configurable Using Cisco APIC Policy?

Name

VMM domain name

Yes (Derived from Domain)

Description

"APIC Virtual Switch"

No

Folder Name

VMM domain name

Yes (Derived from Domain)

Version

Highest supported by vCenter

Yes

Discovery Protocol

LLDP

Yes

Uplink Ports and Uplink Names

8

No

Uplink Name Prefix

uplink

No

Maximum MTU

9000

Yes

LACP policy

disabled

Yes

Port mirroring

0 sessions

Yes

Alarms

2 alarms added at the folder level

No

VDS Port Group Parameters Managed by APIC

VMware VDS Port Group

Default Value

Configurable using APIC Policy

Name

Tenant Name | Application Profile Name | EPG Name

Yes (Derived from EPG)

Port binding

Static binding

No

VLAN

Picked from VLAN pool

Yes

Load balancing algorithm

Derived based on port-channel policy on APIC

Yes

Promiscuous mode

Disabled

Yes

Forged transmit

Disabled

Yes

Mac change

Disabled

Yes

Block all ports

False

No

Creating a VMM Domain Profile

VMM domain profiles specify connectivity policies that enable virtual machine controllers to connect to the Cisco ACI fabric. They group VM controllers with similar networking policy requirements. For example, VM controllers can share VLAN pools and application endpoint groups (EPGs). The Cisco APIC communicates with the controller to publish network configurations such as port groups that are then applied to the virtual workloads. For details, see the Cisco Application Centric Infrastructure Fundamentals on Cisco.com.

Beginning with Cisco APIC Release 3.1(1), you also can create a read-only VMM domain. A read-only VMM domain enables you to view inventory information for a VDS in the VMware vCenter that Cisco APIC does not manage. Procedures to configure a read-only VMM domain differ slightly from procedures to create other VMM domains. However, the same workflow and prerequisites apply.

Note

In this section, examples of a VMM domain are vCenter domain.

- GUI Tasks

- Prerequisites for Creating a VMM Domain Profile

- vCenter Domain Operational Workflow

- Creating a Read-Only VMM Domain

- Enhanced LACP Policy Support

- Endpoint Retention Configuration

GUI Tasks

This section shows how to perform tasks using GUI.

- For references to REST API tasks, refer to the REST API tasks section.

- For references to NX-OS Style CLI tasks, refer to the CLI tasks section.

Prerequisites for Creating a VMM Domain Profile

To configure a VMM domain profile, you must meet the following prerequisites:

- All fabric nodes are discovered and configured.

- Inband (inb) or out-of-band (oob) management has been configured on the APIC.

- A Virtual Machine Manager (VMM) is installed, configured, and reachable through the inb/oob management network (for example, a vCenter).

- You have the administrator/root credentials to the VMM (for example vCenter).

Note

If you prefer not to use the vCenter admin/root credentials, you can create a custom user account with minimum required permissions. See Custom User Account with Minimum VMware vCenter Privileges for a list of the required user privileges.

- A DNS policy for the APIC must be configured if you plan to reference the VMM by hostname rather than an IP address.

vCenter Domain Operational Workflow

A Sequential Illustration of the vCenter Domain Operational Workflow

The APIC administrator configures the vCenter domain policies in the APIC. The APIC administrator provides the following vCenter connectivity information:

- The vCenter IP address, vCenter credentials, VMM domain policies, and VMM domain SPAN

- Policies (VLAN pools, domain type such as VMware VDS, Cisco Nexus 1000V switch)

- Connectivity to physical leaf inerfaces (using attach entity profiles)

1. The APIC automatically connects to the vCenter.

2. The APIC creates the VDS—or uses an existing VDS if there is one already created—matching the name of the VMM domain.

Note

If you use an existing VDS, the VDS must be inside a folder with the same name.

Note

If you want to see an existing VDS from the vCenter, you can do so by specifying the Read Only Mode in the Access Mode area when you create a VMM domain with the same name as the VDS in vCenter using the Cisco APIC. This VMM in Read Only Mode is not managed by APIC. You may not be able to modify any properties of this VMM domain except vCenter user credentials and vCenter IP address.

3. The vCenter administrator or the compute management tool adds the ESX host or hypervisor to the APIC VDS and assigns the ESX host hypervisor ports as uplinks on the APIC VDS. These uplinks must connect to the ACI leaf switches.

4. The APIC learns the location of the hypervisor host to the leaf connectivity using LLDP or CDP information of the hypervisors.

5. The APIC administrator creates and associates application EPG policies.

6. The APIC administrator associates EPG policies to VMM domains.

7. The APIC automatically creates port groups in the VMware vCenter under the VDS. This process provisions the network policy in the VMware vCenter.

Note

- The port group name is a concatenation of the tenant name, the application profile name, and the EPG name.

- The port group is created under the VDS, and it was created earlier by the APIC.

8. The vCenter administrator or the compute management tool instantiates and assigns VMs to the port groups.

9. The APIC learns about the VM placements based on the vCenter events. The APIC automatically pushes the application EPG and its associated policy (for example, contracts and filters) to the ACI fabric.

- Creating a vCenter Domain Profile Using the GUI

Creating a vCenter Domain Profile Using the GUI

An overview of the tasks that you perform to create a vCenter Domain are as follows (details are in the steps that follow):

- Create or select a switch profile.

- Create or select an interface profile.

- Create or select an interface policy group.

- Create or select VLAN pool.

- Create vCenter domain.

- Create vCenter credentials.

Procedure

Step 1

On the menu bar, click Fabric > Access Policies.

Step 2

In the navigation pane, click Quick Start, and then in the central pane click Configure an interface, PC, and VPC.

Step 3

In the Configure an interface, PC, and VPC dialog box, perform the following actions:

1. Expand Configured Switch Interfaces.

2. Click the + icon.

3. Make sure that the Quick radio button is chosen.

4. From the Switches drop-down list, choose the appropriate leaf ID.

In the Switch Profile Name field, the switch profile name automatically populates.

5. Click the + icon to configure the switch interfaces.

6. In the Interface Type area, check the appropriate radio button.

7. In the Interfaces field, enter the desired interface range.

8. In the Interface Selector Name field, the selector name automatically populates.

9. In the Interface Policy Group area, choose the Create One radio button.

10. From the Link Level Policy drop-down list, choose the desired link level policy.

11. From the CDP Policy drop-down list, choose the desired CDP policy.

Note

Similarly choose the desired interface policies from the available policy areas.

12. In the Attached Device Type area, choose ESX Hosts.

13. In the Domain area, make sure that the Create One radio button is chosen.

14. In the Domain Name field, enter the domain name.

15. In the VLAN area, make sure that the Create One radio button is chosen.

16. In the VLAN Range field, enter the VLAN range as appropriate.

Note

We recommend a range of at least 200 VLAN numbers. Do not define a range that includes your manually assigned infra VLAN. If you do so, it can trigger a fault, depending on your version of Cisco Application Policy Infrastructure Controller (APIC). There are specific use cases and options to be set if your infra VLAN needs to be extended as part of an OpFlex integration.

17. In the vCenter Login Name field, enter the login name.

18. (Optional) From the Security Domains drop-down list, choose the appropriate security domain.

19. In the Password field, enter a password.

20. In the Confirm Password field, reenter the password.

21. Expand vCenter.

Step 4

In the Create vCenter Controller dialog box, enter the appropriate information, and click OK.

Step 5

In the Configure Interface, PC, And VPC dialog box, complete the following actions:

If you do not specify policies in the Port Channel Mode and the vSwitch Policy areas, the same policies that you configured earlier in this procedure will take effect for the vSwitch.

1. From the Port Channel Mode drop-down list, choose a mode.

2. In the vSwitch Policy area, click the desired radio button to enable CDP or LLDP.

3. From the NetFlow Exporter Policy drop-down list, choose a policy or create one.

A NetFlow exporter policy configures the external collector reachability.

4. Choose values from the Active Flow Time0ut, Idle Flow Timeout, and Sampling Rate drop-down lists.

5. Click SAVE twice and then click SUBMIT.

Step 6

Verify the new domain and profiles, by performing the following actions:

1. On the menu bar, choose Virtual Networking > Inventory.

2. In the Navigation pane, expand VMM Domains > VMware > Domain_name > vCenter_name.

In the work pane, under Properties, view the VMM domain name to verify that the controller is online. In the Work pane, the vCenter properties are displayed including the operational status. The displayed information confirms that connection from the APIC controller to the vCenter Server is established, and the inventory is available.

Creating a Read-Only VMM Domain

Beginning in Cisco APIC Release 3.1(1), you can create a read-only VMM domain. Doing so enables you to view inventory information for a VDS in the VMware vCenter that Cisco APIC does not manage.

After you create the read-only VMM domain, you can view hypervisors, VMs, NIC status, and other inventory information, as with regular VMM domains. You can associate an EPG to the VMM domain and configure policies for it. However, policies are not pushed from the read-only VMM domain to the VDS. Also, no faults are raised for a read-only VMM domain.

You can create a read-only VMM domain using the Cisco APIC GUI, the NX-OS style CLI, or REST API. The detailed procedures for each of these are available in the subsequent sections of this chapter.

- Creating a Read-Only VMM Domain Using the Cisco APIC GUI

Creating a Read-Only VMM Domain Using the Cisco APIC GUI

In order to create a read-only VMM domain, you create the domain in the Create vCenter Domain dialog box under the Virtual Networking tab. Do not follow the procedure in the section Creating a vCenter Domain Profile Using the GUI to create the domain. That procedure does not enable you to set an access mode for the VMM domain.

Before you begin

- Fulfill the prerequisites in the section Prerequisites for Creating a VMM Domain Profile.

- In the VMware vCenter, ensure that under the Networking tab, the VDS is contained by a folder.

Also ensure that the folder and the VDS have the exact same name of the read-only VMM domain that you plan to create.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Choose Virtual Networking > Inventory and then expand the VMM Domains folder.

Step 3

Right-click the VMM Domains folder and choose Create vCenter Domain.

Step 4

In the Create vCenter Domain dialog box, complete the following steps:

1. In the Virtual Switch Name field, enter a name for the domain.

Note

The name of the read-only domain must be the same as the name of the VDS and the folder that contains in the VMware vCenter.

2. In the Virtual Switch area, choose VMware vSphere Distributed Switch.

3. In the Access Mode area, choose Read Only Mode.

4. In the vCenter Credentials area, click the + (plus) icon, and then create the VMware vCenter credentials for the domain.

5. In the vCenter area, click the + (plus) icon, and then add a vCenter controller for the domain.

6. Click Submit.

What to do next

You can attach an EPG to the read-only VMM domain and configure policies for it. However, those policies are not pushed to the VDS in the VMware vCenter.

Enhanced LACP Policy Support

In Cisco Application Policy Infrastructure Controller (APIC) Release 3.2(7), you can improve uplink load balancing by applying different Link Aggregation Control Protocol (LACP) policies to different distributed virtual switch (DVS) uplink port groups.

Cisco APIC now supports VMware's Enhanced LACP feature, which is available for DVS 5.5 and later. Previously, the same LACP policy applied to all DVS uplink port groups. Before Cisco APIC Release 3.2(7), it was not possible to manage VMware link aggregation groups (LAGs) with Cisco APIC.

You can choose from up to 20 different load-balancing algorithms when you create a VMware vCenter virtual machine manager (VMM) domain for Cisco Application Centric Infrastructure (ACI) Virtual Edge or VMware VDS. You apply different policies to different uplink portgroups.

You have eight DVS uplink portgroups, and you must configure at least two uplinks in the same policy. So you can have up to four different LACP policies for each DVS. Enhanced LACP supports only active and passive LACP modes.

Note

For Cisco ACI Virtual Edge VXLAN mode, it is mandatory to use a load-balancing algorithm having a UDP port. We recommend the algorithm Source and Destination TCP/UDP Port. In VLXAN mode, traffic is always sent between VTEP to the FTEP IP. So communication is always between one pair of IP address. So for VXLAN traffic, the only way to distinguish traffic is using the UDP port number.

The following sections provide instructions for configuring multiple LACP policies for DVS uplinks using the Cisco APIC GUI, NX-OS style CLI, or REST API.

- Enhanced LACP Limitations

- Create LAGs for DVS Uplink Port Groups Using the Cisco APIC GUI

- Associate Application EPGs to VMware vCenter Domains with Enhanced LACP Policies Using the Cisco APIC GUI

- Remove the Enhanced LACP Configuration Before a Downgrade

Enhanced LACP Limitations

Be aware of the following limitations when using enhanced Link Aggregation Control Protocol (LACP) policies.

- You cannot fall back to the previous version of LACP after upgrading to enhanced LACP.

- You cannot downgrade to a version of Cisco Application Policy Infrastructure Controller (APIC) earlier than 3.2(7) without removing the enhanced LACP configuration. See the procedure Remove the Enhanced LACP Configuration Before a Downgrade in this guide.

- Cisco Application Centric Infrastructure (ACI) Virtual Edge, VXLAN mode traffic always uses the source IP address as the TEP IP address. To ensure proper load balancing, we recommend the algorithm Source and Destination TCP/UDP Port.

Create LAGs for DVS Uplink Port Groups Using the Cisco APIC GUI

Improve distributed virtual switch (DVS) uplink port group load balancing by putting the port groups into link aggregation groups (LAGs) and associating them with specific load-balancing algorithms. You can perform this task using the Cisco Application Policy Infrastructure Controller (APIC) GUI.

Before you begin

- You must have created a VMware vCenter virtual machine manager (VMM) domain for VMware VDS or Cisco Application Centric Infrastructure (ACI) Virtual Edge.

- If a vSwitch policy container does not exist, create one.

Procedure

Step 1

Log into the Cisco APIC.

Step 2

Go to Virtual Networking > Inventory > VMM Domains > VMware > domain .

Step 3

In the work pane, choose Policy > VSwitch Policy.

Step 4

If you have not already done so, in the Properties area, choose a policy.

Step 5

In the Enhanced LAG Policy area, click the + (plus) icon and then complete the following steps:

1. In the Name field, enter the name of the LAG.

2. From the Mode drop-down list, choose LACP Active or LACP Passive.

3. From the Load Balancing Mode drop-down list, choose a load-balancing method.

4. In the Number of Links selector, choose how many DVS uplink port groups to include in the LAG.

You can put two to eight uplink port groups into a LAG.

5. Click Update and then click Submit.

Step 6

Repeat Step 5 to create other LAGs for the DVS.

What to do next

If you are using VMware VDS, associate endpoint groups (EPGs) to the domain with the enhanced LACP policy. If you are using Cisco Application Centric Infrastructure (ACI) Virtual Edge, associate internally created inside and outside port groups with the enhanced LACP policy, then associate EPGs to the domain with the policy.

Associate Application EPGs to VMware vCenter Domains with Enhanced LACP Policies Using the Cisco APIC GUI

Associate application endpoint groups (EPGs) with the VMware vCenter domain with LAGs and a load-balancing algorithm. You can perform this task using the Cisco Application Policy Infrastructure Controller (APIC) GUI.

Before you begin

You must have created link aggregation groups (LAGs) for distributed virtual switch (DVS) uplink port groups and associated a load-balancing algorithm to the LAGs.

Note

This procedure assumes that you have not yet associated an application EPG with a VMware vCenter domain. If you have already done so, you edit the domain association.

Procedure

Step 1

Log into Cisco APIC.

Step 2

Go to Tenants > tenant > Application Profiles > application_profile > Application EPGs > EPG > Domains (VMs and Bare-Metals).

Step 3

Right-click Domains (VMs and Bare-Metals) and choose Add VMM Domain Association.

Step 4

In the Add VMM Domain Association dialog box, complete the following steps:

1. From the VMM Domain Profile drop-down list, choose the domain that you want to associate the EPG to.

2. From the Enhanced Lag Policy, choose the policy configured for the domain that you want to apply to the EPG.

3. (Optional) In the Delimiter field, enter one of the following: |, ~, !, @, ^, +, or =.

If you do not enter a symbol, the system default | delimiter will appear in the policy.

4. Add remaining values as desired for the domain association, and then click Submit.

Step 5

Repeat Step 2 through Step 4 for other application EPGs in the tenant as desired.

Remove the Enhanced LACP Configuration Before a Downgrade

Before you downgrade Cisco Application Policy Infrastructure Controller (APIC) to a release earlier than 3.2(7), you must remove the enhanced LACP configuration. Complete the steps in this procedure to remove the configuration.

Procedure

Step 1

Reassign uplinks on all ESXi hosts from link aggregation groups (LAGs) to normal uplinks.

Step 2

Remove LAG associations from all EPGs associated with the distributed virtual switch (DVS).

You can expect traffic loss while performing this step.

Step 3

Change port channel settings to static channel or MAC pinning, which will cause traffic to recover once the port channel is up.

Step 4

Remove all LAG-related configuration from the virtual machine manager (VMM).

Step 5

Verify that all LAG-related policies are deleted from VMware vCenter.

What to do next

Downgrade to a Cisco APIC release earlier than 3.2(7).

Endpoint Retention Configuration

After you create a vCenter domain, you can configure endpoint retention. This feature enables you to delay the deletion of an endpoint, reducing the chances of dropped traffic.

You can configure endpoint retention in the APIC GUI or with the NX-OS style CLI or the REST API. For details, see the subsequent procedures.

- Configuring Endpoint Retention Using the GUI

Configuring Endpoint Retention Using the GUI

Before you begin

You must have created a vCenter domain.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Choose VM Networking > Inventory.

Step 3

In the left navigation pane, expand the VMware folder and then click the vCenter domain that you created earlier.

Step 4

In the central Domain work pane, make sure that the Policy and General tabs are selected.

Step 5

In the End Point Retention Time (seconds) counter, choose the number of seconds to retain endpoints before they are detached.

You can choose between 0 and 600 seconds. The default is 0.

Step 6

Click Submit.

Creating VDS Uplink Port Groups

Each VMM domain appears in the vCenter as a vSphere Distributed Switch (VDS). The virtualization administrator associates hosts to the VDS created by the APIC and selects which vmnics to use for the specific VDS. The configuration of the VDS uplinks are performed from the APIC controller by changing the vSwitch configuration from the Attach Entity Profile (AEP) that is associated with the VMM domain. You can find the AEP in the APIC GUI in the Fabric Access Policies configuration area.

Note

When working with ACI and vSphere VMM integration, Link Aggregation Groups (LAGs) are not a supported method of creating interface teams on distributed switches created by the APIC. The APIC pushes the necessary interface teaming configuration based on the settings in the Interface Policy Group and/or AEP vSwitch policy. It is not supported or required to manually create interface teams in vCenter.

Creating a Trunk Port Group Using the GUI

This section describes how to create a trunk port group using the GUI.

Before you begin

- Trunk port group must be tenant independent.

Procedure

Step 1

Log in to the APIC GUI.

Step 2

On the menu bar, choose Virtual Networking.

Step 3

In the navigation pane, choose VMM Domains > VMware > Domain_name > Trunk Port Groups and right-click Create Trunk Port Group.

Step 4

In the Create Trunk Port Group dialog box, perform the following actions:

1. In the Name field, enter the EPG name.

2. For the Promiscuous Mode buttons, click either Disabled or Enabled. The default is Disabled.

3. For the Trunk Portgroup Immediacy buttons, click either Immediate or On Demand. The default is On Demand.

4. For the MAC changes buttons, click either Disabled or Enabled. The default is Enabled.

5. For the Forged transmits buttons, click either Disabled or Enabled. The default is Enabled.

6. In the VLAN Ranges field, choose the + icon and enter the VLAN range (vlan-100 vlan-200).

Note

If you do not specify a VLAN Range, the VLAN list will be taken from the domain’s VLAN namespace.

7. Click Update.

Step 5

Click Submit.

Guidelines for Cisco UCS B-Series Servers

When integrating blade server systems into Cisco ACI Cisco Application Centric Infrastructure for purposes of VMM integration (for example, integrating Cisco Unified Computing System (UCS) blade servers or other non-Cisco blade servers) you must consider the following guidelines:

Note

This example shows how to configure a port channel access policy for integrating Cisco UCS blade servers. You can use similar steps to set up a virtual port channel or individual link access policies depending upon how your Cisco UCS blade server uplinks are connected to the fabric. If no port channel is explicitly configured on the Cisco Application Policy Infrastructure Controller (APIC) for the UCS blade server uplinks, the default behavior will be mac-pinning.

- The VM endpoint learning relies on either the Cisco Discovery Protocol (CDP) or Link Layer Discovery Protocol (LLDP). If supported, CDP must be enabled all the way from the leaf switch port through any blade switches and to the blade adapters.

- Ensure the management address type, length, and value (TLV) is enabled on the blade switch (CDP or LLDP protocol) and advertised towards servers and fabric switches. Configuration of management TLV address must be consistent across CDP and LLDP protocols on the blade switch.

- The Cisco APIC does not manage fabric interconnects and the blade server, so any UCS specific policies such as CDP or port channel policies must be configured from the UCS Manager.

- VLANs defined in the VLAN pool used by the attachable access entity profile on the Cisco APIC, must also be manually created on the UCS and allowed on the appropriate uplinks connecting to the fabric. This must include the infrastructure VLAN if applicable. For details, see the Cisco UCS Manager GUI Configuration Guide.

- When you are working with the Cisco UCS B-series server, both CDP and LLDP are supported, beginning from UCSM 2.2.4b. If UCS B-series server is using earlier firmware, LLDP is not supported.

- CDP is disabled by default in Cisco UCS Manager. In Cisco UCS Manager, you must enable CDP by creating a Network Control Policy.

- Do not enable fabric failover on the adapters in the UCS server service profiles. Cisco recommends that you allow the hypervisor to handle failover at the virtual switch layer so that load balancing of traffic is appropriately performed.

Note

Symptom: The change of management IP of the unmanaged node such as blade switch or fabric interconnect gets updated in the VMware vCenter, but the VMware vCenter does not send any events to Cisco APIC.

Condition: This causes the Cisco APIC to be out of sync with VMware vCenter.

Workaround: You need to trigger an inventory pull for the VMware vCenter controller that manages ESX servers behind the unmanaged node.

Setting up an Access Policy for a Blade Server Using the GUI

Before you begin

To operate with the Cisco APIC, the Cisco UCS Fabric Interconnect must be at least a version 2.2(1c). All components, such as the BIOS, CIMC, and the adapter must be a version 2.2(1c) or later. For further details, see the Cisco UCS Manager CLI Configuration Guide.

Procedure

Step 1

On the menu bar, choose Fabric > Access Policies.

Step 2

In the navigation pane, click Quick Start.

Step 3

In the central pane, click Configure an interface, PC, and VPC.

Step 4

In the Configure Interface, PC, and VPC dialog box, click the + icon to select switches.

Step 5

In the Switches field, from the drop-down list, choose the desired switch IDs.

Step 6

Click the + icon to configure the switch interfaces.

Step 7

In the Interface Type field, click the VPC radio button.

Step 8

In the Interfaces field, enter the appropriate interface or interface range that is connected to the blade server.

Step 9

In the Interface Selector Name field, enter a name.

Step 10

From the CDP Policy drop-down list, choose default

The default CDP policy is set to disabled. (Between the leaf switch and the blade server, CDP must be disabled.)

Step 11

From the LLDP Policy drop-down list, choose default.

The default LLDP policy is set to enabled for the receive and transmit states. (Between the leaf switch and the blade server, LLDP must be enabled.)

Step 12

From the LACP Policy drop-down list, choose Create LACP Policy.

Between the leaf switch and the blade server, the LACP policy must be set to active.

Step 13

In the Create LACP Policy dialog box, perform the following actions:

1. In the Name field, enter a name for the policy.

2. In the Mode field, the Active radio button is checked.

3. Keep the remaining default values and click Submit.

Step 14

From the Attached Device Type field drop-down list, choose ESX Hosts .

Step 15

In the Domain Name field, enter a name as appropriate.

Step 16

In the VLAN Range field, enter the range.

Step 17

In the vCenter Login Name field, enter the login name.

Step 18

In the Password field, and the Confirm Password field, enter the password.

Step 19

Expand the vCenter field, and in the Create vCenter Controller dialog box, enter the desired content and click OK.

Step 20

In the vSwitch Policy field, perform the following actions:

Between the blade server and the ESX hypervisor, CDP must be enabled, LLDP must be disabled, and LACP must be disabled so Mac Pinning must be set.

1. Check the MAC Pinning check box.

2. Check the CDP check box.

3. Leave the LLDP check box unchecked because LLDP must remain disabled.

Step 21

Click Save, and click Save again. Click Submit.

The access policy is set.

Troubleshooting the Cisco ACI and VMware VMM System Integration

For troubleshooting information, see the following links:

- Cisco APIC Troubleshooting Guide

- ACI Troubleshooting Book

Custom User Account with Minimum VMware vCenter Privileges

Setting VMware vCenter privileges allows the Cisco Application Policy Infrastructure Controller (APIC) to send VMware API commands to VMware vCenter for the creation of the DVS. Setting privileges also allows Cisco APIC to publish port groups and relay all necessary alerts.

To configure the VMware vCenter from Cisco APIC, your credentials must allow the following minimum set of privileges within the VMware vCenter:

- Alarms (read/write/modify)

Cisco APIC creates two alarms in the folder, one for DVS and another for port-group. The alarm is raised when the EPG or Domain policy is deleted on Cisco APIC. However, the alarms cannot be deleted for DVS or port-group because of the virtual machines (VMs) that are attached.

- Distributed Switch (read/write/modify)

- dvPort Group (read/write/modify)

- Folder (read/write/modify)

- Network (read/write/modify)

Cisco APIC manages the network settings such as add or delete port-groups, setting host/DVS MTU, LLDP/CDP, LACP.

- Host (read/write/modify)

- Host.Configuration.Advanced settings

- Host.Local operations.Reconfigure virtual machine

- Host.Configuration.Network configuration

- Virtual machine (read/write/modify)

If you use Service Graph in addition to the already listed privileges, you need the Virtual machine privilege for the virtual appliances that are used for Service Graph.

- Virtual machine.Configuration.Modify device settings

- Virtual machine.Configuration.Settings

If you want to deploy service VMs using the service VM orchestration feature, enable the following privileges in addition to the preceding privileges.

For information about the feature, see the "Service VM Orchestration" chapter of the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide.

- Datastore (read/write/modify)

- Allocate space

- Browse datastore

- Low level file operations

- Remove file

- Host (read/write/modify)

- Local operations.Delete virtual machine

- Local operations.Reconfigure virtual machine

- Resource (read/write/modify)

- Assign virtual machine to resource pool

- Virtual machine (read/write/modify)

- Inventory.Create new

- Inventory.Create from existing

- Inventory.Remove

- Configuration.Add new disk

- Provisioning.Deploy template

- Provisioning.Clone template

- Provisioning.Clone virtual machine

- Provisioning.Customize

- Interaction (all)

- Global (read/write/modify)

- Manage Custom Attributes

- Set Custom Attribute

Quarantine Port Groups

The quarantine port group feature provides a method to clear port group assignments under certain circumstances. In the VMware vCenter, when a VMware vSphere Distributed Switch (VDS) is created, a quarantine port group is created in the VDS by default. The quarantine port group default policy is to block all ports.

As part of integration with Layer 4 to Layer 7 virtual service appliances, such as a load balancer or firewall, the Application Policy Infrastructure Controller (APIC) creates service port groups in vCenter for service stitching and orchestrates placement of virtual appliances, such as service virtual machines (VMs), in these service port groups as part of the service graph rendering mechanism. When the service graph is deleted, the service VMs are automatically moved to the quarantine port group. This auto-move to a quarantine port group on delete is only done for service VMs, which are orchestrated by the APIC.

You can take further action with the port in quarantine port group as desired. For example, you can migrate all of the ports from the quarantine port group to another port group, such as a VM network.

The quarantine port group mechanism is not applicable to regular tenant endpoint groups (EPGs) and their associated port groups and tenant VMs. Therefore, if the tenant EPG is deleted, any tenant VMs present in the associated port group remains intact and they will not be moved to the quarantine port group. The placement of tenant VMs into the tenant port group is outside the realm of the APIC.

On-Demand VMM Inventory Refresh

Triggered Inventory provides a manual trigger option to pull and refresh Cisco Application Policy Infrastructure Controller (APIC) inventory from the virtual machine manager (VMM) controller. It is not required in normal scenarios. Use it with discretion only when errors occur.

When there is a process restart, leadership change, or background periodic 24-hour inventory audit, Cisco APIC pulls inventory to keep VMM inventory aligned with the VMM controller inventory. At certain times, VMware vCenter APIs can error out, and Cisco APIC may not have fully downloaded the inventory from the VMware vCenter despite retries. Cisco APIC indicates this condition with a user-visible fault. In this case, triggered inventory allows you to start an inventory pull from the Cisco APIC VMM to the VMware vCenter.

Cisco APIC does not maintain any synchronization between the VMM configuration and the VMware vCenter VDS configuration. If you directly change VDS settings from the VMware vCenter, Cisco APIC does not try to overwrite the user settings (except for PVLAN configuration).

Physically Migrating the ESXi Host

Complete the tasks in this procedure to physically migrate ESXi hosts.

Procedure

Step 1

Put the host into maintenance mode or evacuate the virtual machine (VM) workload by another method.

Step 2

Remove the ESXi host from the VMware VDS, Cisco Application Centric Infrastructure (ACI) Virtual Edge, or Cisco Application Virtual Switch.

Step 3

Physically recable the ESXi host to the new leaf switch or pair of leaf switches

Step 4

Add the ESXi host back to the VMware VDS, Cisco Application Centric Infrastructure (ACI) Virtual Edge, or Cisco Application Virtual Switch.

Guidelines for Migrating a vCenter Hypervisor VMK0 to an ACI Inband VLAN

Follow the guidelines below to migrate the default vCenter hypervisor VMK0 out of bound connectivity to ACI inband ports. An ACI fabric infrastructure administrator configures the APIC with the necessary policies, then the vCenter administrator migrates the VMK0 to the appropriate ACI port group.

- Create the Necessary Management EPG Policies in APIC

- Migrate the VMK0 to the Inband ACI VLAN

Create the Necessary Management EPG Policies in APIC

As an ACI fabric infrastructure administrator, use the following guidelines when creating the management tenant and VMM domain policies:

- Choose a VLAN to use for ESX management.

- Add the VLAN chosen for ESX management to a range (or Encap Block) in the VLAN pool associated with the target VMM domain. The range where this VLAN is added must have allocation mode set to static allocation.

- Create a management EPG in the ACI management tenant (mgmt).

- Verify that the bridge domain associated with the management EPG is also associated with the private network (inb).

- Associate the management EPG with the target VMM domain as follows:

- Use resolution immediacy as pre-provision.

- Specify the management VLAN in the Port Encap field of the VM domain profile association.

As a result, APIC creates the port group under vCenter with VLAN specified by the user. APIC also automatically pushes the policies on the leaf switches associated with the VMM domain and Attach Entity Profile (AEP).

Migrate the VMK0 to the Inband ACI VLAN

By default vCenter configures the default VMK0 on the hypervisor management interface. The ACI polices created above enable the vCenter administrator to migrate the default VMK0 to the port group that is created by APIC. Doing so frees up the hypervisor management port.

Chapter 4. Microsegmentation with Cisco ACI

This chapter contains the following sections:

- Microsegmentation with Cisco ACI

Microsegmentation with Cisco ACI

Microsegmentation with the Cisco Application Centric Infrastructure (ACI) enables you to automatically assign endpoints to logical security zones called endpoint groups (EPGs). These EPGs are based on various network-based or virtual machine (VM)-based attributes.

This chapter contains conceptual information about Microsegmentation with Cisco ACI and instructions for configuring microsegment (uSeg) EPGs. We assume that you are familiar with EPGs, tenants, contracts, and other key concepts relating to Cisco ACI policies. For more information, see Cisco Application Centric Infrastructure Fundamentals.

Supported Endpoints

The Cisco Application Policy Infrastructure Controller (APIC) manages microsegmentation policies, and the Cisco ACI fabric enforces the policies. Microsegmentation with Cisco ACI supports virtual endpoints that are attached to the following:

- Cisco ACI Virtual Edge

Note

Microsegmentation with Cisco ACI is supported for Cisco ACI Virtual Edge endpoints only when Cisco ACI Virtual Edge is not part of Cisco ACI Virtual Pod (vPod).

- Cisco Application Virtual Switch (AVS)

- Microsoft Hyper-V Virtual Switch

- VMware vSphere Distributed Switch (VDS)

Microsegmentation with network-based attributes also supports bare-metal environments. See the section "Using Microsegmentation with Network-based Attributes on Bare Metal" in the Cisco APIC Basic Configuration Guide, Release 3.x .

Microsegmentation with Cisco ACI also supports physical endpoints using EPGs with IP-based attributes.

Note

You can configure Microsegmentation with Cisco ACI for physical and virtual endpoints, and you can share the same EPGs for both physical and virtual endpoints.

Layer 4 to Layer 7 service graphs are supported for contracts between microsegmented EPGs and between microsegmented EPGs and regular EPGs. See the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide on Cisco.com for detailed information and configuration procedures.

Limitations

- If you use Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch, note the following: If you want to use a MAC-based EPG and any attribute other than IP for virtual endpoints, do not configure any overlapping IP attribute filters for physical endpoints or virtual endpoints on a VDS VMM domain. If you do so, the Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch, microsegmentation EPG classification is overwritten.

- Configuring direct server return (DSR) with a L4-L7 virtual IP (VIP) address under a microsegmented EPGs is not supported.

- Benefits of Microsegmentation with Cisco ACI

- How Microsegmentation Using Cisco ACI Works

- Attributes for Microsegmentation with Cisco ACI

- Methods of Filtering VMs for uSeg EPGs

- Precedence of Operators

- Scenarios for Using Microsegmentation with Cisco ACI

- Configuring Microsegmentation with Cisco ACI

Benefits of Microsegmentation with Cisco ACI

Endpoint groups (EPGs) are used to group virtual machines (VMs) within a tenant and apply filtering and forwarding policies to them. Microsegmentation with Cisco ACI adds the ability to group endpoints in existing application EPGs into new microsegment (uSeg) EPGs and configure network or VM-based attributes for those uSeg EPGs. This enables you to filter with those attributes and apply more dynamic policies. Microsegmentation with Cisco ACI also allows you to apply policies to any endpoints within the tenant.

Example: Microsegmentation with Cisco ACI Within a Single EPG or Multiple EPGs in the Same Tenant

You might assign web servers to an EPG so that you can apply the similar policies. By default, all endpoints within an EPG can freely communicate with each other. However, if this web EPG contains a mix of production and development web servers, you might not want to allow communication between these different types of web servers. Microsegmentation with Cisco ACI allows you to create a new EPG and autoassign endpoints based on their VM name attribute, such as "Prod-xxxx" or "Dev-xxx".

Example: Microsegmentation for Endpoint Quarantine

You might have separate EPGs for web servers and database servers, and each one contains both Windows and Linux VMs. If a virus affecting only Windows threatens your network, you can isolate Windows VMs across all EPGs by creating a new EPG called, for example, "Windows-Quarantine" and applying the VM-based operating systems attribute to filter out all Windows-based endpoints. This quarantined EPG could have more restrictive communication policies, such as limiting allowed protocols or preventing communication to any other EPGs by not having any contract. A microsegment EPG can have a contract or not have a contract.

How Microsegmentation Using Cisco ACI Works

Microsegmentation using Cisco ACI involves the Cisco APIC, vCenter or Microsoft System Center Virtual Machine Manager (SCVMM), and leaf switches. This section describes the workflow for microsegmentation using Cisco ACI Virtual Edge, Cisco AVS, VMware VDS, or Microsoft Hyper-V Virtual Switch.

Cisco APIC

1. The user configures a VMM domain for Cisco ACI Virtual Edge, Cisco AVS, VMware VDS, or Microsoft Hyper-V Virtual Switch in the Cisco APIC.

2. The Cisco APIC connects to vCenter or SCVMM and does the following:

1. Creates an instance of Cisco ACI Virtual Edge, Cisco AVS, VMware VDS, or Microsoft Hyper-V Virtual Switch.

2. Pulls VM and hypervisor inventory information from the associated VMware vCenter or Microsoft SCVMM.

3. The user creates an application EPG and associates it with a vCenter/SCVMM domain. In each vCenter/SCVMM domain, a new encapsulation is allocated for this application EPG. The application EPG does not have any attributes.

The vCenter/SCVMM administrator assigns virtual endpoints to this application EPG—not to any microsegment (uSeg) EPGs. It is the application EPG that appears in vCenter/SCVMM as a port group.

4. The user creates an uSeg EPG and associates it with the VMM domain.

The uSeg EPG does not appear in vCenter/SCVMM as a port group; it has a special function: The uSeg EPG has VM-based attributes to match filter criteria. If a match occurs between the uSeg EPG VM attributes and VMs, the Cisco APIC dynamically assigns the VMs to the uSeg EPG.

The endpoints are transferred from the application EPG to the uSeg EPG. If the uSeg EPG is deleted, the endpoints are assigned back to the application EPG.

The uSeg EPG must be assigned to a VMM domain in order for it to take effect. When you associate an uSeg EPG to a VMM domain, its criteria is applied for that VMM domain only. If you have VMware VDS, you must also assign the uSeg EPG to the same bridge domain as the application EPG.

In the case of VMware VDS, its criteria is applied for that VMM domain and bridge domain.

Leaf Switch

1. The physical leaf switch pulls the attribute policies from the Cisco APIC.

2. The Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch sends a VM attach message to the physical leaf switch using the OpFlex protocol when a VM attaches to Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch.

VMware vSphere Distributed Switch (VDS) does not send a VM attach message using the OpFlex protocol.

3. The physical leaf switch matches the VM against the configured attribute policies for the tenant.

4. If the VM matches the configured VM attributes, the physical leaf switch pushes the uSeg EPG—along with the corresponding encapsulation—to the Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch.

Note that this action does not change the original port-group assignment for the VM in vCenter/SCVMM.

For VMware VDS, the physical leaf switch does not push the microssegmented EGP. The leaf switch performs the attribute-based microsegmentation.

Packet Forwarding for Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch

1. When the VM sends the data packets, Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch tags the packets using encapsulation corresponding to the uSeg EPG, not the application EPG.

2. The physical leaf hardware sees an attribute-based encapsulated VM packet and matches it with the configured policy.

The VM is dynamically assigned to an uSeg EPG, and the packet is forwarded based on the policy defined for that particular uSeg EPG.

Packet Forwarding for VMware VDS

When you enable Microsegmentation with Cisco ACI, Cisco APIC allocates a pair of VLANs (PVLANs) and configures a PVLAN portgroup on VMware vCenter. Doing so forces traffic to go to the leaf switch even if two VMs in the same portgroup try to talk to each other.

You must configure a PVLAN on blade switches of ESXi servers are not directly connected to leaf switches.

Note

If you configure Microsegmentation with Cisco ACI for EPGs associated with VMware VDS VMM domains, you may experience brief traffic disruption.

Attributes for Microsegmentation with Cisco ACI

Applying attributes to uSeg EPGs enables you to apply forwarding and security policies with greater granularity than you can apply policies to EPGs without attributes. Attributes are unique within the tenant.

There are two types of attributes that you can apply to uSeg EPGs: network-based attributes and VM-based attributes.

Network-Based Attributes

The network-based attributes are IP (IP address filter) and MAC (MAC Address Filter). You can apply one or more MAC or IP addresses to a uSeg EPG.

For IP addresses, you simply specify the address or the subnet; for MAC addresses, you simply specify the address.

Note

If you want to use a network-based attribute and classify IP addresses in the same subnet, you must use the MAC-based network attribute. IP-based microsegmented EPGs do not support classification for IP addresses in the same subnet. IP-based microsegmented EPGs are supported only when traffic requires Layer 3 routing. If the traffic is bridged, the microsegmentation policy cannot be enforced.

VM-Based Attributes

You can apply multiple VM-based attributes to VMware VDS, Cisco AVS, or Cisco ACI Virtual Edge uSeg EPGs. The VM-based attributes are VMM Domain, Operating System, Hypervisor Identifier, Datacenter, VM Identifier, VM Name, VNic Dn (vNIC domain name), Custom Attribute, and Tag.

Note

The attribute Datacenter corresponds to Cloud for Microsoft Hyper-V Virtual Switch.

Note

The attribute VM Folder also appears in the GUI. This feature is for beta testing only, and should not be deployed in production environments.

When you create any VM-based attribute, in addition to naming the attribute, you must do the following:

1. Specify the attribute type, such as VM Name or Hypervisor Identifier.

2. Specify the operator, such as Equals, or Starts With.

3. Specify the value, such as a particular vNIC or name of the operating system.

Custom Attribute and Tag Attribute

The Custom Attribute and the Tag attribute allow you to define attributes based on criteria that are not used in other attributes. For example, you might want to define a Custom Attribute called "Security Zone" in VMware vCenter and then associate this attribute to one or more VMs with such values as "DMZ" or "Edge." The APIC administrator can then create an uSeg EPG based on that VM Custom Attribute.

The Custom Attribute and the Tag attribute appear in the APIC GUI as VM attributes:

- Custom Attribute

- Available for Cisco ACI Virtual Edge, Cisco AVS, and VMware VDS as a VM attribute configured in VMware vCenter

- Available for Microsoft Hyper-V Virtual Switch as a Custom Property configured in Microsoft SCVMM

- Tag Attribute—Available for Cisco ACI Virtual Edge, Cisco AVS, and VMware VDS only

If you want to use a Custom Attribute or a Tag attribute for Cisco ACI Virtual Edge, Cisco AVS, or VMware VDS, must also add it in VMware vSphere Web Client. If you want to use a Custom Attribute for Microsoft Hyper-V Virtual Switch, you must also add it as a Custom Property in Microsoft SCVMM. We recommend doing so before configuring the uSeg EPG. That allows you to choose the Custom Attribute or Tag attribute in the drop-down list while configuring microsegmentation policy in Cisco APIC.

You can add the Custom Attribute or Tag attribute in vSphere Web Client or SCVMM after you configure the uSeg EPG in Cisco APIC. However, if you do so, you do not see the Custom Attribute or Tag attribute in the drop-down list in Cisco APIC, although you can type the name of the Custom Attribute or Tag attribute in the text box.

See VMware vSphere ESXi and VMware vCenter Server documentation for instructions for adding a Custom Attribute or Tag attribute in vSphere Web Client. See Microsoft documentation for instructions for adding a Custom Attribute in SCVMM.

Although similar to the Custom Attribute, the Tag attribute differs from in it several ways:

- The Tag attribute can be applied to any object in VMware vCenter, such as a host or data center; the Custom Attribute can be applied only to VMs and ESXi hosts. However, only the Tag attribute for VMs is relevant to microsegmentation.

- The Tag attribute does not have a name and value like the Custom Attribute. Tags are simply labels that get applied or not to objects.

- To configure a Custom Attribute, you provide details about the controller and VM as well as an operator and a value. To configure the Tag attribute, you provide the attribute type, category, operator, and tag name.

Note

- The Tag attribute can be defined for a microsegmented EPG only when the VMware vCenter is running vSphere 6 .0 or later.

- To enable Microsegmentation with Cisco ACI using the Tag attribute, enable the VMware vCenter tag collection on Cisco APIC. You do this with a REST API call for each VMM domain, as shown in the following example:

https://APIC-IPA/api/node/mo.xml
Body:
<polUni>
<vmmProvP vendor="VMware">
<vmmDomP name="Domain-Name" enableTag="yes">
</vmmDomP>
</vmmProvP>
</polUni>

Ensure that the domain name is correct.

Uniqueness of Attributes Within a Tenant

Attributes must be unique within a tenant. Uniqueness depends on the value of the attribute.

For example, for a network-based attribute, you can use the attribute IP Address Filter multiple times within a tenant. You can do so provided that the attribute has a different value for the IP address each time it is used. So you cannot use the IP Address Filter attribute with the address 192.168.33.77 more than once; however, you can use the IP Address Filter attribute a second time, provided that the IP address is different, for example 192.168.33.78.

Methods of Filtering VMs for uSeg EPGs

You can configure uSeg EPGs with multiple attributes. However, VMs can become part of only one uSeg EPG. When a VM has attributes matching more than one uSeg EPG in the tenant, Cisco APIC puts the VM into a uSeg EPG based on filtering rules.

Depending on how you define the attributes, you can use different filtering rules:

- Matching any attribute—You can match any attribute, and Cisco APIC follows a the default precedence among attributes in deciding which uSeg EPG that a VM will join.

For more information, see the section VM Filtering when Matching Any Attributein this guide.

- Matching all attributes—You can match all of the VM-based attributes defined for the uSeg EPG. You cannot match all for multiple network-based attributes.

For more information, see the section VM Filtering when Matching All Attributes in this guide.

- Using simple or block statements—You can create multiple statements to filter for multiple attributes, or you can create block, or nested, statements to create precise filtering rules.

For more information, see the section VM Filtering when Using Simple or Block Statements in this guide.

- Overriding existing rules—When you create a uSeg EPG, you can set its precedence, overriding other rules. You can set the precedence when you match any attribute or match all attributes. You need to set match precedence to break ties across EPGs in the tenant. You can match all attributes and not set match precedence; however, in such cases, if you have multiple uSeg EPGs with similar attributes, the VM can get matched to any of the uSeg EPGs arbitrarily.

For more information, see the section VM Filtering when Using EPG Match Precedence in this guide.

- VM Filtering when Matching Any Attribute

- VM Filtering when Matching All Attributes

- VM Filtering when Using Simple or Block Statements

- VM Filtering when Using EPG Match Precedence

VM Filtering when Matching Any Attribute

Matching any attribute defined for a uSeg EPG is the default.

If you have multiple attributes and match any, Cisco APIC filters for VMs matching any of the attributes and—if VMs match other EPGs in the tenant—puts them into uSeg EPG based on the precedence of attributes.

How Rules for Attribute Precedence are Applied

The following table lists the attributes that can be specified for an uSeg EPG:

Attribute

Type

Precedence Order

Example

MAC

Network

1- Cisco ACI Virtual Edge/Cisco AVS/Microsoft Hyper-V Virtual Switch

2- VMware VDS

5c:01:23:ab:cd:ef

IP

Network

1- VMware VDS

2- Cisco ACI Virtual Edge/Cisco AVS/Microsoft Hyper-V Virtual Switch

192.168.33.77

10.1.0.0/16

VNic Dn (vNIC domain name)

VM

3

a1:23:45:67:89:0b

VM Identifier

VM

4

VM-598

VM Name

VM

5

HR_VDI_VM1

Hypervisor Identifier

VM

6

host-25

VMM Domain

VM

7

AVS-SJC-DC1

Datacenter

VM

8

SJC-DC1

Custom Attribute

VM

9

SG_DMZ

Operating System

VM

10

Windows 2008

Tag

(Cisco ACI Virtual Edge, Cisco AVS, and VMware VDS only)

VM

11

Linux

VM Folder

(Cisco ACI Virtual Edge, Cisco AVS, and VMware VDS only)

Note

The VM Folder attribute is for beta testing only, and should not be deployed in production environments. Contact Cisco for information about this feature.

VM

12

VM_Folder_1

Note

Precedence of MAC-based and IP-based attributes differ for VMware VDS and Cisco ACI Virtual Edge, Cisco AVS, and Microsoft Hyper-V Virtual Switch.

Examples of how Rules for Precedence are Applied

You might have four uSeg EPGs containing attributes that match the same VM, and each uSeg EPG has a different network or VM attribute: Operating System, Hypervisor Identifier, IP; and another has MAC.

Rules for Cisco AVS and Microsoft Hyper-V Virtual Switch are applied in this order: MAC, IP, Hypervisor Identifier, and Operating System. The rule is applied to MAC, and the subsequent rules are skipped. However, if the uSeg EPG with the MAC attribute is deleted, the rule is applied to IP Address Filter, and the subsequent rules are skipped—and so on with the other attributes.

Rules for VMware VDS are applied in this order: IP Address Filter, MAC Address Filter, Hypervisor Identifier, and Operating System. The rule is applied to IP, and the subsequent rules are skipped. However, if the uSeg EPG with the IP attribute is deleted, the rule is applied to MAC and the subsequent rules are skipped—and so on with the other attributes.

In another case, you might have uSeg EPGs containing the same VM, and each uSeg EPG has a different VM attribute: VMM Domain, Datacenter, Custom Attribute, and VNic Dn. The rule is applied to VNic Dn, and the subsequent rules as skipped. However, if the uSeg EPG with the VNic Dn attribute is deleted, the rule is applied to VMM Domain, and the subsequent rules are skipped–and so on with the other attributes.

VM Filtering when Matching All Attributes

You can filter by matching all VM-based attributes defined for a uSeg EPG. You do so by choosing Match All from a drop-down list in the APIC GUI or specify matching in the NX-OS CLI or REST API.

If you match all attributes, Cisco APIC does not put any VM into the uSeg EPG unless it matches all the attributes defined for the uSeg EPG.

For example, you have a uSeg EPG with the following attributes: Hypervisor Identifier where the hypervisor is host-25, VM Name that contains "vm," and Operating System of Linux. Cisco APIC puts into the uSeg EPG only those VMs that have the hypervisor host-25, a VM Name containing "vm," and have the operating system Linux. It would not put into the uSeg EPG a VM that matches the first two attributes but has the operating system Microsoft.

Note

Matching all attributes is supported for VM-based attributes only. You cannot choose Match All for network-based attributes.

If you want to match all VM-based attributes, you might want to set the EPG match precedence when you create the uSeg EPG. Doing so allows you to decide which uSeg EPG should override other uSeg EPGs. However, you can set EPG match precedence whether you match any attribute or all attributes. For more information, see the section VM Filtering when Using EPG Match Precedence in this guide.

Note

If you use Microsoft Hyper-V Virtual Switch and want to downgrade to APIC Release 2.3(1) from a later release, you first need to delete any uSegs configured with the Match All filter. The Match All filter is supported for Microsoft beginning with APIC Release 3.0(1).

VM Filtering when Using Simple or Block Statements

When you define attributes for a uSeg EPG, you can define multiple attributes in simple statements or in block statements. You can combine simple and block statements to create complex filters for attributes.

Simple statements contain a single attribute. You can have as many simple statements as you want for each uSeg EPG. You can match any of the attributes or all of the attributes.

Block statements contain multiple attributes at different levels in a hierarchy. You can have only two sublevels within a block statement. You can match any of the attributes or all of the attributes for each level of the block statement.

Note

You cannot put network-based attributes into sublevels of block statements. However, you can create sublevels for network-based attributes if the network-based attribute is at the top level of a block statement.

When you have block statements, Cisco APIC first filters for attributes defined on the top level. It then filters on the next-highest level, and then the next-highest level.

You can create simple and block statements in the APIC GUI, the NX-OS CLI, and the REST API.

Example of Using Block Statements

You want to put some VMs into a uSeg EPG so you can update Linux on them. The VMs are within a single data center, but you want to limit the update to VMs within two VMM domains. You can use block statements to set up filtering for those VMs.

Because you are filtering for VMs that run Linux and are in a single data center, you create two simple statements: one for the Operating System attribute with the value Linux and one for the attribute Datacenter with the value of datacenter3. For these statements you choose Match All because you want to capture all VMs in the tenant that run Linux and belong to datacenter 3.

However, among VMs that run Linux and belong to datacenter3, you now want to capture VMs that belong only to the VMM domains mininet2 or mininet4. You create a block statement as a sublevel of the two simple statements. The blocks statement contains two attributes: one for the attribute VMM domain with the value of mininet 2 and one for the attribute VMM domain with the value of mininet 4. You choose match any for the block statement because you want to capture VMs that are in either VMM domain.

Once you define the attributes, Cisco APIC first filters for VMs that run Linux and also are in datacenter3. It then searches among those VMs for the ones that belong to either mininet2 or mininet4.

VM Filtering when Using EPG Match Precedence

EPG Match Precedence enables you to override default precedence rules for uSeg EPGs when filtering for VM-based attributes. You configure it when you create the uSeg EPG in the GUI, NX-OS CLI, or REST API.

EPG Match Precedence is optional when matching any attribute or matching all attributes. However, when you match all attributes—filtering on multiple attributes—setting precedence enables Cisco APIC to break ties between uSeg EPGs.

Note

You cannot use EPG Match Precedence when filtering network-based attributes. If you try to do so, you see an error message.

When you configure EPG Match Precedence, you give the uSeg EPG an integer value; the higher the number the higher the precedence. You can have nearly 4.3 billion (2^32) levels of precedence. The default is 0, which does not set any precedence.

For example, you might have two uSeg EPGs, each with only one attribute. One has the attribute VM Name, and the other has Operating System. A VM might match both uSeg EPGs. By default, Cisco APIC would assign the VM to the uSeg EPG with the VM Name attribute because that attribute has higher precedence than the attribute Operating System.

However, if you give the uSeg EPG with the attribute Operating System a precedence of 10 and give the uSeg EPG with the attribute VM Name a precedence of 7, Cisco APIC will give the VM matching both uSeg EPGs to the uSeg EPG with the Operating System attribute.

Precedence of Operators

In addition to applying filtering rules based on attributes of uSeg EPGs within a tenant, Cisco APIC applies filtering rules within VM-based attributes based on the operator type.

When you configure a microsegment with a VM-based attribute, you select one of four operators: Contains, Ends With, Equals, or Starts With. Each operators specifies the string or value match for the specific attribute.

For example, you might want to create a microsegment with the VM Name attribute and want to filter for VMs with names that start with "HR_VM" or VMs that contain "HR" anywhere in their name. Or you might want to configure a microsegment for a specific VM and filter for the name "HR_VM_01."

How Rules for Operator Precedence are Applied

The operators for a specific VM attribute within a tenant determine the order in which the VM-based attributes for microsegments are applied. They also determine which operator will have precedence among a group of microsegments that share the same attribute and overlapping values. The table below shows the default operator precedence for Cisco ACI Virtual Edge, Cisco AVS, and Microsoft Hyper-V Virtual Switch:

Operator Type

Precedence Order

Equals

1

Contains

2

Starts With

3

Ends With

4

Examples of how Rules for Precedence are Applied

You have three Human Resources VM machines in a datacenter cluster under the same tenant: VM_01_HR_DEV, VM_01_HR_TEST, and VM_01_HR_PROD. You have created two microsegmented EPGs based on the VM Name attribute:

Criterion

Microsegment CONTAIN-HR

Microsegment HR-VM-01-PROD

Attribute type

VM Name

VM Name

Operator type

Contains

Equals

Value

VM_01_HR

VM_01_HR_PROD

Because the operator type Equals has precedence over the operator type Contains, the value VM_01_HR_PROD is matched before the value VM_01_HR. So the VM named VM_01_HR_PROD will be put into microsegment HR-VM-01-PROD because it is an exact criterion match and because the operator Equals has precedence over the operator Contains, even though the VM name matches both microsegments. The other two VMs will be put in the Microsegment CONTAIN-HR.

Scenarios for Using Microsegmentation with Cisco ACI

This section contains examples of circumstances in which you might find Microsegmentation useful in your network.

- Using Microsegmentation with Cisco ACI with VMs Within a Single Application EPG

- Using Microsegmentation with Cisco ACI with VMs in Different Application EPGs

- Using Microsegmentation with Network-based Attributes

Using Microsegmentation with Cisco ACI with VMs Within a Single Application EPG

You can use Microsegmentation with Cisco ACI to create new, uSeg EPGs to contain VMs from a single application EPG. By default, VMs within an application EPG can communicate with each other; however, you might want to prevent communication between groups of VMs, if VRF is in enforced mode and there is no contract between uSeg EPGs.

For more information about Intra-EPG Isolation knob, that controls communication between VMs within the EPG, see Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch.

Example: Putting VMs from the Same Application EPG into a Microsegmented EPG

Your company deploys a virtual desktop infrastructure (VDI) for its Human Resources, Finance, and Operations departments. The VDI virtual desktop VMs are part of a single application EPG called EPG_VDI with identical access requirements to the rest of the application EPGs.

Service contracts are built in such a way such that the EPG-VDI has access to Internet resources and internal resources. But at the same time, the company must ensure that each of the VM groups—Human Resources, Finance, and Operations—cannot access the others even though they belong to the same application EPG, EPG_VDI.

To meet this requirement, you can create filters in the Cisco APIC that would check the names of the VMs in the application EPG, EPG_VDI. If you create a filter with the value "HR_VM," Cisco APIC creates a uSeg EPG—a microsegment—for all Human Resource VMs. Cisco APIC looks for matching values in all the EPGs in a tenant even though you want to group the matching VMs within one EPG. So when you create VMs, we recommend that you choose names unique within the tenant.

Similarly, you can create filters with the keyword "FIN_VMs" for Finance virtual desktops and "OPS_VMs" for Operations virtual desktops. These uSeg EPGs are represented as new EPGs within the Cisco APIC policy model. You can then apply contracts and filters to control access between the VM groups even though they belong to the same application EPG.

Microsegmentation with Cisco ACI with VMs from a Single Application EPG

In the illustration above, all the virtual desktop VMs from the Human Resources, Finance, and Operations groups have been moved from the application EPG, EPG_VDI, to new, uSeg EPGs: EPG_OPS_MS, EP_FIN_MS, and EPG_HR_MS. Each uSeg EPG has the attribute type VM Name with a value to match key parts of the VM's name. EPG_OPS_MS has the value OPS_VM, so all VMs in the tenant containing OPS_VM in their names become part of EPG_OPS_MS. The other uSeg EPGs have corresponding values, resulting in the movement of VMs in the tenant with matching names to the uSeg EPGs.

Using Microsegmentation with Cisco ACI with VMs in Different Application EPGs

You can configure Microsegmentation with Cisco ACI to put VMs that belong to different application EPGs into a new uSeg EPG. You might want to do this to apply policy to VMs that share a certain characteristic although they belong to different application EPGs.

Example: Putting VMs in Different Application EPGs into a New uSeg EPG

Your company deploys a three-tier web application. The application is built on VMs that run different operating systems and different versions of the same operating system. For example, the VMs might run Linux, Windows 2008, and Windows 2008 R2. The application is distributed; the company has divided the VMs into three different EPGs: EPG_Web, EPG_App, and EPG_DB.

Because of a recent vulnerability in the Windows 2008 operating system, your company's security team decided to quarantine VMs running Windows 2008 in case those VMs are compromised. The security team also decided to upgrade all Windows 2008 VMs to Windows 2012. It also wants to microsegment all production VMs across all EPGs and restrict external connectivity to those VMs.

To meet this requirement, you can configure a uSeg EPG in the Cisco APIC. The attribute would be Operating System, and the value of the attribute would be Windows 2008.

You can now quarantine the VMs running Windows 2008 and upgrade them to Windows 2012. Once the upgrade is complete, the VMs will no longer be part of the uSeg EPG you created for VMs running Windows 2008. This change will be reflected dynamically to Cisco APIC, and those virtual machines revert to their original EPGs.

Microsegmentation with Cisco ACI in Different Application EPGs

In the illustration above, the new uSeg EPG EPG_Windows has the attribute type Operating System and the value Windows. The VMs App_VM_2, DB_VM_1, DB_VM_2, and Web_VM_2, run Windows as their operating system—and so have been moved to the new uSeg EPG EPG_Windows. However, the VMs App_VM_1, DB_VM_3, and Web_VM_1 run Linux and so remain in their application EPGs.

Using Microsegmentation with Network-based Attributes

You can use Cisco APIC to configure Microsegmentation with Cisco ACI to create a new, uSeg EPG using a network-based attribute, a MAC address or one or more IP addresses. You can configure Microsegmentation with Cisco ACI using network-based attributes to isolate VMs within a single application EPG or VMs in different EPGs.

Using an IP-based Attribute

You can use an IP-based filter to isolate a single IP address, a subnet, or multiple of noncontiguous IP addresses. Isolating multiple IP addresses in a single microsegment can be more convenient that specifying VMs by name. You might want to isolate VMs based on IP addresses as a quick and simply way to create a security zone, similar to using a firewall.

Using a MAC-based Attribute

You can use a MAC-based filter to isolate a single MAC address or multiple MAC addresses. You might want to do this if you have a server sending bad traffic in he network; by creating a microsegment with a MAC-based filter, you can isolate the server.

Configuring Microsegmentation with Cisco ACI

The following sections contain instructions for configuring Microsegmentation with Cisco ACI Virtual Edge, Cisco AVS, VMware VDS or Microsoft Hyper-V Virtual Switch using the Cisco APIC GUI and NX-OS style CLI. You can adapt the procedures for your network's specific needs.

Note

If VXLAN load balancing is enabled in the VMware vCenter domain profile, Microsegmentation with Cisco ACI is not supported on the domain.

- Prerequisites for Configuring Microsegmentation with Cisco ACI

- Workflow for Configuring Microsegmentation with Cisco ACI

- Configuring Microsegmentation with Cisco ACI Using the GUI

Prerequisites for Configuring Microsegmentation with Cisco ACI

Before you can configure Microsegmentation with Cisco ACI for Cisco ACI Virtual Edge, Cisco AVS, VMware VDS or Microsoft Hyper-V Virtual Switch, you need to fulfill the following prerequisites.

- Ensure you meet the microsegmentation hardware requirements:

Microsegmentation hardware support

Cisco Nexus 9332PQ, 9372PX, 9372TX, 9396PX, 9396TX, 93120TX, and 93128TX Switches

Cisco Nexus 9372PX-E and 9372TX-E Switches

Cisco Nexus 93108TC-EX, 93180YC-EX, and 93180LC-EX Switches

Cisco Nexus 93180YC-FX, 93108TC-FX, and 9348GC-FXP Switches

Cisco ACI Virtual Edge uSeg (VM, IP, MAC)

Yes

Yes

Yes

Yes

Microsoft uSeg (VM, IP, MAC)

Yes

Yes

Yes

Yes

VDS uSeg (VM, IP, MAC)

No

No

Yes

Yes

Bare-Metal (IP-EPG)

No

Yes

Yes

Yes

Bare-Metal (MAC-EPG)

N/A

Yes

Yes

Yes

Note

Cisco AVS microsegments work with all hardware.

- You must already have VMs with names that can be used with the filters that you will use when creating the uSeg EPGs.

If you do not have VMs with names that can be used, you can go ahead and create the uSeg EPGs and then change the VM names that can be used with the filters. Cisco APIC will automatically make the VMs part of the new uSeg EPGs.

- You must already have an application EPG.

- The corresponding bridge domain must have an IP subnet defined. Otherwise, the VMs will not be able to communicate.

- You must have chosen your own attributes, names, and values.

Attributes, names, and values used in the preceding scenarios were provided as examples.

- You must create a contract before creating a microsegment with one or more attributes if you want to associate the EPG with a contract.

- If you have Cisco ACI Virtual Edge, Cisco AVS, or VMware VDS and want to use a VM Custom Attribute, you also need to add it in VMware vSphere Web Client. If you have Microsoft Hyper-V Virtual Switch and want to use a VM Custom Attribute, you also need to add it in Microsoft SCVMM.

We recommend adding the Custom Attribute in VMware vSphere Web Client or in Microsoft SCVMM before configuring Microsegmentation in Cisco APIC. This enables you to choose the Custom Attribute in the drop-down list while configuring the microsegment in he Cisco APIC GUI.

See VMware vSphere ESXi and vCenter Server documentation for instructions for adding a Custom Attribute in vSphere Web Client. See Microsoft documentation for instructions for adding a Custom Attribute in SCVMM.

- For Microsoft Hyper-V Virtual Switch based microsegmentation, one of the following is required:

- SCVMM 2012 R2 Build 3.2.8145.0 or newer

- SCVMM 2016 Build 4.0.1662.0 or newer

These builds include a feature called "Enable Dynamic VLAN on the vNIC of a virtual machine," which will be automatically enabled by the Cisco SCVMM Agent to allow live migration of virtual machines that use Microsegmentation with ACI. For more information, see Microsoft's documentation: https://support.microsoft.com.

- If you have VMware VDS or a bare-metal server, make sure to set the VRF policy-enforcement direction to "ingress." Otherwise, there will be a fault.

- If you have VMware VDS, make sure the PVLANs are set up on the blade switch. Also make sure that static VLANs are deployed so that VLAN usage is consistent.

Workflow for Configuring Microsegmentation with Cisco ACI

This section provides a high-level description of the tasks that you need to perform in order to configure Microsegmentation with Cisco ACI.

1

Create the uSeg EPG: Specify a name and bridge domain for the new uSeg EPG and choose a network-based or VM-based attribute for the EPG.

Note

For VMware VDS, you need to choose the same bridge domain for the new uSeg EPG that is use by the application EPG. Otherwise, the VDS uSeg will not match VM attributes or place the VM into the uSeg EPG.

2

Associate the new uSeg EPG with a VMM domain profile; you need to associate it with the same VMM domain profile used by the application EPG.

3

Configure attributes for the uSeg EPG.

4

Verify that the end points have moved from the application EPG to the uSeg EPG.

Follow the instructions for these steps in the Configuring Microsegmentation with Cisco ACI section in this guide.

Configuring Microsegmentation with Cisco ACI Using the GUI

You can use Cisco APIC to configure Microsegmentation with Cisco ACI to put VMs that belong to different application EPGs or the same EPG into a new uSeg EPG. The task is essentially the same for Cisco ACI Virtual Edge, Cisco AVS, VMware VDS, and Microsoft Hyper-V Virtual Switch; the slight differences are noted in the procedure.

Procedure

Step 1

Log into the Cisco APIC.

Step 2

Choose Tenants and then choose the tenant where you want to create a microsegment.

Step 3

In the tenant navigation pane, expand the tenant folder, the Application Profiles folder, and the profile folder.

Step 4

Complete one of the following actions:

- If you are using Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch, skip the following substeps and continue with Step 5.

- If you are using VMware VDS, complete the following substeps.

1. Expand the Application EPGs folder and the folder for the application EPG.

2. Right-click on the folder Domains (VMs and Bare-Metals).

3. In the Add VMM Domain Association dialog box, after you choose the VMM domain, check the Allow Micro-Segmentation check box.

If you are using VMware VDS, you also must configure all the required parameters.

4. Click Submit.

Step 5

In the tenant navigation pane, right-click the uSeg EPGs folder, and then choose Create Useg EPG.

Step 6

In the Create USeg EPG Step 1 > Identity dialog box, complete the following steps to begin creation of an uSeg EPG for a group of VMs:

1. In the Name field, enter a name.

We recommend that you choose a name that indicates that the new uSeg EPG is a microsegment.

2. In the intra-EPG isolation field, select enforced or unenforced.

If you select enforced, Cisco ACI prevents all communication between the endpoint devices within this uSeg EPG.

3. In the Bridge Domain area, choose a bridge domain from the drop-down list.

Note

For VMware VDS, you must choose the same bridge domain that is used for the application EPG. Otherwise, the VDS uSeg will not match VM attributes and will not place the VM into a uSeg EPG.

4. (Optional) In the Epg Match Precedence field, choose an integer to set the precedence for the uSeg EPG among other VM-based attribute uSeg EPGs, overriding default rules.

The larger the integer, the higher the precedence.

5. Click Next.

Step 7

In the Create USeg EPG Step 2 > Domains, complete the following steps to associate the uSeg EPG with a VMM domain.

1. Click the + (plus) icon at the right of the dialog box.

2. From the Domain Profile drop-down list, choose a profile.

If you have a Cisco ACI Virtual Edge, Cisco AVS, or a VMware VDS, choose a VMware domain; if you have a Microsoft Hyper-V Virtual Switch, choose a Microsoft domain.

Note

You must choose the same domain that is used by the application EPG.

3. From the Deploy Immediacy drop-down list, accept the default On Demand if you have Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch; choose Immediate if you have VMware VDS.

4. From the Resolution Immediacy drop-down list, accept the default Immediate.

5. In the Encap Mode drop-down list, accept the default Auto.

6. In the Port Encap (or Secondary VLAN for Micro-Seg) field, accept the default value if you are using VMware VDS; accept the default value if you are using Cisco ACI Virtual Edge, Cisco AVS, or Microsoft Hyper-V Virtual Switch.

7. If you have Cisco ACI Virtual Edge, from the Switching Mode drop-down list, choose a mode.

Choose AVE to switch the uSeg EPG through Cisco ACI Virtual Edge; choose native to switch the uSeg EPG through the VMware VDS.

8. Click Update and then click Finish.

Step 8

In the navigation page for the tenant, open the folder for the uSeg EPG that you just created.

Step 9

Click the uSeg Attributes folder.

The uSeg Attributes work pane appears, where you configure attributes to filter for VMs that you want to put into the uSeg EPG.

Step 10

(Optional) If you will filter using VM-based attributes, in the uSeg Attributes work pane, from the match drop-down list, choose Match Any or Match All.

The match feature enables you to use multiple attributes to filter VMs for the uSeg EPG. The default is Match Any. The match all feature is supported for VM-based attributes only. See the sections "VM Filtering when Matching Any Attribute" and "VM Filtering when Matching All Attributes" in the microsegmentation chapter of the Cisco ACI Virtualization Guide.

Step 11

Click the + or the +( icon to add a filtering statement.

The + icon allows you to create a simple statement, one that creates a filter for a single attribute. You can add multiple simple statements to filter for multiple attributes. The +( icon allows you to create a block, or nested, statement, which allows you to set attributes in a hierarchy, which filters for the highest-level attribute first and then filters for lower-level attributes. See the section VM Filtering when Using Simple or Block Statements in this guide for more information.

Step 12

Complete one of the following series of steps to configure the filter.

If you want to use...

Then...

An IP-based attribute

1. From the Select a type... drop-down list, choose IP.

2. From the Use EPG Subnet? drop-down list, choose Yes or No.

If you choose Yes, you will use a previously defined subnet as the IP attribute filter.

If you choose No, enter the VM IP address or a subnet with the appropriate subnet mask in the field to the right of the Use EPG Subnet? drop-down list.

3. (Optional) Create a second IP Address filter by repeating substeps a through c.

You might want to create a second IP Address filter to include discontinuous IP addresses in the microsegment.

4. Click Submit.

A MAC-based attribute

1. From the Select a type... drop-down list, choose MAC.

2. In the right field, enter the MAC address of the VM.

3. Click Submit.

A VM-based Custom Attribute

1. From the Select a type... drop-down list, choose VM - Custom Attribute.

2. Click the search icon next to the field to the right of the Select a type... drop-down list.

3. In the Select Custom Attribute dialog box, choose a controller from the Controller drop-down list.

4. From the VM drop-down list, choose a VM.

5. From the Attribute Name drop-down list, choose the name, and then click Select.

6. From the operator drop-down list, choose an operator, and then enter a value in the field to the right of the drop-down list.

7. Click Submit.

A VM-based Tag attribute (Cisco ACI Virtual Edge, Cisco AVS and VMware VDS only)

1. From the Select a type... drop-down list, choose VM - Tag.

2. Click the magnifying glass icon next to the Category field, and in the Select VM Category dialog box, choose the category from the Category Name drop-down list, and then click Select.

The category that you enter must be identical to the one assigned earlier for the tag in VMware vCenter.

3. From the operator drop-down list, choose the appropriate operator.

4. Click the magnifying glass icon next to the field on the right, and in the Select VM Tag dialog box, select a tag from the Tag Name drop-down list and then click Select.

5. Click Submit.

Any other VM-based Attribute

1. From the Select a type... drop-down list, choose a VM attribute.

2. From the operator drop-down list, choose the appropriate operator.

3. Complete one of the following steps:

- If you chose the Datacenter VM-based attribute, enter the name of the data center in the field to the right of the operator drop-down list.

- If you chose any other VM-based attribute, click the search icon next to the field to the right of the operator drop-down list, choose appropriate values for the attribute in the Select VM Identifier dialog box, and then click Select.

4. Click Submit.

Step 13

Click the + or the +( icon to add additional attributes for the uSeg EPG.

Step 14

Repeat Step 2 through Step 13 to create additional uSeg EPGs.

What to do next

Verify that the uSeg EPG was created correctly.

If you configured a VM-based attribute, complete the following steps:

1. In the Cisco APIC navigation pane, click the new microsegment.

2. In the work pane, click the Operational tab and then ensure that the Client End-Points tab is active.

3. In the work pane, verify that the VMs that you wanted to move from the application EPG appear as endpoints for the new uSeg EPG.

If you configured an IP- or MAC-based attribute, make sure that traffic is running on the VMs that you put into the new microsegments.

Chapter 5. Intra-EPG Isolation Enforcement and Cisco ACI

This chapter contains the following sections:

- Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch

Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch

Intra-EPG Isolation is an option to prevent physical or virtual endpoint devices that are in the same base EPG or microsegmented (uSeg) EPG from communicating with each other. By default, endpoint devices included in the same EPG are allowed to communicate with one another. However, conditions exist in which total isolation of the endpoint devices from on another within an EPG is desirable. For example, you may want to enforce intra-EPG isolation if the endpoint VMs in the same EPG belong to multiple tenants, or to prevent the possible spread of a virus.

A Cisco ACI virtual machine manager (VMM) domain creates an isolated PVLAN port group at the VMware VDS or Microsoft Hyper-V Virtual Switch for each EPG that has intra-EPG isolation enabled. A fabric administrator specifies primary encapsulation or the fabric dynamically specifies primary encapsulation at the time of EPG-to-VMM domain association. When the fabric administrator selects the VLAN-pri and VLAN-sec values statically, the VMM domain validates that the VLAN-pri and VLAN-sec are part of a static block in the domain pool.

Note

When intra-EPG isolation is not enforced, the VLAN-pri value is ignored even if it is specified in the configuration.

VLAN-pri/VLAN-sec pairs for the VMware VDS or Microsoft Hyper-V Virtual Switch are selected per VMM domain during the EPG-to-domain association. The port group created for the intra-EPG isolation EPGs uses the VLAN-sec tagged with type set to PVLAN. The VMware VDS or the Microsoft Hyper-V Virtual Switch and fabric swap the VLAN-pri/VLAN-sec encapsulation:

- Communication from the Cisco ACI fabric to the VMware VDS or Microsoft Hyper-V Virtual Switch uses VLAN-pri.

- Communication from the VMware VDS or Microsoft Hyper-V Virtual Switch to the Cisco ACI fabric uses VLAN-sec.

Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch

Note these details regarding this illustration:

1. EPG-DB sends VLAN traffic to the Cisco ACI leaf switch. The Cisco ACI egress leaf switch encapsulates traffic with a primary VLAN (PVLAN) tag and forwards it to the Web-EPG endpoint.

2. The VMware VDS or Microsoft Hyper-V Virtual Switch sends traffic to the Cisco ACI leaf switch using VLAN-sec. The Cisco ACI leaf switch drops all intra-EPG traffic because isolation is enforced for all intra VLAN-sec traffic within the Web-EPG.

3. The VMware VDS or Microsoft Hyper-V Virtual Switch VLAN-sec uplink to the Cisco ACI Leaf is in isolated trunk mode. The Cisco ACI leaf switch uses VLAN-pri for downlink traffic to the VMware VDS or Microsoft Hyper-V Virtual Switch.

4. The PVLAN map is configured in the VMware VDS or Microsoft Hyper-V Virtual Switch and Cisco ACI leaf switches. VM traffic from WEB-EPG is encapsulated in VLAN-sec. The VMware VDS or Microsoft Hyper-V Virtual Switch denies local intra-WEB EPG VM traffic according to the PVLAN tag. All intra-ESXi host or Microsoft Hyper-V host VM traffic is sent to the Cisco ACI leaf using VLAN-Sec.

Related Topics

For information on configuring intra-EPG isolation in a Cisco ACI Virtual Edge environment, see the chapter "Intra-EPG Isolation Enforcement for Cisco ACI Virtual Edge" in the Cisco ACI Virtual Edge Configuration Guide .

For information on configuring intra-EPG isolation in a Cisco AVS environment, see the chapter "Intra-EPG Isolation Enforcement for Cisco AVS" in the Cisco Application Virtual Switch Configuration Guide .

- Configuring Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch using the GUI

Configuring Intra-EPG Isolation for VMware VDS or Microsoft Hyper-V Virtual Switch using the GUI

Procedure

Step 1

Log into Cisco APIC.

Step 2

Choose Tenants > tenant .

Step 3

In the left navigation pane expand the Application Profiles folder and appropriate application profile.

Step 4

Right-click the Application EPGs folder and then choose Create Application EPG.

Step 5

In the Create Application EPG dialog box, complete the following steps:

1. In the Name field, add the EPG name.

2. In the Intra EPG Isolation area, click Enforced.

3. In the Bridge Domain field, choose the bridge domain from the drop-down list.

4. Associate the EPG with a bare metal/physical domain interface or with a VM Domain.

- For the VM Domain case, check the Associate to VM Domain Profiles check box.

- For the bare metal case, check the Statically Link with Leaves/Paths check box.

5. Click Next.

6. In the Associated VM Domain Profiles area, click the + icon.

7. From the Domain Profile drop-down list, choose the desired VMM domain.

For the static case, in the Port Encap (or Secondary VLAN for Micro-Seg) field, specify the secondary VLAN, and in the Primary VLAN for Micro-Seg field, specify the primary VLAN. If the Encap fields are left blank, values will be allocated dynamically.

Note

For the static case, a static VLAN must be available in the VLAN pool.

Step 6

Click Update and click Finish.

Chapter 6. Cisco ACI with Cisco ACI vPod

- Cisco ACI with Cisco ACI vPod

Cisco ACI with Cisco ACI vPod

Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) is a software-only solution that enables you to virtually extend the Cisco ACI fabric into bare-metal cloud environments and other remote locations. It is in general availability beginning in Cisco APIC Release 4.0(2).

You can deploy Cisco ACI vPod wherever you have at least two servers on which you can run the ESXi hypervisor. It allows you to use Cisco ACI Virtual Edge where you do not have a physical leaf.

Cisco ACI vPod and its components—a pair of virtual spines (vSpines), a pair of virtual leafs (vLeafs), and Cisco ACI Virtual Edge—run on the ESXi hypervisor. The vSpines and vLeafs handle control plane management, and the Cisco ACI Virtual Edge handles packet forwarding, policy enforcement, and all data plane management.

Cisco ACI vPod manages a data center defined by the VMware vCenter Server. You can have up to eight instances of Cisco ACI Virtual Edge in each Cisco ACI vPod in the remote location. You use Cisco APIC to manage Cisco ACI vPod nodes and enforce Cisco ACI policy in the virtual data center.

Cisco ACI vPod communicates with a physical, on-premises pod or multipod over an interpod network. You configure the physical pod or multipod, the interpod connection, and Cisco ACI vPod in Cisco APIC. You then use the Cisco ACI vCenter plug-in, a Python script, or PowerCLI to deploy Cisco ACI vPod component virtual machines (VMs).

Cisco ACI vPod is compatible with any server hardware listed in the VMware Hardware Compatibility Guide.

You can find information about Cisco ACI vPod in the following documents on Cisco.com:

- Cisco ACI Virtual Pod Release Notes

- Cisco ACI Virtual Pod Installation Guide

- Cisco ACI Virtual Pod Getting Started Guide

Chapter 7. Cisco ACI with Cisco ACI Virtual Edge

- Cisco ACI with Cisco ACI Virtual Edge

Cisco ACI with Cisco ACI Virtual Edge

Beginning with the Cisco APIC Release 3.1(1), the Cisco Application Centric Infrastructure (ACI) supports Cisco ACI Virtual Edge. Cisco ACI Virtual Edge is the next generation of the Application Virtual Switch (AVS) for Cisco ACI environments. Cisco ACI Virtual Edge is a hypervisor-independent distributed service VM that leverages the native distributed virtual switch that belongs to the hypervisor. Cisco ACI Virtual Edge runs in the user space, operates as a virtual leaf, and is managed by the Cisco Application Policy Infrastructure Controller (APIC).

You can find information about Cisco ACI Virtual Edge in the following documents on Cisco.com:

- Cisco ACI Virtual Edge Release Notes

- Cisco ACI Virtual Edge Installation Guide

- Cisco ACI Virtual Edge Configuration Guide

Chapter 8. Cisco ACI with Cisco AVS

- Cisco ACI with Cisco AVS

Cisco ACI with Cisco AVS

Cisco Application Virtual Switch is a hypervisor-resident distributed virtual switch that is specifically designed for Cisco Application Centric Infrastructure (ACI) and managed by the Application Policy Infrastructure Controller (APIC). Cisco AVS implements the OpFlex protocol for control plane communication.

Cisco AVS supports two modes of traffic forwarding: local switching and no local switching. The forwarding mode is selected during Cisco AVS installation.

Cisco AVS is supported as a vLeaf for Cisco APIC with the VMware ESXi hypervisor. It manages a data center defined by the VMware vCenter Server.

Cisco AVS is compatible with any upstream physical access layer switch that complies with the Ethernet standard, including Cisco Nexus switches. Cisco AVS is compatible with any server hardware listed in the VMware Hardware Codmpatibility Guide.

You can find information about Cisco AVS in the following documents on Cisco.com:

- Cisco AVS Release Notes

- Cisco AVS Installation Guide

- Cisco AVS Configuration Guide

- Cisco AVS Troubleshooting Guide

Chapter 9. Cisco ACI with VMware vRealize

This chapter contains the following sections:

- About Cisco ACI with VMware vRealize

- Getting Started with Cisco ACI with VMware vRealize

- Cisco ACI with VMware vRealize Upgrade Workflow

- Cisco ACI with VMware vRealize Downgrade Workflow

- Use Case Scenarios for the Administrator and Tenant Experience

- Troubleshooting

- Removing the APIC Plug-in

- Plug-in Overview

- Configuring a vRA Host for the Tenant in the vRealize Orchestrator

- Configuring an IaaS Host in the vRealize Orchestrator

About Cisco ACI with VMware vRealize

Cisco Application Centric Infrastructure (ACI), in addition to integrating with VMware vCenter, integrates with VMware's products vRealize Automation (vRA) and vRealize Orchestrator (vRO). vRA and vRO are parts of the VMware vRealize Suite for building and managing multivendor hybrid cloud environments.

Beginning with Cisco Application Policy Infrastructure Controller (APIC) Release 2.0(1), vRA and vRO support Cisco AVS in addition to VMware DVS. Beginning with Cisco APIC Release 3.1(1), vRA and vRO support Cisco Application Centric Infrastructure (ACI) Virtual Edge (Cisco ACI Virtual Edge).

Note

In the Cisco APIC GUI, Cisco ACI Virtual Edge is indicated with the term AVE.

- Cisco ACI with VMware vRealize Solution Overview

- Physical and Logical Topology

- About the Mapping of ACI Constructs in VMware vRealize

- Event Broker VM Customization

Cisco ACI with VMware vRealize Solution Overview

vRA integration is delivered through a set of service blueprints imported into vRA. The service blueprints leverage the vRO Application Policy Infrastructure Controller (APIC) workflows, providing a set of catalog items in a self-service portal that allows Tenants to build, manage, and remove networking components. Multi-machine with ACI workflows achieve following functionalities:

- Auto-create Tenant Endpoint Groups (EPGs)

- Required policies in APIC

- Create VMs and portgroups in vCenter

- Auto-place the VMs is respective port groups

- Created by APIC

- Create security policy with access lists

- Configure L4-L7 services, and provide external connectivity

This consumption model allows users to deploy single and multi-tier application workloads in single click with pre-defined as well as customizable compute and network policies. Catalog items are published by infrastructure administrators, whereby granular entitlements can be added or removed on a per-tenant basis.

The integration offers two modes of networking:

Mode

Description

Shared

Shared mode is for Tenants who do not have a preference for what IP address space they use and a shared address space with shared context (VRF) is used across tenants. Isolation is provided using ACI Endpoint Groups (EPGs) and connectivity among EPGs are enabled using a white listing method.

Virtual Private Cloud (VPC)

VPC mode is a bring your own address space architecture, where network connectivity is isolated via a unique context (VRF) per tenant and external connectivity is provided via a common shared L3 out.

Physical and Logical Topology

This section shows the logical model of the vRealize ACI Integration and comparison between a Shared Services Plan and Virtual Private Cloud Plan.

This figure shows a logical model of the vRealize ACI Integration.

This figure shows the comparison between a Shared Services Plan and Virtual Private Cloud Plan.

For details, see the Cisco APIC Basic Configuration Guide.

About the Mapping of ACI Constructs in VMware vRealize

This table shows the mapping between the features of Cisco ACI policy and vRealize policy

Cisco ACI

VMware vRealize

Tenant

Tenant

EPGs

Networks

Layer 3 external connectivity

External routed network

Contract

Security policy

Filter

Rule entry list

L4-L7 service device

Shared load balancer or firewall

This list provides details regarding the features:

- Tenant—Tenants can be employees within an organization, business units, application owners, or applications. Or if you are a service provider, they can be hosting customers (individuals or organizations that pay you to provide IT services).

- Networks—In Cisco ACI, the term “network” refers to EPGs, which are used to provide a new model for mapping applications to the network. Rather than using forwarding constructs, such as addresses or VLANs, to apply connectivity and policy, EPGs use a grouping of application endpoints. EPGs are mapped to networks in the vRealize portal. The isolated networks act as containers for collections of applications, or of application components and tiers, that can be used to apply forwarding and policy logic. They allow the separation of network policy, security, and forwarding from addressing and instead apply these to logical application boundaries. When a network is created in vRealize, in the back end it is created as a port group in vCenter. A vRealize tenant can use vCenter to manage the computing resources and can attach the virtual machine to the appropriate network.

- Layer 3 external connectivity—The Cisco ACI fabric connects to the outside through Layer 3 external networks. These constructs are also available for vRealize tenants to access other services within the data center, across the data center, or on the internet.

- Security policy—Cisco ACI is built on a highly secure model, in which traffic between EPGs (isolated networks) is denied, unless explicitly allowed by policy contracts. A Cisco ACI contract is mapped to a security policy in the vRealize portal. The security policy describes which networks (EPGs) will provide and consume a service. The security policy contains one or more rule entry lists (filters), stateless firewall rules that describe a set of Layer 4 TCP or User Datagram Protocol (UDP) port numbers that define the communication between the various applications.

- Shared load balancer and firewall—Cisco ACI treats services as an integral part of an application. Any services that are required are managed as a service graph that is instantiated on the Application Policy Infrastructure Controller (APIC) . Users define the service for the application, and service graphs identify the set of network and service functions that are needed by the application. Cisco ACI has an open ecosystem of L4-7 service vendors whose services integrate natively with Cisco ACI. This integration is achieved through device packages written and owned by the vendors. The APIC manages the network services and inserts the services according to the Cisco ACI policy model. For vRealize, Cisco ACI offers F5 and Citrix load balancers and Cisco ASA firewalls, both in virtual and physical form factors, which are connected to the Cisco ACI fabric and shared across the various vRealize tenants. After the device has been integrated into Cisco ACI, the vRealize administrator can choose to add the device as a premium service and upsell the plan. The vRealize administrator manages the virtual IP address range for the shared device, to simplify the vRealize tenant’s workflow.

- VPC plan—In a VPC plan, vRealize tenants can define their own address spaces, bring a DHCP server, and map their address spaces to networks. A VPC tenant can also be offered services, such as load balancing, from the shared service plan. In this scenario, a device would have multiple virtual NICs (vNICs). One vNIC would connect to the private address space, and another would connect to the shared service infrastructure. The vNIC that connects to the shared service infrastructure would have an address assigned by the infrastructure and would also consume a shared load balancer owned by the infrastructure.

Event Broker VM Customization

vRealize Automation Event Broker is a workflow subscription service for vRealize Automation to call workflows from the vRealize Orchestrator under predefined conditions the user sets. It is supported beginning with Cisco APIC 3.0(1).

A deployment of a single or multitier application is automatically subscribed to the Event Broker. Machine operations such as creation or deletion on any machine, configured by the vRA, trigger the Event Broker. This invokes the preconfigured operations to the Cisco APIC defined by the Property Groups associated to a single or multitier application.

To add the Cisco APIC workflow subscription, follow the instructions at Setting Up the VMware vRealize Automation Appliance for ACI. The workflow subscription then will be added automatically.

Getting Started with Cisco ACI with VMware vRealize

This section describes how to get started with Cisco ACI with VMware vRealize.

You must download and unzip the Cisco ACI and VMware vRealize file for the 2.2(1) release before installing Cisco ACI with VMware vRealize.

Procedure

Step 1

Go to Cisco's Application Policy Infrastructure Controller (APIC) Website:

http://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

Step 2

Choose All Downloads for this Product.

Step 3

Choose the release version and the apic-vrealize-2.2.1x.tgz file.

Step 4

Click Download.

Step 5

Unzip the apic-vrealize-2.2.1x.tgz file.

Note

Cisco ACI with VMware vRealize only supports ASCII characters. Non-ASCII characters are not supported.

- Prerequisites for Getting Started with Cisco ACI with VMware vRealize

- Cisco ACI with VMware vRealize Installation Workflow

Prerequisites for Getting Started with Cisco ACI with VMware vRealize

Before you get started, ensure that you have verified that your vRealize computing environment meets the following prerequisites:

- vRealize Automation Release 7.0-7.4 must be installed.

See VMware's vRealize documentation.

- The vRealize ACI plug-in version and the Cisco APIC version must match.

- A tenant is configured in vRealize automation and associated with identity store. The tenant must have one or more users configured with "Infra Admin", "Tenant Admin", and "Tenant user" roles.

See VMware's vRealize documentation.

- The tenant must have one more "Business group" configured.

See VMware's vRealize documentation.

- Configure vRealize Orchestrator as an end-point.

See VMware's vRealize documentation.

- Configure vCenter as an endpoint.

See VMware's vRealize documentation.

- Configure "Reservations" using the vCenter compute resources.

See VMware's vRealize documentation.

- Set up the vRealize Appliance.

See VMware's vRealize documentation.

- If Layer 3 (L3) Out policies are to be consumed by a tenant, you must configure a BGP route reflector.

See the Cisco APIC Basic Configuration Guide about Configuring an MP-BGP Route Reflector Using the Basic GUI or Configuring an MP-BGP Route Reflector.

- Setup a vRA handle in vRO.

This is used for Installing the ACI service catalog workflow.

- Setup a IAAS handle in vRO.

This is used for Installing the ACI service catalog workflow.

See Setting Up an IaaS Handle in vRealize Orchestrator.

- Install the vCAC/vRA Custom Property Toolkit for vCO/vRO. You can download the package from the following URL:

https://communities.vmware.com/docs/DOC-26693

- The embedded vRO in vRA has the vCAC vRO plug-in that is installed by default. If you are using a standalone vRO, the vCAC vRO plug-in must be installed. You can download the plug-in from the following URL:

https://solutionexchange.vmware.com/store/products/vmware-vrealize-orchestrator-plug-in-for-vra-6-2-0

- Setting Up an IaaS Handle in vRealize Orchestrator

Setting Up an IaaS Handle in vRealize Orchestrator

This section describes how to set up an Infrastructure as a Service (IaaS) handle in the vRealize Orchestrator (vRO).

Procedure

Step 1

Log in to the VMware vRealize Orchestrator as administrator.

Step 2

Once the VMware vRealize Ochestrator GUI appears, from the drop-down list, choose Run from the menu bar.

Step 3

In the navigation pane, choose the Workflows icon.

Step 4

Choose Adminstrator@vra_name > Library > vRealize Automation > Configuration > Add the IaaS host of a vRA host.

Step 5

Right-click Add the IaaS host of a vRA host and choose Start Workflow.

Step 6

In the Start Workflow: Add the IaaS host of a vRA host dialog box, perform the following actions:

1. In the vRA host field, enter your vRealize Handle.

2. Click Next.

Step 7

In the next screen, perform the following actions:

1. In the Host Name field, enter a name.

2. In the Host URL field, enter the URL of your IaaS host.

3. Use the default values for the remaining fields.

4. Click Next.

Step 8

In the next screen, perform the following actions:

1. In the Session mode drop-down list, choose Shared Session.

2. In the Authentication user name field, enter the authentication user name.

3. In the Authentication password field, enter the password.

4. Click Next.

Step 9

In the next screen, perform the following actions:

1. In the Workstation for NTLM authentication field, enter the name of the workstation that you will use for NTLM authentication.

2. In the Domain for NTLM authentication field, enter the domain that is used in the IaaS host URL.

3. Click Submit.

Cisco ACI with VMware vRealize Installation Workflow

This section describes the Cisco ACI with VMware vRealize installation workflow.

Procedure

Step 1

Install the APIC plug-in on the vRealize Orchestrator (vRO).

For more information, see Installing the APIC Plug-in on the vRealize Orchestrator.

Step 2

Set up the VMware vRealize Automation Appliance for ACI.

For more information, see Setting Up the VMware vRealize Automation Appliance for ACI.

- Installing the APIC Plug-in on the vRealize Orchestrator

- Setting Up the VMware vRealize Automation Appliance for ACI

- Day-0 Operations of ACI

- Associating AEP with VMware VMM Domain

Installing the APIC Plug-in on the vRealize Orchestrator

This section describes how to install APIC plug-in on the vRealize Orchestrator.

Procedure

Step 1

Once you have unzipped the package, save the aci-vra-plugin-3.0.1000.N.dar file in a known directory.

Step 2

Log in to the vRA appliance as root using SSH, enter:

$ ssh root@<vra_ip>

Step 3

Start the configurator to enable the configurator services web interface, enter the following commands:

# service vco-configurator start
.
.
.
Tomcat started.
Status:           Running as PID=15178

Ensure the status is running.

Step 4

Log in to the VMware appliance using the Firefox browser, enter:

https:// applicance_address :8283/vco-controlcenter

Note

Cisco recommends using the Firefox browser.

Do not use the Internet Explorer or the Chrome browser for the first time. There is a known issue when you use the default username and password. It does not login properly.

For more information, see https://communities.vmware.com/thread/491785.

1. In the VMware vRealize Orchestrator Configuration GUI, enter the default username and password which is vmware/vmware. You will then be required to change the password.

Step 5

Under the Plug-Ins section, click Manage Plug-Ins.

Step 6

Under Install plug-in, click the Browse... button and perform the following steps:

1. Locate where you saved the aci-vra-plugin-3.0.1000.N.dar file and choose the aci-vra-plugin-3.0.1000.N.dar file.

2. Click Install on the right, and when the Cisco APIC Plug-in displays, click Install again.

- A message highlighted in green displays, saying that the plug-in is installed.

- A message highlighted in yellow displays, saying "The Orchestrator server must be restarted for the changes to take effect. The restart can be performed from the Startup Options page."

Step 7

Click Startup Options.

You will be directed to the Startup Options page.

Step 8

Click Restart to restart the server. Wait until the Current Status displays RUNNING.

Step 9

Navigate back to the Manage Plug-Ins page by clicking Home on the top left and then clicking Manage Plug-Ins under the Plug-Ins section.

Step 10

Verify the Cisco APIC plug-in has been installed by looking for it under Plug-Ins.

The plug-in will be displayed first with the Cisco icon.

Setting Up the VMware vRealize Automation Appliance for ACI

This section describes how to set up the VMware vRealize Automation Appliance for Cisco ACI.

Procedure

Step 1

Log in to the VMware vRealize Automation Appliance as the administrator through your tenant portal using the browser:

https:// applicance_address /vcac/org/ tenant_id

Example:

https://192.168.0.10/vcac/org/tenant1

Enter the admin username and password.

Step 2

In the VMware vRealize Automation Appliance GUI, perform the following actions:

1. Choose Administration > Users & Groups > Custom Groups

2. In the Custom Group pane, click Add to add a custom group.

3. Enter the name of the custom group. (Service Architect)

4. In the Roles to this group field, select the custom group you created in the previous step. (Service Architect)

5. Choose the Member pane, enter and select the user name(s).

6. Click Add.

This creates a custom group with members.

7. In the Custom Group pane, choose the custom group you created. (Service Architect)

8. In the Edit Group pane, you can verify the members in the Members pane.

Step 3

In the browser, enter the vRealize Automation Appliance.

https://applicance_address

For example:

https://vra3-app.ascisco.net

1. Choose the vRealize Orchestrator Client to download the client.jnlp file.

2. The Downloads dialog box will appear, launch the client.jnlp file.

Step 4

Log in to the VMware vRealize Orchestrator as administrator.

Step 5

Once the VMware vRealize Orchestrator GUI appears, from the drop-down list, choose Run from the menu bar.

Step 6

In the Navigation pane, choose the Workflows icon.

Step 7

Choose Adminstrator@vra3-app.ascisco.net > Cisco APIC Workflows > Utils > Install ACI Service Catalog.

Step 8

Right-click Install ACI Service Catalog and choose Start Workflow.

Step 9

In the Start Workflow - Install ACI Service Catalog dialog box, perform the following actions:

1. In the APIC Hostname/IP Address field, enter the APIC hostname or IP address.

2. In the APIC Admin Password field, enter the APIC admin password.

3. In the vRealize Automation IP Address field, provide the IP address for the vRA.

4. In the vRealize Automation handle field, click Not set, navigate and choose the vRealize automation handle for this appliance.

5. In the Business group field, click Not set to choose business group.

Note

If running vRealize 7.0, you need to select the Business Group from Business Group (Deprecated).

Note

Usernames need to include the domain name. For example: admin1@vsphere.local

6. In the Admin User field, enter the tenant admin user.

7. In the vRealize Automation Admin Password field, enter the admin password for the vRA.

8. In the End users field, click Not set and enter the user names to enable privilege for.

Note

Do not copy and paste the end user names, you should type the user names.

9. In the JSON File containing vRealize Properties field, click Not set, navigate and choose the JSON file containing the vRealize properties. (aci-vra-properties-3.0.1000.x.json)

Note

Usernames need to include the domain name. For example: admin1@vsphere.local

10. In the Zip file containing the service blueprints field, click Not set, navigate and choose the zip file containing the service blueprints. (aci-vra-asd-3.0.1000.x.zip)

11. Click Submit.

Step 10

In the Navigation pane, you will see a green check mark next to the Install ACI Service Catalog, if the installation was successful.

Step 11

In the Navigation pane, choose the Workflows icon.

Step 12

Right-click Install ACI Property Definitions and choose Start Workflow.

Step 13

In the Start Workflow - Install ACI Property Definitions dialog box, click Net set, navigate and choose the IaaS host.

1. Click Submit.

In the Navigation pane, you will see a green checkmark next to the Install ACI Property Definitions, if the installation was successful.

Step 14

To verify as a tenant, log in to the vRealize Automation Appliance as tenant, choose Catalog and you will see the services.

Step 15

To verify as an administrator, log in to the vRealize Automation Appliance as administrator, choose Catalog and you will see the services.

1. Choose Infrastructure > Blueprints > Property Definitions and you will see the properties.

Day-0 Operations of ACI

This section describes day-0 operations of ACI.

Before you begin

- Fabric bring-up

Bring up the fabric and all topologies are supported.

- Access policies

- Attach Entity Policy (AEP)

- Configure access policies between the leaf switches and ESXi hosts to ensure CDP and LLDP is enabled between the leaf and host.

- Layer 3 (L3) Out configuration

- Create any L3 Out configurations in the common tenant that you wish to be consumed user tenants.

- You can choose any name for the L3 policy.

- External EPG must be named "[L3OutName|InstP]".

- Create two policies.

For shared plan, specify "default" and for VPC plan, specify "vpcDefault".

For more information, see About L3 External Connectivity.

- Service graph templates and devices

Create any service graph devices in the common tenant.

For more information, see Configuring the Services on APIC Using XML POST.

- Security domains and tenant user

- vRealize plug-in requires two user accounts.

The first account needs administrator privileges. This account allows you to create, read, update, and destroy objects in the tenant common, access policies, and VMM domains.

The second account needs restricted tenant privileges. This account allows you to only read common tenant and VMM domains, but you can create, read, update, and destroy objects in their own tenant.

- Role-based access control (RBAC) rules are enforced through the APIC not the plug-in.

Procedure

See the Cisco APIC Basic Configuration Guide for more information.

Associating AEP with VMware VMM Domain

This section describes how to associate an attachable entity profile (AEP) with VMware VMM domain.

Note

You do not need to perform this procedure if the domain type is Cisco AVS.

Procedure

Step 1

Log in to the APIC GUI, and then choose Fabric > Access Policies.

Step 2

In the navigation pane, expand Policies > Global > Attachable Access Entity Profiles and then click the profile .

Step 3

In the work pane, perform the following actions:

1. In the Domains (VMM, Physical or External) Associated to Interfaces field, click the + to expand.

2. In the unformed field, choose a VMM domain and click Update.

Cisco ACI with VMware vRealize Upgrade Workflow

This section describes the Cisco ACI with VMware vRealize upgrade workflow.

Procedure

Step 1

Upgrade the APIC image.

Step 2

Upgrade the APIC plug-in on the vRealize Orchestrator (vRO).

For more information, see Upgrading the APIC Plug-in on the vRealize Orchestrator.

Step 3

Set Up the VMware vRealize Automation Appliance for ACI.

For more information, see Setting Up the VMware vRealize Automation Appliance for ACI.

Step 4

Verify the connection between APIC and vRealize.

For more information, see Verifying the Connection Between APIC and vRealize.

- Upgrading the APIC Plug-in on the vRealize Orchestrator

- Verifying the Connection Between APIC and vRealize

Upgrading the APIC Plug-in on the vRealize Orchestrator

This section describes how to upgrade the APIC plug-in certificate on the vRealize Orchestrator.

Procedure

Step 1

To upgrade, first follow the directions in Installing the APIC Plug-in on the vRealize Orchestrator.

Step 2

Upgrade your service blueprints, service categories, and entitlements, see Setting Up the VMware vRealize Automation Appliance for ACI.

Verifying the Connection Between APIC and vRealize

After you have upgraded the Application Policy Infrastructure Controller (APIC) controller and the switch software, you must verify the connection from the vRealize Orchestrator to APIC.

Before you begin

- Ensure the APIC controller and the switch software is upgraded.

For more information, see the Cisco ACI Firmware Management Guide.

Procedure

Step 1

Log in to the vRealize Orchestrator as administrator.

Step 2

In the navigation pane, choose the Inventory icon.

Step 3

Expand the Cisco APIC Plugin, choose the APIC and check the following:

1. In the General pane, check if the controllers are showing in the Name field.

2. Check if you can maneuver through the nested hierarchy below the APIC. This ensures you are communicating with APIC.

If the connection from vRO to APIC is not established, then next to the APIC name the string down will be present, indicating that the connection is down.

Cisco ACI with VMware vRealize Downgrade Workflow

This section describes the Cisco ACI with VMware vRealize downgrade workflow.

Procedure

Step 1

Downgrade the APIC image.

Step 2

Delete the APIC plug-in package and all the APIC workflows.

For more information, see Deleting Package and Workflows.

Step 3

Install the APIC plug-in on the vRealize Orchestrator (vRO).

For more information, see Upgrading the APIC Plug-in on the vRealize Orchestrator.

Step 4

Set up the VMware vRealize Automation Appliance for ACI.

For more information, see Setting Up the VMware vRealize Automation Appliance for ACI.

Step 5

Verify the connection between APIC and vRealize.

For more information, see Verifying the Connection Between APIC and vRealize.

- Deleting Package and Workflows

Deleting Package and Workflows

This section describes how to delete the package and workflows.

Procedure

Step 1

Log in to the vRO client as administrator.

Step 2

Choose the Design role.

Step 3

Choose the Packages tab.

Step 4

Right-click on the com.cisco.apic.package and choose Delete element with content.

Step 5

Choose Keep Shared in the pop-up window.

Step 6

Choose the Workflows tab.

Step 7

Ensure that all workflows in the "Cisco APIC workflows" folder and subfolders are deleted.

To delete the workflow: Select the workflow, right-click and choose Delete.

Use Case Scenarios for the Administrator and Tenant Experience

This section describes use case scenarios for the administrator and tenant experience.

- Overview of Tier Application Deployment

- About Plan Types

- About vRealize Service Categories and Catalog Items

- ACI Administrator Services in vRealize

- ACI Tenant Services in vRealize

- Entitlements for ACI catalog-items in vRealize

- ACI Plug-in in vRealize Orchestrator

- About Load Balancing and Firewall Services

- About L3 External Connectivity

- Administrator Experiences

- Application Deployment Scenarios

- About Property Groups

- About Service Blueprints

- Documentation of APIC Workflows in vRealize Orchestrator

- Writing Custom Workflows Using the APIC Plug-in Method

- Multi-Tenancy and Role based Access Control Using Security Domains

- APIC Credentials for Workflows

Overview of Tier Application Deployment

This section describes the overview of 3-tier application deployment.

Deployment of a single-tier application using property groups

See Deploying a Single-Tier Application Using Property Groups.

Deployment of a 3-tier application using a multi-machine blueprint

See Deploying a 3-Tier Application Using a Multi-Machine Blueprint.

- Deploying a Single-Tier Application Using Property Groups

- Deploying a 3-Tier Application Using a Multi-Machine Blueprint

Deploying a Single-Tier Application Using Property Groups

This section describes how to deploy a single-tier application using property groups.

Procedure

Step 1

Connect to the vRealize Automation appliance by pointing your browser to the following URL:

https://appliance_address/vcac/org/tenant_id

Step 2

Enter the tenant administrator username and password.

Step 3

Choose Catalog.

Step 4

Click Configure Property Groups.

You will configure the database tier.

Step 5

Click Request.

Step 6

In the Request Information tab, enter a description of the request.

Step 7

Click Next.

Step 8

In the Common tab, perform the following actions:

1. In the IaaS Host for vRealize field, click Add.

2. Put a check in the box next to the desired IaaS host.

3. Click Submit.

4. In the APIC Tenant field, click Add.

5. Expand apic_name > Tenants.

6. Put a check in the box next to the desired tenant's name.

Example:

green

7. Click Submit.

8. In the Property Group Name field, enter a name for the property group.

Example:

green-app-bp

9. In the Plan Type (Shared or VPC) field, click Shared.

10. In the VMM Domain/DVS field, click Add.

11. Expand apic_name > Vcenters > vcenter_name

12. Put a check in the box next to the desired vCenter's name.

Example:

green

13. Click Submit.

Step 9

Click Next.

Step 10

In the VM Networking tab, leave all of the fields at their default values.

Step 11

Click Next.

Step 12

In the Security tab, perform the following actions:

1. In the Configure Security Policy drop-down list, choose No.

Step 13

In the Load Balancer tab, from the drop-down list, choose No.

Step 14

In the Firewall tab, from the drop-down list, choose No.

Step 15

Click Submit.

Step 16

Click OK.

Step 17

To verify your request, choose the Requests tab.

1. Choose the request you submitted and click view details. Ensure the status is Succesful.

Step 18

(Optional) To edit a blueprint in the property group, choose Infrastructure > Blueprints > Property Groups.

1. In the Property Group pane, choose the property group you created (green-app-bp) and click edit.

2. In the Edit Property Group pane, choose the property group you want to edit and click on the pencil icon to edit a certain blueprint.

3. Once you have completed your edits, click OK.

Step 19

Attach the property group to the VMs, choose Infrastructure > Blueprints.

Step 20

In the Blueprints pane, click New Blueprint, from the drop-down list, choose Virtual > vSphere (vCenter).

Step 21

In the New Blueprint vSphere (vCenter)pane, perform the following actions:

1. In the Blueprint Information tab, enter the information to create your blueprint and click OK. See VMware's documentation for details on how to create your machine blueprint.

2. In the Build Information tab, enter the information to create your property group and click OK. See VMware's documentation for details on how to create your machine blueprint.

Step 22

In the Properties tab, perform the following actions:

1. In the Property Groups field, choose your property group that you created (green-app-bp) and click OK.

2. Click on the magnifying glass icon for the newly created property group (green-app-bp).

3. In the Property Group Custom Properties dialog box, ensure that the properties match your property group and this makes a connection with the VM and the ACI networking.

4. In the New Blueprint vSphere (vCenter)pane, click OK.

Step 23

In the Blueprints pane, perform the following actions:

1. Choose your property group that you created (green-app-bp), hover and choose Publish.

2. Click OK.

3. Choose Aministration > Catalog Management > Catalog Items.

Step 24

In the Catalog Items pane, perform the following actions.

1. Find and choose the blueprint that you created (Green App Tier).

Step 25

In the Configure Catalog Item pane, perform the following actions.

1. In the Details tab, in the Service field, choose VM Services.

2. Check the check box for New and noteworthy.

3. Click Update.

You now have deployed a single-tier application using property groups.

Step 26

To verify the deployment of the single-tier application, log out of the administrator session and log back in as the tenant.

1. Click the Catalog tab.

2. In the navigation pane, choose VM Services.

3. In the Work pane, choose the blueprint you created.

4. In the Catalog Item Details pane, verify the properties of the blueprint and click Request.

5. In the New Request pane, click Submit and then OK.

This provisions a new virtual machine, ACI networking, and connects the two together.

Deploying a 3-Tier Application Using a Multi-Machine Blueprint

VMware vRealize multi-machine blueprints are groupings of one or more machine blueprints to be deployed simultaneously. A common use case is a three-tier web application, where the web, app, and database tiers are deployed together. From a networking perspective, you must push the application policy into Cisco Application Centric Infrastructure (ACI) to enable secure communication between tiers that need to communicate. This is achieved by creating a security policy and associating the relevant machines dynamically at deployment time.

When configuring a blueprint that will be used in a multi-machine blueprint, a security policy must be created. During the creation process, the consumer and provider must be provided. The provider is always the machine that you are building, and the consumer can be any other machine or network.

As an example, say that you have a MySQL database machine blueprint that provides a service on port 3306. The application tier machines need to access this database, but the web tier machines do not. Under the Security Policy section of the Configure Property Group workflow, you create a policy with the "app" tier as the consumer, listing port 3306 as permissible (everything else is denied by default) and the blueprint will automatically place the "db" tier as the provider.

The "app" tier also must provide a service; in this example a server is listening on port 8000. The web tier will then consume this service. The security policy must be specified in the "app" tier property group.

Note

Machine prefixes generate a unique name for each virtual machine that is deployed. An example prefix for a tenant named "Green" could be "green-web-", plus three unique digits for each machine. The sequence would be: "green-web-001", "green-web-002", "green-web-003", and so on. It is important that you follow a similar scheme with your machine prefixes so that the Application Policy Infrastructure Controller (APIC) plug-in can accurately predict the name of the consumer endpoint group. Additionally, every machine must be on the same prefix number. For example, the names for a 3-tier app must be: green-db-001, green-app-001, and green-web-001. If any tier were not aligned, the security policy would fail to form a correct relationship. This is a requirement because vRealize does not provide the name of the sibling tiers, so the plug-in must infer the siblings' names based on its own name.

When configuring a security policy under a property group, the consumer name should be the second word of the machine prefix. For the example prefix "green-web-", the consumer name would be "web".

This section describes how to deploy a 3-tier application using a multi-machine blueprint.

Procedure

Step 1

Connect to the vRealize Automation appliance by pointing your browser to the following URL:

https://appliance_address/vcac/org/tenant_id

Step 2

Enter the tenant administrator username and password.

Step 3

Choose Catalog.

Step 4

Click Configure Property Group.

You will configure the database tier.

Step 5

Click Request.

Step 6

In the Request Information tab, enter a description of the request.

Step 7

Click Next.

Step 8

In the Common tab, perform the following actions:

1. In the IaaS Host for vRealize field, click Add.

2. Put a check in the box next to the desired IaaS host.

3. Click Submit.

4. In the APIC Tenant field, click Add.

5. Expand apic_name > Tenants.

6. Put a check in the box next to the desired tenant's name.

Example:

green

7. Click Submit.

8. In the Property Group Name field, enter a name for the property group.

Example:

green-db-mm

9. In the VMM Domain/DVS field, click Add.

10. Expand apic_name > Vcenters > vcenter_name

11. Put a check in the box next to the desired vCenter's name.

Example:

green

12. Click Submit.

Step 9

Click Next.

Step 10

In the VM Networking tab, leave all of the fields at their default values.

Step 11

Click Next.

Step 12

In the Security tab, perform the following actions:

1. In the Configure Security Policy drop-down list, choose Yes.

2. In the Consumer Network/EPG Name of Security Policy field, enter the name of the consumer network, without the full machine prefix.

Example:

app

The database tier must have the application tier as the consumer.

3. In the Starting Port Number in Security Policy field, enter the starting port number.

Example:

3306

4. In the Ending Port Number in Security Policy field, enter the ending port number.

Example:

3306

5. For the other fields, leave their values at the defaults.

Step 13

Click Next.

Step 14

In the Load Balancer tab, leave the field at its default value.

Step 15

Click Next.

Step 16

In the Firewall tab, leave the field at its default value.

Step 17

Click Submit.

Step 18

Click OK.

Step 19

Click Configure Property Group.

This time, you will configure the application tier.

Step 20

Click Request.

Step 21

In the Request Information tab, enter a description of the request.

Step 22

Click Next.

Step 23

In the Common tab, perform the following actions:

1. In the IaaS Host for vRealize field, click Add.

2. Put a check in the box next to the desired IaaS host.

3. Click Submit.

4. In the APIC Tenant field, click Add.

5. Expand apic_name > Tenants.

6. Put a check in the box next to the desired tenant's name.

Example:

green

7. Click Submit.

8. In the Property Group Name field, enter a name for the property group.

Example:

green-app-mm

9. In the VMM Domain/DVS field, click Add.

10. Expand apic_name > Vcenters > vcenter_name

11. Put a check in the box next to the desired vCenter's name.

Example:

green

12. Click Submit.

Step 24

Click Next.

Step 25

In the VM Networking tab, leave all of the fields at their default values.

Step 26

Click Next.

Step 27

In the Security tab, perform the following actions:

1. In the Configure Security Policy drop-down list, choose Yes.

2. In the Consumer Network/EPG Name of Security Policy field, enter the name of the consumer network, without the full machine prefix.

Example:

web

The application tier must have the web tier as the consumer.

3. In the Starting Port Number in Security Policy field, enter the starting port number.

Example:

8000

4. In the Ending Port Number in Security Policy field, enter the ending port number.

Example:

8000

5. For the other fields, leave their values at the defaults.

Step 28

Click Next.

Step 29

In the Load Balancer tab, leave the field at its default value.

Step 30

Click Next.

Step 31

In the Firewall tab, leave the field at its default value.

Step 32

Click Submit.

Step 33

Click OK.

Step 34

Click Configure Property Group.

You will configure the web tier.

Step 35

Click Request.

Step 36

In the Request Information tab, enter a description of the request.

Step 37

Click Next.

Step 38

In the Common tab, perform the following actions:

1. In the IaaS Host for vRealize field, click Add.

2. Put a check in the box next to the desired IaaS host.

3. Click Submit.

4. In the APIC Tenant field, click Add.

5. Expand apic_name > Tenants.

6. Put a check in the box next to the desired tenant's name.

Example:

green

7. Click Submit.

8. In the Property Group Name field, enter a name for the property group.

Example:

green-web-mm

9. In the VMM Domain/DVS field, click Add.

10. Expand apic_name > Vcenters > vcenter_name

11. Put a check in the box next to the desired vCenter's name.

Example:

green

12. Click Submit.

Step 39

Click Next.

Step 40

In the VM Networking tab, leave all of the fields at their default values.

Step 41

Click Next.

Step 42

In the Security tab, leave the field at its default value.

Because this is a consumer policy, you do not need to configure the security policy.

Step 43

Click Next.

Step 44

In the Load Balancer tab, leave the field at its default value.

Step 45

Click Next.

Step 46

In the Firewall tab, leave the field at its default value.

Step 47

Click Submit.

Step 48

Click OK.

About Plan Types

The administrator creates the plan with their own values. The plan types are as follows:

Shared Infrastructure

Virtual Private Cloud (VPC)

Isolated Networks

Yes

Yes

Firewall

Yes

Yes

Provider DHCP

Yes

Yes

Shared Load Balancer

Yes

Yes

Public Internet Access

Yes

Yes

Shared Services between Tenants

Yes

Yes

Bring your own address space (Private Address Space) and DHCP Server

No

Yes

About vRealize Service Categories and Catalog Items

This section describes the vRealize services categories and catalog items. The list of all catalog items they are grouped into services and each of these services are assigned an entitlement. ACI entitlement is assigned to certain users.

For more information, see ACI Administrator Services in vRealize.

For more information, see ACI Tenant Services in vRealize.

For more information, see Entitlements for ACI catalog-items in vRealize.

- Mapping of the ACI Plan Types to vRealize Service Categories

Mapping of the ACI Plan Types to vRealize Service Categories

This section shows the mapping of the Cisco ACI plan types to vRealize service categories.

vRA - User, Entitlements, Services and Blueprints

vRA Catalog Category

List of Blueprints

Admin service blueprints

Add APIC with Admin credentials
Add APIC with Tenant credentials
Add Provider for Shared Service (Contract)
Add or Update Tenant
Add VIP Pool
Add VMM Domain, AVS Local Switching with Vlan Encap
Add VMM Domain, AVS Local Switching with Vxlan Encap
Add VMM Domain, AVS No Local Switching
Add VMM Domain, AVE Local Switching with Vlan Encap
Add VMM Domain, AVE Local Switching with Vxlan Encap
Add VMM Domain, AVE No Local Switching
Add VMM Domain, DVS and Vlan Pool
Add or Delete Bridge Domain in Tenant-common
Add or Delete Consumer for Shared Service (Contract)
Add or Delete L3 context (VRF) in Tenant-common
Add or Delete Router Id
Add or Delete Subnets in Bridge Domain for Tenant-Common
Update FW Policy (DFW) association to AVS or AVE VMM Domain
Configure Property Group
Create FW Policy (DFW) and Associate to AVS or AVE VMM Domain
Delete APIC
Delete FW Policy (DFW)
Delete Provider Shared Service (Contract)
Delete Tenant
Delete VIP Pool
Delete VMM Domain, AVS or AVE, and VLAN, Multicast Pool
Delete VMM Domain, DVS and Vlan Pool
Generate and Add Certificate to APIC
Rest API
Update FW Policy (DFW) AVS or AVE
Update Vlan Pool, AVS or AVE
Update Multicast Pool, AVS
Update VMM Domain DVS security domain mapping
Update AVS or AVE VMM Domain Security Domain Mapping

Tenant Shared Plan service blueprints

Add a Useg Network - Shared Plan
Add FW and LB to Tenant Network - Shared Plan
Add FW to Tenant Network - Shared Plan
Add Loadbalancer to Tenant Network - Shared plan
Add Tenant Network - Shared plan
Delete a Useg Network - Shared Plan
Delete FW and LB from Tenant Network - Shared Plan
Delete FW from Tenant Network - Shared Plan
Delete Loadbalancer from Tenant Network - Shared Plan
Delete Tenant Network - Shared plan

Tenant VPC Plan service blueprints

Add a Useg Network - VPC Plan
Add FW and LB to Tenant Network - VPC Plan
Add FW to Tenant Network - VPC Plan
Add Loadbalancer to Tenant Network - VPC plan
Add Tenant Network - VPC plan
Delete a Useg Network - VPC Plan
Delete FW and LB from Tenant Network - VPC Plan
Delete Loadbalancer from Tenant Network - VPC Plan
Delete Tenant Network - VPC  plan

Network Security service blueprints

Add Security Policy (Contracts)
Delete Security Policy (Contracts)
Update Access List Security Rules

Tenant Network Service blueprints

Add or Delete Bridge domain in Tenant
Add or Delete L3 Context (VRF) in Tenant
Add or Delete Subnets in Bridge domain
Add or Delete Useg Attribute
Attach or Detach L3 external connectivity to Network
Update Tenant Network

ACI Administrator Services in vRealize

This section describes the ACI Administrator Services in vRealize.

- List of Admin Services Catalog Items for ACI Administrator Services

List of Admin Services Catalog Items for ACI Administrator Services

This section provides a list of the admin services catalog items for ACI administrator services.

Catalog Item

Description

Add APIC with Tenant Credentials

This creates the Application Policy Infrastructure Controller (APIC) handle with tenant credentials.

Add APIC with Admin Credentials

This creates the APIC handle with Admin credentials.

Add or Delete Bridge Domain in Tenant-common

This adds or deletes the bridge domain in tenant-common.

Add or Delete Consumer for Shared Service (Contract)

This adds or deletes consumer for shared service (Contract).

Add or Delete L3 context (VRF) in Tenant-common

This adds or deletes Layer 3 context (VRF) in tenant-common.

Add or Delete Subnets in Bridge Domain for Tenant-Common

This adds or deletes subnets in the bridge domain for tenant-common.

Add Provider for Shared Service (Contract)

This adds provider for shared service (Contract).

Add or Delete Router Id

This adds or deletes the router Id.

Add or Update Tenant

This adds or updates a tenant.

If the tenant wants to use the Firewall between EPGs, set "Enable inter-EPG Firewall" to Yes. Also the number application tiers should be set. To use typical 3-tier web, app, db application the number of tiers should be set to 3.

Add VIP Pool

This adds the Virtual IP Pool.

Configure Property Group

This configures the property group.

Delete APIC

This deletes the APIC.

Delete Provider Shared Service (Contract)

This deletes the provider shared service (Contract).

Delete Tenant

This deletes a tenant.

Delete VIP Pool

This deletes the Virtual IP Pool.

Generate and Add Certificate to APIC

This blueprints can be used to generate a certificate for a given user. This certificate then be used in the certificate based access to APIC.

REST API

This is the REST API.

This section provides a list of the admin services catalog items for ACI administrator services for the VMM domain type DVS.

Catalog Item

Description

Add VMM Domain, DVS and VLAN Pool

This adds VMM Domain, DVS, and VLAN Pool.

Ensure all hosts of the data-center that has the APIC created DVS in vCenter, must have at least one physical NIC attached. This ensures that the port-groups of the DVS are available for virtual NIC placements.

Delete VMM Domain, DVS, and VLAN Pool

This deletes the VMM Domain, DVS and VLAN Pool.

Update Vlan Pool (encap blocks)

This updates the Vlan Pool (encap blocks).

Update VMM Domain DVS security domain mapping

This updates the VMM Domain DVS security domain mapping.

This section provides a list of the admin services catalog items for ACI administrator services for the VMM domain type Cisco AVS or Cisco ACI Virtual Edge (AVE).

Catalog Item

Description

Add VMM Domain, AVS or AVE Local Switching with Vlan Encap

This creates a VMM domain in Cisco APIC with VLAN as the default encapsulation mode. It also creates a VLAN pool and multicast address pool (in the case of mixed mode). This item also creates an associated Cisco AVS or Cisco ACI Virtual Edge with local switching in vCenter.

Add VMM Domain, AVS or AVE Local Switching with Vxlan Encap

This creates a VMM domain in Cisco APIC with VXLAN as the default encapsulation mode. It also creates a multicast address pool and VLAN pool (in the case of mixed mode). This item also creates an associated Cisco AVS or Cisco ACI Virtual Edge with local switching in vCenter.

Add VMM Domain, AVS or AVE No Local Switching

This adds VMM domain, multicast address pool in Cisco APIC and creates an associated Cisco AVS or Cisco ACI Virtual Edge with no local switching in vCenter.

Update Multicast Pool, AVS or AVE

This updates the multicast pool for Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Update VLAN Pool, AVS or AVE

This updates the VLAN pool for the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Update AVS or AVE VMM Domain Security Domain Mapping

This updates the security domain mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Delete VMM Domain AVS or AVE, Vlan, Multicast Pool

This deletes the Cisco AVS or Cisco ACI Virtual Edge VMM Domain and VLAN Pools and Multicast Pool in Cisco APIC and deletes the associated Cisco AVS or Cisco ACI Virtual Edge in vCenter.

Create FW Policy (DFW) and Associate to AVS or AVE VMM Domain

This creates a Distributed Firewall policy and associates it to the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Update FW Policy (DFW) association to AVS or AVE VMM Domain

This associates/dissociates an existing Distributed Firewall policy to the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Update FW Policy (DFW)

This updates the existing Distributed Firewall Policy.

Delete FW Policy (DFW)

This deletes the existing Distributed Firewall Policy.

To submit a request:

1. Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

ACI Tenant Services in vRealize

This section describes the ACI tenant services in the vRealize.

- List of Network Security Catalog Items for ACI Tenant Services

- List of Tenant Network Services Catalog Items for ACI Tenant Services

- List of Tenant Shared Plan Catalog Items for ACI Tenant Services

- List of Tenant VPC Plan Catalog Items for ACI Tenant Services

- List of VM Services Catalog Items for ACI Tenant Services

List of Network Security Catalog Items for ACI Tenant Services

This section provides a list of the Network Security catalog items for ACI tenant services.

Catalog Item

Description

Add Security Policy (Contracts)

This creates the security policy between tenant networks. For example: APIC contracts between consumer EPG and provider EPG.

Delete Security Policy (Contracts)

This deletes the security policy between tenant networks. For example: APIC contracts between consumer EPG and provider EPG.

Update Access List Security Rules

This adds or removes access list rules associated with a Security Policy Filter created in APIC (using Add Security Policy (Contracts)). The access list rules are of the format <source-port, destination-port, protocol, ethertype>.

Note

The Source and Dest Ports are not allowed for arp, icmp, icmpv6 rules. Ports are valid only for tcp and udp protocols. The access list rules are deployed and enforced in ACI fabric and they are stateless in nature.

In addition this blueprint also has an option to update the stateful firewall rules on a Firewall appliance such as Cisco-ASA for a specific service graph that is provided as an input.

To submit a request:

1. Log in to the vRealize Automation as admin, choose Catalog > Network Security.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

List of Tenant Network Services Catalog Items for ACI Tenant Services

The following table lists the Tenant Network Services catalog items for ACI tenant services. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant Network Services catalog items.

Catalog Item

Description

Add or Delete Bridge Domain in Tenant

This adds or deletes the bridge domain in tenant.

Add or Delete L3 Context (VRF) in Tenant

This adds or deletes Layer 3 context (VRF) in tenant.

Add or Delete Subnets in Bridge domain

This adds or deletes subnets in the bridge domain.

Attach or Detach L3 external connectivity to Network

This attaches or detaches Layer 3 external connectivity to the network.

Update Tenant Network

This updates the tenant network.

The following table lists the Tenant Network Services catalog items for VMM domain of type Cisco AVS and Cisco ACI Virtual Edge only. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant Network Services catalog items.

Catalog Item

Description

Add or Delete Useg Attribute

This adds or deletes an attribute for a microsegment EPG.

To submit a request:

1. Log in to the vRealize Automation as tenant admin, choose Catalog > Tenant Network Services.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

List of Tenant Shared Plan Catalog Items for ACI Tenant Services

The following table lists the Tenant Shared Plan catalog items for ACI tenant services. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant Shared Plan catalog items.

Catalog Items

Description

Add Tenant Network

This adds the tenant network in a shared plan.

Add FW and LB to Tenant Network - Shared Plan

This adds a firewall and load balancer to the tenant network in a shared plan.

Add FW to Tenant Network - Shared Plan

This adds a firewall to the tenant network in a shared plan.

Add Load Balancer to Tenant Network - Shared Plan

This adds load balancer to the tenant network in a shared plan.

Delete FW and LB from Tenant Network - Shared Plan

This deletes the firewall and load balancer from the tenant network in a shared plan.

Delete FW from Tenant Network - Shared Plan

This deletes the firewall from the tenant network in a shared plan.

Delete Load Balancer from Tenant Network - Shared Plan

This deletes load balancer from the tenant network in a shared plan.

Delete Tenant Network - Shared Plan

This deletes the tenant network in a shared plan.

The following table lists the Tenant Shared Plan catalog items for VMM domain of type Cisco AVS only. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant Shared Plan catalog items.

Catalog Item

Description

Add a Useg Network - Shared Plan

This adds a microsegment EPG in a shared plan.

Delete a Useg network - Shared Plan

This deletes a microsegment EPG in a shared plan.

To submit a request:

1. Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

Note

Symptom: You might see errors in the VMware vCenter during the deletion of the service graph through the vRealize Automation (vRA) workflow.

Condition: During the deletion of the service graph, if a port group is deleted before service devices such as VPX or F5 are configured, then these errors are seen. This sequence cannot be controlled through vRA.

Workaround: There is no workaround. These errors are transitory and will stop once the reconfiguration of the service devices is done.

List of Tenant VPC Plan Catalog Items for ACI Tenant Services

The following table lists the Tenant Virtual Private Cloud (VPC) Plan catalog items for ACI tenant services. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant VPC Plan catalog items

Catalog Item

Description

Add Tenant Network - VPC Plan

This adds the tenant network in a VPC plan.

Add FW and LB to Tenant Network - VPC Plan

This adds the firewall and load balancer to the tenant network in a VPC plan.

Add FW to Tenant Network - VPC Plan

This adds the firewall to the tenant network in a VPC plan.

Add Load-balancer to Tenant Network - VPC Plan

This adds the load balancer to tenant network in a VPC plan.

Delete FW and LB from Tenant Network - VPC Plan

This deletes the firewall and load balancer from tenant network in a VPC plan.

Delete Load-balancer from Tenant Network - VPC Plan

This deletes load balancer from tenant network in a VPC plan.

Delete Tenant Network - VPC Plan

This deletes the tenant network in a VPC plan.

The following table lists the Tenant VPC Plan catalog items for VMM domain of type Cisco AVS or Cisco ACI Virtual Edge only. You must log in to the tenant portal with tenant administrator privileges to execute the Tenant VPC Plan catalog items.

Catalog Item

Description

Add a Useg Network - VPC plan

This adds a microsegment EPG in a VPC plan.

Delete a Useg Network - VPC plan

This deletes a microsegment EPG in a VPC plan.

To submit a request:

1. Log in to the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

List of VM Services Catalog Items for ACI Tenant Services

This section provides a list of the VM services catalog items for ACI tenant services.

This service category has the tenant catalog items based on single machine and multi-machine blueprints. For example, for typical three tier application, it contains 3 catalog items "Web", "App", "Db" using single-machine blueprints and 1 catalog item "Web-App-Db" using multi-machine blueprint.

Catalog Item

Description

App

This is the application VM.

Db

This is the database VM.

Test

This is the single-machine VM blueprint for testing property groups.

Web

This is the web VM.

Web-Db-App

This multi-machine blueprint creates a 3-tier application, load balancer attached to the Web tier and the security policy configuration.

To submit a request:

1. Log in to the vRealize Automation as admin, choose Catalog > VM Services.

2. Choose a request, enter the information in the fields and click Submit.

To view your request:

1. In the vRealize Automation GUI, choose Requests.

2. Choose the request you submitted and click view details.

Entitlements for ACI catalog-items in vRealize

This section describes the entitlements for ACI catalog-items in vRealize. Each service category must have an entitlement. Entitlement enables the catalog items to be available for the users.

You can create and manage entitlements to control the access to the catalog items, actions, and specify the approval policies to apply the catalog requests. You can update the priority of the entitlement to determine which approval policy applies to a particular request.

- List of Entitlements for ACI Catalog Items

List of Entitlements for ACI Catalog Items

This section provides a list of the entitlements for ACI catalog items.

Name

VMs Entitlements

Admin Entitlements

Tenant Shared Plan Entitlements

Tenant VPC Plan Entitlements

Common Network Services Entitlements

Tenant Network Services Entitlements

Tenant-common Network Services

Network Security Entitlements

To edit an entitlement:

1. Log in to the vRealize Automation as admin, choose Administration > Catalog Management > Entitlements.

2. Choose an entitlement to edit, enter the information in the fields and click Update.

ACI Plug-in in vRealize Orchestrator

The service category and the catalog item maps to a workflow.

- APIC Workflows

- APIC Inventory View

APIC Workflows

These are the service categories and the catalog items and each catalog items is implemented as a workflow in the vRealize Orchestrator and the catalog items parameter are exactly same as the workflow parameters.

Service Categories

Description

Admin Services

Admin catalog-items to be executed by the global administrator

Network Security

Catalog-items for configuring security policies

Tenant Network Services

For configuring network services (bridge-domain, subnets)

Tenant Shared Plan

For configuring EPG/networks, microsegment EPGs, consuming load balancer, and firewall services in shared mode

Tenant VPC Plan

For configuring EPG/networks, microsegment EPGs, consuming load balancer, and firewall services in VPC mode

VM Services

Single-machine and multi-machine blueprints configured with ACI property groups

APIC Inventory View

In the Inventory view of the vRealize Orchestrator GUI, the Cisco APIC Plugin is a read only view. The Cisco APIC Plugin for vRealize Orchestrator maps to the APIC. For example, if you look at an object in the vRealize Orchestrator GUI it provides the MultiApicDn in the Cisco APIC GUI.

About Load Balancing and Firewall Services

VLAN, virtual routing and forwarding (VRF) stitching is supported by traditional service insertion models, the Application Policy Infrastructure Controller (APIC) can automate service insertion while acting as a central point of policy control. The APIC policies manage both the network fabric and services appliances. The APIC can configure the network automatically so that traffic flows through the services. The APIC can also automatically configure the service according to the application's requirements, which allows organizations to automate service insertion and eliminate the challenge of managing the complex techniques of traditional service insertion.

Perimeter Firewall is typically used to provide state-full firewall services for all incoming external traffic to the application. Once the traffic passes the firewall, another typical service that is inserted is the load balancing. The external traffic is sent towards, a virtual IP. The load balancer terminates this traffic and load balances the incoming traffic among the available servers (such as web servers) behind the load balancers.

See the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide for more information.

APIC vRealize plug-in can be used to create new multi-tier applications while inserting the load balancer and/or firewall services for the traffic between them or it can be used to insert the firewall and load-balancer services for traffic between existing application end-point groups. For creating a multi-tier application with L4-7 services, a property group has to be created using "Configure Property Group" catalog-item in the "Admin Services". In addition of L4-7 services between existing application end-point groups can be done by choosing the appropriate catalog-item from the "Tenant Shared Services" items.

Note

In this release, only support for Shared-Plan is supported for Load balancer and Firewall services.

- Prerequisites for Enabling Services

- Configuring the Services on APIC Using XML POST

- Deleting the Services Configuration

Prerequisites for Enabling Services

This section describes the prerequisites for enabling services.

You must perform the following tasks to deploy Layer 4 to Layer 7 services using the APIC vRealize plug-in:

- Device package for load balancer needs to be uploaded by APIC admin.

Use the link to download the required Citrix, F5, and Cisco ASA device packages:

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-734587.html

Ensure the device package version is certified for the APIC release that you are using.

- Device cluster for load balancer, firewall needs to be created in tenant "common" by APIC-admin. Citrix and F5 are the supported vendors for load balancers. Cisco ASA is the supported vendor for firewall.

- For stand-alone firewall or load balancer service, a service graph template with single node must be configured. For the firewall and load balancer service, a service graph template with two nodes must be configured.

- For the abstract service graph, the firewall node (vnsAbsNode) must be named FW, and the load balancer node must be named SLB.

- For the load balancer only abstract service graph name (vnsAbsGraph) should be same as the load balancer device cluster (vnsLdevVip).

- For the load balancer only service, the consumer L3 connectivity policy must be configured in the "default" VRF of the tenant common.

- For the firewall, the consumer L3 connectivity policy must be configured in the separate VRF ("outside") of the tenant common.

- The firewall device needs to be deployed in the routed mode. For firewall device connectivity, two additional L3 connectivity policy must be configured. One must be configured in the "outside" VRF, and is used as the external connection to the firewall device. The other must be configured in the "default" VRF and is used as the internal connection to the firewall device. These two L3 connectivity policies, attached to the firewall enables the firewall to do the VRF stitching and re-direct the traffic appropriately between the VRFs. The administrator has to ensure that appropriate prefixes with the correct import and export flags are configured under the L3 external connectivity policies.

- The following convention should be used when configuring the L3 connectivity policies. For the L3 connectivity policy should be named as L3ExtName , the child L3 instance should be named as L3ExtNameInst.

- The interface IP addresses that are used on the firewall and load balancer devices need to be configured in the abstract graph.

- For the 2-node abstract graph, an access list to permit all traffic needs to be configured for the firewall node.

Configuring the Services on APIC Using XML POST

Only the administrator can configure and post the XML POST. The template POSTs are located in the apic-vrealize package under the services directory.

Before you begin

- The device package file should be uploaded on the Application Policy Infrastructure Controller (APIC).

See the Cisco APIC Layer 4 to Layer 7 Device Package Development Guide for more information.

- The tenant common should have the two bridge domains named "default" and "vpcDefault". Ensure that the subnets being used by the tenant who is consuming the load balancer are added to these bridge domains. Typically you would have created these bridge domains and subnets while setting up the DHCP infrastructure for vRealize tenants.

- For a non-Virtual Private Cloud (VPC) plan, the backend interface of the load balancer should be placed in the default EPG under the tenant common that was created above. For a VPC plan, the EPG should be "vpcDefault".

- Ensure that the VIP subnet is linked with L3. One VIP per EPG will be allocated from the VIP pool associated with the tenant.

- Prerequisites for the service scripts:

- Python 2.7

- Python libraries:

- jinja2

- yaml

- glob

- json

- requests

- xml

- re

Procedure

Step 1

Use the following link to download the required device packages Citrix, F5, and ASA. Ensure that the device package version is certified for the APIC release that you are using. Store the device package zip files in this directory:

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-734587.html

Step 2

Replace the VENDOR-DEVICE-PACKAGE.zip entries in the shared.cfg or vpc.cfg file with the correct device package files.

Step 3

Edit the setup.yaml file and change the variables to according to your setup.

The template variables in the setup.yaml file are:

TEMPLATE_VARS:
VCENTER: "vcenter1"
ASA_IP: "1.1.1.1"
ASA_CLUSTER: "AsaCluster1"
ASA_VM: "asav-service5"
OUTSIDE_CTX: "outside"
INSIDE_CTX: "default"
FW_GRAPH: "FWOnlyGraph"
FW_SLB_GRAPH: "FWAndSLBGraph"
BD_WEB: "default"
CITRIX_MGMT_IP: "1.1.1.1"
FW_NODE: "FW"
SLB_NODE: "SLB"
CITRIX_GRAPH: "CitrixCluster1_L3"
CITRIX_CLUSTER: "CitrixCluster1_L3"
CITRIX_GRAPH: "CitrixCluster1_L3"
CITRIX_VM: "NS-service4"
F5_BD: "F5Cluster1_L3"
F5_EPG: "F5Cluster1_L3"
F5_CLUSTER: "F5Cluster1_L3"
F5_MGMT_IP: "1.1.1.1"
F5_GRAPH: "F5Cluster1_L3"
F5_ABS_NODE: "SLB"
# Use deleted to generate the "deleted" version of the posts
# STATUS: "deleted"
STATUS: ""

Step 4

Enter the following commands:

For Shared Plan:

Example:

../jinja.py setup.yaml tn-common-template.xml > tn-common.xml
../jinja.py setup.yaml Shared-Plan-Citrix-graph-template.xml > Shared-Plan-Citrix-graph.xml
../jinja.py setup.yaml Shared-Plan-F5-graph-template.xml > Shared-Plan-F5-graph.xml

For VPC Plan:

Example:

../jinja.py setup.yaml VPC-tn-common-template.xml > VPC-tn-common.xml
../jinja.py setup.yaml VPC-Plan-Citrix-LB-graph-template.xml > VPC-Plan-Citrix-LB-graph.xml
../jinja.py setup.yaml VPC-Plan-F5-LB-graph-template.xml > VPC-Plan-F5-LB-graph.xml

If you see python errors, ensure that the prerequisite python libraries are installed in the system.

Step 5

Edit the shared.cfg or vpc.cfg file and set the values for hosts: <YOUR_APIC_IP> and passwd: <YOUR_APIC_ADMIN_PASSWD>.

Sample of the shared.cfg file:

Example:

host:   <YOUR_APIC_IP>:443
name:   admin
passwd: <YOUR_APIC_ADMIN_PASSWD>
tests:
- type: file
path: /ppi/node/mo/.xml
#      file: asa-device-pkg-1.2.2.1.zip
#      Replace actual ASA Device package file in the line below
file: ASA-DEVICE-PACKAGE.zip
wait: 2
- type: file
path: /ppi/node/mo/.xml
#      file: CitrixNetscalerPackage.zip
#      Replace actual Citrix Device package file in the line below
file: CITRIX-DEVICE-PACKAGE.zip
wait: 2
- type: file
path: /ppi/node/mo/.xml
#      file: CitrixNetscalerPackage.zip
#      Replace actual F5 Device package file in the line below
file: F5-DEVICE-PACKAGE.zip
wait: 2
- type: xml
path:  /api/node/mo/.xml
file: tn-common.xml
wait: 0
- type: xml
path:  /api/node/mo/.xml
file: Shared-Plan-Citrix-graph.xml
wait: 0
- type: xml
path:  /api/node/mo/.xml
file: Shared-Plan-F5-graph.xml
wait: 0

Step 6

Post the templates.

For Shared Plan, enter the following command:

Example:

../request.py shared.cfg

For VPC Plan, enter the following command:

Example:

../request.py vpc.cfg

Deleting the Services Configuration

This section describes how to delete the services configuration. Only the administrator can configure and post the XML POST. The template POSTs are located in the apic-vrealize package under the services directory.

Procedure

Step 1

Edit the shared.cfg file and set the values for hosts: <YOUR_APIC_IP> and passwd: <YOUR_APIC_ADMIN_PASSWD>.

Step 2

Edit the setup.yaml file and set the STATUS variable to deleted to generate the deleted version of the posts.

Step 3

Run the following commands:

./jinja.py setup.yaml tn-common-template.xml > tn-common-del.xml
./jinja.py setup.yaml Shared-Plan-Citrix-graph-template.xml > Shared-Plan-Citrix-graph-del.xml
./jinja.py setup.yaml Shared-Plan-F5-graph-template.xml > Shared-Plan-F5-graph-del.xml

Step 4

Post the templates:

./request.py shared_del.cfg

About L3 External Connectivity

Layer 3 (L3) external connectivity is an Cisco Application Centric Infrastructure (ACI) feature to connect ACI fabric to an external network by L3 routing protocols, including static routing, OSPF, EIGRP, and BGP. By setting up L3 external connectivity for vRealize, it allows a tenant network to initiate outgoing traffic destined outside the fabric and to attract traffic from outside. The assumption of this feature is the tenant virtual machine IP addresses are visible outside the fabric without NAT, ACI L3 external connectivity does not include NAT.

- Prerequisites for Configuring L3 External Connectivity for vRealize

Prerequisites for Configuring L3 External Connectivity for vRealize

To configure Layer 3 (L3) external connectivity for vRealize, you must meet the following prerequisites:

- Ensure you have logged in to the Application Policy Infrastructure Controller (APIC) GUI, on the menu bar, choose TENANT > common.

- Create a l3ExtOut called “default”, refer to BD “default”.

- Create l3extInstP name="defaultInstP" under the l3ExtOut. This is to be used by shared service tenants.

See Cisco APIC Basic Configuration Guide for L3 external connectivity configuration.

- Ensure you have logged in to the APIC GUI, on the menu bar, choose TENANT > common.

- Create a l3ExtOut called "vpcDefault", refer to BD "vpcDefault".

- Create l3extInstP name="vpcDefaultInstP" under this l3ExtOut.

This is to be used by VPC tenants.

See Cisco APIC Basic Configuration Guide for configuring external connectivity for tenants.

vRealize leverages the common l3ExtOut configuration with no special requirement other than the naming convention highlighted above

Administrator Experiences

- Cisco ACI with Cisco AVS or Cisco ACI Virtual Edge

- Cisco AVS or Cisco ACI Virtual Edge VMM Domain Creation

- Update of Cisco AVS or Cisco ACI Virtual Edge VMM Domain Encapsulation Pools

- Deletion of Cisco AVS or Cisco ACI Virtual Edge and the VMM Domain

- Cisco AVS or Cisco ACI Virtual Edge VMM Domain Security Domain Mapping

- Distributed Firewall Policy

Cisco ACI with Cisco AVS or Cisco ACI Virtual Edge

See the following documentation for general information about Cisco Application Virtual Switch (AVS) or Cisco ACI Virtual Edge:

- Cisco AVS—See the chapter "Cisco ACI with Cisco AVS" in the latest version of the Cisco ACI Virtualization Guide or the Cisco AVS guides on Cisco.com

- Cisco ACI Virtual Edge—See the Cisco ACI Virtual Edge documentation on Cisco.com.

Cisco AVS or Cisco ACI Virtual Edge VMM Domain Creation

You can create VMM domains for Cisco AVS or Cisco ACI Virtual Edge using VLAN or VXLAN encapsulation or with no local switching.

Beginning with Cisco APIC Release 2.1(1), you can mix encapsulation modes. That is, you can configure a VMM domain to use VLAN or VXLAN and later add EPGs that override the domain's default encapsulation. For details, see the section "Mixed-Mode Encapsulation Configuration" in the Cisco Application Virtual Switch Configuration Guide or the chapter "Mixed-Mode Encapsulation" in the Cisco ACI Virtual Edge Configuration Guide.

You also can create a Cisco AVS or Cisco ACI Virtual Edge VMM domain with no local switching. In local switching mode, the leaf forwards all traffic, and VXLAN is the only allowed encapsulation type. See the Cisco Application Virtual Switch Installation Guide or the Cisco ACI Virtual Edge Installation Guide.

After you create a Cisco AVS or Cisco ACI Virtual Edge VMM domain, you can update the domain's encapsulation pools and delete the Cisco AVS or Cisco ACI Virtual Edge and the VMM domain.

- Creating a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

- Verifying Cisco AVS or Cisco ACI Virtual Edge Creation in vCenter

- Verifying Creation of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain on Cisco APIC

Creating a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

This section shows how to create a Cisco AVS or a Cisco ACI Virtual Edge VMM Domain supporting no encapsulation, VLAN, or VXLAN encapsulation. When you choose the virtual switch (Cisco AVS or Cisco AVE) and the switching preference (Local Switching or No Local Switching), the vRealize GUI shows or hides mandatory or optional field inputs.

Note

Beginning with Cisco APIC Release 5.0(1), Cisco Application Virtual Switch (AVS) is no longer supported. If you use Cisco AVS and upgrade to Cisco APIC Release 5.0(1), in case of issues, the fabric will not be supported. Also, a fault will be raised for the Cisco AVS domain.

If you use Cisco AVS, we recommend that you migrate to Cisco ACI Virtual Edge. See the Cisco ACI Virtual Edge Installation Guide on Cisco.com.

Before you begin

We recommend that you created an attachable access entity profile (AAEP) as part of day-0 operation of Cisco ACI.

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Add VMM Domain and AVS or AVE.

Step 3

In the New Request dialog box, complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Domain name field, enter the VMM domain name.

4. For the Virtual Switch selector, choose Cisco AVS or Cisco AVE.

5. For the Switching Preference selector, choose Lo Local Switching or Local Switching.

6. If you chose Local Switching, for the Encap mode selector choose VLAN or VXLAN.

Encap mode is applicable only for Local Switching.

7. In the AAEP Name field, enter an attachable access entity profile (AAEP) name to associate it to the VMM domain.

If the AAEP that you enter doesn’t exist, it is created.

8. For the VLAN Ranges to be allocated, click Not set and then add values to create VLANs.

For Encap_Block_Role, specify external or internal.

9. (Optional) In the AVS Fabric-wide Multicast Address or AVE Fabric-wide Multicast Addressfield, enter a valid multicast address between 224.0.0.0 and 239.255.255.255, inclusive, for the multicast address block range.

10. (Optional) In the Multicast Address Start field, enter the starting multicast address between 224.0.0.0 and 239.255.255.255, inclusive, for the multicast address block range.

11. Optional) In the Multicast Address End field, between 224.0.0.0 and 239.255.255.255, inclusive, for the multicast address block range.

12. In the AAA Domain area, click the green cross, choose a security domain, and then click Next.

13. In the Vcenter IP (or Hostname) field, enter the host name or IP address.

If you use the host name, you already must have configured a DNS policy on Cisco APIC. If you do not have a DNS policy configured, enter the IP address of the vCenter server.

14. From the DVS Version drop-down list, choose the DVS version.

15. In the Username field, enter the user name for logging in to the vCenter.

16. In the Password field, enter the password for logging into the vCenter.

17. In the vCenter Datacenter field, enter the data center name.

Note

The name that you enter for the data center must match exactly the name in vCenter. The name is case-sensitive.

Verifying Cisco AVS or Cisco ACI Virtual Edge Creation in vCenter

Procedure

Step 1

Open a vSphere Client connection to a vCenter server.

Step 2

In vCenter, choose Home > Inventory > Networkingview.

Step 3

Choose the data center.

Step 4

Under the data center, ensure that the Cisco AVS or the Cisco ACI Virtual Edge and its folder are created.

Verifying Creation of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain on Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, choose VMM Domains > VMware.

Step 4

In the work pane, under Properties, in the vCenter Domains field, ensure that the newly created VMM domain is listed.

Update of Cisco AVS or Cisco ACI Virtual Edge VMM Domain Encapsulation Pools

After you create a Cisco AVS VMM or Cisco ACI Virtual Edge domain, you can update VLAN or multicast address pools. You should then verify the update.

- Updating the VLAN Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

- Verifying the Update of the VLAN Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain in Cisco APIC

- Updating the Multicast Address Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

- Verifying the Update of a Multicast Address Pool on Cisco APIC

Updating the VLAN Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Procedure

Step 1

Log in to the vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Update Vlan Pool, AVS or Update Vlan Pool, AVE.

Note

This update operation is only supported for dynamic VLAN pools. Static VLAN pools are not supported.

Step 3

View the Service Blueprint Information for the input fields and then click Request.

Step 4

In the New Request dialog box, complete the following steps:

1. Add the description and then click Next.

2. In the Vlan Pool Name field, enter the name of the existing VLAN pool.

3. In the List of encap blocks area, click the green cross next to New.

4. For each Encap block, in the VlanStart column, enter the starting VLAN.

5. In the VlanEnd column, enter the ending VLAN.

6. For encapRole, specify external or internal.

7. Tick the check box in IsAddoperation to add encap blocks to the VLAN pool.

Leave the check box unchecked to remove an entered encap block from a VLAN pool.

8. Click Submit.

What to do next

Complete the procedure Verifying the Update of the VLAN Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain in Cisco APIC.

Verifying the Update of the VLAN Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain in Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, expand the Pools folder.

Step 4

Expand the VLAN folder.

Step 5

Choose the VLAN pool.

Step 6

In the work pane, ensure that the VLAN pool is updated.

Updating the Multicast Address Pool of a Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Update Multicast Pool, AVS or AVE.

Step 3

View the Service Blueprint Information for the input fields and then click Request.

Step 4

In the New Request dialog box, complete the following steps:

1. In the Multicast Pool Name field, enter the name of the existing multicast address pool.

2. In the List of Multicast Address Range area, click the green cross next to New.

3. For each multicast address block, enter the starting multicast address between 224.0.0.0 and 239.255.255.255, inclusive, in the MulticastAddressStart column.

4. In the MulticastAddressEnd column, enter the ending multicast address between 224.0.0.0 and 239.255.255.255, inclusive.

5. Check the check box in the column IsAddOperation to add multicast address blocks to the multicast address pool.

Leave the check box unchecked to remove an entered multicast address block from the multicast address pool.

6. Click Submit.

What to do next

Complete the procedure Verifying the Update of a Multicast Address Pool on Cisco APIC.

Verifying the Update of a Multicast Address Pool on Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies.

Step 3

in the Policies navigation pane, expand the Pools folder.

Step 4

Expand the Multicast Address folder.

Step 5

Choose the multicast address pool.

Step 6

In the work pane, ensure that the multicast address pool is updated.

Deletion of Cisco AVS or Cisco ACI Virtual Edge and the VMM Domain

You can delete the Cisco AVS or Cisco ACI Virtual Edge and the VMM domain. After you do so, you should verify the deletion.

- Deleting the Cisco AVS or Cisco ACI Virtual Edge and the VMM Domain

- Verifying Cisco AVS or Cisco ACI Virtual Edge Deletion in vCenter

- Verifying VMM Domain Deletion on Cisco APIC

- Verifying VLAN Pool Deletion on Cisco APIC

- Verifying Multicast Address Pool Deletion on Cisco APIC

Deleting the Cisco AVS or Cisco ACI Virtual Edge and the VMM Domain

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Delete VMM Domain, AVS or AVE.

Step 3

View the Service Blueprint Information for the input fields and then click Request.

Step 4

In the New Request dialog box, complete the following steps:

1. Add a description and then click Next.

2. In the Domain name field, enter the name of the VMM domain that you want to delete.

Note

If the VMM domain has an associated multicast address pool (Domain/AVS or AVE name_mcastpool) or a VLAN pool (Domain/AVS or AVE name_vlanpool), it also will be deleted.

3. Click Submit.

What to do next

Complete the following procedures:

- Verifying Cisco AVS or Cisco ACI Virtual Edge Deletion in vCenter

- Verifying VMM Domain Deletion on Cisco APIC

- Verifying VLAN Pool Deletion on Cisco APIC

- Verifying Multicast Address Pool Deletion on Cisco APIC

Verifying Cisco AVS or Cisco ACI Virtual Edge Deletion in vCenter

Procedure

Step 1

Open a vSphere Client connection to a vCenter server.

Step 2

In vCenter, choose Home > Inventory > Networking view.

Step 3

Choose the data center.

Step 4

Under the data center, ensure that the Cisco AVS or Cisco ACI Virtual Edge and its folder are deleted.

Verifying VMM Domain Deletion on Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand the VMM Domains folder and the VMware folder.

Step 4

Under VMware, ensure that the deleted VMM domain is not present.

Verifying VLAN Pool Deletion on Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies

Step 3

In the Policies navigation pane, expand the Pools folder.

Step 4

Choose the VLAN folder.

Step 5

In the work pane, under Pools - VLAN, ensure that the VLAN pool ( Domain/AVS name_vlanpool) is deleted.

Verifying Multicast Address Pool Deletion on Cisco APIC

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, expand the Pools folder.

Step 4

Choose the Multicast Address folder.

Step 5

In the work pane, under Pools - Multicast Address, ensure that the multicast address pool ( Domain/AVS or AVE name_mcastpool) is deleted.

Cisco AVS or Cisco ACI Virtual Edge VMM Domain Security Domain Mapping

You can update the security domain mapping for the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

- Updating the Security Domain Mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Updating the Security Domain Mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Update AVS or AVE VMM Domain Security Domain Mapping and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the AVS/VMM-domain name field, enter the VMM domain name.

4. In the AAA Domain list table, click New and enter the AAA domain name.

For each entry, specify the existing security domain in the aaaDomainName column. Check the check box in the IsAddOperation column to add the AVS or AVE VMM domain to the AAA domain. If unchecked, the AVS or AVE VMM domain is removed from the AAA domain.

5. Click Submit.

What to do next

Complete the procedure Verifying the Security Domain Mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain.

- Verifying the Security Domain Mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Verifying the Security Domain Mapping of the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Virtual Networking > Inventory > VMM Domains > VMware.

Step 3

Choose the VMM domain.

Step 4

In the work pane, under Properties, ensure that the Security Domains field has been updated.

Distributed Firewall Policy

You can create, update, and delete a Distributed Firewall (DFW) policy and update the DFW policy association with the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

For detailed information about Distributed Firewall, see the one of the following:

- The section "Distributed Firewall in the Cisco ACI AVS Configuration Guide

- The chapter "Distributed Firewall" in the Cisco ACI Virtual Edge Configuration Guide

- Creating a Distributed Firewall Policy

- Updating a Distributed Firewall Policy

- Deleting a Distributed Firewall Policy

- Updating a Distributed Firewall Policy Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

Creating a Distributed Firewall Policy

This section describes how to create a DFW policy and associate it with a Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Create FW Policy (DFW) and Associate to AVS or AVE VMM Domain and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add the description and click Next.

3. In the FW Policy Name field, enter a name for the policy.

4. From the Mode drop-down list, choose Learning, Enabled, or Disabled.

- Learning—Cisco AVS or Cisco ACI Virtual Edge monitors all TCP communication and creates flows in a flow table but does not enforce the firewall. Learning mode lets you enable the firewall without losing traffic.

- Enabled—Enforces the Distributed Firewall. If you upgrade from an earlier version of Cisco AVS—one that does not support Distributed Firewall—and are upgrading Cisco AVS only, you must first upgrade all the Cisco AVS hosts in that VMM domain and then enable Distributed Firewall.

- Disabled—Does not enforce the Distributed Firewall and removes all flow information from the Cisco AVS or Cisco ACI Virtual Edge. Choose this mode only if you do not want to use the Distributed Firewall.

5. In the VMM Name field, enter the name of the existing Cisco AVS or Cisco ACI Virtual Edge VMM domain to which you want to associate the DFW policy and then click Next.

6. In the Syslog Form page, choose Disabled or Enabled from the Administrative State drop-down list.

7. Cisco AVS or Cisco ACI Virtual Edge reports the flows that are permitted or denied by the Distributed Firewall to the system log (syslog) server. Do the following:

- From the Permitted flows drop-down list, choose yes if you want Cisco AVS or Cisco ACI Virtual Edge to report permitted flows to the syslog server. Choose no if you do not want Cisco AVS or Cisco ACI Virtual Edge to report permitted flows to the syslog server.

- From the Denied flows drop-down list, choose yes if you want Cisco AVS or Cisco ACI Virtual Edge to report denied flows to the syslog server. Choose no if you do not want Cisco AVS or Cisco ACI Virtual Edge to report denied flows to the syslog server.

8. In the Polling Interval (seconds) area, enter an interval from 60 to 86,400 seconds.

9. From the Log Level drop-down list, choose a logging severity level that is greater than or equal to the severity level defined for the syslog server.

10. In the Dest Group area, enter an existing syslog monitoring destination group.

11. Click Submit.

What to do next

Complete the procedure Verifying Distributed Firewall Policy Creation on Cisco APIC.

- Verifying Distributed Firewall Policy Creation on Cisco APIC

Verifying Distributed Firewall Policy Creation on Cisco APIC

This section describes how to verify the creation of a distributed firewall policy on Cisco APIC.

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, choose Policies > Interface > Firewall.

Step 4

In the work pane, under Interface - Firewall, confirm that the corresponding firewall policy is created.

Step 5

To view the distributed firewall policy association with a VMM domain, do the following:

1. Choose Virtual Networking > Inventory > VMM Domains > VMware.

2. Click the corresponding VMM domain.

3. In the work pane, click VSwitch Policy, and then confirm that the created distributed firewall policy is present in the Firewall Policy field.

Updating a Distributed Firewall Policy

This section describes how to update an existing DFW policy.

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Update FW Policy (DFW) and complete the following steps:

In the service blueprint, some drop-down lists have a <NO CHANGE> option that you can choose if you do not want to change the configured value.

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add the description and click Next.

3. In the FW Policy Name field, enter an updated name for the policy.

4. From the Mode drop-down list, choose Learning, Enabled, Disabled, or <NO CHANGE>. Click Next.

5. In the Syslog Form page, choose Disabled, Enabled, or <NO CHANGE> from the Administrative State drop-down list.

6. From the Permitted flows drop-down list, choose yes, no, or <NO CHANGE>.

7. From the Denied flows drop-down list, choose yes, no, or <NO CHANGE>.

8. In the Polling Interval (seconds) area, update the interval to a value from 60 to 86,400 seconds.

Note

If you do not specify an interval, no update occurs.

9. From the Log Level drop-down list, choose a logging severity level that is greater than or equal to the severity level defined for the syslog server. Choose <NO CHANGE> if you do not want to change the log level.

10. In the Dest Group area, enter a new or existing syslog monitoring destination group.

Note

If you do not enter a new or existing syslog monitoring destination group, no update occurs.

11. Click Submit.

- Verifying a Distributed Firewall Policy Update on Cisco APIC

Verifying a Distributed Firewall Policy Update on Cisco APIC

This section describes how to verify an update to a distributed firewall policy on Cisco APIC.

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, choose Policies > Interface > Firewall.

Step 4

In the work pane, under Interface - Firewall, double-click the required firewall policy and confirm that it is updated.

Deleting a Distributed Firewall Policy

This section describes how to delete a DFW policy.

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Delete FW Policy (DFW) and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add the description and click Next.

3. In the FW Policy Name field, enter the name of the DFW policy that you want to delete.

4. Click Submit.

- Verifying a Distributed Firewall Policy Deletion on Cisco APIC

Verifying a Distributed Firewall Policy Deletion on Cisco APIC

This section describes how to verify the deletion of a distributed firewall policy on Application Policy Infrastructure Controller.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Choose Fabric > Access Policies.

Step 3

In the Policies navigation pane, choose Policies > Interface > Firewall.

Step 4

In the work pane, under Interface - Firewall, confirm that the deleted firewall policy is not present.

Updating a Distributed Firewall Policy Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

This section describes how to update a DFW policy that is associated with a Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Procedure

Step 1

Log in to vRealize Automation as the administrator and then choose Catalog.

Step 2

Choose Update FW Policy (DFW) association to AVS or AVE VMM Domain and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add the description and click Next.

3. In the FW Policy Name field, enter a name for the policy.

4. In the VMM Domain name field, enter an existing Cisco AVS or Cisco ACI Virtual Edge VMM domain name.

5. From the Operation drop-down list, choose one of the following options:

- add—Associates the DFW policy with the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

- del—Disassociates the DFW policy from the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

6. Click Submit.

What to do next

Complete the procedure Verifying Microsegment Association Updates with Cisco AVS or Cisco ACI Virtual Edge VMM Domains on APIC

- Verifying a Distributed Firewall Policy Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain on APIC

Verifying a Distributed Firewall Policy Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain on APIC

This section describes how to verify the association of a distributed firewall policy with Cisco AVS or Cisco ACI Virtual Edge on Cisco APIC.

Procedure

Step 1

Log in to Cisco APIC as the administrator.

Step 2

Choose Virtual Networking > Inventory > VMM Domains > VMware.

Step 3

Click the required VMM domain.

Step 4

In the Work pane, under Properties, confirm that the distributed firewall policy is associated with the VMM domain in the Firewall Policy field for vSwitch Policies.

Creating Networks in a Shared Plan

This section describes how to create a network in a shared plan.

Procedure

Step 1

Log in to the vRealize Automation as the tenant administrator, choose Catalog.

Step 2

In the navigation pane, choose Tenant Shared Plan.

Step 3

In the Tenant Shared Plan pane, choose Add Tenant Network - Shared Plan and perform the following actions:

1. View the Service Blueprint Information for the input fields and click Request.

2. In the Request Information pane, add the description and click Next.

3. In the Step pane, perform the following actions:

4. In the NetworkEPG name field, enter the name of the new shared network (new-shared-network).

5. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter , and then select the DVS.

6. From the encapMode drop-down list, choose either Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only if the VMM domain type is Cisco AVS or Cisco ACI Virtual Edge (Local Switching). Selecting VLAN or VXLAN for VDS VMM Domain, may lead into unpredictable results.

7. In the Application Tier Number field, enter a numeric value from 1 to 10.

8. In the Intra EPG Deny field, select a value either Yes or No.

9. In the Allow Microsegmentation field, select a value, either Yes or No.

Note

The Allow Microsegmentation field is applicable only if the VMM domain type is VDS VMM Domain.

10. In the Use Default BD? field, select a value either Yes or No.

If you selected No, choose a custom bridge domain by clicking on Add.

- Expand your_apic_user > Tenants > your_tenant > Networking > BridgeDomains > your_bridgedomain and select this bridge domain.

11. In the Switching Mode selector, choose native or AVE.

The native option is default switching; AVE is for Cisco ACI Virtual Edge switching.

12. Click Submit.

Verifying the Newly Created Network on VMware vRealize and APIC

This section describes how to verify the newly created network on VMware vRealize and Application Policy Infrastructure Controller (APIC) .

Procedure

Step 1

Log in to the vRealize Automation as the tenant administrator, choose Request and ensure your request status is successful.

Step 2

Log into the APIC GUI as the Tenant, choose Tenants.

Step 3

In the navigation pane, expand the Tenant name > Application Profiles > default > Application EPGs > EPG new-shared-network.

Step 4

In the Properties pane, ensure the Received Bridge Domain field is common/default.

Step 5

In the navigation pane, choose Domains (VMs and Bare-Metals), ensure it is bound to VMware/your_vmm_domain.

Creating a Bridge Domain in a VPC Plan

This section describes how to create a bridge domain in a VPC plan.

Procedure

Step 1

Log in to the vRealize Automation as the tenant administrator, choose Catalog.

Step 2

In the navigation pane, choose Tenant Network Services.

Step 3

In the Tenant Network Services pane, choose Add or Delete Bridge domain in Tenant and perform the following actions:

1. View the Service Blueprint Information for the input fields and click Request.

2. In the Request Information pane, add the description and click Next.

3. In the Step pane, perform the following actions:

4. In the Add a bridge domain field, choose Yes.

5. In the Bridge Domain name field, enter the bridge domain name (new-bd).

6. In the Enable ARP Flooding field, choose No.

7. In the Enable flooding for L2 Unknown Unicast field, choose hardware-proxy.

8. In the Enable flooding for L3 Unknown Multicast field, choose flood.

9. In the L3 context (VRF) field, click Add, expand your_apic > Tenants > your_tenant > Networking > VRFs and select the VRF (ctx1).

10. Click Submit.

11. In the Operation field, choose Add.

12. Click Submit.

- Verifying the Newly Created Bridge Domain on APIC

Verifying the Newly Created Bridge Domain on APIC

This section describes how to verify the newly created bridge domain on Application Policy Infrastructure Controller (APIC).

Procedure

Step 1

Log into the APIC GUI as the tenant, choose Tenants.

Step 2

In the navigation pane, expand the Tenant name > Networking > Bridge Domain > your_newly_created_bd .

Step 3

In the Properties pane, ensure the fields are the same as in the VMware vRealize GUI.

Creating a Network and Associating to a Bridge Domain in a VPC Plan

This section describes how to create a network and associating to a bridge domain in a VPC Plan.

Procedure

Step 1

Log in to the vRealize Automation as the tenant administrator, choose Catalog.

Step 2

In the navigation pane, choose Tenant VPC Plan.

Step 3

In the Tenant VPC Plan pane, choose Add Tenant Network - VPC Plan and perform the following actions:

1. View the Service Blueprint Information for the input fields and click Request.

2. In the Request Information pane, add the description and click Next.

3. In the Step pane, perform the following actions:

4. In the NetworkEPG name field, enter the name of the new shared network (new-vpc-network).

5. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter and select the DVS.

6. From the encapMode drop-down list, choose either Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only if the VMMdomain type is Cisco AVS or Cisco ACI Virtual Edge (Local Switching). Selecting VLAN or VXLAN for VDS VMM Domain, may lead into unpredictable results.

7. In the Application Tier Number field, enter a numeric value from 1-10.

8. In the Intra EPG Deny field, select a value either Yes or No.

9. In the Allow Microsegmentation field, select a value either Yes or No.

Note

The Allow Microsegmentation field is applicable only if the VMMdomain type is VDS VMM Domain.

10. In the Use Default BD? field, select a value either Yes or No.

If you selected No, choose a custom bridge domain by clicking on Add.

- Expand your_apic_user > Tenants > your_tenant > Networking > BridgeDomains > your_bridgedomain and select this bridge domain.

11. In the Subnet Prefix field, enter the gateway IP address and the subnet mask (10.1.1.1/24).

12. Click Submit.

- Verifying the Network and Association to the Bridge Domain in a VPC Plan on APIC

Verifying the Network and Association to the Bridge Domain in a VPC Plan on APIC

This section describes how to verify the newly created bridge domain on APIC.

Procedure

Step 1

Log into the APIC GUI as the Tenant, choose Tenants.

Step 2

In the navigation pane, expand the Tenant name > Application Profiles > default > Application EPGs > EPG new-vpc-network.

Step 3

In the Properties pane, ensure the Bridge Domain is your_tenant/bd1.

Step 4

In the navigation pane, choose Domains (VMs and Bare-Metals), ensure it is bound to VMware/your_vmm_domain.

Step 5

In the navigation pane, expand the Tenant name > Networking > Bridge Domain > bd1 > Subnets.

Step 6

In the Subnets pane, ensure the gateway IP address and subnet mask that you enter when creating a network and associating to a bridge domain in a VPC plan (10.1.1.1/24) and the scope is Private to VRF.

Step 7

On the menu bar, choose Virtual Networking.

Step 8

In the navigation pane, expand the VMM Domains > VMware > your_vmm_domain > Controllers > vcenter1 > DVS - your_vmm_domain > Portgroups and ensure you see the port group with the tenant application profile EPG name.

Creating a Security Policy Within the Tenant

This section describes how to create a security policy within the tenant.

This figure shows that Web and App are in the same bridge domain, but there is no communication. Web and App are isolated, but they can communicate to their gateway. You need to create a security policy for Web and App to communicate.

Before you begin

Ensure you have set up two shared networks with two virtual machines (VMs).

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Network Security.

Step 2

Choose Add Security Policy (Contracts)

Step 3

Choose Request.

Step 4

In the Request Information tab, enter a description of the request.

Step 5

Choose Next.

Step 6

In the Step tab, perform the following actions:

1. In the Rule Entry List field, enter the values and click Save.

This table shows the values for each Rule Entry:

Rule Entry List

Values

dstFormPort

- Blank

- Unspecified

- 1-65535

dstToPort

- Blank

- Unspecified

- 1-65535

protocol

- icmp

- icmpv6

- tcp

- udp

- Blank

etherType

- IP

- ARP

2. In the Consumer Network/EPG name field, click Add to locate and choose the consumer network/EPG. (web-host)

3. Click Submit.

4. In the Provider Network/EPG name field, click Add to locate and choose the provider network/EPG. (app-host)

5. Click Submit.

Step 7

Click Submit.

Step 8

Click OK.

- Verifying the Security Policy Within the Tenant on APIC

- Verifying the Connectivity of the Security Policy within the Tenant

Verifying the Security Policy Within the Tenant on APIC

This section describes how to verify the security policy within the tenant on APIC.

Procedure

Step 1

Log in to Cisco APIC and then choose TENANTS.

Step 2

In the navigation pane, expand Tenant your_tenant > Networking > Security Policies > Contracts.

1. Ensure the name nested under Contracts is the provider and consumer name. (app-host_ctrct_web-hosts)

Step 3

In the navigation pane, expand Tenant your_tenant > Networking > Security Policies > Filters.

1. Ensure the name nested under Filters is the provider and consumer name. (app-host_flt_web-hosts)

Step 4

In the navigation pane, expand Tenant your_tenant > Networking > Application Profiles > default > Application EPGs > EPG web-hosts > Contracts.

1. In the work pane, ensure the consumer is Comsumed.

Step 5

In the navigation pane, expand Tenant your_tenant > Networking > Application Profiles > default > Application EPGs > EPG app-hosts > Contracts.

1. In the work pane, ensure the provider is Provided.

Verifying the Connectivity of the Security Policy within the Tenant

This section describes how to verify the connectivity of the security policy within the tenant.

Procedure

Step 1

Log in to the virtual machine (web-host), from the command line, ping the other VM (app-host).

Step 2

Log in to the virtual machine (app-host), from the command line, ping the other VM (web-host).

This ensure the VMs are communicating with each other.

Consuming a Shared Service in the Common Tenant

This section describes consuming a shared service in the common tenant.

Before you begin

You must have an EPG in the common tenant that has a bridge domain relationship to "common/default".

Procedure

Step 1

Log in to the vRealize Automation as tenant, choose Catalog > Network Security.

Step 2

Choose Add Security Policy (Contracts)

Step 3

Choose Request.

Step 4

In the Request Information tab, enter a description of the request.

Step 5

Choose Next.

Step 6

In the Step tab, perform the following actions:

1. In the Rule Entry List field, enter the values and click Save.

This table shows the values for each Rule Entry:

Rule Entry List

Values

dstFormPort

- Blank

- Unspecified

- 1-65535

dstToPort

- Blank

- Unspecified

- 1-65535

protocol

- icmp

- icmpv6

- tcp

- udp

- Blank

etherType

- IP

- ARP

2. In the Consumer Network/EPG name field, click Add to locate and choose the consumer network/EPG. (web-host)

3. Click Submit.

4. In the Provider Network/EPG name field, click Add to locate and choose the provider network/EPG. (SYSLOG-EPG)

5. Click Submit.

Step 7

Click Submit.

Step 8

Click OK.

- Verifying the Security Policy in the Tenant Common on APIC

- Verifying the Connectivity of the Security Policy in the Tenant Common

Verifying the Security Policy in the Tenant Common on APIC

This section describes how to verify the security policy in the tenant common on APIC.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose TENANTS.

Step 2

In the navigation pane, expand Tenant your_tenant > Networking > Security Policies > Contracts.

1. Ensure the name nested under Contracts is the provider and consumer name. (SYSLOG-EPG_ctrct_web-hosts)

Step 3

In the navigation pane, expand Tenant your_tenant > Networking > Security Policies > Filters.

1. Ensure the name nested under Filters is the provider and consumer name. (SYSLOG-EPG_flt_web-hosts)

Step 4

In the navigation pane, expand Tenant your_tenant > Networking > Application Profiles > default > Application EPGs > EPG web-hosts > Contracts.

1. In the work pane, ensure the consumer is Comsumed.

Step 5

In the navigation pane, expand Tenant your_tenant > Networking > Application Profiles > default > Application EPGs > EPG SYSLOG-EPG-hosts > Contracts.

1. In the work pane, ensure the provider is Provided.

Verifying the Connectivity of the Security Policy in the Tenant Common

This section describes how to verify the connectivity of the security policy in the tenant common.

Procedure

Step 1

Log in to the virtual machine (web-host), from the command line, ping the other VM (SYSLOG-EPG).

Step 2

Log in to the virtual machine (SYSLOG-EPG), from the command line, ping the other VM (web-host).

This ensure the VMs are communicating with each other.

Updating Security Policies (Access Control Lists)

This section describes how to update security policies (access control lists).

Procedure

Step 1

Log in to the vRealize Automation as tenant, choose Catalog > Network Security.

Step 2

Choose Update Security policies (Access Control Lists)

Step 3

Choose Request.

Step 4

In the Request Information tab, enter a description of the request.

Step 5

Choose Next.

Step 6

In the Step tab, perform the following actions:

1. In the apic security filter name field, click Add to locate and choose a filter that been pushed by vRealize.

2. In the Rule Entry List field, enter the values and click Save. You must recreate the rule entry list.

Note

This updating security policies access control lists will push new rules in including over writing existing rule of the same name.

This table shows the values for each Rule Entry:

Rule Entry List

Values

dstFormPort

- Blank

- Unspecified

- 1-65535

dstToPort

- Blank

- Unspecified

- 1-65535

protocol

- icmp

- icmpv6

- tcp

- udp

- Blank

etherType

- IP

- ARP

3. In the Update firewall access-list field, if the access-list being use by a firewall, click Yes otherwise click No.

4. Click Submit.

Step 7

Click OK.

Step 8

To verify your request, choose the Requests tab.

1. Choose the request you submitted and click view details. Ensure the status is Succesful.

Deleting Security Policies (Access Control Lists)

This section describes how to delete security policies (access control lists).

Procedure

Step 1

Log in to the vRealize Automation as tenant, choose Catalog > Network Security.

Step 2

Choose Delete Security policies (Access Control Lists)

Step 3

Choose Request.

Step 4

In the Request Information tab, enter a description of the request.

Step 5

Choose Next.

Step 6

In the Step tab, perform the following actions:

1. In the Comsume Network/EPG name field, click Add to locate and choose the provider network/EPG. (web-host)

2. In the Provider Network/EPG name field, click Add to locate and choose the provider network/EPG. (app-host)

3. Click Submit.

Step 7

Click OK.

Step 8

To verify your request, choose the Requests tab.

1. Choose the request you submitted and click view details. Ensure the status is Succesful.

Creating the Network in the VPC Plan

This section describes how to create the network in the VPC plan.

Procedure

Step 1

Log in to the vRealize Automation Appliance as the tenant, choose Catalog > Tenant VPC Plan > Add Tenant Network - VPC plan and click Request.

Step 2

In the Request Information pane, perform the following actions:

1. In the Description field, enter the description.

2. Click Next.

Step 3

In the Step pane, perform the following actions:

1. In the Network/EPG name field, enter the Network/EPG name. (web-hosts-vpc)

2. In the Domain Type field, from the drop-down list, choose either VmmDomain (Dynamic Binding) for connecting to virtual machines or PhysDomain (Static Binding) for connecting to physical infrastructure. Cisco recommends choosing VmmDomain (Dynamic Binding) to use the full features of the vRealize plug-in.

3. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter , and then select the DVS.

4. From the encapMode drop-down list, choose either Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only if the VMM domain type is Cisco AVS or Cisco ACI Virtual Edge (Local Switching). Selecting VLAN or VXLAN for VDS VMM Domain, may lead into unpredictable results.

5. In the Application Tier Number field, enter a numeric value from 1 to 10.

6. In the Intra EPG Deny field, select a value either Yes or No.

7. In the Allow Microsegmentation field, select a value either Yes or No.

Note

The Allow Microsegmentation field is applicable only if the VMM domain type is VDS VMM Domain.

8. In the Use Default BD? field, select a value either Yes or No.

If you selected No, choose a custom bridge domain by clicking on Add.

- Expand your_apic_user > Tenants > your_tenant > Networking > BridgeDomains > your_bridgedomain and select this bridge domain.

9. In the Subnet prefix field, enter the gateway IP address and the subnet mask. (192.168.1.1/24)

The subnet prefix is the subnet that this VPC will have available to any hosts.

10. Click Submit.

11. Click OK.

Step 4

Choose Requests.

Step 5

Choose the request you submitted and click view details.

Step 6

Ensure that your request status is Successful.

- Verifying the Network in the VPC Plan on APIC

- Verifying the Network in the VPC Plan on vCenter

Verifying the Network in the VPC Plan on APIC

This section describes how to verify the network in the VPC plan on APIC.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > Application EPGs > EPG web-hosts-vpc

Step 3

In the properties pane, in the Bridge Domain field, verify your tenant name and bd1 is present. (green/bd1)

Step 4

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > Application EPGs > EPG web-hosts-vpc > Domains (VMs and Bare-Metals).

Step 5

Ensure the state is formed and the domain profile is VMware/vmmdomain_you_specified.

Step 6

In the navigation pane, choose Tenant your_tenant > Networking > Bridge Domains > bd1 > Subnets.

Step 7

Under Subnets, ensure the subnet prefix that you specified is present.

Verifying the Network in the VPC Plan on vCenter

This section describes how to verify the network in the VPC plan on vCenter.

Procedure

Step 1

Log in to vSphere Web Client GUI, choose the Networking icon.

Step 2

In the navigation pane, choose vCenter_IP/Host > Datacenter > green > distributed_virtual_switch > port_group and ensure it is present.

The port_group name is in the following format: Tenant Name|Application Profile Name|Application EPG Name.

Updating a Tenant Network Association with the VMM Domain

This section describes how to update a tenant network association with the VMM domain.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and choose Catalog.

Step 2

In the navigation pane, choose Tenant Network services.

Step 3

Choose Update Tenant Network and perform the following actions:

1. View the Service Blueprint Information for the input fields and click Request.

2. In the Request Information pane, add the description and click Next.

3. In the Tenant name field, input the name of corresponding tenant.

4. In the Network/EPG field, click Add, expand your_apic > Tenants > your_tenant > End-Point-Groups, and then select the EPG.

5. From the Domain Type drop-down list, choose the domain type. The domain type is VmmDomain (Dynamic Binding) for VMware VDS or Cisco AVS or Cisco ACI Virtual Edge.

6. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter , and then select the DVS to associate the tenant network (EPG) to the VMM domain.

7. From the encapMode drop-down list, choose Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only when associating an EPG to a VMM domain of the Cisco AVS or Cisco ACI Virtual Edge (Local Switching) type. That association is performed in the following step.

8. From the Operation drop-down list, choose add to associate the tenant network with the VMM domain or choose delete to disassociate the tenant network from the VMM domain.

9. In the Switching Mode selector, choose native or AVE.

The native option is default switching, and AVE is for Cisco ACI Virtual Edge.

10. Click Submit.

- Verifying Tenant Network Association with VMM Domains on APIC

Verifying Tenant Network Association with VMM Domains on APIC

This section describes how to verify a tenant Network association with VMM domains on APIC.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant.

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > Application EPGs > your_tenant_network > Domains (VMs and Bare-Metals).

Step 3

Confirm that any associations with VMM domains are correct.

Microsegmentation

This section describes microsegmentation in shared and VPC plans and explains the usage-related service blueprints.

Note

Starting with the Cisco APIC vRealize Plug-In 2.0(1) release, the service blueprints related to microsegmentation are supported only for Cisco AVS VMM domains.

- Microsegmentation with Cisco ACI

- Microsegmentation in a Shared Plan

- Microsegmentation in a VPC Plan

- Updating Microsegment Attributes

- Verifying a Microsegmentation Attributes Update on APIC

- Updating a Microsegment Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

- Verifying Microsegment Association Updates with Cisco AVS or Cisco ACI Virtual Edge VMM Domains on APIC

Microsegmentation with Cisco ACI

Microsegmentation with the Cisco ACI provides the ability to automatically assign endpoints to logical security zones called endpoint groups (EPGs) based on various attributes.

For detailed information about Microsegmentation, see the chapter "Microsegmentation with Cisco ACI" in the Cisco ACI Virtualization Guide.

Microsegmentation in a Shared Plan

You can create, update, and delete a microsegment in a shared plan.

- Creating a Microsegment in a Shared Plan

- Verifying Microsegmentation Creation in a Shared Plan on APIC

- Deleting a Microsegment in a Shared Plan

- Verifying Microsegmentation Deletion on APIC

Creating a Microsegment in a Shared Plan

This section describes how to create a microsegment in a shared plan.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant Shared Plan.

Step 3

Choose Add a Useg Network - Shared Plan and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Tenant name field, enter the name of the corresponding tenant.

4. In the Network/EPG name field, enter the name of the microsegment (uSeg) that you want to create.

5. From the Domain Type drop-down list, choose the domain type. For the Cisco AVS or Cisco ACI Virtual Edge VMM domain, the domain type is VmmDomain (Dynamic Binding).

6. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter , and then and select the DVS (Cisco AVS or Cisco ACI Virtual Edge VMM domain) to associate the uSeg to the VMM domain.

7. From the encapMode drop-down list, choose Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only if the VMMdomain type is Cisco AVS or Cisco ACI Virtual Edge (Local Switching).

8. In the Application Tier Number field, enter the number of the tier to which the uSeg belongs. The default tier number is 1. The tier number that you enter must be less than or equal to the number of application tiers that were created as part of the tenant creation via the service blueprint Add or Update Tenant option.

For example, if you enter tier number 2, the uSeg will be placed in BD (common/cmnbd2), which is part of VRF (common/default). See the following table for reference.

Tier Number

BD

VRF

1

common/default

common/default

2

common/cmnbd2

common/default

3

common/cmnbd3

common/default

9. From the Intra EPG Deny drop-down list, choose Yes to enforce intra-EPG isolation. Choose No if you do not want to enforce intra-EPG isolation.

Intra-EPG isolation is not supported in AVS or Cisco ACI Virtual Edge VLAN mode, DVS-VXLAN mode, or for Microsoft VMM domains. If you enforce intra-EPG isolation for those modes or domains, ports might go into blocked state.

10. In the Ip Criteria table, click New and enter the IP criteria (or IP attribute). The following columns apply to each entry:

- Name—Name of the IP criteria (or IP attribute).

- Description—Description of the IP criteria.

- IP—For IP addresses, specify the address or the subnet (for example, 1.1.1.1 or 1.1.1.0/30).

11. In the Mac Criteria table, click New and enter the MAC criteria (or MAC attribute). The following columns apply to each entry:

- Name—Name of the MAC criteria (or MAC attribute).

- Description—Description of the MAC criteria.

- MAC—For MAC addresses, specify the address (for example, 00:50:56:44:44:5D).

12. In the VM Criteria table, click New and enter the VM criteria (or VM attribute). The following columns apply to each entry:

- Name—Name of the VM criteria (or VM attribute).

- Type—The following table lists the supported attribute types, their mapping in APIC, and examples. (The MAC attribute and IP attribute have precedence 1 and 2, respectively.)

Type in vRealize

Type in APIC (Mapping)

Precedence

Examples

vnic

VNic Dn

3

00:50:56:44:44:5D

vm

VM Identifier

4

vm-821

vmName

VM Name

5

HR_VDI_VM1

hv

Hypervisor Identifier

6

host-43

domain

VMM Domain

7

AVS-SJC-DC1

datacenter

Datacenter

8

DCI

customLabel

Custom Attribute

9

SG_DMZ

guestOS

Operating System

10

Windows 2008

- Operator—The following table lists the supported operators and their mapping in APIC.

Operator in vRealize

Operator in APIC (Mapping)

equals

Equals

contains

Contains

startsWith

Starts With

endsWith

Ends With

- AttributeName—Enter an attribute name. In the VM Criteria table, the AttributeName applies only to the customLabel attribute type.

- VmmDomain_vC_VmName—In the VM Criteria table, it is applicable only for the type vnic, operator equals. The format to input is <VmmDomain>/<vC>/<VmName>, where <VmmDomain> (AVS VMM domain) and <vC> (vCenter) belong to a controller instance. For example: vmmdomain1/vcenter1/VM1.

- Value—Enter the attribute type value. Examples of each attribute type are listed in the preceding Type table.

13. Click Submit.

What to do next

Complete the procedure Verifying Microsegmentation Creation in a Shared Plan on APIC.

Verifying Microsegmentation Creation in a Shared Plan on APIC

This section describes how to verify that microsegmentation creation in a shared plan has been successful on Application Policy Infrastructure Controller.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs.

Step 3

In the uSeg EPGs pane, double-click the required uSeg to view its properties.

Step 4

In the Properties pane, confirm that the configuration is correct.

Step 5

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs > your_useg > Domains (VMs and Bare-Metals).

Step 6

Confirm that the state is formed and that the domain profile is VMware/vmmdomain_you_specified.

Deleting a Microsegment in a Shared Plan

This section describes how to delete a microsegment.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant Shared Plan.

Step 3

Choose Delete a Useg Network - Shared Plan and then complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Tenant name field, confirm that the tenant name is hard coded to the corresponding tenant.

4. In the Network/EPG field, click Add, expand priapic > Tenants > appurtenant > Useg-End-Point-Groups, and then select the microsegment EPG.

5. Click Submit.

What to do next

Complete the procedure Verifying Microsegmentation Deletion on APIC.

Verifying Microsegmentation Deletion on APIC

This section describes how to verify microsegmentation deletion on Application Policy Infrastructure Controller.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs.

Step 3

In the uSeg EPGs pane, confirm that the deleted uSeg is not present.

Microsegmentation in a VPC Plan

You can create, update, and delete a microsegment in a VPC plan.

- Creating a Microsegment in a VPC Plan

- Verifying Microsegmentation Creation in a VPC Plan on APIC

- Deleting a Microsegment in a VPC Plan

Creating a Microsegment in a VPC Plan

This section describes how to create a microsegment in a VPC plan.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant VPC Plan.

Step 3

Choose Add a Useg Network - VPC Plan and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Tenant name field, enter the name of the corresponding tenant.

4. In the Network/EPG name field, enter the name of the microsegment (uSeg) that you want to create.

5. From the Domain Type drop-down list, choose the domain type.

6. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter , and then select the DVS (Cisco AVS or Cisco ACI Virtual Edge VMM domain) to associate the uSeg to the VMM domain.

7. From the encapMode drop-down list, choose Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only if the VMM domain type is Cisco AVS or Cisco ACI Virtual Edge (Local Switching).

8. In the Subnet field, enter the gateway IP address and the subnet mask (1.1.1.1/24).

9. In the Application Tier Number field, enter the number of the tier to which the uSeg belongs. The default tier number is 1. The tier number that you enter must be less than or equal to the number of application tiers that were created as part of the tenant creation through the service blueprint Add or Update Tenant option.

For example, for a tenant named coke, if you enter tier number 2, the uSeg is placed in BD (coke/bd2), which is part of VRF (coke/ctx1). See the following table for reference.

Tier Number

BD

VRF

1

coke/bd1

coke/ctx1

2

coke/bd2

coke/ctx1

3

coke/bd3

coke/ctx1

10. From the Intra EPG Deny drop-down list, choose Yes to enforce intra-EPG isolation. Choose No if you do not want to enforce intra-EPG isolation.

Intra-EPG isolation is not supported in Cisco AVS or Cisco ACI Virtual Edge VLAN mode, DVS-VXLAN mode, or for Microsoft VMM domains. If you enforce intra-EPG isolation for those modes or domains, ports may go into blocked state.

11. In the Ip Criteria table, click New and enter the IP criteria (or IP attribute). The following columns apply to each entry:

- Name—Name of the IP criteria (or IP attribute).

- Description—Description of the IP criteria.

- IP—For IP addresses, specify the address or the subnet (for example, 1.1.1.1 or 1.1.1.0/30).

12. In the Mac Criteria table, click New and enter the MAC criteria (or MAC attribute). The following columns apply to each entry:

- Name—Name of the MAC criteria (or MAC attribute).

- Description—Description of the MAC criteria.

- MAC—For MAC addresses, specify the address (for example, 00:50:56:44:44:5D).

13. In the VM Criteria table, click New and enter the VM criteria (or VM attribute). The following columns apply to each entry:

- Name—Name of the VM criteria (or VM attribute).

- Description—Description of the VM criteria.

- Type—The following table lists the supported attribute types, their mapping in APIC, and examples. (The MAC attribute and IP attribute have precedence 1 and 2, respectively.)

Type in vRealize

Type in APIC (Mapping)

Precedence

Examples

vnic

VNic Dn

3

00:50:56:44:44:5D

vm

VM Identifier

4

vm-821

vmName

VM Name

5

HR_VDI_VM1

hv

Hypervisor Identifier

6

host-43

domain

VMM Domain

7

AVS-SJC-DC1

datacenter

Datacenter

8

DCI

customLabel

Custom Attribute

9

SG_DMZ

guestOS

Operating System

10

Windows 2008

- Operator—The following table lists the supported operators and their mapping in APIC.

Operator in vRealize

Operator in APIC (Mapping)

equals

Equals

contains

Contains

startsWith

Starts With

endsWith

Ends With

- AttributeName—Enter an attribute name. In the VM Criteria table, the AttributeName applies only to the customLabel attribute type.

- VmmDomain_vC_VmName—In the VM Criteria table, it is applicable only for the type vnic, operator equals. The format to input is <VmmDomain>/<vC>/<VmName> where <VmmDomain> (AVS VMM domain) and <vC> (vCenter) belong to a controller instance. For example: vmmdomain1/vcenter1/VM1.

- Value—Enter the attribute type value. Examples of each attribute type are listed in the preceding Type table.

14. Click Submit.

What to do next

Complete the procedure Verifying Microsegmentation Creation in a VPC Plan on APIC.

Verifying Microsegmentation Creation in a VPC Plan on APIC

This section describes how to verify microsegmentation creation in a VPC plan on Application Policy Infrastructure Controller.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs.

Step 3

In the uSeg EPGs pane, double-click the required uSeg to view its properties.

Step 4

In the Properties pane, confirm that the configuration is correct.

Step 5

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs > your_useg > Domains (VMs and Bare-Metals).

Step 6

Confirm that the state is formed and that the domain profile is VMware/vmmdomain_you_specified.

Step 7

In the navigation pane, choose Tenant your_tenant > Networking > Bridge Domains > corresponding_bd > Subnets.

Step 8

Under Subnets, confirm that the subnet prefix that you specified is present.

Deleting a Microsegment in a VPC Plan

This section describes how to delete a microsegment.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant VPC Plan.

Step 3

Choose Delete a Useg Network - VPC Plan and then complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Tenant name field, confirm that the tenant name is hard coded to the corresponding tenant.

4. In the Network/EPG field, click Add, expand your_apic > Tenants > your_tenant > Useg-End-Point-Groups and select the uSeg EPG.

5. Click Submit.

What to do next

Complete the procedure Verifying Microsegmentation Deletion on APIC.

Updating Microsegment Attributes

This section describes how to update an existing microsegment.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant Network services.

Step 3

Choose Add or Delete Useg Attribute and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add a description and then click Next.

3. In the Network/EPG field, click Add, expand your_apic > Tenants > your_tenant > Useg-End-Point-Groups and select the uSeg EPG.

4. In the Tenant name field, enter the name of the corresponding tenant.

5. If you want to add IP criteria, in the Add Ip Criteria table, click New and enter the IP criteria (or IP attribute). The following columns apply to each entry:

- Name—Name of the IP criteria (or IP attribute).

- Description—Description of the IP criteria.

- IP—For IP addresses, specify the address or the subnet (for example, 1.1.1.1 or 1.1.1.0/30).

6. If you want to add Mac criteria, in the Add Mac Criteria table, click New and enter the MAC criteria (or MAC attribute). The following columns apply to each entry:

- Name—Name of the MAC criteria (or MAC attribute).

- Description—Description of the MAC criteria.

- MAC—For MAC addresses, specify the address (for example, 00:50:56:44:44:5D).

7. If you want to add VM criteria, in the Add Vm Criteria table, click New and enter the VM criteria (or VM attribute). The following columns apply to each entry:

- Name—Name of the VM criteria (or VM attribute).

- Type—The following table lists the supported attribute types, their mapping in APIC, and examples. (The MAC attribute and IP attribute have precedence 1 and 2, respectively.)

Type in vRealize

Type in APIC (Mapping)

Precedence

Examples

vnic

VNic Dn

3

00:50:56:44:44:5D

vm

VM Identifier

4

vm-821

vmName

VM Name

5

HR_VDI_VM1

hv

Hypervisor Identifier

6

host-43

domain

VMM Domain

7

AVS-SJC-DC1

datacenter

Datacenter

8

DCI

customLabel

Custom Attribute

9

SG_DMZ

guestOS

Operating System

10

Windows 2008

- Operator—The following table lists the supported operators and their mapping in APIC.

Operator in vRealize

Operator in APIC (Mapping)

equals

Equals

contains

Contains

startsWith

Starts With

endsWith

Ends With

- AttributeName—Enter an attribute name. In the VM Criteria table, the AttributeName applies only to the customLabel attribute type.

- Value—Enter the attribute type value. Examples of each attribute type are listed in the preceding Type table.

- VmmDomain_vC_VmName—In the VM Criteria table, it is applicable only for the type vnic, operator equals. The format to input is <VmmDomain>/<vC>/<VmName>, where <VmmDomain> (AVS VMM domain) and <vC> (vCenter) belong to a controller instance. For example: vmmdomain1/vcenter1/VM1.

8. If you want to delete existing IP criteria, in the Delete IP Criteria table, click New and enter the name of the IP criteria (or IP attribute) to delete.

9. If you want to delete existing Mac criteria, in the Delete Mac Criteria table, click New and enter the name of the MAC criteria (or MAC attribute) to delete.

10. If you want to delete existing VM criteria, in the Delete Vm Criteria table, click New and enter the name of the VM criteria (or VM attribute) to delete.

11. Click Submit.

What to do next

Complete the procedure Verifying a Microsegmentation Attributes Update on APIC.

Verifying a Microsegmentation Attributes Update on APIC

This section describes how to verify that microsegmentation attributes have been updated on Application Policy Infrastructure Controller.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs.

Step 3

In the uSeg EPGs pane, double-click the required uSeg to view its properties.

Step 4

In the Properties pane, confirm that the attributes in the uSeg Attributes field have been updated.

Updating a Microsegment Association with the Cisco AVS or Cisco ACI Virtual Edge VMM Domain

This section describes how to update a microsegment that is associated with a Cisco AVS or Cisco ACI Virtual Edge VMM domain.

Procedure

Step 1

Log in to vRealize Automation as the tenant administrator and then choose Catalog.

Step 2

In the navigation pane, choose Tenant Network services.

Step 3

Choose Update Tenant Network and complete the following steps:

1. View the Service Blueprint Information for the input fields and then click Request.

2. In the Request Information pane, add the description and click Next.

3. In the Tenant name field, enter the name of the corresponding tenant.

4. In the Network/EPG field, click Add, expand your_apic > Tenants > your_tenant > Useg-End-Point-Groups and select the uSeg EPG.

5. From the Domain Type drop-down list, choose the domain type. For the Cisco AVS or Cisco ACI Virtual Edge VMM domain, the domain type is VmmDomain (Dynamic Binding).

6. In the Domain/DVS field, click Add, expand your_apic > vCenters > your_vcenter and then select the DVS (Cisco AVS or Cisco ACI Virtual Edge VMM domain) to associate the uSeg to the VMM domain.

7. From the encapMode drop-down list, choose Auto, VLAN, or VXLAN for the encapsulation mode.

Note

The encapMode field is applicable only when associating an EPG to a VMM domain of the Cisco AVS or Cisco ACI Virtual Edge (Local Switching) type. That association is performed in the following step.

8. From the Operation drop-down list, choose add to associate the microsegment with the Cisco AVS or Cisco ACI Virtual Edge domain. Choose delete to disassociate the microsegment from the Cisco AVS or Cisco ACI Virtual Edge VMM domain.

9. Click Submit.

What to do next

Complete the procedure Verifying Microsegment Association Updates with Cisco AVS or Cisco ACI Virtual Edge VMM Domains on APIC.

Verifying Microsegment Association Updates with Cisco AVS or Cisco ACI Virtual Edge VMM Domains on APIC

This section describes how to verify updates to microsegment associations with Cisco AVS or Cisco ACI Virtual Edge VMM domains on Cisco APIC.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose Tenants > your_tenant .

Step 2

In the navigation pane, choose Tenant your_tenant > Application Profiles > default > uSeg EPGs > your_useg > Domains (VMs and Bare-Metals).

Step 3

Confirm that any associations with VMM domains are correct.

Creating the VMs and Attaching to Networks Without Using the Machine Blueprints

This section describes how to verify the creating machines (VMs) and attaching to networks without using the machine blueprints.

Procedure

Step 1

Log in to vSphere Web Client GUI, choose the Networking icon.

Step 2

In the pane, choose vCenter_IP/Host > Datacenter > Unmanaged and choose the virtual machine you want to attach ACI network to.

Step 3

In the Summary pane, in the VM Hardware section, click Edit Settings.

Step 4

In the Edit Settings dialog box, choose the network adapter that you want to connect to the ACI network and from the drop-down list, choose the port group you created. (green|default|web-hosts-vpc (green))

Step 5

Click OK.

Now this VM can take advantage of the ACI networking.

About Adding the Load Balancer to the Tenant Network

This section covers the configuration steps to add a load balancer service to a tenant network (APIC's EPG). This release only supports shared plan for load balancer. In subsequent releases we will have support for VPC plan.

In this plan, the load balancer is deployed in tn-common thereby offering consumption model for vRA and APIC tenant using shared infrastructure.

Shared Plan - Load Balancer Overview

VPC Plan - Load Balancer Only

- Configuration Prerequisites on APIC

- Adding the VIP Pool

- Deleting the VIP Pool

- Adding the Load Balancer to the Tenant-Network in a Shared Plan

- Adding the Load Balancer to the Tenant-Network in a VPC Plan

- Deleting the Load Balancer from the Tenant-Network in a Shared Plan

- Deleting the Load Balancer from the Tenant-Network in a VPC Plan

Configuration Prerequisites on APIC

This section describes the configuration prerequisites on APIC.

- Device package for load balancer needs to be uploaded by APIC admin.

- Device cluster for load balancer needs to be created in tn-common or tenant "common" by APIC-admin. Citrix and F5 are the supported vendors for load balancers.

- Shared Plan load balancer service graph templates for Citrix and F5 needs to be created in tn-common by APIC-admin.

Adding the VIP Pool

This section describes how to add the VIP Pool.

Before you begin

Before vRA-Tenant can consumer Load balancer services, vRA admin needs to create a Virtual-IP pool per vRA tenant, using the "Add VIP pool" service blueprint in Admin catalog.

For example for Tenant-Red, VIP pool is 6.1.1.1 to 6.1.1.30 and for Tenant-Green, VIP pool is 6.1.2.1 to 6.1.2.30.

Note

The VIP pool should be in one of the subnets defined under BD "default" in the tenant "common"

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

Step 2

Choose Add VIP Pool and perform the following actions:

1. In the Tenant field, enter the Tenant name.

2. In the VIP address start field, enter the VIP address start.

3. In the VIP Address End field, enter the VIP address end.

4. In the Internal VIP for Inter-EPG in VPC plan field, select Yes or No.

5. Click Submit.

Deleting the VIP Pool

This section describes how to delete the VIP Pool.

This blueprint is to do necessary cleanup of VIP pool, once all the load balancer services consumed in the tenant are deleted.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

Step 2

Choose Delete VIP Pool, perform the following action items.

1. In the Tenant field, click Add, expand your_apic > Tenants and select the tenant.

2. In the VIP address start field, enter the VIP address start.

3. In the VIP Address End field, enter the VIP address end.

4. In the Internal VIP for Inter-EPG in VPC plan field, select Yes or No.

5. Click Submit.

Adding the Load Balancer to the Tenant-Network in a Shared Plan

vRA-Tenant can add a load balancer (LB) to Tenant-Network. The required parameters are Network-Name, LB device cluster, LB-endpoint (protocol, port), Vendor Type, and Consumer EPG or L3out. As part of this workflow, all the required service graph instance and contract (security policy) with chosen Tenant-Network as Provider-EPG is created. The consumer of this load balanced endpoint could be L3out in tenant common, or it could be another Tenant-Network belonging to the tenant.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Add Load Balancer to Tenant Network - Shared Plan, click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Adding the Load Balancer to the Tenant-Network in a VPC Plan

This section describes how to add the load balancer to the tenant-network in a VPC Plan.

Note

In a VPC plan, the Inter-EPG load balancer is not supported. Only the load balancer between L3out and First-Tier (Web) is supported in release 1.2(2x).

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Add Load Balancer to Tenant Network - VPC Plan, click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Load Balancer from the Tenant-Network in a Shared Plan

You can delete the load balancer service (lb-port, lb-protocol) from an existing tenant network or endpoint group.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Delete Load Balancer to Tenant Network - Shared Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Load Balancer from the Tenant-Network in a VPC Plan

You can delete the load balancer service (lb-port, lb-protocol) from an existing tenant network or endpoint group.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Delete Load Balancer to Tenant Network - VPC Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Configuring the Firewall

This section discusses the configuration steps to add a firewall service to a tenant network (the Application Policy Infrastructure Controller's endpoint group).

Shared Plan - Perimeter Firewall Only Overview

Note

The perimeter firewall only service is not supported in VPC Plan. In VPC plan, the firewall service can be configured between EPGs.

- Adding the Firewall to the Tenant-Network in a Shared Plan

- Deleting the Firewall from the Tenant-Network in a Shared Plan

Adding the Firewall to the Tenant-Network in a Shared Plan

You can add the firewall to an existing tenant network or endpoint group. The consumer of the firewall must have a Layer 3 out connectivity policy configured in another VRF for example, "outside" VRF.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Add FW to Tenant Network - Shared Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Firewall from the Tenant-Network in a Shared Plan

You can delete the firewall from an existing tenant network or endpoint group.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Delete FW from Tenant Network - Shared Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Configuring the Firewall and Load Balancer

This section covers the configuration steps to add a firewall and load balancer service to a tenant network (the Application Policy Infrastructure Controller's endpoint group).

In this plan, the firewall and load balancer devices are deployed in the "common" tenant, there by offering consumption model for vRealize Automation (vRA) and the APIC tenant using the shared infrastructure.

Shared Plan - Firewall and Load Balancer Overview

VPC Plan - Perimeter Firewall and Load Balancer

- Adding the Firewall and Load Balancer to the Tenant-Network in a Shared Plan

- Adding the Firewall and Load Balancer to the Tenant-Network in a VPC Plan

- Deleting the Firewall and Load Balancer from the Tenant-Network in a Shared Plan

- Deleting the Firewall and Load Balancer from the Tenant-Network in a VPC Plan

Adding the Firewall and Load Balancer to the Tenant-Network in a Shared Plan

The virtual IP address pool must be added to the tenant before using the firewall and load balancer service.

See Adding the VIP Pool.

The firewall and load balancer can be added to an existing tenant network or endpoint group. The consumer of the firewall must have a Layer 3 out connectivity policy configured in the "outside" VRF.

Before you begin

For both Firewall and Load-Balancer only services have to be met before a firewall and load balancer service can be deployed.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Add FW and LB to Tenant Network - Shared Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Adding the Firewall and Load Balancer to the Tenant-Network in a VPC Plan

This section describes how to add the firewall and load balancer to the Tenant-Network in a VPC Plan.

Note

Whenever a firewall and load balancer (LB) workflow is executed then external leg of LB is pointing to "default" Bridge Domain (BD). Customers should always deploy internal leg of firewall in "default" BD under tn-common. This ensures that both the firewall and load balancer point to same BD and traffic flows in an uninterrupted way.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Add FW and LB to Tenant Network - VPC Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Firewall and Load Balancer from the Tenant-Network in a Shared Plan

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant Shared Plan.

Step 2

Choose Delete FW and LB from Tenant Network - Shared Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Firewall and Load Balancer from the Tenant-Network in a VPC Plan

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Delete FW and LB from Tenant Network - VPC Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Configuring the Inter-EPG Firewall

This section describes how to configure the inter-EPG firewall service to a tenant network (the Application Policy Infrastructure Controller's endpoint group).

VPC Plan - Inter EPG FW

- Adding the Firewall to the Tenant-Network in a VPC Plan

- Deleting the Firewall from the Tenant-Network in a VPC Plan

Adding the Firewall to the Tenant-Network in a VPC Plan

This section describes how to add the firewall to an existing tenant network or endpoint group (EPG). When adding the tenant, "Enable Inter-EPG Firewall" should be set to "yes" and the number of tiers used in the application should be configured. When configuring the network (EPG) tier number should be set. In this scenario, the firewall is configured between a provider EPG and consumer EPG.

Procedure

Step 1

Log into the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Add FW to Tenant Network - VPC Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Deleting the Firewall from the Tenant-Network in a VPC Plan

This section describes how to delete the firewall from an existing tenant network or endpoint group (EPG).

Procedure

Step 1

Log into the vRealize Automation as admin, choose Catalog > Tenant VPC Plan.

Step 2

Choose Delete FW from Tenant Network - VPC Plan and click Request.

Step 3

Enter the requested information in the fields.

Step 4

Click Submit.

Attaching an External L3 Network Internet Access

This section describes how to attach an external Layer 3 (L3) Network Internet Access.

Before you begin

- You can choose any name for the L3 policy.

- External L3 policy instance must be named [L3OutName|InstP].

Procedure

Step 1

Log in to the vRealize Automation as tenant, choose Catalog > Tenant Network service.

Step 2

Choose Attach or Detach L3 external connectivity to Network

Step 3

Choose Request.

Step 4

In the Request Information tab, enter a description of the request.

Step 5

Choose Next.

Step 6

In the Step tab, perform the following actions:

1. In the Rule Entry List field, enter the values and click Save.

This table shows the values for each Rule Entry:

Rule Entry List

Values

dstFormPort

- Blank

- Unspecified

- 1-65535

dstToPort

- Blank

- Unspecified

- 1-65535

protocol

- icmp

- icmpv6

- tcp

- udp

- Blank

etherType

- IP

- ARP

2. In the L3out Policy field, click Add to locate and choose the L3 connectivity policy in the common tenant. (default)

3. In the Network/EPG name field, click Add to locate and choose the network/EPG in the common tenant. (web-host)

4. In the EPG/Network plan type field, click Add to locate and choose the network/EPG in the common tenant. (web-host)

5. In the Operation field, click Add to add a Layer3 Out.

Step 7

To verify your request, choose the Requests tab.

1. Choose the request you submitted and click view details. Ensure the status is Succesful.

- Verify the Security and L3 Policy on the APIC

- Verifying the Network Connectivity

Verify the Security and L3 Policy on the APIC

This section describes how to verifying the security and Layer 3 (L3) policy on APIC.

Procedure

Step 1

Log in to Cisco APIC as the tenant, and then choose TENANTS > common.

Step 2

In the navigation pane, expand Tenant Common > Networking > Security Policies > Contracts.

1. Nested under Contracts there should be a new contract with the end_user_tenant name-L3ext_ctrct_network_name that you connected to. (green-L3ext_ctrct_web-hosts)

2. Expand the end_user_tenant name-L3ext_ctrct_network_name. (green-L3ext_ctrct_web-hosts)

3. Choose the end_user_tenant name-L3ext_ctrct_network_name. (green-L3ext_ctrct_web-hosts)

4. In the Property pane, in the Filter field, click the filter. (green-L3ext_filt_web-hosts)

5. In the Properties pane, you can see the filter is mapped to vRealize.

Step 3

In the navigation pane, expand Tenant Common > Networking > External Routed Networks > default > Networks > defaultInstP.

1. In the Properties pane, in the Provided Contracts field, you should see the end_user_tenant name-L3ext_ctrct_network_name. (green-L3ext_filt_web-hosts)

2. In the Consumed Contracts field, you should see the end_user_tenant name-L3ext_ctrct_network/EPG_name. (green-L3ext_filt_web-hosts)

Step 4

On the menu bar choose TENANTS > your_tenant .

Step 5

In the navigation pane, expand Tenant your_tenant > Application Profile > default > Application EPGs > EPG web-hosts > Contracts.

1. In the Contracts pane, you can verify the contract and consumes a contract is present.

Verifying the Network Connectivity

This section describes how to verify the network connectivity.

Procedure

Log in to the virtual machine (web-host), from the command line, ping the other VM.

Application Deployment Scenarios

The following table shows the supported deployment scenarios:

Deployment Scenario

Description

Web > L3out

Web Tier to L3 external connectivity policy connected using security policy (L3out configured in "default" VRF)

Web > Firewall > L3out

Web Tier with Firewall and L3out (L3out configured in "outside" VRF)

Web > Load Balancer > L3out

Web Tier with Load balancer connected to L3out (L3out configured in "default")

Web > Load Balancer and Firewall > L3out

Web Tier with Load balancer and Firewall service connected to L3out (L3out configured in "outside")

Application > Web

App tier to Web tier, connected using security policy

Database > Application

Db tier to App tier, connected using security policy

Application > Load Balancer > Web

App tier to Web tier using Load balancer. Traffic from Web tier towards App tier is load balanced.

Application > Firewall > Web

App tier to Web tier using firewall.

In a multi-tenant deployment, there are some restrictions in the service deployment configuration. The administrator must decide whether the applications in this deployment will use firewall services or a load balancer-only service at the first (web) tier.

The following table shows the supported combinations of services in the shared plan:

Deployment Type

FW + LB > L3out

LB only > L3out

FW > L3out

LB between EPGs

FW between EPGs

Firewall only or Firewall and Load balancer

Yes

Yes

Yes

Yes

Load Balancer only

Yes

Yes

In case of multi-tenancy, you should use a dedicated service device for each tenant.

About Property Groups

Property groups are a vRealize Automation (vRA) construct that provide virtual machine customization. Using property groups, vRA can invoke workflows in vRealize Orchestration (vRO) at given stage of virtual machine's life cycle. This virtual machine extension capability is used by Application Policy Infrastructure Controller (APIC) vRealize to invoke APIC vRA workflows and configure APIC policies.

APIC vRealize supports a number of application deployment scenarios. In a multi-tier application, the APIC security policy or the load balancing or firewall services can be inserted between tiers. This is achieved by the following steps:

1. Execute the Configure Property Group catalog-item in the Admin Services catalog to create a property group.

2. Use the Security Policy, Load Balancer, and Firewall tabs to customize the property group.

3. Enable the property group in the single-machine blueprint at the Infrastructure > Blueprints > Single Machine Blueprint level in vRealize.

About Service Blueprints

This section describes the service blueprints.

In vRealize there are two sets of blueprints one is a machine blueprints that is for compute for installing, setting up VMs, and spinning VMs. There is a single- and a multi-machine blueprint for launching single-tier application workload or multi-tier application workload that is called machine blueprint for networking workflows.

Admin workflow:

- Create APIC handles

- Create VMM domains

- Create Tenants

- Create subnets in common

- Create Layer 4-7 devices

Tenant workflow:

- Create EPGs

- Create contracts

- Provide contracts

- Consume contracts

- Consume L3Outs

- Consume Layer 4-7 devices

- Integration with vRealize Network Profiles (IPAM)

Integration with vRealize Network Profiles (IPAM)

vRealize IP address management (IPAM) uses the network profiles concept to assign a pool of addresses to one or more networks. You can assign network profiles to ACI backed networks in the same fashion as a regular vRealize network.

To integrate with the vRealize IPAM:

Procedure

Step 1

Ensure the subnet exists to the bridge domain.

See Add or Delete Subnets in Bridge Domain for Tenant-Common.

Step 2

Create a network profile.

See VMware's documentation for creating a network profile.

Step 3

This depends on if your blueprint generates a new network or not:

If you use the same network for each machine blueprint:

Under your vCenter reservation find the EPG (Network Path) and assign the network profile to it.

1. In the vCenter, navigate to Infrastructure > Reservations.

2. Find "Your Reservation", hover and click Edit.

3. Navigate to Network > Find desired Network Path (EPG), from the drop-down list, choose the Network Profile and click Ok.

If you generate a network per VM:

Add a property to your property group with the network profile as the value.

1. In the vCenter, navigate to Infrastructure > Blueprints > Property Groups.

2. Find "Your Blueprint", hover and click Edit.

3. Click + New Property.

4. Set the Name to "VirtualMachine.NetworkX.NetworkProfileName".

where X is the VM NIC number (in the range [0-9]).

5. Set the Value to the name of the Network Profile you created.

6. Click the green tick icon to confirm and click Ok.

New applications will be assigned an address from this pool.

Step 4

Use guest customizations to assign the IP address to the server.

See VMware's documentation for guest customizations.

Documentation of APIC Workflows in vRealize Orchestrator

To get documentation on the APIC methods and types, the vRO API search can be used.

1. Log in to the vRO GUI, choose Tools > API Search

2. Enter APIC.

This shows the list of all APIC methods and types.

- List of Methods in ApicConfigHelper Class

List of Methods in ApicConfigHelper Class

This section provides a list of methods in ApicConfigHelper class.

- This adds an APIC host to the repository and does a login to the APIC:

ApicHandle addHost(String hostName,
String hostIp0,
String hostIp1,
String hostIp2,
String  userName,
String pwd,
int port,
boolean noSsl,
String role,
String tenantName)

- This gets the APIC handle give the APIC name:

ApicHandle getApicHandle(String hostName)

- This gets the list of APIC handles for a given <role, username>:

List<ApicHandle> getApicHandleByRole(String role, String userName)

- This removes an APIC host from the repository:

boolean removeHost(String inApicName)

- This creates Tenant endpoint group and association to vmmDomain in APIC:

ApicResponse addNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String bdName,
String ctxName,
String subnet,
String domName,
boolean vmm,
boolean vpc,
boolean intraEpgDeny,
boolean allowUseg,
String  encapMode)

- This updates the domain of the endpoint group by adding or deleting:

ApicResponse updateNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String domName,
boolean vmm,
boolean add,
String encapMode)

- This adds or deletes subnets to the bridge domain in the virtual private cloud (VPC) tenant:

ApicResponse updateSubnets(ApicHandle handle,
String tenantName,
String bdName,
fvSubnet subnetList[],
boolean add)

- This adds or deletes the bridge domain to or from the tenant:

ApicResponse updateBD(ApicHandle handle,
String tenantName,
String bdName,
String ctxName,
boolean arpFlooding,
String l2UnknownUnicast,
String l3UnknownMulticast,
boolean add)

- This adds or deletes the context (Ctx) to or from the tenant:

ApicResponse updateCtx(ApicHandle handle,
String tenantName,
String ctxName,
boolean add)

- This adds or deletes the following based on add or delete:

ApicResponse addOrDeleteLBToNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String bdName,
String ctxName,
boolean vpc,
String planName,
String lbVendor,
String ldevName,
String graphName,
boolean sharedLb,
String protocol,
String port,
String consumerDn,
String snipIntAddress,
String snipIntNetMask,
String snipExtAddress,
String snipExtNetMask,
String snipNextHopGW,
boolean addOperation)

- This opens a connection to the URL, sends the postBody string to the URL location, and returns result:

ApicResponse addOrDelFWReq(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String ctrctName,
String graphName,
vzEntry entryList[],
String consumerDn,
boolean addOp,
boolean updateOp)

- This adds the firewall service to an endpoint group in the shared and VPC plan:

ApicResponse addFWToNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName,
boolean vpc,
String fwVendor,
String ldevName,
String graphName,
vzEntry entryList[],
String fwL3extExternal,
String fwL3extInternal,
boolean skipFWReq,
String consumerDn)

- This deletes the firewall from the endpoint group in the shared and VPC Plan:

ApicResponse deleteFWFromNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName,
boolean vpc,
String graphName,
String ctrctName,
String protocol,
String startPort,
boolean skipFWReq,
String consumerDn)

- This implements the REST API to APIC:

String apicRestApi(ApicHandle handle,
String apiUrl,
String method,
String postBody)

- This adds or deletes the router ID in a tenant:

ApicResponse addOrDelRouterId(ApicHandle handle,
String rtrId,
boolean addOp)

- This deletes the tenant endpoint group and the association:

ApicResponse deleteNetwork(ApicHandle handle,
String tenantName,
String apName,
String epgName)

- This creates the tenant, bridge domain and the context (Ctx) in APIC:

ApicResponse addTenant(ApicHandle handle,
String tenantName,
String bdName,
String ctxName,
String aaaDomain)

- This deletes the tenant in APIC:

ApicResponse deleteTenant(ApicHandle handle,
String tenantName)

- This adds VlaNS, vmmDomP, vmmCtrlP, vmmUsrAccp and required relation objects to the APIC:

ApicResponse addVmmDomain(ApicHandle handle,
String dvsName,
String vcenterIP,
String userName,
String passwd,
String datacenter,
String vlanPoolName,
int vlanStart,
int vlanEnd,
String aaaDomain)

- This deletes VlanNS and vmmDomP objects from the APIC:

ApicResponse deleteVmmDomain(ApicHandle handle,
String domName,
String vlanPoolName)

- This adds or deletes encap blocks in the VLAN pool:

ApicResponse updateVlanPool(ApicHandle handle,
String vlanPoolName,
fvnsEncapBlk encapList[])

- This adds the security policy (contract entry):

ApicResponse addSecurityPolicySet(ApicHandle handle,
String tenant,
String ap,
String srcEpg,
String dstEpg,
vzEntry entryList[],
boolean createFlg
)

- This updates the security policy (contract entry):

ApicResponse updateSecurityFilters(ApicHandle handle,
String tenant,
String filterName,
vzEntry entryList[]
)

- This adds or removes the consumer contract interface:

ApicResponse updateSharedSvcConsumer(ApicHandle handle,
String tenant,
String ap,
String consumerEpg,
vzBrCP contract,
boolean add
)

- This updates the security policy (contract entry):

ApicResponse updateL3outPolicy(ApicHandle handle,
String tenant,
String ap,
String dstEpg,
vzEntry entryList[],
l3extOut l3out,
boolean vpc,
boolean add
)

- This deletes all the security policy (contracts):

ApicResponse deleteSecurityPolicy(ApicHandle handle,
String tenant,
String ap,
String srcEpg,
String dstEpg
)

- This creates VIP address block in the tn-common:

ApicResponse addVipPool(ApicHandle handle,
String planName,
String addrStart,
String addrEnd)

- This deletes VIP address block in the tn-common:

ApicResponse deleteVipPool(ApicHandle handle,
String planName,
String addrStart,
String addrEnd)

- This adds or deletes the security domain associations:

ApicResponse updateVmmDomain(ApicHandle handle,
String domName,
aaaDomainRef aaaList[])

- This deletes a shared service provider (endpoint group) from a contract:

ApicResponse deleteSharedServiceProvider(ApicHandle handle,
String tenant,
String ap,
String srcEpg,
String dstEpg,
vzBrCP contract)

- This creates a Cisco AVS VMM domain and adds related objects to the APIC:

ApicResponse addAvsVmmDomain(ApicHandle handle,
String dvsName,
String aepName,
String vcenterIP,
String userName,
String passwd,
String dvsVersion,
String datacenter,
String mcastIP,
String poolName,
String rangeStart,
String rangeEnd,
String aaaDomain,
int domType,
String secondRangeStart,
String secondRangeEnd,
String secondPoolName)

- This updates the pools (VLAN, Multicast Address) relevant to a Cisco AVS VMM domain:

ApicResponse updateAvsVlanMcastPool(ApicHandle handle,
String poolName,
fvnsEncapBlk encapList[],
int poolType)

- This deletes a Cisco AVS VMM domain:

ApicResponse deleteAvsVmmDomain(ApicHandle handle,
String domName,
String poolName,
int poolType)

- This deletes a Cisco AVS VMM domain which is in mixed mode:

ApicResponse deleteAvsVmmDomainMixedmode(ApicHandle handle,
String domName )

- This creates Distributed Firewall for a Cisco AVS VMM domain:

ApicResponse createFWPol(ApicHandle handle,
String polName,
String vmmName,
String polMode,
String pInterval,
String logLevel,
String adminState,
String destGrpName,
String inclAction,
int caseVal)

- This updates Distributed Firewall association with a Cisco AVS VMM domain:

ApicResponse updateFWPolMapping(ApicHandle handle,
String polName,
String vmmName,
Boolean opValue)

- This deletes Distributed Firewall:

ApicResponse deleteFWPol(ApicHandle handle,
String polName)

- This adds or deletes attribute(s) for a Microsegment EPG:

ApicResponse addOrDelUsegAttr(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String criteriaName,
fvVmAttrV addFvVmAttrList[],
fvMacAttr addFvMacAttrList[],
fvIpAttr addFvIpAttrList[],
fvVmAttr delFvVmAttrList[],
fvMacAttr delFvMacAttrList[],
fvIpAttr delFvIpAttrList[])

- This adds a microsegment EPG:

ApicResponse addUsegEpg(ApicHandle handle,
String tenantName,
String apName,
String epgName,
String bdName,
String ctxName,
String subnet,
String domName,
String criteriaName,
boolean vmm,
boolean vpc,
boolean intraEpgDeny,
fvVmAttrV fvVmAttrList[],
fvMacAttr fvMacAttrList[],
fvIpAttr fvIpAttrList[],
String encapMode)

Writing Custom Workflows Using the APIC Plug-in Method

This section describes how to write custom workflows using the Application Policy Infrastructure Controller (APIC) plug-in method. Tenants might have unique requirements for their logical network topology that are not covered by the out-of-box designs. Existing Cisco APIC workflows can be combined together into a custom workflow that enables limitless network designs.

All workflows expect a set of input parameters, and workflows that create new objects will export a set of output parameters. Output parameters can be chained into the input parameter of the next workflow.

The following example procedure creates a custom workflow that builds a new network, and then directly passes the newly created network into the input of the attach Layer 3 workflow.

Procedure

Step 1

Log in to the vRealize Orchestrator.

Step 2

Switch to the Design mode.

Step 3

In the Navigation pane, create a folder named "Custom Workflows".

Step 4

Choose the Custom Workflows folder.

Step 5

In the Work pane, click the New workflow button.

Step 6

In the Workflow name dialog box, enter a name for the workflow.

Example:

Create_Network_Attach_L3

Step 7

Click OK.

Step 8

Choose the Schema tab.

Step 9

In the Navigation pane, expand All Workflows > Administrator > Cisco APIC workflows > Tenant Shared Plan

Step 10

Drag and drop Add Tenant Network - Shared Plan onto the blue arrow in the Work pane.

Step 11

In the Do you want to add the activity's parameters as input/output to the current workflow? dialog box, click Setup....

Step 12

In the Promote Workflow Input/Output Parameters dialog box, click Promote.

Leave all of the values at their defaults.

Step 13

In the Navigation pane, expand All Workflows > Administrator > Cisco APIC workflows > Advanced Network Services.

Step 14

Drag and drop Attach or Detach L3 external connectivity to Network onto the blue arrow that is to the right of the Add Tenant Network object in the Work pane.

Step 15

In the Do you want to add the activity's parameters as input/output to the current workflow? dialog box, click Setup....

Step 16

In the Promote Workflow Input/Output Parameters dialog box, click Promote.

Leave all of the values at their defaults.

Step 17

Choose the Inputs tab.

The screen displays the inputs for the workflow. You can verify that the inputs are all exposed and that the created endpoint group is an output parameter.

Step 18

Choose the Schema tab.

Step 19

In the Work pane, click Validate to verify that the custom workflow is valid.

Step 20

Click Close.

Step 21

Click Run to test the workflow.

Step 22

In the Start Workflow dialog box, click Submit to start the workflow.

Multi-Tenancy and Role based Access Control Using Security Domains

APIC and vRA both supports multi-tenancy natively. vRA tenant user is mapped one-to-one with a APIC tenant user and thus Tenant names need to match exactly on both systems.

For every vRA tenant, APIC admin needs to ensure that an user account and required security domains and roles are created in APIC as part of Day-0 operation.

As a next step, vRA-Admin would execute Add Tenant service blueprint (part of Admin catalog), to create/update Tenant in APIC and associate it with the right security Domain. For eg: Tenant-Green on vRA is mapped to Tenant-Green in APIC with association to Security Domain "Domain-Green" enabled for "User-Green".

By associating tenant to right security domains, Role based access control is enforced and it allows for granular as well stricter Tenant policy enforcement.

- Adding the Tenant

- Deleting the Tenant

Adding the Tenant

This section describes how to add the tenant.

In this blueprint, a tenant identified by input parameter "Tenant" is created in APIC with association the security domain that is provided as second input.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

Step 2

Choose Add Tenant, enter the information in the fields and click Submit.

Deleting the Tenant

This section describes how to delete the tenant from APIC.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

Step 2

Choose Delete Tenant, enter the information in the fields and click Submit.

APIC Credentials for Workflows

As part of ACI-integration with vRA, this release supports pairing up vRA with a ACI fabric managed by a APIC-cluster.

The network service blueprints are categorized into Admin and Tenant workflows and accordingly vRA admin has to setup APIC connection handles for APIC-Admin credential as well as APIC-Tenant credential for every vRA-Tenant.

As part of plug-in, the right handles (Admin vs Tenant) are auto-selected implicitly based on the workflow context and the privileges needs to create and managed objects in APIC. This provides stronger access control and isolation among tenants.

- Adding APIC with Admin Credentials

- Adding APIC with Tenant Credentials

Adding APIC with Admin Credentials

This section describes how to add APIC with admin credentials.

All the blueprints and workflows that are part of catalog items in Admin portal are performed using the Admin-credential.

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Network Security.

Step 2

Choose Add APIC with Admin Credentials, enter the information in the fields and click Submit.

Step 3

To access APIC using certificates, set the "Use certificate authentication" to yes and enter the Certificate Name and Private Key parameters.

Adding APIC with Tenant Credentials

This section describes how to using tenant admin credentials (security domain).

Procedure

Step 1

Log in to the vRealize Automation as admin, choose Catalog > Admin Services.

Step 2

Choose Add APIC with Tenant credentials, enter the information in the fields and click Submit.

Step 3

To access APIC using certificates, set the "Use certificate authentication" to yes and enter the Certificate Name and Private Key parameters.

Troubleshooting

This section describes the troubleshooting techniques.

- Collecting the Logs to Report

- Installing the ACI Helper Scripts

Collecting the Logs to Report

This section describes how to collect the log files from the vRealize Appliance to report.

Procedure

To collect the log files, enter the following commands:

tar xvfz apic-vrealize-1.2.1x.tgz
cd apic-vrealize-1.2.1x
cd scripts/
./get_logs.sh
Usage:  get_logs.sh  [-u] [-p <password>] [-s <vra_setup>]
-p    password (can be skipped for default passwd)
-s    vra_setup
-u    un-compress (ie., don't create .tar.gz file)

Example:
./get_logs.sh –p ***** -s vra-app
…
VMware vRealize Automation Appliance
Compressing Logs
logs/
logs/app-server/
logs/app-server/catalina.out
logs/app-server/server.log
logs/configuration/
logs/configuration/catalina.out
Logs saved in vra_logs_201511251716.tar.gz

Installing the ACI Helper Scripts

This section describes how to install the helper scripts. The ACI helper scripts provide the following:

- Restarts the vco-server and vco-configurator

- Uninstalls the APIC plug-in

Procedure

To install the helper scripts, enter the following commands:

cd scripts
./install_apic_scripts.sh
Usage:  install_apic_scripts.sh  [-p <password>] [-s <vra_setup>]
-p    password
-s    vra_setup

Example:
./install_apic_scripts.sh -p ***** -s vra-app
Copying APIC scripts 'rmapic', 'restart' to vra: vra-app

Removing the APIC Plug-in

This section describes how to remove the APIC plug-in.

Procedure

Step 1

Log into the VMware vRealize Orchestrator as administrator.

Step 2

Run the Delete APIC workflow for all APIC handles.

Step 3

Install the ACI helper scripts, which can be found in Installing the ACI Helper Scripts.

Step 4

Log in to the VRA appliance as root, using SSH:$ssh root@vra_ip.

Step 5

Change the permissions to the rmapic bash script to be executable:

$ chmod a+x rmapic

Step 6

Execute the rmapic bash script to remove the APIC plug-in:

$ ~/rmapic

Step 7

To verify that the plug-in has been uninstalled, log in to the VMware appliance using the Firefox browser: https://appliance_address:8283/vco-controlcenter

Step 8

Under the Plug-Ins section, click Manage Plug-Ins.

Step 9

Verify that the Cisco APIC Plug-in is no longer listed under Plug-In.

Plug-in Overview

vRA Blueprints input parameters

vRO Javascript Object Name

APIC Managed Object Name

Tenant

ApicTenant

com.cisco.apic.mo.fvTenant

Bridge Domain

ApicBridgeDomain

com.cisco.apic.mo.fvBD

VRF

ApicL3Context

com.cisco.apic.mo.fvCtx

Tenant Network (EPG)

ApicEPG

com.cisco.apic.mo.fvAEPg

Security Policy (Contracts)

ApicSecurityPolicy

com.cisco.apic.mo.vzBrCP

Security Filters

ApicSecurityFilter

com.cisco.apic.mo.vzFilter

Security Rules

ApicSecurityRule

com.cisco.apic.mo.vzEntry

AAA Domain

ApicAAADomain

com.cisco.apic.mo.aaaDomain

VMM Domain

ApicVmmDomain

com.cisco.apic.mo.vmmDomP

VMM Controller

ApicVmmController

com.cisco.apic.mo.vmmCtrlrP

Physical Domain

ApicPhysicalDomain

com.cisco.apic.mo.physDomP

L4-L7 Device Cluster

ApicLogicalLBDevice

com.cisco.apic.mo.vnsLDevVip

L3 external connectivity

ApicL3Connectivity

com.cisco.apic.mo.l3extOut

Configuring a vRA Host for the Tenant in the vRealize Orchestrator

This section describes how to configure a vRA host for the tenant in the vRealize Orchestratorr (vRO).

Note

There will be one vRA host handle already created by default. This is for the global tenant and is used for administration purposes and to create the IaaS host handle.

Procedure

Step 1

Log in to the VMware vRealize Orchestrator as administrator.

Step 2

Once the VMware vRealize Ochestrator GUI appears, from the drop-down list, choose Run from the menu bar.

Step 3

In the Navigation pane, choose the Workflows icon.

Step 4

Choose Adminstrator@vra_name > Library > vRealize Automation > Configuration > Add a vRA host.

Step 5

Right-click Add a vRA host and choose Start Workflow.

Step 6

In the Start Workflow: Add a vRA host dialog box, perform the following actions:

1. In the Host Name field, enter the host's name.

2. In the Host URL field, enter the host's URL.

3. For Autotmatically install SSL certificates, choose Yes.

4. In the Connection timeout field, enter "30".

5. In the Operation timeout field, enter "60".

6. For Session Mode, choose Shared session.

7. In the Tenant field, enter the tenant's name.

8. In the Authentication username field, enter your tenant administrator username.

9. In the Authentication pwd field, enter your tenant administrator password.

10. Click Submit.

Configuring an IaaS Host in the vRealize Orchestrator

This section describes how to configure an IaaS host in the vRealize Orchestratorr (vRO).

Procedure

Step 1

Log in to the VMware vRealize Orchestrator as administrator.

Step 2

Once the VMware vRealize Ochestrator GUI appears, from the drop-down list, choose Run from the menu bar.

Step 3

In the Navigation pane, choose the Workflows icon.

Step 4

Choose Adminstrator@vra_name > Library > vRealize Automation > Configuration > Add the Iaas host of a vRA host.

Step 5

Right-click Add the Iaas host of a vRA host and choose Start Workflow.

Step 6

In the Start Workflow: Add the Iaas host of a vRA host dialog box, perform the following actions:

1. In the vRA Host drop-down list, choose the default vRA host that was created by the system. Do not choose the tenant handle.

2. In the Host Name field, leave the auto-filled name as is.

3. In the Host URL field, enter the vRA host's URL.

4. In the Connection timeout field, enter "30".

5. In the Operation timeout field, enter "60".

6. For Session Mode, choose Shared session.

7. In the Authentication username field, enter your IaaS administrator username.

8. In the Authentication pwd field, enter your IaaS administrator password.

9. In the Workstation for NTLM authentication field, enter your IaaS host name.

10. In the Domain for NTLM authentication field, enter your IaaS domain name.

11. Click Submit.

Chapter 10. Cisco ACI vCenter Plug-in

This chapter contains the following sections:

- About Cisco ACI with VMware vSphere Web Client

- Cisco ACI vCenter Plug-in Features and Limitations

- Upgrading VMware vCenter when Using the Cisco ACI vCenter Plug-in

- Uninstalling the Cisco ACI vCenter Plug-in

- Upgrading the Cisco ACI vCenter Plug-in

- Troubleshooting the Cisco ACI vCenter Plug-in Installation

About Cisco ACI with VMware vSphere Web Client

The Cisco ACI vCenter plug-in is a user interface that allows you to manage the ACI fabric from within the vSphere Web client.

This allows the VMware vSphere Web Client to become a single pane of glass to configure both VMware vCenter and the ACI fabric.

The Cisco ACI vCenter plug-in empowers virtualization administrators to define network connectivity independently of the networking team while sharing the same infrastructure.

No configuration of in-depth networking is done through the Cisco ACI vCenter plug-in. Only the elements that are relevant to virtualization administrators are exposed.

- Cisco ACI vCenter Plug-in Overview

Cisco ACI vCenter Plug-in Overview

The Cisco Application Centric Infrastructure (ACI) vCenter plug-in for the VMware vSphere Web Client, adds a new view to the GUI called Cisco ACI Fabric.

The Cisco Application Centric Infrastructure (ACI) vCenter plug-in does not change existing integration of ACI with vCenter, it allows you to configure an EPG, uSeg EPG, contract, tenant, VRF, and bridge domain from the VMware vSphere Web Client.

Cisco Application Centric Infrastructure (ACI) vCenter plug-in is stateless, fetches everything from Application Policy Infrastructure Controller (APIC) and does not store any information.

The following is a brief overview of the features provided by Cisco ACI vCenter plug-in:

For more detailed information, see Cisco ACI vCenter Plug-in Features and Limitations.

The Cisco ACI vCenter plug-in provides the possibility to create, read, update and delete (CRUD) the following object on the ACI Fabric:

- Tenant

- Application Profile

- EPG / uSeg EPG

- Contract

- VRF

- Bridge Domain

The Cisco ACI vCenter plug-in also provides a more limited operation regarding the usage of L2 and L3 Out, where all of the advanced configuration needs to be done in APIC beforehand.

- Preconfigured L2 and L3 Out can be used as providers or consumers of a contract.

- Cannot be created, edited or deleted.

The Cisco ACI vCenter plug-in also allows to consume preconfigured L4-L7 Services, by applying existing graph template to a Contract.

- Can use existing graph templates, not create them.

- Only empty mandatory parameter of the function profile will be displayed and configurable.

The Cisco ACI vCenter plug-in also has troubleshooting capabilities:

- Endpoint to endpoint sessions (Faults, Audits, Events, Stats, Contract, Traceroute )

Cisco ACI vCenter Plug-in Software Requirements

The Cisco ACI vCenter plug-in Software Requirements:

Platform Series

Recommended Release

vCenter

Cisco APIC supports any version of Linux Appliance and Windows Server that VMware supports. See VMware documentation for details.

Application Policy Infrastructure Controller (APIC)

Release 3.2(2) and later

Required APIC Configuration

This sections describes the required APIC configuration.

At least one VMM domain should already exists between the APIC and the vCenter where the plug-in is being installed.

For more information, see the Cisco Application Centric Infrastructure Fundamentals Guide.

Installing the Cisco ACI vCenter Plug-in

This section describes how to install the Cisco Application Centric Infrastructure (ACI) vCenter plug-in. You must have working HTTPS traffic between your VMware vCenter and Cisco Application Policy Infrastructure Controller (APIC). That is because VMware vCenter downloads the plug-in directly from the Cisco APIC.

If you cannot enable HTTPS traffic between your VMware vCenter and Cisco APIC, and you wish to use your own web server to host the Cisco ACI vCenter plug-in zip file, see the Alternative Installation of the Cisco ACI vCenter Plug-in.

If you are using VMware vCenter 5.5 (Update 3e or later) or vCenter 6.0 (Update 2 or later), follow the procedure in this section. If you are using an earlier release of vCenter 5.5 or 6.0, see the Alternative Installation of the Cisco ACI vCenter Plug-in.

To install a plug-in, the vCenter must download the plug-in from a Web server. In the following procedure, the Cisco APIC is used as the Web server, and the VMware vCenter downloads the plug-in directly from the Cisco APIC.

Before vCenter 5.5 Update 3e or vCenter 6.0 Update 2, vCenter uses TLSv1 for the HTTPS communication, which is now obsolete. For security reasons Cisco APIC only supports TLSv1.1 and TLSv1.2, therefore the vCenter will not be able to download the plug-in from the Cisco APIC. The plug-in must be put on a separate Web server, that allows TLSv1 or that does not use HTTPS.

Note

If you log out of VMware vCenter 6 .7 and then log back in, you may not see the vCenter plug-in icon. If that occurs, clear the cookies and history or log in using another browser.

Before you begin

- Make sure all of the prerequisites are met.

For more information, see the Cisco ACI vCenter Plug-in Software Requirements and Required APIC Configuration sections.

- Ensure HTTPS traffic is allowed between your vCenter server and APIC.

- If you are installing the Cisco ACI vCenter plug-in for VMware vCenter 6.7, you need PowerCLI version 11.2.0 or later.

Note

During installation, you may see the following error on the console:

Error: Invalid server certificate. Use Set-PowerCLIConfiguration to set the value for the InvalidCertificationAction option to Prompt if you'd like to connect once or to add a permanent exception for this server.
To avoid seeing this error, enter the following command before installation:Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false

Procedure

Step 1

Go to the following URL:

Example:

https://<APIC>/vcplugin

Step 2

Follow the instructions on that web page.

Connecting the Cisco ACI vCenter Plug-in to your Cisco ACI Fabric

This section describes how to connect the Cisco Application Centric Infrastructure (ACI) vCenter plug-in to your Cisco ACI fabric.

Note

- The registration is VMware vCenter-wide and it does not take into account the user that performs it. It is a configuration for the whole VMware vCenter, not just for the logged-in user that performs it.

- Role Based Access Control (RBAC) is based on the credentials used upon registration. Permission of the Cisco Application Policy Infrastructure Controller (APIC) account used for the registration defines configuration restriction on the Cisco ACI vCenter plug-in.

You can connect the plug-in to your Cisco ACI fabric, using one of the following ways:

Connect the Cisco ACI vCenter plug-in to your Cisco ACI fabric using credentials.

For more information, see Connecting vCenter Plug-in to the Cisco ACI Fabric Using Credentials.

Connect the Cisco ACI vCenter plug-in to your Cisco ACI fabric using an existing certificate.

For more information, see Connecting vCenter Plug-in to your ACI Fabric Using an Existing Certificate.

Connect the Cisco ACI vCenter plug-in to your Cisco ACI fabric by creating a new certificate.

For more information, see Connecting vCenter Plug-in to your ACI Fabric by Creating a New Certificate.

- Connecting vCenter Plug-in to the Cisco ACI Fabric Using Credentials

- Connecting vCenter Plug-in to your ACI Fabric Using an Existing Certificate

- Connecting vCenter Plug-in to your ACI Fabric by Creating a New Certificate

Connecting vCenter Plug-in to the Cisco ACI Fabric Using Credentials

This section describes how to connect the Cisco Application Centric Infrastructure (ACI) vCenter plug-in to your Cisco ACI fabric using credentials.

Before you begin

Ensure the Cisco ACI vCenter plug-in is installed. For more information, see Installing the Cisco ACI vCenter Plug-in.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Cisco ACI Fabric.

Step 3

In the Getting Started pane, choose Connect vSphere to your ACI Fabric.

Step 4

In the Register a new ACI Fabric dialog box, click Yes to register a new ACI fabric.

Step 5

In the Register a new APIC Node dialog box, perform the following actions:

1. In the IP/FQDN field, enter the IP address or the fully qualified domain name (FQDN).

2. In the Use Certificate field, do not put a check in the Use Certificate check box to use Cisco Application Policy Infrastructure Controller (APIC) authentication.

3. In the Username field, enter the user name (admin).

4. In the Password field, enter the password.

5. Click OK.

Step 6

In the Information dialog box, click OK.

The Cisco APIC node was successfully added to the Cisco ACI fabric.

Step 7

In the ACI Fabric pane, you will see the new registered Cisco APIC discover the other Cisco APICs.

The Cisco ACI vCenter plug-in always uses a single Cisco APIC for its requests. However, it switches the Cisco APIC if the Cisco APIC currently used is no longer available.

Note

Registering the Cisco ACI fabric with the Cisco ACI vCenter plug-in is not supported for remote users.

Connecting vCenter Plug-in to your ACI Fabric Using an Existing Certificate

This section describes how to connect the vCenter plug-in to your ACI fabric using an existing certificate.

Before you begin

- A certificate is already setup on the APIC for the admin user.

- You have the name and private key of the certificate.

Procedure

Step 1

Log in to the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Cisco ACI Fabric.

Step 3

In the Getting Started pane, choose Connect vSphere to your ACI Fabric.

Step 4

In the Register a new ACI Fabric dialog box, click Yes to register a new ACI fabric.

Step 5

In the Register a new APIC Node dialog box, perform the following actions:

1. In the IP/FQDN field, enter the IP address or the fully qualified domain name (FQDN).

2. In the Use Certificate field, check the Use Certificate check box.

Step 6

In the Action section, choose Use an existing certificate.

Step 7

In the Name field, enter the certificate name.

Step 8

In the Private Key section, paste the private key of the certificate.

Step 9

Click Check Certificate.

The status switches to Connection Success.

Note

If connection failure is displayed, check that the certificate name and private key are correct, and try again.

Step 10

Click OK.

Step 11

In the Information dialog box, click OK .

The APIC node was successfully added to the ACI fabric.

Step 12

In the ACI Fabric pane the newly registered APIC discovers the other APICs.

The Cisco ACI vCenter plug-in always uses a single APIC for its requests. If the currently used APIC is no longer available, the Cisco ACI vCenter plug-in switches APICs.

Connecting vCenter Plug-in to your ACI Fabric by Creating a New Certificate

This section describes how to connect the vCenter plug-in to your ACI fabric by creating a new certificate.

Before you begin

- Ensure the plug-in is installed.

- You have access to the APIC admin credentials.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Cisco ACI Fabric.

Step 3

In the Getting Started pane, choose Connect vSphere to your ACI Fabric.

Step 4

In the Register a new ACI Fabric dialog box, click Yes to register a new ACI fabric.

Step 5

In the Register a new APIC Node dialog box, perform the following actions:

1. In the IP/FQDN field, enter the IP address or the fully qualified domain name (FQDN).

2. In the Use Certificate field, check the Use Certificate check box.

Step 6

In the Action field, choose Generate a new certificate.

Step 7

In the Name field, enter the new certificate name.

Step 8

Click the Generate certificate button.

Step 9

Copy the displayed certificate.

From -----BEGIN CERTIFICATE----- included, to -----END CERTIFICATE----- included.

Step 10

Add this certificate to the admin user in APIC. Make sure to use the same certificate name.

1. Log into the APIC GUI as admin.

2. On the menu bar, choose Admin.

3. In the Navigation pane, expand Security Management > Local Users > admin.

4. In the Work pane, in the User Certificate section, click the plus icon to add the certificate.

5. In the Name field, enter the certificate name.

6. In the Data field, paste the certificate content that you copied in step 8.

7. Click Submit.

Step 11

In the vCenter plug-in, click Check Certificate.

The status changes to Connection Success.

Note

If a Connection Failure message displays, check that the certificate is correctly added on the APIC and that the certificate names are the same.

Step 12

Click OK.

Step 13

In the Information dialog box, click OK.

The APIC node is successfully added to the ACI fabric.

Step 14

In the ACI Fabric pane, the newly registered APIC discovers the other APICs.

The Cisco ACI vCenter plug-in always uses a single APIC for its requests. If the currently used APIC is no longer available, the Cisco ACI vCenter plug-in switches APICs.

Cisco ACI vCenter Plug-in Features and Limitations

This section describes the possible operations provided by the Cisco ACI vCenter plug-in, for all object types it manages. It also goes over intentional configuration limitations.

For more information about the objects, see the Cisco Application Centric Infrastructure Fundamentals Guide.

Tenants

The Cisco ACI vCenter plug-in allows CRUD operations on the Tenant object. The following attributes are exposed in the plug-in:

- Name: The name of the tenant.

- Description (Optional): The description of the tenant.

When a tenant is created by the plug-in, a VRF <tenant_name>_default and a Bridge Domain <tenant_name>_default connected to that VRF are automatically created inside. An Application Profile <tenant_name>_default is also created inside it.

The infrastructure Tenant (infra) and the management Tenant (mgmt) are not exposed in the plug-in.

Note

The tenants visible in the plug-in will also depends on the permissions associated with the account used while registering the ACI fabric into the plug-in.

Application Profiles

The Cisco ACI vCenter plug-in allows CRUD operations on the Application Profile objects. The following attributes are exposed in the plug-in:

- Name: The name of the Application Profile.

- Description (Optional): The description of the Application Profile.

Endpoint Groups

The Cisco ACI vCenter plug-in allows CRUD operations on the Endpoint Group objects. The following attributes are exposed in the plug-in:

- Name: The name of the Endpoint Group.

- Description (Optional): The description of the Endpoint Group

- Bridge Domain: The Bridge Domain associated with this Endpoint Group.

- Intra-EPG Isolation: This allows to deny all traffic between the virtual machines that are connected to an EPG. By default, all virtual machines in the same EPG can talk to each other.

- Distributed Switch: The DVS/Cisco AVS where the EPG is deployed. This correspond to the association with a VMM domain in ACI

By default, all EPGs created with the plug-in are associated with the VMM Domain pointing to the vCenter where the plug-in is used. If there are multiple VMM Domains pointing to the same vCenter, you must choose at least one, in the form of selected on which DVS to deploy the EPG.

Allow microsegmentation (only for DVS, not Cisco AVS): This allows you to create a “Base EPG” . All the virtual machines connected to this EPG are candidates to apply microsegmentation rules of a uSeg EPG. Microsegmented EPG rules only applies to virtual machine that are connected to a “Base EPG” .

Note

All EPGs are considered as base EPGs if the distributed switch is Cisco AVS.

An EPG linked to a VMM domain pointing to the vCenter where the plug-in is being used is displayed as "Virtual." Other EPGs are displayed as "Physical."

Update and Delete actions are only authorized for EPGs linked to a VMM domain that is pointing to the vCenter (Virtual). Others EPGs (Physical) are read-only. Updates are still authorized to make EPGs consume or provide contracts, regardless of their VMM domain.

uSeg EPGs

The Cisco ACI vCenter plug-in allows CRUD operations on the mircosegemented EPG objects. The following attributes are exposed in the plug-in:

- Name: The name of the microsegmented EPG.

- Description (Optional): The description of the microsegmented EPG.

- Bridge Domain: The Bridge Domain associated with this microsegmented EPG.

- Intra-EPG Isolation: This allows to deny all traffic between the virtual machines that are connected to an EPG. By default, all virtual machines in the same EPG can talk to each other.

- Distributed Switch: The DVS/Cisco AVS where the EPG is deployed. This correspond to the association with a VMM domain in ACI

By default, all EPGs created with the plug-in are associated with the VMM Domain pointing to the vCenter where the plug-in is used. If there are multiple VMM Domains pointing to the same vCenter, you must choose at least one, in the form of selected on which DVS to deploy the EPG.

- Miro-segmentation attributes: List of rules that decide which VM belongs to this microsegmented EPG. Rules options include: IP, MAC, VM name, OS, Host, VM id, VNic, Domain, Data Center, Custom Attribute.

Note

Domain attributes (VMM Domain) only allow you to select VMM domains to the local vCenter. You choose a domain by selecting the corresponding DVS/Cisco AVS.

Custom attributes can only be chosen. They cannot be set by the plug-in. They must be set by the VMware vSphere Client. To create custom labels, see the documentation on the VMware website.

L2 and L3 External Networks

Layer 2 and Layer 3 External Networks must be created and configured on the APIC by the network administrator. They are read-only on the vCenter plug-in.

The only plug-in operations permitted on these objects are to make them consume or provide contracts.

The visible information for an L3 External Network is:

- Name: The name of the L3 External Network

- Subnets: External subnets represented by this L3 external network

- VRF: The VRF this L3 External Network belongs to

- Connected Bridge Domains: The Bridge Domains connected to this L3 External Network

The visible information for an L2 External Network is:

- Name: The name of the L2 External Network

- Bridge Domain: The bridge domain associated with this Bridge Domain

- VLAN ID: The VLAN ID associated with this L2 External Network

VRF

The Cisco ACI vCenter plug-in allows CRUD operations on the VRF objects. The following attributes are exposed in the plug-in:

- Name: The name of the VRF

- Description (Optional): The description of the VRF

- Enforce policies: Determine if the contracts need to be enforced for the EPG in this VRF.

Bridge Domains

The Cisco ACI vCenter plug-in allows CRUD operations on the Bridge Domain objects. The following attributes are exposed in the plug-in:

- Name: The name of the Bridge Domain

- Description (Optional): The description of the Bridge Domain

- Private Subnets: List of gateways for this Bridge Domain.

Note

- Shared and advertised subnets are read only. They cannot be configured by the plug-in. Only the private subnets can be added or deleted.

- If the Bridge Domain has been connected to an L3/L2 Out by the APIC, it cannot be deleted.

Contracts

The Cisco ACI vCenter plug-in allows CRUD operations on the Contract objects. The following attributes are exposed in the plug-in:

- Name: The name of the contract

- Description (Optional): The description of the contract.

- Consumers: The consumers for the contract (EPG, uSeg EPGs, L2/L3 External Networks)

- Providers: The providers for the contract (EPG, uSeg EPGs, L2/L3 External Networks)

- Filters: List of filters associated with the contract

- Apply both direction: Indicate if the specified Filters are applying only from consumers to providers or also from providers to consumers.

- L4-L7 Graph Template: It is possible to associate existing graph template to a Contract. See L4-L7 Service section below.

Note

- Subject is not exposed. The plug-in only manages contracts with a single subject. Contracts with multiple subjects are seen, but not editable.

- If the consumer and the contract are not in the same tenant, a contract interface is automatically created (named to_Tenant-name_contract-name).

Filters

The Cisco ACI vCenter plug-in allows CRUD operations on the Filter objects. All parameters from the APIC are exposed.

L4-L7 Services

- L4-L7 services can only be added on contracts that have a single provider.

- The graph template cannot be created by the plug-in (only consume existing graph templates)

- The graph template must be configured so that it contains:

- Association with devices

- Association with a function profile

- Only support graph templates with a maximum of two nodes

- The Function Profile folders naming and hierarchy must be valid as the plug-in does not allow folder manipulation.

- Only empty mandatory parameters of the function profile are editable by the plug-in.

- Graph connectors can be configured.

- All parameters from the APIC are exposed

- You can only consume redirect policies, if needed, not create them

Troubleshooting

- Only endpoint to endpoint troubleshooting sessions are supported.

- You can choose an existing session or create a new one

- The physical topology (spine / leaf) is not displayed.

- The topology display is VM-centric, focusing on Host, VM, vNIC, and the EPG the vNICs connect to

- Available information in a session:

- Faults

- Contracts: A table listing all the Contract/Filters/Entries between the two EPGs (hit counts are not displayed)

- Drop/Stats

- Audits/Events

- Traceroute

- Atomic Counter and SPAN are not available

- A more basic troubleshooting tool is available between objects that are not endpoints (VM, EPG, L3 Out), that only display configured contracts between two selected objects.

- A view of VMs and their connection to EPGs is available.

- For a given VM, it is possible to view the EPGs to which its VNICs are connected.

- If a L4-L7 connecter is used as source or destination of a troubleshooting session, then it is expected to get the following error on the Contract section of the troubleshooting wizard:

The feature required the source and destination endpoint to both be part on an EPG.

You can safely ignore the error message.

Cisco AVS Installation and Upgrade

The Cisco ACI vCenter plug-in enables you to install, uninstall, upgrade, or downgrade Cisco AVS from the vSphere Web Client:

- Once the vCenter plug-in is connected to the ACI fabric, it allows you to see all the Cisco AVS domains present on Cisco APIC, and to install, uninstall, upgrade, or downgrade Cisco AVS for some or all of the hosts in the data center associated with the Cisco AVS domains.

- New versions of Cisco AVS that have been downloaded from Cisco.com can be uploaded to the vCenter using the GUI. These versions can then be installed on the hosts in a given domain.

- You can see all hosts if they are connected to a given Cisco AVS domain. You also can see the hosts' OpFlex Agent status and the current version of Cisco AVS, if installed.

When installing or upgrading Cisco AVS, the vCenter plug-in automatically performs the following steps on a ESXi host:

1. Places the host into maintenance mode.

2. Uploads the appropriate VIB file to the host data store.

3. Installs or reinstalls Cisco AVS software.

4. Deletes the VIB file from the host data store.

5. Takes the host out of maintenance mode.

Note

- The vCenter plug-in only installs or uninstalls Cisco AVS VIBs on the hosts; you need to manually connect or disconnect the host to the Cisco AVS switch.

- If the host is part of an HA/DRS cluster, when the host is placed in maintenance mode, the VMs will be migrated automatically. If the VMs can’t be migrated automatically, you need to migrate them or turn off all the VMs on the host for the installation or upgrade to succeed.

For more information see the following sections in the Cisco AVS Installation Guide:

- "Installing Cisco AVS Using the VMware vCenter Plug-in"

- "Upgrading or Downgrading Cisco AVS Using the VMware vCenter Plug-in"

- "Uninstalling Cisco AVS using the VMware vCenter Plug-in"

- Role-based Access Control for Cisco ACI vCenter Plug-in

- Recommended RBAC Configuration for Cisco ACI vCenter Plug-in

Role-based Access Control for Cisco ACI vCenter Plug-in

Starting with Cisco APIC Release 3.1(1), the Cisco ACI vCenter plug-in supports enhanced role-based access control (RBAC) based on Cisco APIC user roles and security domains.

The UI of the Cisco ACI vCenter plug-in reflects the read and write privileges of Cisco APIC users. For example, if the user tries to access contract features but does not have read privilege for contracts, a gray screen displays with message saying the user does not have permission. A user who does not have write privileges sees a disabled link or action.

Setting Read and Write Roles

The following table describes how each privilege should be set for read and write roles in order to enable or disable the different features of Cisco ACI vCenter plug-in RBAC.

Note

You must create Cisco APIC roles and associate them when assigning a security domain to a user or users. You also must add security domains to any tenant the user will have access to.

Cisco ACI vCenter Plug-in RBAC Privileges

Roles

Workflow

Limited Read Role

Write Role

Mandatory settings for all roles

vmm-connectivity and vmm-ep

Application Profile

List

tenant-network-profile or tenant-epg

Create/Delete

tenant-network-profile

EPG

List

tenant-epg, tenant-connectivity-l2, and tenant-connectivity-l3

Create/Delete

tenant-connectivity-l2 and tenant-connectivity-l3

tenant-epg

VRF

List

tenant-connectivity-l2 and tenant-connectivity-l3

Create/Delete

tenant-connectivity-l2 and tenant-connectivity-l3

Bridge Domain

List BD

tenant-connectivity-l2 and tenant-connectivity-l3

Create/Delete BD

tenant-connectivity-l2 and tenant-connectivity-l3

List BD Subnet

tenant-connectivity-l2 and tenant-connectivity-l3

Create/Delete BD Subnet

tenant-connectivity-l2 and tenant-connectivity-l3

Contract

List Contract

tenant-security and tenant-epg

Create/Delete Contract

tenant-security and tenant-epg

List Filter

tenant-security and tenant-epg

Create/Delete Filter

tenant-epg

tenant-security

L4L7

List

tenant-security, tenant-epg, and nw-svc-policy

Create/Delete

tenant-epg

tenant-security and nw-svc-policy

Troubleshooting

List Session

admin*

Create/Delete Session

admin*

L2 Out

List L2Outs

tenant-ext-connectivity-l2

Contract creation

tenant-ext-connectivity-l2

tenant-security

L3 Out

List L3Outs

tenant-ext-connectivity-l3

Contract creation

tenant-ext-connectivity-l3

tenant-security

Note

In the preceding table, you must add roles marked with an asterisk (*) with the security domain "all."

For more information about Cisco APIC user roles and security domains, see the section "User Access: Roles, Privileges, and Security Domains" in Cisco ACI Fundamentals.

Recommended RBAC Configuration for Cisco ACI vCenter Plug-in

We recommended that you define two user roles with privileges to be created on APIC for aaaUser:

- vcplugin_read—defines the read permissions of aaaUser.

- vcplugin_write—defines the write permissions of aaaUser.

You can register the Cisco ACI fabric only as a local user on Cisco APIC. If the default log-in domain is local, you can log in as admin or any local username and password.

However, if the default login domain is not local, you can still register the fabric by specifying the local domain in the username:

apic#local domain\username

The local domain name must exist on Cisco APIC before you enter the local domain and username.

Note

Any RBAC configuration requires that you assign the security domain or domains of aaaUser to the VMM domain between Cisco APIC and VMware vCenter.

Note

The Cisco ACI vCenter plug-in adapts to any combination of user roles that follow the permissions described in the RBAC privileges table in Role-based Access Control for Cisco ACI vCenter Plug-in in this guide.

Upgrading VMware vCenter when Using the Cisco ACI vCenter Plug-in

If you are upgrading VMware vCenter from version 6.0 to version 6.5, and you are using the Cisco ACI vCenter plug-in, you need to take an additional step before you proceed with the upgrade.

Note

It is a best practice to uninstall the vCenter plug-in before you upgrade the VMware vCenter and then reinstall it after the upgrade.

Procedure

Delete the folder C:\ProgramData\cisco_aci_plugin\ on the vCenter.

If you do not delete the folder, and you try to register a fabric again after the upgrade, you see the following error message: "Error while saving setting in C:\ProgramData\cisco_aci_plugin\user_domain.properties" where the user is the user currently logged in to the vSphere Web Client, and the domain is the domain to which it belongs.

Although you can still register a fabric, you do not have rights to override settings that were created in the old VMware vCenter. You need to enter any changes in Cisco APIC configuration again after restarting VMware vCenter.

Cisco ACI vCenter Plug-in GUI Architecture Overview

Main Menu

This section describes the Cisco ACI vCenter plug-in GUI architecture Overview.

Main Menu

1

Home—Displays the Cisco ACI vCenter plug-in home page and has a Getting Started and an About tab.

The Getting Started tab that allows you to perform basic tasks such as Create a new Tenant, Create a new Application Profile, Create a new Endpoint Group and click the Cisco Application Centric Infrastructure (ACI) link to explore the ACI website.

The About tab displays the current version of the Cisco ACI vCenter plug-in.

2

ACI Fabric—Used to register an ACI Fabric in the plug-in and manage the tenants of the fabrics.

3

Application Profile—Used to manage application profiles by a drag and drop interface of EPG, uSeg EPG, L2/L3Out and Contract. Provides visibility on an application health, Stats and Faults.

4

Networking—Drag and Drop interface to manage VRFs and Bridge Domains.

5

Troubleshooting—View contracts defined between to entity, Start endpoint to endpoint troubleshooting sessions, browse the virtual machines (VMs) and view their connections to the endpoint groups (EPGs).

6

Cisco AVS—Install, upgrade, or uninstall Cisco AVS.

See the Cisco Application Virtual Switch Installation Guide for information.

7

Cisco ACI Virtual Edge—Install or uninstall Cisco ACI Virtual Edge, or migrate from Cisco AVS or VMware VDS to ACI Virtual Edge.

See the Cisco ACI Virtual Edge Installation Guide for information.

8

Resources—Allows you to browse in a hierarchical view of all objects managed by the plug-in.

Note

While navigating through Application Profile, Networking and Resources sections, a selection bar at the top of each screen allows you to select an active tenant. Content displayed for each section is specific to the tenant selected in that bar.

Cisco ACI vCenter Plug-in Overview

Home

This section describes the Cisco ACI vCenter plug-in GUI overview.

Note

All of the times for faults, stats, event and audits are shown in the local timezone of the browser. If the Cisco APIC time zone does not match the time zone of your system, the time stamp can have a different time zone.

In the VMware vSphere Web Client, in the Navigator pane, choose Home. In the Work pane displays the following tabs:

- Getting Started tab

The bottom of the Getting Started pane enables you to do the following things:

- Click Create a new Tenant to create a new tenant.

- Click Create a new Application Profile to create a new application profile.

- Click Create a new Endpoint Group to create a new endpoint group.

- Click the Cisco Application Centric Infrastructure (ACI) link to explore the ACI website.

- About tab

The About pane displays the Cisco ACI vCenter plug-in version.

ACI Fabric

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric. In the Work pane displays the following tabs:

- ACI Fabric tab

The ACI Fabric pane enables you to do the following things:

- Click Register a new ACI Fabric / ACI Node to register a new ACI fabric or ACI node.

- View information about the current Cisco APIC states of the fabric.

Note

When the plug-in detects the Cisco APIC as unavailable, it stops trying to connect to it and will not update its status anymore. To avoid having to wait for the timeout that comes with trying to connect to an unresponsive Cisco APIC. Click Reload to refresh the Cisco APIC state. This forces it to try to reconnect to each Cisco APIC, even to the unavailable ones. This updates their status, if they are available again.

- Tenants tab

The Tenants pane enables you to do the following things:

- Manage the different tenants present in the registered ACI Fabrics.

- Click Create a new Tenant to create a new tenant.

- View the different tenants.

If you select a tenant in the table, you can delete a tenant if you click Delete Tenant <tenant_name>.

If you select a tenant in the table, you can edit the tenant description if you right-click the <tenant_name> and choose Edit settings.

ACI Fabric - Home

Application Profile

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Application Profile. In the Work pane enables you to do the following things:

- Choose an active tenant and the application profile.

- Click Create a new Application Profile to create a new application profile.

- Use the Drag and drop to configure section to drag and drop the different elements to configure your Application Profiles fully. The elements are:

- Endpoint Group

- uSeg

- L3 External Network

- L2 External Network

- Contract

- View the Policy, Traffic Stats, Health, Faults, Audit Logs, and Events by using the tabs.

In the Policy tab, you can switch back to Consumer and Provider view or traffic view.

Networking

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Networking. In the Work pane enables you to do the following things:

- Set up your own addressing for all endpoint groups by creating isolated VRFs that are populated with bridge domains. An endpoint group will be associated with one bridge domain.

- Choose an active tenant.

- Use the Drag and drop to configure section to drag and drop the following elements:

- VRF

- Bridge Domain

Note

The available Layer 3 and Layer 2 endpoint groups are displayed here, but are not configurable.

Troubleshooting

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Troubleshooting. In the Work pane displays the following tabs:

- Policy Checker tab

The Policy Checker tab enables you to select two entities (Virtual Machine, endpoint group, Layer 3 external network or endpoint), and view all of the contracts and Layer 4 to Layer 7 services that are enforced between those 2 entities.

You can also start a troubleshooting session between two endpoints:

- Choose the time frame of the session in the From, To and fixed time check box.

- You can configure the time frame by putting a check in the Fix Time check box.

- In the Source Destination section, you can choose the source and destination endpoints. Click on Start Troubleshooting session to start a new troubleshooting session.

- In the Troubleshooting Session, you can inspect faults, configured contracts, event, audits, and traffic stats.

- You can start a trace route between the two endpoints if you click Traceroute.

- You can click the icon next to an elements to get details that correspond to the category that you chose in the left pane.

- You can get a topology that represents, for each endpoint, the corresponding vNIC, VM, and host, and the EPG to which the vNIC is connected.

- Virtual Machines tab

This view is to visualize if the network interface cards of your virtual machine are connected to any endpoint groups.

- You can restrict the list by using the search field.

- You view each of the VMs if the vNICs are connected to an EPG.

- You can quickly view if the associated EPG has good health or any faults, and view the tenant and application profile to which it belongs.

Resources

- Network

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Resources > Network. In the Work pane displays the following tabs:

- Endpoint Groups tab

Configure the network infrastructure by creating endpoint groups. Each endpoint group has a corresponding VMware Distributed Port Group where you can connect your virtual machines. You can organize your different endpoint groups into application profiles.

- Choose an active tenant.

- Click Create a new Application Profile to create a new application profile.

- Choose an application in the table and click Create a new Endpoint Group to create a new endpoint group.

- View the table to see the application profiles and endpoint groups of an active tenant.

- Choose an endpoint group to view all of the VMs that are connected to it.

- VRFs tab

For all endpoint groups, you can setup your own addressing by creating isolated VRFs that are populated with bridge domains. An endpoint group will be associated with one bridge domain.

- Choose an active tenant.

- Click Create a new VRF to create a new VRF.

- Click Create a new Bridge Domain to create a new bridge domain.

- View the table to see the VRFs.

- Security

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Resources > Security. In the Work pane displays the following tabs:

- Contracts tab

Contracts allows you to define security policies between different endpoint groups and security policies between endpoint groups and Layer 3 and Layer 2 external networks.

- Choose an active tenant.

- Click Create a new Contract to create a new contract.

- View the table to see the contracts.

- Filters tab

Filters are entities that matches a given type of traffic (based on protocol, port, etc.). They are used by contracts to define the authorized services between endpoint groups and Layer 3 external networks.

- Choose an active tenant.

- Click Create a new Filter to create a new filter.

- View the table to see the filters.

- External Connectivity

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Resources > External Connectivity. In the Work pane displays the following tabs:

- L3 External Networks tab

Layer 3 external networks are defined by the Cisco APIC administrator. You have the possibility to consume the defined networks in your contracts and Layer 4 to Layer 7 services, in order to bring external connectivity to your infrastructure.

- Choose an active tenant.

- View the table to see the Layer 3 external networks.

- L2 External Networks tab

Layer 2 external networks are defined by the Cisco APIC administrator. You have the possibility to consume the defined networks in your Contracts and Layer 4 to Layer 7 services, in order to bring external connectivity to your infrastructure.

- Choose an active tenant.

- View the table to see the Layer 2 external networks.

- L4-7 Services

In the VMware vSphere Web Client, in the Navigator pane, choose Cisco ACI Fabric > Resources > External Connectivity. In the Work pane displays the following:

- Layer 4 to Layer 7 services enables you to add pre-provisoned firewalls and load balancers between your endpoint groups and Layer 3 external networks.

- Choose an active tenant.

- View the table to see the Layer 4 to Layer 7 graph instances currently deployed inside the tenant.

GUI Tips

This section provides GUI tips.

- You can right-click on ACI object displayed in tables or in graph, to get associated actions.

- When a Virtual Machine object is displayed inside a table in the vCenter plug-in, you can double-click on it to navigate to that Virtual Machine in the vSphere Web Client.

Creating a New Tenant

This section describes how to create a new tenant.

Before you begin

Ensure that an ACI fabric is registered. For more information, see Connecting vCenter Plug-in to the Cisco ACI Fabric Using Credentials.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Work pane, choose Cisco ACI Fabric.

Step 3

In the Navigator pane, choose ACI Fabric.

Step 4

In the ACI Fabric pane, choose the Tenants tab.

Step 5

In the Tenants pane, click Create a new Tenant.

Step 6

In the New Tenant dialog box, perform the following actions:

1. In the Enter a name for the Tenant field, enter the tenant name.

2. (Optional) In the Enter a description for the Tenant field, enter the description for the tenant.

3. Click OK.

Creating a New Application Profile

This section describes how to create a new application profile.

Before you begin

- Ensure that a tenant has been created.

For more information, see Creating a New Tenant.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Work pane, choose Cisco ACI Fabric.

Step 3

In the Navigator pane, choose Resources > Network.

Step 4

In the Network pane, under the Endpoint Groups tab, perform the following actions:

1. From the Tenant drop-down list, choose the tenant name.

2. Click Create a new Application Profile.

Step 5

In the New Application Profile dialog box, perform the following actions:

1. In the Name field, the application profile name.

2. (Optional) In the Description field, enter the description of the application profile name.

3. Click OK.

Creating an EPG Using the Drag and Drop Method

This section describes how to create an endpoint group (EPG) using the drag and drop method.

Before you begin

- Ensure that a tenant has been created.

For more information, see Creating a New Tenant.

- Ensure that an application profile has been created.

For more information, see Creating a New Application Profile.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Application Profile.

Step 3

In the Application Profile pane, perform the following actions:

1. In the Tenant field, from the drop-down list, choose a tenant.

2. In the Application Profile field, from the drop-down list, choose an application profile.

3. In the Drag and drop to configure element area, drag and drop Endpoint Group.

Step 4

In the New Endpoint Group dialog box, perform the following actions:

1. In the Name field, enter the name of the endpoint group.

2. (Optional) In the Description field, enter the description of the EPG.

3. In the Bridge Domain field, choose any bridge domain from common or from the tenant where the EPG is created. The default bridge domain is common/default. Click the pen icon to choose another bridge domain.

Step 5

In the Distributed Switch field, perform the following actions:

1. Put a check in at least one distributed switch check box to connect the EPG to the chosen distributed switches.

2. Put a check in the Allow micro-segmentation check box to allow micro-segmentation.

The Allow micro-segmentation check box only shows if the distributed switch is DVS. If the distributed switch is AVS, then the GUI does not show the Allow micro-segmentation check box. All EPGs are considered to be base EPGs if the distributed switch is AVS.

This allows you to create a base EPG. All of the virtual machines that are connected to this EPG are candidates to apply the micro-segmentation rules of a uSeg EPG. Micro-segmented EPG rules only apply to virtual machines that are connected to a base EPG.

3. Put a check in the Intra EPG isolation check box to isolate the EPG.

This allows you to deny all traffic between the virtual machines that are connected to this EPG. This rule also applies to machines that are seen under a microsegmented EPG. By default, all virtual machines in the same EPG can talk to each other.

Step 6

Click OK to push the new EPG on APIC.

You will see the new EPG that you created in the topology.

Creating a New uSeg EPG Using the Drag and Drop Method

This section describes how to create a new uSeg EPG using the drag and drop method.

Before you begin

- Ensure that a tenant has been created

For more information, see Create a New Tenant.

- Ensure that an application profile has been created.

For more information, see Creating a New Application Profile.

- (DVS only, not Cisco AVS) Ensure you have created a base EPG, and connected all the VMs that needs to participate in micro-segmentation to that base EPG. For more information, see Creating a new Endpoint Group.

Procedure

Step 1

Log in to the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Application Profile.

Step 3

In the Application Profile pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant.

2. From the Application Profile drop-down list, choose an application profile.

3. In the Drag and drop to configure element area, drag and drop the uSeg into the topology.

Step 4

In the New Endpoint Group dialog box, perform the following actions:

1. In the Name field, enter the name of the EPG.

2. In the Description field, enter the description of the EPG.

Step 5

In the Distributed Switch field, choose which distributed switch needs to be associated with that uSeg EPG.

Note

If there is only one DVS, no check box is displayed as it is chosen by default.

Step 6

In the Bridge Domain field, choose any bridge domain from common or from the tenant where the uSeg EPG is created. The default bridge domain is common/default. Click the pen icon to select another bridge domain.

Step 7

Put a check in the Intra EPG isolation check box to isolate the EPG.

Step 8

In the Microsegmentation section, click the + icon.

Step 9

In the New micro-segmentation Attribute dialog box, perform the following actions:

1. In the Name field, enter the name of the new attribute.

2. (Optional) In the Description field, enter the description of the new attribute.

3. In the Type section, choose the type on which to filter.

4. In the Operator section, choose Contains the operator you wish to use.

5. If available, click the Browse button to choose a specific object, instead of manually entering a value.

6. Click OK to add the new attribute to the uSeg EPG.

Step 10

Repeat Step 7 and Step 8 to add other attributes to the uSeg EPG.

Step 11

Click OK.

Creating a Contract Between Two EPGs Using the Drag and Drop Method

This section describes how to create a contract between two endpoint groups (EPGs) using the drag and drop method.

Before you begin

- Ensure that two EPGs have been created.

For more information, see Creating an EPG Using the Drag and Drop Method.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Work pane, choose Cisco ACI Fabric.

Step 3

In the Navigator pane, choose Application Profile.

Step 4

In the Application Profile pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant.

2. From the Application Profile drop-down list, choose an application profile.

Step 5

In the Drag and drop to configure element area, drag and drop the contract on the source EPG.

Step 6

Click on the destination EPG. An arrow will display, going from the source EPG to the destination EPG.

Step 7

In the New Contract dialog box, perform the following actions:

1. In the Consumers field, verify that it displays the correct EPG.

2. In the Providers field, verify that it displays the correct EPG.

3. In the Name field, enter the name of the contract.

4. (Optional) In the Description field, enter the description of the contract.

5. In the Filters field, click the + icon to add filters to the contract.

6. In the new dialog box, drag and drop all the filters you wish to add to the Contract from the list on the left to the list on the right and click OK.

7. (Optional) Check the Configure L4-7 service check box to configure Layer 4 to Layer 7 services.

8. Click OK to create the contract.

Adding an EPG to an Existing Contract Using Drag and Drop Method

This section describes how to add an EPG to an existing contract using the drag and drop method.

Before you begin

- Ensure that a contract has been created.

- Ensure that an EPG has been created.

For more information, see Creating an EPG Using the Drag and Drop Method.

- Ensure that the contract is visible on the Application Profile pane. For example, if another EPG of the Application Profile is already using the contract. If this is not the case, follow the steps of Adding EPG to an Existing Contract Using the Security Tab.

Procedure

Step 1

Log into the VMware vSphere Web Client. In the Navigator pane, choose Application Profile .

Step 2

In the Navigator pane, choose Application Profile .

Step 3

In the Application Profile pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant.

2. From the Application Profile drop-down list, choose an application profile.

Step 4

In the Drag and drop to configure element area, drag and drop the contract, and do one of the following:

- To have the EPG consume the contract:

1. Drag and drop the Contract on the EPG that needs to consume the contract.

2. Choose the relevant contract (an arrow is displayed going from the EPG to the contract), and click the contract to make the EPG consume the contract.

- To have the EPG provide the contract:

1. Drag and drop the Contract on the contract that the EPG needs to provide.

2. Choose the relevant contract (an arrow is displayed going from the contract to the EPG), and click the Contract to make the EPG provide that contact.

Adding an EPG to an Existing Contract using the Security Tab

Before you begin

- Ensure that a contract has been created.

- Ensure that an EPG has been created.

For more information, see Creating an EPG Using the Drag and Drop Method.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Resources > Security.

Step 3

From the Tenant drop-down list, choose a tenant.

Step 4

Click on the contract where the EPG needs to be added in the list of contract.

Step 5

Click on the + icon of either the Consumers or Providers columns (to respectively have the EPG consume or provide the contract).

Step 6

From the menu that opens, choose Add Endpoint Groups.

Step 7

In the dialog box, perform the following actions:

1. Expand the tenant where the EPG is located.

2. Expand the Application Profile where the EPG is located.

3. Drag and drop the EPG from the list on the left to the list on the right.

4. Click OK.

Setting up L3 External Network

This section describes how to connect an a Layer 3 external network.

Note

You cannot do any configuration with a Layer 3 external network. You can only set up a Layer 3 external network that exists in APIC.

Before you begin

- Ensure that a Layer 3 (L3) external network on APIC is configured. For more information, see the ACI Basic Configuration Guide.

- Ensure that an EPG has been created. For more information, see Creating an EPG Using the Drag and Drop Method.

Procedure

Step 1

Log in to the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Application Profile.

Step 3

In the Application Profile pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant.

2. From the Application Profile drop-down list, choose an application profile (app).

3. In the Drag and drop to configure element area, drag and drop the L3 External Network into the topology.

Step 4

In the Select an object dialog box, expand Tenant <tenant_name> (tenant1), choose the Layer 3 external network and click OK.

Step 5

In the Drag and drop to configure element area, drag and drop the Contract on top of the Layer 3 external network and drag to connect the EPG (WEB).

Step 6

In the New Contract dialog box, perform the following actions:

1. In the Consumers field, verify that it displays the correct Layer 3 external network (L3ext).

2. In the Providers field, verify that it displays the correct EPG (WEB).

3. In the Name field, enter the name of the contract (L3ext-to-WEB).

4. (Optional) In the Description field, enter the description of the contract.

5. In the Filters field, you can add traffic filters by clicking the + icon.

6. In the new dialog box, drag and drop all the filters you wish to add to the contract from the list on the left to the list on the right and click OK.

7. (Optional) Check the Configure L4-7 service check box to configure Layer 4 to Layer 7 services.

8. Click OK to create the contract.

The contract is connected to the Layer 3 external network in the topology.

Setting up L2 External Network

This section describes how to connect Layer 2 (L2) External Network.

Note

You cannot do any configuration with an L2 External Network. You can only set up an L2 External Network that exists in the APIC.

Before you begin

- Ensure that a L2 external network on APIC is configured. For more information, see the ACI Basic Configuration Guide

- Ensure that a EPG exists.

Procedure

Step 1

Log in to the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Application Profile.

Step 3

In the Application Profile pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant (tenant1).

2. From the Application Profile drop-down list, choose Expenses.

3. In the Drag and drop to configure element area, drag and drop the L2 External Network into the topology.

4. In the Drag and drop to configure element area, drag and drop the Contract on top of the L2 external network, and then drag to connect the EPG (WEB).

Step 4

In the New Contract dialog box, perform the following actions:

1. In the Consumers field, verify that it displays the correct L2 External Network (L2ext).

2. In the Providers field, verify that it displays the correct EPG (WEB).

3. In the Name field, enter the name of the contract (L2ext-to-WEB).

4. In the Description field, enter the description of the contract.

5. In the Filters field, you can add traffic filters by clicking the + icon.

6. In the new dialog box, drag and drop all the filters you wish to add to the contract from the list on the left to the list on the right and click OK.

7. (Optional) Check the Configure L4-7 service check box to configure Layer 4 to Layer 7 services.

8. Click OK.

The contract is connected to the L2 external network in the topology.

Creating a VRF Using the Drag and Drop Method

This sections describes how to create a VRF using the drag and drop method.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Work pane, choose Networking.

Step 3

In the Networking pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant

2. In the Drag and drop to configure element area, drag and drop the VRF into the pane.

Step 4

In the New VRF dialog box, perform the following actions:

1. In the Name field, enter the name of the VRF.

2. (Optional) In the Description field, enter the description of the VRF.

3. In the Security section, check the Enforce Policies check box. Enforce Policies determines if the security rules (Contracts) should be enforced or not for that VRF.

4. Click OK.

Creating a Bridge Domain

This section describes how to create a bridge domain.

Before you begin

- Ensure that a VRF (Private Network) exists.

Procedure

Step 1

Log in to the VMware vSphere Web Client.

Step 2

In the Navigator pane, choose Networking.

Step 3

In the Networking pane, perform the following actions:

1. From the Tenant drop-down list, choose a tenant (tenant1).

2. In the Drag and drop to configure element area, drag and drop the Bridge Domain on top of the VRF in the topology.

Step 4

In the New Bridge Domain dialog box, perform the following actions:

1. In the Name field, enter the name of the bridge domain (BD2).

2. (Optional) In the Description field, enter the description of the bridge domain.

3. In the Private Subnets section, enter the private subnets (2.2.2.2/24) and click the + icon to add the subnet to the bridge domain.

4. (Optional) Repeat substeps c and d to add the desired number of subnets to the bridge domain.

5. Click OK.

The bridge domain connects to the VRF in the topology.

Start a New Troubleshooting Session Between Endpoints

This section describes how to start a new troubleshooting session between endpoints.

Procedure

Step 1

Log into the VMware vSphere Web Client.

Step 2

In the Work pane, choose Cisco ACI Fabric.

Step 3

In the Navigator pane, choose Troubleshooting.

Step 4

In the Policy Checker tab, in the Session name section, enter the new session name.

Step 5

In the Source and Destination section, click Select source.

Step 6

From the Menu that opens, click on Select Endpoint.

Step 7

In the new dialog box that opens, select the endpoint to use as source and click OK.

Step 8

In the Source and Destination section, click Select destination.

Step 9

From the Menu that opens, click on Select Endpoint.

Step 10

In the new dialog box that opens, select the endpoint to use as destination and click OK

Step 11

Click Start Troubleshooting Session.

Step 12

In the Troubleshooting pane, you can inspect the faults, configured contracts, event, audits and traffic stats.

A topology displays your configuration for each endpoint, the corresponding vNIC, VM, host, and the EPG to which the vNIC is connected. You can click the icon next to an elements to get details, corresponding to the category selected in the left pane.

Step 13

In the Navigation pane, click Traceroute to start a traceroute between the two endpoints.

Start an Exisiting Troubleshooting Session Between Endpoints

This section describes how to start an existing troubleshooting session between endpoints.

Before you begin

Procedure

Step 1

Log into the VMware vSphere Web Client, in the Work pane, choose Cisco ACI Fabric.

Step 2

In the Navigator pane, choose Troubleshooting.

Step 3

In the Policy Checker tab, in the Session name section, click Select an existing session.

1. In the Select a section dialog box, choose a troubleshooting session.

2. Click OK.

You can only do endpoint to endpoint troubleshooting.

Step 4

Click Start Troubleshooting Session.

Step 5

In the Troubleshooting pane, you can inspect the faults, configured contracts, event, audits and traffic stats.

A topology displays your configuration for each endpoint, the corresponding vNIC, VM, host, and the EPG to which the vNIC is connected. You can click the icon next to an elements to get details, corresponding to the category selected in the left pane.

Step 6

In the Navigation pane, click Traceroute to start a traceroute between the two endpoints.

Uninstalling the Cisco ACI vCenter Plug-in

This section describes how to uninstall the VMware vCenter Plug-in.

Before you begin

- You must have a PowerCLI console available.

- You must have the ACIPlugin-Uninstall.ps1 script available.

You can find the script inside the plug-in archive, or you can download it from: https://APIC_IP/vcplugin/ACIPlugin-Uninstall.ps1.

Procedure

Step 1

Open a PowerCLI console.

Step 2

Run the ACIPlugin-Uninstall.ps1 script.

Step 3

When prompted, in the vCenter IP / FQDN field, enter the vCenter where the plug-in needs to be uninstalled.

Step 4

In the dialog box that appears, enter the root privilege credentials of the vCenter.

you should see the following message in the console if the uninstallation was successful:

[x] Uninstalled ACI vCenter Plugin

Upgrading the Cisco ACI vCenter Plug-in

This section describes how to upgrade the Cisco ACI vCenter Plug-in.

Procedure

To upgrade the Cisco ACI vCenter Plug-in, you must follow the installation procedure.

For more information, see Installing the Cisco ACI vCenter Plug-in.

Troubleshooting the Cisco ACI vCenter Plug-in Installation

This section describes how to troubleshoot the Cisco ACI vCenter plug-in installation.

If the Cisco ACI vCenter plug-in is not seen the VMware vSphere Web Client GUI, perform the following actions:

- Make sure the .zip file can be downloaded from the vCenter by ensuring that HTTPS/HTTP traffic is working between the vCenter and web server where the .zip is hosted.

- Ensure that you have enabled HTTP download if your using a HTTP web server.

- Ensure that the thumbprint used is correct if you are using HTTPS.

- Check if the registration has happened by going to the following URL:

https://<VCENTER_IP>/mob/?moid=ExtensionManager&doPath=extensionList%5b"com%2ecisco%2eaciPlugin"%5d

You should see the Cisco ACI vCenter plug-in details.

If you do not and the page is blank, this indicates that the registration did not succeed. This means an error occurred while executing the registration script. To resolve this, you must perform the installation procedure again and note if an error is displayed by the registration scripts.

- Check the vSphere Web Client logs.

- Linux Appliance: /var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log

- 5.5 Windows 2008: C:\ProgramData\VMware\vSphere Web Client\serviceability\logs\vsphere_client_virgo.log

- 6.0 Windows 2008: %ALLUSERSPROFILE%\VMWare\vCenterServer\logs\vsphere-client\logs\vsphere_client_virgo.log

- Searching for ‘vcenter-plugin’ or ‘com.cisco.aciPlugin’ in the log displays relevant information about the install/upgrade.

An Example of a successful upgrade:

[2016-05-31T19:32:56.780Z] [INFO ] -extensionmanager-pool-11139 70002693 100019
200004 com.vmware.vise.vim.extension.VcExtensionManager
Downloading plugin package from https://172.23.137.72/vcenter-plugin-2.0.343.6.zip
(no proxy defined)
[2016-05-31T19:32:56.872Z] [INFO ] m-catalog-manager-pool-11128 70002693 100019 200004
com.vmware.vise.vim.cm.CmCatalogManager
Detected service providers (ms):206
[2016-05-31T19:32:56.872Z] [INFO ] m-catalog-manager-pool-11128 70002693 100019 200004
com.vmware.vise.vim.cm.CmCatalogManager
No new locales or service infos to download.
[2016-05-31T19:32:57.678Z] [INFO ] -extensionmanager-pool-11139 70002693 100019 200004
com.vmware.vise.vim.extension.VcExtensionManager
Done downloading plugin package from https://172.23.137.72/vcenter-plugin-2.0.343.6.zip
[2016-05-31T19:32:58.438Z] [INFO ] -extensionmanager-pool-11139 70002693 100019 200004
com.vmware.vise.vim.extension.VcExtensionManager
Done expanding plugin package to /etc/vmware/vsphere-client/vc-packages/vsphere-client-
serenity/com.cisco.aciPlugin-2.0.343.6
[2016-05-31T19:32:58.440Z] [INFO ] -extensionmanager-pool-11139 70002693 100019 200004
com.vmware.vise.extensionfw.ExtensionManager
Undeploying plugin package 'com.cisco.aciPlugin:2.0.343.5’.

Alternative Installation of the Cisco ACI vCenter Plug-in

This section describes how to install the Cisco ACI vCenter plug-in. If you cannot enable HTTPS traffic between your vCenter and APIC and you wish to use your own web server to host the Cisco ACI vCenter plug-in zip file, follow this procedure.

Before you begin

- Make sure that all the prerequisites are met.

For more information, see Cisco ACI vCenter Plug-in Software Requirements.

For more information, see Required APIC Configuration.

- Have a PowerCLI console available.

For more information, see VMware documentation.

Procedure

Step 1

Make the .zip file available on a Web server.

1. If the Web server is not HTTPS: By default, vCenter will only allow a download from HTTPS sources. To allow from HTTP, open and edit the following configuration file for your vCenter version:

- vCenter 5.5 Linux Appliance: /var/lib/vmware/vsphere-client/webclient.properties

- vCenter 6.0 Linux Appliance: /etc/vmware/vsphere-client/webclient.properties

- vCenter 5.5 Windows 2008: %ALLUSERSPROFILE%\VMware\vSphere Web Client\webclient.properties

- vCenter 6.0 Windows 2008: C:\ProgramData\VMware\vCenterServer\cfg\vsphere-client\webclient.properties

2. Add allowHttp=true at the end of the file.

3. If the Web server is not HTTPS, restart the vSphere Web Client service using the '/etc/init.d/vsphere-client restart' command.

Step 2

Run the script using the PowerCLI console or Python:

Option

Description

To use the PowerCLI console

1. Open a PowerCLI console.

2. Run the ACIPlugin-Install.ps1 script.

When prompted, enter the following information:

- In the vCenter IP / FQDN field, enter the vCenter where the plug-in needs to be installed.

- In the Plugin .zip file URL field, enter the URL where the vCenter will be able to download the plug-in.

Note

Ensure you have not renamed the .zip file.

- If you are using HTTP, leave the SHA1 thumbprint field empty. If you are using HTTPS, enter the SHA1 thumbprint of the Web server used, using one of the following formats:

- Separated by colons:

xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx

- Separated by spaces:

xx xx xx xx xx xx xx xx xx xx xx

Note

Some browsers on Windows might display the certificate thumbprint as a single non-delimited string (for example, xxxxxxxxxxxxxxxxxx), which the installation script does not process correctly. Make sure that the SHA1 thumbprint of the Web server uses one of the correct formats. Otherwise, the Cisco ACI vCenter plug-in appears to fail.

3. In the dialog box, enter the root privilege credentials of the vCenter.

To use Python

Note

You must use Python 2.7.9 or higher and have the pyvmomi package installed in the Python environment.

Run the Python script: python deployPlugin.py

When prompted, enter the following information:

- In the vCenter IP field, enter the vCenter where the plug-in needs to be installed.

- In the vCenter Username & Password field, enter the root privilege credentials of the vCenter.

- In the Plugin .zip file URL field, enter the URL where the vCenter will be able to download the plug-in.

Ensure you have not renamed the .zip file.

- In the Https server thumbprint field, Leave this empty, if you are using HTTP. Otherwise, enter the SHA1 thumbprint of the Web server used. The fields are separated with colons. For example:

D7:9F:07:61:10:B3:92:93:E3:49:AC:89:84:5B:03:80:C1:9E:2F:8B

Note

There is also a deploy.cfg file available, where you can pre-enter your information. You can then run the script with the file as argument. For example:

$ python deployPlugin.py deploy.cfg

Step 3

Log into the vSphere Web Client once the registration is completed.

Note

First login may take longer, as the vCenter will be downloading and deploying the plug-in from the Web server.

Once the VMware vSphere Web Client loads, you will see the Cisco ACI Fabric in the Navigator pane. This allows you to manage your ACI fabric.

Note

After you register the plug-in, when you launch the web client for the first time, an error message might display asking to reload the web client. Click Reload to refresh the page and the error message will not appear again.

Chapter 11. Cisco ACI with Microsoft SCVMM

This chapter contains the following sections:

- About Cisco ACI with Microsoft SCVMM

- Getting Started with Cisco ACI with Microsoft SCVMM

- Upgrading the Cisco ACI with Microsoft SCVMM Components

- Uninstalling the Cisco ACI with Microsoft SCVMM Components

- Downgrading the Cisco APIC Controller and the Switch Software with Cisco ACI and Microsoft SCVMM Components

- Exporting APIC OpFlex Certificate

About Cisco ACI with Microsoft SCVMM

The Application Policy Infrastructure Controller (APIC) integrates with Microsoft VM management systems and enhances the network management capabilities of the platform. The Cisco Application Centric Infrastructure (ACI) integrates at the following levels of the Microsoft VM Management systems:

- Cisco ACI with Microsoft System Center Virtual Machine Manager (SCVMM)—When integrated with Cisco ACI, SCVMM enables communication between ACI and SCVMM for network management.

Note

Migrating from SCVMM to SCVMM HA is not supported by Microsoft.

- Cisco ACI and Microsoft Windows Azure Pack—For information about how to set up Cisco ACI and Microsoft Windows Azure Pack, see Cisco ACI with Microsoft Windows Azure Pack Solution Overview.

- Cisco ACI with Microsoft SCVMM Solution Overview

- Physical and Logical Topology of SCVMM

- About the Mapping of ACI Constructs in SCVMM

- SCVMM Fabric Cloud and Tenant Clouds

Cisco ACI with Microsoft SCVMM Solution Overview

At this integration point the Application Policy Infrastructure Controller (APIC) and Microsoft System Center Virtual Machine Manager (SCVMM) communicate with each other for network management. Endpoint groups (EPGs) are created in APIC and are created as VM networks in SCVMM. Compute is provisioned in SCVMM and can consume these networks.

Physical and Logical Topology of SCVMM

This figure shows a representative topology of a typical System Center Virtual Machine Manager (SCVMM) deployment with Cisco Application Centric Infrastructure (ACI) fabric. The Microsoft SCVMM service can be deployed as a Standalone Service or as a Highly Available Service on physical hosts or virtual machines, but will logically be viewed as a single SCVMM instance which communicates to the APIC.

Connectivity between an SCVMM Service and the Application Policy Infrastructure Controller (APIC) is over the management network.

Topology with ACI Fabric and SCVMM

About the Mapping of ACI Constructs in SCVMM

This section shows a table and figure of the mapping of Application Policy Infrastructure Controller (APIC) constructs in Microsoft System Center Virtual Machine Manager (SCVMM).

Mapping of APIC and SCVMM constructs

APIC

System Center

VMM Domain

Logical Switch and Logical Network

VMM Controller

SCVMM

SCVMM Cloud Name

Cloud (Fabric)

EPG

VM Network

Infrastructure VLAN

One infrastructure VM network for each logical switch

Mapping of ACI and SCVMM constructs

The mapping is bound by the following rule:

- One VMM domain cannot map to the same SCVMM more than once.

SCVMM Fabric Cloud and Tenant Clouds

Microsoft System Center Virtual Machine Manager (SCVMM) provides an object called "Cloud", which acts as a container of logical and physical fabric resources. ACI Integration with SCVMM automatically creates the various logical networking pieces and enables the logical networks at your designated cloud. When configuring ACI Integration with SCVMM, the fabric cloud is the cloud that is specified as the root container on the Application Policy Infrastructure Controller (APIC), while the tenant cloud is an SCVMM cloud that contains a subset of the host groups specified in the fabric cloud. SCVMM contains all the host groups that will be used to deploy the logical switch. Once the fabric cloud is set up and the logical switch has been deployed to the hosts in the host groups, an SCVMM Admin can then create tenant clouds and enable the apicLogicalNetwork on that tenant cloud, enabling Windows Azure Pack tenants to create and deploy tenant networks on the fabric.

Example:

SCVMM Cloud Name:  Fabric_Cloud
Host Groups:  All Hosts
Host Group HumanResources:
HyperV Node:  Node-2-24
Host Group Engineering:
HyperV Node:  Node-2-25

SCVMM Cloud Name:  HR_Cloud
Host Groups: HumanResources

SCVMM Cloud Name:  Engineering_Cloud
Host Groups:  Engineering

Getting Started with Cisco ACI with Microsoft SCVMM

This section describes how to get started with Cisco Application Centric Infrastructure (ACI) with Microsoft System Center Virtual Machine Manager (SCVMM).

You must download and unzip the Cisco ACI and Microsoft Integration file for the 2.2(1) release before installing Cisco ACI with Microsoft Windows Azure Pack.

1. Go to Cisco's Application Policy Infrastructure Controller (APIC) Website:

http://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

2. Choose All Downloads for this Product.

3. Choose the release version and the aci-msft-pkg-2.2.1x.zip file.

4. Click Download.

5. Unzip the aci-msft-pkg-2.2.1x.zip file.

Note

Cisco ACI with Microsoft System Center Virtual Machine Manager (SCVMM) only supports ASCII characters. Non-ASCII characters are not supported.

Ensure that English is set in the System Locale settings for Windows, otherwise ACI with SCVMM will not install. In addition, if the System Locale is later modified to a non-English Locale after the installation, the integration components may fail when communicating with the APIC and the ACI fabric.

- Prerequisites for Getting Started with Cisco ACI with Microsoft SCVMM

- Installing, Setting Up, and Verifying the Cisco ACI with Microsoft SCVMM Components

Prerequisites for Getting Started with Cisco ACI with Microsoft SCVMM

Before you get started, ensure that you have verified that your computing environment meets the following prerequisites:

- Ensure that one of the following Microsoft System Center Virtual Machine Manager (SCVMM) versions with the Administrator Console Builds are met:

- 2016 RTM (Build 4.0.1662.0) or newer

- 2012 R2 with Update Rollup 9 (Build 3.2.8145.0) or newer

- Ensure Windows Server 2016 or 2012 R2 is installed on the Hyper-V server with the Hyper-V role enabled.

See Microsoft's documentation.

- Ensure the cloud is configured in SCVMM and appropriate hosts added to that cloud.

See Microsoft's documentation.

- Ensure "default" AEP exists with infrastructure VLAN enabled.

- If there are switches between the Cisco Application Centric Infrastructure (ACI) leaf switch and the Hyper-V host (such as a Fabric Interconnect), you must allow the infrastructure VLAN on these intermediary devices.

- Ensure you have the Cisco MSI files for Application Policy Infrastructure Controller(APIC) SCVMM and the Host Agent.

See Getting Started with Cisco ACI with Microsoft SCVMM.

- Ensure that you scheduled a maintenance window for the SCVMM Installation. The Cisco ACI SCVMM Installation process with automatically restart the current running SCVMM service instance.

Note

If the VMs in SCVMM are configured with Dynamic MAC, then it takes time for the APIC to update the VM Inventory as the SCVMM takes time to learn or discover these MAC addresses.

- Ensure the Hyper-V Management Tools is installed on the Hyper-V hosts as well as the SCVMM server.

To install the Hyper-V Management Tools feature:

1. In the Remote Server Administration Tools, Add Roles and Features > Feature > Remote Server Administration Tools > Role Administration Tools > Hyper-V Management Tools and finish the wizard to install the feature.

2. Repeat for each Hyper-V and the SCVMM server.

This installs the Hyper-V PowerShell cmdlets needed for the APIC SCVMM and host agent.

Installing, Setting Up, and Verifying the Cisco ACI with Microsoft SCVMM Components

This section describes how to install, set up, and verify the Cisco Application Centric Infrastructure (ACI) with Microsoft System Center Virtual Machine Manager (SCVMM) components.

Component

Task

Install the APIC SCVMM Agent on SCVMM or on a Highly Available SCVMM

See Installing the APIC SCVMM Agent on SCVMM.

See Installing the APIC SCVMM Agent on a Highly Available SCVMM

For the Windows Command Prompt method, see Installing the APIC Agent on SCVMM Using the Windows Command Prompt.

Generate the OpflexAgent certificate

See Generating APIC OpFlex Certificate.

Add the OpFlex certificate policy to APIC

See Adding the OpFlex Certificate Policy to APIC.

Install the OpflexAgent certificate

See Installing the OpflexAgent Certificate.

Configure APIC IP Settings with APIC credentials on the SCVMM Agent or on the SCVMM Agent on a Highly Available SCVMM

See Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent.

or

See Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent on a Highly Available SCVMM.

Install the APIC Hyper-V Agent on the Hyper-V server

See Installing the APIC Hyper-V Agent on the Hyper-V Server.

For the Windows Command Prompt method, see Installing the APIC Hyper-V Agent on the Hyper-V Server Using the Windows Command Prompt.

Verify the APIC SCVMM Agent installation on SCVMM or on a Highly Available SCVMM

See Verifying the APIC SCVMM Agent Installation on SCVMM.

or

See Verifying the APIC SCVMM Agent Installation on a Highly Available SCVMM.

Verify the APIC Hyper-V Agent installation on the Hyper-V server

See Verifying the APIC Hyper-V Agent Installation on the Hyper-V Server.

Create SCVMM Domain Profiles

See Creating SCVMM Domain Profiles and Creating a SCVMM Domain Profile Using the GUI.

For the NX-OS Style CLI method and the REST API method, see the respective procedures included later in this chapter:

- Creating a SCVMM Domain Profile Using the NX-OS Style CLI

- Creating a SCVMM Domain Profile Using the REST API

Verify the SCVMM VMM Domain and SCVMM VMM

See Verifying the SCVMM VMM Domain and SCVMM VMM.

Deploy the logical switch to the host on SCVMM

See Deploying the Logical Switch to the Host on SCVMM.

Enable the Logical Network on Tenant Clouds

See Enabling the Logical Network on Tenant Clouds.

- Installing the APIC SCVMM Agent on SCVMM

- Installing the APIC SCVMM Agent on a Highly Available SCVMM

- Generating APIC OpFlex Certificate

- Adding the OpFlex Certificate Policy to APIC

- Installing the OpflexAgent Certificate

- Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent

- Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent on a Highly Available SCVMM

- Installing the APIC Hyper-V Agent on the Hyper-V Server

Installing the APIC SCVMM Agent on SCVMM

This section describes how to install the Application Policy Infrastructure Controller (APIC) SCVMM agent on System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Log in to the SCVMM server with SCVMM administrator credentials.

Step 2

On the SCVMM server in Explorer, locate the APIC SCVMM Agent.msi file.

Step 3

Right-click APIC SCVMM Agent.msi file and select Install.

Step 4

In the Cisco APIC SCVMM Agent Setup dialog box, perform the following actions:

1. Click Next.

2. Check the I accept the terms in the License Agreement check box and click Next.

3. Enter your account name and password credentials.

Provide the same credentials that you used for the SCVMM console. The Cisco APIC SCVMM agent requires these credentials for the SCVMM operations to be able to function.

The installation process verifies the entered account name and password credentials. If the installation fails, the SCVMM shows an error message and you must re-enter valid credentials.

4. After successful validation of the account name and password credentials, click Install.

5. Click Finish.

Note

You can configure only one APIC cluster per SCVMM, as one SCVMM can interact with only one APIC cluster.

Installing the APIC SCVMM Agent on a Highly Available SCVMM

This section describes how to install the Application Policy Infrastructure Controller (APIC) SCVMM agent on a Highly Available System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Log in to the Current Owner Node of the Highly Available SCVMM installation.

Step 2

On the SCVMM server in File Explorer, locate the APIC SCVMM Agent.msi file.

Step 3

Right-click APIC SCVMM Agent.msi file and select Install.

Step 4

In the Cisco APIC SCVMM Agent Setup dialog box, perform the following actions:

1. Click Next.

2. Check the I accept the terms in the License Agreement check box and click Next.

3. Enter your account name and password credentials.

Provide the same credentials that you used for the SCVMM console. The Cisco APIC SCVMM agent requires these credentials for the SCVMM operations to be able to function.

The installation process verifies the entered account name and password credentials. If the installation fails, the SCVMM shows an error message and you must re-enter valid credentials.

4. After successful validation of the account name and password credentials, click Install.

5. Click Finish.

Step 5

Repeat steps 1-4 for each Standby Node in the Windows Failover Cluster.

Generating APIC OpFlex Certificate

This section describes how to generate APIC OpFlex certificate to secure communication between the Application Policy Infrastructure Controller (APIC) and SCVMM agents.

Note

This should only be done once per installation.

Procedure

Step 1

Log in to the SCVMM server, choose Start > Run > Windows Powershell, and then, in the app bar, click Run as administrator.

Step 2

Load ACISCVMMPsCmdlets and create a new OpflexAgent.pfx certificate file, by entering the following commands:

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. All rights reserved.

PS C:\Users\Administrator.INSCISCO> cd \
PS C:\> cd '.\Program Files (x86)\ApicVMMService'
PS C:\Program Files (x86)\ApicVMMService> Import-Module .\ACIScvmmPsCmdlets.dll
PS C:\Program Files (x86)\ApicVMMService> Get-Command -Module ACIScvmmPsCmdlets

CommandType     Name                      ModuleName
-----------     ----                      ----------
Cmdlet          Get-ACIScvmmOpflexInfo    ACIScvmmPsCmdlets
Cmdlet          Get-ApicConnInfo          ACIScvmmPsCmdlets
Cmdlet          Get-ApicCredentials       ACIScvmmPsCmdlets
Cmdlet          New-ApicOpflexCert        ACIScvmmPsCmdlets
Cmdlet          Read-ApicOpflexCert       ACIScvmmPsCmdlets
Cmdlet          Set-ApicConnInfo          ACIScvmmPsCmdlets
Cmdlet          Set-ApicCredentials       ACIScvmmPsCmdlets

Step 3

Generate a new OpFlex Certificate, by entering the following commands. The "New-ApicOpflexCert" PowerShell command will both generate the PFX certificate package file for use on other machines and install the certificate to the local machine's Certificate Store.

PS C:\Program Files (x86)\ApicVMMService> $pfxpassword = ConvertTo-SecureString "MyPassword" -AsPlainText -Force
PS C:\Program Files (x86)\ApicVMMService> New-ApicOpflexCert -ValidNotBefore 1/1/2015 -ValidNotAfter 1/1/2020
-Email t0@domain.com -Country USA -State CA -Locality "San Jose" -Organization MyOrg –PfxPassword $pfxpassword
Successfully created:
C:\Program Files (x86)\ApicVMMService\OpflexAgent.pfx

PS C:\Program Files (x86)\ApicVMMService>

Step 4

Display the certificate information to be used on APIC using the REST API.

See Displaying the Certificate Information to be Used on APIC Using the REST API.

- Displaying the Certificate Information to be Used on APIC Using the REST API

Displaying the Certificate Information to be Used on APIC Using the REST API

This section describes how to display the certificate information to be used on APIC using the REST API.

Procedure

To display the certificate information to be used on the APIC.

PS C:\Program Files (x86)\ApicVMMService> $pfxpassword = ConvertTo-SecureString "MyPassword"
-AsPlainText -Force
PS C:\Program Files (x86)\ApicVMMService> Read-ApicOpflexCert -PfxFile
"C:\Program Files (x86)\ApicVMMService\OpflexAgent.pfx" -PfxPassword $pfxpassword
-----BEGIN CERTIFICATE-----
MIIDojCCAoqgAwIBAgIQHz+F2luuOpFKK0p3jxWRfjANBgkqhkiG9w0BAQ0FADBfMRwwGgYJKoZI
hvcNAQkBFg10MEBkb21haW4uY29tMQ4wDAYDVQQKDAVNeU9yZzELMAkGA1UECAwCQ0ExDDAKBgNV
BAYTA1VTQTEUMBIGA1UEAwwLT3BmbGV4QWdlbnQwHhcNMTUwMTAxMDAwMDAwWhcNMjAwMTAxMDAw
MDAwWjBfMRwwGgYJKoZIhvcNAQkBFg10MEBkb21haW4uY29tMQ4wDAYDVQQKDAVNeU9yZzELMAkG
A1UECAwCQ0ExDDAKBgNVBAYTA1VTQTEUMBIGA1UEAwwLT3BmbGV4QWdlbnQwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCzQS3rvrIdxiHfeAUqtX68CdjIL1+nDtqBH8LzDk0RBVb0KU6V
9cYjCAMwW24FJo0PMt4XblvFJDbZUfjWgEY1JmDxqHIAhKIujGsyDoSZdXaKUUv3ig0bzcswEGvx
khGpAJB8BCnODhD3B7Tj0OD8Gl8asd1u24xOy/8MtMDuan/2b32QRmn1uiZhSX3cwjnPI2JQVIif
n68L12yMcp1kJvi6H7RxVOiES33uz00qjxcPbFhsuoFF1eMT1Ng41sTzMTM+xcE6z72zgAYN6wFq
T1pTCLCC+0u/q1yghYu0LBnARCYwDbe2xoa8ClVcL3XYQlEFlp1+HFfd//p1ro+bAgMBAAGjWjBY
MBIGA1UdEwEB/wQIMAYBAf8CAQAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFGuzLCG5
4DEcP+bPiFbiDjMDQ3tMMA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0BAQ0FAAOCAQEANc5kKvN4
Q62tIYa1S2HSyiwjaMq7bXoqIH/ICPRqEXu1XE6+VnLnYqpo3TitLmU4G99uz+aS8dySNWaEYghk
8jgLpu39HH6yWxdPiZlcCQ17J5B5vRu3Xjnc/2/ZPqlQDEElobrAOdTko4uAHG4lFBHLwAZA/f72
5fciyb/pjNPhPgpCP0r7svElQ/bjAP1wK8PhCfd7k2rJx5jHr+YX8SCoM2jKyzaQx1BAdufspX3U
7AWH0aF7ExdWy/hW6CduO9NJf+98XNQe0cNH/2oSKYCl9qEK6FesdOBFvCjlRYR9ENqiY4q7xpyB
tqDkBm80V0JslU2xXn+G0yCWGO3VRQ==
-----END CERTIFICATE-----
PS C:\Program Files (x86)\ApicVMMService>

Adding the OpFlex Certificate Policy to APIC

This section describes how to add the OpFlex certificate policy to theApplication Policy Infrastructure Controller (APIC) .

Procedure

Add the AAA policy to allow authenticate this certificate on the APIC server. The Hyper-V agent certificate policy can be added in APIC through the GUI or REST Post:

- GUI method:

1. Log in to the APIC GUI, on the menu bar, choose ADMIN > AAA.

2. In the Navigation pane, choose Security Management > Local Users and click on admin.

3. In the PROPERTIES pane, choose Actions > Create X509 Certificate, in the drop-down list, enter the name and data.

4. In the Create X509 Certificate dialog box, in the Name field, you must enter "OpflexAgent".

5. On the SCVMM server, enter the output of the PowerShell Read-ApicOpflexCert cmdlet.

6. When you run the Read-ApicOpflexCert cmdlet, provide the full link when prompted for the name of the pfx file: C:\Program Files (x86)\ApicVMMService\OpflexAgent.pfx, then enter the password.

7. Copy from the beginning of "-----BEGIN CERTIFICATE-----" to the end of "-----END CERTIFICATE-----"and paste it in the DATA field.

8. Click SUBMIT.

9. In the PROPERTIES pane, under the User Certificates field, you will see the user certificate displayed.

- REST Post method:

POST
http://<apic-ip>/api/policymgr/mo/uni/userext/user-admin.json?rsp-subtree=full
{"aaaUserCert":{"attributes":
{"name":"OpflexAgent", "data":"
-----BEGIN CERTIFICATE-----
MIIDojCCAoqgAwIBAgIQHz+F2luuOpFKK0p3jxWRfjANBgkqhkiG9w0BAQ0FADBfMRwwGgYJKoZI
hvcNAQkBFg10MEBkb21haW4uY29tMQ4wDAYDVQQKDAVNeU9yZzELMAkGA1UECAwCQ0ExDDAKBgNV
BAYTA1VTQTEUMBIGA1UEAwwLT3BmbGV4QWdlbnQwHhcNMTUwMTAxMDAwMDAwWhcNMjAwMTAxMDAw
MDAwWjBfMRwwGgYJKoZIhvcNAQkBFg10MEBkb21haW4uY29tMQ4wDAYDVQQKDAVNeU9yZzELMAkG
A1UECAwCQ0ExDDAKBgNVBAYTA1VTQTEUMBIGA1UEAwwLT3BmbGV4QWdlbnQwggEiMA0GCSqGSIb3
DQEBAQUAA4IBDwAwggEKAoIBAQCzQS3rvrIdxiHfeAUqtX68CdjIL1+nDtqBH8LzDk0RBVb0KU6V
9cYjCAMwW24FJo0PMt4XblvFJDbZUfjWgEY1JmDxqHIAhKIujGsyDoSZdXaKUUv3ig0bzcswEGvx
khGpAJB8BCnODhD3B7Tj0OD8Gl8asd1u24xOy/8MtMDuan/2b32QRmn1uiZhSX3cwjnPI2JQVIif
n68L12yMcp1kJvi6H7RxVOiES33uz00qjxcPbFhsuoFF1eMT1Ng41sTzMTM+xcE6z72zgAYN6wFq
T1pTCLCC+0u/q1yghYu0LBnARCYwDbe2xoa8ClVcL3XYQlEFlp1+HFfd//p1ro+bAgMBAAGjWjBY
MBIGA1UdEwEB/wQIMAYBAf8CAQAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwHQYDVR0OBBYEFGuzLCG5
4DEcP+bPiFbiDjMDQ3tMMA4GA1UdDwEB/wQEAwIBBjANBgkqhkiG9w0BAQ0FAAOCAQEANc5kKvN4
Q62tIYa1S2HSyiwjaMq7bXoqIH/ICPRqEXu1XE6+VnLnYqpo3TitLmU4G99uz+aS8dySNWaEYghk
8jgLpu39HH6yWxdPiZlcCQ17J5B5vRu3Xjnc/2/ZPqlQDEElobrAOdTko4uAHG4lFBHLwAZA/f72
5fciyb/pjNPhPgpCP0r7svElQ/bjAP1wK8PhCfd7k2rJx5jHr+YX8SCoM2jKyzaQx1BAdufspX3U
7AWH0aF7ExdWy/hW6CduO9NJf+98XNQe0cNH/2oSKYCl9qEK6FesdOBFvCjlRYR9ENqiY4q7xpyB
tqDkBm80V0JslU2xXn+G0yCWGO3VRQ==
-----END CERTIFICATE-----

Installing the OpflexAgent Certificate

This section describes how to install the OpflexAgent Certificate.

Procedure

Step 1

Log in to the SCVMM server with administrator credentials.

Step 2

Use one of the following methods:

- For large-scale deployments, see Microsoft's documentation for Deploy Certificates by Using Group Policy:

https://technet.microsoft.com/en-us/library/cc770315(v=ws.10).aspx.

- For small-scale deployments follow these steps:

You must add OpFlex security certificate to the local machine. The Microsoft SCVMM agent has a security certificate file named OpflexAgent.pfx located in the C:\Program Files (x86)\ApicVMMService folder on the SCVMM server. If the following steps are not performed on your SCVMM servers, the APIC SCVMM Agent cannot communicate with the Application Policy Infrastructure Controller (APIC) .

Install the OpFlex security certificate on the SCVMM Windows Server 2012 local machine’s certificate repository. On each SCVMM server, install this certificate by performing the following steps:

1. Choose Start > Run.

2. Enter mmc and click OK.

3. In the Console Root window, on the menu bar, choose Add/Remove Snap-in.

4. In the Available Snap-ins field, choose Certificates and click Add.

5. In the Certificates snap-in dialog box, choose the Computer Account radio button, and click Next.

6. In the Select Computer dialog box, choose the Local Computer radio button, and click Finish.

7. Click OK to go back to the main MMC Console window.

8. In the MMC Console window, double-click Certificates (local computer) to expand its view.

9. Right-click Certificates under Personal and choose All Tasks > Import.

10. In the Certificates Import Wizard dialog box, perform the following actions:

1. Click Next.

2. Browse to the Opflex Agent file and click Next.

11. Enter the password for the certificate that was provided when you installed MSI.

12. You must choose the Mark this key as exportable. This will allow you to back up or transport your keys at a later time radio button.

13. Choose the Include all extended properties radio button.

14. Choose the Place all certificates in the following store radio button, browse to locate Personal, and click Next.

15. Click Finish.

16. Click OK.

Step 3

Repeat steps 1 through 5 for each SCVMM server.

Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent

This section describes how to configure the Cisco Application Policy Infrastructure Controller (APIC) IP settings with OpflexAgent Certificate on the System Center Virtual Machine Manager (SCVMM) agent.

Procedure

Step 1

Log in to the SCVMM server and choose Start > Run > Windows PowerShell.

Step 2

Load ACISCVMMPsCmdlets by entering the following commands:

Example:

Note

Get-ApicCredentials and Set-ApicCredentials are now deprecated, use Get-ApicConnInfo and Set-ApicConnInfo.

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. All rights reserved.

PS C:\Users\Administrator.INSCISCO> cd \
PS C:\> cd '.\Program Files (x86)\ApicVMMService'
PS C:\Program Files (x86)\ApicVMMService> Import-Module .\ACIScvmmPsCmdlets.dll
PS C:\Program Files (x86)\ApicVMMService> Get-Command -Module ACIScvmmPsCmdlets

CommandType     Name                                               ModuleName
-----------     ----                                               ----------
Cmdlet          Get-ACIScvmmOpflexInfo                             ACIScvmmPsCmdlets
Cmdlet          Get-ApicConnInfo                                   ACIScvmmPsCmdlets
Cmdlet          Get-ApicCredentials                                ACIScvmmPsCmdlets
Cmdlet          New-ApicOpflexCert                                 ACIScvmmPsCmdlets
Cmdlet          Read-ApicOpflexCert                                ACIScvmmPsCmdlets
Cmdlet          Set-ApicConnInfo                                   ACIScvmmPsCmdlets
Cmdlet          Set-ApicCredentials                                ACIScvmmPsCmdlets

PS C:\Program Files (x86)\ApicVMMService>

Step 3

Set up Cisco APIC connection parameters for the SCVMM agent by entering the following commands, adding at least one Cisco APIC:

PS C:\Users\administrator.APIC> Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP
-CertificateSubjectName OpflexAgent

Apic Credential is successfully set to APIC SCVMM service agent.

If you enter more than one -ApicNameOrIPAddress , use the following format: "APIC_1_IP;APIC_2_IP;APIC_3_IP;APIC_N_IP"

If you enter the wrong information in Set-ApicCredentials, the information fails to apply and validate on the Cisco APIC. This information is not preserved.

PS C:\Program Files (x86)\ApicVMMService> Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP
-CertificateSubjectName O
pflexAgentWrong
Failed cmdlet with Error: Invalid APIC Connection Settings.
Set-ApicConnInfo : The remote server returned an error: (400) Bad Request.
At line:1 char:1
+ Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP -CertificateSubjectName Opf ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Set-ApicConnInfo], WebException
+ FullyQualifiedErrorId : Failed cmdlet with Error: Invalid APIC Connection Settings.,Cisco.ACI.SCVMM.
PowerShell.SetApicConnInfo

Step 4

Verify that the Cisco APIC connection parameters are set properly on Cisco APIC SCVMM Agent by entering the following command:

PS C:\Program Files (x86)\ApicVMMService> Get-ApicConnInfo

EndpointAddress        :
Username               :
Password               :
ApicAddresses          : 172.23.139.224
ConnectionStatus       : Connected
adminSettingsFlags     : 0
certificateSubjectName : OpflexAgent
ExtensionData          :

PS C:\Program Files (x86)\ApicVMMService>

Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent on a Highly Available SCVMM

This section describes how to configure the Application Policy Infrastructure Controller (APIC) IP settings with OpflexAgent Certificate on the System Center Virtual Machine Manager (SCVMM) agent.

Procedure

Step 1

Log in to the Owner Node SCVMM server and choose Start > Run > Windows PowerShell.

Step 2

Load ACISCVMMPsCmdlets by entering the following commands:

Example:

Note

Get-ApicCredentials and Set-ApicCredentials are now deprecated, use Get-ApicConnInfo and Set-ApicConnInfo.

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. All rights reserved.

PS C:\Users\Administrator.INSCISCO> cd \
PS C:\> cd '.\Program Files (x86)\ApicVMMService'
PS C:\Program Files (x86)\ApicVMMService> Import-Module .\ACIScvmmPsCmdlets.dll
PS C:\Program Files (x86)\ApicVMMService> Get-Command -Module ACIScvmmPsCmdlets

CommandType     Name                                               ModuleName
-----------     ----                                               ----------
Cmdlet          Get-ACIScvmmOpflexInfo                             ACIScvmmPsCmdlets
Cmdlet          Get-ApicConnInfo                                   ACIScvmmPsCmdlets
Cmdlet          Get-ApicCredentials                                ACIScvmmPsCmdlets
Cmdlet          New-ApicOpflexCert                                 ACIScvmmPsCmdlets
Cmdlet          Read-ApicOpflexCert                                ACIScvmmPsCmdlets
Cmdlet          Set-ApicConnInfo                                   ACIScvmmPsCmdlets
Cmdlet          Set-ApicCredentials                                ACIScvmmPsCmdlets

PS C:\Program Files (x86)\ApicVMMService>

Step 3

Set up Cisco APIC connection parameters for the SCVMM agent by entering the following commands, adding one or more Cisco APIC:

PS C:\Users\administrator.APIC> Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP
-CertificateSubjectName OpflexAgent

Apic Credential is successfully set to APIC SCVMM service agent. 10:25 AM

If you enter more than one -ApicNameOrIPAddress , use the following format: "APIC_1_IP;APIC_2_IP;APIC_3_IP;APIC_N_IP"

If you enter the wrong information in Set-ApicCredentials, the information fails to apply and validate on the Cisco APIC. This information is not preserved.

PS C:\Program Files (x86)\ApicVMMService> Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP
-CertificateSubjectName O
pflexAgentWrong
Failed cmdlet with Error: Invalid APIC Connection Settings.
Set-ApicConnInfo : The remote server returned an error: (400) Bad Request.
At line:1 char:1
+ Set-ApicConnInfo -ApicNameOrIPAddress APIC_1_IP -CertificateSubjectName Opf ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (:) [Set-ApicConnInfo], WebException
+ FullyQualifiedErrorId : Failed cmdlet with Error: Invalid APIC Connection Settings.,Cisco.ACI.SCVMM.
PowerShell.SetApicConnInfo

Step 4

Verify that the Cisco APIC connection parameters are set properly on the Cisco APIC SCVMM Agent by entering the following command:

PS C:\Program Files (x86)\ApicVMMService> Get-ApicConnInfo

EndpointAddress        :
Username               :
Password               :
ApicAddresses          : 172.23.139.224
ConnectionStatus       : Connected
adminSettingsFlags     : 0
certificateSubjectName : OpflexAgent
ExtensionData

Installing the APIC Hyper-V Agent on the Hyper-V Server

This section describes how to install the APIC Hyper-V agent on the Hyper-V server.

Before you begin

Scheduled downtime for the Hyper-V node. For more information regarding Hyper-V Maintenance Mode behavior, see: https://technet.microsoft.com/en-us/library/hh882398.aspx

Procedure

Step 1

Log on to the SCVMM server and bring the Hyper-V node into Maintenance Mode.

Step 2

Log in to the Hyper-V server with administrator credentials.

Step 3

On the Hyper-V server in File Explorer, locate the APIC Hyper-V Agent.msi file.

Step 4

Right-click the APIC Hyper-V Agent.msi file and choose Install.

Step 5

In the ApicHypervAgent Setup dialog box, perform the following actions:

1. Check the I accept the terms in the License Agreement check box.

2. Click Install.

3. Click Finish.

Step 6

Follow the steps in Microsoft's documentation to view and bring the apicVSwitch Logical Switch into compliance. Also referred to in this guide as Host Remediate or Logical Switch Instance Remediation: https://technet.microsoft.com/en-us/library/dn249415.aspx

Step 7

Use one of the following methods:

- For large-scale deployments, see Microsoft's documentation for Deploy Certificates by Using Group Policy:

https://technet.microsoft.com/en-us/library/cc770315(v=ws.10).aspx

- For small-scale deployments follow these steps:

You must add OpFlex security certificate in the local system. The Microsoft Hyper-V agent has a security certificate file named OpflexAgent.pfx located in the C:\Program Files (x86)\ApicVMMService folder on the SCVMM server. If the following steps are not performed on your Hyper-V servers, the APIC Hyper-V Agent cannot communicate with the Cisco Application Centric Infrastructure (ACI) fabric leaf switches.

Install the OpFlex security certificate on the Hyper-V Windows Server 2012 local machine’s certificate repository. On each Hyper-V server, install this certificate by performing the following steps:

1. Choose Start > Run.

2. Enter mmc and click OK.

3. In the Console Root window, on the menu bar, choose Add/Remove Snap-in.

4. In the Available Snap-ins field, choose Certificates and click Add.

5. In the Certificates snap-in dialog box, choose the Computer Account radio button, and click Next.

6. In the Select Computer dialog box, choose the Local Computer radio button, and click Finish.

7. Click OK to go back to the main MMC Console window.

8. In the MMC Console window, double-click Certificates (local computer) to expand its view.

9. Right-click Certificates under Personal and choose All Tasks > Import.

10. In the Certificates Import Wizard dialog box, perform the following actions:

1. Click Next.

2. Browse to the Opflex Agent file and click Next.

11. Enter the password for the certificate that was provided when you installed MSI.

12. You must choose the Mark this key as exportable. This will allow you to back up or transport your keys at a later time radio button.

13. Choose the Include all extended properties radio button.

14. Choose the Place all certificates in the following store radio button, browse to locate Personal, and click Next.

15. Click Finish.

16. Click OK.

Step 8

Log on to the SCVMM Sserver and bring the Hyper-V node out of Maintenance Mode.

Step 9

Repeat steps 1 through 8 for each Hyper-V server.

Verifying the APIC SCVMM Agent Installation on SCVMM

This section describes how to verify the APIC SCVMM agent installation on System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Choose Start > Control Panel.

Step 2

In the Control Panel window, enter Control Panel\Programs\Programs and Features in the address bar.

Step 3

Locate Cisco APIC SCVMM Agent. If Cisco APIC SCVMM Agent is present, then the product is installed.

If Cisco APIC SCVMM Agent is not present, then the product is not installed. See the Installing the APIC SCVMM Agent on SCVMM or Installing the APIC Agent on SCVMM Using the Windows Command Prompt section.

Step 4

Verify the ApicVMMService is in RUNNING state through the GUI or CLI:

- GUI method: Choose Start > Run and enter services.msc. In the Service pane, locate the ApicVMMService and verify the state is RUNNING.

- CLI method: From the command prompt, enter the sc.exe query ApicHypervAgent command and verify the state is RUNNING:

sc.exe query ApicVMMService

SERVICE_NAME: ApicVMMService
TYPE : 10 WIN32_OWN_PROCESS
STATE : 4 RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE : 0 (0x0)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0

Verifying the APIC SCVMM Agent Installation on a Highly Available SCVMM

This section describes how to verify the APIC SCVMM agent installation on a Highly Available System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Choose Start > Control Panel.

Step 2

In the Control Panel window, enter Control Panel\Programs\Programs and Features in the address bar.

Step 3

Locate Cisco APIC SCVMM Agent. If Cisco APIC SCVMM Agent is present, then the product is installed.

If Cisco APIC SCVMM Agent is not present, then the product is not installed. See the Installing the APIC SCVMM Agent on SCVMM or Installing the APIC Agent on SCVMM Using the Windows Command Prompt section.

Step 4

Verify the ApicVMMService is in RUNNING state through the GUI or CLI:

- GUI method: Choose Start > Run and enter services.msc. In the Service pane, locate the ApicVMMService and verify the state is RUNNING.

- CLI method: From the command prompt, enter the sc.exe query ApicHypervAgent command and verify the state is RUNNING:

sc.exe query ApicVMMService

SERVICE_NAME: ApicVMMService
TYPE : 10 WIN32_OWN_PROCESS
STATE : 4 RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE : 0 (0x0)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0

Step 5

Choose Start > PowerShell and enter the following commands:

PS C:\Users\administrator.APIC\Downloads> Get-ClusterResource -Name ApicVMMService

Name            State         OwnerGroup          ResourceType
----            -----         ----------          ------------
ApicVMMService  Online        clustervmm07-ha     Generic Service

PS C:\Users\administrator.APIC\Downloads> Get-ClusterCheckpoint -ResourceName ApicVMMService

Resource        Name
--------        ----
ApicVMMService  SOFTWARE\Wow6432Node\Cisco\Apic

PS C:\Users\administrator.APIC\Downloads> Get-ClusterResourceDependency -Resource ApicVMMService

Resource        DependencyExpression
--------        --------------------
ApicVMMService  ([VMM Service clustervmm07-ha])

Verifying the APIC Hyper-V Agent Installation on the Hyper-V Server

This section describes how to verify the APIC Hyper-V agent installation on the Hyper-V server.

Procedure

Step 1

Choose Start > Control Panel.

Step 2

In the Control Panel window, enter Control Panel\Programs\Programs and Features in the address bar.

Step 3

Locate Cisco APIC Hyperv Agent. If Cisco APIC Hyperv Agent is present, then the product is installed.

If Cisco APIC Hyperv Agent is not present, then the product is not installed. See the Installing the APIC Hyper-V Agent on the Hyper-V Server or Installing the APIC Hyper-V Agent on the Hyper-V Server Using the Windows Command Prompt section.

Step 4

Verify the ApicHypervAgent is in RUNNING state through the GUI or CLI:

- GUI method: Choose Start > Run and enter services.msc. In the Service pane, locate the ApicHypervAgent and verify the state is RUNNING.

- CLI method: From the command prompt, enter the sc.exe query ApicHypervAgent command and verify the state is RUNNING:

sc.exe query ApicHypervAgent

SERVICE_NAME: ApicHypervAgent
TYPE : 10 WIN32_OWN_PROCESS
STATE : 4 RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE : 0 (0x0)
SERVICE_EXIT_CODE : 0 (0x0)
CHECKPOINT : 0x0
WAIT_HINT : 0x0

Creating SCVMM Domain Profiles

In this section, the examples of a VMM domain are System Center Virtual Machine Manager (SCVMM) domains. The example tasks are as follows:

- Configuring the VMM domain name and SCVMM controller.

- Creating an attach entity profile and associating it to the VMM domain.

- Configuring a pool.

- Verifying all configured controllers and their operational states.

- Creating a SCVMM Domain Profile Using the GUI

- Configuring the Port Channel Policy

Creating a SCVMM Domain Profile Using the GUI

Before you begin

Before you create a VMM domain profile, you must establish connectivity to an external network using in-band or out-of-band management network on the Application Policy Infrastructure Controller (APIC).

Procedure

Step 1

Log in to the APIC GUI, and then choose Virtual Networking > Inventory.

Step 2

In the Navigation pane, expand VMM Domains, right-click the VM Provider Microsoft and choose Create SCVMM Domain.

Step 3

In the Create SCVMM domain dialog box, in the Name field, enter the domain's name (productionDC).

Step 4

Optional: In the Delimiter field, enter one of the following: |, ~, !, @, ^, +, or =. If you do not enter a symbol, the system default | delimiter will appear in the policy.

Step 5

In the Associated Attachable Entity Profile field, from the drop-down list, choose Create Attachable Entity Profile, and perform the following actions to configure the list of switch interfaces across the span of the VMM domain:

1. In the Create Attachable Access Entity Profile dialog box, in the Profile area, in the Name field, enter the name (profile1), and click Next.

2. In the Association to Interfaces area, expand Interface Policy Group.

3. In the Configured Interface, PC, and VPC dialog box, in the Configured Interfaces, PC, and VPC area, expand Switch Profile.

4. In the Switches field, from the drop-down list, check the check boxes next to the desired switch IDs (101 and 102).

5. In the Switch Profile Name field, enter the name (swprofile1).

6. Expand the + icon to configure interfaces.

7. Choose the appropriate interface ports individually in the switch image (interfaces 1/1, 1/2, and 1/3).

The Interfaces field gets populated with the corresponding interfaces.

8. In the Interface Selector Name field, enter the name (selector1).

9. In the Interface Policy Group field, from the drop-down list, choose Create Interface Policy Group.

10. In the Create Access Port Policy Group dialog box, in the Name field, enter the name (group1).

11. Click Submit.

12. Click Save, and click Save again.

13. Click Submit.

14. In the Select the interfaces area, under Select Interfaces, click the All radio button.

15. Verify that in the vSwitch Policies field, the Inherit radio button is selected.

16. Click Finish.

The Attach Entity Profile is selected and is displayed in the Associated Attachable Entity Profile field.

Step 6

In the VLAN Pool field, from the drop-down list, choose Create VLAN Pool. In the Create VLAN Pool dialog box, perform the following actions:

1. In the Name field, enter the VLAN pool name (VlanRange).

2. In the Allocation Mode field, verify that the Dynamic Allocation radio button is selected.

3. Expand Encap Blocks to add a VLAN block. In the Create Ranges dialog box, enter a VLAN range.

Note

We recommend a range of at least 200 VLAN numbers. Do not define a range that includes the reserved VLAN ID for infrastructure network because that VLAN is for internal use.

4. Click OK, and click Submit.

In the VLAN Pool field, "VlanRange-dynamic" is displayed.

Step 7

Expand SCVMM. In the Create SCVMM Controller dialog box, verify that the Type is SCVMM, and then perform the following actions:

1. In the Name field, enter the name (SCVMM1).

2. To connect to a SCVMM HA Cluster, specify the SCVMM HA Cluster IP address or the SCVMM Cluster Resource DNS name, which was specified during the SCVMM HA installation. See How to Connect to a Highly Available VMM Management Server by Using the VMM Console: https://technet.microsoft.com/en-us/library/gg610673.aspx

3. In the Host Name (or IP Address) field, enter the Fully Qualified Domain Name (FQDN) or IP address of your SCVMM.

4. In the SCVMM Cloud Name field, enter the SCVMM cloud name (ACI-Cloud).

5. Click OK.

6. In the Create SCVMM Domain dialog box, click Submit.

Step 8

Verify the new domain and profiles, by performing the following actions:

1. On the menu bar, choose Virtual Networking > Inventory.

2. In the navigation pane, choose VMM Domains > Microsoft > productionDC > SCVMM1.

3. In the Work pane, view the VMM domain name to verify that the controller is online.

4. In the Work pane, the SCVMM1 properties are displayed including the operational status. The displayed information confirms that connection from the APIC controller to the SCVMM server is established, and the inventory is available.

Configuring the Port Channel Policy

This section describes how to configure the port channel policy.

- Modifying the Interface Port Channel Policy

- Overriding the VMM Domain VSwitch Policies for Blade Servers

Modifying the Interface Port Channel Policy

The Cisco ACI SCVMM Agent synchronizes the SCVMM uplink port profile with the aggregated interface port channel policies and performs an automated update when there are changes to the policy.

To update the policy for Hyper-V servers, perform the following steps.

Procedure

Step 1

Log in to the Cisco APIC GUI, and on the menu bar, choose Fabric > Access Policies.

Step 2

In the navigation pane, expand Interfaces > Leaf Interfaces > Policy Groups.

Step 3

Choose the policy group and check the name of the policy group.

Step 4

Navigate to the policy group and update it based on your requirements (for example, LACP or MAC pinning).

Overriding the VMM Domain VSwitch Policies for Blade Servers

When Blade servers are connected to ACI fabric interface port channel policy will be used between interface and fabric interconnect. When fabric interconnect is configured for LACP you will need to configure the Hyper-V server for MAC pinning mode.

To configure the Hyper-V server for MAC pinning mode perform the following steps.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose Virtual Networking.

Step 2

In the navigation pane, expand VMM Domains > Microsoft > Domain_Name .

Step 3

In the Work pane, click ACTIONS and choose Create VSwitch Policies.

Step 4

On the port channel policy, select the existing policy for mac pinning or create a new policy.

Note

If the hosts are already connected to logical switch, then the SCVMM admin should perform host remediate for all the hosts for uplink policy to take effect.

Verifying the SCVMM VMM Domain and SCVMM VMM

Procedure

In the System Center Virtual Machine Manager Console GUI, the following object has been created by the SCVMM agent for the newly created SCVMM VMM domain and VMM Controller’s rootContName (SCVMM Cloud Name):

1. Click Fabric at the bottom left side pane and under fabric verify the following objects:

Example:

1. Choose Networking > Logical Switches and in the right side pane, the logical switch name is apicVSwitch_VMMdomainName .

2. Choose Networking > Logical Networks and in the right side pane, the logical network name is apicLogicalNetwork_VMMdomainName .

3. Choose Networking > Port Profiles and in the right side pane, the port profile name is apicUplinkPortProfile_VMMdomainName .

2. Click VMs and Services in the bottom left side pane.

Example:

1. Choose VM Networks.

2. In the right side pane, the VM network name is apicInfra|10.0.0.30|SCVMM Controller HostNameORIPAddress filed value|VMMdomainName .

You must use infra VM Network to create VTEP on the Hyper-V server.

Deploying the Logical Switch to the Host on SCVMM

This section describes how to deploy the logical switch to the host on System Center Virtual Machine Manager (SCVMM).

Note

If SCVMM upgrade is performed and hosts are already connected to logical switch then SCVMM admin should perform host remediation for all the hosts for hosts to establish connection to leaf.

Procedure

Step 1

Log in to the SCVMM server, in the Navigation pane, choose Fabric on the bottom left.

Step 2

In the Navigation pane, expand Networking > Logical Switches to ensure the logical switch is created (apicVswitch_cloud1).

Step 3

In the Navigation pane, choose VMs and Services on the bottom left.

Step 4

In the Navigation pane, expand All Hosts.

Step 5

Choose the Hyper-V host folder (Dev8).

Step 6

Right-click the Hyper-V host (Dev8-HV1) and choose Properties.

Step 7

In the Dev8-HV1.inscisco.net Properties dialog box, choose Virtual Switches and perform the following actions:

1. Choose + New Virtual Switch.

2. Choose New Logical Switch.

3. In the Logical switch field, from the drop-down list, choose a logical switch (apicVswitch_cloud1).

4. In the Adapter field, from the drop-down list, choose an adapter (Leaf1-1-1 - Intel(R) Ethernet Server Adapter X520-2 #2).

5. In the Uplink Port Profile field, from the drop-down list, choose an Uplink Port Profile (apicUplinkPortProfile_Cloud01).

6. Click New Virtual Network Adapter, choose the unnamed virtual network adapter, and enter the name (dev8-hv1-infra-vtep).

7. Click Browse.

8. In the Dev8-HV1.inscisco.net Properties dialog box, choose the VM network (apicInfra|10.0.0.30|dev8-scvmm.apic.net|Cloud01) and click OK.

9. In the Virtual Machine Manager dialog box, click OK.

Step 8

Click Jobs on the bottom left.

Step 9

In the History pane, you can check the status of the Change properties of virtual machine host job to ensure that the job has completed.

Step 10

You must refresh the host under SCVMM for the Hyper-V server to reflect proper Hyper-V Host IP address in SCVMM. Once it has been refreshed, the APIC GUI reflects the updated Hyper-V Host IP information.

Enabling the Logical Network on Tenant Clouds

This section describes how to enable the Cisco ACI Integration with SCVMM Tenant Clouds. For more information, see the SCVMM Fabric Cloud and Tenant Clouds.

Procedure

Step 1

Log in to the SCVMM server with SCVMM administrator credentials, and open up the SCVMM Admin Console.

Step 2

On the SCVMM Admin Console, navigate to VMs and Services.

Step 3

In the Navigation pane, expand Clouds, right-click on your target Tenant Cloud (HR_Cloud) and choose Properties.

Step 4

In the Pop-Up Window, in the Navigation pane, choose Logical Networks

1. Locate the logical network which was automatically created as part of associating the VMM Domain to this SCVMM.

2. Click the logical network check box (apicLogicalNetwork_MyVmmDomain).

3. Click OK.

The tenant cloud is now ready to be used within ACI Integration at the Windows Azure Pack Plan configuration page.

Upgrading the Cisco ACI with Microsoft SCVMM Components

If you are trying to upgrade to SCVMM 2016, you must follow the Microsoft procedure and then install the Cisco ACI with Microsoft SCVMM components as a fresh install.

Prerequisites:

If upgrading to SCVMM 2012 R2, Microsoft servers that you integrate into ACI must be updated with the KB2919355 and KB3000850 update rollups prior to upgrading ACI to the 2.2(1) release. The KB2919355 update rollup includes the 2929781 patch, which adds new TLS cipher suites and changes the cipher suite priorities in Windows 8.1 and Windows Server 2012 R2.

You must patch the following Microsoft servers:

- Microsoft Windows Azure Pack Resource Provider Servers

- Microsoft Windows Azure Pack Tenant Site Servers

- Microsoft Windows Azure Pack Admin Site Servers

- Microsoft System Center Service Provider Foundation/Orchestration Servers

- Microsoft System Center 2012 R2 Servers

- Microsoft HyperV 2012 R2 Servers

- Upgrading the ACI Microsoft SCVMM Components Workflow

- Upgrading the APIC SCVMM Agent on SCVMM

- Upgrading the APIC SCVMM Agent on a High Available SCVMM

- Upgrading the APIC Hyper-V Agent

Upgrading the ACI Microsoft SCVMM Components Workflow

This sections describes upgrading the ACI Microsoft SCVMM components workflow.

Procedure

Step 1

Upgrade the APIC Controller and the Switch Software.

For more information, see the Cisco APIC Firmware Management Guide.

Step 2

Upgrade the APIC SCVMM Agent on SCVMM or Upgrade the APIC SCVMM Agent on a Highly Available SCVMM.

For more information, see Upgrading the APIC SCVMM Agent on SCVMM.

For more information, see Upgrading the APIC SCVMM Agent on a High Available SCVMM.

Step 3

Upgrade the APIC Hyper-V Agent.

For more information, see Upgrading the APIC Hyper-V Agent.

Upgrading the APIC SCVMM Agent on SCVMM

This section describes how to upgrade the APIC SCVMM agent on System Center Virtual Machine Manager (SCVMM).

Before you begin

Scheduled downtime for the Microsoft SCVMM Server. The upgrade process will automatically restart the Microsoft System Center Virtual Machine Manager Service, resulting in the SCVMM Service to be temporarily unable to handle any change or query requests.

Procedure

Upgrade the APIC SCVMM agent on SCVMM.

If upgrading from release 1.1(2x) or later:

1. Follow the steps outlined in the Installing the APIC SCVMM Agent on SCVMM.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

If upgrading from a prior release of 1.1(2x):

1. Follow the steps outlined in the Installing the APIC SCVMM Agent on SCVMM.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

2. Follow the steps outline in the Exporting APIC OpFlex Certificate.

3. Follow the steps outline in the Installing the OpflexAgent Certificate.

4. Follow the steps outline in the Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent or Configuring APIC IP Settings with OpflexAgent Certificate on the SCVMM Agent on a Highly Available SCVMM.

Upgrading the APIC SCVMM Agent on a High Available SCVMM

This section describes how to upgrade the APIC SCVMM agent on a high available System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Log in to a Standby node of the Highly Available SCVMM installation.

Step 2

On the SCVMM server in File Explorer, locate the APIC SCVMM Agent.msi file.

Step 3

Right-click APIC SCVMM Agent.msi file and select Install.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

Step 4

In the Cisco APIC SCVMM Agent Setup dialog box, perform the following actions:

1. Click Next.

2. Check the I accept the terms in the License Agreement check box and click Next.

3. Enter your account name and password credentials.

Provide the same credentials as used for the SCVMM console. The Cisco APIC SCVMM agent requires these credentials for the SCVMM operations to be able to function.

The installation process verifies the entered account name and password credentials. If the installation fails, the SCVMM shows an error message and you must re-enter valid credentials.

4. After successful validation of the account name and password credentials, click Install.

5. Click Finish.

Step 5

Repeat steps 1-4 for each Standby Node in the Windows Failover Cluster.

Step 6

Failover from the Current Owner Node of the Highly Available SCVMM installation to one of the newly upgrade Standby Nodes.

Step 7

Follow steps 2-4 on the final Standby Node of the Windows Failover Cluster.

Upgrading the APIC Hyper-V Agent

This section describes how to upgrade the APIC Hyper-V agent.

Before you begin

Scheduled downtime for the Hyper-V node. For more information regarding Hyper-V Maintenance Mode behavior, see: https://technet.microsoft.com/en-us/library/hh882398.aspx

Procedure

Upgrade the APIC Hyper-V agent.

If upgrading from release 1.1(2x) or later:

1. Follow steps 1-8 in the Installing the APIC Hyper-V Agent on the Hyper-V Server. Skip step 7. Step 7 is not required for upgrades as the OpflexAgent certificate is already installed on the Hyper-V node.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

If upgrading from a prior release of 1.1(2x):

1. Follow the steps outlined in the Uninstalling the APIC Hyper-V Agent.

2. Follow steps 1-8 in the Installing the APIC Hyper-V Agent on the Hyper-V Server. Skip step 7. Step 7 is not required for upgrades as the OpflexAgent certificate is already installed on the Hyper-V node.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

Deployment Tenant Policies Prerequisites

Ensure that your computing environment meets the following prerequisites:

- Ensure you have installed the APIC SCVMM Agent.

For details, see Installing the APIC SCVMM Agent on SCVMM.

- Ensure you have installed the APIC Hyper-V Agent.

For details, see Installing the APIC Hyper-V Agent on the Hyper-V Server.

- Ensure you have created a logical switch.

See Microsoft's documentation.

- Ensure you have created a virtual switch.

See Microsoft's documentation.

Creating a Tenant

Procedure

Step 1

On the menu bar, choose TENANTS, and perform the following actions:

1. Click Add Tenant.

The Create Tenant dialog box opens.

2. In the Name field, add the tenant name (ExampleCorp).

Step 2

Click Finish.

See the Cisco APIC Basic Configuration Guide for more information.

Creating an EPG

This section describes how to create an endpoint group (EPG).

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > Tenant Name.

Step 2

In the Navigation pane, expand Tenant Name > Application Profiles > Application Profile Name, right-click Application EPGs, and choose Create Application EPG.

Step 3

In the Create Application EPG dialog box, perform the following actions:

1. In the Name field, enter the name (EPG1).

2. In the Bridge Domain field, from the drop-down list, choose one to associate with the bridge domain.

3. In the Associate to VM Domain Profiles field, click the appropriate radio button and click Next.

4. In the Associated VM Domain Profiles field, click the + icon, and choose a cloud to add (Cloud10).

You have now created an EPG.

Associating the Microsoft VMM Domain with an EPG

This section describes how to create a VM Network by associating the Microsoft VMM domain with an endpoint group (EPG).

Note

Content in the Hypervisors, Virtual Machines, and Virtualization Ratio areas of the Cisco APIC capacity dashboard appears as 0 when SCVMM endpoints are learned in Pre-Provision mode.

Before you begin

Ensure you have created an EPG.

Procedure

Step 1

Log in to the Cisco APIC GUI and on the menu bar, choose Tenants > Tenant Name.

Step 2

In the Navigation pane, expand Tenant Name > Application Profiles > Application Profile Name > Application EPGs and select an existing EPG.

Step 3

In the Navigation pane, choose Domains (VMs and Bare-Metals).

Step 4

In the Domains (VM and Bare-Metals) pane, click on the ACTIONS and choose Add VMM Domain Association.

Step 5

In the Add VMM Domain Association dialog box, click the Deploy Immediacy field radio button for either Immediate or On Demand.

See EPG Policy Resolution and Deployment Immediacy for more information.

Step 6

In the Add VMM Domain Association dialog box, click the Resolution Immediacy field radio button for either Immediate, On Demand, or Pre-Provision.

See EPG Policy Resolution and Deployment Immediacy for more information.

You have now created a VM Network.

Step 7

Optional: In the Delimiter field, use a single character as the VM Network Name delimiter, enter one of the following: |, ~, !, @, ^, +, or = . If you do not enter a symbol, the system default of | will be used.

Verifying the EPG is Associated with the VMM Domain on APIC

This section describes how to verify the endpoint group association with the VMM domain on Application Policy Infrastructure Controller (APIC).

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose Virtual Networking > Inventory.

Step 2

In the navigation pane, expand VMM Domains > Microsoft > Cloud10 > Controller > Controller1 > Distributed Virtual Switch > SCVMM|Tenant|SCVMM|EPG1|Cloud1.

The name of the new VM Network is in the following format: Tenant Name|Application Profile Name|Application EPG Name|Microsoft VMM Domain .

Step 3

In the PROPERTIES pane, verify the EPG associated with the VMM domain, the VM Network, and the details such as NIC NAME, VM NAME, IP, MAC, and STATE.

Verifying the EPG is Associated with the VMM Domain on SCVMM

This section describes how to verify the endpoint group (EPG) associated with the VMM domain on System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Open the Virual Machine Manager Console icon on your desktop.

Step 2

In the bottom left pane, click on VMs and Services or press Ctrl+M.

Step 3

In the VMs and Services pane, click on VM Networks and verify the EPG associated with the VMM domain.

The EPG associated with the VMM domain is in the following format: Tenant Name|Application Profile Name|Application EPG Name|Microsoft VMM Domain .

Creating a Static IP Address Pool

Static IP Address Pools enable an Microsoft SCVMM Server to statically assign IP Address to virtual machines during the VM Template Deployment phase. This feature removes the need to request a DHCP address from a DHCP Server. This feature is most often used to deploy server VMs which require statically assigned IP Addresses in the network such as: Windows Active Directory Domain Controllers, DNS Servers, DHCP Servers, Network Gateways, etc.

For more information regarding Static IP address pools, see the Microsoft Documentation: https://technet.microsoft.com/en-us/library/jj721568.aspx#BKMK_StaticIPAddressPools

With Cisco ACI SCVMM Integration - the Cisco APIC can automate the deployment of a Static IP Address Pool to a VM Network, bypassing the need to perform these operations on the Microsoft SCVMM Server itself.

Before you begin

Ensure an EPG is associated to a Microsoft SCVMM VMM Domain.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > Tenant Name.

Step 2

In the Navigation pane, expand Tenant Name > Application Profiles > Application Profile Name > Application EPGs > Your Target EPG , right-click Subnets, and choose Create EPG Subnet.

Step 3

In the Create EPG Subnet dialog box, perform the following actions:

1. Enter a default Gateway IP in address/mask format.

2. Click Submit.

Step 4

Right-click on the newly created subnet and choose Create Static IP Pool Policy.

Step 5

In the Create Static IP Pool Policy dialog box, perform the following actions:

1. Enter a Name (IP).

2. Enter a Start IP and End IP.

3. Enter optional Static IP Pool policies.

The DNS Servers, DNS Search Suffix, Wins Servers fields Allow a list of entries, simply use semicolon to separate the entries. For example within the DNS Servers Field:

192.168.1.1;192.168.1.2

Note

When configuring the Start IP and End IP, ensure they are within the same Subnet as the Gateway defined in Step 3. If not deployment of the Static IP Address Pool to SCVMM fails.

Only 1 Static IP Address Pool will be used for a given EPG. Do not create multiple Static IP Pool Policies under a Subnet as the others will not take effect.

The Static IP Address Pool Policy follows the VMM Domain association. If this EPG is deployed to multiple SCVMM Controllers in the same VMM Domain, then the same Static IP Addresses will be deployed, causing duplicate IP Addresses. For this scenario, deploy an addition EPG with a non-overlapping Address pool and create the necessary policies and contracts for the endpoints to communicate.

Connecting and Powering on the Virtual Machine

This section describes how to connect and power on the virtual machine.

Procedure

Step 1

Log in to the SCVMM server, choose VMs and Services > All Hosts, and choose one of the hosts.

Step 2

In the VMs pane, right-click on the VM host that you want to associate to the VM Network and choose Properties.

Step 3

In the Properties dialog box, choose Hardware Configuration, and choose a network adapter (Network Adapter 1).

Step 4

In the Network Adapter 1 pane, perform the following actions to connect to a VM network:

1. Click the Connect to a VM network radio button.

2. Click the Browse button.

3. Verify the list of VM networks, which lists all of the VM networks to which the hypervisor is associated.

Step 5

Power on the virtual machine.

Verifying the Association on APIC

This section describes how to verify the association on Application Policy Infrastructure Controller (APIC).

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose Virtual Networking > Inventory.

Step 2

In the navigation pane, expand VMM Domains > Microsoft > Cloud10 > Controller > Controller1 > Hypervisors > Hypervisor1 > Virtual Machines to verify the association.

Viewing EPGs on APIC

This section describes how to view endpoint groups (EPGs) on the Application Policy Infrastructure Controller (APIC).

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > Tenant Name.

Step 2

In the Navigation pane, expand Tenant Name > Application Profiles > VMM > Application EPGs > EPG1.

Step 3

In the Application EPG - EPG1 pane, click the OPERATIONAL button, and verify if the endpoint group is present.

Troubleshooting APIC to SCVMM Connectivity

Use the ApicVMMService logs to debug the System Center Virtual Machine Manager (SCVMM) server.

Procedure

Step 1

Log in to the SCVMM server, go to the ApicVMMService logs. Located at C:\Program Files (X86)\ApicVMMService\Logs.

Step 2

Check the ApicVMMService logs to debug.

If you are unable to debug, on the SCVMM server copy all the ApicVMMService logs from C:\Program Files (X86)\ApicVMMService\Logs and send them to Cisco Tech Support.

Troubleshooting Leaf to Hyper-V Host Connectivity

Use the ApicHypervAgent logs to debug the Hyper-V servers.

Procedure

Step 1

Log in to the Hyper-V servers, go to the ApicHypervAgent logs. Located at C:\Program Files (x86)\ApicHypervAgent\Logs.

Step 2

Check the ApicHypervAgent logs to debug.

If you are unable to debug, on the Hyper-V servers copy all the ApicHypervAgent logs from C:\Program Files (x86)\ApicHypervAgent\Logs and send them to Cisco Tech Support.

Troubleshooting the EPG Configuration Issue

If during the lifetime of the endpoint group (EPG), the VLAN ID of the EPG changes on the APIC, then SCVMM needs to update the VLAN configuration on all virtual machines for the new setting to take effect.

Procedure

To perform this operation run the following PowerShell commands on the SCVMM server:

Example:

$VMs = Get-SCVirtualMachine
$VMs | Read-SCVirtualMachine
$NonCompliantAdapters=Get-SCVirtualNetworkAdapter -All | Where-Object
{$_.VirtualNetworkAdapterComplianceStatus -eq "NonCompliant"}
$NonCompliantAdapters | Repair-SCVirtualNetworkAdapter

Installing the APIC Agent on SCVMM Using the Windows Command Prompt

This section describes how to install the APIC Agent on System Center Virtual Machine Manager (SCVMM) using the Windows Command Prompt.

Procedure

Step 1

Log in to the SCVMM server with SCVMM administrator credential.

Step 2

Launch the command prompt, change to the folder where you copied the APIC SCVMM Agent.msi file, and execute following commands:

Example:

C:\>cd MSIPackage

C:\MSIPackage>dir
Volume in drive C has no label.
Volume Serial Number is 726F-5AE6

Directory of C:\MSIPackage

02/24/2015  01:11 PM    <DIR>          .
02/24/2015  01:11 PM    <DIR>          ..
02/24/2015  05:47 AM         3,428,352 APIC SCVMM Agent.msi
1 File(s)      3,428,352 bytes
2 Dir(s)  37,857,198,080 bytes free

C:\MSIPackage>msiexec.exe /I "APIC SCVMM Agent.msi" /Qn ACCOUNT="inscisco\Administrator"
PASSWORD="MyPassword" /log "C:\InstallLog.txt"
C:\MSIPackage>sc.exe query ApicVMMService

SERVICE_NAME: ApicVMMService
TYPE               : 10  WIN32_OWN_PROCESS
STATE              : 4  RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE    : 0  (0x0)
SERVICE_EXIT_CODE  : 0  (0x0)
CHECKPOINT         : 0x0
WAIT_HINT          : 0x0

Step 3

If the msiexec.exe installer package succeeds, it finishes without any warning or error messages. If it fails, it displays the appropriate warning or error message.

Installing the APIC Hyper-V Agent on the Hyper-V Server Using the Windows Command Prompt

This section describes how to install the APIC Hyper-V Agent on the Hyper-V server using the windows Command Prompt.

Procedure

Step 1

Log in to the Hyper-V server with administrator credentials.

Step 2

Launch the command prompt, change to the folder where you copied the APIC Hyper-V Agent.msi file, and execute the following commands:

Example:

C:\>cd MSIPackage

C:\MSIPackage>dir
Volume in drive C has no label.
Volume Serial Number is C065-FB79

Directory of C:\MSIPackage

02/24/2015  01:11 PM    <DIR>          .
02/24/2015  01:11 PM    <DIR>          ..
02/24/2015  05:44 AM           958,464 APIC Hyper-V Agent.msi
1 File(s)        958,464 bytes
2 Dir(s)  749,486,202,880 bytes free

C:\MSIPackage>msiexec.exe /I "APIC Hyper-V Agent.msi" /log "C:\InstallLog.txt"

C:\MSIPackage>msiexec.exe /I "APIC Hyper-V Agent.msi" /Qn /log "C:\InstallLog.txt"

C:\MSIPackage>sc.exe query ApicHyperVAgent

SERVICE_NAME: ApicHyperVAgent
TYPE               : 10  WIN32_OWN_PROCESS
STATE              : 4  RUNNING
(STOPPABLE, NOT_PAUSABLE, ACCEPTS_SHUTDOWN)
WIN32_EXIT_CODE    : 0  (0x0)
SERVICE_EXIT_CODE  : 0  (0x0)
CHECKPOINT         : 0x0
WAIT_HINT          : 0x0

Step 3

Repeat steps 1 through 2 for each Hyper-V server.

If the msiexec.exe installer package succeeds, it finishes without any warning or error messages. If it fails, it displays the appropriate warning or error message.

ACI SCVMM PowerShell Cmdlets

This section describes how to list the Cisco Application Centric Infrastructure (ACI) System Center Virtual Machine Manager (SCVMM) PowerShell cmdlets, help, and examples.

Procedure

Step 1

Log in to the SCVMM server, choose Start > Run > Windows PowerShell.

Step 2

Enter the following commands:

Example:

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. All rights reserved.

PS C:\Program Files (x86)\ApicVMMService> cd C:\Program Files (x86)\ApicVMMService>
PS C:\Program Files (x86)\ApicVMMService> Import-Module .\ACIScvmmPsCmdlets.dll
PS C:\Program Files (x86)\ApicVMMService> Add-Type -Path .\Newtonsoft.Json.dll
PS C:\Program Files (x86)\ApicVMMService> Get-Command -Module ACIScvmmPsCmdlets

CommandType     Name                      ModuleName
-----------     ----                      ----------
Cmdlet          Get-ACIScvmmOpflexInfo    ACIScvmmPsCmdlets
Cmdlet          Get-ApicConnInfo          ACIScvmmPsCmdlets
Cmdlet          Get-ApicCredentials       ACIScvmmPsCmdlets
Cmdlet          New-ApicOpflexCert        ACIScvmmPsCmdlets
Cmdlet          Read-ApicOpflexCert       ACIScvmmPsCmdlets
Cmdlet          Set-ApicConnInfo          ACIScvmmPsCmdlets
Cmdlet          Set-ApicCredentials       ACIScvmmPsCmdlets

Step 3

Generating help:

Example:

commandname -?

Step 4

Generating examples:

Example:

get-help commandname -examples

MAC Address Configuration Recommendations

This section describes the MAC address configuration recommendations.

- Both Dynamic and Static MAC are supported.

- Static MAC for the VM Network adapter is recommended if you want the VM inventory to show up quickly on APIC.

- If you choose Dynamic MAC there is a delay for the VM inventory to show up on APIC. The delay is because Dynamic MACs are not learned by SCVMM right away.

Note

The Data plane works fine even though the VM inventory does not show up.

Shows the MAC address section in the Properties pane.

Uninstalling the Cisco ACI with Microsoft SCVMM Components

This section describes how to uninstall the Cisco Application Centric Infrastructure (ACI) with Microsoft System Center Virtual Machine Manager (SCVMM) components.

Procedure

Step 1

Detach all virtual machines from the VM networks.

See Microsoft's documentation.

Step 2

Delete the Infra VLAN tunnel endpoint (VTEP) and APIC logical switches on all Hyper-Vs.

See Microsoft's documentation.

Step 3

Verify the APIC GUI to make sure all the VMs and hosts are disconnected.

Step 4

Delete the VMM Domain from the Application Policy Infrastructure Controller (APIC).

See Guidelines for Deleting VMM Domains.

Step 5

Verify the logical switch and logical networks are removed from SCVMM.

Step 6

Uninstall the APIC SCVMM Agent on SCVMM or on a Highly Available SCVMM.

See Uninstalling the APIC SCVMM Agent.

See Uninstalling the APIC SCVMM Agent on a Highly Available SCVMM

- Uninstalling the APIC SCVMM Agent

- Uninstalling the APIC SCVMM Agent on a Highly Available SCVMM

Uninstalling the APIC SCVMM Agent

This section describes how to uninstall the APIC SCVMM Agent.

Procedure

Step 1

Log in to the SCVMM server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click ApicVMMService and choose Uninstall.

This uninstalls the APIC SCVMM Agent.

Step 4

To verify if the APIC SCVMM Agent is uninstalled, in the Programs and Features window, verify that ApicVMMService is not present.

Uninstalling the APIC SCVMM Agent on a Highly Available SCVMM

This section describes how to install the Application Policy Infrastructure Controller (APIC) SCVMM agent on a Highly Available System Center Virtual Machine Manager (SCVMM).

Procedure

Step 1

Log in to any node within the Highly Available SCVMM Failover Cluster.

Step 2

Open the Failover Cluster Manager Application.

Step 3

In the Windows Failover Cluster Manager window, select ApicVMMService in the Highly Available SCVMM Roles/Resources tab.

Step 4

Right-click on the ApicVMMService Role and choose Take Offline.

Step 5

Once the Role is offline, right-click on the ApicVMMService Role and choose Remove.

Step 6

On each node within the Highly Available SCVMM Failover Cluster, perform the following actions to uninstall the APIC SCVMM Agent:

1. Log in to the SCVMM server.

2. Choose Start > Control Panel > Uninstall a Program.

3. In the Programs and Features window, right-click ApicVMMService and choose Uninstall.

This uninstalls the APIC SCVMM Agent.

4. To verify if the APIC SCVMM Agent is uninstalled, in the Programs and Features window, verify that ApicVMMService is not present.

Downgrading the Cisco APIC Controller and the Switch Software with Cisco ACI and Microsoft SCVMM Components

This section describes how to downgrade the Cisco APIC and the switch software with Cisco ACI and Microsoft System Center Virtual Machine Manager (SCVMM) components.

Procedure

Step 1

Uninstall the Cisco APIC SCVMM Agent on SCVMM or on a highly available SCVMM.

See Uninstalling the APIC SCVMM Agent.

See Uninstalling the APIC SCVMM Agent on a Highly Available SCVMM.

Step 2

Downgrade the Cisco APIC Hyper-V Agent by Completing the following steps:

1. Log in to the SCVMM server and bring the Hyper-V node into maintenance mode.

2. Log in to the Hyper-V server with administrator credentials.

3. Uninstall the Cisco APIC Hyper-V agent.

4. Install Cisco APIC Hyper-V agent to the version that the Cisco ACI fabric is being downgraded to.

Step 3

Downgrade the switch software.

Step 4

Downgrade the Cisco APIC.

See the Cisco APIC Firmware Management Guide for details.

Step 5

On the SCVMM server, install the SCVMM agent for the version that the Cisco ACI fabric is being downgraded to.

See Installing the APIC SCVMM Agent on SCVMM

See Installing the APIC SCVMM Agent on a Highly Available SCVMM

Step 6

Follow the steps in Microsoft's documentation to view and bring the Cisco APIC vSwitch logical switch into compliance.

See How to View Host Network Adapter Settings and Increase Compliance with Logical Switch Settings in VMM .

Exporting APIC OpFlex Certificate

This section describes how to back up APIC OpFlex certificate to a file which can be used to deploy new Hyper-V nodes, System Center Virtual Machine Manager (SCVMM) and Windows Azure Pack Resource Provider servers to the ACI Fabric when the original OpFlex certificate cannot be located.

Procedure

Step 1

Log in to a Hyper-V node which is currently a member of the ACI Fabric.

Step 2

Export the certificate from the Hyper-V node by performing the following actions:

1. Choose Start > Run and type certlm.msc to launch the Certificate Manager.

2. In the navigation pane, right-click on Certificates - Local Computer and choose Find Certificates.

3. In the Find Certificate dialog box, perform the following actions:

- In the Find in field, from the drop-down list, choose All certificate stores.

- In the Contains field, enter OpflexAgent.

- In the Look in Field field, from the drop-down list, choose Issued By.

- Click Find Now.

Your result list should have a single Certificate in the list.

4. Right-click on the newly found OpflexAgent certificate and choose Export.

The Certificate Export Wizard will appear.

Step 3

In the Certificate Export Wizard dialog box, perform the following actions:

1. In the Welcome to the Certificate Export Wizard dialog box, click Next

2. In the Export Private Key dialog box, choose the Yes, export the private key radio button, and click Next.

3. In the Export File Format dialog box, choose the Personal Information Exchange - PKCS #12 (.PFX) radio button, check the Include all certificates in the certificate path if possible and Export all extended properties check box. Click Next.

4. In the Security dialog box, check the Password check box, enter your PFX password and enter your PFX password again to confirm. Click Next.

Your PFX password will be used later to import the PFX file on the target machine.

5. In the File to Export dialog box, enter the filename you wish to save the exported file (C:\OpflexAgent.pfx) and click Next.

6. In the Completing the Certificate Export Wizard dialog box, review all your specified settings are correct and click Finish.

7. The Certificate Export Wizard dialog box will appear with The export was successful. and click Ok.

Step 4

Copy the PFX file to a known location.

You can deploy the certificate through an Active Directory Group Policy or copy the file to your various Microsoft Servers which host your SCVMM, Windows Azure Pack Resource Provider, and Hyper-V services for integration into the ACI Fabric.

Chapter 12. Cisco ACI with Microsoft Windows Azure Pack

This chapter contains the following sections:

- About Cisco ACI with Microsoft Windows Azure Pack

- Getting Started with Cisco ACI with Microsoft Windows Azure Pack

- Upgrading the Cisco ACI with Microsoft Windows Azure Pack Components

- Use Case Scenarios for the Administrator and Tenant Experience

- Uninstalling the Cisco ACI with Microsoft Windows Azure Pack Components

- Downgrading Cisco APIC and the Switch Software with Cisco ACI and Microsoft Windows Azure Pack Components

About Cisco ACI with Microsoft Windows Azure Pack

Cisco Application Centric Infrastructure (ACI) integrates in Microsoft Windows Azure Pack to provide a self-service experience for the tenant.

ACI enhances the network management capabilities of the platform. Microsoft Windows Azure Pack is built on top of an existing Microsoft System Center Virtual Machine Manager (SCVMM) installation. Cisco ACI has integration points at each of these layers, enabling you to leverage the work performed in a SCVMM environment and use it in a Microsoft Windows Azure Pack installation.

- Cisco ACI with Microsoft Windows Azure Pack—Microsoft Windows Azure Pack for Windows Server is a collection of Microsoft Azure technologies that include the following capabilities:

- Management portal for tenants

- Management portal for administrators

- Service management API

- Cisco ACI with Microsoft System Center Virtual Machine Manager —For information about how to set up Cisco ACI with Microsoft System Center Virtual Machine Manager (SCVMM), see details in Cisco ACI with Microsoft SCVMM Solution Overview.

Note

You cannot configure direct server return (DSR) through Windows Azure Pack. If you want to configure DSR, you must do so in Cisco APIC. See the chapter "Configuring Direct Server Return" in the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide for information.

- Cisco ACI with Microsoft Windows Azure Pack Solution Overview

- Physical and Logical Topology

- About the Mapping of ACI Constructs in Microsoft Windows Azure Pack

Cisco ACI with Microsoft Windows Azure Pack Solution Overview

Cisco Application Centric Infrastructure (ACI) integrates in Microsoft Windows Azure Pack to provide a self-service experience for tenants. ACI resource provider in Windows Azure Pack drives the Application Policy Infrastructure Controller (APIC) for network management. Networks are created in System Center Virtual Machine Manager (SCVMM) and are available in Windows Azure Pack for respective tenants. ACI Layer 4 to Layer 7 capabilities for F5 and Citrix load balancers and stateless firewall are provided for tenants. For details, see the About Load Balancing.

Windows Azure Pack for Windows Server is a collection of Microsoft Azure technologies, available to Microsoft customers at no additional cost for installation into your data center. It runs on top of Windows Server 2012 R2 and System Center 2012 R2 and, through the use of the Windows Azure technologies, enables you to offer a rich, self-service, multi-tenant cloud, consistent with the public Windows Azure experience.

Windows Azure Pack includes the following capabilities:

- Management portal for tenants—a customizable self-service portal for provisioning, monitoring, and managing services such as networks, bridge domains, VMs, firewalls, load balancers, external connectivity, and shared services. See the User Portal GUI.

- Management portal for administrators—a portal for administrators to configure and manage resource clouds, user accounts, and tenant offers, quotas, pricing, Web Site Clouds, Virtual Machine Clouds, and Service Bus Clouds.

- Service management API—a REST API that helps enable a range of integration scenarios including custom portal and billing systems.

See Use Case Scenarios for the Administrator and Tenant Experience for details.

Physical and Logical Topology

Topology of a typical Windows Azure Pack deployment with ACI Fabric

The above figure shows a representative topology of a typical Windows Azure Pack deployment with Cisco Application Centric Infrastructure (ACI) fabric. Connectivity between Windows Azure Pack and Application Policy Infrastructure Controller (APIC) is over the management network. Tenants interface is only with Windows Azure Pack either through the GUI or REST API. Tenants do not have direct access to APIC.

ACI in Resource Provider Framework

About the Mapping of ACI Constructs in Microsoft Windows Azure Pack

This section shows a table of the mapping of Cisco Application Centric Infrastructure (ACI) constructs in Microsoft Windows Azure Pack.

Mapping of ACI and Windows Azure Pack constructs

Windows Azure Pack

ACI

Subscription

Tenant

Network

EPG

Firewall Rule

Intra-tenant contract

Shared Service

Inter-tenant contract

SCVMM Cloud

VM Domain

Getting Started with Cisco ACI with Microsoft Windows Azure Pack

This section describes how to get started with Cisco ACI with Microsoft Windows Azure Pack.

Before you install Cisco ACI with Microsoft Windows Azure Pack, download and unzip the folder containing the Cisco ACI and matching Microsoft integration files for the Cisco APIC release.

1. Go to Cisco's Application Policy Infrastructure Controller (APIC) website.

2. Choose All Downloads for this Product > APIC Software.

3. Choose the release version and the matching zipped folder.

4. Click Download.

5. Unzip the zipped folder.

Note

Cisco ACI with Microsoft Windows Azure Pack only supports ASCII characters. Non-ASCII characters are not supported.

Ensure that English is set in the System Locale settings for Windows, otherwise Cisco ACI with Windows Azure Pack will not install. Also, if the System Locale is modified to a non-English Locale after installation, the integration components may fail when communicating with Cisco APIC and the Cisco ACI fabric.

- Prerequisites for Getting Started with Cisco ACI with Microsoft Windows Azure Pack

- Installing, Setting Up, and Verifying the Cisco ACI with Microsoft Windows Azure Pack Components

Prerequisites for Getting Started with Cisco ACI with Microsoft Windows Azure Pack

Before you get started, ensure that you have verified that your computing environment meets the following prerequisites:

- Ensure Cisco Application Centric Infrastructure (ACI) with Microsoft System Center Virtual Machine Manager (SCVMM) has been set up.

For more information, see Getting Started with Cisco ACI with Microsoft SCVMM.

- Ensure that Microsoft Windows Azure Pack Update Rollup 5, 6, 7, 9, 10, or 11 is installed.

See Microsoft's documentation.

- Ensure that Windows Server 2016 is installed.

See Microsoft's documentation.

- Ensure that Hyper-V Host is installed.

See Microsoft's documentation.

- Ensure a cloud is configured on SCVMM.

See Microsoft's documentation.

- Ensure a VM cloud is configured on Windows Azure Pack.

See Microsoft's documentation.

- Ensure "default" AEP exists with infrastructure VLAN enabled.

- Ensure "default" and "vpcDefault" bridge domains and corresponding "default" and "vpcDefault" EPGs exist in tenant common.

- Ensure you have the Cisco MSI files for APIC Windows Azure Pack Resource and the Host Agent.

For more information, see Getting Started with Cisco ACI with Microsoft SCVMM.

Note

Symptom: When you either create or update a plan it may fail with an error message.

Condition: If you have configured Microsoft's Windows Azure Pack without the FQDN, you will encounter the following error message:

Cannot validate the new quota settings because one of the underlying services failed to respond. Details: An error has occurred.

Workaround: When you configure the VM Clouds, follow Microsoft's Windows Azure Pack UI instructions which informs you to use the FQDN for your SCVMM server.

Installing, Setting Up, and Verifying the Cisco ACI with Microsoft Windows Azure Pack Components

This section describes how to install, set up, and verify the Cisco ACI with Microsoft Windows Azure Pack components.

Component

Task

Install ACI Azure Pack Resource Provider

See Installing ACI Azure Pack Resource Provider.

Install the OpflexAgent certificate

See Installing the OpflexAgent Certificate.

Configure ACI Azure Pack Resource Provider Site

See Configuring ACI Azure Pack Resource Provider Site.

Install ACI Azure Pack Admin site extension

See Installing ACI Azure Pack Admin Site Extension.

Install ACI Azure Pack tenant site extension

See Installing ACI Azure Pack Tenant Site Extension.

Set up the ACI

See Setting Up ACI.

Verify the Windows Azure Pack Resource Provider

See Verifying the Windows Azure Pack Resource Provider.

- Installing ACI Azure Pack Resource Provider

- Installing the OpflexAgent Certificate

- Configuring ACI Azure Pack Resource Provider Site

- Installing ACI Azure Pack Admin Site Extension

- Installing ACI Azure Pack Tenant Site Extension

- Setting Up ACI

- Verifying the Windows Azure Pack Resource Provider

Installing ACI Azure Pack Resource Provider

This section describes how to install ACI Azure Pack Resource Provider on the Windows Azure Pack server.

Procedure

Step 1

Log in to the Microsoft Service Provider Foundation Server which provides VM Clouds in the Windows Azure Pack environment. Locate and copy over ACI Azure Pack - Resource Provider Site.msi file.

Step 2

Double-click the ACI Azure Pack - Resource Provider Site.msi file.

Step 3

In the Setup dialog box, perform the following actions to install ACI Azure Pack - Resource Provider:

1. Check the I accept the terms in the License Agreement check box.

2. Click Install.

3. Click Install.

4. Click Finish.

Installing the OpflexAgent Certificate

This section describes how to install the OpflexAgent Certificate.

Procedure

Step 1

Log in to the Windows Azure Pack server with administrator credentials.

Step 2

Use one of the following methods:

- For large-scale deployments, see Microsoft's documentation for Deploy Certificates by Using Group Policy:

https://technet.microsoft.com/en-us/library/cc770315(v=ws.10).aspx.

- For small-scale deployments follow these steps:

You must add OpFlex security certificate to the local system. The ACI Windows Azure Pack resource provider uses the same security certificate file from the Cisco ACI SCVMM installation process located on your SCVMM Server at: C:\Program Files (x86)\ApicVMMService\OpflexAgent.pfx. Copy this file to the Windows Azure Pack Resource Provider Server. If the following steps are not performed on your ACI Windows Azure Pack resource provider servers, the APIC ACI Windows Azure Pack resource provider cannot communicate with the Application Policy Infrastructure Controller (APIC) .

Install the OpFlex security certificate on the ACI Windows Azure Pack resource provider Windows Server 2012 local machine’s certificate repository. On each ACI Windows Azure Pack resource provider server, install this certificate by performing the following steps:

1. Choose Start > Run.

2. Enter mmc and click OK.

3. In the Console Root window, on the menu bar, choose Add/Remove Snap-in.

4. In the Available Snap-ins field, choose Certificates and click Add.

5. In the Certificates snap-in dialog box, choose the Computer Account radio button, and click Next.

6. In the Select Computer dialog box, choose the Local Computer radio button, and click Finish.

7. Click OK to go back to the main MMC Console window.

8. In the MMC Console window, double-click Certificates (local computer) to expand its view.

9. Right-click Certificates under Personal and choose All Tasks > Import.

10. In the Certificates Import Wizard dialog box, perform the following actions:

1. Click Next.

2. Browse to the Opflex Agent file and click Next.

11. Enter the password for the certificate that was provided when you installed MSI.

12. You must choose the Mark this key as exportable. This will allow you to back up or transport your keys at a later time radio button.

13. Choose the Include all extended properties radio button.

14. Choose the Place all certificates in the following store radio button, browse to locate Personal, and click Next.

15. Click Finish.

16. Click OK.

Configuring ACI Azure Pack Resource Provider Site

This section describes how to configure ACI Azure Pack Resource Provider IIS Site on the Windows Azure Pack server.

Procedure

Step 1

Log in to the Windows Azure Pack server and open the Internet Information Services Manager Application.

Step 2

Navigate to Application Pools > Cisco-ACI.

Step 3

Click the Advanced Settings in the Actions tab.

1. Locate the Identity field and click on the ellipses to the left of the scroll bar.

2. Select Custom Account and input your account name and password credentials for Service Provider Foundation Administrator. The Service Provider Foundation Administrator user account should have the following group memberships: Administrators, SPF_Admin. This user account is required as the Resource Provider queries the attached SCVMM servers. In addition, the User Credentials must have permission to write to the Local Machine Registry and have Read/Write access to the following directory for Resource Provider Logging:

C:\Windows\System32\config\systemprofile\AppData\Local

3. Click OK to exit Application Pool Identity.

Step 4

Click OK to exit Advanced Settings

Installing ACI Azure Pack Admin Site Extension

This section describes how to install ACI Azure Pack Admin Site Extension on the Windows Azure Pack server.

Procedure

Step 1

Log in to the Windows Azure Pack server and locate the ACI Azure Pack - Admin Site Extension.msi file.

Step 2

Double-click the ACI Azure Pack - Admin Site Extension.msi file.

Step 3

In the Setup dialog box, perform the following actions to install ACI Azure Pack - Admin Site Extension:

1. Check the I accept the terms in the License Agreement check box.

2. Click Install.

3. Click Finish.

Installing ACI Azure Pack Tenant Site Extension

This section describes how to install ACI Azure Pack Tenant Site Extension on the Windows Azure Pack server.

Procedure

Step 1

Log in to the Windows Azure Pack server and locate the ACI Azure Pack - Tenant Site Extension.msi file.

Step 2

Double-click the ACI Azure Pack - Tenant Site Extension.msi file.

Step 3

In the Setup dialog box, perform the following actions to install ACI Azure Pack - Tenant Site Extension:

1. Check the I accept the terms in the License Agreement check box.

2. Click Install.

3. Click Finish.

Setting Up ACI

This section describes how to setup ACI.

Procedure

Step 1

Log in to the Service Management Portal.

Step 2

In the navigation pane, choose ACI.

If you do not see ACI, click Refresh.

Step 3

Click the QuickStart icon.

Step 4

In the QuickStart pane, perform the following actions in order:

1. Click on Register your ACI REST endpoint.

2. In the ENDPOINT URL field, enter the resource provider address:Cisco-ACI port (http://resource_provider_address:50030).

3. In the USERSNAME field, enter the user name (domain administrator).

4. In the PASSWORD field, enter the password (domain administrator password).

Step 5

Choose the ACI > Setup tab, and perform the following actions:

1. In the APIC ADDRESS field, enter the APIC IP Address(es).

2. In the CERTIFICATE NAME field, enter OpflexAgent.

Verifying the Windows Azure Pack Resource Provider

This section describes how to verify the Windows Azure Pack Resource Provider.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose the QuickStart Cloud icon.

Ensure the Register your ACI REST Endpoint link is greyed out.

Step 4

In the aci pane, choose SETUP.

Ensure that you see the APIC Address has valid apic addresses and the Certificate name is OpflexAgent.

Upgrading the Cisco ACI with Microsoft Windows Azure Pack Components

Prerequisites:

Microsoft servers that you integrate into ACI must be updated with the KB2919355 and KB3000850 update rollups prior to upgrading ACI to the 2.0(1) release. The KB2919355 update rollup includes the 2929781 patch, which adds new TLS cipher suites and changes the cipher suite priorities in Windows 8.1 and Windows Server 2012 R2.

You must patch the following Microsoft servers:

- Microsoft Windows Azure Pack Resource Provider Servers

- Microsoft Windows Azure Pack Tenant Site Servers

- Microsoft Windows Azure Pack Admin Site Servers

- Microsoft System Center Service Provider Foundation/Orchestration Servers

- Microsoft System Center 2012 R2 Servers

- Microsoft HyperV 2012 R2 Servers

To upgrade the .msi files for each Cisco ACI with Windows Azure Pack Integration follow the Microsoft general guidelines for upgrading Windows Azure Pack Components listed per Update Rollup. The general guidelines are:

- If the system is currently operational (handling customer traffic), schedule downtime for the Azure servers. The Windows Azure Pack does currently not support rolling upgrades.

- Stop or redirect customer traffic to sites that you consider satisfactory.

- Create backups of the computers.

Note

If you are using virtual machines (VMs), take snapshots of their current state.

If you are not using VMs, take a backup of each MgmtSvc-* folder in the inetpub directory on each machine that has a Windows Azure Pack component installed.

Collect information and files that are related to your certificates, host headers, or any port changes.

Once the upgrade is complete and has been verified, follow Hyper-V best practices regarding managing VM snapshots: https://technet.microsoft.com/en-us/library/dd560637(v=ws.10).aspx

- Upgrading the ACI Windows Azure Pack Workflow

- Upgrading the ACI Windows Azure Pack Resource Provider

- Upgrading the ACI Azure Pack Admin Site Extension

- Upgrading the ACI Azure Pack Tenant Site Extension

Upgrading the ACI Windows Azure Pack Workflow

This section describes upgrading the ACI Windows Azure Pack Workflow.

Procedure

Step 1

Upgrade the APIC Controller and the Switch Software.

See the Cisco APIC Firmware Management Guide.

Step 2

Upgrade the ACI Windows Azure Pack.

If upgrading from a prior release of 1.1(2x):

1. You must uninstall the APIC Windows Azure Pack Resource Provider, see Uninstalling the APIC Windows Azure Pack Resource Provider.

2. Follow the steps that are outlined in the Installing, Setting Up, and Verifying the Cisco ACI with Microsoft Windows Azure Pack Components.

3. Skip to step 6, Upgrade the APIC SCVMM Agent on SCVMM or Upgrade the APIC SCVMM Agent on a Highly Available SCVMM.

If upgrading from release 1.1(2x) or later:

1. Proceed to step 3.

Step 3

Upgrade the ACI Windows Azure Pack Resource Provider.

For more information, see Upgrading the ACI Windows Azure Pack Resource Provider.

Step 4

Upgrade the ACI Azure Pack Admin Site Extension.

For more information, see Upgrading the ACI Azure Pack Admin Site Extension.

Step 5

Upgrade the ACI Azure Pack Tenant Site Extension.

For more information, see Upgrading the ACI Azure Pack Tenant Site Extension.

Step 6

Upgrade the APIC SCVMM Agent on SCVMM or Upgrade the APIC SCVMM Agent on a Highly Available SCVMM.

For more information, see Upgrading the APIC SCVMM Agent on SCVMM.

For more information, see Upgrading the APIC SCVMM Agent on a High Available SCVMM.

Step 7

Upgrade the APIC Hyper-V Agent.

For more information, see Upgrading the APIC Hyper-V Agent.

Upgrading the ACI Windows Azure Pack Resource Provider

This section describes how to upgrade the ACI Windows Azure Pack resource provider.

Procedure

Upgrade the ACI Windows Azure Pack resource provider.

If upgrading from release 1.1(2x) or later:

1. Follow the steps outlined in the Installing ACI Azure Pack Resource Provider.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

2. Follow the steps outline in the Configuring ACI Azure Pack Resource Provider Site.

If upgrading from a prior release of 1.1(2x):

1. Follow the steps outlined in the Uninstalling the APIC Windows Azure Pack Resource Provider.

2. Follow the steps outlined in the Installing ACI Azure Pack Resource Provider.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

3. Follow the steps outline in the Configuring ACI Azure Pack Resource Provider Site.

Upgrading the ACI Azure Pack Admin Site Extension

This section describes how to upgrade the ACI Azure Pack Admin site extension.

Procedure

Upgrade the ACI Azure Pack Admin site extension.

1. Follow the steps outlined in the Installing ACI Azure Pack Admin Site Extension.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

Upgrading the ACI Azure Pack Tenant Site Extension

This section describes how to upgrade the ACI Azure Pack Tenant site extension.

Procedure

Upgrade the ACI Azure Pack Tenant site extension.

1. Follow the steps outlined in the Installing ACI Azure Pack Tenant Site Extension.

The MSI packages handles uninstalling the previous version and installing the new version as part of the upgrade.

Use Case Scenarios for the Administrator and Tenant Experience

This section describes the use case scenarios for the administrator and tenant experience.

Note

If the shared service consumer is in a different VRF than the provider, route leaking between the VRFs will automatically occur in order to enable the communication.

Use case

Shared Plan

VPC Plan

User

Task

Creating a plan

This allows the administrator to create plans with their own moderation values.

Yes

Yes

Admin

1. See About Plan Types.

Admin

2. See Creating a Plan.

Creating a tenant

This allows the administrator to create a tenant.

Yes

Yes

Admin

See Creating a Tenant.

Creating and verifying networks in a shared plan

This allows the tenant to create and verify networks in a shared plan.

Yes

No

Tenant

1. See Creating Networks in a Shared Plan.

Tenant

2. See Verifying the Network you Created on Microsoft Windows Azure Pack on APIC.

Creating the network in VPC plan

This allows the tenant to create networks in a VPC plan.

No

Yes

Tenant

See Creating the Network in VPC Plan.

Creating a bridge domain in a VPC plan and creating a network and associating to the bridge domain

This applies only in a virtual private cloud (VPC) plan. This allows a tenant to bring its own IP address space for the networks.

No

Yes

Tenant

1. See Creating a Bridge Domain in a VPC Plan.

Tenant

2. See Creating a Network and Associating to a Bridge Domain in a VPC Plan.

Creating a firewall within the same subscription.

This allows the tenant to create a firewall within the same subscription.

Yes

Yes

Tenant

See Creating a Firewall Within the Same Subscription.

Allowing tenants to provide shared services

This allows tenants to create networks, attach compute services (servers) to those networks, and offer the connectivity to these services to other tenants. The administrator needs to explicitly enable this capability in the plan.

Yes

Yes

Admin

1. See Allowing Tenants to Provide Shared Services.

Tenant

2. See Providing a Shared Service.

Tenant

3. See Adding Access Control Lists or Deleting Access Control Lists.

Admin

4. See Allowing Tenants to Consume Shared Service.

Tenant

5. See Setting up the Shared Service to be Consumed.

Admin

6. See Viewing the Shared Service Providers and Consumers.

Allowing tenants to consume NAT firewall and ADC load balancer services

No

Yes

Admin

1. See Allowing Tenants to Consume NAT Firewall and ADC Load Balancer Services.

Tenant

2. See Adding NAT Firewall Layer 4 to Layer 7 Services to a VM Network.

Tenant

3. See Adding NAT Firewall Port-Forwarding Rules for a VM Network.

Tenant

4. See Adding NAT Firewall With a Private ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network.

Tenant

5. See Adding a Public ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network.

Tenant

6. See Adding ADC Load Balancer Configuration for a VM Network.

Managing shared services

This allows the administrator to deprecate a shared service from new tenants and revoke a tenant access from a shared service.

Yes

Yes

Admin

See Deprecating a Shared Service from New Tenants.

See Revoking a Tenant from a Shared Service.

Creating VMs and attaching to networks

Yes

Yes

Tenant

See Creating VMs and Attaching to Networks.

Creating the load balancer

Yes

Yes

Admin

1. See About Load Balancing.

Admin

2. See Importing the Device Package on APIC.

Admin

3. See Configuring the Load Balancer Device on APIC using XML POST.

Admin

4. See Creating a Load Balancer to a Plan.

Tenant

5. See Configuring the Load Balancer.

Creating external connectivity

This allows a tenant network to initiate outgoing traffic destined outside the fabric and to attract traffic from outside.

Yes

Yes

APIC Admin

1. See About L3 External Connectivity.

APIC Admin

2. See Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack.

APIC Admin

3. See Creating a Contract to be Provided by the l3extinstP "default".

APIC Admin

4. See Creating a Contract to be Provided by the l3extinstP "vpcDefault".

Tenant

5. See Creating a Network for External Connectivity.

Tenant

6. See Creating a Firewall for External Connectivity.

APIC Admin

7. See Verifying Tenant L3 External Connectivity on APIC.

- Tenant Tasks

About Plan Types

The administrator creates the plan with their own values. The plan types are as follows:

Shared Infrastructure

Virtual Private Cloud

Isolated Networks

Yes

Yes

Firewall

Yes

Yes

Provider DHCP

Yes

Yes *

Shared Load Balancer

Yes

Yes *

Public Internet Access

Yes

Yes

Shared Services between Tenants

Yes

Yes

Bring your own address space (Private Address Space) and DHCP Server

No

Yes

* In a Virtual Private Cloud (VPC) plan, a load balancer and DHCP is not supported for private address space. Both features are still offered to a tenant, but owned by the shared infrastructure.

About Plan Options

This section describes about the plan options.

- APIC Tenant: Disable Auto Creation of an APIC Tenant

- Default: Unselected.

Unselected: Cisco ACI Azure Pack Resource Provider will automatically create/delete an APIC tenant. The APIC tenant name will be the Subscription ID (GUID) of the Windows Azure Pack tenant. No manual intervention by the APIC admin is required as the Resource Provider will handle all the necessary mapping.

Selected: Cisco ACI Azure Pack Resource Provider will NOT automatically create/delete an APIC tenant. The APIC tenant must be explicitly mapped to a Windows Azure Pack Subscription ID. Once this mapping is established on the APIC, the Azure Pack Tenant will be able to perform his normal operations of working with networks, firewalls, load balancers, etc.

- Features enabled by Disabling Auto Creation of an APIC Tenant

- SCVMM and Windows Azure Pack VM Network names take on the APIC Tenant Name rather than a GUID. This increases readability for an SCVMM Admin and Azure Pack Tenant as VM Networks will have a friendly name rather than a GUID.

- Plan Quotas: Azure Pack Plan Admins can now create Plans which limit the number of EPGs, BDs, and VRFs an Azure Pack Tenant can create.

- The EPG, BD, and VRF created by the APIC admin under an APIC Tenant count against their quota for Azure Pack Plan.

- Example 1: Plan Admin creates an Azure Pack plan with a limit of 5 EPGs. Azure Pack Tenant creates 4 EPGs and the APIC Admin creates an EPG for the Azure Pack Tenant. The Azure Pack Tenant has now reached his plan quota and cannot create EPGs until he is below plan quota.

- Example 2: Plan Admin creates an Azure Pack plan with a limit of 5 EPGs. Azure Pack Tenant creates 5 EPGs. An APIC Admin creates an EPG for the Azure Pack Tenant. The Azure Pack Tenant has now reached his plan quota and cannot create EPGs until he is below plan quota.

- These quotas are enforced for the Azure Pack Tenant, but do not apply to the APIC Admin. An APIC admin can continue to create EPGs, BDs, and VRFs for an Azure Pack Tenant even when the Tenant has gone beyond his quota.

- All Plan Types - Publishing EPGs

- Ability for an APIC admin to push EPGs to Windows Azure Pack tenants.

- An APIC admin can now create EPGs for their Azure Pack Tenants by creating the EPG on the APIC and associating it to the VMM Domain (SCVMM Cloud) associated with the Tenant’s Plan.

- The “default” Application Profile under the tenant is considered Azure Pack Tenant owned space. This means that the Azure Pack Tenant is allowed to create contracts with it and delete it.

- All other Application Profiles will be considered APIC Admin owned space. These EPGs will be available to the Azure Pack Tenant for consumption, but the Azure Pack tenant will not be allowed to modify, delete, or work with the EPG outside of associating with a Virtual Machine Network Adapter.

Creating a Plan

This allows the administrator to create plans with their own values.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose PLANS.

Step 3

Choose NEW.

Step 4

In the NEW pane, choose CREATE PLAN.

Step 5

In the Let's Create a Hosting Plan dialog box, enter the name for your plan (Bronze) and click the arrow for next.

Step 6

In the Select services for a Hosting Plan dialog box, choose your features. Check the check box for VIRTUAL MACHINE CLOUDS, NETWORKING (ACI), and click the arrow for next.

Step 7

In the Select add-ons for the plan dialog box, click the checkmark for next.

Step 8

In the plans pane, wait for the plan (Bronze) to be created and choose the (Bronze) plan arrow to configure it.

Step 9

In the Bronze pane under plan services, choose Virtual Machine Clouds arrow.

Step 10

In the virtual machine clouds pane, perform the following actions:

1. In the VMM MANAGEMENT SERVER field, choose the VMM management server (172.23.142.63).

2. In the VIRTUAL MACHINE CLOUD field, choose the cloud name (Cloud01).

3. Scroll down and choose Add templates.

4. In the Select templates to add to this plan dialog box, check the check box for your template(s) and click the checkmark for next.

5. Scroll down to Custom Settings, check the Disable built-in network extensions for tenants check box for SCVMM.

6. Click SAVE at the bottom.

7. Once completed, click OK.

Step 11

In the Service Management Portal, click the back arrow which takes you back to the Bronze pane.

Step 12

In the Bronze pane under plan services, click Networking (ACI) and perform the following actions:

1. In the PLAN TYPE field, from the drop-down list, choose the plan type.

2. For Virtual Private Cloud plan type, enter a valid value between 1 to 4000 number for the “Maximum EPG allowed per tenant”, “Maximum BD allowed per tenant” and “Maximum CTX allowed per tenant”.

For Shared Infrastructure Plan type, enter a valid value between 1 to 4000 number for the “Maximum EPG allowed per tenant”.

3. Click SAVE.

Step 13

Click OK.

You have now created a plan.

Creating a Tenant

This allows the administrator to create a tenant.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose USER ACCOUNTS.

Step 3

Choose NEW.

Step 4

In the NEW pane, scroll down and choose USER ACCOUNTS.

Step 5

In the NEW pane, choose QUICK CREATE and perform the following actions:

1. In the ENTER EMAIL ADDRESS field, enter the email address (tenant@domain.com).

2. In the ENTER PASSWORD field, enter the password.

3. In the CONFIRM PASSWORD field, enter the password again.

4. In the CHOOSE PLAN field, choose a plan (BRONZE).

5. Click CREATE.

6. Click OK.

You have now created a tenant.

Step 6

For Windows Azure Pack Tenants associated with Plans that “Disable Auto Creation of an APIC Tenant”, Take note of the Azure Pack Tenant Login and Subscription ID.

1. Log in to the APIC GUI, on the menu bar, choose TENANTS > Tenant Name. The Tenant is the intended APIC Tenant targeted for Azure Pack Subscription mapping.

2. Select the Policy Tab.

3. In the GUID section, click the + icon to add a new Azure Pack subscription mapping.

4. Populate the GUID with the Azure Pack Tenant Subscription ID and the Account Name with the Azure Pack Login Account.

5. Click Submit to save the changes.

Note

An APIC Tenant can only map to a single Azure Pack Tenant Subscription ID.

Allowing Tenants to Provide Shared Services

This option allows tenants to create networks, attach compute services (servers) to those networks, and offer the connectivity to these services to other tenants. The administrator needs to explicitly enable this capability in the plan.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose PLANS.

1. Choose a plan.

2. Click Networking (ACI) under plan services.

Step 3

In the networking (aci) pane, check the allow tenants to provide shared services check box and click SAVE.

Allowing Tenants to Consume Shared Service

Even though tenants are allowed to create a shared service to be used by other tenants, the administrator needs to select the services which can be shared across tenants. This procedure shows how Windows Azure Pack admin can choose the shared services for the plan:

Before you begin

- Ensure the administrator has allowed tenants to provide shared services.

- Ensure the tenant has provided a shared service.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose PLANS.

Step 3

In the plans pane, choose PLANS.

1. Click on the plan (Gold).

Step 4

In the Gold pane, choose Networking (ACI).

Step 5

In the networking (aci) pane, check the shared service check box you want to give access to (DBSrv).

Step 6

Click SAVE.

Allowing Tenants to Consume NAT Firewall and ADC Load Balancer Services

Cisco Application Centric Infrastructure (ACI) has the concept of service graphs, which allows a tenant to insert service nodes performing various Layer 4 to Layer 7 functions between two endpoint groups (EPGs) within the fabric.

Windows Azure Pack with ACI integration now includes the ability to easily and seamlessly provision and deploy services graphs in a Virtual Private Cloud (VPC) setting where the external NAT firewall IP and external ADC load balancer sit within a shared space. The most common use-case for this is the service provider model where a limited number externally accessible IP addresses are available for use, in which case various port-forwarding techniques or load balancing of an entire EPG is done against the one external IP.

Tenants within Azure Pack can utilize a strict VPC model where all their networking is contained within the tenant virtual routing and forwarding (VRF) or a split VRF model where an APIC admin can configure a set of L3Out which is accessible by all tenants utilizing the ACI fabric. The following are instructions on providing a split VRF workflow allowing Azure Pack tenants to consume the Layer 4 to Layer 7 service devices as well as being allocated public addresses for the services provided from within the tenant VRF:

Before you begin

- Ensure the Application Policy Infrastructure Controller (APIC) administrator has configured at least 1 Layer 4 to Layer 7 resource pool in tenant common. For information, see the chapter "Configuring Layer 4 to Layer 7 Resource Pools" in the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide .

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose PLANS.

Step 3

In the plans pane, choose PLANS.

1. Click on the plan (Gold).

Step 4

In the Gold pane, choose Networking (ACI).

Step 5

In the networking (aci) pane, choose the Layer 4 to Layer 7 services pool provisioned by the APIC admin for Azure Pack consumption.

Step 6

Click SAVE.

Viewing the Shared Service Providers and Consumers

This allows the administrator to view the shared service providers and consumers.

Before you begin

- Ensure the administrator has allowed tenants to provide shared services.

- Ensure the tenant has provided a shared service.

- Ensure the administrator has enabled the shared service on a plan.

- Ensure the tenant has set up the shared service to be consumed.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the ACI pane, choose SHARED SERVICES to view the shared service providers.

Step 4

Click on the provider.

Step 5

Click INFO to display all the users that are consuming this shared service.

Deprecating a Shared Service from New Tenants

This allows the administrator to deprecate a shared service from new tenants.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose PLANS.

Step 3

In the plans pane, choose the plan (Gold).

Step 4

In the gold pane, choose Networking (ACI).

Step 5

In the networking (aci) pane, uncheck the service from the plan and click SAVE.

You have deprecated the shared service from tenants.

Revoking a Tenant from a Shared Service

This allows the administrator to revoke a tenant from a shared service.

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose the shared service (DBSrv).

Step 4

Click INFO to ensure that the user you want to revoke is present in that shared service.

Step 5

In the navigation pane, choose PLANS.

Step 6

In the plans pane, choose the plan (Gold).

Step 7

In the gold pane, choose Networking (ACI).

Step 8

In the networking (aci) pane, uncheck the service from the plan and click SAVE.

Step 9

In the navigation pane, choose ACI.

Step 10

In the aci pane, choose SHARED SERVICES.

Step 11

In the aci pane, choose the shared service (DBSrv) and click INFO.

Step 12

In the Revoke Consumers of DBSrv dialog box, check the check box of the user you want to revoke.

Step 13

Click the checkmark.

About Load Balancing

VLAN, virtual routing and forwarding (VRF) stitching is supported by traditional service insertion models, the Application Policy Infrastructure Controller (APIC) can automate service insertion while acting as a central point of policy control. The APIC policies manage both the network fabric and services appliances. The APIC can configure the network automatically so that traffic flows through the services. The APIC can also automatically configure the service according to the application's requirements, which allows organizations to automate service insertion and eliminate the challenge of managing the complex techniques of traditional service insertion.

See the Cisco APIC Layer 4 to Layer 7 Services Deployment Guide for more information.

You must perform the following tasks to deploy Layer 4 to Layer 7 services using the APIC GUI:

Import the device package.

Only the administrator can import the device package.

See Importing the Device Package on APIC.

Configure and post the XML POST to Application Policy Infrastructure Controller (APIC)

Refer to Microsoft's Windows Azure Pack Services section about the device package.

Only the administrator can configure and post the XML POST.

See Configuring the Load Balancer Device on APIC using XML POST.

Creating a load balancer to a plan

The VIP range to Windows Azure Pack is set.

Only the administrator can create a load balancer to a plan.

See Creating a Load Balancer to a Plan.

Configure the load balancer

Only the tenant can configure the load balancer.

See Configuring the Load Balancer.

- Importing the Device Package on APIC

- Configuring the Load Balancer Device on APIC using XML POST

- Creating a Load Balancer to a Plan

Importing the Device Package on APIC

Only the administrator can import the device package. The administrator can import a device package into the Application Policy Infrastructure Controller (APIC) so that the APIC knows what devices you have and what the devices can do.

Before you begin

Ensure you have downloaded the device package.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose L4-L7 SERVICES > PACKAGES.

Step 2

In the navigation pane, choose Quick Start.

Step 3

In the Quick Start pane, choose Import a Device Package.

Step 4

In the Import Device Package dialog box, perform the following action:

1. Click BROWSE and locate your device package such as F5 or Citrix device package.

2. Click SUBMIT.

Configuring the Load Balancer Device on APIC using XML POST

Only the administrator can configure and post the XML POST.

Before you begin

- The device package file should be uploaded on the Application Policy Infrastructure Controller (APIC).

See Cisco APIC Layer 4 to Layer 7 Device Package Development Guide for more information.

- The tenant common should have the two bridge domains named "default" and "vpcDefault". Ensure that the subnets being used by the tenant who is consuming the load balancer is added to these bridge domains. Typically you would have created these bridge domains and subnets while setting up the DHCP infrastructure for Windows Azure Pack tenants.

- For a non-VPC plan, the backend interface of the load balancer should be placed in the default EPG under the tenant common that was created above. For a VPC plan, the EPG should be "vpcDefault".

- The VIP interface of the load balancer should be placed in an EPG of your choice which should be linked to external world.

See Cisco APIC Layer 4 to Layer 7 Device Package Development Guide for L3 extOut external connectivity outside the Fabric.

- (Optional) If desired, ensure the VIP subnet is linked with L3 or L2 extOut. One VIP per EPG will be allocated.

Procedure

Step 1

These are example XML POSTs for Citrix and F5:

1. Citrix example XML POST:

Example:

<polUni dn="uni">
<fvTenant dn="uni/tn-common" name="common">

<vnsLDevVip name="MyLB" devtype="VIRTUAL">

<!-- Device Package -->
<vnsRsMDevAtt tDn="uni/infra/mDev-Citrix-NetScaler-1.0"/>

<!-- VmmDomain -->
<vnsRsALDevToDomP tDn="uni/vmmp-VMware/dom-mininet"/>

<vnsCMgmt name="devMgmt" host="172.31.208.179" port="80"/>
<vnsCCred name="username" value="nsroot"/>
<vnsCCredSecret name="password" value="nsroot"/>

<vnsDevFolder key="enableFeature" name="EnableFeature">
<vnsDevParam key="LB" name="lb_1" value="ENABLE"/>
<vnsDevParam key="CS" name="cs_1" value="ENABLE"/>
<vnsDevParam key="SSL" name="ssl_1" value="ENABLE"/>
</vnsDevFolder>
<vnsDevFolder key="enableMode" name="EnableMode_1">
<vnsDevParam key="USIP" name="usip_1" value="DISABLE"/>
<vnsDevParam key="USNIP" name="usnip_1" value="ENABLE"/>
</vnsDevFolder>

<vnsCDev name="ADC1" devCtxLbl="C1">
<vnsCIf name="1_1"/>
<vnsCIf name="mgmt"/>

<vnsCMgmt name="devMgmt" host="172.31.208.179" port="80"/>
<vnsCCred name="username" value="nsroot"/>
<vnsCCredSecret name="password" value="nsroot"/>
</vnsCDev>

<vnsLIf name="C5">
<vnsRsMetaIf tDn="uni/infra/mDev-Citrix-NetScaler-1.0/mIfLbl-outside"/>
<vnsRsCIfAtt tDn="uni/tn-common/lDevVip-MyLB/cDev-ADC1/cIf-[1_1]"/>
</vnsLIf>
<vnsLIf name="C4">
<vnsRsMetaIf tDn="uni/infra/mDev-Citrix-NetScaler-1.0/mIfLbl-inside"/>
<vnsRsCIfAtt tDn="uni/tn-common/lDevVip-MyLB/cDev-ADC1/cIf-[1_1]"/>
</vnsLIf>

</vnsLDevVip>

<vnsAbsGraph name ="MyLB">

<!-- Node2 Provides SLB functionality -->
<vnsAbsNode name = "Node2" funcType="GoTo" >

<vnsRsDefaultScopeToTerm tDn="uni/tn-common/AbsGraph-MyLB/AbsTermNodeProv-Output1/outtmnl"/>

<vnsAbsFuncConn name = "C4">
<vnsRsMConnAtt tDn="uni/infra/mDev-Citrix-NetScaler-1.0/mFunc-LoadBalancing/mConn-external" />
</vnsAbsFuncConn>

<vnsAbsFuncConn name = "C5" attNotify="true">
<vnsRsMConnAtt tDn="uni/infra/mDev-Citrix-NetScaler-1.0/mFunc-LoadBalancing/mConn-internal" />
</vnsAbsFuncConn>

<vnsAbsDevCfg>
<vnsAbsFolder key="Network"
name="network"
scopedBy="epg">
<vnsAbsFolder key="nsip" name="snip1">
<vnsAbsParam key="ipaddress" name="ip1" value="5.5.5.251"/>
<vnsAbsParam key="netmask" name="netmask1" value="255.255.255.0"/>
<vnsAbsParam key="hostroute" name="hostroute" value="DISABLED"/>
<vnsAbsParam key="dynamicrouting" name="dynamicrouting" value="ENABLED"/>
<vnsAbsParam key="type" name="type" value="SNIP"/>
</vnsAbsFolder>
</vnsAbsFolder>
</vnsAbsDevCfg>

<vnsAbsFuncCfg>
<vnsAbsFolder key="internal_network"
name="internal_network"
scopedBy="epg">
<vnsAbsCfgRel name="internal_network_key"
key="internal_network_key"
targetName="network/snip1"/>
</vnsAbsFolder>
</vnsAbsFuncCfg>

<vnsRsNodeToMFunc tDn="uni/infra/mDev-Citrix-NetScaler-1.0/mFunc-LoadBalancing"/>
</vnsAbsNode>

<vnsAbsTermNodeCon name = "Input1">
<vnsAbsTermConn name = "C1"/>
</vnsAbsTermNodeCon>

<vnsAbsTermNodeProv name = "Output1">
<vnsAbsTermConn name = "C6"/>
</vnsAbsTermNodeProv>

<vnsAbsConnection name = "CON1" adjType="L2">
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsTermNodeCon-Input1/AbsTConn" />
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsNode-Node2/AbsFConn-C4" />
</vnsAbsConnection>

<vnsAbsConnection name = "CON3" adjType="L2">
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsNode-Node2/AbsFConn-C5" />
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsTermNodeProv-Output1/AbsTConn" />
</vnsAbsConnection>

</vnsAbsGraph>

</fvTenant>
</polUni>

2. F5 example XML POST:

Example:

<polUni dn="uni">
<fvTenant name="common">

<fvBD name="MyLB">
<fvSubnet ip="6.6.6.254/24" />
<fvRsCtx tnFvCtxName="default"/>
</fvBD>

<vnsLDevVip name="MyLB" devtype="VIRTUAL">
<vnsRsMDevAtt tDn="uni/infra/mDev-F5-BIGIP-1.1.1"/>
<vnsRsALDevToDomP tDn="uni/vmmp-VMware/dom-mininet"/>
<vnsCMgmt name="devMgmt" host="172.31.210.88" port="443"/>
<vnsCCred name="username" value="admin"/>
<vnsCCredSecret name="password" value="admin"/>

<vnsLIf name="internal">
<vnsRsMetaIf tDn="uni/infra/mDev-F5-BIGIP-1.1.1/mIfLbl-internal"/>
<vnsRsCIfAtt tDn="uni/tn-common/lDevVip-MyLB/cDev-BIGIP-1/cIf-[1_1]"/>
</vnsLIf>

<vnsLIf name="external">
<vnsRsMetaIf tDn="uni/infra/mDev-F5-BIGIP-1.1.1/mIfLbl-external"/>
<vnsRsCIfAtt tDn="uni/tn-common/lDevVip-MyLB/cDev-BIGIP-1/cIf-[1_2]"/>
</vnsLIf>

<vnsCDev name="BIGIP-1">
<vnsCIf name="1_1"/>
<vnsCIf name="1_2"/>

<vnsCMgmt name="devMgmt" host="172.31.210.88" port="443"/>
<vnsCCred name="username" value="admin"/>
<vnsCCredSecret name="password" value="admin"/>

<vnsDevFolder key="HostConfig" name="HostConfig">
<vnsDevParam key="HostName" name="HostName" value="example22-bigip1.ins.local"/>
<vnsDevParam key="NTPServer" name="NTPServer" value="172.23.48.1"/>
</vnsDevFolder>

</vnsCDev>

</vnsLDevVip>
<vnsAbsGraph name = "MyLB">
<vnsAbsTermNodeCon name = "Consumer">
<vnsAbsTermConn name = "Consumer">
</vnsAbsTermConn>
</vnsAbsTermNodeCon>
<!-- Node1 Provides Virtual-Server functionality -->
<vnsAbsNode name = "Virtual-Server" funcType="GoTo">

<vnsAbsFuncConn name = "internal" attNotify="yes">
<vnsRsMConnAtt
tDn="uni/infra/mDev-F5-BIGIP-1.1.1/mFunc-Virtual-Server/mConn-internal" />
</vnsAbsFuncConn>
<vnsAbsFuncConn name = "external">
<vnsRsMConnAtt
tDn="uni/infra/mDev-F5-BIGIP-1.1.1/mFunc-Virtual-Server/mConn-external" />
</vnsAbsFuncConn>
<vnsRsNodeToMFunc
tDn="uni/infra/mDev-F5-BIGIP-1.1.1/mFunc-Virtual-Server"/>
<vnsAbsDevCfg>
<vnsAbsFolder key="Network" name="webNetwork">

<!-- Active Bigip SelfIP -->
<vnsAbsFolder key="ExternalSelfIP" name="External1" devCtxLbl="ADC1">
<vnsAbsParam key="SelfIPAddress" name="seflfipaddress"
value="6.6.6.251"/>
<vnsAbsParam key="SelfIPNetmask" name="selfipnetmask"
value="255.255.255.0"/>
<vnsAbsParam key="Floating" name="floating"
value="NO"/>
</vnsAbsFolder>
<vnsAbsFolder key="InternalSelfIP" name="Internal1" devCtxLbl="ADC1">
<vnsAbsParam key="SelfIPAddress" name="seflfipaddress"
value="12.0.251.251"/>
<vnsAbsParam key="SelfIPNetmask" name="selfipnetmask"
value="255.255.0.0"/>
<vnsAbsParam key="Floating" name="floating"
value="NO"/>
</vnsAbsFolder>
<vnsAbsFolder key="Route" name="Route">
<vnsAbsParam key="DestinationIPAddress" name="DestinationIPAddress"
value="0.0.0.0" />
<vnsAbsParam key="DestinationNetmask" name="DestinationNetmask"
value="0.0.0.0"/>
<vnsAbsParam key="NextHopIPAddress" name="NextHopIP"
value="6.6.6.254"/>
</vnsAbsFolder>
</vnsAbsFolder>
</vnsAbsDevCfg>
<vnsAbsFuncCfg>
<vnsAbsFolder key="NetworkRelation" name="webNetwork">
<vnsAbsCfgRel key="NetworkRel" name="webNetworkRel"
targetName="webNetwork"/>
</vnsAbsFolder>
</vnsAbsFuncCfg>
</vnsAbsNode>
<vnsAbsTermNodeProv name = "Provider">
<vnsAbsTermConn name = "Provider" >
</vnsAbsTermConn>
</vnsAbsTermNodeProv>
<vnsAbsConnection name = "CON3" adjType="L3">
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsTermNodeCon-Consumer/AbsTConn" />
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsNode-Virtual-Server/AbsFConn-external" />
</vnsAbsConnection>
<vnsAbsConnection name = "CON1" adjType="L2">
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsNode-Virtual-Server/AbsFConn-internal" />
<vnsRsAbsConnectionConns tDn="uni/tn-common/AbsGraph-MyLB/AbsTermNodeProv-Provider/AbsTConn" />
</vnsAbsConnection>
</vnsAbsGraph>
</fvTenant>

</polUni>

Step 2

These are the configurable parameters for Citrix and F5:

1. Configurable parameters for Citrix:

Parameter

Sample Value

Description

vnsLDevVip name

"MyLB"

This value is an identifier for your load balancer and is shown in the Windows Azure Pack admin portal in the plan section for the load balancer selection. You can modify this globally throughout the XML POST with the same alternate value.

vnsRsALDevToDomP tDn

"uni/vmmp-VMware/dom-mininet"

This is the VMM Domiain where your load balancer VM sits. For example, if you have a virtual load balancer you can associate it with a vCenter VMM domain, a SCVMM, or a physical domain.

Note

Whichever domain you give it should have an associated VLAN range with it.

vnsCMgmt name="devMgmt" host

"172.31.208.179"

This is the IP address of the load balancer that communicates to Cisco Application Centric Infrastructure (ACI) fabric.

vnsCCred name

"username"

This is the username.

vnsCCredSecret name

"password"

This is the password.

vnsAbsParam key

"ipaddress"

This is the IP address which the fabric identifies for this device.

vnsAbsParam key="ipaddress" name="ip1" value

"5.5.5.251"

This IP address should be one of your bridge domains.

2. Configurable parameters for F5:

Parameter

Sample Value

Description

fvBD name

"MyLB"

This value is an identifier for your load balancer and is shown in the Windows Azure Pack admin portal in the plan section for the load balancer selection. You can modify this globally throughout the XML POST with the same alternate value.

vnsRsALDevToDomP tDn

"uni/vmmp-VMware/dom-mininet"

This can be any VMM domain with a valid VLAN ENCAP Block.

Note

In this Windows Azure Pack load balancer configuration, this VMM domain has no other relevance for the LB configuration. This is used for backward compatibility.

vnsCMgmt name="devMgmt" host

"172.31.210.88"

This is the IP address of the load balancer that communicates to ACI fabric.

vnsCCred name

"username"

This is the username.

vnsCCredSecret name

"password"

This is the password.

Step 3

POST one of the device packages for either F5 or Citrix.

Creating a Load Balancer to a Plan

Only the administrator can import the device package.

Before you begin

- Import the device package.

- Configure and post the XML POST to Application Policy Infrastructure Controller (APIC) .

Procedure

Step 1

Log in to the Service Management Portal (Admin Portal).

Step 2

In the Navigation pane, choose PLANS.

Step 3

In the plans pane, choose the plan that you want to add a load balancer (shareplan).

Step 4

In the shareplan pane, choose Networking (ACI).

Step 5

In the networking (aci) pane, perform the following actions to add a shared load balancer:

1. Check the shared load balancer check box

2. In the LB DEVICE ID IN APIC field, from the drop-down list, choose the load balancer (MyLB).

3. In the VIP RANGE field, provide the VIP range (5.5.5.1 - 5.5.5.100).

4. Click SAVE.

Note

You can have a single load balancer that is shared across different plans as long as the VIP ranges do not over lap.

About L3 External Connectivity

Layer 3 (L3) external connectivity is an Cisco Application Centric Infrastructure (ACI) feature to connect ACI fabric to an external network by L3 routing protocols, including static routing, OSPF, EIGRP, and BGP. By setting up L3 external connectivity for Microsoft Windows Azure Pack, it allows a tenant network to initiate outgoing traffic destined outside the fabric and to attract traffic from outside. The assumption of this feature is the tenant virtual machine IP addresses are visible outside the fabric without NAT, ACI L3 external connectivity does not include NAT.

- Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack

- Creating a Contract to be Provided by the l3extinstP "default"

- Creating a Contract to be Provided by the l3extinstP "vpcDefault"

Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack

To configure Layer 3 (L3) external connectivity for Windows Azure Pack, you must meet the following prerequisites:

- Ensure you have logged in to the Application Policy Infrastructure Controller (APIC) GUI, on the menu bar, choose TENANT > common.

- Create a l3ExtOut called “default”, refer to BD “default”.

- Create l3extInstP name="defaultInstP" under the l3ExtOut. This is to be used by shared service tenants.

See the Cisco APIC Basic Configuration Guide for L3 external connectivity configuration.

- Ensure you have logged in to the APIC GUI, on the menu bar, choose TENANT > common.

- Create a l3ExtOut called "vpcDefault", refer to BD "vpcDefault".

- Create l3extInstP name="vpcDefaultInstP" under this l3ExtOut.

This is to be used by VPC tenants.

See the Cisco APIC Basic Configuration Guide for configuring external connectivity for tenants.

Windows Azure Pack leverages the common l3ExtOut configuration with no special requirement other than the naming convention highlighted above

Creating a Contract to be Provided by the l3extinstP "default"

This section describes how to creating a contract to be provided by the l3extinstP "default".

See Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack.

Make sure the scope is "Global". This contract allows all traffic from consumer to provider, and only allow TCP established from provider to consumer.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > common.

Step 2

In the Navigation pane, expand Tenant Name > Security Policies > Contracts.

Step 3

Click ACTION, from the drop-down list, choose Create Contract.

Step 4

In the Create Contract dialog box, perform the following actions:

1. In the Name field, enter the name (L3_DefaultOut).

2. In the Scope field, from the drop-down list, choose Global.

3. In the Subjects field, click the + icon.

4. In the Create Contract Subject dialog box, perform the following actions:

5. In the Name field, enter the name of your choice.

6. Uncheck Apply Both direction.

7. In the Filter Chain For Consumer to Provider field, click the + icon, from the drop-down list, choose default/common, and click Update.

8. In the Filter Chain For Provider to Consumer field, click the + icon, from the drop-down list, choose est/common, and click Update.

9. Click OK to close the Create Contract Subject dialog box.

10. Click OK to close the Create Contractdialog box.

You have now creating a contract to be provided by the l3extinstP "default".

Creating a Contract to be Provided by the l3extinstP "vpcDefault"

This section describes how to creating a contract to be provided by the l3extinstP "vpcDefault".

See Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack.

Make sure the scope is "Global". This contract allows all traffic from consumer to provider, and only allow TCP established from provider to consumer.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > common.

Step 2

In the Navigation pane, expand Tenant Name > Security Policies > Contracts.

Step 3

Click ACTION, from the drop-down list, choose Create Contract.

Step 4

In the Create Contract dialog box, perform the following actions:

1. In the Name field, enter the name (L3_VpcDefaultOut).

2. In the Scope field, from the drop-down list, choose Global.

3. In the Subjects field, click the + icon.

4. In the Create Contract Subject dialog box, perform the following actions:

5. In the Name field, enter the name of your choice.

6. Uncheck Apply Both direction.

7. In the Filter Chain For Consumer to Provider field, click the + icon, from the drop-down list, choose default/common, and click Update.

8. In the Filter Chain For Provider to Consumer field, click the + icon, from the drop-down list, choose est/common, and click Update.

9. Click OK to close the Create Contract Subject dialog box.

10. Click OK to close the Create Contractdialog box.

You have now creating a contract to be provided by the l3extinstP "vpcDefault".

Tenant Tasks

This section describes the tenant tasks.

Note

If the shared service consumer is in a different VRF than the provider, route leaking between the VRFs will automatically occur in order to enable the communication.

- Shared or Virtual Private Cloud Plan Experience

Shared or Virtual Private Cloud Plan Experience

This is an experience of a tenant in a shared or virtual private cloud (VPC) plan.

- Creating Networks in a Shared Plan

- Creating a Bridge Domain in a VPC Plan

- Creating a Firewall Within the Same Subscription

- Creating the Network in VPC Plan

- Creating VMs and Attaching to Networks

- Providing a Shared Service

- Setting up the Shared Service to be Consumed

- Configuring the Load Balancer

- Adding Access Control Lists

- Deleting Access Control Lists

- Preparing a Tenant L3 External Out on APIC for Use at Windows Azure Pack

- Creating a Network for External Connectivity

- Creating a Firewall for External Connectivity

- Verifying Tenant L3 External Connectivity on APIC

- Adding NAT Firewall Layer 4 to Layer 7 Services to a VM Network

- Adding NAT Firewall Port-Forwarding Rules for a VM Network

- Adding NAT Firewall With a Private ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network

- Requesting Additional NAT Firewall Public IP Addresses for a VRF

- Adding a Public ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network

- Adding ADC Load Balancer Configuration for a VM Network

Creating Networks in a Shared Plan

This allows the administrator to create networks in a shared plan.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the ACI pane, choose NETWORKS.

Step 4

Click NEW.

Step 5

In the NEW pane, choose NETWORKS and perform the following actions:

1. In the NETWORK NAME field, enter the name of the network (S01).

2. Click CREATE.

3. Click REFRESH.

- Verifying the Network you Created on Microsoft Windows Azure Pack on APIC

Verifying the Network you Created on Microsoft Windows Azure Pack on APIC

This section describes how to verify the network you created on Microsoft Windows Azure Pack on APIC.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS.

Step 2

In the Navigation pane, expand Tenant 018b2f7d-9e80-43f0-abff-7559c026bad5 > Application Profiles > default > Application EPGs > EPG Network01 to verify that the network you created on Microsoft Windows Azure Pack was created on APIC.

Creating a Bridge Domain in a VPC Plan

This applies only in a virtual private cloud (VPC) plan. This allows a tenant to bring its own IP address space for the networks.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose BRIDGE DOMAIN.

Step 5

In the BRIDGE DOMAIN field, enter the bridge domain name (BD01).

Step 6

If the current tenant is subscribed to multiple Azure Pack Plans, select the Subscription to create the Bridge Domain against.

Step 7

Optional: In the SUBNET'S GATEWAY field, enter the subnet's gateway (192.168.1.1/24).

Step 8

In the CONTEXT field, select a Context that is already part of the subscription or choose Create One to create a new Context for the Bridge Domain.

Step 9

Click CREATE.

- Creating a Network and Associating to a Bridge Domain in a VPC Plan

Creating a Network and Associating to a Bridge Domain in a VPC Plan

This allows the tenant to create a network and associate to a bridge domain in a VPC plan.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose NETWORK.

Step 5

In the NETWORK NAME field, enter the network name (S01).

Step 6

In the BRIDGE NAME field, enter the bridge name (BD01).

Step 7

Click CREATE.

Step 8

In the aci pane, choose NETWORKS.

You will see the network is now associated to the bridge domain.

Creating a Firewall Within the Same Subscription

This allows the tenant to create a firewall within the same subscription.

Before you begin

Ensure two networks have been created.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose FIREWALL.

Step 5

In the FROM NETWORK field, in the drop-down list, choose the network name (WEB01).

Step 6

In the TO NETWORK field, in the drop-down list, choose another network name (WEB02).

Step 7

In the PROTOCOL field, enter the protocol (tcp).

Step 8

In the PORT RANGE BEGIN field, enter the beginning port range (50).

Step 9

In the PORT RANGE END field, enter the end of the port range (150).

Step 10

Click CREATE.

You have added a firewall within the same subscription.

Creating the Network in VPC Plan

This allows the tenant to create networks in a VPC plan.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the Navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose ACI > NETWORK and perform the following actions:

1. In the NETWORK NAME field, enter the network name (Network01).

2. Option 1: Creating a network in a shared Bridge Domain.

- In the BRIDGE DOMAIN field, from the drop-down, choose the bridge domain. (default).

- Click CREATE.

This could take a few minutes for this process to complete.

3. Option 2: Creating a network in a Tenant Bridge Domain.

- In the BRIDGE DOMAIN field, from the drop-down, choose the bridge domain (myBridgeDomain).

4. Optional: To deploy the Network with a Static IP Address Pool, perform the following actions:

- Enter a Gateway in Address/Mask format (192.168.1.1/24). The resultant Static IP Address Pool will use the full range of the Gateway Subnet.

- Enter DNS Servers. If more than one is required, separate out the list with semicolons (192.168.1.2;192.168.1.3)

Note

The Subnet will be validated against all other subnets in the Context. The Network create will return an error if an overlap is detected.

- Click CREATE.

This could take a few minutes for this process to complete.

Creating VMs and Attaching to Networks

This allows the tenant to create VMs and attach to networks.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose STANDALONE VIRTUAL MACHINE > FROM GALLERY.

Step 5

In the Virtual Machine Configuration dialog box, choose your configuration (LinuxCentOS).

Step 6

Click the arrow for next.

Step 7

In the Portal Virtual Machine Settings dialog box, perform the following actions:

1. In the NAME field, enter the VM name (SVM01).

2. In the ADMINISTRATOR ACCOUNT field, root displays.

3. In the NEW PASSWORD field, enter a new password.

4. In the CONFIRM field, re-enter the password to confirm.

5. Click the arrow for next.

Step 8

In the Provide Virtual Machine Hardware Information dialog box, perform the following actions:

1. In the NETWORK ADAPTER 1 field, from the drop-down list, choose the network adapter to associate and compute (6C6DB302-aObb-4d49-a22c-151f2fbad0e9|default|S01).

2. Click the checkmark.

Step 9

In the navigation pane, choose Virtual Machines to check the status of the VM (SVM01).

Providing a Shared Service

This allows the tenant to provide a shared service.

Before you begin

Ensure the administrator has allowed tenants to provide shared services.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the ACI pane, choose SHARED SERVICE.

Step 4

In the SHARED SERVICES dialog box, perform the following actions:

1. In the ACTION field, from the drop-down list, choose PROVIDE A SHARED SERVICE CONTRACT.

2. In the NETWORK field, from the drop-down list, choose the network (WEB01).

3. In the SERVICE NAME field, enter the service name (DBSrv).

4. In the DESCRIPTION field, enter the description.

5. In the PROTOCOL field, enter the protocol (tcp).

6. In the PORT RANGE BEGIN field, enter the beginning port range (139).

7. In the PORT RANGE END field, enter the end port range (139).

8. Click the checkmark.

Setting up the Shared Service to be Consumed

This allows the tenant to setup the shared service to be consumed.

Before you begin

- Ensure the administrator has allowed tenants to provide shared services.

- Ensure the tenant has provided a shared service.

- Ensure the administrator has enabled the shared service on a plan.

- If the shared service consumer is in a different VRF than the provider, route leaking between the VRFs will automatically occur in order to enable the communication.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI > SHARED SERVICE.

Step 3

In the SHARED SERVICE dialog box, perform the following actions:

1. In the Network field, choose the network (V1).

2. In the Consumed Services field, check the service check box (DBSrv).

3. Check the checkmark.

Step 4

In the aci pane, choose SHARED SERVICES to check the consumer of the plan.

Configuring the Load Balancer

This allows the tenant to configure the load balancer.

Before you begin

- Ensure the administrator imported the device package.

- Ensure the administrator configured and posted the XML POST to Application Policy Infrastructure Controller (APIC).

- Ensure the administrator added the load balancer to a plan.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose LOAD BALANCER.

Step 5

In the NETWORK NAME field, enter the network name (WEB01).

Step 6

In the PORT field, enter the port (80).

Step 7

In the PROTOCOL field, enter the protocol (tcp).

Step 8

Click CREATE.

Step 9

In the ACI pane, choose LOAD BALANCER to check the network, virtual server, application server, port, and protocol of the load balancer.

The bridge domain should have the following subnets:

- SNIP subnet

- Host subnet

- VIP subnet

If you want the VIP subnet, it should be linked with L3 or L2 extOut.

Adding Access Control Lists

This allows the tenant to add access control lists (ACLs) to the shared service.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose SHARED SERVICES.

Step 4

In the aci pane, choose a shared service to which you want to add more ACLs (DBSrv).

Step 5

Click +ACL to add ACLs.

Step 6

In the Add ACL for DBSrv dialog box, perform the following actions:

1. In the PROTOCOL field, enter the protocol (tcp).

2. In the PORT NUMBER BEGIN field, enter the beginning port number (301).

3. In the PORT NUMBER END field, enter the end port number (400).

4. Click the checkmark.

Deleting Access Control Lists

This allows the tenant to delete access control lists (ACLs) from the shared service.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, perform the following actions:

1. Choose SHARED SERVICES.

2. Choose a shared service from which you want to delete ACLs (DBSrv).

3. Click Trash ACL to delete ACLs.

Step 4

In the Delete ACL from DBSrv dialog box, check the ACLs check box that you want to delete and click the checkmark.

Preparing a Tenant L3 External Out on APIC for Use at Windows Azure Pack

This section describes how to prepare a tenant L3 External Out on APIC for use at Windows Azure Pack.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS > Tenant Name.

Step 2

In the Navigation pane, expand Tenant Name > Networking > External Routed Networks, right-click External Routed Networks, and choose Create Routed Outside.

Step 3

In the Create Route Outside dialog box, perform the following actions:

1. Enter a Name (myRouteOut).

2. Select a VRF (3b4efb29-f66e-4c93-aed4-dc88ed4be8f2/CTX_01).

3. Configure the current dialog box according to your network config requirements. The following website provides more information about ACI Fabric Layer 3 Outside Connectivity: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/basic-config/b_ACI_Config_Guide/b_ACI_Config_Guide_chapter_0110.html

4. Click Next.

5. Click Finish.

Step 4

In the Navigation pane, expand Tenant Name > Networking > External Routed Networks > Route Outside Name, right-click Logical Node Profiles, and choose Create Node Profile.

Step 5

Follow the L3ExtOut Guide to complete your Node Profile Creation. The following website provides more information about ACI Fabric Layer 3 Outside Connectivity: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/basic-config/b_ACI_Config_Guide/b_ACI_Config_Guide_chapter_0110.html

Step 6

In the Navigation pane, expand Tenant Name > Networking > External Routed Networks > Route Outside Name, right-click Networks, and choose Create External Network.

Step 7

In the Create External Network dialog box, perform the following actions:

1. Enter the Name in the following format: <RouteOutsideName>InstP. For example: Route Outside Name is myRoutOut, my External Network Name is myRoutOutInstP.

2. In the Subnet section, click the + icon .

3. Enter your External Subnet details in the Create Subnet dialog box per your network design.

4. In the Create Subnet dialog box, click OK to complete.

5. In the Create External Network dialog box, click Submit.

Step 8

In the Navigation pane, expand Tenant Name > Networking > Bridge Domains > Bridge Domain Name, select the L3 Configurations tab and perform the following actions:

1. Click the + icon to the right of Associated L3 Outs.

2. In the drop-down list, select the L3 Out (3b4efb29-f66e-4c93-aed4-dc88ed4be8f2/myRouteOut).

3. Click UPDATE.

4. Click Submit on the Bridge Domain - <Name> Page.

Step 9

Optional: For Tenant Networks which do not use the ACI Integrated Windows Azure Pack Integrated Static IP Address Pool feature.

In the Navigation pane, expand Tenant Name > Networking > Bridge Domains > Bridge Domain Name, select the L3 Configurations tab and perform the following actions:

1. Click the + icon to the right of Subnets.

2. In the Create Subnet dialog box, perform the following actions:

- Enter a Gateway IP in Address/Mask format.

- Check the Advertised Externally check box .

- Click Submit.

Creating a Network for External Connectivity

This allows the tenant to create a network for external connectivity.

External Connectivity can be established either through the ACI Common L3ExtOut or through a user defined L3ExtOut.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose NETWORK.

Step 5

In the NETWORK NAME field, enter the network name (wapL3test).

Step 6

Option 1: Uses the Bridge Domain's Subnet for Route Advertisement.

Click CREATE.

Step 7

Option 2: Uses the EPG's Subnet for Route Advertisement.

Enter a Gateway in Address/Mask format (192.168.1.1/24).

1. Click CREATE.

Creating a Firewall for External Connectivity

This allows the tenant to create a firewall for external connectivity.

External Connectivity can be established either through the ACI Common L3ExtOut or through a user defined L3ExtOut.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

Click NEW.

Step 4

In the NEW pane, choose FIREWALL.

Step 5

Option 1: For Shared Windows Azure Pack Plans or VPC Windows Azure Pack Plans using the ACI Common L3ExtOut *External:default.

1. In the FROM NETWORK field, in the drop-down list, choose the network name (*External:default).

Option 2: For VPC Windows Azure Pack Plans using a user defined External Network.

1. In the FROM NETWORK field, in the drop-down list, choose the network name (External:myRouteOut).

Step 6

In the TO NETWORK field, in the drop-down list, choose another network name (wapL3test).

Step 7

In the PROTOCOL field, enter the protocol (tcp).

Step 8

In the PORT RANGE BEGIN field, enter the beginning port range (12345).

Step 9

In the PORT RANGE END field, enter the end of the port range (45678).

Step 10

Click CREATE.

You have added a firewall for external connectivity.

Verifying Tenant L3 External Connectivity on APIC

This section describes how to verify the Tenant L3 External Connectivity on APIC.

Procedure

Step 1

Log in to the APIC GUI, on the menu bar, choose TENANTS.

Step 2

In the Navigation pane, expand Tenant b81b7a5b-7ab8-4d75-a217-fee3bb23f427 > Application Profiles > Application EPG, ensure the network you created in Creating a Network for External Connectivity exists (wapL3test).

Step 3

In the Navigation pane, expand EPG wapL3test > Contracts, ensure the contract name exists in the format of L3+EPG name+protocols+port range (L3wapL3testtcp1234545678), the contract is Provided by the EPG, and the STATE is formed.

Step 4

Option1: For Shared L3 Out deployments, where the contract was created with *External:default, on the menu bar, choose TENANTS > common.

Option 2: For Tenant owned L3 Out deployments, on the menu bar, choose TENANTS > <your tenant-id>.

Step 5

In the Navigation pane, expand Security Policies > Imported Contracts, ensure the contract that you verified in step 3 is imported as an contract interface.

Step 6

Option 1: For Shared L3 Out deployments, where the contract was created with *External:default, on the menu bar, choose TENANTS > common.

Option 2: For Tenant owned L3 Out deployments, choose TENANTS > <your tenant-id>.

Step 7

In the External Network Instance Profile -defaultInstP pane, in the Consumed Contracts field, search for the contract interface that you verified in step 5 and ensure it exists and the STATE is formed.

Step 8

On the menu bar, choose TENANTS.

Step 9

In the Navigation pane, expand Tenant b81b7a5b-7ab8-4d75-a217-fee3bb23f427 > Application Profiles > Application EPG > EPG wapL3test > Contracts.

Step 10

In the Contracts pane, in the Consumed Contracts field, ensure the default contract that you defined in Prerequisites for Configuring L3 External Connectivity for Windows Azure Pack for either shared service tenant or for VPC tenant is consumed by this EPG and the STATE is formed.

Step 11

Option 2: For VPC Windows Azure Pack Plans using a user defined External Network with a Tenant Network with a Gateway specified.

In the Navigation pane, select Tenant Name > Application Profiles > Application EPG > EPG wapL3test > Subnets > Subnet Address, verify that the Scope is marked as Advertised Externally.

Adding NAT Firewall Layer 4 to Layer 7 Services to a VM Network

This provisions an Adaptive Security Appliance (ASA) firewall or firewall context, dynamically allocate a network address translation (NAT) IP from the external IP address pool, configure dynamic PAT on the ASA to allow outbound traffic, and provision the rest of the service graph for an easy deployment.

Before you begin

- Ensure the Azure Pack plan is configured to access an Layer 4 to Layer 7 service pool.

- Ensure the ACI VM network has been created with a gateway or subnet.

- If the private subnet of the Layer 4 to Layer 7 resource pool was not provided by the APIC admin, attempting to add Layer 4 to Layer 7 services with an overlapping subnet results in an error and no configuration will be pushed. In this case, delete and recreate the VM network with an alternate subnet.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, click on the arrow to enter further network configuration.

Step 4

Click the Enable direct internet access using NAT check box.

Step 5

Click SAVE.

Adding NAT Firewall Port-Forwarding Rules for a VM Network

This configures the network address translation (NAT) firewall to forward traffic from the NAT IP to the internal IP within the VM network.

Before you begin

- Ensure the Cisco Application Centric Infrastructure (ACI) VM network has been configured to enable NAT.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, click on the arrow to enter further network configuration.

Step 4

In the NETWORKS pane, choose RULES.

Step 5

Click ADD at the bottom panel.

Step 6

Input the required information for the Port-Forwarding Rule.

Note

The destination IP address should be an IP address within the bounds of the VM network subnet.

Step 7

Click the SAVE checkmark.

Adding NAT Firewall With a Private ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network

In addition to deploying a NAT firewall, this configuration will also deploy an internal load balancer. In this scenario, the load balancer VIPs are dynamically allocated from the Layer 4 to Layer 7 private IP address subnet (per tenant VRF). In this 2-Node service graph deployment, it is assumed that the tenant creates a Port-Fowarding Rule to forward traffic to the internal load balancer for traffic load balancing.

Before you begin

- Ensure the Azure Pack Plan is configured to access an Layer 4 to Layer 7 service pool.

- Ensure the ACI VM network has been created with a gateway or subnet.

- If the private subnet of the Layer 4 to Layer 7 resource pool was not provided by the APIC admin, attempting to add Layer 4 to Layer 7 services with an overlapping subnet results in an error and no configuration will be pushed. In this case, delete and recreate the VM network with an alternate subnet.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, click on the arrow to enter further network configuration.

Step 4

Click the Enable direct internet access using NAT check box.

Step 5

Click the Enable internal load balancer (internal) check box.

Step 6

Click SAVE.

Requesting Additional NAT Firewall Public IP Addresses for a VRF

Use this procedure to allocate additional public IP addresses for use with NAT rules. You can request this public IP address from any EPG where NAT is enabled. It is therefore available for all EPGs in the VRF.

NAT rules are saved for each EPG. So we recommend that the destination IP of the NAT rule points only to an endpoint within the EPG and not somewhere else in the VRF.

Before you begin

Ensure the Cisco ACI VM network has been configured for the NAT firewall.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, and then click the arrow to enter further network configuration.

Step 4

In the NETWORKS pane, choose IP ADDRESS.

Step 5

At the bottom panel, click REQUEST IP ADDRESS.

Step 6

Click OK.

If there is an available public IP address in the L4-L7 resource pool, an IP address is allocated and be present in this table. This IP address also is present in the RULES tab, for configuring inbound NAT rules.

Adding a Public ADC Load Balancer Layer 4 to Layer 7 Services to a VM Network

This provisions a load balancer, dynamically allocate a VIP from the external IP address pool, add the necessary routes and provision the rest of the service graph for an easy deployment.

Before you begin

- Ensure the Azure Pack Plan is configured to access an Layer 4 to Layer 7 service pool.

- Ensure the ACI VM network has been created with a gateway or subnet.

- If the private subnet of the Layer 4 to Layer 7 resource pool was not provided by the APIC admin, attempting to add Layer 4 to Layer 7 services with an overlapping subnet results in an error and no configuration will be pushed. In this case, delete and recreate the VM network with an alternate subnet.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, click on the arrow to enter further network configuration.

Step 4

Click the Enable load balancer (public) check box.

Step 5

(Optional) Click the Allow Outbound Connections check box.

Note

This option is only available if NAT has NOT been configured for this VM network.

Step 6

Click SAVE.

Adding ADC Load Balancer Configuration for a VM Network

This configures either the public, private ADC load balancer, listening on the VIP allocated to the VM network and forwarding load balancing traffic to the real servers based on the one with the least number of connections. The entire VM network will be load balanced. As VMs or VNICs come online, they will be added to the load balancer automatically. Since the entire VM Network is load balanced, it is assumed that all endpoints in the VM network are the same and can service the load balancer configuration defined.

Before you begin

- Ensure the ACI VM network has been configured for either public or private load balancing.

Procedure

Step 1

Log in to the Service Management Portal (Tenant Portal).

Step 2

In the navigation pane, choose ACI.

Step 3

In the aci pane, choose NETWORKS, click on the arrow to enter further network configuration.

Step 4

In the NETWORKS pane, choose LOAD BALANCERS.

Step 5

Click ADD at the bottom panel.

Step 6

Input the required information for the load balancer (Name: HTTP, Protocol: TCP, Port: 80).

Step 7

Click the SAVE checkmark.

Troubleshooting as an Admin

Procedure

Windows Azure Pack Administrator can look at all networks deployed by tenants in the admin portal. In case there is an issue, use the APIC GUI to look for any faults on the following objects:

1. VMM domain

2. Tenant and EPG corresponding to the Windows Azure Pack tenant networks.

Troubleshooting as a Tenant

If there is an error message, provide the error message along with the description of the workflow and action to your Administrator.

Troubleshooting the EPG Configuration Issue

If during the lifetime of the endpoint group (EPG), the VLAN ID of the EPG changes on the APIC then SCVMM needs to update the VLAN configuration on all virtual machines for the new setting to take effect.

Procedure

To perform this operation, run the following PowerShell commands on the SCVMM server:

Example:

$VMs = Get-SCVirtualMachine
$VMs | Read-SCVirtualMachine
$NonCompliantAdapters=Get-SCVirtualNetworkAdapter -All | Where-Object
{$_.VirtualNetworkAdapterComplianceStatus -eq "NonCompliant"}
$NonCompliantAdapters | Repair-SCVirtualNetworkAdapter

ACI Windows Azure Pack PowerShell Cmdlets

This section describes how to list the Cisco Application Centric Infrastructure (ACI) Windows Azure Pack PowerShell cmdlets, help, and examples.

Procedure

Step 1

Log in to the Windows Azure Pack server, choose Start > Run > Windows PowerShell.

Step 2

Enter the followings commands:

Example:

Windows PowerShell
Copyright (C) 2013 Microsoft Corporation. All rights reserved.

PS C:\Users\administrator> cd C:\inetpub\Cisco-ACI\bin
PS C:\inetpub\Cisco-ACI\bin> Import-Module .\ACIWapPsCmdlets.dll
PS C:\inetpub\Cisco-ACI\bin> Add-Type -Path .\Newtonsoft.Json.dll
PS C:\inetpub\Cisco-ACI\bin> Get-Command -Module ACIWapPsCmdlets

CommandType     Name                                      ModuleName
-----------     ----                                      ----------
Cmdlet          Add-ACIWAPEndpointGroup                   ACIWapPsCmdlets
Cmdlet          Get-ACIWAPAdminObjects                    ACIWapPsCmdlets
Cmdlet          Get-ACIWAPAllEndpointGroups               ACIWapPsCmdlets
Cmdlet          Get-ACIWAPBDSubnets                       ACIWapPsCmdlets
Cmdlet          Get-ACIWAPConsumersForSharedService       ACIWapPsCmdlets
Cmdlet          Get-ACIWAPEndpointGroups                  ACIWapPsCmdlets
Cmdlet          Get-ACIWAPEndpoints                       ACIWapPsCmdlets
Cmdlet          Get-ACIWAPLBConfiguration                 ACIWapPsCmdlets
Cmdlet          Get-ACIWAPOpflexInfo                      ACIWapPsCmdlets
Cmdlet          Get-ACIWAPPlans                           ACIWapPsCmdlets
Cmdlet          Get-ACIWAPStatelessFirewall               ACIWapPsCmdlets
Cmdlet          Get-ACIWAPSubscriptions                   ACIWapPsCmdlets
Cmdlet          Get-ACIWAPTenantCtx                       ACIWapPsCmdlets
Cmdlet          Get-ACIWAPTenantPlan                      ACIWapPsCmdlets
Cmdlet          Get-ACIWAPTenantSharedService             ACIWapPsCmdlets
Cmdlet          Get-ACIWAPVlanNamespace                   ACIWapPsCmdlets
Cmdlet          New-ApicOpflexCert                        ACIWapPsCmdlets
Cmdlet          Read-ApicOpflexCert                       ACIWapPsCmdlets
Cmdlet          Remove-ACIWAPEndpointGroup                ACIWapPsCmdlets
Cmdlet          Remove-ACIWAPPlan                         ACIWapPsCmdlets
Cmdlet          Remove-ACIWAPTenantCtx                    ACIWapPsCmdlets
Cmdlet          Set-ACIWAPAdminLogin                      ACIWapPsCmdlets
Cmdlet          Set-ACIWAPBDSubnets                       ACIWapPsCmdlets
Cmdlet          Set-ACIWAPLBConfiguration                 ACIWapPsCmdlets
Cmdlet          Set-ACIWAPLogin                           ACIWapPsCmdlets
Cmdlet          Set-ACIWAPOpflexOperation                 ACIWapPsCmdlets
Cmdlet          Set-ACIWAPPlan                            ACIWapPsCmdlets
Cmdlet          Set-ACIWAPStatelessFirewall               ACIWapPsCmdlets
Cmdlet          Set-ACIWAPTenantSharedService             ACIWapPsCmdlets
Cmdlet          Set-ACIWAPUpdateShareServiceConsumption   ACIWapPsCmdlets
Cmdlet          Set-ACIWAPVlanNamespace                   ACIWapPsCmdlets

Step 3

Generating help:

Example:

commandname -?

Step 4

Generating examples:

Example:

get-help commandname -examples

Uninstalling the Cisco ACI with Microsoft Windows Azure Pack Components

This section describes how to uninstall the Cisco Application Centric Infrastructure (ACI) with Microsoft Windows Azure Pack components.

Note

Uninstall involves removing artifacts such as VM and logical networks. Uninstalling succeeds only when no other resource, such as a VM or a host, is consuming them.

Component

Task

Detach all virtual machines from the VM networks

See Microsoft's documentation.

Delete VXLAN tunnel endpoint (VTEP) logical switch on all hyper-Vs

See Microsoft's documentation.

Delete cloud on System Center Virtual Machine Manager (SCVMM)

See Microsoft's documentation.

To uninstall the ACI with Microsoft Windows Azure Pack 1.1(1j) release, uninstall the APIC Windows Azure Pack Resource Provider

See Uninstalling the APIC Windows Azure Pack Resource Provider.

To uninstall this release of ACI with Microsoft Windows Azure Pack, uninstall the following:

- ACI Azure Pack Resource Provider

- ACI Azure Pack Admin Site Extension

- ACI Azure Pack Tenant Site Extension

See Uninstalling the ACI Azure Pack Resource Provider.

See Uninstalling the ACI Azure Pack Admin Site Extension.

See Uninstalling the ACI Azure Pack Tenant Site Extension.

Uninstall the APIC Hyper-V Agent

See Uninstalling the APIC Hyper-V Agent.

- Uninstalling the APIC Windows Azure Pack Resource Provider

- Uninstalling the ACI Azure Pack Resource Provider

- Uninstalling the ACI Azure Pack Admin Site Extension

- Uninstalling the ACI Azure Pack Tenant Site Extension

- Uninstalling the APIC Hyper-V Agent

Uninstalling the APIC Windows Azure Pack Resource Provider

This section describes how to uninstall the APIC Windows Azure Pack Resource Provider.

Procedure

Step 1

Log in to the Windows Azure Pack server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click APIC Windows Azure Pack Resource Provider and choose Uninstall.

This uninstalls the APIC Windows Azure Pack Resource Provider from the Windows Azure Pack server.

Step 4

To verify if the APIC Windows Azure Pack Resource Provider is uninstalled, perform the following actions:

1. Choose Start > Control Panel > Uninstall a Program.

2. In the Programs and Features window, verify that APIC Windows Azure Pack Resource Provider is not present.

Uninstalling the ACI Azure Pack Resource Provider

This section describes how to uninstall the ACI Azure Pack Resource Provider.

Procedure

Step 1

Log in to the Windows Azure Pack server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click ACI Azure Pack Resource Provider and choose Uninstall.

This uninstalls the ACI Azure Pack Resource Provider from the Windows Azure Pack server.

Step 4

To verify if the ACI Azure Pack Resource Provider is uninstalled, perform the following actions:

1. Choose Start > Control Panel > Uninstall a Program.

2. In the Programs and Features window, verify that ACI Azure Pack Resource Provider is not present.

Uninstalling the ACI Azure Pack Admin Site Extension

This section describes how to uninstall the ACI Azure Pack Admin Site Extension.

Procedure

Step 1

Log in to the Windows Azure Pack server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click ACI Azure Pack Admin Site Extension and choose Uninstall.

This uninstalls the ACI Azure Pack Admin Site Extension from the Windows Azure Pack server.

Step 4

To verify if the ACI Azure Pack Admin Site Extension is uninstalled, perform the following actions:

1. Choose Start > Control Panel > Uninstall a Program.

2. In the Programs and Features window, verify that ACI Azure Pack Admin Site Extension is not present.

Uninstalling the ACI Azure Pack Tenant Site Extension

This section describes how to uninstall the ACI Azure Pack Tenant Site Extension.

Procedure

Step 1

Log in to the Windows Azure Pack server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click ACI Azure Pack Tenant Site Extension and choose Uninstall.

This uninstalls the ACI Azure Pack Tenant Site Extension from the Windows Azure Pack server.

Step 4

To verify if the ACI Azure Pack Tenant Site Extension is uninstalled, perform the following actions:

1. Choose Start > Control Panel > Uninstall a Program.

2. In the Programs and Features window, verify that ACI Azure Pack Tenant Site Extension is not present.

Uninstalling the APIC Hyper-V Agent

This section describes how to uninstall the APIC Hyper-V Agent.

Procedure

Step 1

Log in to the Hyper-V server.

Step 2

Choose Start > Control Panel > Uninstall a Program.

Step 3

In the Programs and Features window, right-click Cisco APIC HyperV Agent and choose Uninstall.

This uninstalls the APIC Hyper-V Agent from the Hyper-V server.

Step 4

To verify if the APIC Hyper-V Agent is uninstalled, perform the following actions:

1. Choose Start > Control Panel > Uninstall a Program.

2. In the Programs and Features window, verify that Cisco APIC HyperV Agent is not present.

Step 5

Repeat steps 1-4 for each Hyper-V server.

Downgrading Cisco APIC and the Switch Software with Cisco ACI and Microsoft Windows Azure Pack Components

This section describes how to downgrade the Cisco APIC and the switch software with Cisco ACI with Microsoft Windows Azure Pack components.

Note

Layer 4 to Layer 7 resource pool configurations created and used in Cisco APIC 3.1(1) and later are not compatible with older Cisco APIC/Windows Azure Pack builds. Steps 1 to 3 apply when downgrading from Cisco APIC 3.1(1) or later to earlier versions.

Procedure

Step 1

Review the list of Layer 4 to Layer 7 resource pools on the Cisco APIC.

Note the list of resource pools that were created in Cisco APIC 3.1(1) or later. These resource pools have the Function Profiles tab in the GUI and have version normalized in the NX-OS Style CLI configuration.

Step 2

Windows Azure Pack Tenants Portal: Perform the following steps for each Cisco ACI VM network that has a Virtual Private Cloud using Layer 4 to Layer 7 Cloud orchestrator mode resource pools (resource pools created in Cisco APIC 3.1(1) or later):

1. Log in to the Service Management Portal (Tenant Portal).

2. In the navigation pane, choose ACI.

3. In the aci pane, choose NETWORKS, click the arrow to enter further network configuration.

4. Uncheck the box Enable direct internet access using NAT if it is checked.

5. Uncheck the box Enable internal load balancer (internal) if it is checked.

6. Uncheck the box Enable load balancer (public) if it is checked.

7. Click SAVE.

Step 3

Windows Azure Pack Admin: Perform the following steps for each Windows Azure Pack plan where ACI Networking has been added as a Plan Service and the Plan is using Layer 4 to Layer 7 cloud orchestrator mode resource pools.

1. Log in to the Service Management Portal (Admin Portal).

2. In the navigation pane, choose PLANS.

3. In the plans pane, choose PLANS, and then click the plan (Gold).

4. In the Gold pane, choose Networking (ACI).

5. In the networking (aci) pane, perform one of the following steps:

- Choose the Layer 4 to Layer 7 resource pools provisioned by the Cisco APIC admin in Cisco APIC 3.0(x) or earlier for Azure Pack consumption.

- Choose Choose one… to disable Virtual Private Cloud NAT Firewall and ADC Load Balancer services for Azure Pack Tenants.

6. Click SAVE.

Step 4

Uninstall Cisco ACI with Microsoft Windows Azure Pack components.

See Uninstalling the Cisco ACI with Microsoft Windows Azure Pack Components.

Step 5

Downgrade the APIC controller and the switch software.

See the Cisco APIC Firmware Management, Installation, Upgrade, and Downgrade Guide.

Step 6

Install the downgrade version of Cisco ACI with Microsoft Windows Azure Pack components.

See the Installing, Setting Up, and Verifying the Cisco ACI with Microsoft Windows Azure Pack Components.

Preface

This preface includes the following sections:

- Audience

- Document Conventions

- Related Documentation

- Documentation Feedback

- Obtaining Documentation and Submitting a Service Request

Audience

This guide is intended primarily for data center administrators with responsibilities and expertise in one or more of the following:

- Virtual machine installation and administration

- Server administration

- Switch and network administration

- Cloud administration

Document Conventions

Command descriptions use the following conventions:

Convention

Description

bold

Bold text indicates the commands and keywords that you enter literally as shown.

Italic

Italic text indicates arguments for which the user supplies the values.

[x]

Square brackets enclose an optional element (keyword or argument).

[x | y]

Square brackets enclosing keywords or arguments separated by a vertical bar indicate an optional choice.

{x | y}

Braces enclosing keywords or arguments separated by a vertical bar indicate a required choice.

[x {y | z}]

Nested set of square brackets or braces indicate optional or required choices within optional or required elements. Braces and a vertical bar within square brackets indicate a required choice within an optional element.

variable

Indicates a variable for which you supply values, in context where italics cannot be used.

string

A nonquoted set of characters. Do not use quotation marks around the string or the string will include the quotation marks.

Examples use the following conventions:

Convention

Description

screen font

Terminal sessions and information the switch displays are in screen font.

boldface screen font

Information you must enter is in boldface screen font.

italic screen font

Arguments for which you supply values are in italic screen font.

< >

Nonprinting characters, such as passwords, are in angle brackets.

[ ]

Default responses to system prompts are in square brackets.

!, #

An exclamation point (!) or a pound sign (#) at the beginning of a line of code indicates a comment line.

This document uses the following conventions:

Note

Means reader take note. Notes contain helpful suggestions or references to material not covered in the manual.

Caution

Means reader be careful. In this situation, you might do something that could result in equipment damage or loss of data.

Warning

IMPORTANT SAFETY INSTRUCTIONS

This warning symbol means danger. You are in a situation that could cause bodily injury. Before you work on any equipment, be aware of the hazards involved with electrical circuitry and be familiar with standard practices for preventing accidents. Use the statement number provided at the end of each warning to locate its translation in the translated safety warnings that accompanied this device.

SAVE THESE INSTRUCTIONS

Related Documentation

Application Policy Infrastructure Controller (APIC) Documentation

The following companion guides provide documentation for APIC:

- Cisco APIC Getting Started Guide

- Cisco APIC Basic Configuration Guide

- Cisco ACI Fundamentals

- Cisco APIC Layer 2 Networking Configuration Guide

- Cisco APIC Layer 3 Networking Configuration Guide

- Cisco APIC NX-OS Style Command-Line Interface Configuration Guide

- Cisco APIC REST API Configuration Guide

- Cisco APIC Layer 4 to Layer 7 Services Deployment Guide

- Cisco ACI Virtualization Guide

- Cisco Application Centric Infrastructure Best Practices Guide

All these documents are available at the following URL: https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html

Cisco Application Centric Infrastructure (ACI) Documentation

The broader ACI documentation is available at the following URL: https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html.

Cisco Application Centric Infrastructure (ACI) Simulator Documentation

The Cisco ACI Simulator documentation is available at https://www.cisco.com/c/en/us/support/cloud-systems-management/application-centric-infrastructure-simulator/tsd-products-support-series-home.html.

Cisco Nexus 9000 Series Switches Documentation

The Cisco Nexus 9000 Series Switches documentation is available at https://www.cisco.com/c/en/us/support/switches/nexus-9000-series-switches/tsd-products-support-series-home.html.

Cisco Application Virtual Switch Documentation

The Cisco Application Virtual Switch (AVS) documentation is available at https://www.cisco.com/c/en/us/support/switches/application-virtual-switch/tsd-products-support-series-home.html.

Cisco Application Centric Infrastructure (ACI) Integration with OpenStack Documentation

Cisco ACI integration with OpenStack documentation is available at https://www.cisco.com/c/en/us/support/cloud-systems-management/application-policy-infrastructure-controller-apic/tsd-products-support-series-home.html.

Documentation Feedback

To provide technical feedback on this document, or to report an error or omission, please send your comments to apic-docfeedback@cisco.com. We appreciate your feedback.

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What's New in Cisco Product Documentation at: http://www.cisco.com/c/en/us/td/docs/general/whatsnew/whatsnew.html

Subscribe to What’s New in Cisco Product Documentation, which lists all new and revised Cisco technical documentation as an RSS feed and delivers content directly to your desktop using a reader application. The RSS feeds are a free service.

Full Cisco Trademarks with Software License

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.

Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)