Cisco UCS CPA Workflows

This chapter contains the following sections:

Workflows for Big Data

Cisco UCS Director Express for Big Data defines a set of workflows in the UCS CPA folder under Orchestration page.

  • UCS CPA Multi-UCSM Hadoop Cluster WF—This workflow is triggered if you use multiple UCSM Hadoop accounts.

  • UCS CPA Multi-UCSM Splunk Cluster WF—This workflow is triggered if you use multiple UCSM Splunk accounts.

  • UCS CPA Single UCSM Server Configuration WF—This workflow is triggered for every UCSM account.

  • UCS CPA Node Bare Metal—This workflow is triggered per node in the cluster.

  • UCS CPA Node Bare Metal with Storage Profile WF—This workflow is triggered using Cisco UCS S3260 storage server having SSD Server Boot Drive under SAS RAID controller to deploy OS.

    Note 

    Disk group policies are automatically created in the Cisco UCS Director Express for Big Data using workflows only for Cisco UCS Manager 3.2 or later version.

    Table 1. Disk Group Policy
    Servers Details

    Disk Group Policy Required?

    Disk Policy Allocation Details

    Servers having PCH Controller and and Data drive slots with SSD disks

    No

    Not applicable

    SAS controller with dedicated boot drive slots (20x) and Data drive slots with SSD disks

    Yes

    For OS boot, if you want to manually allocate the SSD slot numbers to RAID 0 or RAID 1, BD_V1_SSM_RAID0 or BD_V1_SSM_RAID1 should be created. Currently dedicated boot drive slots 201 and 202 are supported only by Cisco UCS S3260 Storage Server.

    SAS controller with dedicated boot drive slots (20x) with SSD disks and Data drive slots without SSD disks

    Yes

    For OS boot, if you want to automatically allocate the SSD slot numbers to RAID 0 or RAID 1, Boot_SSD_RAID0 or Boot_SSD_RAID1 should be created.

    SAS controller and Data drive slots with HDD only

    Yes

    For OS boot, if you want to automatically allocate the HDD slot numbers to RAID 0 or RAID 1, Boot_HDD_RAID0 or Boot_HDD_RAID1 should be created.

    Note 

    You can create Boot_HDD_RAID0 or Boot_HDD_RAID1 if SSD disks are available in Data drive slots in addition to HDD.

    SAS controller and Data drive slots with SDD and HDD disks

    Yes

    During OS boot, if you want to manually allocate the slot numbers to RAID 1, BD_V1_ANYM_RAID1 should be created.

    Note

    In some cases, Wait for Complete Association might fail for Cisco UCS 326x storage servers during UCS CPA Node Bare Metal with Storage Profile after a time out. In such cases, association process might have already failed after Bind UCS Service Profile to Template post 90% of FSM with Storage System initialisation error. You need to perform a re-acknowledgement on the Cisco UCS server and resubmit the Bare Metal workflow.

  • UCS CPA Delete Node—This workflow is triggered if you delete a node from the Hadoop cluster.

  • UCS CPA Cloudera Add Live Node—This workflow is triggered if you add a Live Node for Cloudera.

  • UCS CPA MapR Add Live Node—This workflow is triggered if you add a Live Node for MapR.

  • UCS CPA Hortonworks Add Live Node—This workflow is triggered if you add a Live Node for Hortonworks.

  • UCS CPA Add New Disks—This workflow is triggered if you add new disks.

  • UCS CPA Configure NFS Clients and Servers—This workflow is triggered if you configure NFS clients and servers.

  • UCS CPA Enable HTTPS Modify Protocol and Port Number for Hadoop—This workflow is triggered if you modify protocol and port numbers for Hadoop.

  • UCS CPA Enable HTTPS Modify Protocol and Port Number for Splunk—This workflow is triggered if you modify protocol and port numbers for Splunk.

  • UCS CPA Migrate Splunk Cluster to Multi-Site—This workflow is triggered if you want to migrate an existing Splunk cluster to a multi-site Splunk cluster.

  • UCS CPA Remove Disks—This workflow is triggered if you remove disks.

  • UCS CPA Splunk Upgrade—This workflow is triggered if you upgrade the Splunk cluster.

  • UCS CPA Instant Hadoop Cluster WF—This workflow is triggered if you create an instant Hadoop cluster based on the node count and other mandatory inputs such as the IP address range, memory, and number of interfaces. Cisco UCS Director Express for Big Data automatically creates one UCS service profile and a Hadoop cluster profile template at the back end that are required to create an instant Hadoop cluster. This saves you the effort of manually creating a service profile and a cluster profile.

  • UCS CPA Customized Hadoop Cluster WF—This workflow is triggered if you choose to use a specific UCS service profile. A Hadoop cluster profile template with the specified number of nodes to be created in the Hadoop cluster.

  • UCS CPA Disable High Availability WF—This workflow is triggered when you disable high availability.

  • UCS CPA Enable High Availability WF—This workflow is triggered when you enable high availability.

  • UCS CPA Shutdown Big Data Cluster WF—This workflow is triggered when you shut down the Hadoop cluster.

  • UCS CPA Start Big Data Cluster WF—This workflow is triggered when you power up the Hadoop cluster.

  • UCS CPA Splunk Add Live Archival Node—This workflow is triggered when if you add a Live Archival Node for Splunk.

  • UCS CPA Upgrade Cluster WF—This workflow is triggered when you upgrade the Hadoop cluster.

  • UCS CPA Cloudera Add New Service WF—This workflow is triggered when you add a new service for Cloudera.

  • UCS CPA MapR Add New Service WF–This workflow is triggered when you add a new service for MapR.

  • UCS CPA Hortonworks Add New Service WF—This workflow is triggered when you add a new service for Hortonworks.

  • UCS CPA Multi-Node Bare Metal provisioning WF—This workflow is triggered when you deploy baremetal OS account using Cisco UCS server.

    Table 2. Disk Group Policy
    Servers Details

    Disk Group Policy Required?

    Disk Policy Allocation Details

    Servers having PCH Controller

    No

    Disk group policy is not applicable for PCH controllers, for example, Lewisburg SATA Controller. The LUN gets created automatically by the PCH controller based on the controller definition.

    Servers having SAS controller

    Yes

    The disks are configured for RAID 1. You can manually allocate two disks.

    Servers having SATA controller

    Yes

    The disks are configured for RAID 1. You can manually allocate two disks.

  • Bare-metal OS Deployment —This workflow is triggered per server to deploy baremetal OS. UCS CPA Multi-Node Bare Metal provisioning WF will automatically trigger the Bare-metal OS Deployment workflow when you deploy baremetal OS account per server and monitor it.

About Service Requests for Big Data

Cisco UCS Director Express for Big Data leverages Cisco UCS Director service requests and workflow orchestration for the overall deployment of Hadoop clusters. Each service request is a provisioning workflow that is created during a cluster creation.

For example, one UCS CPA Multi-UCSM Hadoop Cluster W/F, one Single UCSM Server Configuration W/F, and four UCS CPA Node Bare Metal W/Fs are created for a four-node Hadoop cluster. When the workflows are complete, the cluster is created under Solutions > Big Data > Accounts for that UCSM account.

A set of service requests is created under Organizations > Service Requests during a cluster creation.

  • UCS CPA Multi-UCSM Hadoop Cluster WF—This workflow is triggered if you use multiple UCSM Hadoop accounts. It also applies when you create an instant or customized Hadoop cluster.

  • UCS CPA Multi-UCSM Splunk Cluster WF—This workflow is triggered if you use multiple UCSM Splunk accounts. It also applies to when you create an instant or customized Splunk cluster.

  • UCS CPA Single UCSM Server Configuration WF—This workflow is triggered for every UCSM account.

  • UCS CPA Node Bare Metal—This workflow is triggered per node in the cluster.

The following service requests are created when you add a Bare Metal Node or a Live Node to the cluster.

  • UCS CPA Cloudera Add Live Node—This workflow is triggered if you add a Live Node for Cloudera.

  • UCS CPA MapR Add Live Node—This workflow is triggered if you add a Live Node for MapR.

  • UCS CPA Hortonworks Add Live Node—This workflow is triggered if you add a Live Node for Hortonworks.

The following service requests are created as explained, below:

  • UCS CPA Disable High Availability WF—This workflow is triggered when you disable high availability.

  • UCS CPA Enable High Availability WF—This workflow is triggered when you enable high availability.

  • UCS CPA Shutdown Big Data Cluster WF—This workflow is triggered when you shut down the Hadoop cluster.

  • UCS CPA Start Big Data Cluster WF—This workflow is triggered when you power up the Hadoop cluster.

  • UCS CPA Upgrade Cluster WF—This workflow is triggered when you upgrade the Hadoop cluster.

  • UCS CPA Cloudera Add New Service WF—This workflow is triggered when you add a new service for Cloudera.

  • UCS CPA Hortonworks Add New Service WF—This workflow is triggered when you add a new service for Hortonworks.

  • UCS CPA MapR Add New Service WF–This workflow is triggered when you add a new service for MapR.

The following service requests are created as explained below:

  • UCS CPA Multi-Node Bare Metal provisioning WF—This workflow is triggered if you deploy a baremetal OS account.

  • Bare-metal OS Deployment - This workflow is triggered per server while you deploy a baremetal OS account.

For more information on the service requests and workflows, see the following guides:

  • Cisco UCS Director Self-Service Portal Guide

  • Cisco UCS Director Orchestration Guide

Monitoring Service Requests for Big Data

Before you begin

Create and customize a Cluster Deploy Template to monitor a service request.

Procedure

Step 1

Choose Organizations > Service Requests.

Step 2

Click Service Requests.

Step 3

Select the service request that you want to monitor and click View Details.

One of the following Request Status is displayed:

  • Complete
  • In Progress
  • Cancelled
  • Failed

Viewing UCS CPA Workflow Tasks

From the Service Request Status screen, you can view the following:

  • Workflow Status

  • Log

  • Objects Created and Modified

  • Input/Output


Note

You can only modify inputs for failed service requests.

Procedure

Step 1

Choose Organizations > Service Requests.

You can see the list of user-specific service requests added to a specific group. For example, All User Groups.
Step 2

Choose the Service Request ID that you want to view.

Step 3

Double-click the Service Request ID that opens the Service Request Status screen. (You can also choose the Service Request ID by the workflow name associated with it, and click View Details. For example, choose the UCS CPA Node Bare Metal workflow and click View Details).

On the Service Request Status screen, you can view the following tasks for the workflow:

UCS CPA Multi-UCSM Hadoop Cluster WF/UCS CPA Multi-UCSM Splunk Cluster WF

UCS CPA Single UCSM Server Configuration WF

UCS CPA Node Bare Metal

The following tasks are associated with the UCS CPA Multi-UCSM Hadoop Cluster WF/UCS CPA Multi-UCSM Splunk Cluster WF:

  1. Initiated by Admin

  2. Multi-UCSM Hadoop Cluster profile/Multi-UCSM Splunk Cluster profile

  3. Setup Big Data Cluster Env

  4. User (Compute) Approval Required

  5. User (OS) Approval Required

  6. Multi-UCSM Configuration WF

  7. Multi Bare Metal WF Monitor

  8. Synchronized Command Execution

  9. User (Hadoop) Approval Required

  10. Custom SSH Command

  11. Monitor Shell Script Status

  12. Provision Hadoop Cluster/Provision Splunk Cluster

  13. SSH Command

  14. Monitor Shell Script Status

  15. Edit Hadoop Account/Edit Splunk Account

  16. Assign Big Data Account to Group

  17. Custom SSH Command

  18. Complete

The following tasks are associated with the Single UCSM Server Configuration WF:

  1. Initiated by Admin

  2. Create UCS Service Profile Template

  3. Change Maintenance Policy UCS SP Template

  4. Muti-Bare Metal OS Install WF

  5. Create UCS Disk Group Policy

  6. Create UCS Disk Group Policy

  7. Create UCS Disk Group Policy

  8. Create UCS Disk Group Policy

  9. Multi-Bare Metal WF Monitor

  10. Change Maintenance Policy for UCS SP Template

  11. Complete

The following tasks are associated with the UCS CPA Node Bare Metal:

  1. Initiated by Admin

  2. Modify Workflow Priority (High)

  3. Assign Bare Metal SR ID

  4. Create Service Profile from Template

  5. Unbind UCS Service Profile from Template

  6. Modify UCS Service Profile Boot Policy

  7. Add VLAN to Service Profile

  8. Associate UCS Service Profile

  9. Assign ServerIdentity

  10. PXE Boot With BMA Selection

  11. Setup RAID Commands

  12. UCS Blade Reset Action

  13. PXE Boot Wait

  14. Monitor RAID Configuration

  15. Custom Archive PXE Boot Request

  16. UCS Blade Power OFF Action

  17. PXE Boot With BMA Selection

  18. Setup RAID Commands

  19. Custom Wait for Complete Association of SP

  20. UCS Blade Reset Action

  21. PXE Boot Wait

  22. Modify UCS Service Profile Boot Policy

  23. Delete VLAN from Service Profile vNIC

  24. Bind UCS Service Profile to Template

  25. Wait for complete Association

  26. UCS Blade Reset Action

  27. Assign IP Status

  28. Custom SSH Command

  29. Monitor Shell Script Status

  30. Linux Shell Script Execution

  31. Linux Shell Script Execution

  32. UCS Blade Power OFF Action

  33. UCS Blade Power ON Action

  34. Synchronized Command Execution

  35. Assign UCS Server to Group

  36. Assign Service Profile to Group

  37. Complete
Step 4

Click Close.

Viewing UCS CPA Workflow Tasks for BareMetal OS

From the Service Request Status screen, you can view the following:

  • Workflow Status

  • Log

  • Objects Created and Modified

  • Input/Output


Note

You can only modify inputs for failed service requests.

Procedure

Step 1

Choose Organizations > Service Requests. You can see the list of user-specific service requests added to a specific group. For example, All User Groups.

Step 2

Choose the Service Request ID that you want to view.

Step 3

Double-click the Service Request ID that opens the Service Request Status screen. (You can also choose the Service Request ID by the workflow name associated with it, and click View Details. For example, choose the UCS CPA Node Bare Metal workflow and click View Details). On the Service Request Status screen, you can view the following tasks for the workflow:

UCS CPA Multi-Node Bare Metal provisioning WF

Bare-metal OS Deployment

The following tasks are associated with the UCS CPA Multi-Node Bare Metal provisioning WF:

  1. Initiated by Admin

  2. Get UCSM account authentication

  3. Get UCS Organization

  4. Create Multi UCS Policy for Linux Account

  5. Create Maintenance Policy

  6. Create BIOS Policy

  7. Create Local Boot Policy

  8. Create LAN Boot Policy

  9. Check Scrub Policy is required

  10. Create Scrub Policy (note only executed when check scrub policy is required is true)

  11. Check Selected slot is SATA

  12. Create Local Disk Config Policy (note only executed when check Selected slot is SATA is true)

  13. Create Service Profile Template

  14. Get UCSM Account Name

  15. Create PXE VLAN

  16. Modify Org Permission for PXE VLAN

  17. Get PXE VLAN ID

  18. Check if server have PCH

  19. Add UCS Storage Profile (only executed when Check if server have PCH is true)

  20. Create Storage Profile (only executed when Check if server have PCH is false)

  21. Get UCS Storage Profile (only executed when Check if server have PCH is false)

  22. Collect Inventory

  23. Request UCSM Inventory Collection By DN (for service profile template inventory)

  24. Wait for PXE VLAN

  25. Check if server have PCH

  26. Request UCSM Inventory Collection By DN (for storage profile inventory & executed only when Check if server have PCH is false)

  27. Wait for Service Profile Template Status

  28. Install Multi Linux on UCS Server for Linux Account (it will automatically trigger the Bare-metal OS Deployment workflow per server)

  29. Multi Bare Metal Account WF Monitor (it will monitor the Bare-metal OS Deployment workflow SR’s)

  30. Create Baremetal OS Account (once all the Bare-metal OS Deployment is successful, the baremetal OSs account will get created )

  31. Complete

The following tasks are associated with the Bare-metal OS Deployment:

  1. Initiated by Admin

  2. Create Service Profile from Template

  3. Unbind UCS Service Profile from Template

  4. Modify UCS Service Profile Boot Policy

  5. Get UCS Organization

  6. Add PXE VLAN to Service Profile (PXE VLAN is already created in the UCS CPA Multi-Node Bare Metal provisioning WF and passed here)

  7. Associate Storage Profile to Service Profile (Storage Profile is already created in the UCS CPA Multi-Node Bare Metal provisioning WF and passed here)

  8. Checking Set JBOD contains "true"

  9. Set JBOD to Unconfigured Good (only executed when checking Set JBOD contains "true")

  10. Select UCS Server

  11. Check if Delete LUN is enabled

  12. Delete LUNs From Server (only execute when check if Delete LUN is enabled)

  13. Reserve UCS IP address

  14. Associate UCS Service Profile

  15. Check if server have SAS

  16. Get UCS LUN ID (only execute when Check if server have SAS is true)

  17. Setup PXE Boot (OS Type: RHEL7.6) (only execute when Check if server have SAS is true)

  18. Setup PXE Boot (OS Type: RHEL7.6) (only execute when Check if server have SAS is false)

  19. Modify PXE Configuration for SATA Controller (only execute when Check if server have SAS is false)

  20. Wait for Complete Association

  21. UCS Blade Power ON Action

  22. Check if server have SAS

  23. Monitor PXE Boot

  24. Delete PXE VLAN from MGMT vNIC

  25. Bind UCS Service Profile to Template

  26. UCS Blade Power OFF Action

  27. UCS Blade Power ON Action

  28. IPconfig

  29. vNIC Bonding

  30. Complete
Step 4

Click Close.

Workflow Customization to Deploy a Hadoop or Splunk Cluster

You can customize the following UCS CPA workflows and use them to deploy a Hadoop or Splunk cluster. You can add installation packages (RPMs) required for your cluster environment in the Cisco UCS Director Express for Big Data Bare Metal Agent.

  • Rename the UCS CPA Multi-UCSM Hadoop Cluster WF

  • Rename the UCS CPA Multi-UCSM Splunk Cluster WF

  • Rename the Single UCSM Server Configuration WF

  • Rename the UCS CPA Node Bare Metal

Deploying a Hadoop or Splunk Cluster Through Workflow Customization

Before you begin

For more information on workflow orchestration, see the Cisco UCS Director Orchestration Guide.

  • Customize UCS CPA Node Bare Metal workflows that you want to use in the cloned Single UCSM Server Configuration WF.

  • Customize the Single UCSM Server Configuration WF that you want to use in the UCS CPA Multi-UCSM Hadoop Cluster WF or UCS CPA Multi-UCSM Splunk Cluster WF.

Procedure
Step 1

Choose Orchestration.

Step 2

Click the UCS CPA folder from Workflows.

Step 3

Double-click the workflow that you want to customize in the Workflow Designer. For instance, double-click the UCS CPA Multi-UCSM Hadoop Cluster WF.

Step 4

Double-click the Muti-UCSM Configuration WF task in the Workflow Designer.

Step 5

Click Next on the Workflow Task Basic Information page.

Step 6

On the User Input Mappings to Task Input Attributes page, select the attributes that you want to map to the workflow input fields or provide values in the next step. If necessary, check the Map to User Input check box to provide user inputs.

Step 7

Click Next on the User Input Mappings to Task Input Attributes page.

Step 8

Enter the task values which are not mapped to workflow inputs. For example, enter the name of the cloned Single UCSM Server Configuration WF in the Workflow Name field.

Step 9

Click Revalidate to validate task input values.

Step 10

Click Next.

Step 11

On the User Output mappings to Task Output Attributes page, select the attributes that you want to map to the workflow output fields.

Step 12

Check the Map to User Output check box and choose the value from the User Output drop-down list.

Step 13

Click Submit.

Assigning Big Data Accounts to User Groups

Procedure
Step 1

Choose Orchestration and click Workflows.

Step 2

Click Add Workflow.

Step 3

On the Add Workflow Details page, enter the workflow name and choose a folder. Click Next.

Step 4

On the Workflow Task Basic Information page, enter the required details and click Next.

Step 5

On the User Input Mappings to Task Input Attributes page, select the attributes that you want to map to the workflow input fields or provide values in the next step. If necessary, check the Map to User Input check box to provide user inputs..

Step 6

On the User Input Mappings to Task Input Attributes page, choose big data account type and big data account.

Step 7

Check the Assign Users checkbox and click Submit. The big data accounts are assigned to specified users if the Allow resource assignment to users option is enabled in the Users Groups page. When the Assign Users checkbox is not checked, the big data accounts are assigned to a specified group.

Unassigning Big Data Accounts

Before you begin
Procedure
Step 1

Choose Orchestration and click Workflows.

Step 2

Click Add Workflow.

Step 3

On the Add Workflow Details page, enter the workflow name and choose a folder. Click Next.

Step 4

On the Workflow Task Basic Information page, enter the required details and click Next.

Step 5

On the User Input Mappings to Task Input Attributes page, select the attributes that you want to map to the workflow input fields or provide values in the next step. If necessary, check the Map to User Input check box to provide user inputs..

Step 6

On the User Input Mappings to Task Input Attributes page, choose big data account type and big data account.

Step 7

Check the Unassign Users checkbox and click Submit. The big data accounts are unassigned from the group.

Cloning UCS CPA Workflows

To customize cluster deployment through bare metal workflows, you can clone the following workflows in the UCS CPA folder:

  • Clone the UCS CPA Node Bare Metal workflows.

  • Rename the Single UCSM Server Configuration WF using the cloned UCS CPA Node Bare Metal workflows.

  • Rename the UCS CPA Multi-UCSM Hadoop Cluster WF using the cloned Single UCSM Server Configuration WF.

Procedure
Step 1

Choose Orchestration.

Step 2

Click the UCS CPA folder from Workflows and choose the workflow that you want to clone.

Step 3

Click Clone Workflow.

Step 4

On the Workflow Details page of the Clone Workflow wizard, complete the following fields:

Name

Description

Workflow Name field

A unique name for the workflow.

Version field

The current version of the workflow that you are cloning. This is a display-only field.

Description field

The description of the workflow.

Workflow Context drop-down list

The workflow context. Workflow Orchestration supports the following options:
  • Any—Enables you to use the workflow in any context.
  • Selected VM—Enables you to use the execute workflow. This option can be selected only when you choose a VM.

  • Check the Save As Compound Task check box to define the workflow as a compound task.

  • Check the Place in New Folder check box, and enter the folder name in the Folder Name field, to assign the workflow to a new folder other than the UCS CPA folder.

Select Folder drop-down list

Choose a folder. UCS CPA is the default folder for Big Data.

Notify status of execution to initiator User check box

Check the check box to notify the user through email, then enter appropriate email addresses in the Additional User(s) to send Email Notification field.

Step 5

Click Next.

Step 6

On the Workflow User Inputs page of the Clone Workflow wizard, complete the following fields:

Name

Description

Associate to Activity check box

If the check box is checked then any existing workflow's user input is overridden by any selected activity user input.

Activity drop-down list.

Choose an activity. The user-input table is updated based on the selected activity.

Workflow User Inputs table

On the Workflow User Inputs page:

  1. Click the + icon to add workflow input properties.

  2. On the Add Entry to screen, complete the following fields:

    1. Enter the name for the activity workflow input in the Input Label field.

    2. Enter the description for the activity workflow input in the Input Description field.

    3. Check the Optional check box to set the input as optional during workflow execution.

    4. Click Select. On the Select screen, click Input Type.

    5. Click Submit.

Step 7

Click Next.

Step 8

On the Workflow User Outputs page of the Clone Workflow wizard, do the following:

Name

Description

Workflow User Outputs table

On the Workflow User Outputs page:

  1. Click the + icon to add workflow output properties.

  2. On the Add Entry to screen, complete the following fields:

    1. Enter the name for the activity workflow output in the Output Label field.

    2. Enter the description for the activity workflow output in the Output Description field.

    3. Check the Optional check box to set the output as optional during workflow execution.

    4. Click Select. On the Select screen, click Output Type.

    5. Click Submit.

Step 9

Click Submit.