Cisco UCS Director Express for Big Data Management Guide, Release 3.0

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Book Contents

Find Matches in This Book

Available Languages

Download Options

Book Title

Cisco UCS Director Express for Big Data Management Guide, Release 3.0

Chapter Title

Managing Hadoop and Splunk Clusters

PDF - Complete Book (3.28 MB) PDF - This Chapter (1.22 MB)
View with Adobe Reader on a variety of devices
ePub - Complete Book (391.0 KB)
View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi - Complete Book (821.0 KB)
View on Kindle device or Kindle app on multiple devices

Results

Updated:: September 15, 2016

Chapter: Managing Hadoop and Splunk Clusters

Managing a Hadoop Cluster
Managing Nodes in a Cluster
Delete Node and Delete Node to Bare Metal Actions in Cloudera and Hortonworks
Adding Managed Nodes to the Hadoop Cluster
Adding Live Nodes to the Hadoop Cluster
Adding Bare Metal Nodes to the Hadoop Cluster
Service Roles
Managing a Splunk Cluster

Managing Hadoop and Splunk Clusters

This chapter contains the following sections:

Managing a Hadoop Cluster
Managing Nodes in a Cluster
Delete Node and Delete Node to Bare Metal Actions in Cloudera and Hortonworks
Adding Managed Nodes to the Hadoop Cluster
Adding Live Nodes to the Hadoop Cluster
Adding Bare Metal Nodes to the Hadoop Cluster
Service Roles
Managing a Splunk Cluster

Managing a Hadoop Cluster

You can manage an existing cluster.

Step 1

On the menu bar, choose Solutions > Big Data > Accounts.

Step 2

Click the Hadoop Accounts tab.

Step 3

Select an account and click View Details.

Step 4

Click each tab to view the details.

Name

Description

Summary

Displays statistics data report for the selected Hadoop Account and the high-level report on the cluster and node account.

Hosts

See Managing Nodes in a Cluster to understand the functionality of this tab.

Hadoop Clusters

Displays the list of actions that you can perform on an existing Hadoop cluster.

Click Role Topology to view the topology of the nodes. (This tab is not editable.)
Click View Details to view the inputs for the Hadoop cluster you have created, and to view the virtual network interface configuration. (This tab is not editable.)
Click Shutdown Cluster to stop all the services and power off all the nodes in the Hadoop cluster.
Click Start Cluster to power up all the nodes in the Hadoop cluster and start all services in the Hadoop cluster.
Click Restart Cluster to power off and then power up all nodes in the Hadoop cluster.
Click Rebalance to configure the threshold percentage to rebalance Hadoop clusters. For MapR cluster, configure the threshold percentage using CLDB Balancer Disk Paused and CLDB Balancer Disk Max Switches in Nodes.

Click Upgrade Cluster to upgrade Hadoop distributions from the current version, if available.

Note

For a derived account, install and configure pssh and clush for the nodes in the Hadoop cluster and password less between nodes.

Click Upgrade Cluster.
Choose the JDK version from the Oracle JDK drop-down list.
Choose the Hadoop distribution that you want to upgrade from the current version from the Available Version drop-down list.
Check the Enable HA check box to enable high availability for the Hadoop cluster, if Cloudera is the Hadoop distribution.
Click Submit.

Enable Hadoop cluster high availability.

Click Enable High Availability.
From the Enable High Availability dialog box, access the Standby Name Node drop-down list and choose the Standby Name Node.

Check a minimum of three nodes from the Journal Nodes table.

Note

Journal nodes selection is recommended to be on first three Master nodes and should be increased by the node count in odd number only.

Click Submit.

Disable Hadoop cluster high availability. This action is supported only for Cloudera.

Click Disable High Availability.
From the Disable High Availability dialog box, access the Standby Name Node drop-down list and choose the Standby Name Node.

Check a minimum of three nodes from the Journal Nodes table.

Note

Journal nodes selection is recommended to be on first three Master nodes and should be increased by the node count in odd number only.

Click Submit.

Click Cluster Snapshot to view the snapshot.
Click View Reports to view performance and monitoring reports.

Hadoop Services

Displays the list of Hadoop services and their status. You can do the following:

Start All Services - Start all Hadoop services, depending on their status.
Stop All Services - Stop all Hadoop services, depending on their status.
Add New Service - Add a new Hadoop service.
Edit Service - Start and stop a particular Hadoop service.

Hadoop Service Roles

Displays the list of Hadoop services.

To add a role to the cluster, do the following in the Add Role dialog box:

From the Hadoop Service Name drop-down list, choose the Hadoop service.
From the Role Type drop-down list, choose the role type.
From the Node Name drop-down list, choose the node name.
From the Role Name drop-down list, choose the role name.
Click Submit.

To start or stop any role that you have created, do the following in the Start/Stop Role dialog box:

From the Hadoop Service Roles tab, choose the Hadoop service.
Click Start/Stop Role.
Click Submit.

To delete a role in the cluster, do the following in the Delete Role dialog box:

From the Hadoop Service Roles tab, choose the Hadoop service.
Click Delete.
Click Submit.

More Reports

Lists additional reports that you can generate about data usage and CPU utilization.

View Hadoop Cluster Details

For each Big Data Account, use the Hadoop Clusters tab to view details of all Hadoop clusters associated with the account. See Managing a Hadoop Cluster

You can view the following details by clicking the Hadoop Clusters tab.

Name	Description
Big Data Account name	The name of the Big Data account.
UCS SP Template for Big Data	The UCS SP Template for Big Data that you used to create service profiles for the servers in the Hadoop cluster.
Hadoop Cluster Profile Template	The Hadoop cluster profile template that you used to configure the cluster services.
Hadoop Cluster Deploy Template	A unique name that you used for the Hadoop cluster deployment template.
UCSM Policy Name Prefix	The UCSM Policy Name prefix.
Hadoop Cluster Name	A unique name that you used for the Hadoop cluster.
Hadoop Node Count	The number of nodes in the Hadoop cluster.
Hadoop Node Prefix	The Host Node prefix for the cluster.
OS Version	The operating system that you installed on the servers for the Hadoop cluster.
Hadoop Distribution	The Hadoop distribution that you used for this cluster.
Hadoop Distribution Version	The Hadoop distribution version that used for this cluster.
PXE VLAN ID	The VLAN ID used for PXE boot of the servers.
UCS Service Profile Template	The UCS Service Profile Template that you specified parameters for installing the OS and Hadoop distribution software.
Host Maintenance Policy	_
Host Firmware Package	_
UCSM Version	Displays the Cisco UCS Manager version
vNIC: eth0	Displays IPv4 network information for the management interface and the management VLAN ID.
vNIC: eth1	Displays IPv4 network information for the DATA1 interface and the VLAN ID. This filed is not displayed when the Use one vNIC check box is selected while creating a Cisco UCS Service Profile Template for Big Data.
vNIC: eth2	Displays IPv4 network information for the DATA2 interface and the VLAN ID. This filed is not displayed when the Use one vNIC check box is selected while creating a Cisco UCS Service Profile Template for Big Data.

Viewing a Cluster Snapshot

A cluster snapshot displays configuration details of a Hadoop cluster, such as hosts, roles, and services. To view the current snapshot of a cluster, do the following:

Step 1	On the menu bar, choose Solutions > Big Data > Accounts.
Step 2	Click the Hadoop Accounts tab.
Step 3	Choose the Hadoop Account for which you want to view the snapshot and click View Details.
Step 4	Click the Hadoop Clusters tab.
Step 5	Choose the Hadoop cluster for which you want to view the snapshot and click Cluster Snapshot.
Step 6	Click Submit. You can view the snapshot for the selected Hadoop cluster.

Adding a New Hadoop Service

Step 1

On the menu bar, choose Solutions > Big Data > Accounts.

Step 2

Choose a Cloudera account for which you can add a new Hadoop service.

Adding a new Hadoop service is not supported for MapR and Hortonworks distributions.

Step 3

Click View Details.

Step 4

Click the Hadoop Services tab.

Step 5

Click Add New Service.

In the Add New Service dialog box, complete the following fields:

Name	Description
Workflow Inputs
(Hadoop) Account Name	Choose the Cloudera Account.
Service Type	Enter the Hadoop service for Cloudera.
Role Assignment Pairs	Enter Role Assignment Pairs, separated by commas. For example, RoleType1:hostname1, RoleType2:hostname2
Dependant Services	Enter the list of dependent services. Use commas to separate list entries.
Pre Install Commands	Enter the list of commands. Use the "\n" command to insert a new line after each list entry.
Post Install Commands	Enter the list of commands. Use the "\n" command to insert a new line after each list entry.

Step 6

Click Submit.

Managing Nodes in a Cluster

You can add, delete, decommission, and recommission nodes in a cluster.

Managed Node—Any node that was already a member (managed) in the cluster and deleted, which can be added again in the cluster.
Live Node—Any node that has the operating system installed and is reachable from the Hadoop cluster.
Bare Metal Node—Any node that is available and is not associated with the Hadoop cluster.

Step 1

On the menu bar, choose Solutions > Big Data > Accounts.

Step 2

Click the Hadoop Accounts tab.

Step 3

Select an account and click View Details.

Step 4

Click the Hosts tab to perform the following actions:

Name	Description
Refresh	Refreshes the page.
Favorite	Adds the page to Favorites.
Add Managed Node	Adds managed nodes to the Hadoop cluster.
Add Live Node	Adds live nodes to the Hadoop cluster.
Add Bare Metal Nodes	Adds bare metal nodes to the Hadoop cluster.

Step 5

Select a host that allows you to perform the following actions:

Name

Description

View Details

Displays the summary of the CPU usage, the I/O status of the hosts disks, and so on.

Note

If you see a License Status tab, it indicates a licensing issue.

Delete Node

Deletes node from the cluster.

Assign Rack

Assigns the node to the rack server.

Recommission Node/ Decommission Node

Decommissioning or Recommissioning a node depends on its status.

Note

When the node is in decommissioned status, it means that all the roles for that node have been withdrawn.

Delete Node to Bare Metal

The node is removed from the cluster and disassociated from the service profile. The node becomes a bare metal server.

Host Mappings

Lists DNS entries of all the hosts in the Hadoop cluster.

Delete Node and Delete Node to Bare Metal Actions in Cloudera and Hortonworks

When you perform the Delete Node or Delete Node to Bare Metal actions, the UCS CPA Delete Node (a new workflow) is created for Cloudera and Hortonworks. You can also execute this workflow for MapR to perform the delete node operation. If you execute the UCS CPA Delete Node workflow for Cloudera and Hortonworks, this workflow also provides the functionality of Delete Node to Bare Metal action. This functionality is based on the Delete Node to Bare Metal flag setting as true or false. In addition, the Rollback UCS CPA Node Bare Metal workflow is created.

Adding Managed Nodes to the Hadoop Cluster

Add managed nodes to the Hadoop cluster.

Note

This Add Managed Nodes functionality is not supported for Hortonworks 2.3 and later versions, and MapR distribution, but you can use the Add Live Nodes functionality.

This feature allows you to add nodes that are available only from the following URLs, but not the members of the cluster.

Cloudera—http://serverip:7180/api/v6/hosts, where the serverIP is the IPv4 address of the administration node.
Hortonworks—http://serverIP:8080/api/v1/hosts, where the serverIP is the IPv4 address of the administration node

Step 1	On the menu bar, choose Solutions > Big Data > Accounts.
Step 2	Click the Hadoop Accounts tab.
Step 3	Double-click the Hadoop account.
Step 4	Click the Hosts tab.
Step 5	Click Add Managed Node.
Step 6	From the Host Name drop-down list, choose the host name.
Step 7	Click Submit.

Adding Live Nodes to the Hadoop Cluster

Add live nodes to the Hadoop cluster.

Step 1	On the menu bar, choose Solutions > Big Data > Accounts.
Step 2	Click the Hadoop Accounts tab.
Step 3	Double-click the Hadoop account.
Step 4	Click the Hosts tab.
Step 5	Click Add Live Node.
Step 6	Enter the IPv4 address in the Host Management IPv4 Address field.
Step 7	Enter the name of the rack server in the Rack Name field.
Step 8	Enter the password in the (New Node) Password field for that rack server.
Step 9	Choose the Cluster Manager Version for the Hadoop distribution from the Cluster Management Version drop-down list.
Step 10	Choose the operating system to be installed on the servers in this cluster from the OS Version drop-down list.
Step 11	Choose a MapR cluster template from the Hadoop Template Name drop-down list. This field is displayed only when you select the MapR cluster.
Step 12	Click Submit.

Adding Bare Metal Nodes to the Hadoop Cluster

Add bare metal nodes to the Hadoop cluster.

Step 1

On the menu bar, choose Solutions > Big Data > Accounts.

Step 2

Click the Big Data Accounts tab.

Step 3

Double-click the Big Data account.

Step 4

Click the Hosts tab.

Step 5

Click Add Bare Metal Nodes.

Step 6

In the Add Bare Metal Nodes dialog box, complete the following fields:

Name

Description

Big Data Account Name field

The name of the Big Data account.

UCSM Policy Name Prefix field

The UCSM Policy Name prefix.

Hadoop Cluster Name field

A unique name for the Hadoop cluster.

Hadoop Node Count field

The number of nodes in the Hadoop cluster.

Host Node Prefix field

The Host Node prefix for the cluster.

OS Version drop-down list

Choose the operating system to be installed on the servers in this cluster.

Hadoop Distribution drop-down list

Choose the Hadoop distribution to be used for this cluster.

Hadoop Distribution Version drop-down list

Choose the Hadoop distribution version.

Oracle JDK Version drop-down list

Choose the Oracle JDK version.

External Database drop-down list

Choose an external database. You can also configure a new database from here.

Multi-UCSM check box

Check the Multi-UCSM check box if you use multiple UCSM accounts.

Note

If you use the multiple UCSM accounts option, you can configure the Hadoop Server Roles as described in Step 7. You can add UCSM Specific Inputs in the Add Entry to UCSM Specific Inputs table.

The following workflows are created during an Instant and Customized Hadoop Cluster creation:

UCS CPA Multi-UCSM Hadoop Cluster WF.
Single UCSM Server Configuration WF. (This WF is triggered per UCSM Account. For example, UCSM 120, UCSM121.)
UCS CPA Node Bare Metal. (This WF is triggered per Node.)

The UCSM Specific Input area is displayed when the Multi-UCSM check box is checked.

UCS Manager Account drop-down list

Choose the Cisco UCS Manager account for this cluster.

Organization drop-down list

Choose the organization in which the servers for this cluster are located.

SSD Boot Drives Available for OS check box

Check this check box if you do not want to validate the server disk availability for RAID level OS disks. Ensure that the servers contain Solid-State Drive (SSD).

If the check box is not selected, the disk availability for both the OS disk and data disk are validated based on their RAID level.

Hadoop Template Name drop-down list

Choose a template for a MapR cluster. This field is displayed only when you select the MapR cluster.

UCS SP Template table

Choose an existing UCS Service Profile Template for Hadoop cluster creation.

PXE VLAN ID field

Enter the PXE VLAN ID.

UCSTemplate Name table

Check the UCS Service Profile Template check box that you want to use and click Submit to confirm the selection.

Step 7

If you want to edit a Hadoop Server Role, select the row for that role, and click Edit.

Step 8

In the Edit Hadoop Server Roles Entry dialog box, complete the following fields and click Submit.

Name

Description

Node Type field

Displays the Hadoop node role.

Node Count field

The number of nodes in the Hadoop cluster for the selected node type.

SSD Boot Drives Available for OS check box

Check this check box if you do not want to validate the server disk availability for RAID level OS disks. Ensure that the servers contain SSD.

If the check box is not selected, the disk availability for both the OS disk and data disk are validated based on their RAID level.

Note

This check box is not displayed when the UCSM version is greater than or equal to 3.

Server Pool table

Enter the server pool that you want to use for the cluster for the selected node type.

The Cisco UCS Manager account and the organization that you choose determine which server pools are displayed in this area.

Step 9

In the vNIC Template table, verify the vNIC templates available for the cluster.

Step 10

If you want to edit a vNIC template, select the row for that template and click Edit.

Step 11

In the Edit vNIC Template Entry dialog box, complete the following fields and click Submit.

Name	Description
vNIC Name drop-down list	The vNIC name in the selected template. This field is for your information only.
IP Pool drop-down list	Choose the Big Data IP pool that you want to use for IP addresses assigned to this vNIC.
MAC Address Pool drop-down list	Choose the MAC address pool that you want to use for this cluster.
First MAC Address field	Enter the MAC address.
Size field	Enter the size.
VLAN ID field	The VLAN ID for this cluster.

Step 12

Click Submit.

Service Roles

If you use Add Node Bare Metal, Add Managed Node, and Add Live Node actions, the following nodes-specific roles are added for each Hadoop distribution.

Service Roles	Cloudera	MapR	Hortonworks
FileServer	No	Yes	No
DataNode	Yes	No	Yes
NodeManager	Yes	Yes	Yes
Ganglia Monitor	No	No	Yes
NFS Gateway Service	No	Yes	No

Note

If NFS gateway service is available in the MapR cluster, the new nodes are installed with NFS gateway service. The service is not supported for fresh cluster deployment.

Managing a Splunk Cluster

You can manage the Splunk cluster from the Hosts tab.

Step 1

On the menu bar, choose Solutions > Big Data > Accounts.

Step 2

Click the Splunk Accounts tab.

Step 3

Click the Hosts tab to perform the following actions:

Name	Description
Refresh	Refreshes the page.
Favorite	Adds the page to Favorites.
Add Bare Metal Nodes	Add bare metal nodes to the Splunk cluster. You can add Indexer, Search Head, or Administrative node through Add Bare Metal workflow. You need to provide the Replication Factor based on the Inderxer count.

Note

You can also start, stop, or restart the Splunk cluster.

Step 4

Select a host that allows you to perform the following actions:

Name

Description

View Details

Displays the summary of the CPU usage, the I/O status of the hosts disks, and so on.

Note

If you see a License Status tab, it indicates a licensing issue.

Start

Starts the services on the node.

Stop

Stops the services on the node.

View Details

Restarts the services on the node.

Restart

Deletes node from the cluster.

Delete Node to Bare Metal

The node is removed from the cluster and disassociated from the service profile. The node becomes a Bare Metal server.

Step 5

Select an account and click View Details.

You can see only the Hosts tab. You can start, stop, or restart the Splunk cluster.

Step 6

Click the Performance tab.

Step 7

Click Run Test.

The Performance tab displays a default Big Data Metrics Report. This report shows the statistics collected for each host before the Splunk cluster creation and the reports post Splunk cluster creation only when you check the Memory Test, Network Test, and Disk Test check boxes in the Pre Cluster Performance Tests section of the Management tab. If you enable the precluster disk test, it impacts Splunk cluster creation.

Step 8

Click Submit, and then click OK.

For the following actions, choose the performance report:

Name	Description
View	Displays the metrics in the Big Data Metrics Report.
Compare	Compares and displays the metrics in the Big Data Metrics Report.
View Graph Report	Displays graphically the following reports from the Summary tab: Average TRIAD Rate (MB/Sec) Average Network Bandwidth (MB/Sec)
Delete	Deletes the Big Data Metrics Report.
More Reports	Displays the metrics on an hourly, daily, weekly, or monthly basis.

Step 9

Click the Monitoring tab.

Every time an inventory collection cycle is triggered, an entry listing the aggregate CPU, network bandwidth, and disk utilization metrics appears on the Monitoring Page.

Step 10

Select the entry you want to analyze and click View Details.

Step 11

Click Back to return to the Monitoring page.

Was this Document Helpful?

Feedback

Contact Cisco

Open a Support Case
(Requires a Cisco Service Contract)