Monitoring and Reporting

This chapter contains the following sections:

About Monitoring and Reporting

Cisco UCS Director Express for Big Data can monitor the virtual infrastructure and system resources, and provide a wide array of reports.

Cisco UCS Director Express for Big Data monitors a wide range of cluster events:

  • High CPU usage

  • Memory usage

  • Disk capacity

  • Disk IO utilization

Cisco UCS Director Express for Big Data displays statistics from the respective pages for the selected Big Data Account and Hosts. You can also generate reports and these reports help you to understand system details and provide an insight into how the system is performing.

Cisco UCS Director Express for Big Data Dashboard

The Cisco UCS Director Express for Big Data provides complete system visibility through real-time and historical monitoring. See Reports


Note

The Dashboard tab shows up in the menu bar only after a summary report is added to the dashboard.

The customizable dashboard displays key processing, memory, storage, and network utilization metrics.

  • Per node statistics: CPU, memory, and disk

  • Health of Hadoop cluster components: HDFS, MapReduce jobs

  • Graphs based on historical data

You can:

Note

Drag and Drop summary report icons from the Customize drawer to the Summary and Hosts tabs where you can expand and close reports.

  1. Add summary reports to the dashboard from the Big Data Account Summary tab and customize these summary reports to display the statistics for a specific duration in the dashboard, or export these reports from the dashboard.

  2. Add summary reports to the dashboard from the Big Data Account Hosts tab and customize these summary reports to display the statistics for a specific duration in the dashboard, or export these reports from the dashboard.


    Note

    Not all summary reports apply to MapR.

  3. Add UCSM Accounts summary reports to the dashboard from Physical > Compute.

  4. Add Data Center summary reports to the dashboard from Physical > Compute .

View Hadoop Cluster Details

You can view the inputs for all the Hadoop clusters you have created in Cisco UCS Director Express for Big Data. For each Big Data Account, you can view the details of all Hadoop clusters that are associated with this account from the Hadoop Clusters tab. See Managing a Hadoop Cluster

Name

Description

Big Data Account name

The name of the Big Data account.

UCS SP Template for Big Data

The UCS SP Template for Big Data that you used to create service profiles for the servers in the Hadoop cluster.

Hadoop Cluster Profile Template

The Hadoop cluster profile template that you used to configure the cluster services.

Hadoop Cluster Deploy Template

A unique name that you used for the Hadoop cluster deployment template.

UCSM Policy Name Prefix

The UCSM Policy Name prefix.

Hadoop Cluster Name

A unique name that you used for the Hadoop cluster.

Hadoop Node Count

The number of nodes in the Hadoop cluster.

Hadoop Node Prefix

The Host Node prefix for the cluster.

OS Version

The operating system that you installed on the servers for the Hadoop cluster.

Hadoop Distribution

Hadoop distribution that you used for this cluster.

Hadoop Distribution Version

Hadoop distribution version that used for this cluster.

PXE VLAN ID

The VLAN ID used for PXE boot of the servers.

UCS Service Profile Template

The UCS Service Profile Template that you specified parameters for installing the OS and Hadoop distribution software.

vNIC: eth0

Displays IPv4 network information for management interface and the management VLAN ID.

vNIC: eth1

Displays IPv4 network information for DATA1 interface and the VLAN ID.

vNIC: eth2

Displays IPv4 network information for DATA2 interface and the VLAN ID.

Viewing Deployed Cluster Report

You can generate the Big Data Account Summary Report with or without credentials to view the details of the deployed clusters.

Step 1   On the menu bar, choose Solutions > Big Data Containers.
Step 2   Click the Deployed Clusters tab.
Step 3   Choose the deployed cluster, and click View Report.
Step 4   In the View Report dialog box, choose the report type that you want to generate.
Step 5   Click Submit.

Reports

UCSM Account Summary Reports

  • Overview

  • UCS Chassis Inventory

  • UCS Server Inventory

  • UCS Fabric Interconnect Inventory

  • UCS Servers Associated vs Unassociated

  • Rack Server Discovery Policy

  • Power Policy

  • Global Power Allocation Policy

Big Data Account Summary Reports

  • Overview

Cluster Specific Metrics Supported per Hadoop Distribution

Metrics

Cloudera Distribution

Hortonworks Distribution

MapR Distribution

Remarks

Average CPU IO Wait (%)

Yes

Yes

Yes

Average CPU idle (%)

Yes

Yes

Yes

Average CPU Nice (%)

Yes

Yes

Yes

Average CPU System (%)

Yes

Yes

Yes

Average CPU User

Yes

Yes

Yes

CPU Percentage Across Hosts

Yes

Metrics for Host CPU usage across hosts

Cluster Disk IO

Yes

Metrics for Total Disk Write/Read Bytes Across Disks

Cluster Network IO

Yes

Metrics for Total Bytes Transmitted/Received Across Network Interfaces

HDFS IO

Yes

Metrics for Total Bytes written/Read Across Data Nodes

Running MapReduce Job CPU Utilization

Yes [MapR 3.1.1 only]

Running MapReduce Job Memory

Yes [MapR 3.1.1 only]

Job Map Input Bytes Rate

Yes [MapR 3.1.1 only]

Job Map Input Records Rate

Yes [MapR 3.1.1 only]

Job Map Output Bytes Rate

Yes [MapR 3.1.1 only]

Job Map Output Records Rate

Yes [MapR 3.1.1 only]

Job Reduce CPU

Yes [MapR 3.1.1 only]

Job Running Map Task Count

Yes [MapR 3.1.1 only]

Job Running Reduce Task Count

Yes [MapR 3.1.1 only]

Job Running Task Count

Yes [MapR 3.1.1 only]

Total Space Utilization

Yes

CPU

Yes

Load

Yes

Memory

Yes

Network

Yes

Process

Yes

Host Specific Metrics Supported per Hadoop Distribution

Metrics

Cloudera Distribution

Hortonworks Distribution

MapR Distribution

Average CPU IO Wait (%)

Yes

Yes

Yes

Average CPU idle (%)

Yes

Yes

Yes

Average CPU Nice (%)

Yes

Yes

Yes

Average CPU System (%)

Yes

Yes

Yes

Average CPU User

Yes

Yes

Yes

Load Average(%)

Yes

Host CPU Usage

Yes

Host Memory Usage

Yes

Host Network Throughput

Yes

Disk Latency

Yes

Aggregate Disk Throughput

Yes

Role-CPU Usage

Yes

HOST CPU

Yes

DISK Usage

Yes

LOAD Usage

Yes

Memory-CPU Usage

Yes

Network Usage

Yes

Process Usage

Yes