The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter provides the overview to a set of visibility plugins that can be used to get an integrated view of your Big Data cluster.
In a network, throughput-sensitive traffic has a direct impact on latency-sensitive traffic. For example, in the Hadoop framework, running a map or reduce job next to a HBase read request, can cause large spikes in the read latency due to buffer (queue) build up on the network. To understand these scenarios, it is important to correlate the actual source of queuing the map or reduce job and any other traffic that may be present at the same time. To do this, you can use the programmability function of the Cisco Nexus 3000 Series switches and provide a high resolution view of the buffer occupancy of the actual map or reduce jobs running on the switch, along with details of the jobs. Cisco NX-OS Release 6.0(2)U3(1) introduces a set of Python scripts that provide you with an integrated view of your Big Data cluster.
The scripts are separate for the server and switch. They are:
This python script is for a server that can do the following:
If the client sends a value of '1' to the server, the server requests the list of active trackers from the switch (client) side. If the client sends a value of '2' to the server, the server requests a list of active jobs from the switch. If the client sends a value of '101', then the server requests a list of the MAC addresses of all devices connected to the switch.
This script also uses a connection to a remote server (port: 34567) to retrieve the fully qualified domain name (FQDN) of each server connected to the switch. The port and FQDN of each server is formatted in a table in the output.
This script also establishes a connection to a remote server (port: 34567) and uses it to list out all servers that are connected to the switch, which have trackers running on them. The output of the script identifies each server's name, its physical port on the switch side, and the listening port on the server side.
This script operates on a particular switch, it establishes a connection with a remote server on the port 34567, uses it to retrieve the names of all the servers connected to the switch, and then lists out the buffer usage corresponding to each server at specific time intervals.
This script includes the functionality of the bufferServerMap.py script, and it will also provide more details (running time, priority, user) on the jobs that were running during peak buffer usage.
For more information on these scripts and their output, refer to
Python Version 2.7 is needed to run these scripts.
There are two components to these scripts. Server side and the switch side. The server side component (appInfo.py) must be run on any Linux-based server, which has access to the hadoop cluster nodes that are connected to the switch.
To run the script on the server, type:
The appInfo.py script is designed to run infinitely and provide any server-related information requested by the Cisco Nexus 3000 Series switch. The scripts under the Nexus3548 are supposed to be run on the Nexus 3548. They connect to the server component (appInfo.py) to gather application-related information.
For example, to run bufferServerMap.py, use: