What You Will Learn
The Cisco Nexus™ 7000 Series Switches combine the highest levels of scalability with operational flexibility. TheCisco Nexus 7000 Series is a modular data center-class product line designed for highly scalable 10 Gigabit Ethernet networks with a fabric architecture that scales beyond 15 terabits per second (Tbps). Cisco® NX-OS Software, a state-of-the-art operating system, powers the Cisco Nexus 7000 Series platform. Cisco NX-OS is a data center-class operating system built with modularity, resiliency, and serviceability at its foundation. Cisco NX‑OS helps ensure serviceability for mission-critical data center environments by providing a comprehensive set of features, including a built-in protocol analyzer based on the popular open source Wireshark software.
This document begins with a brief introduction to the Wireshark protocol analyzer and describes the Wireshark-like functions provided by Cisco NX-OS. The document explains how to use the Wireshark protocol analyzer for real‑time analysis of control-plane and data-plane traffic. This document also summarizes factors to consider when using this function and the effect it can have on Cisco Nexus 7000 Series supervisors’ CPUs (Figure 1).
Introduction to Wireshark
Wireshark, formerly known as Ethereal, is the world’s foremost network protocol analyzer and the standard across multiple industries and within many educational institutions. Wireshark development thrives thanks to the contributions of networking experts across the globe. Wireshark can interactively browse packet data from a live network or from a previously saved capture file. Wireshark’s packet capturing is performed using the pcap library; its native capture file format is the libpcap format, which is also the format used by tcpdump and various other tools.
Wireshark’s main window shows three views of a packet: a summary line briefly describes the packet type, the protocol field of interest can be shown and analyzed in the portion of the window directly below the summary line, and a hexadecimal dump shows exactly what the packet looks like when it goes across the wire. In addition, Wireshark has some features that make it unique; for example, it can assemble all the packets in a TCP conversation and highlight the ASCII data in that conversation. The display filters in Wireshark are powerful; more fields are filterable in Wireshark than in other protocol analyzers. Figure 2 shows an example of the Wireshark GUI.
Wireshark Capability in the Cisco Nexus 7000 Series
Network administrators often have difficulty gaining a complete knowledge of the nature of the control-plane traffic flowing through their network, but visibility into control-plane traffic is critical to full control over the network. For this reason, Cisco decided to embed a protocol analyzer within the Cisco NX-OS Software running on the Cisco Nexus 7000 Series Switches.
Cisco NX-OS is a modern, modular operating system running a Linux kernel (the MontaVista Linux 2.6.10 kernel). The architecture makes it easy to embed productive tools used by network administrators who are working in Linux-based environments. The most significant example of this integration is support for an integrated packet analyzer for the network traffic destined to or generated by the Cisco Nexus 7000 Series supervisor. Using the command-line version of Wireshark, called TShark (for Terminal Wireshark), as a basis, Cisco developed the Cisco NX-OS Ethanalyzer. The current version of TShark, on which the Cisco Ethanalyzer is based, is TShark Version 1.0.8, and the libpcap library is based on TShark Version 0.9.8.
Capturing and Analyzing Live Traffic on the Cisco Nexus 7000 Series Supervisor
Ethanalyzer is available only in the default virtual device context (VDC). Ethanalyzer can interactively analyze packets being sent to (or generated by) the supervisor; more specifically, it can capture traffic received by the supervisor from both the out-of-band management port (mgmt0) and the I/O modules:
You cannot specify the explicit I/O module interface on which to perform the capture. As will be explained later in this document, however, you can configure capture filters that capture only traffic of interest.
After specifying where to capture traffic, you can choose among several Ethanalyzer options to specify how and what to capture:
● capture-filter: This powerful option restricts the capture to only the traffic of interest. The capture-filter option has the same syntax, and thus the flexibility, of the Linux tcpdump utility. Here some examples:
Capture only traffic to or from the particular IP address 172.16.7.3:
Capture traffic to or from a range of IP addresses:
Capture traffic from a range of IP addresses:
Capture traffic to a range of IP addresses:
Capture only Domain Name System (DNS) traffic:
Capture traffic that is not HTTP or Simple Mail Transfer Protocol (SMTP) traffic:
Capture traffic except Address Resolution Protocol (ARP) and DNS traffic:
Capture traffic within a range of Layer 4 ports:
Capture only Ethernet type Authentication Protocol over LAN (EAPOL) traffic:
Reject Ethernet frames belonging to the Link Layer Discovery Protocol (LLDP) multicast group:
● More capture filtering options are available from these resources:
● decode-internal: This option is for Cisco Technical Assistance Center (TAC) use only, as it does not provide any meaningful information to the network administrator.
● detailed-dissection: This option enables the user to see a detailed view of the captured packet. Every protocol field is decoded and presented in a clear and organized format for in-depth analysis. If this option is not specified, Ethanalyzer will print a summary line for each packet it captures; the line will show the most important protocol fields.
● display-filter: Ethanalyzer supports post-capture display filters. Here are some examples:
Show only SMTP (port 25) and ICMP traffic:
Show only traffic in the LAN (172.16.x.x) between clients and servers:
The TCP buffer is full, so the source instructs the destination to stop sending data:
Filter on Microsoft Windows; filter out noise while watching Microsoft Windows client and data center exchanges:
You can also search for characters appearing anywhere in a field or protocol by using the “matches” operator. For example, you can match packets that contains the 3-byte sequence 0x71, 0x6A, 0xE3 anywhere in the User Datagram Protocol (UDP) header:
The “slice” feature is useful for filtering on the vendor’s organizational unique identifier (OUI) of the MAC address; thus, you can restrict the display to only packets from a specific device manufacturer:
The use and syntax of the display filters are described in the Wireshark user’s guide:.
● dump-pkt: This option causes Ethanalyzer to print a hexadecimal and ASCII dump of the packet data after printing a summary line that helps quickly identify the packet type.
● limit-captured-frames: With this option, you can specify the number of packets to be captured. The default value is 10 packets. A value of zero tells the system to capture packets indefinitely until Ethanalyzer is explicitly stopped.
● limit-frame-size: This option specifies how many bytes of the packets will be displayed.
● write: This very useful option allows you to write the capture data to a file in one of the storage devices available on the Cisco Nexus 7000 Series Switch for later analysis. The capture file size is limited to 10 MB. When the capture data is saved to a file, the captured packets are not displayed in the terminal window. Thedisplay-write option will force Cisco NX-OS to display the packets while also saving the capture data to a file.
Here is an example of a live capture of Hot Standby Router Protocol (HSRP) packets reaching the Cisco Nexus 7000 Series supervisor:
The entire packet and all its fields are displayed when the detailed-dissection option is part of the command:
For those who can read hexadecimal code, the dump-pkt keyword provides a good exercise:
To stop Ethanalyzer, press Ctrl-C.
Analyzing a Previously Saved Capture
Ethanalyzer can decode previously saved captures stored in any of the storage devices available on Cisco Nexus 7000 Series Switches. The display options available for live captures are also available when reading a previously saved capture:
On-Demand Analysis of Data-Plane Traffic
Ethanalyzer is part of the software running on the supervisor, and its main design goal is the capture and analysis of the traffic directed to and generated by the supervisor. A Cisco Nexus 7000 Series Switch is a fully distributed forwarding system in which the hardware-based data plane is implemented in the I/O modules. The data traffic is not seen by the supervisor or by Ethanalyzer, because all traffic is forwarded by the I/O modules. However, the Cisco Nexus 7000 Series does provide a method for enabling Ethanalyzer to capture data traffic.
In this mode of operation, Ethanalyzer gives network administrators a powerful, easy-to-use tool that increases visibility into application behavior and increases their ability to exert control over the network environment.
Ethanalyzer can analyze application traffic on demand with a few simple steps:
1. Identify the application characteristics: for example, Layer 4 ports.
2. Create an impromptu access control list (ACL) to match (and permit) the application flow between two known servers.
3. Add the “log” keyword to the Access Control Entries (ACEs) to send copies of the matching traffic to the supervisor. This mechanism allows the supervisor to receive the traffic of interest and allows Ethanalyzer to capture it.
4. Set the configurable hardware-based rate limiter, access-list-log, to limit the rate at which copies of the packets are sent to the supervisor. This function is extremely important because it protects the Cisco Nexus 7000 Series supervisor from the eventual high rates of the packet copies generated by the ACL logging mechanism.
If you want to capture and analyze the traffic at line rate, then Ethanalyzer is not the right tool; the Cisco Nexus 7000 Series provides other features such as NetFlow and Switched Port Analyzer (SPAN) that are designed explicitly for that purpose.
5. Now open a different terminal session to the Cisco Nexus 7000 Series Switch and start Ethanalyzer with the appropriate capture filters in place. No packets should be captured at this time, as the ACL has not been applied to any interface yet.
6. In the original terminal window, now apply the impromptu ACL to the interfaces at which the traffic of interest is expected to be received.
After the ACL is in place, the forwarding engine will start to generate the copies of the matching traffic, while the original packets are forwarded with no effect on performance.
The copies allowed by the rate limiter will then reach the supervisor, where Ethanalyzer can easily capture and analyze them.
The following example illustrates how to implement these steps.
Consider an application using TCP port 5600 between server 220.127.116.11 and client 18.104.22.168. The first step is to create the impromptu ACL:
The access-list-log hardware-based rate limiter is responsible for making sure that the copies of the matching traffic (generated by the logging mechanism) do not overwhelm the supervisor. The default value of the rate limiter is 100 packets per second (pps) and can be tuned by the network administrator. The rate limiting occurs on a per‑forwarding-engine basis. This implies that if the ACL is applied to interfaces on different line cards, the rate of the copies will be N times the value of the rate limiter, where N is the number of forwarding engines serving the interfaces being analyzed. Cisco’s recommendation is not to exceed 1000 pps of aggregated access-list-log rate‑limited traffic.
The value of the rate limiter can be modified and shown as follows:
With the setting shown here, the supervisor will not receive more than 250 pps of copies from each forwarding engine.
On another terminal window, the network administrator can start Ethanalyzer:
Capturing on inband
The copy of the data traffic generated by the application is now being captured, and the capture is stored in the bootflash memory for later analysis.
Effect of Ethanalyzer on CPU
Ethanalyzer is part of the software running on the supervisor. It is important to understand its effect to the supervisor’s CPU. Testing has shown an increase in the supervisor’s CPU utilization of just under 5 percent. Theutilization can be decreased by 1 or 2 percent by saving the capture data in a file (by using the write option).
General Public License Considerations
The copyrights to certain works contained in the Cisco NX-OS Software are owned by third parties and used and distributed under license. Certain components of this software are licensed under GNU General Public License (GPL) Version 2.0 or GNU Lesser General Public License (LGPL) Version 2.1. A copy of each license is available at:
Cisco NX-OS helps ensure serviceability for mission-critical data center networks by providing a comprehensive set of features, including a built-in protocol analyzer, Ethanalyzer, based on the popular open source Wireshark protocol analyzer.
Network administrators have difficulty gaining complete knowledge of the control-plane traffic that flows through their networks. Visibility into this critical component of every network environment is critical to attaining increased control over the network. Ethanalyzer provides a simple tool for analyzing the network traffic destined to and generated by the supervisor.
For More Information