Cisco® NX-OS Software is a state-of-the-art operating system for the Cisco Nexus® Family switching platform. Cisco NX-OS is a data center-lass operating system built with modularity, resiliency, and serviceability at its foundation. To help ensure serviceability for mission-critical data center environments, Cisco NX-OS provides comprehensive features, including a built-in protocol analyzer called Ethanalyzer.
Ethanalyzer provides sniffing capabilities to Cisco NX-OS within the operating system, simplifying the need for a third-party network probe to capture control traffic sourced from or destined to the switch, including Spanning Tree Protocol and Link Aggregation Control Protocol (LACP) traffic.
This document explains the various packet capture tools and the Ethanalyzer implementation, based on the open source Terminal Wireshark (TShark) utility and providing additional functions from tcpdump. The document describes the functions available in Ethanalyzer and the circumstances in which they can be used. It also explains how to configure Ethanalyzer to capture and read packet captures in real time. This document illustrates Ethanalyzer configuration with examples.
Packet-capture tools, also known as sniffers or packet analyzers, intercept and log traffic that is passing over a network or a part of a network. When data streams flow across the network, the sniffer can capture each packet. If needed, it can decode a packet's raw data, showing the values of the various fields and analyzing the packet's content using RFC and other specifications.
Three of the most popular open source packet capture tools are tcpdump and TShark for UNIX-based systems and Wireshark (previously Ethereal) for Microsoft Windows and Mac OS.
Tcpdump is the most common command-line-based packet analyzer tool. It is distributed under the free Berkeley Software Distribution (BSD) license. It operates on most UNIX-based operating systems: BSD, Linux, Solaris, Mac OS, etc. Tcpdump uses the libpcap library to capture packets. It can capture packet data from a live network or read packets from a previously saved capture file, either printing a decoded form of those packets to the standard output or writing the packets to a file.
Wireshark is very similar to Tcpdump, with the addition of an open source GUI and visual tools for information sorting and filtering. Wireshark allows the user to monitor the network interfaces, wired or wireless. On Linux, BSD, and Mac OS X, Wireshark uses libpcap; and for Microsoft Windows, it uses the WinPcap library.
Terminal-based TShark is the command-line network protocol analyzer of Wireshark. In addition to tcpdump's functions, it displays a running count of captured packets while the capture is in progress. TShark's native capture file format is also the libpcap format.
Libpcap is a portable C/C++ UNIX library for packet capture (pcap). It consists of an API for capturing network traffic.
Cisco Nexus NX-OS Implementation
Cisco NX-OS is a modular operating system running a Linux Montavista kernel, Version 2.6.10. This implementation facilitates addition of embedded Linux tools to the environment. Cisco implemented Ethanalyzer on Cisco NX-OS, a wrapper over TShark. TShark, as described in the previous section uses libpcap. This implementation enables Ethanalyzer to capture and decode packets using TShark and the libpcap library.
The implementation of Ethanalyzer directly in the Cisco NX-OS command line makes the utility very easy to use, without the need for any additional steps, such as use of a dedicated shell mode.
Effect of Ethanalyzer on the CPU
Since Ethanalyzer is part of the software running on the supervisor, understanding its effect to the supervisor's CPU is important. Testing has shown an average increase in the supervisor's CPU utilization of just under 5 percent. Utilization can be decreased by 1 or 2 percent by saving the capture data in a file using the write option, described later in this document.
When to Use Ethanalyzer on Cisco NX-OS
Ethanalyzer is useful when troubleshooting problems related to the switch itself. The packets captured by Ethanalyzer need to be generated or destined for the switch supervisor CPU itself.
Ethanalyzer offers these main benefits:
• Ethanalyzer, as part of the Cisco NX-OS integrated management tool set, improves troubleshooting and reduces time to resolution.
• Ethanalyzer helps preserve and improve the operational continuity of the network infrastructure.
• Ethanalyzer achieves its benefits by allowing network administrators to learn about the amount and nature of the control-plane traffic within their switches.
Ethanalyzer enables you to accomplish the following functions:
• Capture packets sent to and received by the switch supervisor CPU
• Set the number of packets to be captured
• Set the length of the packets to be captured
• Display packets with either very detailed protocol information or a one-line summary
• Open and save the packet data captured
• Filter packet captures based on many criteria
• Filter packet displays based on many criteria
• Decode the internal header of the control packet
Ethanalyzer does not capture hardware switched traffic between data ports of the switch. For this type of packet capture, you can use Cisco Switch Protocol Analyzer (SPAN). For more information about SPAN, please refer to the related white papers and configuration guides.
How to Configure Live Packet Captures on Cisco NX-OS
First decide what is the traffic of interest to be captured and where it is expected on the switch: the inband ports or the out-of-band management mgmt0 port (Figure 1).
The Cisco Nexus 5000 and 3000 Series Switches have three types of Ethernet interfaces used by Ethanalyzer:
• Eth0 is the mgmt0 port of the switch. It is defined as mgmt in Ethanalyzer.
• Eth3 is an inband interface handling traffic from the data ports of the switch. It handles low-priority control packets destined for the switch CPU such as Internet Group Management Protocol (IGMP), TCP, User Datagram Protocol (UDP), IP, and Address Resolution Protocol (ARP) traffic. It is defined as inbound-low in Ethanalyzer.
• Eth4 is also an inband interface handing traffic from the data ports of the switch. It handles high-priority control packets destined for the switch CPU, such as Spanning Tree Protocol, Link Aggregation Control Protocol (LACP), Cisco Discovery Protocol, Data Center Bridging Exchange (DCBX), Fibre Channel, and Fibre Channel over Ethernet (FCoE) traffic. It is defined as inbound-hi in Ethanalyzer.
The Cisco Nexus 7000 Series has two types of Ethernet interfaces used by Ethanalyzer:
• Eth1 is the mgmt0 port of the switch. It is defined as mgmt in Ethanalyzer.
• Eth0 is an inband interface handling traffic from the data ports of the switch. It handles all control packets destined for the switch CPU such as IGMP, TCP, UDP, IP, ARP, Spanning Tree Protocol, LACP, Cisco Discovery Protocol, DCBX, Fibre Channel, and FCoE . It is defined as inband in Ethanalyzer.
Figure 1. Ethanalyzer Port Types
Note: All in-band Ethernet ports sending or receiving data over the switch supervisor will be captured with the inband-hi or inband-low option. However, display or capture filtering can be applied.
To start a packet capture with Ethanalyzer, use these commands:
Switch# ethanalyzer local interface [inbound-hi|inbound-low|mgmt] (options)
Table 1 lists the options.
Table 1. Ethanalyzer Packet Capture Options
Capture autostop condition
Filter on Ethanalyzer capture
Capture ring buffer option
Include internal system header decoding
Display detailed protocol information
Display filter on frames captured
Maximum number of frames to be captured (default is 10)
Capture only a subset of a frame
Filename to which to save capture
By default, Ethanalyzer captures up to 10 frames. Use the limit-captured-frames option to change the value; to remove the limit, set the value to 0.
There are two filtering approaches for configuring a packet capture: you can use the display filter, or you can use the capture filter. Both filtering methods can be used when displaying or saving a capture to a file. You can also capture in full detail all the traffic and then export the capture file to the Wireshark utility and filter it in the Wireshark user interface.
Further information about the display and capture filters can be found at:
2011-05-15 22:16:01.412162 00:05:73:b4:f3:a4 -> c8:4c:75:5b:25:40 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412361 c8:4c:75:5b:25:40 -> 00:05:73:b4:f3:a4 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412468 00:05:73:b4:f3:a4 -> c8:4c:75:5b:25:40 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412623 c8:4c:75:5b:25:40 -> 00:05:73:b4:f3:a4 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412749 00:05:73:b4:f3:a4 -> 68:ef:bd:62:28:80 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412872 c8:4c:75:5b:25:40 -> 00:05:73:b4:f3:a4 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412940 00:05:73:b4:f3:a4 -> c8:4c:75:5b:25:40 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
2011-05-15 22:16:01.412948 68:ef:bd:62:28:80 -> 00:05:73:b4:f3:a4 LLC U, func=UI; SNAP, OUI 0x00000C (Cisco), PI
10 packets captured
Program exited with status 0.
Note: By default, the capture is limited to 10 frames. This value can be changed with the option limit-captured-frames <0-2147483647>. Capture is stopped after the specified number of frames (0 means no limit).
Use the display filter to display only Cisco Discovery Protocol packets:
switch# ethanalyzer local interface inbound-hi decode-internal display-filter "cdp"
Capturing on eth4
2011-05-15 20:50:08.971379 00:05:73:ce:47:c9 -> 01:00:0c:cc:cc:cc CDP Device ID: switch2(SSI15040AM0) Port ID: E
2011-05-15 20:50:08.971392 00:05:73:ce:47:f7 -> 01:00:0c:cc:cc:cc CDP Device ID: switch2(SSI15040AM0) Port ID: E
2011-05-15 20:50:10.401985 00:05:73:ce:47:c9 -> 01:00:0c:cc:cc:cc CDP Device ID: switch2(SSI15040AM0) Port ID: E
2011-05-15 20:50:10.401997 00:05:73:ce:47:f7 -> 01:00:0c:cc:cc:cc CDP Device ID: switch2(SSI15040AM0) Port ID: E
2011-05-15 20:50:15.459076 00:22:55:79:b1:c1 -> 01:00:0c:cc:cc:cc CDP Device ID: switch3(TBM12347542) Port
Capture the full frame detail:
switch# ethanalyzer local interface inbound-hi detail limit-captured-frames 1
Capturing on eth4
Frame 1 (68 bytes on wire, 68 bytes captured)
Arrival Time: May 15, 2011 22:01:11.892278000
[Time delta from previous captured frame: 1305496871.892278000 seconds]
[Time delta from previous displayed frame: 1305496871.892278000 seconds]
[Time since reference or first frame: 1305496871.892278000 seconds]
Frame Number: 1
Frame Length: 68 bytes
Capture Length: 68 bytes
[Frame is marked: False]
[Protocols in frame: eth:vlan:llc:stp]
Ethernet II, Src: 00:05:73:b4:f3:c6 (00:05:73:b4:f3:c6), Dst: 01:00:0c:cc:cc:cd (01:00:0c:cc:cc:cd)
Network administrators have difficulty gaining complete knowledge of the control-plane traffic. Visibility into this critical component of every network environment crucial to attaining increased control over the network. Cisco NX-OS helps ensure serviceability for mission-critical data center networks by providing a comprehensive set of features, including a built-in protocol analyzer: Ethanalyzer. Ethanalyzer is based on the popular open source Wireshark protocol analyzer, which provides additional features from tcpdump. Ethanalyzer provides a simple tool for analyzing the network traffic destined to and generated by the supervisor and is directly accessible from the Cisco NX-OS CLI, without the need for additional steps and with very limited impingement on the switch CPU.
For More Information
Refer to the documentation (white papers, configuration guides, troubleshooting guides, etc.) at:
The copyrights to certain works contained in Cisco NX-OS Software are owned by third parties and used and distributed under license. Certain components of this software are licensed under GNU General Public License (GPL) Version 2.0 or GNU Lesser General Public License (LGPL) Version 2.1. A copy of each license is available at: