Packet-tracer is an inbuilt utility on the Nexus 9000 that can be used to trace the path of the packet through the switch. It can be invoked using the command line and can be configured to match IP address and or layer 4 attributes. It cannot be used to match ARP traffic.
This tool will provide confirmation on whether a flow is traversing through the switch. It also provides a counter to track flow statistics which can be useful for scenerios of intermittent/complete packet loss.
Cisco recommends that you have basic knowledge of these topics:
Cisco Nexus 9000 hardware architecture
The information in this document is based on these software and hardware versions:
Cisco Nexus 9500
SW Version 7.0(3)I2(2a)
Use Case Scenerios
Applicable for IPv4 flows only (IPv6 and non IP not supported)
This tool does not display the packet inner details as shown by wireshark.
Intermittent packet loss : Ping or any other utility can provide a definite symptom of lost packets
Complete packet loss
Only line cards/Fabric Modules or TORs with Broadcom Trident II asics are supported. List is below:
Note: Please reach out to TAC if a specific line card/TOR is not listed
How to Use Packet Tracer
Packet-tracer commands are EXEC level commands.
N9K-9508#test packet-tracer src_ip <src_ip> dst_ip <dst_ip> <==== provide your src and dst ip N9K-9508#test packet-tracer start <==== Start packet tracer N9K-9508#test packet-tracer stop <==== Start packet tracer N9K-9508#test packet-tracer show <==== Check for packet matches
The above commands programs the trigger on every Broadcom Trident II Asic that exist on the line card or the fabric modules. When a flow with the matching attributes passes through these modules, it will show the counters being hit thereby helping identify the path within the switch (Ingress module--->One of the fabric module---->egress module).
The counters can be used to corelate drops.
Fabric modules interconenct I/O module slots. All fabric moduels are active and carry traffic. Two Broadcom Trident II ASIC (T2) instances per fabric module.
PACL (Port Access-list) is used to see if a particular physical interface received our interested traffic. However on Nexus platform, some of the linecards do not have TCAM carved for PACL. TCAM carving requires module reload. In those cases, use packet tracer to match the interested traffic. You can also trace the packet going up to fabric ports and going towards egress module. So packet tracer gives you more insight into how traffic is being forwarded within the switch.
Packet tracer uses TCAM entries carved for SPAN.
NS - North Star ASIC T2 - Trident II ASIC NFE - Network Forwarding Engine ALE - ACI Leaf Engine
For more infomation about Nexus 9000 Switch Architecture, refer to:
switch#test packet-tracer src_ip <src_ip> dst_ip <dst_ip> protocol <> <==== provide your src and dst ip and protocol (protocol option 1 is for icmp) switch#test packet-tracer start <==== Start packet tracer switch#test packet-tracer show <==== Check for packet match statistics
You don't need to apply it to any paritcular interface. Above config installs filter ACL across all LC’s/FM’s on all instances of T2 ASIC. It will show packet count on the module on which traffic ingressed. This matches our interested traffic ingressing on a module , both linecard and Fabric.
Here is a configuraiton example:
N9K-9508# test packet-tracer src-ip 10.1.1.1 dst-ip 10.2.2.1 protocol 1 <=== Protocol 1 matches ICMP traffic N9K-9508# test packet-tracer start
Here is how to interpret 'test packet-tracer show" output:
N9K-9508# test packet-tracer show Packet-tracer stats --------------------- Module 1: <=== Slot #. Same output will be displayed for other Linecards's and Fabric modules. Filter 1 installed: src-ip 10.1.1.1 dst-ip 10.2.2.1 <==== Our filter #1 ASIC instance 0: <==== Trident ASIC instance #0 Entry 0: id = 7425, count = 0, active, fp, <==== pakcet match count on front panel port. it could be any port Entry 1: id = 7426, count = 0, active, hg, <==== packet match count from fabric module to T2 ASIC on the linecard ASIC instance 1: Entry 0: id = 7425, count = 0, active, fp, Entry 1: id = 7426, count = 0, active, hg, Filter 2 uninstalled: Filter 3 uninstalled: Filter 4 uninstalled: Filter 5 uninstalled:
test packet-tracer remove-all <=== Removes all configured filters test packet-tracer clear <filter #> <=== Clear counters for all filters or specified filter test packet-tracer src_ip <.> dst_ip <> l4-dst-port <dst_port> | l4-src-port <src_port> | protocol <=== Matches based on L4 src_port, L4 dst_port or protocol.