This document describes how to perform datapath packet tracing for Cisco IOS®-XE software via the Packet Trace feature.
In order to identify issues such as misconfiguration, capacity overload, or even the ordinary software bug while troubleshooting, it is necessary to understand what happens to a packet within a system. The Cisco IOS-XE Packet Trace feature addresses this need. It provides a field-safe method that is used for accounting and in order to capture per-packet processing details based on a class of user-defined conditions.
Cisco recommends that you have knowledge of the packet trace feature that is available in Cisco IOS-XE Versions 3.10 and later, as well as in all platforms that run the Cisco IOS-XE software, such as the Cisco 1000 Series Aggregation Services Routers (ASR1K), the Cisco 1000V Series Cloud Services Router (CSR1000v), and the Cisco 4451-X Series Integrated Services Router (ISR4451-X).
The information in this document is based on these software and hardware versions:
Cisco IOS-XE Software Release 3.10S (15.3(3)S) and later
The information in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure you understand the potential impact of any command used.
This diagram illustrates the topology that is used for the examples that are described in this document:
Packet Tracing in Use
In order to illustrate the use of the packet trace feature, the example that is used throughout this section describes a trace of the Internet Control Message Protocol (ICMP) traffic from the local workstation 172.16.10.2 (behind the ASR1K) to the remote host 172.16.20.2 (the ingress direction for the ASR1K at the Gig0/0/1 interface).
You can trace packets on the ASR1K with these two steps:
Enable the platform conditional debugs in order to select the packets or traffic that you want to trace on the ASR1K.
Enable the platform packet trace (path-trace or Feature Invocation Array (FIA) trace).
Quick Start Guide
Here is a quick start guide if you are already familiar with the contents of this document, and want a section for a quick look at the CLI. These are only a few examples to illustrate the use of the tool. Refer to the later sections that discuss the syntaxes in detail, and ensure you use the configuration that is appropriate to your requirement.
Configure platform conditions:
debug platform condition ipv4 10.0.0.1/32 both --> matches in and out packets with source or destination as 10.0.0.1/32
debug platform condition ipv4 access-list 198 egress --> (Ensure access-list 198 is defined prior to configuring this command) - matches egress packets corresponding to access-list 198
debug platform condition interface gig 0/0/0 ingress --> matches all ingress packets on interface gig 0/0/0
debug platform condition mpls 10 1 ingress --> matches MPLS packets with top ingress label 10
debug platform condition ingress --> matches all ingress packets on all interfaces (use cautiously)
After a platform condition is configured, start platform conditions with this CLI command:
debug platform condition start
Configure packet trace:
debug platform packet-trace packet 1024 -> basic path-trace, and automatically stops tracing packets after 1024 packets. You can use "circular" option if needed
debug platform packet-trace packet 1024 fia-trace -> enables detailed fia trace, stops tracing packets after 1024 packets
debug platform packet-trace drop [code <dropcode>] -> if you want to trace/capture only packets that are dropped. Refer to Drop Trace section for more details.
Note: In earlier Cisco IOS-XE 3.x releases, the command debug platform packet-trace enable is also required to start the packet-trace feature. This is no longer required in Cisco IOS-XE 16.x releases.
Enter this command in order to clear the trace buffer and reset packet-trace:
clear platform packet-trace statistics --> clear the packet trace buffer
The command to clear both platform conditions and the packet trace configuration is:
clear platform condition all --> clears both platform conditions and the packet trace configuration
Verify the platform condition and packet trace configuration after you apply the previous commands in order to ensure you have what you need.
show platform conditions --> shows the platform conditions configured
show platform packet-trace configuration --> shows the packet-trace configurations
show debugging --> this will show both platform conditions and platform packet-trace configured
Here are the commands to check the traced/captured packets:
show platform packet-trace statistics --> statistics of packets traced
show platform packet-trace summary --> summary of all the packets traced, with input and output interfaces, processing result and reason.
show platform packet-trace packet 12 -> Tracing the 12th packet, with complete path trace or FIA trace details.
Enable Platform Conditional Debugs
The Packet Trace feature relies on the conditional debug infrastructure in order to determine the packets to be traced. The conditional debug infrastructure provides the ability to filter traffic based on:
IP address and mask
Access Control List (ACL)
Traffic direction (ingress or egress)
These conditions define where and when the filters are applied to a packet.
For the traffic that is used in this example, enable platform conditional debugs in the ingress direction for ICMP packets from 172.16.10.2 to 172.16.20.2. In other words, select the traffic that you want to trace. There are various options that you can use in order to select this traffic.
ASR1000#debug platform condition ? egress Egress only debug feature For a specific feature ingress Ingress only debug interface Set interface for conditional debug ipv4 Debug IPv4 conditions ipv6 Debug IPv6 conditions start Start conditional debug stop Stop conditional debug
In this example, an access-list is used in order to define the condition, as shown here:
Note: Enter the clear platform condition all command in order to clear all of the platform debug conditions and the packet trace configurations and data.
In summary, this configuration data has been used thus far in order to enable packet-trace:
debug platform packet-trace packet 16
Egress Condition Limitation with Packet Traces
The conditions define the conditional filters and when they are applied to a packet. For example, debug platform condition interface g0/0/0 egress means that a packet is identified as a match when it reaches the output FIA on interface g0/0/0, so any packet processing that takes place from ingress until that point is missed.
Note: Cisco highly recommends that you use ingress conditions for packet traces in order to get the most complete and meaningful data possible. The egress conditions can be used, but be aware of the limitations.
Display the Packet Trace Results
Note: This section assumes that path-trace is enabled.
Three specific levels of inspection are provided by the packet trace:
Per-packet path data
When five ICMP request packets are sent from 172.16.10.2 to 172.16.20.2, these commands can be used in order to view the packet trace results:
ASR1000#show platform packet-trace statistics Packets Traced: 5 Ingress 5 Inject 0 Forward 5 Punt 0 Drop 0 Consume 0
Note: The third command provides an example that illustrates how to view the packet trace for each packet. In this example, the first packet traced is shown.
From these outputs, you can see that five packets are traced and that you can view the input interface, the output interface, the state, and the path trace.
The packet is scheduled/queued for delivery, to be forwarded to next hop via an egress interface.
The packet is punted from the Forwarding Processor (FP) to the Route Processor (RP) (control plane).
The packet is dropped on the FP. Run FIA trace, use global drop counters, or use datapath debugs in order to find more details for drop reasons.
The packet is consumed during a packet process, such as during the ICMP ping request or the crypto packets.
The ingress and inject counters in the packet trace statistics output correspond to the packets that enter via an external interface and packets that are seen as injected from the control plane, respectively.
The FIA holds the list of features that are executed sequentially by the Packet Processor Engines (PPE) in the Quantum Flow Processor (QFP) when a packet is forwarded either ingress or egress. The features are based on the configuration data that is applied on the machine. Thus, a FIA trace helps to understand the flow of the packet through the system as the packet is processed.
You must apply this configuration data in order to enable packet trace with FIA:
Note: This section assumes that FIA trace is enabled. Also, when you add or modify the current packet trace commands, the buffered packet trace details are cleared, so you must send some traffic again so that you can trace it.
Send five ICMP packets from 172.16.10.2 to 172.16.20.2 after you enter the command that is used in order to enable the FIA trace, as described in the previous section.
When you enable the platform conditional debugs, it is added to the FIA as a feature. Depending upon the location that it is added to the list, you might need to adjust your platform conditions, such as when you trace pre-encap and post-encap packets.
This output shows the order of the features in the FIA for the platform conditional debugging that is enabled in the ingress direction:
ASR1000#show platform hardware qfp active interface if-name GigabitEthernet 0/0/1
QfpEth Physical Information DPS Addr: 0x11215eb8 Submap Table Addr: 0x00000000 VLAN Ethertype: 0x8100 QOS Mode: Per Link
Note: The CBUG_INPUT_FIA and the DEBUG_COND_INPUT_PKT correspond to the conditional debug features that are configured on the router.
Dump the Traced Packets
You can copy and dump the packets as they are traced, as this section describes. This example shows how to copy a maximum of 2,048 bytes of the packets in the ingress direction (172.16.10.2 to 172.16.20.2).
Drop trace is available in Cisco IOS-XE Software Release 3.11 and later. It enables packet trace only for dropped packets. Here are some highlights of the feature:
It optionally allows you to specify the retention of packets for a specific drop code.
It can be used without global or interface conditions in order to capture drop events.
A drop event capture means that only the drop itself is traced, not the life of the packet. However, it still allows you to capture summary data, tuple data, and the packet in order to help refine conditions or provide clues to the next debug step.
Here is the command syntax that is used in order to enable drop-type packet traces:
debug platform packet-trace drop [code <code-num>]
The drop code is the same as the drop ID, as reported in the show platform hardware qfp active statistics drop detail command output:
ASR1000#show platform hardware qfp active statistics drop detail -------------------------------------------------------------------------------- ID Global Drop Stats Packets Octets -------------------------------------------------------------------------------- 60 IpTtlExceeded 3 126 8 Ipv4Acl 32 3432
Example Drop Trace Scenario
Apply this ACL on the Gig 0/0/0 interface of the ASR1K in order to drop traffic from 172.16.10.2 to 172.16.20.2:
access-list 199 deny ip host 172.16.10.2 host 172.16.20.2 access-list 199 permit ip any any interface Gig 0/0/0 ip access-group 199 out
With the ACL in place, which drops the traffic from the local host to the remote host, apply this drop-trace configuration:
The inject and punt packet trace feature was added in Cisco IOS-XE Software Release 3.12 and later in order to trace punt (packets that are received on the FP that are punted to the control plane) and inject (packets that are injected to the FP from the control plane) packets.
Note: The punt trace can work without the global or interface conditions, just like a drop trace. However, the conditions must be defined for an inject trace to work.
Here is an example of a punt and inject packet trace when you ping from the ASR1K to an adjacent router:
ASR1000#debug platform condition ipv4 172.16.10.2/32 both ASR1000#debug platform condition start ASR1000#debug platform packet-trace punt ASR1000#debug platform packet-trace inject ASR1000#debug platform packet-trace packet 16 ASR1000# ASR1000#ping 172.16.10.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 172.16.10.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 14/14/15 ms ASR1000#
Now you can verify the punt and inject trace results:
With this example, a site-to-site VPN tunnel is used between the ASR1K and the Cisco IOS router in order to protect the traffic that flows between 172.16.10.0/24 and 172.16.20.0/24 (local and remote subnets).
Here is the platform condition and packet trace configuration that is used in order to trace the VPN traffic that flows from 172.16.10.2 to 172.16.20.2 on the Gig 0/0/1 interface:
When five ICMP packets are sent from 172.16.10.2 to 172.16.20.2, which are encrypted by the VPN tunnel between the ASR1K and the Cisco IOS router in this example, these are the packet trace outputs:
Note: The packet traces show the QFP Security Association (SA) handle in the trace that is used in order to encrypt the packet, which is useful when you troubleshoot IPsec VPN issues in order to verify that the correct SA is used for encryption.
Packet trace buffers consume QFP DRAM, so be mindful of the amount of memory that a configuration requires and the amount of memory that is available.
The performance impact varies, dependent upon the packet trace options that are enabled. The packet trace only affects the forwarding performance of the packets that are traced, such as those packets that match the user-configured conditions. The more granular and detailed information that you configure the packet trace to capture, the greater it will impact resources.
As with any troubleshooting, it is best to take an iterative approach and only enable the more-detailed trace options when a debug situation warrants it.
The QFP DRAM usage can be estimated with this formula:
memory needed = (stats overhead) + num of pkts * (summary size + path data size + copy size)
Note: Where the stats overhead and summary size are fixed at 2 KB and 128 B, respectively, the path data size and copy size are user-configurable.