Cisco Nexus MTU troubleshooting counters
Available Languages
Contents
Introduction
This document describes about the different MTU configuration and also discusses scenarios that involves the behavior with different combinations and padding.
Background
Fragmentation takes place in L3 path not at L2
Padding is basically used to make sure that the IP packet header has a length that is a multiple of 32 bits
Prerequisite
IP fragmentation and reassembly
Information about Padding
- Sender[initiator] will perform the padding, intermittent[transit] devices will not perform padding
- Padding should not get modified when a packet goes through cut-through switch
- Switch will consider the packet as undersize frame if originator is not capable of doing the padding
- Wireshark capture will takes place before Padding
- Basically switch will add extra bytes even if the packet size it is about to send to the wire is less than 64B
- When a 64 byte 802.1q tagged Ethernet framed is received over a trunk port on a L2/L3 and is routed/forwarded to an untagged access port the 802.1q tag is reduced and frame size reduced by 4 bytes
- During the process of untagging a frame the frame no longer meets the 64 byte minimum MTU as specified in the IEEE 802.1q spec the switch should pad the frame back to 64 bytes
Fragmentation and MTU mis-match
- If Path is L3, fragmentation takes place, packet will not be dropped.
- If Path is L2, no fragmentation takes place, packet will be dropped completely
- Initiate [ICMP] with packet-size 1540B & has L2 in path still the you dont see the drops, where total size becomes 1568 [1540+20+8]
- Initiate [ICMP with ]packet-size 1541B, total packet becomes 1569, and you see the drops, and drops are seen as Giants counters
- If MTU-mismatch following counters increment - Jumbo, Giants, Runt etc. on the basis of scenario and configuration.
Topology
9K = MTU 9K [Jumbo]
1.5K = MTU 1.5K + configured as L2
Above lab topology has been divided in multiple scenarios as follows:
MTU troubleshooting using ping test
Ping with packet size 1500
Initiated ping and succssefull with out any ping drop.
Even-though we have L2, we dont see the drop as the ping size it took is default one 1500.
N5K-1# ping 10.1.1.2 count 10
PING 10.1.1.2 (10.1.1.2): 56 data bytes
64 bytes from 10.1.1.2: icmp_seq=0 ttl=254 time=3.228 ms
64 bytes from 10.1.1.2: icmp_seq=1 ttl=254 time=4.832 ms
Ping with packet size 5000
Initiated ping with the packet-size 5000 with packet count 50 from N5k1 to Nexus-Sw2 and dropped at transit L2
N5K-1# ping 10.1.1.2 packet-size 5000 count 50
PING 10.1.1.2 (10.1.1.2): 5000 data bytes
Request 0 timed out
Request 1 timed out
Packet considered as Jumbo at ingress of Nexus-sw1
Nexus-Sw1# sh interface ethernet 3/3 | i MTU|jumbo
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec
50 jumbo packets 0 storm suppression packets >>>>>>> exact 50 jumbo packets are seeing in the RX counter.
Packet considered as Jumbo at Egress of Nexus-sw1
Nexus-Sw1# sh interface ethernet 3/1 | i MTU|jumbo >>>>>>> Intertace connected towards to N7k2 with MTU 1500
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
50 jumbo packets >>>>>>> Exact 50 jumbo packets are egress in the TX.
Packet dropped at ingress of Nexus-sw2
Nexus-Sw2# sh interface et3/1 | i MTU|giant >>>>>>> Interface connected towards Nexus-Sw1 with e3/1 MTU 1500
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
0 runts 50 giants 0 CRC/FCS 0 no buffer >>>>>>> Exact 50 input error and 50 Giants packets observed in the RX counter.
50 input error 0 short frame 0 overrun 0 underrun 0 ignored
Nexus-Sw2# sh interface et3/4 | i MTU|giant|error >>>>>>> Interface with MTU 1500
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec
0 runts 0 giants 0 CRC/FCS 0 no buffer >>>>>>> No counter seen
0 output error 0 collision 0 deferred 0 late collision >>>>>>> No counter seen
Feedback