Troubleshoot Packet Drops Related to SCTP Max MTU Size

Introduction

This document describes SCTP fragmentation and chunk bundling mechanisms in the Cisco MME and how fragmentation and bundling affect packet drops.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on these software and hardware versions:

• QVPC-SI software version 21.28.m18

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Stream Control Transmission Protocol (SCTP) is a reliable transport protocol operating on top of a connectionless packet network such as IP.  It can fragment data to conform to discovered path MTU size and bundle multiple user messages into a single SCTP packet by using chunk bundling. This document provides two methods to remedy packet drops that occur due to path MTU limitations.

A case study displays fragmentation and bundling mechanisms and a means to demonstrate that packet drops, including packet drops that occur outside of the Cisco MME, rather than due to the configuration of SCTP Max MTU size.

Per RFC 4960 (Stream Control Transmission Protocol) 6.9 (Fragmentation and Reassembly):

"If an endpoint supports fragmentation, it MUST fragment a user message if the size of the user message to be sent causes the outbound SCTP packet size to exceed the current MTU."

In addition, multiple SCTP data chunks can be bundled into one outbound SCTP packet. The total size of the resultant IP datagram, including the SCTP packet and IP headers, MUST be less that or equal to the current Path MTU. On the MME the Path MTU is configured on the IP interface that is processing these packets.

Fragmentation

There can be multiple streams in a SCTP association, each identified by a Stream Identifier (SI). Stream Sequence Number (SSN) identifies the order for each fragment within a specific SCTP stream. Multiple sessions can use the same SCTP stream. The Transmission Sequence Number (TSN) identifies the order for each fragment within the entire SCTP association.

When fragmented, the SCTP Payload data chunk indicates in the B and E fields whether the fragment is the beginning fragment, a middle fragment, or the end fragment: 

0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   Type = 0    | Reserved|U|B|E|    Length                     |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                              TSN                              |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |      Stream Identifier S      |   Stream Sequence Number n    |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                  Payload Protocol Identifier                  |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       \                                                               \
       /                 User Data (seq n of Stream S)                 /
       \                                                               \
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               B E                  Description
            ============================================================
            |  1 0 | First piece of a fragmented user message          |
            +----------------------------------------------------------+
            |  0 0 | Middle piece of a fragmented user message         |
            +----------------------------------------------------------+
            |  0 1 | Last piece of a fragmented user message           |
            +----------------------------------------------------------+
            |  1 1 | Unfragmented message                              |
            ============================================================
            |             Table 1: Fragment Description Flags          |
            ============================================================

Gaps in Stream Sequence Numbers can indicate that fragments within the steam or association are being dropped. Wirshark filters can help to identify gaps in SCTP streams:

sctp.sack_gap_block_start or sctp.sack_gap_block_end

SCTP Chunk Bundling

Multiple chunks can be bundled into one SCTP packet up to the MTU size, except for the INIT, INIT ACK, and SHUTDOWN COMPLETE chunks:

   An SCTP packet is composed of a common header and chunks.  A chunk
   contains either control information or user data.

   The SCTP packet format is shown below:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                        Common Header                          |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                          Chunk #1                             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                           ...                                 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                          Chunk #n                             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The chunk length for each SCTP Data Chunk is listed within each data chunk:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   Chunk Type  | Chunk  Flags  |        Chunk Length           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       \                                                               \
       /                          Chunk Value                          /
       \                                                               \
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Configurations Impacting Sizing

SCTP-Param-Template SCTP MTU Sizes

In the SCTP-Param-Template the SCTP-Max-MTU-Size is configured by use of the sctp-max-mtu-size option and/or the sctp-start-mtu-size option, configured in bytes. 1500 bytes is the default SCTP Max MTU Size and default SCTP Start MTU Size:

Exec > Global Configuration > SCTP Parameter Template Configuration
[Local]ASR5500-2# sctp-param-template template_name
[Local]ASR5500-2(sctp-param-template)# sctp-max-mtu-size 1500
[Local]ASR5500-2(sctp-param-template)# sctp-start-mtu-size 1500

In the MME Service the SCTP-Param-Template is associated to the MME Context:

Exec > Global Configuration > Context Configuration > MME Service
[mme]ASR5500-2(config-mme-service)# associate sctp-param-template S1_MME_SCTP

MME Context Ethernet Interface Configuration

The interface to be used by the MME Service is configured along with the desired IP MTU size in bytes (1500 is default):

Exec > Global Configuration > MME Context Configuration
[mme]ASR5500-2(config-ctx)# interface int1/10_s1mme
[mme]ASR5500-2(config-if-eth)# ip address <IP ADDRESS> <SUBNET MASK>
[mme]ASR5500-2(config-if-eth)# ip mtu 2000
[mme]ASR5500-2(config-if-eth)# exit

S1AP UE Radio Capability IE Size

In the MME Serice configuration, S1AP UE Radio Capability IE message sizes are configured. The default value is 9000 bytes:

[context_name]host_name(config-mme-service)# s1-mme ue-radio-cap size 9000

Case Study

Here is an example of a fragmented INITIAL CONTEXT SETUP REQUEST/UE CAPABILITY INFORMATION message, which SCTP fragments to meet the configured SCTP Max MTU Size.

In subscriber traces the ICMP message "Fragmentation needed" is returned from the transport router toward the MME after the transmission of the SCTP fragmented INTIAL CONTEXT SETUP REQUEST message.

In Frame 5, multiple SCTP packets (2) are clubbed together under one IP packet:

The first data chunk is the Last Segment of that fragmented message, as signified by the 1 mark in the E-Bit information element.

The second data chunk is not part of a fragmented message, as by the B-Bit and E-Bit information elements are marked with a number 1:

The first SCTP data chunk has 1383 bytes in length. The second SCTP data chunk is 89 bytes in length, so each individual SCTP packet does not exceed the configured SCTP-Max-MTU-Size of 1500 bytes:

Because the total size of the resultant IP datagram, including the SCTP packet and IP headers is smaller than the configured IP MTU size of 2000, these SCTP chunks grouped together within one IP packet at the IP stack level on the MME.

Solution

S1AP UE Radio Capability IE message sizes are allowed in the configuration at the default value of 9000 bytes. Fragmentation must occur at the SCTP layer to carry these messages at a level smaller than the configured SCTP Max MTU size of 1500.

sctp-max-mtu-size being 1500 means any SCTP packet transmitted does not exceed 1500 bytes.

The IP interface MTU size being 2000 means that any IP packet cannot exceed 2000 bytes.

One IP packet can have multiple SCTP packets and each SCTP packet can have multiple chunks. So long as the individual SCTP packet size adheres to SCTP MTU size and total size of all SCTP packets combined together adheres to IP MTU size, the node is behaving as expected.

Packet drops are not occurring within the node, if:

• The total size of SCTP chunks within packet trace frames are less than the configured value of SCTP-Max-MTU-Size in the SCTP-Param-Template and
• The total value of the SCTP PDU + the IP header are less than the configured interface MTU size on the node.

In our case study example, investigation into the transport router revealed that the transport router had an IP MTU size of 1500. Since transport node in between has an MTU of 1500, the packets were dropped by the IP interface on the transport node, promptimg the transport router to send a message to the MME indicating "Destination Unreachable".

Workarounds to Resolve Drops

Workaround #1: Reduce the SCTP Max MTU Size

Reduce the SCTP MTU size by configuring sctp-max-mtu-size to lower values until you do not see the drops.

Note: When lowering the sctp-max-mtu-size, also ensure sctp-start-mtu-size is configured to a value less than or equal to the sctp-max-mtu-size.

Example: Modify the sctp-param-template S1_MME_SCTP (sctp-start-mtu-size from 1500 to 1460),

Workaround #2: Increase the Transport Node MTU Size to Greater Than 1500

Related Information

• RFC 4960 Stream Control Transmission Protocol

• MME Administration Guide, StarOS Release 21.28 Chapter: UE Radio Capability IE Size
• Command Line Interface Reference, Modes E - F, StarOS Release 21.28 Chapter: Ethernet Interface Configuration Mode Commands IP MTU Size
• Command Line Interface Reference, Modes E - F, StarOS, Chapter: Ethernet Interface Configuration Mode Commands: ip mtu
• SCTP Parameter Template Configuration Mode Commands: sctp-max-mtu-size
• Cisco TechNote - Configure StarOS MTU Interface, APN, and Local Subscribers