Introduction
This document describes how to troubleshoot Cisco Adaptive Security Appliance (ASA) throughput and connection speed issues.
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
The information in this document is based on the Cisco Adaptive Security Appliance (ASA).
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Background Information
Some customers might experience a problem when they first deploy an ASA or when they test new connectivity. The problem is the TCP throughput for connections that flow through the ASA is much lower than when the ASA is not in the connection path (or the connections are much slower than before the ASA was implemented in the network).
For example, a customer might replace a low-end D-Link router (or other routing device) with an ASA 5505 or an ASA 5510; however, once the router is replaced, connection speed is greatly reduced. Customer might raise a case with Cisco TAC because they believe the ASA caused the reduction in connection speed.
Troubleshooting Methodology
TCP flows slow down when there is packet loss or packet delay on the network. In order to understand the exact cause of the problem, the data must show the actual TCP packets on the wire for that connection and how the network might affect them. Usually a network administrator is alerted to the problem when they perform a specific action, such as an FTP file transfer or an online speed test. Most often the problem can be reproduced. Therefore, the administrator can gather the required data in order to find the root cause.
In order to gather the required data, the show tech command should be run from the ASA before and after the test. This command shows configuration and packet statistics (mainly from show service-policy) and also shows if the interface errors increment.
Bi-directional, simultaneous packet captures (taken from the two ASA interfaces affected that the connection traverses) are required to fully diagnose the cause of the issue.
Refer to these documents for examples of how to apply packet captures to the ASA:
Data Analysis
Once you gather the required data, you can use the packet captures in order to determine which of these issues might have occurred:
- The packets from the outside host are dropped or delayed before they reach the ASA's outside interface.
- The packets are delayed or dropped by the ASA.
- The packets are delayed or dropped somewhere on the inside network.
Note: This analysis assumes the data is sent from a host on the outside interface to a host on the inside interface.
This video shows an example of how to perform the analysis on a packet capture:
TCP stream coalescing is a technical consideration specific to this problem because, when you engage certain features on the ASA, the firewall fully coalesces the TCP stream that passes through it.
For example, if the ASA discovers a missing packet on the network (since it is not received at the ASA), it sends an ACK on behalf of the other TCP endpoint for the missing data. This scenario is most common. If the ASA discovers packets that arrive out of order, the ASA reorders the packets and passes them to the receiver in the proper order. If there are no network drops or packet reordering, there are no side effects to enabling this feature. If all the packets sent by either TCP endpoint successfully passed through the network and the ASA, you would not know this feature is enabled since it does not take action on the packet flows. Only when there is trouble with the TCP connection on the network will enabling this feature further slow down network traffic. The act of coalescing the TCP stream is very resource intensive for the ASA. For every packet dropped on the network the ASA must not only send a TCP packet request the retransmission of that packet, but it must also buffer the packets that the sender continued to send after the packet went missing.
Common Problems
Misconfigured Speed and Duplex Values on Interface that Connects ASA to Adjacent Device
This issue often occurs when a device is replaced by an ASA. If the speed and duplex values on the ASA interface are not the same as the values on the adjacent device, packet drops occur on that interface. Check the speed and duplex values on the ASA interface as well as the adjacent interface.
Check the show interface output of the ASA for obvious errors that are symptoms of this problem:
Interface Ethernet0/0 "Outside", is up, line protocol is up
Hardware is i82546GB rev03, BW 100 Mbps
Auto-Duplex(Half-duplex), Auto-Speed(100 Mbps)
MAC address 0019.2f58.c324, MTU 1500
IP address 192.168.222.122, subnet mask 255.255.255.252
124047996 packets input, 35340918453 bytes, 0 no buffer
Received 3 broadcasts, 0 runts, 0 giants
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 L2 decode drops
156918660 packets output, 40931551514 bytes, 0 underruns
1 output errors, 4286634 collisions, 0 interface resets
0 babbles, 123332 late collisions, 4752834 deferred
0 lost carrier, 0 no carrier
input queue (curr/max blocks): hardware (0/0) software (0/0)
output queue (curr/max blocks): hardware (0/245) software (0/0)
Traffic Statistics for "Outside":
124047995 packets input, 33107957301 bytes
157041993 packets output, 38195084709 bytes
103480 packets dropped
1 minute input rate 2140 pkts/sec, 477200 bytes/sec
1 minute output rate 2630 pkts/sec, 396763 bytes/sec
1 minute drop rate, 0 pkts/sec
5 minute input rate 2152 pkts/sec, 525496 bytes/sec
5 minute output rate 2701 pkts/sec, 421215 bytes/sec
5 minute drop rate, 0 pkts/sec
Send Traffic to IPS Module
When the ASA is configured to send traffic to the IPS module, the TCP stream coalescing feature is engaged on the ASA. Refer to the Data Analysis section of this document for more information on the TCP stream coalescing feature.
ASA Modification of TCP MSS Option Causes Slight Performance Decrease
By default the ASA sets the TCP MSS option in the SYN packets to 1380. Therefore, TCP endpoints should not transmit a TCP segment larger than 1380 bytes. This value is lower than the often default value of 1460 bytes and represents a TCP performance drop of around six percent (6%). Performance might improve is you increase the maximum MSS setting on the ASA or disable the MSS adjustment. Before you modify the default command on the ASA, understand the risks involved with regard to potential fragmentation if the packet is further encapsulated in the path somewhere.
For more information, refer to the sysopt connection tcpmss section of the Cisco ASA 5500 Series Command Reference.
Related Information
Feedback