Guest

Cisco Adaptive Security Appliance (ASA) Software

Wireless Mobility Connections Fail and Do Not Recover When ASA is Rebooted

Techzone Article content

Document ID: 116074

Updated: Apr 04, 2013

Contributed by Jay Johnston, Cisco TAC Engineer.

   Print

Introduction

This document describes a problem where a mobility path connection (using User Datagram Protocol (UDP) and IP protocol 93) that traverses an Adaptive Security Appliance (ASA) might go down and continue to fail until the mobility devices are reloaded, or the mobility path traffic is stopped and left inactive for a short time and then restarted.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

  • Cisco Adaptive Security Appliance (ASA)
  • Wireless LAN Controller (WLC)

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for information on document conventions.

Problem

In this situation a Wireless LAN Controller (WLC) at 10.10.1.2 attempts to communicate with the WLC at 10.10.9.3, but the communication fails.

This problem can be triggered by any of these events:

  • The ASA is rebooted.
  • The routing table is modified by an administrator or routing protocol.
  • An interface is shut down, then brought back up by the administrator.

Besides mobility traffic, this problem might be experienced for any UDP or non-TCP IP protocols. 

This problem is not a bug but a consequence of the network topology and ASA configuration. See below for the cause and solution to this problem.

Sample Network Topology


116074-problem-solution-asa-01.png

ASA routing configuration:

!
route outside 0.0.0.0 0.0.0.0 192.168.4.3 1
route inside 10.0.0.0 255.0.0.0 192.168.254.1 1
!
same-security-traffic permit intra-interface
!

ASA dmz interface configuration:

!
interface Gigabit-Ethernet0/1.10
vlan 10
nameif dmz
security-level 75
ip address 10.10.9.1 255.255.255.240 standby 10.10.9.2
!

Problem Trigger

The problem is triggered when the WLC at 10.10.1.2 sends traffic destined to the WLC at 10.10.9.3. These packets cause the ASA to build a connection in its connection table that sends the mobility traffic out the wrong ASA interface (inside).

116074-problem-solution-asa-02.png

This issue is caused by the destination interface "dmz" of the ASA being in the down/down state at the time the connection was built, which results in the connection being built out a different, non-optimal interface. The dmz interface might be down due to a cable problem, an ethernet or port-channel negotiation issue, or it might be administratively shut down.

At the time of the problem, the mobility path connections can be seen as being created as "intra-interface" of the ASA, which is routing the packets back out the same inside interface they arrived on:

ASA# show conn address 10.10.1.2
15579 in use, 133142 most used
97 inside 10.10.9.3 inside 10.10.1.2, idle 0:00:00, bytes 32210
UDP inside 10.10.9.3:16666 inside 10.10.1.2:16666, idle 0:00:00, bytes 4338, flags -
97 inside 10.10.9.3 inside 10.10.1.2, idle 0:00:00, bytes 157240
ASA#

The mobility endpoint at 10.10.1.2 continues to send traffic destined to 10.10.9.3, which matches these existing connections. Even if the dmz interface were to progress to the up/up state, mobility traffic sourced from 10.10.1.2 would match the existing connections in the table (instead of building a new connection to the dmz interface) which resets the timeout of the connections on the ASA, which prolongs the problem.

In summary, these events can trigger the issue:

  1. The device at 10.10.1.2 sends a protocol 97 or UDP packet to 10.10.9.3.
  2. The ASA receives the packet on the inside interface, but the dmz interface is down, which results in the more specific route to the destination network missing from the routing table. Since the same-security permit intra-interface command is enabled on the ASA, it follows a static route configured for the 10.0.0.0/8 network back through the inside interface, builds a connection in the connection table, and then sends the packet back out the inside interface towards the internal network.
  3. At some point the dmz interface might come back up and the route is added back to the table; however, since the connection for the protocol 97 traffic was already built in step #2, subsequent packets will match the connection and the routing table is overwritten, and the traffic does not reach the server on the dmz.

Solution

Solution 1

One possible solution for this issue is to remove the same-security permit intra-interface command from the ASA. This solution prevents the u-turn connection from being built back out the same interface on which the original packet was received, which allows the correct connection to be built when the interface comes up. However, depending on the routing table of the ASA, this solution might not work (the traffic might be routed to another interface other than the intended destination based on the routing table), and the same-security permit intra-interface command might be necessary for other connections on the ASA.

Solution 2 

For this specific instance, the problem was successfully mitigated by enabling the timeout floating-conn feature. This feature, which is not enabled by default, caused the ASA to tear down these connections one minute after a more preferred route to one of the endpoints is added to the routing table out a new interface of the ASA, which occurs when the dmz interface comes up. The connections are then immediately rebuilt when the next packet arrives at the ASA, using the more preferred interface (dmz, instead of inside for the 10.10.9.3 host).

ASA(config)# timeout floating-conn 0:01:00

116074-problem-solution-asa-03.png

When the problem is mitigated, the correct connections are built in the ASA connection table and connectivity is automatically restored:

ASA# show conn address 10.10.1.2
15329 in use, 133142 most used
97 dmz 10.10.9.3 inside10.10.1.2, idle 0:00:00, bytes 3175742510
UDP dmz 10.10.9.3:16666 inside 10.10.1.2:16666, idle 0:00:00, bytes 40651338, flags -
97 dmz 10.10.9.3 inside10.10.1.2, idle 0:00:00, bytes 1593457240
ASA#

Related Information

Updated: Apr 04, 2013
Document ID: 116074