Contents

Introduction

Background Information

Configuration

Troubleshoot

Bugs

Introduction

This document describes how to troubleshoot High CPU/memory due to  Extensible Authentication Protocol (EAP) framework and Authentication, Authorization, and Accounting (AAA) manager. This is seen on switches that use dot1x/mab authentication.

Background Information

The Cisco IOS Auth Manager handles network authentication requests and enforces authorization policies regardless of the authentication method. The Auth Manager maintains operational data for all port-based network connection attempts, authentications, authorizations, and disconnections and, serves as a session manager.

The switch acts as an intermediary (proxy) between the client and the authentication server, it requests identity information from the client, verifies that information with the authentication server, and relays a response to the client. The switch includes the RADIUS client, which encapsulates and decapsulates the EAP frames and interacts with the authentication server.

Configuration

This section shows a Cisco switch that does MAB/DOT1X (MAC AuthenticationBypass) authentication.

You should understand the concepts of port-based network access control and have an understanding of how to configure port-based network access control on your Cisco platform. This image illustrates workstations that have dot1x/MAB authentication.

This is of a sample configuration:

interface FastEthernet0/8

switchport access vlan 23

switchport mode access

switchport voice vlan 42

authentication host-mode multi-domain

authentication order mab dot1x

authentication priority mab dot1x---> Priority order

authentication port-control auto

authentication periodic

authentication timer reauthenticate <value in sec>---->(Time after which the client auth would be re-negotiated)

authentication violation protect

mab

mls qos trust dscp

dot1x pae authenticator

dot1x timeout tx-period 3

storm-control broadcast level 2.00

no cdp enable

spanning-tree portfast

spanning-tree bpduguard enable

service-policy input Marking

end

Troubleshoot

Switches that use dot1x/MAB authentication sometimes have high CPU/memory spikes due to the EAP Framework and AAA manager. This can impact the production since authentication requests are dropped.

In order to resolve this, these steps are recommended:

Step 1. Enter the show proc cpu sort command in order to check the high CPU usage on the switch and make sure that the EAP Framework and Auth manager processes have the highest usage as shown in this example:

PU utilization for five seconds:

97%

/2%; one minute: 90%; five minutes: 89%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

149   178566915 140683416       1269

64.04% 47.11% 45.63%   0 EAP Framework

141   130564594  55418491       2355

21.61% 29.05% 29.59%   0 Auth Manager

121   305295906 487695245        519  1.74%  1.84%  1.78%   0 Hulc LED Process

144    12070918  31365536        384  0.63%  0.43%  0.49%   0 MAB Framework

258   117344878 885817567        132  0.47%  0.79%  0.86%   0 RADIUS

Step 2. Check the memory usage on the switch for processes like Auth Manager and RADIUS with the show process cpu memory command as shown in this example.

Processor Pool Total:   22559064 Used:   16485936 Free:    6073128

I/O Pool Total:    4194304 Used:    2439944 Free:    1754360

Driver te Pool Total:    1048576 Used:         40 Free:    1048536

PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process

0   0   29936164   13273256   13856236          0          0 *Init*

0   0   34797632   32603736    1091560    2481468     263240 *Dead*

59   0     366860       6760     317940          0          0 Stack Mgr Notifi

141   0

569580564 3357129696

174176    2986956

0

Auth Manager

258   0

1212276148 2456764884     140684   21066696

0

RADIUS

131   0  552345134 541235441      90736      20304          0 HRPC qos reque

Step 3. If you face high resource usage on the switch, you might see the following logs for the authentication failures as shown:

Enter the show logging command.

%DOT1X-5-FAIL: Authentication failed for client (7446.a04b.1495) on Interface Fa0/17 AuditSessionID 0A73340200000224870C28AA

%AUTHMGR-7-RESULT:

Authentication result 'no-response'

from 'dot1x' for client (7446.a04b.1495) on Interface Fa0/17 AuditSessionID 0A73340200000224870C28AA

%AUTHMGR-7-FAILOVER: Failing over from 'dot1x' for client (7446.a04b.1495) on Interface Fa0/17 AuditSessionID 0A73340200000224870C28AA

Step 4. Set the re-authenticate timer to a higher value (for example, 3600 seconds) in order to ensure that you do not authenticate frequently for the clients, which thereby increases the load on the switch.

In order to validate the configuration enter show run interface <interface-name> command:

interface FastEthernet0/8

switchport access vlan 23

switchport mode access

switchport voice vlan 42

authentication host-mode multi-domain

authentication order mab dot1x

authentication priority mab dot1x

authentication port-control auto

authentication periodic

authentication timer reauthenticate 60---------->Make sure we do not have any

aggressive timers set

authentication violation protect

Step 5. Determine how many sessions are seen for MAB/dot1x processes, because sometimes a high number of authenticated sessions can also lead to high CPU. In order to check the number of active sessions, enter these commands:

SW#

show authentication registrations

Auth Methods registered with the Auth Manager:

Handle  Priority  Name

100       0      dot1x

3        1      mab

1        2      webauth

SW#Show authentication method dot1x

SW#Show authentication method mab

SW#Show authentication sessions

Step 6. In order to check the version and potential bugs, enter the show version command.

If the bug is not listed in the "Bugs" section, open a case with the Technical Assistance Center (TAC) and attach all of the logs from steps 1 to 5.

Bugs

CSCus46997 Memory Leak and High CPU in IP Host Track and Auth Manager

CSCtz06177 A catalyst 2960 may run low on memory.

CSCty49762 EAP Framework and AAA AttrL Sub Uses All Process Memory

Tip: For further details, refer to Cisco bug IDs CSCus46997, CSCtz06177 and CSCty49762.