Document ID: 10557
Updated: Aug 30, 2005
Contents
Introduction
After issuing a show processes cpu command, the Catalyst fixed configuration switches 2900XL and 3500XL report a high value for CPU utilization at idle. With other Cisco devices, high CPU utilization at idle is cause for concern; however, it is normal for the 2900XL and 3500XL.
This document explains the reason for the high CPU utilization value and gives some suggestions for monitoring and troubleshooting excessively high CPU utilization.
Before You Begin
Conventions
For more information on document conventions, see the Cisco Technical Tips Conventions.
Prerequisites
There are no specific prerequisites for this document.
Components Used
This document is not restricted to specific software and hardware versions.
Background
The 2900XL/3500XL switches have two separate hardware subsystems, the switching subsystem and the internal CPU engine. Each is described below:
-
The switching subsystem controls the receiving and forwarding of data traffic.
-
The internal CPU engine implements higher-level protocols to control and monitor switch behavior.
As a result, 2900XL/3500XL CPU utilization does not reflect:
-
The number of packets being switched.
-
The total load on the switch.
CPU utilization does reflect the switch CPU processing, which includes the following:
-
spanning-tree processing
-
port status
-
LED indicators
In comparison, Cisco routers such as the Cisco 3600 process and route packets using software; therefore, the output of the show processes cpu command provides insight into the overall load on the router. The CPU utilization is linear for the amount of packet processing and routing being performed on the Cisco 3600 router.
The common observation is that the CPU utilization output for the show processes cpu command on the 2900XL/3500XL switch shows a fairly high value when the switch is idle and has no active ports. When viewing the output of the show processes cpu command, the highest percentage processes are Port Status Proc (Port Status Process) and LED Control Proc (LED Control Process). It is normal behavior for these processes to be a high percentage of the total CPU utilization. These processes run constantly to maintain and monitor the status of the ports and the different LEDs available on the panel of the switch.
Baseline CPU Utilization
The table below summarizes typical CPU utilization seen by 3500XL switches at idle and under load. It is noteworthy that CPU utilization does not increase linearly with increased load.
| Switch | CPU Utilization at IDLE | CPU Utilization at 50 Percent Write Speed Using Unicast Traffic |
|---|---|---|
| 3512 XL | 5 seconds (32 percent/2 percent) 1 minute (32 percent) 5 minutes (32 percent) | 5 seconds (32 percent/2 percent) 1 minute (33 percent) 5 minutes (32 percent) |
| 3548 XL | 5 seconds (44 percent/6 percent) 1 minute (44 percent) 5 minutes (32 percent) | 5 seconds (46 percent/7 percent) 1 minute (33 percent) 5 minutes (33 percent) |
Therefore, it is normal for a 2900XL or 3500XL switch to indicate a CPU utilization value of 30 percent to 50 percent, even under minimal load.
Troubleshoot High CPU Utilization
The CPU is responsible for the following:
-
processing broadcasts
-
spanning-tree Bridge Protocol Data Units (BPDUs)
-
address learning
-
port status
-
LED operations
The processes listed above may contribute to excessive CPU utilization. If the CPU utilization is extremely high (around 80 percent to 99 percent), or if the CPU utilization is higher than normal, follow these recommendations.
-
Create a baseline CPU utilization at no load.
-
Note which processes are generating the most load.
-
Check for consistent MAC address clearing or spanning-tree instability. Use the show spanning-tree vlan vlan-id command to gather this information.
-
Check for network broadcast storm.
-
Create a Service Request with Cisco Technical Support, if necessary.
Check Utilization of 2900XL and 3500XL Switches
The show controller switch command provides indicative information regarding the total switch utilization. This is an example:
Switch#show controller switch Switch registers: Device Type : 0x00040273 Congestion Threshold : 0x00000E95 Peak Total Allocation : 0x0000001A Total Allocation : 0x00000000 Peak Total Bandwidth : 0x00000020 Total Bandwidth : 0x00000000 Total Bandwidth Limit : 0x000003DE Lower Bandwidth Limit : 0x000003DE Switch Mode : 0x00040000 Switch#
The Total Bandwidth Limit varies between different 2900XL and 3500XL models. When the Total Bandwidth reaches the Total Bandwidth Limit value, the switch has reached its full bandwidth capacity and begins to drop packets. The Peak Total Bandwidth is the highest value attained by the Total Bandwidth since the last time the show controller switch command was executed. Note, the values for the above parameters are in hexadecimal.
The Congestion Threshold value is used as conservative value for the maximum global buffer utilization. When the buffer utilization noted by Total Allocation reaches this value, the switch may drop frames. The Peak Total Allocation value shows the highest value attained by the Total Allocation since the last time the show controller switch command was executed. It is possible for the Peak Total Allocation and/or the Total Allocation to be greater than Congestion Threshold. If the Total Allocation reaches or is over the Congestion Threshold amount, the switch is experiencing considerable network activity near its full capacity.
The global buffer utilization may be adversely affected by several configuration issues, described here:
-
Speed mismatch between an ingress and egress port; for example, several 100 Mb clients transferring files to a server connected to the switch at 10 Mb, half-duplex.
-
Multiple input ports feeding a single output port.
-
Duplex mismatch on multiple ports.
-
Numerous ports that are experiencing collisions and/or output errors due to half-duplex configuration or oversubscription of a slow link.
Sample show processes cpu Command Output
Below is sample output from the show processes cpu command.
Switch#show processes cpu CPU utilization for five seconds 35%/4%; one minute 35%; five minutes 35% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 15 44349 0 0.00% 0.00% 0.00% 0 Load Meter 2 18 31 580 0.24% 0.02% 0.00% 0 Exec 3 90945 22514 4039 0.00% 0.01% 0.00% 0 Check heaps 4 3 1 3000 0.00% 0.00% 0.00% 0 Chunk Manager 5 2 3 666 0.00% 0.00% 0.00% 0 Pool Manager 6 0 2 0 0.00% 0.00% 0.00% 0 Timers 7 0 1 0 0.00% 0.00% 0.00% 0 Entity MIB API 8 451 3742 120 0.00% 0.00% 0.00% 0 ARP Input 9 0 1 0 0.00% 0.00% 0.00% 0 RAM Access (dm 0 10 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd 11 1202 89758 13 0.00% 0.00% 0.00% 0 Net Background 12 0 53 0 0.00% 0.00% 0.00% 0 Logger 13 2410 221081 10 0.00% 0.00% 0.00% 0 TTY Background 14 352967 767348 459 0.04% 0.09% 0.08% 0 Per-Second Jobs 15 708 9223 76 0.00% 0.00% 0.00% 0 Net Input 16 70 44350 1 0.00% 0.00% 0.00% 0 Compute load avg 17 37697 3697 10196 0.00% 0.01% 0.00% 0 Per-minute Jobs 18 877166 17911145 48 0.32% 0.21% 0.20% 0 LED Control Proc 19 11915 221085 53 0.00% 0.00% 0.00% 0 Frank Aging 20 23499061 26508015 886 5.73% 5.54% 5.55% 0 Port Status Proc 21 0 1 0 0.00% 0.00% 0.00% 0 VM Prune Events 23 1554627 741189 2097 0.32% 0.41% 0.43% 0 GDS Frame Ager 24 0 1 0 0.00% 0.00% 0.00% 0 RAM Access (gi0/ 25 0 1 0 0.00% 0.00% 0.00% 0 IP NAT Ager 28 0 1 0 0.00% 0.00% 0.00% 0 RAM Access (gi0/ 30 2568178 5890745 435 0.65% 0.73% 0.74% 0 Broadcast Storm 31 0 1 0 0.00% 0.00% 0.00% 0 Port Group Chang 32 2265 5952 380 0.00% 0.00% 0.00% 0 IP Input 33 627 392 1599 0.00% 0.00% 0.00% 0 Address Learning 34 0 1 0 0.00% 0.00% 0.00% 0 RAM Access (dm 1 35 0 1 0 0.00% 0.00% 0.00% 0 RAM Access (dm 2 36 369323 663847 556 0.00% 0.00% 0.00% 0 Enet Aging 37 3073 26588 115 0.00% 0.00% 0.00% 0 CDP Protocol 38 805 70 11500 0.00% 0.00% 0.00% 0 Address Deletion 39 71 35 2028 0.00% 0.00% 0.00% 0 Switch CGMP Prot 40 0 1 0 0.00% 0.00% 0.00% 0 TCP Timer 41 0 1 0 0.00% 0.00% 0.00% 0 TCP Protocols 42 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers 43 0 1 0 0.00% 0.00% 0.00% 0 HTTP Timer 44 0 1 0 0.00% 0.00% 0.00% 0 CGMP Forwarding 45 121 22176 5 0.00% 0.00% 0.00% 0 Cluster RARP 46 184 27803 6 0.00% 0.00% 0.00% 0 Cluster Base 47 105 113 929 0.00% 0.00% 0.00% 0 Spanning Tree 48 6438 36032 178 0.00% 0.00% 0.00% 0 STP Hello 49 283088 128597 2201 0.00% 0.00% 0.00% 0 STP Queue Handle 50 172 93 1849 0.00% 0.00% 0.00% 0 Malibu STP Adjus 51 0 1 0 0.00% 0.00% 0.00% 0 Time Range Proce 52 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro 53 0 2 0 0.00% 0.00% 0.00% 0 Bridge MIB traps
Known Issues
High CPU Utilization Due to Autoconf Process
If you are experiencing high CPU utilization (up to 99 percent) and the show processes cpu command output indicates a large portion of this as the Autoconf process, you are probably hitting a known bug. This problem is filed against Cisco IOS Software Release 12.0(5.4)WC1 and earlier.
The problem occurs when you boot a switch which is configured with VLAN 1 interface in the administratively down state without an IP address, and another VLAN interface exists with an IP address that is administratively up. With this configuration, when the switch boots up, it finds that there is no IP address for VLAN 1 and starts the DHCP-based auto-configuration. This calls a function in the software which checks for interface VLAN 1 to be up before starting the DHCP, and hence comes up in an infinite loop. This results in the high CPU utilization.
The workaround is to configure the following commands, save the configuration, and reload the switch:
-
interface vlan 1
-
no ip address
Note: If you have any other interface VLANs without IP addresses, you will have to follow the same procedure for all those VLANs.
The following command shows the sample output if the switch is hitting this bug:
Switch#show processes cpu CPU utilization for five seconds: 99%/7%; one minute: 99%; five minutes: 99% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4034 538437 7 0.00% 0.00% 0.00% 0 Load Meter 2 0 1 0 0.00% 0.00% 0.00% 0 IP NAT Ager !--- Output suppressed. 62 1888112123 117660129 16047 24.77% 49.89% 51.43% 0 Router Autoconf 63 12583 1254 10034 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro 64 20 9 2222 0.00% 0.00% 0.00% 0 Bridge MIB traps 66 592259 538607 1099 0.00% 0.00% 0.00% 0 Runtime diags 67 0 1 0 0.00% 0.00% 0.00% 0 SNMP Timers !--- Output suppressed. Switch#
The fix for this Autoconf process high CPU issue has been integrated in
code Cisco IOS Software Release 12.0(5)WC2 and later. Refer to Cisco bug ID
CSCdv21552
(registered customers only)
in the
Bug Toolkit
(registered customers only)
.
High CPU Utilization Due to Excess STP Instances
You create a Spanning Tree Protocol (STP) instance when you assign an interface to a VLAN. The STP instance is removed when the last interface is moved to another VLAN. Each VLAN is a separate STP instance. If you have already used up all available STP instances on a switch, adding another VLAN anywhere in the VLAN Trunking Protocol (VTP) domain creates a VLAN that does not run STP on that switch. For example, if 250 VLANs are defined in the VTP domain, you can enable STP on 64 of those VLANs. The rest of the VLANs must operate with STP disabled. There is a limitation on the number of instances of STP that a switch can handle.
The Catalyst 2912 XL, Catalyst 2924 XL, and Catalyst 2924C XL support only 64 STP instances and 64 VLANs. All other Catalyst 2900 XL switches and all Catalyst 3500 XL switches support 64 STP instances and 250 VLANs. Ensure that the total number of logical ports across all instances of STP for different VLANs does not exceed the maximum number supported. If you exceed these STP instance recommendations, the switch exhibits high CPU utilization. The Spanning Tree Algorithm (STA) is generally not processor intensive and has priority over other processes. You can see the CPU usage for the Spanning Tree process from the show processes cpu command output.
Switch#show processes cpu CPU utilization for five seconds: 98%/10%; one minute: 96%; five minutes: 96% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 111 411944 0 0.00% 0.00% 0.00% 0 Load Meter 2 8251 2638 3127 0.90% 0.07% 0.01% 0 Exec 3 753782 209157 3603 0.00% 0.01% 0.00% 0 Check heaps 4 0 1 0 0.00% 0.00% 0.00% 0 Chunk Manager !--- Output suppressed 54 9862 282046 34 0.00% 0.00% 0.00% 0 Cluster Base 55 2050 959972 3791 32.63% 30.03% 17.35% 0 Spanning Tree 56 144660 6673996 21 0.00% 0.00% 0.00% 0 STP Hello 57 0 14 0 0.00% 0.00% 0.00% 0 STP Queue Handle 58 167 193 865 0.00% 0.00% 0.00% 0 Malibu STP Adjus !--- Output suppressed
Issue the show spanning-tree summary command to verify the number of logical ports or interfaces per VLAN in the STP Active column.
Switch#show spanning-tree summary
UplinkFast is disabled
Name Blocking Listening Learning Forwarding STP Active
-------------------- -------- --------- -------- ---------- ----------
VLAN1 19 0 0 7 26
VLAN3 19 0 0 5 24
VLAN4 19 0 0 5 24
VLAN5 19 0 0 5 24
VLAN6 19 0 0 5 24
VLAN7 19 0 0 5 24
VLAN9 19 0 0 5 24
VLAN10 19 0 0 5 24
VLAN11 19 0 0 5 24
VLAN12 19 0 0 5 24
VLAN20 19 0 0 7 26
VLAN21 19 0 0 5 24
!--- Output suppressed
Name Blocking Listening Learning Forwarding STP Active
-------------------- -------- --------- -------- ---------- ----------
VLAN111 19 0 0 5 24
VLAN200 19 0 0 5 24
VLAN222 19 0 0 5 24
VLAN900 19 0 0 5 24
-------------------- -------- --------- -------- ---------- ----------
64 VLANs 1217 0 0 329 1546
Related Information
Open a Support Case 
(Requires a Cisco Service Contract.)
Related Cisco Support Community Discussions
The Cisco Support Community is a forum for you to ask and answer questions, share suggestions, and collaborate with your peers.
Refer to Cisco Technical Tips Conventions for information on conventions used in this document.