Universal Forwarder with High CPU Utilization when Splunk_TA_microsoft-IIS is Installed.

Applicability

Description

Investigation

How to identify it:

a. The universal forwarder has the following apps:

-OS Name Microsoft Windows Server 2008 R2 Enterprise
-Version 6.1.7601 Service Pack 1 Build 7601
-Splunk Universal Forwarder 7.2.0
-Splunk_TA_windows 5.0.0
-Splunk_TA_microsoft-IIS 1.0.1

b. The configuration of Splunk_TA_microsoft-IIS is as follows:

[monitor://C:\inetpub\logs\LogFiles\...\*.log]
disabled = false
sourcetype = ms:iis:auto
index = iis[monitor://D:\inetpub\logs\LogFiles\...\*.log]
disabled = false
sourcetype = ms:iis:auto
index = iis[monitor://D:\inetpub\logs\logfiles\...\*.log]
disabled = false
sourcetype = ms:iis:auto
index = iis[monitor://D:\IISLogs\...\*.log]
disabled = false
sourcetype = ms:iis:auto
index = iis

c. After enabling the Splunk_TA_microsoft-IIS add-on with the configuration above, the CPU usage goes to 70%.

d. Inputs two and three point to the same location, the only difference is the case sensitivity for the path ignored in Windows.



 

Resolution

How to fix it:

Folder Traversals "..." are well known to steal a bunch of CPU resources.

The Splunk_TA_microsoft-IIS add-on official documentation recommends a one-input entry:
Configure inputs using the configuration files

To fix the issue replace all 4 inputs with folder traversal to a one-input (implicit recursive equals true):

[monitor://C:\inetpub\logs\LogFiles]
whitelist = \.log$|\.LOG$
disabled = false
sourcetype = ms:iis:auto

After modifying $HOME_SPLUNK/etc/apps/Splunk_TA_microsoft-IIS/local/inputs.conf and restarting Splunk, the CPU usage will stay normal (~10%).


 

Related Links

Configure inputs using the configuration files