Files Not Ingesting Through Monitor Input Stanza on Windows UF Machine

Applicability

Description

In this situation, it appears that log files are correctly located in the specified path on the Windows Universal Forwarder (UF), and a corresponding monitor stanza has been created in the inputs.conf file to monitor this path. 

However, despite this configuration, the files are not being ingested into Splunk. Although the path and required parameters are as expected.

Investigation

Upon conducting a thorough investigation, it was observed that running the "list monitor" command in the backend of the instance successfully displays all the files in the monitoring results. 

This confirms that the files are being monitored as expected. 

However, despite this, the logs within these files are not being ingested into Splunk.

Location: "C:\Program Files\SplunkUniversalForwarder\bin"

This indicates that the monitoring configuration is correct, but further troubleshooting is needed to determine why the logs are not being processed.

Command : 

./splunk list monitor

Output of the Command:

Additionally, upon reviewing the content of the files, it was discovered that the first 8-10 lines in all the files contain identical data.

Resolution

The observed behavior occurs because, by default, the input performs CRC (Cyclic Redundancy Check) against only the first 256 bytes of a file. 

This mechanism prevents the input from indexing the same file twice, even if it has been renamed (as is common with rolling log files). 

However, since the CRC check is based solely on the first few lines, files with identical headers can produce matching CRCs, leading to legitimate files being skipped during ingestion.

To resolve this, the parameter crcSalt needs to be set to NONE in the input configuration, which ensures that each file is treated uniquely.

Add the following parameter to your input stanza and restart Splunk:

crcSalt = NONE

Example Monitor Stanza: 

[monitor://C:\Program Files\SplunkUniversalForwarder\etc\apps\*]disabled = false
index = Test
sourcetype = test:json
crcSalt = NONE

By applying this configuration, Splunk will perform a more comprehensive check to prevent legitimate files from being skipped.

Related Links

Inputs.conf