ERROR: "The monitor input cannot produce data because splunkd's processing queues are full"

Applicability

Splunk Enterprise Splunk Cloud

Description

During Monitoring it was noticed that:
One of our heavy forwarders on-prem used to send data to Splunk Cloud keeps showing the following processing errors:
 

The monitor input cannot produce data because splunkd's processing queues are full. 
This will be caused by inadequate indexing or forwarding rate, or a sudden burst of 
incoming data.

And these related messages:

WARN TailReader - Could not send data to output queue (parsingQueue), retrying...
INFO TailReader - ...continuing.
WARN TailReader - Could not send data to output queue (parsingQueue), retrying...

Investigation

As part of the Troubleshooting is required to ask a Diag:

It was found that the system specifications were:
- 12 CPU's/24 GB's RAM (good here).
- ulimit open files were 1024 (low).
- THP enabled (Healthcheck).
- Parsing and Aggregate queues were plateaued at +99%.
- High number of below WARN in ERRORS section of healthcheck:

WARN LineBreakingProcessor - Truncating line because limit of 15000 bytes has been exceeded 
with a line length >= 22672 - data_source="proofpoint_message_log", data_host="10.20.30.1", 
data_sourcetype="proofpoint"

Resolution

After updating settings below, that WARN messages were resolved.  

- ulimit open files to 65536 (or 128000, if needed).
Increase set open file limits in Linux

- disabled THP
Transparent huge pages

- Set TRUNCATE = 30,000 in props.conf
LineBreakingProcessor - Truncating line because limit of 10000 has been exceeded
 

Revision History

Revision	Publish Date	Comments
1.0	29-Sep-2026	Initial Release