Indexers are NOT receiving data after Windows universal forwarder upgrade.

Applicability

Spunk Enterprise 8.2.x Splunk Enterprise 9.0.1 Universal Forwarder

Description

Data no longer gets sent to the receiver (indexers).  The Windows universal forwarders upgraded to 8.2.9 or 9.0.1 from 8.0.3.
The Forwarder has useACK = True in the outputs.conf

Symptoms:
The Forwarder tcpout queue is exhausted.
The receiving Indexers are no longer getting the data from the Forwarder.

Investigation

While the UF is on Splunk v9.0.1 or Splunk v8.2.9 is getting

WARN AutoLoadBalancedConnectionStrategy [3480 TcpOutEloop] - Applying quarantine to ip=155.148.110.61 port=9998 connid=0 _numberOfFailures=2

On the Universal forwarder, the tcpout queue will be seen as fully utilized. 


The indexer Error:

ERROR TcpInputProc [6652 FwdDataReceiverThread] - Error encountered for connection from src=xxx.xxx.xx.XX:XXXXX. error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol for both subnets 155.148.84.X and 155.148.91.X.

 

Resolution

This is a known issue with useACK = true

If the UF (Forwarder) instance can't be upgraded to the fixed Splunk version at this time, then there are these workarounds:
Set the outputs.conf useACK=false
OR
autoBatch=false

If neither workaround is feasible, and upgrading the UFs to 9.0.3 where ( issue fixed).
SPL-226003

Related Links

https://community.splunk.com/t5/Getting-Data-In/Why-this-error-after-upgrade-to-9-0-quot-ERROR-TcpOutputQ-lt/m-p/604790#M105160
https://docs.splunk.com/Documentation/Forwarder/9.0.3/Forwarder/Configureforwardingwithoutputs.conf#Common_attributes_for_outputs.conf  

https://conf.splunk.com/files/2019/slides/FN1570.pdf - see "Debugging blocked tcpout queue".

 

Revision History

Revision	Publish Date	Comments
1.0	29-Sep-2026	Initial Release