Splunk Universal Forwarder not sending data

Applicability

Splunk Universal Forwarder Splunk Enterprise

Description

Troubleshooting guide about how to investigate when the UF is not forwarding data or the data is not showing in the Indexers side.

Investigation

Initial basic checks:

1. check if splunk process is running on splunk forwarder

For windows check in services | for linux use below command

ps -ef |grep splunkd

 Or

cd $SPLUNK HOME/bin
./splunk status

Also run:

splunk display listen

2. Check if splunk forwarder forwarding port is open by using below command

netstat -an | grep 9997

Or

netstat -an | grep 9997

If output of above command is blank then your port is not open. Open it.

3. Check on indexer if receiving is enabled on port 9997 and port 9997 is open on indexer

Check if receiving is configured :On indexer go to setting > forwarding and receiving > check if receiving is enabled on port 9997.
If not enable it

4. check if it is possible to ping indexer  from forwarder host

ping indexer name

If ping to the server did not work, then check network issue

5. Make sure the outputs.conf is properly configured and pointing to the correct Indexers ip's or hosts.

For this purpose, lets use btool: 
 

./splunk cmd btool outputs list./splunk cmd btool check   (check for typos on configs in the instance.)./splunk cmd btool outputs list --debug  ( shows where the configuration file and its stanza is located.)

Some other Basic Network investigation steps: 

2.1. Start investigation for network issues, start by confirming the communications within the UFs and the Indexer, for example running the below command:

telnet indexername.domainname.com 9997

The output should be similar like this:
Connected to indexername.domainname.com

2.2.  Make sure 100% there is not firewall issue.
or within a local firewall like ip tables on the indexers ( for example).

To confirm this, please SSH into the indexer and attempt to telnet back onto yourselve via port 9997.
Run the command:

telnet localhost 9997

In case the issue is related to firewall rules like, ip tables you could receive an error like the one below or similar:
telnet: connected to address ::1: Connection refused
If that is the case, it would be a matter of opening the port 9997 via ip tables on your local firewall.

Resolution

As per above steps, it depends of its results, the resolution could be opening 9997 port, of properly configuring firewall rules.
It could also be a typo or missconfiguration in the $SPLUNK_HOME/etc/local/outputs.conf. 

Related Links

Outputs.conf

Forwarding: Receiver connection

Revision History

Revision	Publish Date	Comments
1.0	30-Sep-2026	Initial Release