Different logs received via HEC being sent to the same index.

Applicability

Description

Configured two different Tokens(Akamai/Cylance) to receive and get data on two indexes, however, the data of both tokens were sent to only one index.

It was confirmed in the UI that the Tokens were set with the different indexes, but the issue was still present.

Investigation

The root cause of this issue was due to having two different inputs configured on the same 8443 port.

1. HTTP Event Collector configured to receive data over port 8443.

Configured here:

$SPLUNK_HOME/etc/apps/splunk_httpinput/local/inputs.conf
[http]
disabled = 0
port = 8443

2. Cylance data is configured to receive data on 8443 TCP port, not HTTP.

Configured here:

$SPLUNK_HOME/etc/apps/cylance_protect/local/inputs.conf
[tcp-ssl://8443]
disabled = false
sourcetype = syslog_protect
index = cylance_protect
source = Solium

Because of this conflict, HTTP event collector was not working, but Cylance data is.
The output from the log file:
 

WARN HttpListener - Connection from x.x.x.x:47692 didn't send us any data, disconnecting 
ERROR HttpInputServer - Failed to start HTTP Input server 
ERROR HTTPServer - Cannot set up accepts without binding ports first 
FATAL HTTPServer - Could not bind to port 8443 
ERROR HttpInputServer - Failed to start HTTP Input server 
ERROR HTTPServer - Cannot set up accepts without binding ports first 
FATAL HTTPServer - Could not bind to port 8443

Resolution

Solution for this issue:
Configure one of the inputs (either HTTP or TCP) to another port and this will fix the issue.