Universal Forwarder Connectivity/Handshake Issues with Deployment Server

Available Languages

Updated:September 29, 2026
Document ID:SPLK045301

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Applicability

Splunk Enterprise Splunk Cloud

Description

The followinghas been reported on a Windows-based server

splunkd.log:04-20-2020 11:41:16.028 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:41:16.074 +1000 WARN HttpPubSubConnection - Unable to parse message from PubSubSvr:
04-20-2020 11:41:16.074 +1000 INFO HttpPubSubConnection - Could not obtain connection, will retry after=39.259 seconds.
04-20-2020 11:41:26.891 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:41:38.898 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:41:45.740 +1000 INFO TcpOutputProc - After randomization, current is first in the list. Swapping with last item
04-20-2020 11:41:45.883 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:41:50.904 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:42:02.913 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:42:14.919 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:42:15.811 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:42:16.356 +1000 WARN HttpPubSubConnection - Unable to parse message from PubSubSvr:
04-20-2020 11:42:16.356 +1000 INFO HttpPubSubConnection - Could not obtain connection, will retry after=70.079 seconds.
04-20-2020 11:42:26.933 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:42:38.944 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:42:38.944 +1000 INFO DC:PhonehomeThread - Attempted handshake 110 times. Will try to re-subscribe to handshake reply
04-20-2020 11:42:45.654 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:42:50.956 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:43:02.970 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:43:14.979 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:43:15.544 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:43:26.994 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:43:38.999 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:43:45.396 +1000 INFO TcpOutputProc - Connected to idx=X.X.X.X:9997, pset=0, reuse=0.
04-20-2020 11:43:47.451 +1000 WARN HttpPubSubConnection - Unable to parse message from PubSubSvr:
04-20-2020 11:43:47.451 +1000 INFO HttpPubSubConnection - Could not obtain connection, will retry after=78.879 seconds.
04-20-2020 11:43:51.014 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:44:03.022 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected
04-20-2020 11:44:15.028 +1000 INFO DC:DeploymentClient - channel=tenantService/handshake Will retry sending handshake message to DS; err=not_connected


Command to confirm connectivity is successful for this scenario:

splunk cmd openssl s_client -connect <splunk_server.com>:9997

This result was obtained:

CONNECTED(00000190)
depth=2 CN = Root CA
verify error:num=19:self signed certificate in certificate chain
324:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:.\ssl\s3_pkt.c:1498:SSL alert number 40
324:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:.\ssl\s23_lib.c:177:
---
<snip>.....
Verify return code: 19 (self signed certificate in certificate chain)
---




Investigation

Reason:
Messages like these: INFO TcpOutputProc - Connected to idx=xx.xx.xx.xx:9997, pset=0, reuse=0.
Show up as false positives.


Resolution

Solution:
Perform a (rolling) restart on the indexers via the cluster master. Then restart the SH or SH Cluster (if any), this will make the false positive errors go away.

Revision History

Revision Publish Date Comments
1.0
29-Sep-2026
Initial Release

Was this Document Helpful?

FeedbackFeedback

Contact Cisco

This Document Applies to These Products