ERROR: "The search head cluster captain (https://x.x.x.x.local:8089) is disconnected; skipping configuration replication"

Applicability

Splunk Enterprise Splunk Enterprise Security

Description

Certain conditions can cause a member's baseline to get out-of-sync with the captain's baseline, and thus with the other members' baseline. In particular, a member can be out-of-sync when recovering from a loss of connectivity with the cluster. To remediate this situation, the member must resync with the cluster.

For this issue you will find in splunkd logs on SH ConfReplicationThread failed several times:

WARN SHCMasterHTTPProxy - Low Level http request failure err=failed method=POST 
path=/services/shcluster/captain/jobs/scheduler_c3ZjX3__SplunkEnterpriseSecuritySuite__
RMD5193b00c5a6f570ae_at_1618902960_13697_F8360495-F082-4BFB-9C12-D620D56BEDEF/report_job_completion 
captain=x.x.x.x.local:8089 rc=0 actual_response_code=502 expected_response_code=200 
status_line="Connection closed by peer" socket_error="Connection closed by peer"WARN ConfReplicationThread - The search head cluster captain (https://x.x.x.x.ocal:8089) is 
disconnected; skipping configuration replicationWARN ConfReplicationThread - Error pulling configurations from captain=https://x.x.x.xlocal:8089, 
consecutiveErrors=1 msg="Error in fetchFrom, at=66607192e80d1bd1d3550f8b4e44d2f0dc14111a: 
Network-layer error: Connect Timeout"WARN ConfReplicationThread - Error pushing configurations to captain=https://x.x.x.xlocal:8089, 
consecutiveErrors=1 msg="Error in acceptPush: Non-200 status_code=400: ConfReplicationException: 
Cannot accept push with outdated_baseline_op_id=13deac132fbb1a1df6b450fde943f929d6e8fcb6; 
current_baseline_op_id=c80c675d29e92076a6e5bc9fdf7c3d58f0058f6a"

Resolution

Based on the logs found, recovering from a loss of connectivity should be the root cause of the out-of-sync issue and manual re-sync is required, check the next link for information about synchronization issues:

Replication synchronization issues

Revision History

Revision	Publish Date	Comments
1.0	29-Sep-2026	Initial Release