Indexers fail to start up after upgrade to Splunk Enterprise version 8.1.x

Applicability

Splunk Enterprise

Description

During startup, Splunk runs the locktest utility to validate if the filesystem is supported. Indexers are not starting up after upgrade to Splunk Enterprise version 8.1.x because of filesystem checks failure when executing the locktest utility. Before the upgrade, the indexers were able to pass the filesystem checks and start up successfully.

Here is the error message causing Splunk startup to fail: 

$/opt/splunk/bin/splunk start
Splunk> Take the sh out of IT.Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
coldPath='/opt/splunk_db/var/lib/splunk/<index>/colddb' of index=<index_name> on 
unusable filesystem.
Validating databases (splunkd validatedb) failed with code '1'. If you cannot resolve 
the issue(s) above after consulting documentation, please file a case online at 
http://www.splunk.com/page/submit_issue

Investigation

"mount -v" and "nfsstat -m" indicate that the mount point is NFSV3 which is a supported file system. 
 

Resolution

Splunk is failing the filesystem checks because of a bug (SPL-199608) that is causing the temp lock files (generated to validate the cold path) to be inaccessible. The bug will be fixed from Splunk Enterprise version 8.1.3 onward.

To startup the indexers, the workaround is to set OPTIMISTIC_ABOUT_FILE_LOCKING=1 in $SPLUNK_HOME/etc/splunk-launch.conf to bypass filesystem checks during startup.

Refer to Splunk Enterprise does not start due to unusable filesystem - Splunk Documentation
 

Revision History

Revision	Publish Date	Comments
1.0	29-Sep-2026	Initial Release