Migrating from Mmapv1 to wiredTiger Storage Engine failed

Applicability

Description

 While migrating the storage engine, encountered an error while executing the below command in the bin directory:

splunk migrate kvstore-storage-engine --target-engine wiredTiger 

Encountering the below error:

ERROR: Cannot get the size of the KVStore folder at=E:\SplunkDB\kvstore\mongo, due to reason=3 errors occurred, description of first 3 errors: [{operation:"failed to stat file", error:"Access is denied.", file:"E:\\SplunkDB\\kvstore\\mongo"}, {operation: "failed to stat
ile", error: "Access is denied.", file:"E: \\SplunkDB\\kvstore\\mongo"}, {operation:"failed to stat file", error: "Access is denied.", file:"E: \\SplunkDB\\kvstore\\mongo"} 

Investigation

1) Checked the permissions of the user for the path "E:/SplunkDB/kvstore/mongo" as the error was indicating "Access denied".

2) Inquired if the user is an administrative user and has the respective privileges.

3) Verified the permission for KV store and mongo directories.

4) Traversed through the read/write access for all of the files for the admin user.

5) Tried with another admin user who has domain account access. 

6) Noticed that the domain account has full access and all permissions, however, Splunk was unable to access the mongod folder.

 7) Tried to modify the permissions and inherit all the objects but, it was not successful and returned with the enumerate error:

• Logged in through the GUI and right click > Properties > 

Resolution

Workaround -

-Perform the two commands shown below to take ownership of all objects on the path where the mongo folder is present and then inherit it:

takeown /F E:\SplunkDB\kvstore\mongo /R /D y

icacls E:\SplunkDB\kvstore\mongo /grant Administrator:(D,WDAC) /t

The output should look like the following:

Note - "IP-0ACA10EB\splunker" is taken as an example here.

-Post executing these commands with the successful output, tried to execute the migrate command, and it was successfully migrated and was able to upgrade the Splunk version. 

Related Links

Migrate the KV store storage engine | Splunk Docs

Upgrade the KV store server version - Splunk Documentation