Migrating Splunk Enterprise from init.d to systemd on Linux Systems

Applicability

Description

When migrating Splunk Enterprise from init.d to systemd on Linux systems, users may encounter warning messages related to cgroup permissions such as below:

WARNING: Splunk was configured as a systemd service using old version service file.
WARNING: Updating to new version failed, this may cause splunk fail to start.
WARNING: Please check splunk's systemd service file and manually correct it.

This article outlines the correct migration process, required Linux OS permissions, and how to resolve legacy warning messages.

Investigation

Check the steps to migrate from init.d to systemd:

1. Login to the server via SSH with root account.
2. Switch to the splunk account: 
   

   su - splunk

3. Stop Splunk service:
   

   sudo $SPLUNK_HOME/bin/splunk stop

4. Disable current boot-start configuration: 
   

   sudo $SPLUNK_HOME/bin/splunk disable boot-start

5. Confirm /etc/init.d/splunk is deleted: 
   

   sudo ls /etc/init.d

6. Enable boot-start with systemd: 
   

   sudo $SPLUNK_HOME/bin/splunk enable boot-start -systemd-managed 1 -user splunk -group splunk -create-polkit-rules 1

7. Start Splunk:
   

    $SPLUNK_HOME/bin/splunk start

Required Linux OS Permissions:

• Root access for initial migration steps
• Sudo privileges for the splunk user
• Permissions to modify system files (e.g., in /etc/systemd/system/)
• Permissions to manage systemd services

Resolution

If you encounter a warning message related to cgroup permissions, follow these steps:

1. Edit the systemd unit file: 
   

   sudo vi /etc/systemd/system/Splunkd.service

2. Replace these lines: 
   

   ExecStartPost=-/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/cpu/system.slice/%n" 
   ExecStartPost=-/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/memory/system.slice/%n"

3. Remove this line if present: 
   

   ExecStartPost=-/bin/bash -c "chown -R splunk:splunk /sys/fs/cgroup/system.slice/%n"

4. Save the file and restart Splunk.

Related Links

Configure systemd manually