Unable to remove or modify SAML users via the Splunk User Interface

Applicability

Description

When Splunk is configured with SAML authentication, attempting to delete a user via the Splunk Web interface does not successfully remove the user from the system.

For example, executing the following REST API command removes the user from the local passwd file, but the user still appears in the Splunk User Interface as a SAML user:

curl -k -u admin:<password> --request DELETE https://<search_head>:8089/services/authentication/users/<user_to_be_deleted>

Additionally, attempts to modify user permissions or roles for a SAML-authenticated user via the Splunk Web interface may appear to save successfully, but the changes do not persist. 

This occurs because SAML user attributes and roles are managed externally by the Identity Provider (IdP). For IdPs that do not support SAML Attribute Query Requests (such as Azure Entra ID), Splunk caches the role information from the initial SAML assertion in the UserInfoCache for the duration of the session. Splunk does not automatically re-evaluate role mappings unless a new SAML assertion is processed or the cache is manually cleared.

Investigation

To determine if the issue is related to the user type, verify whether the user is a Local user or a SAML-authenticated user:

1. Verify if the user is a Local user. Local users are modified via the UI and are stored within the $SPLUNK_HOME/etc/passwd file.
2. Verify if the user is a SAML user. SAML users are authenticated and managed via the external IdP, and their configurations are stored within $SPLUNK_HOME/etc/system/local/authentication.conf.
3. Run the following REST query to check the user type:

| rest /services/authentication/users
| search users=<user_in_question>
| table title type roles
| rename title as username type as authentication_type

If the user is identified as a SAML user, the Splunk Web interface will not allow local modifications to their roles or permanent deletion of the user account.

Resolution

Perform the following steps:
To modify roles or permissions for a SAML user:

1. Add or remove the user from the appropriate group within the SAML IdP.
2. Ensure the group is correctly mapped to a Splunk role by navigating to Settings > Authentication Methods > SAML Settings > Groups.
3. Instruct the user to completely log out of Splunk and log back in. This forces Splunk to process a new SAML assertion and refresh the cached roles.

To permanently remove a SAML user from the Splunk environment:

1. Delete the user account within the SAML IdP.
2. Remove the associated entry from the $SPLUNK_HOME/etc/system/local/authentication.conf file.
3. Reload the authentication configuration by running the following command:

$SPLUNK_HOME/bin/splunk reload auth