ITSI backfill of kpis not completing because is risky commands

Applicability

Description

The ITSI process running the backfill for new KPIs may fail. As a consequence the backfill will not be generated, and the Service Analyzer will no show sparkchart for the previous days for that KPI.

The reason is a change in the enforcement of flags for 2 commands used for the ITSI backfill:  collect and mcollect. Those commands are used to summarize the result of the KPI backill searches into the itsi summaries.

When run manually, they can work, but when executed by the backend script, the Splunk backend may not allow them to run.

Investigation

After setting up a new Service, with a new KPI, check the "backfill" checkbox, and save.

If the backfill fails, an error will the generated a few minutes after. Look in the splunk message bar, or search in the internal logs for : 
The backfill is failing because of this error it seems, but we do not have capability to remove safeguard for this riskycommand for internal user.

Example of log : 

Error: "The search that collects data into summary index failed with error: HTTP 403 Forbidden -- b'{"messages":[{"type":"risky_command","text":"Found potentially risky commands:collect ","RiskyCommands":"collect "}". The backfill is failing because of this error it seems, but we do not have capability to remove safeguard for this riskycommand for internal user. Steps to Reproduce: Date Issue Started:2026-10-01 Business Impact:Unable to investigate the incident occurred with CAH (Client Authentication) part of the business Client OS:Linux Client OS Version: Any Recent Changes?No Conf File Name:

Resolution

The issue has been fixed in ITSI 5.0.1 and later. The solution is to upgrade to those versions.

Otherwise on older versions, the possible workaround is to change the risky flag on the 2 commands collect and mcollect on the ITSI search-head, see specifications for commands.conf. In Splunkcloud this will require to open a SplunkCloud support case to edit the file.

Change in : 

[collect]
is_risky = false
[mcollect]
is_risky = false