ITSI service Sandboxes in readonly mode may cause continuous synchronization tasks filling the refresh queue.

Applicability

Description

In the feature Configuration > Services Monitoring > Service Sandboxes, introduced in ITSI 4.19* see documentation. 
ITSI service Sandboxes that were never properly synced with the services/templates may be in an infinite loop of resync attempts
This can lead to continuous new refresh tasks in the ITSI refresh_queue.

Investigation

When the issue occurs, the symptoms are :

• Errors in the execution of the template update in the itsi.consumer process. The message will mention the service_template id, that is the internal Sandbox id. 

2026-07-28 15:43:01,285-0400 process:3650083 thread:MainThread INFO [itsi.consumer] [sandbox_base_service_template_update_handler:151] [deferred] Skipping the Execution for Service template update as Sandbox is not in Editable state . service_template="XXXXXX-XXXX-XXXX-XXXX-XXXXXXXX". Exception="This action can’t be completed because of a conflicting action. Refresh after a minute and try again."

• Constant number of tasks  in the refresh queue, 
  with the job type mentioning the sandbox job_change_type=sandbox_base_service_template_update
  The rate is about 1 new task per minute per sandbox impacted, causing the refresh_queue size to grow overtime.
  
  
  • Use that type of search to see the activity.

index=_internal source=*itsi_consumer.log* job_change_type=sandbox_base_service_template_update | timechart count by host

• Or use the ITSI > Configuration > Additional configuration > Refresh Queue Explorer, and filter per job change type.

Resolution

The long term fix will be planned in ITSI 5.1 and later. Check release notes for but reference "ITSI-4114"

For short term workaround :
identify the problematic Sandboxes (using the service_template id in the messages). From the Sandbox UI, they will be in a read only state. And assess if those sandboxes are still activelly used.

• option A : Reset the sandbox
  Warning : This preserve production changes made to sandbox services after publishing the sandbox, but now the services are no longer associated with the sandbox.
• option B: Unpublish the sandbox, wait for jobs to complete, re-publish the sandbox
  Warning : This loses any changes made in the sandbox.
• option C : Delete the sandbox.

If the old jobs in the refresh_queue do not go away. The solution may require to Pause refresh queue mod_input, delete the specific queued jobs, then resume.

If you need help, open a Splunk Support case for the ITSI product.