Error: "Search could not be obtained" when accessing searches in Enterprise Security Content Management

Applicability

Description

After upgrading to Enterprise Security 8.1.1 and enabling Content Versioning, opening searches in Content Management fail with the error:

Search could not be obtained:

Investigation

1. The error "Search could not be obtained" was first observed when attempting to open searches in Content Management.

2. On reviewing splunkd.log, the following entry was found:

   ERROR pid=xxxx tid=MainThread file=ba_export_utils.py:create_search_job:110 | Error creating search job. status=400 content=b'{"messages":[{"type":"FATAL","text":"All Time Searches are not allowed. Please pick an appropriate time range."}]}'
   

3. This indicated that the search job was being rejected due to restrictions on All-Time searches.

4. Further review of Settings > Workload Management > Admission Rules revealed an admission rule blocking All-Time searches for roles other than sc_admin and outside the splunk_instance_monitoring app.
   
    

5. Since Content Versioning stores detection versions in the cms_main index, retrieval requires All-Time searches. The admission rule prevented these background searches from executing under splunk_system_user, resulting in the observed error.

Resolution

1. Navigate to Settings > Workload Management > Admission Rules.

2. Perform one of the following actions:

   • Disable the rule, or

   • Modify the rule to allow splunk_system_user or admin roles to run All-Time searches.
     
     

   Example modification:

   search_time_range=alltime AND (NOT app=splunk_instance_monitoring) AND NOT (role=sc_admin) AND NOT (role=admin)

3. After performing the above steps, the correlation searches are now visible. 
   

Related Links

Splunk Doc - Use detection versioning in Splunk Enterprise Security