Findings Are Not Present In Analyst Queue

Applicability

Description

This article will help confirm that detections are running and triggering as expected, and address why no Findings are displayed within the Analyst Queue of Mission Control.

Investigation

Verify detections with scheduler logs

Using the search below to look at the scheduler logs for detections it can be confirmed if the alert_actions show the value of notable, which indicates that the detection found the results appropriate to create a Finding. If no results are showing that value, then confirm that there are results within the result_count field and that the suppressed field does not show a value of "1". If result_count shows "0", then no results were found by the detection. If suppressed shows "1", then the Finding was suppressed, so the detection Throttling settings should be verified to prevent the behavior.

index=_internal sourcetype=scheduler status=success

If a detection does show the notable value for alert_actions, but that result is not present in the Analyst Queue, then confirm if it is present within the notable index. 

Verify if Findings are present in notable index

The search below will confirm if the Findings are present in the index and can also be helpful to determine when the Findings stopped flowing, as the last time stamp for an event in this index may point to when the behavior began.

index=notable

If the Finding is found when searching the index directly, then verify that there are no filters within the Analyst Queue that may be hiding the result. Additionally try using the macro below to confirm the Finding is present and that the suppression field is not present for the finding. If suppression is present for the finding, then address the suppression configurations within 'Configure > Findings and Investigations > Suppressions'.

`notable`

Verify if Findings are present within the main index

If the Finding is not found when searching the notable index directly, then try using the search below to confirm if you see events that resemble the expected Findings within the main index. 

index=main sourcetype=stash

If the above search does show events resembling the Findings, then it is important to verify that the default inputs.conf is present at the path below:

/opt/splunk/etc/apps/Splunk_SA_CIM/default/inputs.conf

If the search does not show the Findings, then check the path below on the Enterprise Security search head to determine if there are files present. If files are present here it generally indicates that either indexing queues are blocked or the search head is unable to send data to the indexing layer. Addressing that issue will then allow all of the Findings to flow into the Analyst Queue.

/opt/splunk/var/spool/splunk/

Resolution

After following through the process outlined in the Investigation section Findings should hopefully be flowing once more. If not please open a support case confirming that steps taken thus far, so that the investigation may be continued to resolve the behavior.
https://www.splunk.com/en_us/resources/working-with-splunk-support.html?locale=en_us