[ES] Threat intel error "Message content: FileNotFoundError: [Error 2] No such file or directory: < file name >"

Applicability

Description

ES Search Head captain is showing "Message content: FileNotFoundError: [Error 2] No such file or directory: < file name >" error for new threat intel source that is setup via Threat Intelligence Management "New > Upload".

Issue is only applicable in a SHC environment and when threat intel source is setup on search head members instead of the captain.

Investigation

1. Setup a new threat intel source via Threat Intelligence Management "New > Upload" on a search head member.
2. Choose the desired intel csv file to upload and configure the necessary fields before saving.
3. Threat intel manager will create a ../SA-ThreatIntelligence/lookups/__threat_<csv file name>.dmy file on the member.
4. SH member will run the threatlist to parse the dmy file, however as it is not the captain, it will not parse the file and will inform the captain.
5. SH captain will run the threatlist but throw FileNotFoundError: [Errno 2] No such file or directory: ../SA-ThreatIntelligence/lookups/__threat_<csv file name>.dmy error
6. Checked on captain backend and indeed dmy file was not found, it could only be found on the member that the threat intel source was setup on.

Resolution

Two workaround available:

1. Create the threat intel source on the SH captain, so that the uploaded threat file can be parsed as the captain will have the file. This is because in a cluster environment, only the captain will be running modular inputs for threat intel.
2. Setup a lookup table and definition, this will allow you to update/overwrite the csv file from backend without the need to re-create a newlocal upload threat intel.
   
   1. Create a new "Lookup table files" for the intel csv file
   2. Set app to SA-ThreatIntelligence; Permission to global read and admin write only
   3. Create a new "Lookup definitions" for the lookup table created in step 1
   4. Set app to SA-ThreatIntelligence, Permission to global read and admin write only
   5. Create new threat intelligence source and select "local"
   6. Point it to the lookup definition created in step 3

Related Links

Documentation: Upload the lookup table file

Documentation: Create a CSV lookup definition