SOAR - SASE Event Forwarding failed sending events

Applicability

Description

Configured Splunk version (9.2.1) on the Windows 2019 server or Windows 2022 server and installed Splunk app for SOAR export (v4.3.13). The test connection with SOAR was successful. However, when creating an Event Forwarding to forward to the SOAR environment, the events were not being sent to SOAR when the event forwarding was scheduled to run every 5 minutes. No container or artifact was created on SOAR. 

Investigation

The following error was found in splunkd.log every time the Event Forwarding was scheduled to run.

08-07-2024 08:40:07.738 -0400 WARN Pathname [589732 AlertNotifierWorker-0] - Pathname 'C:\Program Files\Splunk\bin\Python3.exe C:\Program Files\Splunk\etc\apps\search\bin\runshellscript.py phantom_forward.py 1 | savedsearch "IT Unlock Account" | savedsearch "IT Unlock Account" _phantom_app_IT Unlock Account Saved Search [_phantom_app_IT Unlock Account] number of events(1) https://<SPLUNK>:8000/app/phantom/@go?sid=scheduler__nobody__phantom__RMD52ff619c780cd7d25_at_1723034400_23436 scheduler__nobody__phantom__RMD52ff619c780cd7d25_at_1723034400_23436 C:\Program Files\Splunk\var\run\splunk\dispatch\scheduler__nobody__phantom__RMD52ff619c780cd7d25_at_1723034400_23436\results.csv.gz' larger than MAX_PATH, callers: call_sites=[0xea4e92, 0xea6ab1, 0x1840cff, 0x183d533, 0x1568f50, 0x14ef8d7, 0x83daa5, 0x83d054, 0x83cda9, 0x85f718, 0x863fa7]08-07-2024 08:40:09.264 -0400 ERROR script [589732 AlertNotifierWorker-0] - SearchMessage orig_component=script sid=scheduler__nobody__phantom__RMD52ff619c780cd7d25_at_1723034400_23436 message_key= message=command="runshellscript", Error while executing script [WinError 2] The system cannot find the file specified

Please note that "larger than MAX_PATH" is a warning message which can be ignored. The search result was actually created under C:\Program Files\Splunk\var\run\splunk\dispatch\scheduler__nobody__phantom__RMD52ff619c780cd7d25_at_1723034400_23436\results.csv.gz. However, the app was not able to recognize the file. When testing “Send to SOAR” action from the same saved search/alert, events were being sent to SOAR and containers/artifacts were created correctly.

Resolution

The following code line #!/usr/bin/env python3 in phantom_forward.py file is not supported on Windows. This was introduced in the version 4.3.13.

There are two options to fix it temporarily:

1. Manually remove the line from the file and restart Splunk Enterprise.

- Edit the file <SPLUNK_HOME>/etc/apps/phantom/bin/scripts/phantom_forward.py, remove the line #!/usr/bin/env python3

Please note that there is no negative impact for Splunk running on Windows OS.

2. Downgrade Splunk App for SOAR Export to use version 4.3.2.

For permanent fix will be in the Splunk App for SOAR Export future release.