SOAR - Exchange On-Premise EWS failed to add email attachment

Applicability

Description

The email was being retrieved by “get email” action from Microsoft Exchange On-Premise EWS app. The email artifact was created but the attachment zip file was not being added to the container.

Investigation

The error was found out in the spawn.log:

2024-09-09T19:04:26.115722 PID:1964328 TID:1964328 : DEBUG: SPAWN3 : /srv/phantom/alchemy/connectors/spawn3/spawn3.cpp : 1024 : EWSOnPremConnector :part type <class 'email.message.Message'>
2024-09-09T19:04:26.115836 PID:1964328 TID:1964328 : DEBUG: SPAWN3 : /srv/phantom/alchemy/connectors/spawn3/spawn3.cpp : 1024 : EWSOnPremConnector :file_path: /tmp/ph_email_ewsonpremi3i9p3ol/2_[EXTERNAL]Emailalert!🎉.zip_False 
2024-09-09T19:04:26.116184 PID:1964328 TID:1964328 : ERROR: SPAWN3 : /srv/phantom/alchemy/connectors/spawn3/spawn3.cpp : 996 : EWSOnPremConnector :Exception occurred. Exception: 'ascii' codec can't encode character '\U0001f389' in position 63: ordinal not in range(128)
Traceback (most recent call last):
  File "/opt/phantom/apps/microsoftexchangeonpremiseews_badc5252-4a82-4a6d-bc53-d1e503857124/process_email.py", line 619, in _handle_attachment
    with open(file_path, 'wb') as f:
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f389' in position 63: ordinal not in range(128)
2024-09-09T19:04:26.116216 PID:1964328 TID:1964328 : DEBUG: SPAWN3 : /srv/phantom/alchemy/connectors/spawn3/spawn3.cpp : 1024 : EWSOnPremConnector :Error occurred while adding file to Vault. Error Details: /tmp/ph_email_ewsonpremi3i9p3ol/2_[EXTERNAL]Emailalert!🎉.zip_False 

The root cause of the issue is because of the emoji in the attachment zip file name, specifically how the file name is being used to construct the file path. When testing on perm instances for both Centos and Linux and the ingestion worked fine with the emoji, which is the expected behavior since SOAR does ship to work with Unicode characters. However, when the environment variables are not set, the issue could be reproduced.

Resolution

Run locale on the command line and checking what the LANG and LC_ALL environment variables set to in your system. They should be set as "en_US.UTF-8"

$ locale
LANG=en_US.UTF-8
LC_ALL="en_US.UTF-8"