Message Trace input stopped working and getting "urllib3.exceptions.ReadTimeoutError" error.

The `splunk_ta_o365_cloud_messagetrace.log` was investigated, and a timeout error was observed while connecting to the Microsoft API. The affected sourcetype is `o365:reporting:messagetrace`.

2024-02-05 16:24:16,986 level=INFO pid=656879 tid=MainThread logger=splunk_ta_o365.modinputs.cloud_app_security pos=cloud_app_security.py:run:63 | datainput=b'cloud_app_sec_alerts' start_time=1707150256 | message="Error retrieving Cloud Application Security messages." exception=replace() argument 2 must be str, not None
2024-02-05 16:24:16,986 level=ERROR pid=656879 tid=MainThread logger=splunk_ta_o365.modinputs.cloud_app_security pos=utils.py:wrapper:72 | datainput=b'cloud_app_sec_alerts' start_time=1707150256 | message="Data input was interrupted by an unhandled exception."
Traceback (most recent call last):
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/splunksdc/utils.py", line 70, in wrapper
return func(*args, **kwargs)
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/cloud_app_security.py", line 225, in run
return consumer.run()
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/modinputs/cloud_app_security.py", line 47, in run
self._cas_portal_data_center,
File "/opt/splunk/etc/apps/splunk_ta_o365/bin/splunk_ta_o365/common/portal.py", line 913, in o365_cloud_app_security_call
url = url.replace("tenant_subdomain", cas_portal_url)
TypeError: replace() argument 2 must be str, not None

The `splunkd.log` was investigated, but unfortunately, nothing related to the message trace input was found. Subsequently, the message trace logs were examined in splunk_ta_o365_message_trace_exchange_message_trace.log, revealing the following error.

2024-02-01 21:00:57,503 level=ERROR pid=4191500 tid=MainThread logger=splunk_ta_o365.modinputs.message_trace pos=__init__.py:run:376 | datainput=b'exchange_message_trace' start_time=1706821060 | message="An error occurred while collecting data" stack_info=True
Traceback (most recent call last):
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/opt/splunk/lib/python3.7/http/client.py", line 1373, in getresponse
response.begin()
File "/opt/splunk/lib/python3.7/http/client.py", line 319, in begin
version, status, reason = self._read_status()
File "/opt/splunk/lib/python3.7/http/client.py", line 280, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/opt/splunk/lib/python3.7/socket.py", line 589, in readinto
return self._sock.recv_into(b)
File "/opt/splunk/lib/python3.7/ssl.py", line 1079, in recv_into
return self.read(nbytes, buffer)
File "/opt/splunk/lib/python3.7/ssl.py", line 937, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed outDuring handling of the above exception, another exception occurred:Traceback (most recent call last):
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/requests/adapters.py", line 499, in send
timeout=timeout,
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 788, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/packages/six.py", line 770, in reraise
raise value
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 710, in urlopen
chunked=chunked,
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 451, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/opt/splunk/etc/apps/splunk_ta_o365/lib/urllib3/connectionpool.py", line 341, in _raise_timeout
self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='reports.office365.com', port=443): Read timed out. (read timeout=120)