Splunk Add-on for Microsoft Cloud Services (MSCS) not able to read files due to Content-Range header ERRORs.

Applicability

Description

The Splunk Add-on for Microsoft Cloud Services (MSCS) is encountering issues reading files due to errors related to the Content-Range header.

Investigation

While checking internal logs using SPL:

SPL:

index=_internal *mscs_storage_blob* log_level=WARN*

Logs:

2024-08-16 23:13:29,676 +0000 log_level=WARNING, pid=1497, tid=ThreadPoolExecutor-0_4, file=mscs_storage_blob_data_collector.py, func_name=_do_collect_data, code_line_no=252 | [stanza_name="azure_cloudflare_blobs" account_name="cflogssa" container_name="cf-http-logs" blob_name="20231120/20231120T143749Z_20231120T143849Z_ac586a77.log.gz" blob_type="BlockBlob" blob_mode="append" blob_compression="not_compressed"] Failed to collect blob.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 204, in _do_collect_data
    blob_client=blob_client, offset=offset, length=length
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_util.py", line 362, in wrapper
    return func(self, *args, **kwargs)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 325, in _get_blob
    raise e
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 301, in _get_blob
    if_match=if_match,
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_blob_client.py", line 942, in download_blob
    return StorageStreamDownloader(**options)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_download.py", line 367, in __init__
    self._response = self._initial_request()
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_download.py", line 432, in _initial_request
    raise ValueError("Required Content-Range response header is missing or malformed.")
ValueError: Required Content-Range response header is missing or malformed.
2024-08-16 23:13:29,682 +0000 log_level=WARNING, pid=1497, tid=ThreadPoolExecutor-0_7, file=mscs_storage_blob_data_collector.py, func_name=_do_collect_data, code_line_no=252 | [stanza_name="azure_cloudflare_blobs" account_name="cflogssa" container_name="cf-http-logs" blob_name="20231121/20231121T011142Z_20231121T011240Z_d5bff9b0.log.gz" blob_type="BlockBlob" blob_mode="append" blob_compression="not_compressed"] Failed to collect blob.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 204, in _do_collect_data
    blob_client=blob_client, offset=offset, length=length
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_util.py", line 362, in wrapper
    return func(self, *args, **kwargs)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 325, in _get_blob
    raise e
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/mscs_storage_blob_data_collector.py", line 301, in _get_blob
    if_match=if_match,
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_blob_client.py", line 942, in download_blob
    return StorageStreamDownloader(**options)
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_download.py", line 367, in __init__
    self._response = self._initial_request()
  File "/opt/splunk/etc/apps/Splunk_TA_microsoft-cloudservices/lib/azure/storage/blob/_download.py", line 432, in _initial_request
    raise ValueError("Required Content-Range response header is missing or malformed.")
ValueError: Required Content-Range response header is missing or malformed.

 
Found that while configuring input with the default method as per Add-on configurations it uses Access Key to fetch data and using the Access Key method by default it does not create Headers in the API calls.

Resolution

--> Need to change the authentication method from Access Key to SAS Token, as mentioned below :
Headers in Storage Blob.

Related Links

Reference link for ERRORs