Management Activity input stopped collecting new logs configured in Splunk Add-on for Office 365

Applicability

Description

The Management Activity input that utilizes Office 365 Management Activity API to fetch the logs and is configured in the Splunk Add-on for Office 365 is encountering a failure, accompanied by the error message: "Open checkpoint failed. cannot unpack non-iterable int object".

Another observation made was that the other input configured in the same Add-on were working as expected

Investigation

Upon investigating and checking the internal logs, with the below search query to check if there are any error messages in the _internal logs:

SPL:

index=_internal sourcetype="splunk:ta:o365:log" datainput="b'<input_name>'" level=ERROR

Found the below error messages which are generated repeatedly whenever a Management Activity input is executed. The error message reads as below:

Error Logs:

2024-08-14 21:08:48,909 level=ERROR pid=4002853 tid=MainThread logger=splunksdc.checkpoint pos=checkpoint.py:open_always:174 | datainput=b'<input_name>' start_time=1723684128 | message="Open checkpoint failed. cannot unpack non-iterable int object" path=b'/splunk/splunk/var/lib/splunk/modinputs/splunk_ta_o365_management_activity/<input_name>.ckpt'

On further validation and debugging of the internal logs, it was discovered that the root cause of the issue lies within the checkpoints, which became corrupted.

Consequently, whenever an input invocation occurred, it failed with the error: "Open checkpoint failed. cannot unpack non-iterable int object".

This error arose because the input fetched the last checkpoint value available to pull new logs from the Management Activity API.

Resolution

The issue was resolved by re-creating a new input through the Splunk Add-on for Office 365 GUI(Graphical User Interface).