Unable to generate diag - "UnicodeDecodeError: 'utf-8' codec can't decode byte x in position y: invalid start byte"

Applicability

Splunk Enterprise

Description

When the following error appears: 

File "/opt/splunk/lib/python3.7/site-packages/splunk/clilib/cli_common.py", line 126, in bom_aware_readline
if sys.version_info >= (3, 0): return line.decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte x in position y: invalid start byte

 

Investigation

In the output of the text file created by the failed diag generation this verbiage is included: 

UnicodeDecodeError: 'utf-8' codec can't decode byte x in position y: invalid start byte

Resolution

The problem is caused by non-ASCII/UTF-8 characters, that are present in your configuration and are not supported.
You can remediate the problem:

1. Either remove non-ASCII/UTF-8 characters from your configuration files.
2. Or take a backup of '$SPLUNK_HOME/lib/python3.7/site-packages/splunk/clilib/cli_common.py' and in line 127:
Add parameter to "line.decode" - either "errors='replace' ", or "errors='ignore' ".

Eg:

line.decode(errors='replace')

 

Related Links

https://docs.splunk.com/Documentation/Splunk/8.2.5/Admin/Howtoeditaconfigurationfile
https://docs.splunk.com/Documentation/Splunk/8.2.6/ReleaseNotes/KnownIssues
https://docs.splunk.com/Documentation/Splunk/9.0.1/ReleaseNotes/KnownIssues

Revision History

Revision	Publish Date	Comments
1.0	29-Sep-2026	Initial Release