The cli_common.getMgmtUri() routine fails to parse IPv6 management URIs (e.g. https://[::1]:8089), causing repeated TypeError exceptions and noisy ExecProcessor errors in splunkd.log

Applicability

Description

Installation of Splunk Enterprise v10.0.0 on a single instance is reporting ERROR ExecProcessor messages in splunkd.log every minute. These errors are traced back to 

/opt/splunk/bin/python3.9 /opt/splunk/etc/apps/search/bin/quarantine_files.py, specifically within the getMgmtUri function located in /cloudmark/splunk/lib/python3.9/site-packages/splunk/clilib/cli_common.py at line 504.

The exact error observed is:

TypeError: 'int' object is not iterable This error occurs on the line: if tmpProtocol in ['http', 'https'] and tmpHost and all(i in string.digits for i in tmpPort):

This indicates that the tmpPort variable, which is expected to be a string for character-by-character iteration (all(i in string.digits for i in tmpPort)), is instead being passed as an integer.

ERROR ExecProcessor [2560 ExecProcessor] - message from "/opt/splunk/bin/python3.9 /opt/splunk/etc/apps/search/bin/quarantine_files.py" File "/cloudmark/splunk/lib/python3.9/site-packages/splunk/clilib/cli_common.py", line 504, in getMgmtUri

This issue is a product defect related to type handling within the cli_common.py script, preventing it from correctly processing the management URI, regardless of whether it's an IPv4 or IPv6 address. The constant errors can impact system stability and the proper functioning of dependent scripts like quarantine_files.py.

Investigation

1. splunkd.log show ERROR ExecProcessor 
   

   ERROR ExecProcessor [2560 ExecProcessor] - message from "/opt/splunk/bin/python3.9 /opt/splunk/etc/apps/search/bin/quarantine_files.py" File "/cloudmark/splunk/lib/python3.9/site-packages/splunk/clilib/cli_common.py", line 504, in getMgmtUri

   messages every minute, pointing to cli_common.py and quarantine_files.py.
2. The splunkd_uri.txt file contained https://[::1]:8089, leading to an initial assumption of an IPv6 parsing issue in cli_common.py.
3. Manually editing /opt/splunk/var/run/splunk/splunkd_uri.txt to https://127.0.0.1:8089 temporarily resolved the error, but the file reverted to https://[::1]:8089 after a Splunkd restart.
4. The TypeError indicates that the tmpPort variable is an integer, but the Python code expects a string to iterate over its digits. This is a fundamental type mismatch known issue within the Splunk core script. Even if the mgmtHostPort was correctly configured to force IPv4, this TypeError would still occur if tmpPort is an integer at that point in the code.
5. The system has listenOnIPv6 = yes configured, contributing to the default preference for IPv6 loopback ([::1]) when mgmtHostPort is not correctly applied. The python.version = force_python3 setting confirms the use of Python 3.9, which is consistent with the error path.

Resolution

1. Correct mgmtHostPort Configuration:

   • Edit /opt/splunk/etc/system/local/server.conf.
   • Under the [general] stanza, ensure mgmtHostPort is set without the scheme:
      

     [general]
     mgmtHostPort = 127.0.0.1:8089

   • Restart the Splunkd service.
   • Verify if /opt/splunk/var/run/splunk/splunkd_uri.txt now contains https://127.0.0.1:8089 and check splunkd.log for the TypeError. This step aims to address the IPv6 preference, but may not resolve the underlying TypeError.

The TypeError is a core code known issue that requires a fix from Splunk to ensure the tmpPort variable is correctly handled as a string within the getMgmtUri() function.

This issue is will be fixed in v 10.2.x, 9.4.14.