Cisco Tetration Analytics software advisory notice due to bugs CSCvn30664, CSCvn37738, CSCvn46417, CSCvn54430, and CSCvn56472
PDF(90.8 KB) View with Adobe Reader on a variety of devices
Updated:December 14, 2018
Software Advisory Notice
Dear Cisco Customer,
Cisco has identified software issues with the release that you have selected that may affect your use of this software. Review this Software Advisory notice to determine if the issues apply to your environment. If these issues are applicable, upgrade or apply the patch release specified in the "Software solution – Fixed software version" column in the table below.
For more comprehensive information about what is included in this software, refer to the Cisco software Release Notes, available from the Product Selector tool. From this page, choose the product in which you are interested. The Release Notes are under "General Information" on the product page.
Affected Software and Replacement Solution for CSCvj03179, CSCvn37738, CSCvn46417, CSCvn54430, and CSCvn56472
Fixed Software versions
Patch release 126.96.36.199
CSCvn37738, CSCvn54430, CSCvn56472
Patch release 188.8.131.52
2.0, 2.1, 2.2, 2.3.1, 184.108.40.206, 220.127.116.11
Patch releases 18.104.22.168 and 22.214.171.124
Reason for Advisory:
This software advisory addresses five software issues.
Cisco Tetration Software Release version mentioned in the above table.
The sensor ID for a given Cisco Tetration agent is generated by the Cisco Tetration backend service using a combination of the hostname and system UUID. A recent change in the dmidecode command , which was called by the agent to get the system UUID, now causes the system UUID to get returned in lowercase (the UUID was previously returned in uppercase). This change results in a different sensor ID for the same running agent.
If you have a Cisco Tetration agent and you upgraded the dmidecode command, you will see duplicate agent entries in the Cisco Tetration UI for a given host.
This issue has been observed with an upgrade from the RHEL 7.5 release to the 7.6 release, but can be seen with any distribution that contains this dmidecode command being upgraded from an older version to version 3.1 or later.
You can observe this issue if you upgrade the Linux host to a version that contains the updated dmidecode (version 3.1 or later) package and restart the existing agent or reboot the host.
Alternately, upgraded dmidecode to version 3.1 or newer from an older version without upgrading the system OS.
Do not upgrade the dmidecode package to 3.1 or later and do not upgrade the system to the new Linux version that contains the dmidecode package 3.1 or later until Cisco Tetration contains a fix.
Support for the 3.1 or later dmidecode package is available in the Cisco Tetration 126.96.36.199 patch release. Disable the Cisco Tetration agent auto-upgrade until you update your Cisco Tetration cluster to the 188.8.131.52 release.
The Cisco Tetration agent MSI installer uses Microsoft Windows Installer services to perform installations during upgrades. If the installer encounters a file from a previous agent installation that it cannot overwrite or remove, the installer will queue this file and request a reboot from the OS.
In the 184.108.40.206 release, this issue will trigger a reboot immediately. With the 220.127.116.11 release, this issue will prevent the reboot, but the files will still be pending and waiting for a manual reboot.
With the 18.104.22.168 release, this pending reboot can lead to mixed code running in the environment if the files queued for replacement by the Microsoft Windows Installer are executables. If the system is manually rebooted, the installation will finish, the pending files will be copied, and the agent will be fully running on the updated code.
You can observe this issue if a file in the Cisco Tetration installation folder is locked as read-only by third party software, such as an anti-virus software. In this situation, the file cannot be replaced or removed during an upgrade from an older sensor release to the 22.214.171.124 or 126.96.36.199 sensor release when you use the Cisco Tetration GUI.
Disable the Microsoft Windows agent auto-upgrade while upgrading to the 188.8.131.52 release, and only upgrade after the fix is available in the 184.108.40.206 release.
For 220.127.116.11 release sensor installations, you should search the "Cisco Tetration" folder for the existence of *.tmp files. If a .tmp file exists, the sensor might not be fully installed and a server reboot is pending. You can verify this in the Event Viewer as well by looking under Windows Logs > Application and searching for event 1029 from the "Msiinstaller" source. The event message is as follows:
Product: Cisco Tetration Agent. Restart required. The installation or update for the product required a restart for all changes to take effect. The restart was deferred to a later time.
Cisco Tetration is impacted with a Hadoop bug in which inodes and blocks are leaked with certain APIs. These leaked inodes and blocks accumulate over time and cause slowness in the cluster.
You can observe this issue on clusters that have been running longer than 3 months and the total blocks count is higher than the total files count by more than 10 million in the HDFS monitoring dashboard. Use the following procedure to check the total blocks count and total files count in Hawkeye:
1.Under the main GUI, choose Monitoring.
2.Click on Hawkeye [Charts].
3.Choose the "HDFS monitoring" dashboard.
If you are using the 18.104.22.168 or 22.214.171.124 release, upgrade to the 126.96.36.199 patch release.
If you are on a 2.3.1 release, upgrade to the 188.8.131.52 patch release.
The Cisco Tetration agent MSI installer uses the Microsoft Windows Installer services to perform installations during upgrades. If WinPcap already exists in the system, the installer cancels the installation and prints the following error:
MSI (s) (E4:5C) [02:53:50:004]: Product: Cisco Tetration Agent -- WinPcap found, please uninstall WinPcap first before continue
In the Cisco Tetration 2.3.1 releases, the installation does not abort when detecting WinPcap. Instead, Npcap will be installed regardless.
You can observe this issue if WinPcap is already installed on the system when you install the Cisco Tetration agent MSI.
If WinPcap is not required by any applications, you can manually uninstall WinPcap and retry the Cisco Tetration agent installation.
Otherwise, if WinPcap is required, then Cisco recommends that you wait for the fix that will be in the Cisco Tetration 184.108.40.206 release. The fix installs Npcap in WinPcap compatible mode, which enables the installation to succeed.
In the Microsoft Windows installer script (in powershell), the Out-File cmdlet writes to the user.cfg file in Unicode encoding by default. This could prevent some scripts, such as fetch_sensor_id.cmd, from being able to read the user.cfg file. This issue prevents the agent from registering with the Cisco Tetration cluster.
You can observed this issue you install the agent with the PowerShell installer script for Microsoft Windows from the Software Agent Download page and the agent has not registered with the cluster.
Open the user.cfg file in the "C:\Program Files\Cisco Tetration" folder with notepad.exe and Save-As ANSI format with the same filename to overwrite the Unicode version. Re-run the fetch_sensor_id.cmd batch file as an administrator to register the sensor.
This issue will be fixed in the 220.127.116.11 patch release. Until the patch is available, disable the Microsoft Windows Cisco Tetration agent auto upgrade or follow the above workaround.