CUIC Local NTP Client Syslog Message Troubleshooting

Introduction

This document describes a problem that is encountered with the Cisco Unified Intelligence Center (CUIC) when the local Network Time Protocol (NTP) client is off by more than the acceptable threshold of three seconds from its remote NTP system peer. Solutions to this problem are also described.

Problem

You receive this syslog message:

The local NTP client is off by more than the acceptable threshold of 3 seconds from its remote NTP system peer. The normal remedy is for NTP Watch Dog to automatically restart NTP. However, an unusual number of automatic NTP restarts have already occurred on this node. No additional automatic NTP restarts will be done until NTP time synchronization stabilizes. This is likely due to an excessive number of VMware Virtual Machine migrations or Storage VMotions. Please consult your VMware Infrastructure Support Team.

Solution

This section describes two possible causes for the problem and the solutions.

First Cause

The syslog is filled with the previously described NTP messages because you have more than one NTP server configured, and they might not be properly synced.

Resolution

You must remove the additional NTP servers and generate a new license. Once this is complete, the NTP syslog messages are cleared.

Second Cause

There is no available space in the CUIC.

Resolution

With CUIC Version 9.0(1), new CLI commands are introduced so that you can delete the files via normal CLI access. Therefore, you do not need root access through the Cisco Technical Assistance Center (TAC).

Enter the file delete activelog /cuic/logs/jmx/*.log noconfirm command on the secondary node in order to delete the jmx.log files. In this example, the disk usage is reduced to 77 percent after the command is entered:

                        Total            Free            Used
Disk/active         27087452K       15555208K       11257052K (42%)
Disk/inactive       27087420K       14989416K       11822812K (45%)
Disk/logging        94235980K       20874792K       68574224K (77%)

Enter these commands on the primary node in order to delete the CUIC logs for 2013:

• file delete activelog /cuic/logs/cuic/CCBU-cuic.2013* noconfirm
• file delete activelog /cuic/logs/cuic/ERROR-cuic.2013* noconfirm

For future issues, you can enter the show disk usage common directory sort command in order to identify the directories that have the largest size and delete the log files based on the output. However, the deletion of the files described in this section is usually sufficient.

RTMT Issues

It is possible that the Real Time Monitor Tool (RTMT) might freeze when you attempt to open the syslog messages. This occurs because the size of the syslog message is 123 MB, which is too large.

With root access, you can navigate to var > log > active > syslog > messages and clean the content of the file. Once this is complete, the RTMT is able to open the file.

Note: You might have to restart the NTP and confirm that the messages are logged in.

Revision History