This document describes the possible causes and solutions for an exceptionally large disk usage for a Cisco EnergyWise Management (CEM, CEMS or CEWMS) controller installation. CEM uses the controller component in order to communicate with the endpoints or devices from which you want to retrieve or control energy-related information.
The space usage of the controller is normally quite limited and rather stable, but in some cases, the installation directory starts to grow and starts to eat up all the space. The space-problem can have multiple causes and the space usage can be in multiple subdirectories of the installation folder. Most common causes and their solutions can be found in the rest of the article.
1. Large Service/Directory due to Error Logs
Every time the controller quits unexpectedly or experiences an error, it automatically gets recovered by the System Watcher service and the end-user normally doesn't really notice that the error occurred.
When such exception occurs, a logfile (.log) and minidump (.mdmp) file is created in the service/directory of the controller installation directory (by default: C:\Program Files\Cisco Energy Management Controller\service). These files are around 350MB average in size and in case the controller service suffers from multiple exceptions over time, the space used by these files can grow to a considerably large amount as seen here:
The solution for the first cause is rather simple as you can just delete the .log and .mdmp files without any impact on the running controller. Keep in mind that there must be a reason why the files exist and if they appear on a regular basis. It is a good idea to start with the investigation of the cause of the log/dump generation.
2. Problems Cleaning up the Service/DB Directory
The service/db directory contains the logs of the device information received by the controller (by default: C:\Program Files\Cisco Energy Management Controller\service\db). Normally these files get managed and cleaned up automatically.
In some cases, the size of this directory starts to grow and (some) older files do not get cleaned up automatically anymore. Usually, the problem starts with one file and expands quite fast. This can be caused by (a) locked file(s) or too large files to delete. Once the problem appears, that one file causes issues for the remaining ones.
It is normal for this directory to contain some data. In order to check if you really experience an issue related to this, you can see if there are any files in the directory that are older than the DeleteOlderThan setting in settings.json (the default setting is 32 days).
The solution for the second cause requires a little more caution since you can potentially delete useful information or cause harm to a running controller instance. First, you need to ensure that the large files are actually older than the DeleteOlderThan setting in settings.json (the default setting is 32 days). If that's the case, you can perform these actions:
Stop the System Watcher service
Stop the Controller
Delete the contents of CEM Controller\service\db
Start the controller
Start the System Watcher service
3. Large Log/Directory due to the Verbosity of Logfiles or Unset Clean Cycle
It's considered normal behavior that the log/directory (by default: C:\Program Files\Cisco Energy Management Controller\logs) tends to grow over time. Especially when you do some testing/experimenting/upgrade/... there is a chance that the log/directory grows to a considerable amount of space and never shrinks. This might potentially cause problems.
The reason for the directory to grow large is that, by default, the Controller Log File Clean Cycle is disabled.
It's a good idea and a best practice to check these controller settings:
Controller Log Level:
The default Controller Log Level (INFO) is a good setting and gives enough information while you keep the used space to a fair amount. In case somebody changed the Log Level to a more verbose setting like DEBUG or TRACE, the logfiles grow rapidly in space. If there is no special reason for the Log Level to be more verbose, ensure that this setting is for INFO.
Controller Log File Clean Cycle:
The Controller Log File Clean Cycle is the number of days that logfiles must be kept in log/. By default, the clean cycle is disabled which means that the files stay there forever. In order to enable the clean cycle, change the default value of 0 to the number of days that you want to keep logfiles.
You can check and change the Controller Log Level and Controller Log File Clean Cycle in this location of the CEMS interface: Settings > Controllers > Edit Settings > Settings and as shown in the image: