Field Notice: FN - 72548 - Firepower Software: Log Files Might Consume Excessive Disk Space - Software Upgrade Recommended

Notice

THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.

Revision History

Revision	Publish Date	Comments
1.0	23-Mar-23	Initial Release

Products Affected

Affected OS Type	Affected Software Product	Affected Release	Affected Release Number	Comments
NON-IOS	Firepower Threat Defense (FTD) Software	7.0	7.0.0, 7.0.0.1, 7.0.1, 7.0.1.1, 7.0.2, 7.0.2.1, 7.0.3, 7.0.4
NON-IOS	Firepower Threat Defense (FTD) Software	7.1	7.1.0, 7.1.0.1, 7.1.0.2
NON-IOS	Firepower Threat Defense (FTD) Software	7.2	7.2.0, 7.2.1, 7.2.2
NON-IOS	Firepower Threat Defense (FTD) Software	7.3	7.3.0
NON-IOS	Firepower Management Center Software	7.0	7.0.0, 7.0.0.1, 7.0.1, 7.0.1.1, 7.0.2, 7.0.2.1, 7.0.3, 7.0.4
NON-IOS	Firepower Management Center Software	7.1	7.1.0, 7.1.0.1, 7.1.0.2
NON-IOS	Firepower Management Center Software	7.2	7.2.0, 7.2.1, 7.2.2
NON-IOS	Firepower Management Center Software	7.3	7.3.0

Defect Information

Defect ID	Headline
CSCwb34240	Log rotate failure of files process_stdout.log and process_stderr.log - syslog-ng. High disk usage
CSCwd09341	Multiple log files have zero bytes due to logrotate failure
CSCvy26511	Tune unmanaged disk alert thresholds for low end platforms
CSCwd87227	High disk usage due to process_stdout.log and process_stderr.log logrotate failure (no rotation)
CSCwc41661	High disk usage due to process_stdout.log and process_stderr.log logrotate failure (deleted files)
CSCvu32541	Add maxsize to pm.logrotate in Firepower Threat Defense

Problem Description

Some versions of Firepower software might cause log files to consume excessive disk space on the Cisco Secure Firewall and Cisco Secure Firewall Management Center (FMC).

Background

The Firepower software uses a log file rotation (logrotate) process to manage log files that are stored on the disk of the Cisco Secure Firewall and Cisco Secure FMC. The log file management process maintains recent log files while older log files are flagged for deletion in order to conserve disk space.

For some versions of Firepower software, the log file rotation pointers for the process_stdout.log and process_stderr.log log files are not reset correctly, which causes log files to consume additional disk space and prevent older hidden log files from being automatically deleted. This condition might result in the consumption of most or all of the available disk space on the system.

Problem Symptom

There are two symptoms for the issue described in this field notice.

For both cases, if the log files consume all available space on the system disk drive the user might be unable to log into the device and a system reload might be required to regain login access.

Health alerts will be indicated when the disk consumption becomes excessive. The Secure Firewall will indicate "High unmanaged disk usage on /ngfw" and the Secure FMC will indicate "High unmanaged disk usage on /Volume".

• For systems that operate (or that have previously operated) with affected Firepower software versions 7.0.x and 7.1.x, the process_stdout.log and process_stderr.log log files continue to log events and consume excessive disk space after they are flagged for deletion and become hidden files. These files can be viewed with this command:
   

  Firepower #> lsof | grep deleted | grep process_std

syslog-ng   638                        root   33w      REG              253,7    124121812     527928 /var/log/process_stdout.log.1 (deleted)

syslog-ng   638                        root   34w      REG              253,7 161211401889     527776 /var/log/process_stderr.log.1 (deleted)

  In addition, the active process_stdout.log and process_stderr.log log files will show zero bytes in size. These files can be viewed with these commands.

  For Firepower platforms:

  Firepower #> ls -al /ngfw/var/log/process_std*
  
  -rw-rw----  1 1000000511 User     0 Jun  5 04:02 process_stderr.log
  -rw-rw----  1 1000000511 User     0 Jun  5 04:02 process_stdout.log

  For FMC platforms:

  Firepower #> ls -al /var/log/process_std*
  
  -rw-rw----  1 1000000511 User     0 Jun  5 04:02 process_stderr.log
  -rw-rw----  1 1000000511 User     0 Jun  5 04:02 process_stdout.log

• For systems that have only operated with affected Firepower software versions 7.2.x and 7.3.0 (were not upgraded from affected Firepower software versions 7.0.x or 7.1.x), the process_stdout.log and process_stderr.log active log files will continue to log events and consume excessive disk space without being rotated and flagged for deletion. These files can be viewed with these commands:
   

  For Firepower platforms:

  Firepower #> ls -al /ngfw/var/log/process_std*
  
  -rw-rw----  1 1000000511 User     161211401889 Jun  5 04:02 process_stderr.log
  
  -rw-rw----  1 1000000511 User     124121812    Jun  5 04:02 process_stdout.log

  For FMC platforms:

  Firepower #> ls -al /var/log/process_std*
  -rw-rw----  1 1000000511 User     161211401889 Jun  5 04:02 process_stderr.log
  -rw-rw----  1 1000000511 User     124121812    Jun  5 04:02 process_stdout.log

  Note: Unless you migrate to a fixed version Firepower software release, the process_stdout.log and process_stderr.log log files that consume excessive disk space will remain on the system disk drive after Firepower software version upgrades or downgrades.

Workaround/Solution

Solution

Cisco recommends to upgrade the Firepower software to one of the versions shown in this table.

Release Version	Fixed Version
7.0.0, 7.0.0.1, 7.0.1, 7.0.1.1, 7.0.2, 7.0.2.1, 7.0.3, 7.0.4	7.0.5 or later
7.1.0, 7.1.0.1, 7.1.0.2	Upgrade to a fixed release
7.2.0, 7.2.1, 7.2.2	7.2.3 or later
7.3.0	7.3.1 or later

 

Note: The fixed version Firepower software will remove the process_stdout.log and process_stderr.log log files on the disk drive during the installation process.

For More Information

If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:

• Open a service request on Cisco.com
• By email or telephone

Receive Email Notification For New Field Notices

My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.