How to manage container logs with Logrotate on Linux

Why container logs consume so much disk space

When running isolated applications in virtualized environments on Linux, every message printed to the console is recorded by the execution engine into plain text files on the host operating system. In practice, this means if an application is very chatty or suffers from repetitive errors in a loop, these files grow exponentially until they occupy all available space on the partition. As a result, the server can crash unexpectedly, refuse new connections, and corrupt databases due to a lack of space for temporary writes.

Managing this mass of textual data requires an automated strategy, since no team can manually clean up files in the middle of the night when the disk fills up. This exact scenario is where Logrotate comes in, a native Unix and Linux utility designed to automate the rotation, compression, and removal of old log files. Before implementing it, remember that mastering auxiliary sorting tools, such as how to use awk to filter logs in Linux terminal, helps identify which services are generating excessive noise even before applying the cleanup policy.

How Logrotate works in practice

Logrotate operates cyclically, usually triggered once a day by a system task scheduler. When a file reaches a specific size or the clock hits the rollover moment, the program renames the current file, creates a new empty file so the application can continue writing without interruption, and compresses the old file using efficient compression algorithms like gzip. In practice, this means you keep a recent accessible history and discard old data completely autonomously.

To configure this behavior, the utility reads configuration files located in the central system folder or service-specific directories. Each rule defines the time interval or size limit that triggers the action, as well as determining how many old versions should be kept before final deletion. This approach prevents history from growing indefinitely, ensuring that disk consumption remains strictly within a safe and predictable range throughout months of continuous operation.

Step-by-step guide to configuring log rotation

If you want to apply a rotation policy for text files generated by containers that store their data directly on the host filesystem, follow the procedure below carefully. First, make sure the utility is installed and create a dedicated configuration file inside the appropriate directory using your favorite text editor in the terminal.

1. Open the terminal with administrative privileges and create a configuration file for the containers:

   sudo nano /etc/logrotate.d/docker-containers

2. Insert the rotation guidelines specifying the file paths, daily frequency, retention, and compression:

   /var/lib/docker/containers/*/*.log {
       daily
       count 7
       compress
       delaycompress
       missingok
       notifempty
       copytruncate
   }

3. Save the file and force a test run to validate that the syntax is correct without errors:

   sudo logrotate -f /etc/logrotate.d/docker-containers

Essential cautions when rotating container logs

A common mistake made by beginner administrators is applying Logrotate directly to files opened by running processes without using the correct truncation directive. In practice, if you merely move or delete the log file without notifying the process, the application will continue writing to the old unlinked file descriptor, meaning disk space will not actually be freed until the container is restarted. The copytruncate directive resolves this dilemma by copying the current content to the backup file and then emptying the original file, allowing the flow to continue without pauses.

Another critical point involves the retention volume configured in the rules. Keeping dozens of compressed copies might seem safe, but if the daily data volume is massive, accumulated storage can still exhaust the disk in a short time. Therefore, define realistic limits based on your partition's total capacity and the actual need for historical auditing of running applications. To complement this implementation in production environments, see our related technical analyses on Shorted SMD capacitor? How to test on board with a meter and How to measure surface temperature with thermocouple and ESP32.

Final considerations on storage health

Keeping the container environment clean and organized is a fundamental pillar to ensure the stability of any modern infrastructure, whether in corporate servers or local test environments. The proper use of Logrotate combined with good volume planning and continuous monitoring eliminates unpleasant surprises caused by lack of disk space. By automating rotation and compression, the engineering team gains precious time to focus on developing new features, knowing the system foundation operates resiliently and autonomously.