Lock down your logfiles with logrotate
Setting the Records Straight
The simple act of logging can create management and storage nightmares. Logrotate brings creative solutions to your logging needs.
When you are new to running servers and rightly worried about keeping downtime to a minimum, you inevitably find yourself facing a few problems – some predictable and others not so much. A particular aspect of systems that repeatedly causes consternation is logging and its concomitant storage.
To my mind, logging is one of the truly operational aspects of being a sys admin. Even with the best automation in the world, to get system logging right, you need to invest a little forethought along with very occasional housekeeping. After attempting to diagnose a few system problems, you soon come to realize that logs are absolutely essential. I'll reiterate that last statement by saying that they are not just important, they are truly key to keeping your servers functioning correctly.
Logfiles come in all different shapes and sizes and hold all kinds of information, both useful and relatively useless. Some logs are inconsequential if lost, whereas others are so critical to the operation of a server that their presence can mean the difference between a full server rebuild (sometimes adding several days of additional work to your busy week) or a five-minute health check.
With experience, you develop a level of trust in what your logs deliver and a degree of comfort in the level of detail you need to answer a number of questions relating to the kinds of problems you are often called on to investigate. How important is getting that level of detail correct? A few years ago, my least-favorite dialog box (on a certain popular GUI-based operating system) inconveniently popped up and reported the following: There has been an error.
No matter which way you squint, that dialog box isn't going to help much in solving what might be a life or death dilemma – in the sense of ongoing employment at least. The message I'm attempting to convey is: Logs are not only critical to the operation of your server, but they are also an aspect of your systems that needs more than just a flying blind, brief amount of consideration. Without a high level of logging detail, you are inevitably caught out unwittingly when presented with a large number of red herrings (i.e., clues) that lead you down the wrong path in your diagnosis.
Conversely, with a massive amount of detail your disks fill up so quickly that your remote server, at best, complains endlessly when trying to run its applications or, at worst, simply stops working, requiring expensive onsite engineers with hands-on access to recover it.
Doom-mongering aside, a Unix-flavored package called logrotate is so powerful that it swiftly takes care of many of the problems commonly associated with logging and storage without breaking a sweat. That said, it is still important to keep an eye on how your logs are behaving periodically, even with such a sophisticated tool in your toolkit.
Pause for Breath
Imagine a software package with functionality that lets you specify lots of well-considered parameters, including whether to send email after logfiles have reached a certain number, whether to compress logfiles to save disk space, and whether logs should be ignored or highlighted if they have no content (i.e., zero-byte files). The most important of all logrotate's features, however, is revealed by its name: The simple functionality of rotating logs is key to keeping your servers online.
Any additional bells and whistles won't be as useful to you as the simple act of purging old logfiles that are simply not useful any more to reduce disk space usage. Logrotate is so flexible that it can manage all of the above-mentioned tasks with the logfiles of almost any software package, making it a fantastic weapon in your sys admin arsenal.
The end result is peace of mind that your disks will never fill up, causing poor server performance. Coupled with the right level of logging (which is usually set within the configuration parameters of your applications and not within logrotate), you can, for all intents and purposes, strike logging off your daily housekeeping to-do list and relegate it to your monthly to-do list instead.
En Suite
To say the superhero that is logrotate is feature-filled is certainly no understatement. For a number of years now, logrotate has been so warmly embraced by the Linux community that many packages in the repositories ship conveniently with a ready-made logrotate profile that is dropped dutifully into the correct directory, with the path /etc/logrotate.d
.
The result is that, give or take some very infrequent tweaks, everything related to pruning and purging your critical logs is taken care of automatically for you. Additionally, I have seen some less mature packages include an optional profile, even if it's not dropped into place when the package is installed.
However, until you delve into the multiple configuration options, you might not realize exactly what you should be doing with your logs. Now that I've established why it's so important to keep logs ticking over correctly to prevent disaster, I'll run through some basic logging scenarios, as well as a few that might not have occurred to you.
Status Quo
My systems run my preferred flavor of Linux – namely, Debian on my servers and Ubuntu on my desktops. However, like so many Linux packages, logrotate is probably nearly identical to use across all the popular distros, barring a few file path changes. Logrotate likely already exists on your system, but you might want to check at this stage with:
# apt-get install logrotate
Before forging ahead, I'll briefly mention where to check the innards of the package. A "state" file remembers who did what and when, in terms of when it last checked, changed, or rotated a log. That file is /var/lib/logrotate/status
(Figure 1), which stores the last rotation recorded for each logfile by date.
Also, you don't have to worry about triggering logrotate, it simply runs as a cron job, sitting neatly inside the /etc/cron.daily/
directory. Thanks to cron's portability, then, you can simply move the 14 lines or so of configuration to another cron file and run it whenever you want.
The easiest way to dip your toe into the water with logrotate is to start looking at some of the bundled config files, which demonstrate how much of the hard work is already done for you by this well-written software package.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Fedora KDE Approved as an Official Spin
If you prefer the Plasma desktop environment and the Fedora distribution, you're in luck because there's now an official spin that is listed on the same level as the Fedora Workstation edition.
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.
-
Fedora 41 Released with New Features
If you're a Fedora fan or just looking for a Linux distribution to help you migrate from Windows, Fedora 41 might be just the ticket.
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.