All Systems Go

Some people swear by systemd [1], the new systems and services manager. Others swear at it – loudly and frequently. However, now that systemd is starting to be installed by default in most major distributions, you probably need to begin to familiarize yourself with at least its basic concepts, including its basic commands and how it handles runlevels, logs, and starting and stopping systems.

To be sure, the latest version of Debian lets users install without systemd by using preseed [2] or adding an argument at the boot menu [3], but at that point learning systemd seems easier than avoiding it. Additionally, that option may not be available in future releases.

If you have not been following the controversy about systemd, you may wonder what the fuss is about. After all, systemd is generally summarized as a replacement for init [4], the process that controls other processes. Ubuntu's replacement for init, called upstart [5], did not meet with anything like the same opposition, so why should systemd?

Ignoring the clashes of personality involved, the purely technical answer is that describing systemd as an init replacement is like calling a computer an adding machine: The explanation is true but is so limited as to be misleading. The truth is, systemd not only manages the bootup process, but pretty much every other major aspect of your system, introducing dozens of new commands (Table 1) and accompanying configuration files.

These changes are so extensive that typing man system.d.index lists 134 new man pages, many of which have their own options. An orderly naming system for commands and files helps (e.g., systemd commands mostly end with ctl), but systemd may well be the largest set of changes to Linux ever.

Some developers complain that systemd is so all encompassing that it violates the standard Unix tenet of developing one tool for one task. Also, systemd enforces the Linux Standards Base [6] for file hierarchies, reducing long-established directories such as /bin and /sbin to symbolic links that point to subdirectories of /usr. Faced with such changes, groups such as the Debian fork Devuan [7] are determined to avoid being "locked in the overwhelming web of dependencies of systemd" and are building a distribution that continues to use init.

By contrast, KDE developer Aaron Seigo says, "The problems with the venerable SysV init were becoming more and more evident. Systemd is a replacement that modernizes the init system, but just as importantly has consensus support among distributions."

Considering the amount of information to absorb, having an informed opinion about systemd is difficult. However, the best way to judge the issues is to explore systemd more closely until you can form your own conclusions.

Some Systemd Basics

Systemd manages 12 different types of objects required for bootup and system operation. These objects are called units (Table 2). Typical units include a mount unit for each filesystem, a service unit for each daemon, a sockets unit for the CUPS printing system, and a device unit for mounting a sound card. However, the average system has dozens of units – 156 on the computer I am using to write this article. Many have configuration files with similar names.

At any given time, a unit may be in one of five main states: active, inactive, activating, deactivating, or failed. Systemd also manages positive dependencies (what must be present) and reverse dependencies (what cannot be present) for units. The state of units can be changed by applications or other units; in particular, many are created during bootup.

For convenience, units are arranged in control groups, which are defined in the kernel. The processes of control groups are readable in fs//sys/fs/cgroup/systemd/ or using the ps command. Units or control groups receive requests for action from applications, which are known as jobs in systemd.

Contrary to widespread comprehension, this structure remains largely compatible with init. From a user's perspective, files such as /etc/fstab or /etc/group remain much the same. In fact, systemd is aliased to init when used at bootup.

The main difference is that control of the system is centralized to a far greater degree than under init. Much of this control is centralized in the new systemctl command. Systemctl's basic command structure is:

systemctl [OPTIONS] COMMAND [UNIT / OTHERTARGET]

The plain command systemctl lists all units on the system (Figure 1). Units can be specified by a comma-separated list or by using options to specify a type of unit, such as --type=UNITS or --state=STATE (Figure 2).

Figure 1: Without options, commands, or units specified, systemctl lists all units on the system.

Figure 2: Here, systemctl shows socket units on the system.

The command systemctl start UNITS activates units, and similar commands stop, restart, and disable units. For troubleshooting, a useful option is systemctl --failed, which shows the units that failed during bootup. Other combinations of commands and options create and control snapshots, jobs, or systemd environment variables, as specified in the systemctl man page.

If no units are specified, then the command given is run on the entire system. In everyday system administration, this is the situation under which systemctl is most likely to be used. The most common standalone commands are halt, poweroff, reboot, suspend, and hibernate, which are often given symbolic links to the pre-systemd separate commands to ease the transition for users. Other standalone systemctl commands include:

<C>systemctl isolate graphical.target<C>
<C>systemctl isolate multi-user.target<C>

The first command changes to runlevel 5 for a graphical interface, and the second changes to runlevel 3 for a command-line interface.

Another command you might want to learn right away is journalctl (Figure 3). Systemd replaces the old syslog files with its own, which starts recording much earlier in the boot process. The basic command is

journalctl -- all

Figure 3: Systemd offers more detailed logs than syslog. Notice that journalctl begins by logging its own startup.

However, you can filter your reading of logfiles by your choice of options (Table 3).

Most users who are also admins are used to dealing with logs, but systemd also introduces systemd-analyze to help you study the boot process. The command used alone shows how long the last bootup took in kernel space and userspace (Figure 4). Add the blame command, and systemd-analyze shows how long each service took to start (Figure 5), whereas critical-chain creates a tree view that shows the order in which units start and the time they take to start (Figure 6).

Figure 4: The command systemd-analyze shows bootup time in kernel and userspace.

Figure 5: The command systemd-analyze blame shows how long each service took to start at the last bootup.

Figure 6: The command system-analyze critical-chain shows how units depend on each other during bootup.

In theory, you are supposed to be able to create SVG charts of bootup and print them to a file with

systemd-analyze dot | dot -Tsvg > systemd.svg

or

systemd-analyze plot > bootplot.svg

In practice, however, these charts fail to display in any useful fashion in any application that supports SVG format, including Gimp, Inkscape, LibreOffice Draw, or Calligra Suite's Karbon.

Taking the Next Steps

Many users, including me, got the idea while trying systemd when it was still in development that it's more complicated than it actually is. For example, I remember being puzzled at being unable to shut down while logged in as an unprivileged user – and as root as well – before I found the right command. However, such difficulties have disappeared as dozens of symbolic links have been added, and, for the most part, a Linux computer running with systemd now operates much as one without.

Still, the brief explanations given here should provide a starting point for understanding systemd and its unique features more thoroughly. As must be obvious, systemd involves more – much more – than mentioned here. Full documentation of systemd would require at least several hundred pages, as well as in-depth scrutiny of individual units.

When you look more closely at systemd, you may still dislike its design philosophy or the politics surrounding its implementation. At least potentially, systemd appears to offer more thorough and more detailed control of Linux systems than anything available until now. If nothing else, you may end up appreciating it on a technical level – as well as being grateful for having its thorough documentation to guide you.