Tutorials – Build the Linux Kernel

Back in the day, rebuilding the kernel was something of a rite of passage for most Linux users. Typically, a Linux distribution would use a plain kernel that wasn't optimized for any specific CPU type and was bundled in various bits and bobs that users might need. Many other features and drivers – especially experimental ones – were left out, though. I remember having to recompile my kernel just to get audio working on my old Cyrix M3 box running Red Hat 5.1 back in the late 1990s.

Today, many desktop Linux distros include multiple kernel packages built for different CPU types, and almost every feature and driver is available as a module. Very few users actually need to build a custom kernel by hand, so why do it? Well, it's still a very useful technique to learn. Even the most bleeding-edge distros don't always enable every single feature in the kernel, and what if a new kernel is released with an important fix or update you need? You could wait a few weeks or months for your distro to package it up, but if you know how to compile it yourself, you can stay ahead of the game.

Some patches that you might want to try aren't part of the main Linux kernel source code tree, so you have to compile your own kernel to use them. Aside from all of the practical benefits, it's just fascinating to see what's going on inside the guts of a Linux installation and is a good little project to take on if you have a few spare hours on a weekend. Over the next few pages, I'll show you how to get, configure, compile, and install a fresh new kernel directly from Linus Torvalds' computer (well, thereabouts) and show you how to apply patches as well.

Before I start, though, it's worth reiterating exactly what the kernel does. Essentially, the kernel is the "core" process on a Linux installation: the first thing that is loaded and started (by the bootloader). The kernel is responsible for managing memory – making sure that programs don't overwrite one another's data – and sharing CPU time between different processes. Additionally, the kernel talks to your hardware via drivers and provides implementation of various protocols (e.g., networking) and filesystem drivers. Stability in the kernel is of utmost importance: Whereas a crash in Firefox, for example, might be slightly annoying and cause you to lose some work, a kernel crash can completely lock up your system.

Getting Set Up

Many Linux distros include source code packages for the kernels they ship, including extra patches and build scripts. In this tutorial, however, you're going to use the vanilla kernel code available from the main kernel website [1]. This site contains the compressed code archives signed off by Linus Torvalds and others, so it's the "pure" form of the kernel without any distro-specific patches or customizations.

On the kernel website you'll see a big yellow button with "latest stable kernel" – click it to grab the source code in .tar.xz format. (At the time of writing, this was version 4.9, but 4.10 or newer could be available by the time you read this.) This .tar.xz archive is around 90MB, so download it and save it into your home directory. Next, open a terminal, extract the archive, and switch into the resulting directory (using the version number of your download):

tar xfv linux-4.9.tar.xz
cd linux-4.9

After extraction, the code will take up around 780MB of disk space. That might sound huge, especially when you consider that a typical kernel is only a fraction of that! (Look in your /boot directory at files beginning with vmlinuz (Figure 1) – they are only around 7 or 8MB.) The vast majority of kernel source code is for drivers, most of which you'll never need, and are compiled into modules that are only loaded if your hardware requires them.

Figure 1: Have a peek inside your /boot directory, and you'll see vmlinuz files for kernels and config files for their build settings.

Now you are in your kernel source code directory, so enter ls to have a look around. You'll see subdirectories for various parts of the kernel: drivers, firmware, net(working), and so forth, along with a kernel subdirectory that contains the low-level workings of the kernel (see cpu.c for some CPU management routines and kmod.c for the module loader). It's all extremely technical stuff but worth having a look, if just to be wowed by the complexity.

Back in your linux-4.9 directory, you'll need to make sure you have the right tools for compiling a kernel. Specifically, you need GNU Make, GCC (the C/C++ compiler), and the development packages for OpenSSL and ncurses (Figure 2). On a Debian-based distro, you can grab these with:

sudo apt-get install make gcc libssl-dev libncurses5-dev

Figure 2: You'll need a few dependencies installed to build your kernel, namely GCC, GNU Make, and OpenSSL development headers.

If you're running a distro based on Red Hat or Fedora (e.g., CentOS), try the

sudo dnf install make gcc openssl-devel ncurses-devel

command.

Configuring the Kernel

Now the fun part begins. Before you compile the kernel, you have the opportunity to customize it to your liking. You can spend minutes, hours, or even days doing this – it all depends on what you want to achieve. If you're installing a new kernel for practical reasons, such as to get security updates and bug fixes, you might want to use the configuration for your currently running kernel and apply it to the new one. Have a look in your /boot directory, and you'll see one (or more) files beginning with config- followed by a version number (Figure 1).

Take a look inside one of these files; you'll see that they contain several thousand lines of options, followed by y or m. The y means that the option or feature should be compiled into the kernel image (i.e., into the vmlinuz file mentioned beforehand). If a feature is followed by m, it means it should be compiled as a module that is loaded only when it is needed (the sensible choice for the vast majority of kernel features).

If you want to use your existing kernel configuration with the new kernel, then, copy the appropriate config- file in /boot to .config inside the new kernel's source code directory. For instance, in this case, I'm in linux-4.9/ in the home directory, and by running this command, I can see which kernel I'm currently running:

uname -r

This tells me I'm running 4.8.0-30-generic, so I copy the config file for that kernel into my source directory:

cp /boot/config-4.8.0-30-generic .config

If you don't want to make any changes to the configuration at this stage, you can now simply jump ahead to the "Building and Installing" section in this tutorial. If you do want to perform some fine-tuning, however, you have various options.

First, if you don't want to use the existing config file from your /boot directory but would rather generate a fresh (default) one, enter:

make defconfig

Now look inside .config, and you'll see a set of options that have been created for your CPU architecture. These may be OK, but chances are you still want to tweak them a bit. There's an interactive way to generate a .config file, with:

make config

The problem here is, you could spend several days answering questions! You have to answer yes/no/module to most of them, and with thousands of questions it becomes unwieldy. So the tool that almost everyone uses to configure kernels, including distro developers, is menuconfig:

make menuconfig

Now that's more like it. This provides a user-friendly menu-driven system (Figure 3) for exploring and enabling different features, drivers, and options in the kernel configuration. Use the Up and Down arrow keys to browse through menus and options, and hit the spacebar to select or disable them. Options listed with square brackets can only be built into the kernel; in other words, they cannot be loaded as modules. (This usually applies to very low-level features that need to be accessible right from the start of booting, before there's even a filesystem available from which to load modules.)

Figure 3: The menuconfig tool helps you customize your kernel setup in a simple menu-driven user interface.

If an option uses angle brackets (< >), however, it can be enabled as a module; just keep hitting the spacebar until M appears. You'll notice that most drivers are available as modules, and it doesn't make much sense to compile them directly into your kernel image unless you're building an embedded system with extremely restricted requirements. Most features and options have accompanying help text, so to access it, use the Right and Left arrow keys to select Help at the bottom of the window and then hit Enter. To go back to a higher level menu, choose Exit.

The big question now is: What options should you enable? If this is your first time building a kernel, I highly recommend copying an existing config file from your /boot directory into .config before running make menuconfig, as described earlier. This means your new kernel will be set up in much the same way as your existing kernel, so you shouldn't have problems booting later on.

From here, it's all up to you. Try exploring the menu options General setup, Processor type and features, Kernel hacking (if you're a developer), and Security options. Even if a lot of the features are unfamiliar to you, it really highlights just what a flexible and feature-rich kernel Linux is. No surprise then that it's being used on everything from wristwatches to mainframes.

When you're finished, choose Exit at the bottom and choose to save your configuration.

Building and Installing

With your dependencies installed and the kernel configured to your liking, now comes the big moment: compiling it. This step used to be a more involved procedure with several commands, but these days, it all comes down to a single:

make

A better approach, if you have a multicore CPU (like pretty much every PC built in the last five years), is to run a parallel make, specifying the number of cores you have. For instance, on a dual-core machine, use:

make -j2

This option speeds things up considerably. How long the whole build process takes, though, depends entirely on the speed of your machine: If you're rocking the latest CPU and building on all four cores, you could be done in a matter of minutes. Older boxes and low-spec machines like the Raspberry Pi can take a few hours. You can watch the build as it progresses (Figure 4): CC in the output refers to the C compiler doing its work, whereas LD is the linker, which essentially links together the executable binary code generated from each compile so that you end up with a single kernel image. Additionally, [M] indicates that something is being compiled as a module.

Figure 4: Compiling the kernel and modules can take anywhere from five minutes to five hours, depending on the specs of your machine.

Once the build process has finished and you land back at the command prompt, you need to install the newly built kernel and its modules into your filesystem. (If you're on RHEL, Fedora, or CentOS, make sure that you have the grubby package installed at this point). Enter:

sudo make modules_install install

When this command has finished, have a look in /boot and you should see your new kernel (e.g., vmlinuz-4.9.0). Reboot your distro, and bring up the GRUB menu – if it doesn't appear by default, keep tapping Esc just after the BIOS screen appears. You should see a list of kernels, including your shiny new one, which you can now try out (Figure 5). And if anything's wrong with it, simply reboot and choose the older one. Enjoy.

Figure 5: In Ubuntu-based distros, choose Advanced options in the GRUB boot menu to access and boot all available kernels.

If you want to add more custom and experimental features to your kernel, see the "Patching Your Kernel" box, and if you want the latest and greatest, see the "Living on the Bleeding Edge" box.

zcat ../patch-4.9-rt1.patch.gz | patch -p1 --dry-run

zless patch-4.9-rt1.patch.gz

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.gitcd linux