Compressing initrd

By default, Debian's initramfs-tool compresses the initrd with Gzip and adds all the drivers. The image is then only 31MB, but loading and unpacking takes quite a while. You can configure the behavior using the COMPRESS variable in /etc/initramfs-tools/initramfs.conf.

An alternative compression algorithm could provide improved performance: Facebook's Zstandard (or zstd for short) takes longer to compress but produces a smaller archive and unpacks it faster (see Table 1). At the cost of a larger image, LZ4 proves to be even snappier when it comes to unpacking. You might need to install these alternative compression tools before you can use them:

sudo apt install zstd lz4

As you can see in Table 1, despite the larger archive, LZ4 needs only one fifth of the time compared to Gzip, and one third compared to zstd, to decompress in just 84ms. However, the results shown in Table 1 are only valid for the system used in our tests: Depending on the speed of the processor and the hard disk, zstd or Gzip might offer advantages on other systems.

Reducing the Scope

In many cases, you can optimize the startup process by reducing the initrd scope. The initramfs-tools-core package contains some tools you can use for analyzing the contents of initrd (Listing 4, line 1 and 3). Of the 1,377 files on the test system, 665 are kernel modules. You can extract the contents of initrd using the command in line 1 of Listing 5; this command lets you analyze the space used by each directory on the lab system (starting in Line 3).

01 $ lsinitramfs /boot/initrd.img-5.9.0-3-amd64 | wc -l
02 1377
03 $ lsinitramfs /boot/initrd.img-5.9.0-3-amd64 | grep -c ko$
04 665

01 $ unmkinitramfs /boot/initrd.img-5.9.0-3-amd64 5.9.0-3-amd64
02 $ cd 5.9.0-3-amd64
03 $ du -sh *
04 3.4M early
05 98M main
06 $ du -sh main/usr/lib/* | sort -rh
07 84M main/usr/lib/modules
08 9.0M main/usr/lib/x86_64-linux-gnu
09 2.3M main/usr/lib/firmware
10 200K main/usr/lib/udev
11 76K main/usr/lib/klibc-zlroYG-5K-Jd0h1P0SDWOT4wCA.so
12 16K main/usr/lib/systemd
13 16K main/usr/lib/modprobe.d
14 $ <B>du -sh main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/* | sort -rh | head -10<B>
15 29M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/net
16 17M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/scsi
17 2.6M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/infiniband
18 2.5M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/block
19 2.2M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/usb
20 2.2M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/hid
21 1.9M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/ata
22 1.3M main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/mmc
23 940K main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/nvme
24 880K main/usr/lib/modules/5.9.0-3-amd64/kernel/drivers/input

The kernel modules take up the most space, as suspected. However, the diversity offered is only necessary if you want the installation to be compatible with a large number of systems and various scenarios. If the installation only has to work on a single computer, you have some room to optimize.

The MODULES=dep parameter in initramfs.conf makes sure that only those modules necessary for reading the root filesystem end up in initrd. After recreating the image, the size of initrd shrinks quite substantially. LZ4 is still the fastest way to unpack the results (see Table 2).

The kernel modules now only occupy 4.6MB instead of 84MB, and the library directory now takes up most of the space. Another look at the files stored there reveals some big space guzzlers (Listing 6).

$ du -sh main/usr/lib/x86_64-linux-gnu/* | sort -rh | head -10
3.0M main/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
1.8M main/usr/lib/x86_64-linux-gnu/libc-2.31.so
1.3M main/usr/lib/x86_64-linux-gnu/libm-2.31.so
572K main/usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.9.0
432K main/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1
412K main/usr/lib/x86_64-linux-gnu/libext2fs.so.2.4
368K main/usr/lib/x86_64-linux-gnu/libmount.so.1.1.0
332K main/usr/lib/x86_64-linux-gnu/libntfs-3g.so.883.0.0
320K main/usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0
252K main/usr/lib/x86_64-linux-gnu/libfuse3.so.3.10.0

The libcrypto.so.1.1 library needs kmod (libkmod.so.2.3.5) to check the signature of Linux modules [1]. You could basically build kmod with --without-openssl. I tried this on the test system and thus removed the dependency, but this is not a good idea for security reasons. However, the question arises as to why libraries for NTFS and FUSE need to be in initrd. Many programs have a switch to output more detailed messages. Use the -v option for more verbose output with update-initramfs (Listing 7).

$ sudo update-initramfs -uv
Available versions: 5.9.0-3-amd64
5.9.0-2-amd64
[...]
Calling hook ntfs_3g
Adding binary /bin/ntfs-3g
Adding binary-link /usr/lib/x86_64-linux-gnu/libntfs-3g.so.883
Adding binary /usr/lib/x86_64-linux-gnu/libntfs-3g.so.883.0.0
[...]

These hooks – a way for packages to integrate their own scripts – are located in the /usr/share/initramfs-tools/hooks/ directory and are executable. An unneeded script can be disabled by using the chmod command to remove the executable bit (Listing 8). Note that a system update can reverse the change. LVM and encrypted root partitions require dmsetup. On systems where /dev/mapper/ does not contain appropriate files, you can also disable the dmsetup hook.

$ sudo chmod -x /usr/share/initramfs-tools/hooks/{fuse,ntfs_3g,thermal}
$ sudo update-initramfs -uv
[...]
/usr/share/initramfs-tools/hooks/ntfs_3g ignored: not executable
[...]

On systems with firmware that includes and applies the latest microcode updates (Listing 9), you do not also have to include these updates in initrd. On Intel systems, where Linux does not log the update, you can also disable the intel_microcode hook, which saves 500KB.

$ dmesg | grep microcode
[ 0.000000] microcode: microcode updated early to revision 0xde,   date = 2020-05-26
[ 0.529552] microcode: sig=0x906e9, pf=0x2, revision=0xde
[ 0.529744] microcode: Microcode Update Driver: v2.2.

All of these comparatively simple changes help the system unpack initrd in 14.5ms (Listing 10), where it took 419ms before – a massive 28 times faster.

$ sudo dmsg
[...]
[ 0.398422] calling populate_rootfs+0x0/0x109 @ 1
[ 0.398453] Trying to unpack rootfs image as initramfs...
0.413173] Freeing initrd memory: 6968K
[ 0.413254] initcall populate_rootfs+0x0/0x109 returned 0 after 14480 usecs

According to the 80-20 rule, the last milliseconds require far more effort on the user's part. You could still remove a few Linux kernel modules. On the test system, for example, booting does not require the AHCI driver or the MMC driver. The MODULES=list option in initramfs.conf makes sure that only the modules listed in /etc/initramfs-tools/modules are added (Listing 11). This step reduces the time for unpacking and loading initrd to 9.5ms, which means that I have reached the target for the lab system (see the "Potential" box).

crc32c-intel
crc32c_generic
jbd2
mbcache
crc16
ext4
crct10dif_common
crct10dif-pclmul
crct10dif_generic
crc-t10dif
t10-pi
nvme-core
nvme

Crypto Modules

The graph created by systemd-bootchart shows a large share for the dh_init and rsa_init methods in the total boot time (Figure 3). It turns out that these cryptographic modules run hardware tests that take some time (Listing 12). If you value security, you need to trust the developers that these tests are necessary. If you do not want to wait the 124ms, add the cryptomgr.notests option to the GRUB_CMDLINE_LINUX_DEFAULT variable from the /etc/default/grub.

$ sudo dmesg | grep dh_init
[ 0.144678] calling dh_init+0x0/0x20 @ 1
0.245015] initcall dh_init+0x0/0x20 returned 0 after 97656 usecs
$ sudo dmesg | grep rsa_init
[ 0.245015] calling rsa_init+0x0/0x50 @ 1
[ 0.275009] initcall rsa_init+0x0/0x50 returned 0 after 27343 usecs

Figure 3: Instead of 419ms, populate_rootfs now takes less than 10ms. dh_init, rsa_init, and acpi_init now occur in the first second.