Putting It All Together

I assume you have cloned the kvmtool Git repository already, so let's go straight to the build process. Alternatively, you can install kvmtool from the distribution's repositories with sudo apt-get install kvmtool or something similar. Kvmtool has minimum dependencies. If you need a GUI, as I do in this example, you'd want SDL or GTK+ 3.x development headers. I opted for the former:

sudo apt-get install libsdl-dev

Building kvmtool is as simple as typing make. There are no configuration steps as, again, build dependencies are minimal. Compilation takes a few seconds, and you get a command named lkvm as the result. Supposedly, "l" stands for lightweight. The initial name was just kvm, but Qemu had the same, so kvmtool was renamed. Alternatively, you can run vm, which is just an alias.

For usage summary, call ./lkvm --help. You see it implements various sub-commands, one per builtin-*.c file we just dissected. To start a VM, you need ./lkvm run. This is already a working command that opens a simple shell, with some caveats. It runs the guest on the same kernel as the host, so the image (/boot/vmlinuz-$version) must be readable. On my Ubuntu 16.04, this requires root privileges (hence, you need sudo). Moreover, kvmtool needs the guest kernel to support certain features (notably, virtio), which are often shipped as modules. So, you'd need to tell lkvm where the initial RAM drive (initrd) is.

Last but not least, lkvm needs a disk image to run. The default is a small rootfs, which lkvm builds in the ~/.lkvm/default directory in the host and exports as a 9P filesystem (see the "What on Earth Is 9P?" box).

Another option is to use one of the testing images Qemu provides [5]. I'd go this route and download linux-0.2.img. With all these bits in place, a complete command to run a VM could be as follows:

./lkvm run --kernel /boot/vmlinuz-4.8.0-46-generic --initrd /boot/initrd.img-4.8.0-46-generic --disk /boot/linux-0.2.img --cpus 2 --sdl

Note that lkvm itself doesn't need superuser privileges, but you'd want to prefix this command with sudo if your kernel image or initrd requires privileges to read, as explained previously. Here, --cpus 2 sets the number of vCPUs your guests will have, and --sdl tells kvmtool to open it in an SDL window (Figure 3).

Figure 3: Kvmtool running a guest (here, the same kernel as the host) in an SDL window. GTK+ 3 and VNC are also supported.

The command will also print a line on a terminal where you started it, like this:

# lkvm run -k /boot/vmlinuz-4.8.0-46-generic -m 320 -c 2 --name guest-4361

Take note of --name: You'll need it to manage the VM. For example, ./lkvm stop -n guest-4361 will shut down the guest gracefully. Here, the name was autogenerated, but as in this example,

lkvm run --name linux-0.2 <other arguments follow>

you can also assign a VM something more descriptive.

Command of the Month: lkvm debug

Although kvmtool is great for learning, I still tend to run Qemu/KVM in production. Reports are though that kvmtool addresses one specific use-case particularly well. I'm talking now about early boot-time debugging.

You can already specify some debugging switches to lkvm run. Say, you want to enable single-step mode. Just add --debug-single-step, and kvmtool will dump the system state (Figure 4) after every machine code instruction. Naturally, that would make a guest really slow.

Then there is a dedicated debugging sub-command named (you guessed it) debug. Typically, you supply it a guest name (either the one you assigned with --name or autogenerated) and an action to do, like this:

./lkvm debug --name guest-4849 --dump

This makes kvmtool dump the guest system state (again, see Figure 4). Internally, this command sends a SIGUSR1 signal to the vCPU thread, which causes it to request the guest state (such as registers) from the kernel-side KVM via an ioctl.

It is also possible to signal a non-maskable interrupt (NMI) to the guest:

./lkvm debug --name guest-4849 --nmi 0

The integer argument is a vCPU number to send an NMI to. Non-maskable interrupts are a serious weapon, and Linux would complain if it came unexpectedly:

[   45.364335] Uhhuh. NMI received for unknown reason 20 on CPU 0.
[   45.364968] Do you have a strange power saving mode enabled?
[   45.365467] Dazed and confused, but trying to continue

Otherwise, they can be an only option to solicit a feedback from an otherwise irresponsible kernel.

Figure 4: Kvmtool dumping the guest's state. You get the register values, interrupts info, stack contents, and page table entries (not shown here).