Multifaceted backups with Kopia
Tutorial – Mastering Kopia
Data deduplication, encryption, compression, incremental backups, error correction, and support for snapshots and popular cloud storage services: Kopia delivers.
A good backup tool is like a dishwasher: It's not something most of us get excited about, but the degree to which it improves our daily lives is hard to overstate. And like with a dishwasher, no one really wants to spend time attending to a backup tool. Ideally, you'd want to set it up once and let it do its job with the push of a button or have it perform backups automatically, with no user interaction whatsoever.
Picking the right backup tool is not as trivial as choosing a dishwasher, though. Sure, you can whip up a simple shell script that backs up data to a different storage device using good old rsync. But in this day and age, it's simply not enough. If you're serious about keeping your data safe, you want to use a tool that supports incremental backups, deduplication, snapshots, and other useful features. For an offsite backup, you definitely want your backup tool to support mainstream storage services and encryption. On top of that, it wouldn't hurt if all of this were wrapped in a user-friendly interface.
It may sound like a pipe dream, but that's exactly what Kopia [1] has to offer. Plus, this cross-platform tool features a built-in web server and a dedicated desktop graphical application. And it goes without saying that you can use Kopia from the command line. In short, it's pretty much a perfect tool for keeping your data safe.
Getting Started with Kopia
Unsurprisingly, to use Kopia you have to install it on your system first, and the project supports practically every installation option imaginable. There are packages for most mainstream Linux distributions, there are Docker images for those who prefer to go the container route, there are AppImage packages, and you can even grab a single executable binary from the project's GitHub repository.
If you happen to use an Apt-based Linux distribution (Debian, Ubuntu, or Linux Mint), installing the latest version of Kopia is a matter of running the commands in Listing 1.
Listing 1
Apt-based Kopia Installation
curl -s https://kopia.io/signing-key | sudo gpg --dearmor -o /etc/apt/keyrings/kopia-keyring.gpg echo "deb [signed-by=/etc/apt/keyrings/kopia-keyring.gpg] http://packages.kopia.io/apt/ stable main" | sudo tee /etc/apt/sources.list.d/kopia.list sudo apt update sudo apt install kopia
The official documentation also suggests installing the Kopia UI desktop application, but you don't really need it, because you can access and control Kopia via its web UI. In fact, the web UI offers the most straightforward way to learn Kopia's basics. To enable the web UI, you need to configure and start Kopia's built-in server. Normally, this involves creating a Kopia user, configuring permissions, and creating and enabling a certificate. The good news is that you don't need any of that if you only want to access Kopia from the same machine it runs on, and the machine itself is not accessible from outside of the local network. In this case, you can start Kopia's server with all security measures disabled using:
kopia server start --insecure --without-password --disable-csrf-token-checks
When the server is running, point the browser to http://127.0.0.1:51515 to access Kopia's web UI.
Starting the server manually is fine, but a better approach is to let the system do that automatically on boot through a systemd service. To do this, use the following commands to create a dedicated directory for systemd services, and then create a new systemd unit file and open it for editing:
mkdir -p ~/.config/systemd/user/ nano ~/.config/systemd/user/kopia.service
Enter the service definition in Listing 2 and save the changes.
Listing 2
Service Definition
[Unit] Description=Kopia server [Service] Restart=always ExecStart=kopia server start --insecure --without-password --disable-csrf-token-checks ExecStop=/usr/bin/kill -HUP $MAINPID [Install] WantedBy=default.target
Use the following commands to enable and start the service as well as enable it on boot:
systemctl --user daemon-reload systemctl --user enable kopia.service systemctl --user start kopia.service loginctl enable-linger $USER
With the Kopia server up and running, the next step is to create and configure a location for storing backups, called a backup repository. The landing page in the web UI lists all supported storage types (Figure 1), so creating and configuring a new backup repository is a matter of clicking the appropriate button (Figure 2). To set up an external USB storage device as a backup repository, click the Local Directory or NAS button (Figure 1).
Assuming that the target storage device is connected and mounted, enter the root directory or a specific folder on the storage device in the Directory Path field, and click Next. Because Kopia encrypts backups, you must specify a password for the new repository. It goes without saying that if you lose or forget the password, you won't be able to access your backups. The Advanced Options section lets you configure additional settings. While you can leave most of the options at their defaults, you might want to enable the error correction feature that reduces the likelihood of data corruption caused by bitrot or hardware issues. To enable this feature, set the Error Correction Overhead option to the desired value. This value determines how much storage space is used for the error correction code. Keep in mind that the error correction functionality is still experimental. When you've configured the options, click Create Repository to create the backup repository.
Like most modern backup tools, Kopia doesn't simply mirror the data you want to keep safe. Instead the application uses the concept of snapshots. Every time you run a backup job, Kopia creates a snapshot, or a backup catalog that is frozen in time. The data in the snapshot reflects the directory structure and the state of each file as it was at the moment the snapshot was created. The snapshot approach has several advantages compared to a straight backup, key among them being the ability to restore previous versions of specific files and directories. On the downside, the snapshot-based backup approach requires more storage than the source. So it's a good idea to allocate as much storage space for use with Kopia as possible.
But if Kopia creates a new snapshot every time you run a backup job, wouldn't you need infinite storage to keep an infinite number of snapshots? Yes, but in reality, it's unlikely you'd need to keep all snapshots Kopia has ever created. Realistically, you'd want to keep a couple of yearly snapshots, weekly snapshots that go back a month or so, hourly snapshots for the last 48 hours, and so on. You can then let the backup tool automatically remove outdated snapshots, which frees up storage and ensures that you'll never run out of space.
This is exactly what Kopia's snapshot retention mechanism is designed to do. The application allows you to specify how many latest, daily, weekly, monthly, and annual snapshots you want to keep. It could be a bit tricky to figure out how many of each type of snapshots you might need. The good news is that Kopia features sensible defaults, so you don't have to deep-dive into the topic right from the start (Figure 3).
To perform a backup (i.e., create a new snapshot), click the New Snapshot button, and enter the path to the directory with the data you want to back up. You can then configure snapshot settings using the available options. You can specify the number of snapshots you want Kopia to keep in the Snapshot Retention section (as already mentioned, you might want to leave the options at their defaults), and you can list the files and directories you want to exclude from the backup in the Files section (Figure 4).
As any backup application worth its salt, Kopia makes it possible to set up a schedule for automatically creating snapshots (Figure 5). This can be done by configuring the available options in the Scheduling section. The most straightforward way to enable regular automatic backups is to choose the desired interval from the Snapshot Frequency drop-down list. This way, you can configure intervals from every 10 minutes to every 12 hours. If you want Kopia to automatically create snapshots every day at a specific time, specify one or several time entries in the Times of Day section. For example, if you want Kopia to create snapshots at 11am and 5pm, enter 11:00 and 17:00, each on a new line. And here's the clever part: If Kopia detects no new or modified files, it skips the scheduled snapshot operation, which avoids clogging the storage device with identical snapshots.
After you've configured the settings, press the Snapshot Now button to create a new backup snapshot.
If you take a look at backup snapshots on the storage device, you'll see directories with encrypted files in them. In other words, you can't directly access the backup data. Instead, you can browse the backups using Kopia's web UI or the Kopia desktop application. In the Snapshots section, click on the path defined as a backup source, and you should see a list of all snapshots. Click the desired snapshot entry, and you should see the source data as it was at the moment the snapshot was taken. You can traverse the directories to locate the file you need. Click on the file's link to download it. Kopia has another clever trick up its sleeve: It allows you to mount a snapshot as a local filesystem, so you can work with it using a file manager. To mount a snapshot, click the Mount as Local Filesystem button. This mounts the current snapshot and conveniently displays the path to its mount point.
Using Kopia's web UI or a mounted filesystem works fine if you need to restore one or two files or directories, but if you need to perform a full restore of the backed up data, there is a better way to do this (Figure 6). In Kopia's web UI, select the desired snapshot, press the Restore Files and Directories button, specify the destination directory for the restored data (this directory must be empty), and hit Begin Restore. Instead of restoring the entire snapshot, you can restore a single directory and all its content. To do this, navigate to the desired directory in the snapshot, and then press Restore Files and Directories.
Using Kopia from the Command Line
The Kopia desktop application and the web UI offer a user-friendly way of using the application, but nothing beats the efficiency of the command line. In fact, you can set up a new backup repository, connect Kopia to it, and create a snapshot using the three simple commands in Listing 3.
Listing 3
Set Up, Connect, Create
kopia repository create filesystem --path /path/to/repository kopia repository connect filesystem --path /path/to/repository kopia snapshot create /path/to/source
To list all existing snapshots, run the kopia snapshot list
command, and to restore data from a specific snapshot, use the kopia snapshot restore
command followed by the hash of the desired snapshots and the path for saving the restored data:
kopia snapshot restore ke5ba82cc69841df04f5839102f0cd53d/path/to/restore/dir
In most cases, you're likely to keep multiple backup copies, and you can use Kopia to create and manage several repositories. Better still, Kopia makes it possible to synchronize the currently connected repository with another local or remote repository. This means that you can, for example, connect to a repository on a local storage device and then simply synchronize it with a remote repository. Unfortunately, you can only perform synchronization from the command line. Fortunately, it's just a matter of running a single command. You can synchronize the currently connected repository to a remote Backblaze B2 repository (where BUCKET is the actual name of an existing B2 bucket) with:
kopia repository sync-to b2 --bucket BUCKET
By default, the synchronization command doesn't synchronize deletions, but adding the --delete
flag enables that:
kopia repository sync-to b2 --bucket BUCKET --delete
Conclusions
Hopefully by now you know enough to put Kopia to work on backup duties. But there is much, much more to the application beyond its basic functionality. If you are serious about backup, it's worth putting time and effort into learning and understanding Kopia's advanced features. Data loss is as inevitable as death and taxes, and sooner or later a backup will save your bacon. Kopia is equipped to do just that.
Infos
- Kopia: https://kopia.io/
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
New Steam Client Ups the Ante for Linux
The latest release from Steam has some pretty cool tricks up its sleeve.
-
Gnome OS Transitioning Toward a General-Purpose Distro
If you're looking for the perfectly vanilla take on the Gnome desktop, Gnome OS might be for you.
-
Fedora 41 Released with New Features
If you're a Fedora fan or just looking for a Linux distribution to help you migrate from Windows, Fedora 41 might be just the ticket.
-
AlmaLinux OS Kitten 10 Gives Power Users a Sneak Preview
If you're looking to kick the tires of AlmaLinux's upstream version, the developers have a purrfect solution.
-
Gnome 47.1 Released with a Few Fixes
The latest release of the Gnome desktop is all about fixing a few nagging issues and not about bringing new features into the mix.
-
System76 Unveils an Ampere-Powered Thelio Desktop
If you're looking for a new desktop system for developing autonomous driving and software-defined vehicle solutions. System76 has you covered.
-
VirtualBox 7.1.4 Includes Initial Support for Linux kernel 6.12
The latest version of VirtualBox has arrived and it not only adds initial support for kernel 6.12 but another feature that will make using the virtual machine tool much easier.
-
New Slimbook EVO with Raw AMD Ryzen Power
If you're looking for serious power in a 14" ultrabook that is powered by Linux, Slimbook has just the thing for you.
-
The Gnome Foundation Struggling to Stay Afloat
The foundation behind the Gnome desktop environment is having to go through some serious belt-tightening due to continued financial problems.
-
Thousands of Linux Servers Infected with Stealth Malware Since 2021
Perfctl is capable of remaining undetected, which makes it dangerous and hard to mitigate.