File syncing with unison
Command Line – unison

© Photo by pan xiaozhen on Unsplash
Unison is a handy tool for file syncing, backups, and merging. To get the most out of unison, however, you need to invest time perfecting your preference files to meet your needs.
Although unison
has been around for years, chances are you have never heard of unison
. For one thing, rsync
and ssh
tend to be the default commands for file syncing. For another, unison
's documentation is maddeningly incomplete. In the absence of man or info files or usage instructions from the creators, the existing help focuses on building a command to use at the prompt – and, considering that unison
has 88 options, that is not a very attractive option, even without a command history, especially if you have several sets of files with which you regularly work. So far as I can see, the only distribution that mentions how to simplify the use of unison
with preference files is Arch Linux, and even it is incomplete – as is the original documentation that Arch references. Yet once preference files are created, the number of on-the-fly options is greatly reduced, and unison
becomes a handy tool for file syncing, backups, and even merging files.
unison
is a shell for the rsync
protocol; it also calls upon external programs like ssh
and diff
for its operations. It works with two sets of directories and files, updating one so that the two sets match. Its only drawback is that, if you are syncing in separate machines, each must have the same version of unison
installed. The basic command structure is:
unison PATH1 PATH2
Synchronization is two-way: The most recent version of a file in either directory will overwrite the version in the other. Similarly, any file that exists only in one directory will be copied to the other. The resulting output can have three parts:
A standard warning: This warning indicates that this is the first time you have run a synchronization or that archive files do not exist (Figure 1). Scroll past the warning to attempt the synchronization.
Figure 3: The third part of the output shows the changes made and their success or failure.A list of proposed changes: Each must be confirmed unless the
-auto
or-batch
option is used to accept everything automatically. You must enter y (yes) to continue (Figure 2).Figure 3: The third part of the output shows the changes made and their success or failure.A summary of operations: Arrows to the left show that a file is being transferred from the first root, while an arrow to the right shows a file transfer from the second root. At the end of all operations, a summary displays (Figure 3).
Figure 3: The third part of the output shows the changes made and their success or failure.
This basic structure is rapidly complicated by options. All unison
's options are prefixed by a single hyphen, with a name that is usually self-explanatory – a useful feature when dealing with so many options. To further simplify matters, options share identical or at least similar names with the fields in a preference file. However, once a preference file is debugged, no root needs to be specified at the prompt. Instead, the structure becomes:
unison PREFERENCE-FILE
Moreover, the only options you need to add are ones that affect the command's operation, which usually reduces the number of options to add. Often, no options need to be added to the command at all. In addition, preference files can be written to draw on a common file, making the creation of new preference files extremely easy.
Writing Preference Files
Preference files are the sanest way to use unison
. Preferences are stored in .unison
in your home directory, with names that end with a .prf
extension. Typically, you would have a default.prf
that is used when a preference is not specified with the command, and perhaps other preference files whose names are a description of their contents.
To make a profile, follow these four basic steps using commented lines to help keep track of the sections:
- Optionally, begin with
label=
, followed by a description of what contents the preference file is designed to sync. This field has no effect on howunison
interacts with the preference file, being intended only for human convenience. You could add a commented line instead. List the first root directory. For example, for user
bb
, it might be:#Root (source) root= /home/bb
List the second root directory to which files will be synchronized. On a separate line, you can also add any
ssh
argument, such as the port to use and the option to run in quiet mode. For example:#Target Directory root=ssh://nanday.com//home/bb/backup sshargs=-p4000 sshargs=-q
List, one per line, the subdirectories and files of the root or source directory to include in the operation. For instance:
#Directories and Files to include path=Projects path=Fonts path=Downloads path=to-do.txt
These first three sections (steps 2-4 and optionally step 1), in this order, must be in a preference file. Below them, you can add other sections:
List the files to ignore. Regular expressions may be useful in this section:
#Files to Ignore ignore=Virtualbox VMS ignore=*.tmp
As files are synced in the second root directory, they will be overwritten. As a precaution, you may want to back up the original files in the second root directory to another location, using these fields. If
backuplocation
is specified aslocal
, then backup files are stored in the second root directory; if the specification iscentral
, then backup files are stored in another directory that is specified inbackupdir
.Backups use the naming convention
.bak.VERSION.FILENAME
. The version begins with1
for the most recent and can be limited in number by using themaxbackups
field. The filename is the name of the original file, including any extension. This convention can be overridden using thebackupprefix
andbackupsuffix
fields, one of which must contain the version. For example:#Backups backuplocation=central backupdir=/home/bb/backups backup=$VERSION. backupprefix=backup backupsuffix= maxbackups=4
unison
can call on external commands likediff
anddiff -3
with the-diff
option and merge files using-merge PATTERN
. However, for either option to work, the path to the external command and any options to use with the command must be specified in a preference file. For instance:Diffs and merge diff=diff -y --suppress-common-lines merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW
If you are syncing files on different servers or physical devices, you may want to log each use of
unison
. At the bottom of the preference file, add:#Log log=true
The logfile is located in the
.unison
directory.Advanced users can save complexity by using preferences another way:
In
default.prf
, include only the line:include common
- Create a file called
common
(notcommon.prf
). As in the first example,common
will define the two root directories. It must not contain anypath=
fields but may includeignore=
andbackup=
fields. Create preference files that contain only
path=
fields for backups, plus the statementinclude common
:#Directories to include path=Projects path=Fonts path=Downloads include common
Other than the directories to include, these stripped down preference files will take their characteristics from common
. Note that if you run unison
without specifying a preference file when using a common
file, then all subdirectories will be backed up.
Whether you use basic preference files or set up preference files with a common
file, the command is simplified to:
unison PREFERENCE-FILE
However, at least part of the time, you may still want or need to add other options. To give one obvious example, if you are syncing to a flash drive, then chances are you will want to add the -fat
option because many Linux users format flash drives to use FAT32.
Desktop vs. Command Line
unison
also supports a GTK+ graphical interface. This interface is often a separate package from unison
. In Debian, it is called unison-gtk
.
With unison-gtk
, you can create basic preference files using the included wizard. However, if you want to list files to ignore, or any other more advanced feature, you will have to open the preference file in a text editor. In particular, you might want to add keyboard shortcuts in the preference file. For example, the lines:
#Downloads Backup path=Downloads batch=true key=1 include common
This entry will customize unison
so that, when the GUI is open, pressing the 1 key will signal unison
to automatically sync any updates found in the Downloads
directory, using the information found in the common
file.
However, although unison-gtk
is convenient for average users, there may still be many times when you want to run unison
from the command line to take advantage of some of the options. Despite the availability of unison-gtk
, taking advantage of the prompt still requires a terminal.
Entering Options from the Prompt
Using preferences usually means that no options need to be entered at the prompt. Those that might be occasionally needed are normally general ones about how unison
operates, rather than about content. If manually entered options are used, they will override those specified in the preference file.
For example, when the output lists changed files, -sortbysize
, -sortfirst PATTERN
, or -sortnewfirst
will change the order of items on the list. You might also choose -auto
to accept non-conflicting default options without an interactive prompt. Alternatively, you might specify that you are prompted before deleting large numbers of files with -confirmbigdel
or -confirmmerge
before merging files. You might also add -links
to assure the copying of symbolic links.
If one root is on an external drive formatted with FAT32 – a common occurence in Linux – you will need to add the -fat
option. Similarly, if you are making a backup, you will not want to confirm each operation, so you use the -auto
option to disable confirmations.
Generally, the more you use unison
and create preference files to meet your needs, the less likely you are to need to add options manually. If you find yourself adding the same options all the time, check to see if they can be added to the preference file instead. Your use of unison
will be far easier if you can.
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Direct Download
Read full article as PDF:
Price $2.95
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Find SysAdmin Jobs
News
-
CarbonOS: A New Linux Distro with a Focus on User Experience
CarbonOS is a brand new, built-from-scratch Linux distribution that uses the Gnome desktop and has a special feature that makes it appealing to all types of users.
-
Kubuntu Focus Announces XE Gen 2 Linux Laptop
Another Kubuntu-based laptop has arrived to be your next ultra-portable powerhouse with a Linux heart.
-
MNT Seeks Financial Backing for New Seven-Inch Linux Laptop
MNT Pocket Reform is a tiny laptop that is modular, upgradable, recyclable, reusable, and ships with Debian Linux.
-
Ubuntu Flatpak Remix Adds Flatpak Support Preinstalled
If you're looking for a version of Ubuntu that includes Flatpak support out of the box, there's one clear option.
-
Gnome 44 Release Candidate Now Available
The Gnome 44 release candidate has officially arrived and adds a few changes into the mix.
-
Flathub Vying to Become the Standard Linux App Store
If the Flathub team has any say in the matter, their product will become the default tool for installing Linux apps in 2023.
-
Debian 12 to Ship with KDE Plasma 5.27
The Debian development team has shifted to the latest version of KDE for their testing branch.
-
Planet Computers Launches ARM-based Linux Desktop PCs
The firm that originally released a line of mobile keyboards has taken a different direction and has developed a new line of out-of-the-box mini Linux desktop computers.
-
Ubuntu No Longer Shipping with Flatpak
In a move that probably won’t come as a shock to many, Ubuntu and all of its official spins will no longer ship with Flatpak installed.
-
openSUSE Leap 15.5 Beta Now Available
The final version of the Leap 15 series of openSUSE is available for beta testing and offers only new software versions.