Command Line – unison

Although unison has been around for years, chances are you have never heard of unison. For one thing, rsync and ssh tend to be the default commands for file syncing. For another, unison's documentation is maddeningly incomplete. In the absence of man or info files or usage instructions from the creators, the existing help focuses on building a command to use at the prompt – and, considering that unison has 88 options, that is not a very attractive option, even without a command history, especially if you have several sets of files with which you regularly work. So far as I can see, the only distribution that mentions how to simplify the use of unison with preference files is Arch Linux, and even it is incomplete – as is the original documentation that Arch references. Yet once preference files are created, the number of on-the-fly options is greatly reduced, and unison becomes a handy tool for file syncing, backups, and even merging files.

unison is a shell for the rsync protocol; it also calls upon external programs like ssh and diff for its operations. It works with two sets of directories and files, updating one so that the two sets match. Its only drawback is that, if you are syncing in separate machines, each must have the same version of unison installed. The basic command structure is:

unison PATH1 PATH2

Synchronization is two-way: The most recent version of a file in either directory will overwrite the version in the other. Similarly, any file that exists only in one directory will be copied to the other. The resulting output can have three parts:

1. A standard warning: This warning indicates that this is the first time you have run a synchronization or that archive files do not exist (Figure 1). Scroll past the warning to attempt the synchronization.

   

   Figure 3: The third part of the output shows the changes made and their success or failure.

2. A list of proposed changes: Each must be confirmed unless the -auto or -batch option is used to accept everything automatically. You must enter y (yes) to continue (Figure 2).

   

   Figure 3: The third part of the output shows the changes made and their success or failure.

3. A summary of operations: Arrows to the left show that a file is being transferred from the first root, while an arrow to the right shows a file transfer from the second root. At the end of all operations, a summary displays (Figure 3).

   

   Figure 3: The third part of the output shows the changes made and their success or failure.

This basic structure is rapidly complicated by options. All unison's options are prefixed by a single hyphen, with a name that is usually self-explanatory – a useful feature when dealing with so many options. To further simplify matters, options share identical or at least similar names with the fields in a preference file. However, once a preference file is debugged, no root needs to be specified at the prompt. Instead, the structure becomes:

unison PREFERENCE-FILE

Moreover, the only options you need to add are ones that affect the command's operation, which usually reduces the number of options to add. Often, no options need to be added to the command at all. In addition, preference files can be written to draw on a common file, making the creation of new preference files extremely easy.

Writing Preference Files

Preference files are the sanest way to use unison. Preferences are stored in .unison in your home directory, with names that end with a .prf extension. Typically, you would have a default.prf that is used when a preference is not specified with the command, and perhaps other preference files whose names are a description of their contents.

To make a profile, follow these four basic steps using commented lines to help keep track of the sections:

1. Optionally, begin with label=, followed by a description of what contents the preference file is designed to sync. This field has no effect on how unison interacts with the preference file, being intended only for human convenience. You could add a commented line instead.

2. List the first root directory. For example, for user bb, it might be:

   #Root (source)
   root= /home/bb

3. List the second root directory to which files will be synchronized. On a separate line, you can also add any ssh argument, such as the port to use and the option to run in quiet mode. For example:

   #Target Directory
   root=ssh://nanday.com//home/bb/backup
   sshargs=-p4000
   sshargs=-q

4. List, one per line, the subdirectories and files of the root or source directory to include in the operation. For instance:

   #Directories and Files to include
   path=Projects
   path=Fonts
   path=Downloads
   path=to-do.txt

   These first three sections (steps 2-4 and optionally step 1), in this order, must be in a preference file. Below them, you can add other sections:

5. List the files to ignore. Regular expressions may be useful in this section:

   #Files to Ignore
   ignore=Virtualbox VMS
   ignore=*.tmp

6. As files are synced in the second root directory, they will be overwritten. As a precaution, you may want to back up the original files in the second root directory to another location, using these fields. If backuplocation is specified as local, then backup files are stored in the second root directory; if the specification is central, then backup files are stored in another directory that is specified in backupdir.

   Backups use the naming convention .bak.VERSION.FILENAME. The version begins with 1 for the most recent and can be limited in number by using the maxbackups field. The filename is the name of the original file, including any extension. This convention can be overridden using the backupprefix and backupsuffix fields, one of which must contain the version. For example:

   #Backups
   backuplocation=central
   backupdir=/home/bb/backups
   backup=$VERSION.
   backupprefix=backup
   backupsuffix=
   maxbackups=4

7. unison can call on external commands like diff and diff -3 with the -diff option and merge files using -merge PATTERN. However, for either option to work, the path to the external command and any options to use with the command must be specified in a preference file. For instance:

   Diffs and merge
   diff=diff -y --suppress-common-lines
   merge = diff3 -m CURRENT1 OLD CURRENT2 > NEW

8. If you are syncing files on different servers or physical devices, you may want to log each use of unison. At the bottom of the preference file, add:

   #Log
   log=true

   The logfile is located in the .unison directory.

   Advanced users can save complexity by using preferences another way:

9. In default.prf, include only the line:

   include common

10. Create a file called common (not common.prf). As in the first example, common will define the two root directories. It must not contain any path= fields but may include ignore= and backup= fields.

11. Create preference files that contain only path= fields for backups, plus the statement include common:

    #Directories to include
    path=Projects
    path=Fonts
    path=Downloads
    include common

Other than the directories to include, these stripped down preference files will take their characteristics from common. Note that if you run unison without specifying a preference file when using a common file, then all subdirectories will be backed up.

Whether you use basic preference files or set up preference files with a common file, the command is simplified to:

unison PREFERENCE-FILE

However, at least part of the time, you may still want or need to add other options. To give one obvious example, if you are syncing to a flash drive, then chances are you will want to add the -fat option because many Linux users format flash drives to use FAT32.

Desktop vs. Command Line

unison also supports a GTK+ graphical interface. This interface is often a separate package from unison. In Debian, it is called unison-gtk.

With unison-gtk, you can create basic preference files using the included wizard. However, if you want to list files to ignore, or any other more advanced feature, you will have to open the preference file in a text editor. In particular, you might want to add keyboard shortcuts in the preference file. For example, the lines:

#Downloads Backup
path=Downloads
batch=true
key=1
include common

This entry will customize unison so that, when the GUI is open, pressing the 1 key will signal unison to automatically sync any updates found in the Downloads directory, using the information found in the common file.

However, although unison-gtk is convenient for average users, there may still be many times when you want to run unison from the command line to take advantage of some of the options. Despite the availability of unison-gtk, taking advantage of the prompt still requires a terminal.

Entering Options from the Prompt

Using preferences usually means that no options need to be entered at the prompt. Those that might be occasionally needed are normally general ones about how unison operates, rather than about content. If manually entered options are used, they will override those specified in the preference file.

For example, when the output lists changed files, -sortbysize, -sortfirst PATTERN, or -sortnewfirst will change the order of items on the list. You might also choose -auto to accept non-conflicting default options without an interactive prompt. Alternatively, you might specify that you are prompted before deleting large numbers of files with -confirmbigdel or -confirmmerge before merging files. You might also add -links to assure the copying of symbolic links.

If one root is on an external drive formatted with FAT32 – a common occurence in Linux – you will need to add the -fat option. Similarly, if you are making a backup, you will not want to confirm each operation, so you use the -auto option to disable confirmations.

Generally, the more you use unison and create preference files to meet your needs, the less likely you are to need to add options manually. If you find yourself adding the same options all the time, check to see if they can be added to the preference file instead. Your use of unison will be far easier if you can.