Hidden Data

Multimedia files, such as pictures, videos, or music tracks, often contain metadata (additional information not found in the file name or file attributes). Exchangeable Image File Format (EXIF), the standard for storing this metadata, provides details such as recording date, shutter speed, and aperture, among other things. Introduced in 2010, EXIF was originally used for digital camera photos. Today, EXIF has other applications and includes far more data than originally intended. For instance, applications such as Geeqie and digiKam use an image's metadata to control the display order or to narrow a search.

Usually, metadata is defined once and then not edited, because it contains important information about the multimedia data. In practice, there are exceptions where metadata access is both useful and necessary. For instance, many digital cameras still do not store location data in the images. However, if the data exists in the form of a track recorded with a GPS device, it makes sense to include the location data as additional metadata in the file.

For these and many other tasks, Phil Harvey has developed ExifTool [1]. Implemented as a Perl library, ExifTool is the most comprehensive free software for reading, adapting, or adding metadata in multimedia files. Over the years, many users have contributed to ExifTool's development, making it an extremely extensive, but also complex, piece of software [2].

In addition to EXIF, ExifTool also supports Extensible Metadata Platform (XMP), the ISO 16684-1 standard originally developed by Adobe to embed metadata in digital media or store it as a sidecar file, and IPTC-Information Interchange Model (IPTC-IIM), the first multimedia news exchange format. ExifTool can even access makernotes, the metadata not standardized by digital camera manufacturers. ExifTool is also capable of reliably processing RAW formats.

Learning all of ExifTool's options and tag names takes time. This article covers the basics of ExifTool and then introduces a graphical user interface (GUI) for ExifTool, jExifToolGUI, that makes using ExifTool easier.

Options

ExifTool's command-line interface offers precise control via many different options. ExifTool's general options (see Table 1) start with a minus sign, followed by one or more letters, and sometimes an argument. Table 2 shows additional options for special cases.

For multiple options, use the syntax -r -s. If you use -rs, ExifTool will not process the options. Some options are followed by a metadata keyword (a tag). Additionally, the tag may be followed by assignment operators and values.

If you specify a directory as the last argument in the command line, ExifTool automatically processes all files contained in the directory. If you specify individual files in the last argument, ExifTool processes only those files.

EXIF Tags

ExifTool analyzes the EXIF tags in the specified files, evaluates them, or assigns new values to them. The (partially) normalized EXIF tags consist of a name (or tag description) and one or more values. In many cases, the tag descriptions shown in the output do not correspond to the tag names specified as options. You can use -s to find out the correct tag names. Tag names are not case-sensitive.

Due to the different metadata families and tag groups, tags with the same name may occur multiple times. Normally, ExifTool shows only the first occurrence of a tag unless you instruct it to show all occurrences with the -a option.

Without further options, exiftool shows which EXIF tags a multimedia file contains. In the example in Listing 1, the output for the file is very sparse – normally there is much more metadata in a file. The output depends on the software used to create or edit the data.

$ exiftool N1-30km.png
File Name                   : N1-30km.png
Directory                   : .
File Size                   : 1038 kB
File Modification Date/Time : 2020:03:11 11:18:42+01:00
File Access Date/Time       : 2020:03:11 11:18:42+01:00
File Inode Change Date/Time : 2020:03:13 11:24:57+01:00
File Permissions            : rw-r--r--
File Type                   : PNG
File Type Extension         : png
MIME Type                   : image/png
Image Width                 : 1920
Image Height                : 1080
Bit Depth                   : 8
Color Type                  : RGB with Alpha
Compression                 : Deflate/Inflate
Filter                      : Adaptive
Interlace                   : Noninterlaced
Significant Bits            : 8 8 8 8
Image Size                  : 1920x1080
Megapixels                  : 2.1

Many tag names are self-explanatory. For a list of available tags, see the Perl module Image::ExifTool::TagNames. Some distributions offer a special man page (e.g., /usr/share/man/man3/Image::ExifTool::TagNames.3pm.gz). You can also find more information on the ExifTool website [3].

ExifTool classifies tags into seven families (0-6), with each family consisting of several groups. The first three families are of general interest, while the remaining families are for specific uses.

You can organize ExifTool's output based on these family groups by using -g or -G with the family number (0-6) to make the output details more precise. If you don't use the family number after these options, -g inserts lines and subtitles (e.g., ---- File ----) into the output, while -G writes a group identifier at the beginning of each line (e.g., [EXIF]).

ExifTool normally displays a huge amount of information. To limit the output to certain tags, use the syntax shown in Listing 2. You can specify the -TAG parameter as often as you like. If you enter a tag at the command line that is missing in a file's EXIF tags, the corresponding line is omitted in the output.

$ exiftool -TAG... FILE|DIR

The syntax for tags consists of a minus sign followed by the tag name. Tag names are often identical to the descriptions, but spaces can be omitted (e.g, Focal Length is the same as FocalLength, and Circle Of Confusion corresponds to CircleOfConfusion). However, sometimes tag names and descriptions don't match (as with Depth Of Field); in that case, search for the correct name with -s. When reading and writing metadata, it is important to exactly adhere to the syntax, since small deviations often have unexpected effects.

There are two call variants for tags: -TAG enables a tag, while --TAG ignores it. The command

exiftool -keywords .

shows the keywords of all files in the current directory. In contrast,

exiftool --keywords .

returns all tags except for the keywords of all files in the current directory. Table 3 shows possible tag modifications.

Some values cannot be overwritten. For example, if you try to change the megapixels value, ExifTool outputs Warning: Sorry, Megapixels is not writable.

Using ExifTool

The best way to get to know ExifTool is to use it. The automatic backup option, which backs up the original files, prevents serious damage in most cases. Nevertheless, it is advisable to first work with copies of the original files.

To view the tags of a current directory's files, use line 1 of Listing 3. If the output is difficult to read, use the -htmlDump option to output an HTML-formatted binary dump that can be redirected to a file, which you can then view in your web browser.

01 $ exiftool .
02 $ exiftool -P "-Directory<DateTimeOriginal" -d "%Y/%m/%d" DIRECTORY
03 $ exiftool -P "-FileName<CreateDate" -d "%Y%m%d_%H%M%S-%f.%%e" DIRECTORY

Conveniently, ExifTool accepts values from tags as input for actions. For example, the command in line 2 of Listing 3 sorts the images into directories based on their timestamps. This action leaves the file names unchanged; -P also preserves the files' attributes.

If, on the other hand, you want to rename the files based on their timestamps, use the command in line 3 of Listing 3. In the argument for -d, %Y%m%d and %H%M%S represent the same values as the date command in the shell. %e stands for the suffix and %f for the previous file name. In this way you can customize file names individually.

For both examples in Listing 3, ExifTool evaluates the EXIF tags contained in the files. In the first example (line 2), -Directory creates new directories based on the value of DateTimeOriginal; in the second example (line 3), ExifTool builds the file name from the CreateDate tag. Since the redirection character ("<") is included in ExifTool's syntax, you must put the entire expression in quotation marks – otherwise the shell will interpret the operator.

For some situations, you may need to change the EXIF tags. For example, if the camera clock was set incorrectly, it is often useful to correct the timestamps afterwards. You should be able to fix this by changing the DateTimeOriginal tag (Listing 4, line 1).

01 $ exiftool "-DateTimeOriginalOPERATOR=YY:MM:DD d:m:s" .
02 $ exiftool "-AllDatesOPERATOR=YY:MM:DD d:m:s" .

However, several other EXIF tags contain the timestamp. Because not all software used for viewing images works correctly with a certain specification, it is often better to use the AllDates pseudo-tag (Listing 4, line 2).

ExifTool is especially useful for adding geodata from external tracks for recordings that do not yet contain location data. Two options are important here: -geotrack=TRKFILE and -geosync=MIN:SEC. ExifTool can interpret most common geodata formats, especially GPX.

GPS devices and camera clocks are often not in sync. Since the time is used as a reference for the location, -geosync=CORRECTION lets you correct the times when assigning the positions without changing the timestamps in the images. A helpful tip: Before starting to take pictures, take a reference picture of the GPS screen with the time display to show the time difference between the camera and the GPS device.

You can correct fast camera times by using negative values for -geosync=, but it often makes more sense to change the wrong image timestamp directly in the images using -AllDates.

There are six standard and three optional geotags (see Table 4). ExifTool extracts these values – usually only the first six – from the GPS tracks, interpolating the location based on the time.

A simplified call for geodata looks like line 1 of Listing 5; it shows the camera clock is 12 minutes and 34 seconds slower than the GPS device. If you want to use multiple track files, specify them individually as arguments (Listing 5, line 2). Using a wildcard like -geotag=*gpx usually causes many error messages. The command from line 3 of Listing 5 deletes geodata from the files.

01 $ exiftool -geotag=mytrack.gpx -geosync=+12:34 .
02 $ exiftool -geotag=1.gpx -geotag=2.gpx .
03 $ exiftool -geotag= .

The message Warning: Time is too far before track in File:Geotime (ValueConvInv) indicates a problem with the time zone. GPS times are always given as Coordinated Universal Time (UTC), whereas camera clocks usually give the local time. You may have to take this into account when correcting the time difference [4].