Shrink It

Although more sophisticated image formats have existed for some time [1], JPEG is still one of the most common. Whether you are working on a smartphone, compact camera, or professional photo device, you'll find an option for generating images in JPEG format. The flood of JPEG images caused by so many millions of users snapping photos actually sucks up lots of storage space on file hosting servers. This problem led cloud provider Dropbox [2] to look for a better way to store photo images. The result of this effort is the new Lepton image format, which reduces the space requirements for JPEG images by up to 25 percent. Lepton works losslessly, meaning the image remains exactly the same after you reconvert it (Figure 1). Dropbox says it has already applied the new format to 16 billion images and has thus saved multiple petabytes of storage space. The Dropbox cloud transparently converts JPEG images to Lepton format and changes them back into the original format as soon as someone accesses them.

Figure 1: Lepton doesn't make any changes to the image when converting and reconverting: The image remains exactly the same.

Unlike JPEG, Lepton doesn't reduce the visible image – image reduction often results in known artifacts with JPEG. Instead, Lepton is designed to reduce the data needed for the image information using the Middle-out algorithm, a type of arithmetic coding that is also used in the VP8 video format. A Dropbox blog post provides a detailed technical explanation about how the Lepton format reduces the size of a JPEG file [3].

Installing Lepton

Dropbox has additionally released a tool, also called Lepton, that you can use to convert files to and from Lepton format. The Lepton compression tool is available under the Apache 2.0 license. Ubuntu, Debian, openSUSE, and several other Linux distributions provide Lepton packages. You can also download the source code from GitHub [4] and compile Lepton from source (Listing 1) If you choose to compile, you'll need the Git and Cmake tools, which you can install with:

sudo apt-get install git cmake

The Lepton README [5] suggests a few other possibilities for obtaining Lepton, but Git or a package installation are likely the easiest options.

In Use

The Lepton syntax is pretty simple. For example, the command:

lepton image.jpg

converts the specified image into Lepton format. By default, the new file will have the original filename with the .lep extension. The following command:

lepton filename.lep

extracts the original JPEG from a Lepton file.

If the output file already exists, Lepton doesn't overwrite it but instead creates a new file with an underscore (_) at the end of the filename. You can also specify a filename for the output file.

During our tests, it became apparent that Lepton only processes images up to a maximum size of about 17MB; with larger images, the process aborts and the OOM SHORT_READ error message appears. Fortunately, most JPEGs are smaller than 17MB.

Lepton generally works in multithreaded mode, employing multiple CPU cores if they are available. The -singlethread switch disables the multithreading feature, which will significantly slow down the conversion process but will release more resources.

Lepton also has a server mode, which you can run with the command:

lepton -listen=2020

In this case, Lepton listens on TCP port 2020 for incoming connection requests (Figure 2).

Figure 2: Lepton has a server mode that receives images via TCP.

Line 1 of Listing 2 sends a JPEG image to the server; the command in line 2 reconverts the file. In this example, the source and target are on the same computer.

Benchmark and Comparison

We compared Lepton with other compression tools by comparing the output file size for a 10MB JPEG file. Table 1 shows the results of various compression programs.

As you can see in Table 1, not a single conventional compression algorithm succeeded in significantly shrinking a JPEG without losing quality. Lepton, however, was able to shrink the file, although it managed just 2 MB/s – far less than the advertised 5 MB/s encoding rate.