Command Line – Installersn

A well-known xkcd cartoon starts with 14 competing standards [1]. Someone resolves to simplify the situation by developing one standard that covers all the others. The result? Fifteen competing standards. The cartoon reflects a wry truth that has recently been illustrated in a proliferation of software installers that complicates matters for those who install cutting-edge applications. Increasingly, would-be users have to learn a new installer and install more software before they can get to the software they want to try.

When Linux first became popular in the late 1990s, software installation often ended in what was known as "dependency hell," in which users had to track down the correct version of each dependency and risked leaving a package half-installed. One way around this problem was to use static tarballs – archives that included the required dependencies. These were used in the first forays into commercial Linux software, like Loki Entertainment, and they were easy to create with the tar command. However, static tarballs were not always used, probably because they often meant that one hard drive might contain several versions of the same software, which was wasteful at a time when storage was limited.

Another early solution was to install software written in C or C++ from source, creating the necessary binaries. If you were lucky, the install scripts would even list and install dependencies, although less conscientious developers might still strand users in dependency hell. Installing from source typically begins by switching into the directory that contains the source code and running ./configure to check that all the required dependencies are already installed (Figure 1). For instance, one of the first checks that ./configure usually does is to check whether the system has the GCC compiler installed and running correctly. In addition, ./compile creates the files needed to compile, such as the makefile that contains the basic instructions for compiling. If all dependencies are present, you can then compile the binary using the make command. If the build is successful, the command make install installs the application. This system at least provides some minimal guidance in dealing with problems and is still used today, as well as variants like KDE's CMake.

Figure 1: The ./configure command is used to compile C and C++ source code, along with make and make install.

Installation took a giant step forward when Debian became the first distribution to introduce package management. With package management, applications are presented along with an array of scripts that handle dependencies (as well as any alterations to the system before, during, and after installation) and draw from a standard set of online repositories. The efficiency, as well as the economy of memory, meant that package management soon spread to other major distributions. The commands may differ – apt-get or dpkg for Debian, yum or dnf for Fedora, and pacman for Arch Linux – but the structure of package managers is much the same in all distributions, with basic commands for installation, deletion, and repository searching and updating, as well as a centralized database for all installations. Several years ago, Ubuntu introduced apt, a convenient subset of apt-get functions (Figure 2), but the structure of package management has remained the same for over two decades. Most Linux users have likely used a package manager, if only through a graphical interface.

Figure 2: Typical of modern package manager commands, apt explains the results of requests and reports on progress.

Universal Packages

Today, Linux computing is vastly different than it was in its infancy. Memory is abundant now, and the restrictions of earlier times no longer seem relevant to many. For instance, on many systems, package installation no longer needs to be part of system administration in order to manage limited system resources. Instead, ordinary users can install software for their own use. Similarly, there is room for multiple versions of resources, something that package management generally avoided. And, most importantly, many developers would prefer to build a single file for all distributions, in effect making software installation on Linux closer to what it is on Windows or macOS.

Currently, there are three main universal package systems: AppImage, Canonical's Snap, and Fedora's Flatpak. Announced in 2016, Snap and Flatpak helped to renew the interest in AppImage (Figure 3). All three are structured much like traditional package management systems and might be said to be a more sophisticated version of static tarballs. However, although all three are highly promising (e.g., Flatpak's home page declares the format "The Future of Apps on Linux"), distributions differ enough in details like the location of files that one format that fits all is not always easy to create. Several distributions like openSUSE now maintain releases in a universal format, but all three universal package systems have become just more standards among many.

Figure 3: AppImage requires only a download and a change in permissions to run.

The New Installers

Another growing trend is to use the installer included in a programming language. For instance, Carp is the package manager for the Rust programming language. Similarly, Ruby borrows Homebrew from Apple.

The most common of these programming language installers is pip, the Python installer (Figure 4). Until January 2020, a pip version existed for Python releases before 3.x and is undoubtedly still around. For current versions of Python, pip3 is the command. However, somewhat confusingly, at the command prompt, pip3 is referred to merely as pip. The command structure is further complicated by calling pip from Python3:

python3 -m pip install PACKAGE

Figure 4: Python's pip installer closely resembles standard package installers.

The package can be defined by a local path, or the version control or Python repository. An exact version can be specified by PACKAGE==VERSION, while specifying a version and adding <HIGHER-VERSION or >LOWER-VERSION specifies a range of installation candidates. In addition, ~=VERSION specifies an installation candidate compatible with a certain version.

Almost alone among the other installation choices, this syntax is more complicated than that of other package managers discussed here. However, it is included, because, as its use of Debian syntax implies, pip seems designed as an improvement over traditional package management. Its syntax makes pip an installer for experts, providing them with advanced tools.

Cloning from Servers

An increasingly common installation method is to copy the files from a version control repository. The most common command for this purpose is git, using the command git clone URL (Figure 5), but curl and wget are also occasionally used. With all three commands, several scenarios are possible. You might copy only source code and find installation instructions in a README file. If the application is written in an interpreted language like Python, the copied files are ready for use. Alternatively, a developer may have included a compiled binary and all the dependencies. In all these cases, you only need to follow the provided instructions, although you may need to install a tool or two before you install.

Figure 5: Cloning a Git repository can be a quick way to install an application.