Why universal packages aren't universal solutions

Off the Beat: Bruce Byfield's Blog

The initial announcements of Flatpak and Snap presented them as the solution to all of Linux's packaging problems. These claims soon proved to be ahead of actual development, but they linger in the minds of many users. A few weeks ago, for example, someone on Google+ was complaining about how long their distribution was taking to package LibreOffice 5.3. They looked forward, they wrote, to the day when one universal package manager or another would eliminate such delays. However, while universal package managers might one day simplify maintaining a distribution, whether they will ever have the effect that the complainer anticipated seems doubtful -- for which we can all be thankful.

I suspect that the availability of several hundred distributions gave the complainer the impression that preparing a distribution is a relatively easy thing to do. If so, they are being misled.

The majority of distributions are not created from scratch, but by modifications of existing distributions. In fact, two-thirds of the distributions on Distrowatch are derived from Debian packages, other directly, or through Ubuntu. Even Ubuntu and Linux Mint are constructed in this way, and, although they do their own testing and construction, the work is eased by the fact that Debian has already done much of  it in advance. Otherwise, distributions made by two or three people would take so long to release that they would make Debian itself look like a rolling release in comparison.

Anyone who has ever had anything to do with a distribution knows that assembling a release involves more than simply wrapping applications into packages and adding them to a repository. Packages have to be checked for consistency, compatibility, and security. The amount of testing done varies with the distribution, and can be partly automated, but at some point, someone has to do the work. Perhaps some day, universal package managers might help reduce the workload, but they are unlikely to eliminate it entirely any time soon.

Meanwhile, to get a sense of the work involved, look at the Debian Policy Guidelines. The Guidelines are pages after pages about what a Debian package must or can contain, and how it must interact with other packages. Admittedly, few other distributions are so comprehensive, but after reading the Debian Policy Guidelines, you can appreciate why Debian sometimes goes so long between releases -- and also why Debian is favored by the security-conscious and as the source of so many derivative distributions. Making distributions from scratch is hard work, which is why the freeze before a Debian release has lasted several months in the past.

Upstream vs. Distributions
When I tried to point out these basic facts, the original complainer suggested that universal packages were no different from downloading the latest version of Firefox or LibreOffice from an upstream repository. However, the analogy does not hold.

Such upstream packages are generally either standalone packages, or ones with a small number of dependencies, and, when they fail, the desktop environment or operating system that runs them is unlikely to be affected to any great extent -- although you might have broken packages with which to contend.

However, downloading such applications -- which tend to be mostly designed for the desktop -- is one thing, and building an entire distribution, including core packages quite another. The odds that packages from a variety of upstream sources will all interact seamlessly just because they share a package format lie somewhere between remote and impossible.

For that matter, even when upstream packages are available, distributions tend to package their own versions. The upstream packages are the quickest way to see what's new, but the distribution packages that arrive a week or two later tend to be stabler and to suffer from fewer problems, because they have been integrated into the distribution in a way that their upstream equivalents have not.

Nor should this state of affairs be surprising. Upstream projects are not primarily concerned with packaging. For them, packaging is a courtesy, a way of getting more people to try the latest release than they would get if they only posted source code. Even the relative ease of making Flatpak or Snap packages is not likely to change the priorities of an upstream project.

By contrast, packaging is the main business of distributions. Under these circumstances, who do you think is likely to prepare the less trouble-free packages?

Policy, not technology
These comments do not mean that Flatpak or Snap packages have no uses -- just that they are overhyped. If distributions are slow to adopt them, the reason is not that developers are conservative. Rather, they know from experience that what makes their work stable and secure is policy, far more than any technology.

When universal packages are further along in their development, undoubtedly a distribution or two will start to use them. However, when those distributions appear, their success will not be due to their package format, but to the policies used to create a coherent distribution.

And if you doubt that, have another look at how Debian is assembled.