Skip to content

Desktop Linux Platform Issues

Jump to bottom
probonopd edited this page Nov 12, 2017 · 57 revisions

To make "Desktop Linux" a viable platform (as in: Windows, macOS) to develop against, we need to insist on backward compatibility and either need to find the "least common denominator" by experimentation (as we have done for over a decade now), or get the main Linux distributions (Ubuntu, Fedora, openSUSE, Debian, etc.) to agree on a certain set of infrastructure that can be expected to be "there", in a consistant way, without unnecessary differences.

Your thoughts are highly welcome. Please join the discussion here. However, please do not make substantial edits to this page before reaching consensus in the discussion on https://github.com/AppImage/AppImageKit/issues/445.

Credit: MuseScore (CC BY 2.0)

openSUSE Chairman Richard Brown pointed out the need for this in a talk at openSUSE Conference 2017:

This is not to say that the "least common denominator" approach does not work. In fact, the arguably two most successful "Desktop Linux" applications, LibreOffice and Firefox, are built on old build systems and bundle many libraries in order to be able to deliver one set of binaries to (almost) all "Desktop Linux" systems, independent of distributions, and have been doing this for quite some time now. But given some sane platform decisions, life could be made much easier for application developers who want to target the "Desktop Linux" platform.

This page is here to collect ideas on what would be worthwhile to standardize on. It is meant as a basis to get the discussion started rather than finalized recommendations. You are welcome to edit this page to add factual information, but if you fundamentally disagree with the direction of this page please talk to us first before editing this page.

Kernel

In the over a decade experience with application bundles the kernel has proven never to be an issue, since the interface between the kernel and userland is stable, and Linus Torvalds is heavily defending a stable userland interface. So no action required as far as AppImage is concerned. Applications should ensure they can run on a vanilla kernel without extra patches.

Paths

Library filenames

The names of libraries should reflect the version as given by upstream. Currently it is not, resulting in the necessity for each application using it to bundle a private version, since the application cannot rely on the system-provided one.

Example:

  • Ubuntu 12.04 has libarchive.so.12 desipte it being version 3.x
  • Ubuntu 14.04 has libarchive.so.13 desipte it being version 3.x

This should be changed so that every distribution that carries any version of libarchive version 3.x would contain libarchive.so.3.xxx and a symlink to libarchive.so.3, so that applications could rely on being able to link against libarchive.so.3.

The libarchive upstream developers say they can do nothing about this since the library filename is decided by distributions.

We might be able to make the number more consistent across platforms, but it would require someone to do the research into how cmake and autoconf generate build numbers on the many different platforms.

References:

Other example:

  • Mageia has /usr/lib/pulseaudio/libpulsecommon-10.0.so
  • App wants libpulsecommon-4.0.so

Library paths

Despite there being the FHS, filesystem paths are different on different reason for no apparent reason. They should be standardized between all distributions.

Example:

  • Debian, Ubuntu: /lib/i386-linux-gnu/, /lib/x86_64-linux-gnu/
  • openSUSE: /lib/, /lib64/

References:

Certificates

Should be installed to a standard location that is the same in all distributions. Currently this is not the case and where certificates go is apparently entirely random by distribution, as of mid-2017:

"/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
"/etc/pki/tls/certs/ca-bundle.crt",   // Fedora/RHEL
"/etc/ssl/ca-bundle.pem",             // OpenSUSE
"/etc/pki/tls/cacert.pem",            // OpenELEC
"/etc/ssl/certs",               // SLES10/SLES11, https://golang.org/issue/12139
"/system/etc/security/cacerts", // Android

Source: https://serverfault.com/questions/62496/ssl-certificate-location-on-unix-linux

The result is that if we bundle something like libgnutls, then it fails to find the certificates if the target system happens to store them in a different place.

Workaround: Patch whatever consumes certificates (e.g., libgnutls) to look in all known places. Example: https://github.com/darealshinji/vlc-AppImage/issues/1#issuecomment-321041496

Basic libraries

Applications should be able to rely on certain basic infrastructure libraries to be provided by all distributions in the default installation. They should be standardized between all distributions.

glibc

Ideally this library would be frozen indefinitely. Why is it necessary to still keep adding symbols to it, making software compiled against later versions fail on earlier versions? If freezing this is not realistic, then at least build systems should use older versions than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

  • ld-linux.so.2
  • ld-linux-x86-64.so.2
  • libanl.so.1
  • libBrokenLocale.so.1
  • libcidn.so.1
  • libcrypt.so.1
  • libc.so.6
  • libdl.so.2
  • libm.so.6
  • libmvec.so.1
  • libnsl.so.1
  • libnss_compat.so.2
  • libnss_db.so.2
  • libnss_dns.so.2
  • libnss_files.so.2
  • libnss_hesiod.so.2
  • libnss_nisplus.so.2
  • libnss_nis.so.2
  • libpthread.so.0
  • libresolv.so.2
  • librt.so.1
  • libthread_db.so.1
  • libutil.so.1

These files are all part of the GNU C Library which should never be bundled. List was generated from a fresh build of glibc 2.25.

References: https://www.gnu.org/software/libc/

libstdc++.so.6

Ideally this library would be frozen indefinitely. Why is it necessary to still keep adding symbols to it, making software compiled against later versions fail on earlier versions? If freezing this is not realistic, then at least build systems should use older versions than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of it should be guaranteed to still run with later versions of this.

  • libstdc++.so.6

References: https://gcc.gnu.org/wiki/Libstdc++

OpenSSL

This should be abstracted in a way so that software compiled against OpenSSL would work against whatever version of OpenSSL comes with the system. Currently software compiled against a certain "letter" version (e.g., "f", "l") of OpenSSL will fail when another version of OpenSSL is available on the system. Applications written against earlier versions of it should be guaranteed to still run with later versions of this.

Currently applications cannot rely on these:

  • libssl.so.1
  • libssl.so.1.0.0
  • libcrypto.so.1
  • libcrypto.so.1.0.0

They should be standardized across all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.

References:

According to https://twitter.com/Det_Conan_Kudo/status/896508995044880384 OpenSSL, the upstream project "bumps the soname on every version release". According to the source, "Downstream, we fix this. This provides the added value of literally being able to upgrade OpenSSL in a relatively safe way without busting the world in the process".

libgnutls

An application compiled against libgnutls26 should continue to work on systems that are using subsequent versions, such as libgnutls28 and libgnutls30. Possibly the versioning scheme of libgnutls would have to be changed for this, with new major versions (with ABI-breaking changes) coming out as rarely as possible (every 5-7 years or less frequently), and distributions should carry not only the recent version but also the version before to give sufficient time to developers to update.

Currently it looks like libgnutls26, libgnutls28 and libgnutls30 are all in use and not compatible.

Kerberos

The following libraries currently cannot be safely assumed to be part of every distribution:

  • libkrb5.so.26
  • libkrb5.so.3
  • libkrb5support.so.0

Distributions should agree on libkrb.so.5 being the name for Kerberos 5. Then applications could rely on these from the system rather than having to privately bundle their own copies.

Workaround: Patch libcurl so that it does not need Kerberos. https://github.com/AppImage/AppImages/blob/c9e6b6b365fcd97e2980c0844c478904c030a25a/recipes/meta/Recipe#L194

X11 libraries, Xlib, XCB

X11 libraries, Xlib, and the X protocol C-language Binding (XCB) should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

  • libX11.so.6 (uncertain if really all distributions bundle it)
  • libxcb.so.1 (uncertain if really all distributions bundle it)
  • libxcb-dri3.so.0 (uncertain if really all distributions bundle it)
  • libSM.so.6
  • libXp.so.6 (currently not available on Fedora default installations)
  • Many other libX...

OpenGL and Direct Rendering Manager (DRM)

OpenGL and the and Direct Rendering Manager (DRM) should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

  • libGL.so.1
  • libdrm.so.2

References:

ALSA and PulseAudio

Sound should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

  • libasound.so.2
  • PulseAudio libraries (tbd)

Example:

  • Mageia has /usr/lib/pulseaudio/libpulsecommon-10.0.so
  • App wants libpulsecommon-4.0.so

Gtk+ and Gdk

Gtk+ and Gdk should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

Ideally, there would be a defined library to develop against, e.g., libgtk.so.2 for Gtk+2 and libgtk.so.3 and applications would be compiled against those instead of a multitude of individual libraries that make up "Gtk+".

Example:

  • libgdk_pixbuf-2.0.so.0
  • Many more (are they defined somewhere)

Versioning scheme

GTK+ should follow semver.

According to https://blog.gtk.org/2016/09/01/versioning-and-long-term-stability-promise-in-gtk/, "Updates within long-term stable series will be ABI stable". This is good, but in order to minimize developer confusion about which GTK+ version can be relied on, only such long-term stable series should get major version numbers.

nkiesel nails it in https://lwn.net/Articles/691131/

However, there is the psychological problem of such applications always using the "previous" release. So innocent bystanders will see "GTK 5.0 released with all those new cool features", followed by "app FOO upgraded from GTK 3 to GTK 4". Therefore, GTK should make it very clear in it's press release that GTK 5.0 is just the first release candidate of GTK 5 and not intended to be used by external applications.

In this example, I would propose "GTK 5" to be called that only after it has a stable API and should be used by external applications. This would be more in line with the idea of a sane, dependable platform to build on. Everything that is still a moving target and has no API guarantees should not be worthy of a new major version number.

GLib

GLib should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older version than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of this should be guaranteed to still run with later versions of this.

  • libglib-2.0.so.0

Fonts and font rendering

Fonts and font rendering libraries should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

  • libfontconfig.so.1

These would ideally never have to be linked against directly:

  • libpangoft2-1.0.so.0
  • libpangocairo-1.0.so.0
  • libpango-1.0.so.0
  • libpangomm-1.4.so.1

With increasing frequency, we are running into symbol lookup error: /usr/lib64/libpangoft2-1.0.so.0: undefined symbol: hb_buffer_set_cluster_level. Why?

HarfBuzz

Bundling HarfBuzz seems to cause symbol lookup error: /usr/lib64/libpangoft2-1.0.so.0: undefined symbol: hb_buffer_set_cluster_level (suggesting that the version of HarfBuzz that is being bundled is too old for the lib64/libpangoft2-1.0.so.0 on the system; which means that we need to bundle HarfBuzz then we also must bundle libpangoft2-1.0.so.0).

HarfBuzz should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.

References:

Image formats

Currently every distribution names these differently:

  • libtiff (Fedora and openSUSE lack libtiff.so.4)
  • libpng
  • libjpeg

They should be named the same in all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.

NSS

Currently distributions ship no or incompatible versions of:

  • libnss3.so
  • libnssutil3.so
  • libsmime3.so
  • libssl3.so
  • libfreebl3.so
  • libfreeblpriv3.so
  • libnssckbi.so
  • libnssdbm3.so
  • libnsssysinit.so
  • libsoftokn3.so

They should be standardized across all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.

References:

PCRE

libpcre should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of this should be guaranteed to still run with later versions of this

  • openSUSE (as of 2017) lacks libpcre.so.3 but provides libpcre.so.1
  • Ubuntu (as of 2017) lacks libpcre.so.1 but provides libpcre.so.3

According to https://twitter.com/Det_Conan_Kudo/status/896508995044880384 PCRE sonames are wrong in debian, and hence, probably Ubuntu. According to the source, it cannot be fixed "because now it's the ABI contract of Debian." According to https://github.com/AppImage/AppImages/issues/83#issuecomment-322044692, "libpcre.so.3 is a historical Debian/Ubuntu screwup. Everywhere else it is libpcre.so.1."

Weak dependencies

Certain distribution functionality/libraries should be purely optional. Software compiled on distributions should never require these as hard dependencies, but should rather check whether these are available, and fall back gracefully if they are not there.

  • libselinux.so.1. Currently some software compiled on CentOS requires libselinux.so.1 even if SELinux is not used or wanted
  • libapparmor.so.1. According to https://twitter.com/Det_Conan_Kudo/status/896733648619270144, this equally applies to AppArmor. According to the source, apparently binaries on Ubuntu, SUSE and Debian link to it

Workaround: dummy libraries that resolves all of their symbols but just do nothing, e.g., https://github.com/AppImage/AppImages/issues/83#issuecomment-321966519

ELF handling in desktop environments

Desktop environments should handle ELF files with missing executable bits more gracefully. https://github.com/AppImage/AppImageKit/issues/374

SELinux

In the realm of security, distribution vendors are going different paths (Red Hat using NSA-originated SELinux and others using AppArmor or other technologies. Which is fine, as long as these systems don't spill over into each and every application, as is the case with libselinux.so.1, which gets linked into each application compiled on Red Hat based systems like CentOS, while it is not available on some other systems.

Security enhancements in the operating system should not require applications to link to libraries like libselinux.so.1.

Workaround: Bundle libselinux.so.1 with every application that needs it (but since it depends on libpcre.so.3 we need also bundle that because distributions can't be relied on to ship the correct version under the same name). Or, better, replace it by a dummy library that resolves all of its symbols but just does nothing, e.g., https://github.com/AppImage/AppImages/issues/83#issuecomment-321966519

FUSE

Although working out-of-the box in >95% of desktop distributions in the default installation, there are still 5% where this is not the case.

Distributions should ensure that FUSE is working out of the box and that all permissions are set correctly for regular users.

Compilers

Ideally, which version of a compiler was used to compile a binary should not determine which version of libraries the compiled binaries need. However, according to https://stackoverflow.com/a/42705277, the GCC version used determines the minimum version of libstdc++.so.6:

Sometimes you don't control the target machine (e.g. your library needs to run on a locked-down enterprise system). In such a case you will need to recompile your code using the version of GCC that corresponds to their GLIBCXX version. In that case, you can do the following:

  1. Look up the latest version of GLIBCXX supported by the target machine: strings /usr/lib/libstdc++.so.6 | grep GLIBC ... Say the version is 3.4.19.
  2. Use https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html to find the corresponding GCC version. In our case, this is [4.8.3, 4.9.0).

So, in practice, if you want your binary to run on say, all still-supported Ubuntu LTS releases (14.04 at the point of this writing), then you must not use a compiler newer than GCC 4.8.3.

Question: Can the compiler version be decoupled from the libraries used by the resulting binaries by using a cross-compiler? If so, how?

Individual distributions' platform promises

Some distributions make their own platform promises. Whether this can be part of the solution or adds to the complexity remains to be determined. Of course, this is insufficient. We need platform promises for "Desktop Linux".

Red Hat Enterprise Linux

https://access.redhat.com/articles/rhel-abi-compatibility describes the Red Hat Enterprise Linux platform guarantees.

Limit linking applications to the core libraries at compatibility level 1. The core libraries (see Appendix A) are intended to preserve binary compatibility across three consecutive major releases.

Provide compatibility libraries for applications that have been built with libraries that are not at the desired compatibility level, provided the bundled libraries themselves only use the interfaces provided by the core libraries.

Compatibility level 1: APIs and ABIs are stable across three major releases (starting with RHEL 7)

alsa-lib elfutils-libelf glibc glibc-utils gtk2 hesiod krb5-libs libgcc libgfortran libgomp libstdc++ libtbbmalloc_proxy.so libtbbmalloc.so libtbb.so libusb libvirt-client libxml2 libxslt mesa-libGL mesa-libGLU motif pam SDL

This means that we can trust that if you build against these libraries and bundle everything else privately, then your application is expected to work at least in RHEL 7, 8, and 9.

Note that gtk3 at this point is in "Compatibility level 2: APIs and ABIs are stable within one major release (RHEL 7)". Which means that a gtk3 application might break in RHEL 8. Which also means that application developers should not use it before it also enters compatibility level 1, or else they cannot be sure that their application will continue working in the foreseeable future.

Relevant forums

Where should we discuss this? Please add ideas here.

References

Clone this wiki locally