Use base docker image for multiple compiler versions

[uilianries]

Description of Problem, Request, or Question

We are running around our tail when trying to release new entire docker image for each new compiler version. This situation is worst when thinking about the multiple glibc versions available and the age of some images (based on its distro and support).

This issue asks for a new approach: A base image which can be reused by any compiler version.

What's the cost? Build most of compiler from sources and increase the CI.

Pros:

• Better maintenance, fix only one image
• Easier to add a new compiler version, only update docker-compose file
• Same glibc version

What about the old images? Keep then in the repository, but do not keep the support. The new images should have a new name, to avoid any conflict.

closes #204

[Croydon]

The problem is what should the base image be?

If we choose e.g. Xenial that the glib is probably newer than that of the currently RHEL supported versions (haven't checked it yet)

But if we get this sorted out that an unified base image would be wonderful obviously

[uilianries]

We will rebuild the packages in CCI to avoid any problem, also, not all images will be update, it will be from GCC 6.

[ericLemanissier]

so this won't solve conan-io/conan-center-index#213, because all installer images will still be based on xenial, and gcc 4.9 will still be based on trusty. Or am I missing something ?

[uilianries]

that's another case, we need to filter installer packages in CCI and build then using a specific image (old).

[Croydon]

The problem seems to be that packages are first build with GCC 5, meaning that installer packages are ONLY build on GCC 5.

The glibc verison of our GCC 5 container is newer as the one in the GCC 4.9 container, so it will fail to use this executables in GCC 4.9 CI builds.

If this changes only GCC 6 and newer, then it won't solve the problem within our CI builds as @ericLemanissier pointed out.

But such problems don't happen only in our CI builds, it is the same for consumers with Linux distribution versions which use older glibc versions.
That's why INCREASING the glibc version might actually be the opposite of what we want.

---

I have no clue how hard it is to do, but just to throw this idea from my head into the wild: When we already considering building the compilers from source, maybe building a light distribution ourselves with Yocto might be a solution? In (absolute) theory, this could allows us to combine old glic versions with all compiler versions we want?

[uilianries]

Again, this issue is not related to conan-io/conan-center-index#213. Here we want to avoid new glibc version and distro versions.

That issue requires a CCI intervention, doesn't matter how we change the docker images here. The CentOS image it would be nice for that case, but first CCI team needs to filter the installer package (probably filtering the settings, options and package content), and running a specific image.

In (absolute) theory, this could allows us to combine old glic versions with all compiler versions we want?

It's possible I think, but it would require extra effort. IHMO keeping only Ubuntu 16.04 is enough for most of cases.

[danimtb]

We have not decided yet what will happen with "installer" packages. But we aim to have the same treatment for everyone, as it is up to the consumer to use it as a build require for cross-building or as a requirement for a library. What we have in mind is to clearly state the environment used to build the packages, explain the glibc issue and the version we are using and encourage users facing this issue to build from sources.

[ghost]

The compiler and standard library (both C and C++) should be Conan packages themselves. That is the easiest way to manage everything.

[jgsogo]

Copying from #252 (comment)

• The plan is to move to the same base docker image for all the Linux compilers, ATM the chosen one is Ubuntu 16.04 (LTS). We still don't know if this scenario is possible, there are some questions that need answers:

  • Can we get to run all the compilers (from GCC 4.9 to GCC 10, and clang 3.9 to 11) in this version of Ubuntu? We will need to compile from sources some of them... (versions TBD conan-io/conan-center-index#4761)
  • We would need to compile some missing packages too: is libc++11 available for 16.04? can we build it from sources?

  Maybe we realize we cannot use the same base image for all the compilers and it will totally change the perspective from where to look at this issue.

[uilianries]

@jgsogo said:

There are several questions that came to me related to this issue and they will help us a lot with other ongoing efforts as well.

The plan is to move to the same base docker image for all the Linux compilers, ATM the chosen one is Ubuntu 16.04 (LTS). We still don't know if this scenario is possible, there are some questions that need answers:
    Can we get to run all the compilers (from GCC 4.9 to GCC 10, and clang 3.9 to 11) in this version of Ubuntu? We will need to compile from sources some of them... (versions TBD conan-io/conan-center-index#4761)
    We would need to compile some missing packages too: is libc++11 available for 16.04? can we build it from sources?

Maybe we realize we cannot use the same base image for all the compilers and it will totally change the perspective from where to look at this issue.

We need to understand why the libc++ package now is called libc++11, surely there is some ABI break. We just need to confirm it, it is ok to write a demo app to demonstrate it. It won't be a blocker, but we need to know the implications of the decision.
    Can LIBCXX_ABI_VERSION help us? Can we use version 11 but use LIBCXX_ABI_VERSION to target old ABIs and still get all the modern C++ features?

If ABI is incompatible, should we consider it as a new entry to compiler.libcxx for Clang compilers and add libc++11 to the list?

Can we get to run all the compilers (from GCC 4.9 to GCC 10, and clang 3.9 to 11) in this version of Ubuntu? We will need to compile from sources some of them... (versions TBD conan-io/conan-center-index#4761)

Javi, thank you for your thoughts, indeed they are good question about CDT future. Here are my thoughts.

Yes, that's possible. Also, @SSE4 made some effort in the past, creating a Conan recipe for GCC. Maybe, we can use it if necessary. The only problem I see building from sources is CI time, last time I tried to build GCC, it took 3h30. Only if we move to Jenkins we could build it, because no free CI service offers such amount of time.

We would need to compile some missing packages too: is libc++11 available for 16.04? can we build it from sources?

Clang and libc++ or libc++11 are distributed as pre-built on LLVM page, we can download from there.

We need to understand why the libc++ package now is called libc++11, surely there is some ABI break. We just need to confirm it, it is ok to write a demo app to demonstrate it. It won't be a blocker, but we need to know the implications of the decision.

Actually, looking into Ubuntu packages, not only libc++-11-dev is available, but also all versions, like libc++-8-dev, libc++-9-dev, ... The point is, we never aligned them. Part because the mistake of reusing the same Docker recipe for a new Clang version, and part because Ubuntu only releases libc++ months after its original date, coming as -update

If ABI is incompatible, should we consider it as a new entry to compiler.libcxx for Clang compilers and add libc++11 to the list?

I think so, otherwise we are lying.

[Croydon]

Also, @SSE4 made some effort in the past, creating a Conan recipe for GCC

See conan-io/conan-center-index#1694