Bootstrapping

[staticfloat]

Many packages will require some kind of "bootstrapping"; e.g. the ability to run the executables they have just built. We should look into solutions such as Wine for Windows, Darling for MacOS and QEMU for non-Intel Linux.

Ideally, I'd like to make this transparent and automatic. I think the best route here is for us to have binfmt_misc be enabled and setup properly within whatever virtualization environment our builds run, which should be possible if we ditch Docker.

[Keno]

While I think this is a reasonable thing to do, I also think we should very strongly encourage packages to fix their build systems to not require this where possible.

[staticfloat]

The next rootfs will embed wine, wine64 and qemu-user-static. staticfloat/julia-docker#15

• We can add code in BinaryBuilder.jl to auto-register with binfmt_misc on Linux systems. Is it possible to register binfmt_misc within the usernamespace and thereby not require sudo privileges for our unprivileged containers? Or is that kernel interface not integrated within user namespaces yet?

• Can we do qemu-within-qemu for when we're running on OSX? Sure, it'll be slow, but hopefully this is only used for when we're running cmake autoprobe executables anyway.

[Keno]

We can add code in BinaryBuilder.jl to auto-register with binfmt_misc on Linux systems. Is it possible to register binfmt_misc within the usernamespace and thereby not require sudo privileges for our unprivileged containers? Or is that kernel interface not integrated within user namespaces yet?

Yes, I believe this works

Can we do qemu-within-qemu for when we're running on OSX? Sure, it'll be slow, but hopefully this is only used for when we're running cmake autoprobe executables anyway.

Different kinds of qemu. I think that should nest just fine.

[sloede]

Is there any progress on this issue? Another advantage I see from being able to run executables is that it would allow to immediately test the build results, e.g., in cases where the build process provides a binary that can be used to ensure the functionality of the built library. Especially for Windows builds I'd see this as greatly beneficial, since (I assume) many developers probably do not have ready access to a Windows machine for easy testing.

[bramtayl]

This seems to required for Jack1, Jack2, and a couple of optional dependencies for PulseAudio. Is there any way I could help nudge it along or contribute? What's the first step?

[eschnett]

@staticfloat says above "[...] which should be possible if we ditch Docker." Why is this? As far as I know, Docker supports binfmt_misc and qemu. It's enabled by default on macOS, but needs to be explicitly set up on Linux https://docs.docker.com/desktop/multi-arch/ .

[staticfloat]

That is now true (I don't think it was 5 years ago in 2017!) and is in fact supported by Sandbox.jl, which is what the next iteration of BinaryBuilder will be built on top of.