Update our AZLinux 3.0 .NET 9 images to build a custom standard library toolchain

[jkoritzinsky]

Update our docker images to build a toolchain with a static libc++ that uses the target-provided libstdc++ for the C++ Itanium ABI.

This scenario has some limitations (we're limited to API features that don't use any ABI features exposed since 2016), but it works fine for our C++11 code in dotnet/runtime.

The sanitizer instrumentation in libc++ isn't compatible with this old of a libstdc++ though, so this PR provides a separate image for that scenario. That image provides a statically linked libc++ that includes libc++abi. Additionally, it instruments libc++ and libc++abi directly with ASAN so our ASAN testing can have a fully instrumented toolchain.

This isn't required for ASAN, but is required for other sanitizers we may consider adding validation with in the future.

[sbomer]

Looking good! Some questions mostly for my own understanding.

[sbomer]

Why can't we use the cxxabi.h that comes with libstdc++? Is this for the libc++ build, or is it installing headers to be used when building dotnet/runtime with libc++?

[jkoritzinsky]

We can't use the one from libstdc++ as the include directory it is in is also the directory where libstdc++ defines the standard headers.

We need this for the build of createdump, which uses __cxa_demangle to demangle symbol names.

Alternatively, we could copy the cxxabi.h file from the libstdc++ location in the rootfs into the libc++ location.

Or we could have dotnet/runtime define a header that defines the required function prototypes (instead of using the header from the ABI library).

[sbomer]

Alternatively, we could copy the cxxabi.h file from the libstdc++ location in the rootfs into the libc++ location.

This option makes most sense to me. I think using the headers that are installed with the library we will link against is the surest way to make sure they match - but I think you have the best context to make the decision here, so I'll leave it up to you.

[sbomer]

Will this put any non-cross-compilation resources into that directory? With the current images I see:

# clang -print-resource-dir
/usr/local/lib/clang/18
# ls /usr/local/lib/clang/18
include lib

[jkoritzinsky]

Yes, this will put all assets into the cross folder (and Clang will only look in that folder, not one with the major version in the path).

[sbomer]

In that case, as a minor nit I wouldn't name it cross since it has all of the resources (including non-cross compilations).

[jkoritzinsky]

I'll rename it to resources then (or figure out a way to get the major version of clang out of the command line).

I'm trying to reduce the number of places where we need to specify (and update later) the version of LLVM/Clang that we're building.

[sbomer]

Does building libc++ also make it the default for clang on these images? Or does the runtime build need to set some flags to build against libc++?

[jkoritzinsky]

This does make libc++ the default. We'll have to make some changes to the dotnet/runtime build to utilize these new features in the images (by default they'll use the libstdc++ that they've been using).

[jkoritzinsky]

Waiting to merge until usage UX in dotnet/runtime#101773 is agreed upon.

[jkoritzinsky]

Usage UX has been defined, I'll merge this in and (once the new images are built) update the PR in dotnet/runtime with the sanitizer fixes.