Platforms that don't have "fat" multi-architecture binaries should place their runtime libraries in an arch-specific directory

[finagolfin]

Description
This is a long-standing issue where we cannot ship or link against the stdlib for different architectures in the same Swift resource directory, because the compiler places and adds a rpath to them in lib/swift/linux, not lib/swift/linux/x86_64 and lib/swift/linux/aarch64:

> find swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/lib_InternalSwiftStaticMirror.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Glibc.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Glibc.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Glibc.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Glibc.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicuucswift.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_RegexParser.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_RegexParser.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_RegexParser.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_RegexParser.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicuucswift.so.65
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Concurrency.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Concurrency.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Concurrency.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Concurrency.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftSwiftOnoneSupport.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicudataswift.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicui18nswift.so.65.1
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/lib_InternalSwiftSyntaxParser.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicui18nswift.so.65
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libFoundationNetworking.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Differentiation.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Differentiation.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Differentiation.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_Differentiation.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libdispatch.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/RegexBuilder.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/RegexBuilder.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/RegexBuilder.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/RegexBuilder.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicudataswift.so.65.1
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswift_Differentiation.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Swift.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Swift.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswift_RegexParser.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_StringProcessing.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_StringProcessing.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_StringProcessing.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/_StringProcessing.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libFoundationXML.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libBlocksRuntime.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicudataswift.so.65
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libXCTest.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/lib_InternalSwiftScan.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicuucswift.so.65.1
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftDistributed.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftDispatch.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libicui18nswift.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswift_Concurrency.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libFoundation.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftGlibc.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/SwiftOnoneSupport.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/SwiftOnoneSupport.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/SwiftOnoneSupport.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/SwiftOnoneSupport.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftRemoteMirror.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftRegexBuilder.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Distributed.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Distributed.swiftmodule/x86_64-unknown-linux-gnu.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Distributed.swiftmodule/x86_64-unknown-linux-gnu.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/Distributed.swiftmodule/x86_64-unknown-linux-gnu.swiftinterface
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswiftCore.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/libswift_StringProcessing.so
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/Foundation.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/FoundationXML.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/FoundationXML.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/Foundation.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/Dispatch.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/Dispatch.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/glibc.modulemap
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/XCTest.swiftmodule
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/SwiftGlibc.h
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/swiftrt.o
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/FoundationNetworking.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/XCTest.swiftdoc
swift-5.7.3-RELEASE-ubuntu20.04/usr/lib/swift/linux/x86_64/FoundationNetworking.swiftmodule

@compnerd wrote about these issues when the problem was worse four years ago, and while some of that has since been fixed, this remaining issue of the runtime libraries location came up again last year.

Steps to reproduce

1. Build the toolchain for linux x86_64 and install it.
2. Build the toolchain for linux aarch64 and install it to the same directory (build-script currently does not support building two non-Darwin architectures at once, so it has to be run twice.).

Expected behavior
Both architectures' runtime libraries install fine. Instead, the latter will overwrite the former.

Environment

• All Swift versions

Additional context
@compnerd fixed this for Windows already by installing the runtime libraries to arch-specific directories, and having the Swift driver look in those arch-specific directories instead when linking.

I've put together a similar pull for the Unix toolchain and it passes the compiler validation suite natively on Android, after updating some tests. I will submit it soon, once I get it to install the libraries properly when setting up the full toolchain, along with the needed modifications to the corelibs install.

@MaxDesiatov or @kateinoigakukun, what would you like to do for the wasm toolchain? I currently disable these changes in my pull for Darwin, which has fat libraries so doesn't need it, and WASI, because I don't know what that platform requires.

[kateinoigakukun]

@buttaface Thank you for working on this! It does not have a fat library format for WASI, so the same rule can be applied as well as for linux.