SwiftPM built with Swift 6.0 and later snapshots crashes on macOS 14

[kateinoigakukun]

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

• Confirmed reproduction steps with SwiftPM CLI.

Description

You can reproduce the crash by running the following simple command on macOS 14 and later with swift-DEVELOPMENT-SNAPSHOT-2024-04-09-a or later.

$ /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2024-04-09-a.xctoolchain/usr/bin/swift package --version
zsh: segmentation fault   package --version

The segmentation fault is caused by trying to dereference an async function pointer $ss23withCheckedContinuation9isolation8function_xScA_pSgYi_SSyScCyxs5NeverOGXEtYalFTu, which is introduced by #72578 and only available in toolchain stdlib but unavailable in OS stdlib yet.

Note that #72578 itself is not wrong, but it just revealed a long-living issue.

Solution?

All images compiled with snapshot toolchains should use toolchain stdlib instead of OS stdlib because:

1. _stdlib_isOSVersionAtLeast(9999._builtinWordValue, 0._builtinWordValue, 0._builtinWordValue) always returns true after [Stdlib] Special-case major version 9999 as always available. #22658
2. All new stdlib APIs introduced in the next release are marked with @available(macOS 9999, ...) until the version is fixed to be a concrete OS version.
3. if #available and @backDeploy check the running OS version by using _stdlib_isOSVersionAtLeast, and it leads to referencing symbols with @available(macOS 9999) at runtime.

So as long as we use the availability placeholder, executables built with such stdlib must link exactly the same stdlib library at runtime.

Why it doesn't happen on macOS 13 or earlier?

It looks like the /usr/lib/swift/libswiftCore.dylib in the version did not include #22658, so _stdlib_isOSVersionAtLeast(9999._builtinWordValue, 0._builtinWordValue, 0._builtinWordValue) returns false.

Alternative Consideration

You might think we can revert #22658 not to make APIs marked with the placeholder always available and adjust test suites. However the implementation is already shipped as a part of macOS 14, so the issue remains as long as we use macOS 14.

Expected behavior

Not crash

Swift & OS version (output of swift --version ; uname -a)

Apple Swift version 6.0-dev (LLVM 6bf0f1dbf510217, Swift f36060f01135902)
Target: arm64-apple-macosx14.0
Darwin Yutas-Mac-mini.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:41 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T8103 arm64

[kateinoigakukun]

Hmm, due to the absolute install_name specified by LC_ID_DYLIB in libswiftCore.dylib in toolchains, Swift executables always load libswiftCore.dylib from absolute /usr/lib/swift/libswiftCore.dylib without checking @rpath.
So just adding -rpath linker option did not work unfortunately.

[kateinoigakukun]

Another approach might be reverting #22658 and making _stdlib_isOSVersionAtLeast @_alwaysEmitIntoClient.

[kateinoigakukun]

#72801 seems trying to solve the same issue for typed throws

[rauhul]

+1 I'm hitting this locally on my Mac running 14.0

[MaxDesiatov]

The issue only seems to reproduce when SwiftPM is built with 6.0 toolchains. I can't reproduce these crashes when building SwiftPM itself with Swift 5.10. IMO this should be moved to the toolchain repository.

[omochi]

I'll share some information that might be of use.
My mac is on Sonoma 14.1, but I didn't encounter this issue. Therefore, I ran the following code to investigate the behavior of _stdlib_isOSVersionAtLeast which is included in the OS.

#include <stdio.h>
#include <stdint.h>
extern int8_t $ss26_stdlib_isOSVersionAtLeastyBi1_Bw_BwBwtF(int64_t major, int64_t minor, int64_t patch);
int main() {
    int n = 6;
    int64_t vers[][3] = {
        { 13, 0, 0 },
        { 14, 0, 0 },
        { 14, 1, 0 },
        { 14, 2, 0 },
        { 9998, 0, 0 },
        { 9999, 0, 0 },
    };
    for (int i = 0; i < n; i++) {
        int64_t *ver = vers[i];
        int8_t y = $ss26_stdlib_isOSVersionAtLeastyBi1_Bw_BwBwtF(ver[0], ver[1], ver[2]);
        printf("(%lld, %lld, %lld) => %d\n", ver[0], ver[1], ver[2], y);
    }
    return 0;
}

I built and ran this code using the following command:

$ clang -L $(xcrun --show-sdk-path)/usr/lib/swift -lswiftcore v.c && ./a.out

The output was as follows:

(13, 0, 0) => 1
(14, 0, 0) => 1
(14, 1, 0) => 1
(14, 2, 0) => 0
(9998, 0, 0) => 0
(9999, 0, 0) => 0

The true result was obtained for 14.1, and false for 14.2, so it seems that isOSVersionAtLeast was called correctly. Moreover, I got false for version 9999. It appears that the libswiftCore in Sonoma 14.1 does not include #22658. It's curious since it was merged back in 2019.

My Mac is a MacBook Pro 2023, 14-inch, Apple M2 Pro.

Now then, I will try updating to Sonoma 14.4.

[omochi]

I have updated to macOS 14.4.1. Upon retesting, I obtained the following output:

(13, 0, 0) => 1
(14, 0, 0) => 1
(14, 1, 0) => 1
(14, 2, 0) => 1
(14, 3, 0) => 1
(14, 4, 0) => 1
(14, 4, 1) => 1
(14, 4, 2) => 0
(14, 5, 0) => 0
(9998, 0, 0) => 0
(9999, 0, 0) => 1

Special handling for 9999 has occurred. #22658 seems to have been introduced at some point between 14.1 and 14.4.1.

Now, this prevents me from using the latest compiler for Swift for Wasm development. This is troubling...