SwiftWasm modules depending on JavaScriptKit fail to load when compiled with Swift 6.x compiler

[ephemer]

We updated to macOS Sequoia and are trying to get SwiftWasm working again in this environment.

After installing a Swift release toolchain (6.0.1) and a matching SwiftWasm SDK, we are able to build our Swift package, with some changes.

    swift.setInstance(instance);
    wasi.initialize(instance);
    swift.main(); // <-- new

    runCmd(
        "swift",
        [
            "build",
            "--swift-sdk", // <-- new
            "6.0-SNAPSHOT-2024-08-30-a-wasm32-unknown-wasi", // <-- new
            // "--skip-update", // for offline support
            "-c",
            BUILD_CONFIG,
            "--product",
            PRODUCT_NAME,
            "-Xswiftc",
            "-Xclang-linker",
            "-Xswiftc",
            "-mexec-model=reactor",
            "-Xlinker",
            "--export-if-defined=main",
            "-Xlinker",
            "--export-if-defined=__main_argc_argv", // <-- new

            "-Xswiftc",
            "-DJAVASCRIPTKIT_WITHOUT_WEAKREFS",
        ],
        {
            cwd: swiftPackageDir,
            env: { ...process.env, TOOLCHAINS: "Swift 6.0.1" }, // <-- new
            shell: true,
            stdio: "inherit",
        }
    );

After the above changes, almost everything works as expected.

Now here is the issue: previously we had issues with JavaScriptKit using resources in its target definition (in Package.swift) under certain circumstances – I think because we were using an API that "might" be provided by Foundation if it's imported (note that the resources flag depends on Foundation, see here).

This happens due to this line:

.target(
            name: "JavaScriptKit",
            dependencies: ["_CJavaScriptKit"],
            resources: [.copy("Runtime")] // <-- here
        ),

Now it seems we can't escape it. I can remove the resources line and the package will build fine, but doing so would require a fork of JavaScriptKit to get reproducible builds. Do you have any idea what could be going on here? Have you seen this issue yourself at all?

To be clear: we don't want or need Foundation at all in our project. It is being automatically imported due to the resources line in JavaScriptKit's Package.swift. I can't see how to workaround this any more in Swift 6.x: it seems like something has changed such that whenever resources is there, Foundation is required. Any ideas?

[ephemer]

For now, I went with the workaround here, but I do wonder what has changed since Swift 5.7 that means that Foundation is linked even if we're not using it at all. This appears to be a regression to me but I'm not sure it's related to SwiftWasm directly

[kateinoigakukun]

I totally understand your needs. I made an escape hatch for you #264
Does the hatch work for your case? If it works, I can maintain it in this mainline.

[ephemer]

This is interesting, thank you! I will have to check but I think we actually need the resources in there at some point to get a link to the jsRuntime.js file (which we process separately with our own build setup, similar to carton but integrating into our web app, using esbuild).

I just wonder what has changed compared to the older Swift toolchains: resources was always a part of JavaScriptKit but it didn't cause Foundation to be linked until we used it elsewhere (e.g. when we used a Foundation API). Now it seems that Foundation is linked even if we don't use it.

Our current workaround for this is to package our own module called Foundation which just contains a dummy class Bundle {} and the minimal API area for the autogenerated code to build. This way everything builds and links, without the "real" Foundation being pulled in. It's not ideal but it works with our requirement that jsRuntime is actually copied into the build directory by SwiftPM, avoids the runtime crash, and avoids linking the real Foundation.

[kateinoigakukun]

If I remember correctly, the issue that Foundation is always linked when using resources SwiftPM feature has existed since the introduction of the feature. So nothing has been changed in toolchain side. I think we have two directions to solve the issue.

1. Change interface to interact with package resources not to use Foundation API or expose a way to just copy resources without providing a way to access them from program.
2. Given that JSKit won't reference Bundle.module and Foundation doesn't define any retroactive conformance, Foundation dependency can be stripped away in theory. After evolving LTO functionalities, we can avoid linking Foundation in release build.

[ephemer]

Hi @kateinoigakukun, thank you for your reply. Maybe I'm being imprecise with my language.

My experience with this issue is that the SwiftPM generated code to import Foundation and expose the resource URL via Bundle.main has existed for years now. I think that is unchanged.

But: previous to Swift 6, unless we actually did something else that directly used Foundation APIs, e.g. using one of the extensions on String that only Foundation provides, Foundation would not be statically linked into the resulting binary. At the very least: the binary size did show that Foundation was being linked, AND we previously did not have issues with e.g. __CFDateInitialize not being found at load time. Maybe it was due to LTO working better in the past?

I'm not sure what was happening "under the hood" previously that is different now, but there indeed seems to be a regression IMO.

[kateinoigakukun]

Interesting. The new swift-foundation might have introduced some changes to make it difficult to GC at link time.

[kateinoigakukun]

I released a new way to manage the JS resource: https://github.com/swiftwasm/JavaScriptKit/releases/tag/0.24.0

With this way, we no longer depend on SwiftPM's resource feature, so we can make our module completely Foundation-free.