[Build] Add the new fully-static Linux SDK. #71839

al45tair · 2024-02-23T16:10:31Z

Declare a new LINUX_STATIC SDK and configure it.

Add options to set the build architectures for the LINUX and LINUX_STATIC SDKs, similar to what we have for Darwin, because we'll be cross-compiling.

Also add an option to point the build system at the sources for the musl C library, which we're using for LINUX_STATIC.

rdar://123503470

al45tair · 2024-02-23T16:10:56Z

This sits on top of #71838.

al45tair · 2024-02-23T16:11:03Z

@swift-ci Please smoke test

al45tair · 2024-02-27T15:11:11Z

@swift-ci Please smoke test

al45tair · 2024-02-28T12:16:25Z

@swift-ci Please smoke test

al45tair · 2024-03-14T15:34:48Z

@swift-ci Please smoke test

edymtt

The overall approach looks good, given the moderate size I did a first pass with my initial impressions -- loosely wondering if there is a way to break this PR in smaller chunks (e.g. land first the plumbing of the build-script args, and leave last the more substantial changes to the CMake logic that may require more discussion).

utils/build-script

utils/build_swift/build_swift/driver_arguments.py

edymtt · 2024-03-26T22:31:18Z

utils/swift_build_support/swift_build_support/products/llvm.py

@@ -351,6 +352,36 @@ def build(self, host_target):
            llvm_cmake_options.define('LLVM_INCLUDE_TESTS', 'NO')
            llvm_cmake_options.define('CLANG_INCLUDE_TESTS', 'NO')

+        build_root = os.path.dirname(self.build_dir)


What is the purpose of these cfg files? Are those needed only when targeting the static Linux SDK in build-script, or they need to be present in the generated toolchains?

Trying to understand if these need to be generated regardless for every platforms.

We will want them in the generated toolchains, but they are also required for the build.

Do we need then to invoke this logic in the install method as well? Leaning toward yes given your previous reply.

Do we need to gate such logic behind args.build_swift_static? Wondering if e.g. we want to always install it (so the generated toolchain can used to build the static stdlib), but generate in the build folder only if we are building the static library.

I'm slightly torn about this; I had intended to get rid of the .cfg files by patching the compiler drivers, particularly now I've changed the triple to a special one for the static SDK (<arch>-swift-linux-musl instead of <arch>-unknown-linux-musl). I think though that we probably do want to add them in the install phase for now; we can always remove them again later if it's a problem.

utils/build-script-impl

stdlib/cmake/modules/AddSwiftStdlib.cmake

Declare a new `LINUX_STATIC` SDK and configure it. Add options to set the build architectures for the `LINUX` and `LINUX_STATIC` SDKs, similar to what we have for Darwin, because we'll be cross-compiling. Also add an option to point the build system at the sources for the musl C library, which we're using for `LINUX_STATIC`. rdar://123503470

Change the `--linux[-static]-arch` option to `--linux[-static]-archs`, on the basis that it supports multiple values. Other tidying. rdar://123503470

Apparently I accidentally removed a `]` while resolving a conflict in `build_script_invocation.py.` rdar://123503470

The Python linter was complaining about import orders. rdar://123503470

Apparently I'd failed to add a couple of lines here. rdar://123503470

al45tair · 2024-05-02T13:58:08Z

@swift-ci Please smoke test

stdlib/cmake/modules/AddSwiftStdlib.cmake

utils/swift_build_support/swift_build_support/build_script_invocation.py

utils/build-script-impl

edymtt · 2024-05-02T21:15:24Z

utils/swift_build_support/swift_build_support/products/llvm.py

@@ -351,6 +352,36 @@ def build(self, host_target):
            llvm_cmake_options.define('LLVM_INCLUDE_TESTS', 'NO')
            llvm_cmake_options.define('CLANG_INCLUDE_TESTS', 'NO')

+        build_root = os.path.dirname(self.build_dir)


Do we need then to invoke this logic in the install method as well? Leaning toward yes given your previous reply.

Do we need to gate such logic behind args.build_swift_static? Wondering if e.g. we want to always install it (so the generated toolchain can used to build the static stdlib), but generate in the build folder only if we are building the static library.

Fix some indentation issues. Change `build-script-impl` to make `build-linux-static` a positive argument. Fix documentation for `--linux-archs` and `--linux-static-archs` (the options are comma separated for `build-script`, but semicolon separated for `build-script-impl`). Set the default for `linux-static-archs` to `x86_64, aarch64` so that we install the expected content in the toolchain. Add missing default for `test_linux_static`. Make sure to pass down `--skip-build-linux` and `--build-linux-static`. Factor out config file generation and call it from the install step in `llvm.py` as well as from the build step. rdar://123503470

al45tair · 2024-05-03T08:17:10Z

@swift-ci Please smoke test

Remove unused import of `importlib.resources`. Set `test_linux_static` to `False` if tests are disabled. rdar://123503470

al45tair · 2024-05-03T09:47:45Z

@swift-ci Please smoke test

MaxDesiatov · 2024-05-03T13:41:09Z

@swift-ci test webassembly

al45tair · 2024-05-03T14:49:22Z

Failures are:

Failed Tests (1):
  Swift(linux-x86_64) :: TypeRoundTrip/round-trip.swift

because

/usr/bin/ld.gold: error: cannot find -lswiftRemoteInspection
/usr/bin/ld.gold: internal error in read_header_prolog, at ../../gold/dwarf_reader.cc:1678
clang: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

(similar error on macOS).

Not sure this is to do with this PR; looking into it now.

al45tair · 2024-05-03T16:12:00Z

build-script -r; utils/run-test --build-dir ... test/TypeRoundTrip/round-trip.swift works locally on macOS, with an initially empty build directory.

al45tair · 2024-05-03T16:12:14Z

@swift-ci Please smoke test

al45tair · 2024-05-05T17:18:51Z

Hmmm. It's failing because it can't find swiftRemoteInspection, which suggests we're missing a dependency somewhere.

al45tair · 2024-05-05T19:51:57Z

@swift-ci Please smoke test

al45tair · 2024-05-05T20:42:51Z

@swift-ci Please smoke test

`TypeRoundTrip/round-trip.swift` tries to link with `libswiftRemoteInspection`, but that library isn't in the default link path, so we need to add a `-L` option to the command line in the test. Also, we should check that `SWIFT_BUILD_REMOTE_MIRROR` is enabled and disable the test if not, because otherwise `libswiftRemoteInspection` won't have been built. rdar://123503470

al45tair · 2024-05-06T07:47:56Z

@swift-ci Please smoke test

al45tair · 2024-05-06T07:48:59Z

The problem was that libswiftRemoteInspection.a wasn't in the link path, so we need to add an extra -L option to round-trip.swift. We should also only enable that test if we're building Remote Mirror because otherwise that library won't exist.

Looks like I'd unconditionally added -lfts to the Foundation build flags. rdar://123503470

al45tair · 2024-05-06T09:55:30Z

@swift-ci Please smoke test

al45tair · 2024-05-06T13:38:58Z

@swift-ci Please smoke test Linux platform

al45tair · 2024-05-06T13:39:35Z

(The previous failure was because the integration tests tried to treat the .cfg files as ELF binaries. That's wrong, so I fixed the integration tests to ignore them.)

[Build] Add the new fully-static Linux SDK.

This was added by swiftlang#71839 as part of a fix for the `TypeRoundTrip/round-trip.swift` test, but it isn't necessary and it's breaking nightly builds. rdar://128000843

al45tair requested a review from a team as a code owner February 23, 2024 16:10

al45tair requested a review from edymtt February 23, 2024 16:11

This was referenced Feb 23, 2024

[Platform] Update musl modulemap and improve musl platform support. #71840

Merged

[Runtime] Fix generation of .lnk files for static linking. #71843

Merged

al45tair force-pushed the eng/PR-123503470 branch from baa2d02 to ee69cbc Compare February 28, 2024 12:15

al45tair force-pushed the eng/PR-123503470 branch from ee69cbc to 6b8467a Compare February 28, 2024 12:18

al45tair force-pushed the eng/PR-123503470 branch from 6b8467a to 8ee2fe6 Compare March 14, 2024 15:34

al45tair requested a review from etcwilde March 14, 2024 15:35

edymtt reviewed Mar 28, 2024

View reviewed changes

al45tair force-pushed the eng/PR-123503470 branch from 8ee2fe6 to 1a7ea20 Compare May 2, 2024 11:51

al45tair requested a review from edymtt May 2, 2024 11:53

al45tair added 5 commits May 2, 2024 14:56

[Build] Updated following initial comments from Eric.

fdc8ffd

Change the `--linux[-static]-arch` option to `--linux[-static]-archs`, on the basis that it supports multiple values. Other tidying. rdar://123503470

[Build] Fix a conflict resolution mistake.

29e38af

Apparently I accidentally removed a `]` while resolving a conflict in `build_script_invocation.py.` rdar://123503470

[Build] Fix Python linting issues.

6859851

The Python linter was complaining about import orders. rdar://123503470

[Build] Add options to the Python build script tests.

bac73d0

Apparently I'd failed to add a couple of lines here. rdar://123503470

al45tair force-pushed the eng/PR-123503470 branch from dc876ee to bac73d0 Compare May 2, 2024 13:57

edymtt reviewed May 2, 2024

View reviewed changes

[Build] A couple more Python fixes.

62c80a1

Remove unused import of `importlib.resources`. Set `test_linux_static` to `False` if tests are disabled. rdar://123503470

edymtt approved these changes May 3, 2024

View reviewed changes

al45tair force-pushed the eng/PR-123503470 branch from dc9b67f to 7f0e5d8 Compare May 5, 2024 20:42

al45tair force-pushed the eng/PR-123503470 branch from 7f0e5d8 to 2297c3f Compare May 6, 2024 07:47

[Build] Only add -lfts for linux-static-*.

7431b96

Looks like I'd unconditionally added -lfts to the Foundation build flags. rdar://123503470

al45tair merged commit 9e2c13b into swiftlang:main May 6, 2024
3 checks passed

al45tair added a commit to al45tair/swift that referenced this pull request May 6, 2024

Merge pull request swiftlang#71839 from al45tair/eng/PR-123503470

758665a

[Build] Add the new fully-static Linux SDK.

al45tair mentioned this pull request May 6, 2024

[Build] Add the new fully-static Linux SDK. #73455

Merged

al45tair mentioned this pull request May 13, 2024

[Tests] Remove erroneous swiftRemoteInspection dependency. #73591

Merged

al45tair mentioned this pull request May 13, 2024

[6.0][Tests] Remove erroneous swiftRemoteInspection dependency. #73592

Merged

[Build] Add the new fully-static Linux SDK. #71839

[Build] Add the new fully-static Linux SDK. #71839

Uh oh!

Conversation

al45tair commented Feb 23, 2024

Uh oh!

al45tair commented Feb 23, 2024

Uh oh!

al45tair commented Feb 23, 2024

Uh oh!

al45tair commented Feb 27, 2024

Uh oh!

al45tair commented Feb 28, 2024

Uh oh!

al45tair commented Mar 14, 2024

Uh oh!

edymtt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edymtt Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

al45tair May 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

edymtt May 2, 2024

Choose a reason for hiding this comment

Uh oh!

al45tair May 3, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

al45tair commented May 2, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

edymtt May 2, 2024

Choose a reason for hiding this comment

Uh oh!

al45tair commented May 3, 2024

Uh oh!

al45tair commented May 3, 2024

Uh oh!

MaxDesiatov commented May 3, 2024

Uh oh!

al45tair commented May 3, 2024

Uh oh!

al45tair commented May 3, 2024

Uh oh!

al45tair commented May 3, 2024

Uh oh!

al45tair commented May 5, 2024

Uh oh!

al45tair commented May 5, 2024

Uh oh!

al45tair commented May 5, 2024

Uh oh!

al45tair commented May 6, 2024

Uh oh!

al45tair commented May 6, 2024

Uh oh!

al45tair commented May 6, 2024

Uh oh!

al45tair commented May 6, 2024

Uh oh!

al45tair commented May 6, 2024

Uh oh!

Uh oh!

Uh oh!

al45tair May 2, 2024 •

edited

Loading