Support custom host SDK (aka. `SDKROOT`) #266

stevapple · 2022-04-21T15:01:26Z

This PR differentiates host SDK and target SDK, and adds support for setting custom host SDK through "Advanced" configuration.

stevapple · 2022-04-24T06:20:58Z

@adam-fowler Are you able to reproduce the CI error? I didn't think this has touched the behavior if no custom SDK is set, and cannot reproduce in my environments (Windows/5.6.1 & macOS/5.6) either.

adam-fowler · 2022-04-25T11:21:44Z

I can't see why this should fail during CI. Running tests locally works fine. Can you merge in latest changes as it is very hard to see which changes are specific to this PR.

stevapple · 2022-04-27T14:35:07Z

@adam-fowler Do you have time to review this now?

adam-fowler · 2022-04-27T17:32:50Z

So as I understand it you have added a new setting hostSDK, which is used as the SDKROOT environment variable when running many of the swift commands. At the same time a couple of the swift package commands (describe, unedit) have had the runtime and compiler environment removed from them, as it is unnecessary. Any reason you didn't do this for edit.

With this PR plus the two prior PRs we now have settings
SDK: Adds --sdk parameter to swift calls
runtimePath: Added to Path env var when compiling and running
hostSDK: Set SDKROOT environment variable when compiling

adam-fowler · 2022-04-27T17:42:34Z

What is the point of setting SDKROOT env variable and also including --sdk command line parameter?

stevapple · 2022-04-27T17:43:13Z

At the same time a couple of the swift package commands (describe, unedit) have had the runtime and compiler environment removed from them, as it is unnecessary. Any reason you didn't do this for edit.

Could you point out where I may be missing? It feels like any swift-package command won’t be affected by custom SDK flags.

I intentionally want to make:

SDK (and the upcoming "Target") applied to Build Tasks and Run Script Task (and possible debugging on Windows)
host SDK applied to all tasks except for debugging
runtime path applied everywhere

stevapple · 2022-04-27T17:46:40Z

What is the point of setting SDKROOT env variable and also including --sdk command line parameter?

SDKROOT is used to override host SDK, and --sdk is specified for the target. SwiftPM needs to run the manifest on the host, that’s where host SDK should be explicitly used. (It makes sense, but the use case is rare).

BTW SDKROOT only works on macOS and Windows, that’s something that needs to be documented.

adam-fowler · 2022-04-27T17:52:42Z

`SwiftPM needs to run the manifest on the host, that’s where host SDK should be explicitly used.

Which commands are we talking about here?
Also any reason we don't use --sdk to point to the host SDK, in this situation?

stevapple · 2022-04-27T18:05:54Z

Considering compiling a Swift package on macOS without Xcode (wish this could happen, but just imagine) for iOS.

We need to explicitly set the host SDK, or the manifest won’t compile. And we also need to set the target SDK, or the target cannot be supported. So finally it becomes:

SDKROOT=/path/to/macos/sdk swift build --triple arm64-apple-ios --sdk /path/to/ios/sdk

Maybe compiling for Android on Windows is easier to imagine:

SDKROOT=/path/to/windows/sdk swift build --triple aarch64-unknown-linux-android --sdk /path/to/android/sdk

stevapple · 2022-04-27T18:09:41Z

Also any reason we don't use --sdk to point to the host SDK, in this situation?

Host SDK is never decided by CLI options. There’re only two sources: SDKROOT value > xcrun output

adam-fowler · 2022-04-28T07:27:06Z

So what are the reasons you might want to make the "host" sdk configurable? If it is just to compile Package.swift I can't see what that would be.

stevapple · 2022-04-28T07:32:21Z

If you don’t need to cross-compile, you’re recommended to set host SDK instead of custom SDK. The latter should be used explicitly for cross-compilation.

Also, setting SDKROOT is required for using an alternative Windows toolchain now. There is no second way.

adam-fowler · 2022-04-28T07:55:09Z

If you don’t need to cross-compile, you’re recommended to set host SDK instead of custom SDK. The latter should be used explicitly for cross-compilation.

The "host" SDK is explicitly set already (when you install Xcode, and as I understand it SDKROOT is set when installing Swift on Windows, I may be wrong there).

Also, setting SDKROOT is required for using an alternative Windows toolchain now. There is no second way.

So does that mean you only use --sdk when doing cross compilation.

stevapple · 2022-04-28T08:00:11Z

The "host" SDK is explicitly set already (when you install Xcode, and as I understand it SDKROOT is set when installing Swift on Windows, I may be wrong there).

You can have multiple macOS SDKs in parallel. macOS SDKs are standalone and you can extract older or newer SDKs from other versions of Xcode if you want.

So does that mean you only use --sdk when doing cross compilation.

At least when compiling to another triple (eg. a custom minimal OS version). I don’t think there should be a second usage.

Maybe we should hide it for now, and wait until custom target support?

adam-fowler · 2022-04-28T08:37:46Z

So on macOS what do we gain from setting SDKROOT when we can already set the path to the swift compiler? The only thing I can think of is xctest.

stevapple · 2022-04-28T08:42:05Z

So on macOS what do we gain from setting SDKROOT when we can already set the path to the swift compiler?

On macOS toolchain and SDK is not tied so closely. Using alternative toolchain doesn’t affect the SDK (thanks to ABI stability). You always need to set SDKROOT if you want to use an SDK outside your current Xcode installation.

stevapple · 2022-04-28T08:43:24Z

The only thing I can think of is xctest.

XCTest on macOS is part of the toolchain. Because it is found using xcode-select, we have no option to override it yet.

adam-fowler · 2022-04-28T15:39:09Z

You always need to set SDKROOT if you want to use an SDK outside your current Xcode installation.

But can't you use --sdk in that case?

stevapple · 2022-04-28T15:47:31Z

--sdk <sdk> should be used for cross-compilation;
Manifest compiler won’t take this.

stevapple · 2022-05-02T09:30:49Z

@adam-fowler Any review? Better get this in 0.5.0

stevapple · 2022-05-02T09:47:48Z

BTW, plugins in SwiftPM 5.6+ will also stick to host SDK.

adam-fowler

In general this looks fine. I'm not sure about the terminology host SDK. I don't remember seeing it anywhere else. Given people who are changing this will know it as SDKROOT call it that.

I'm not including this in 0.5.0 though. As I don't want to rush it in.

adam-fowler · 2022-05-02T11:01:14Z

src/toolchain/toolchain.ts

@@ -198,17 +198,22 @@ export class SwiftToolchain {
                return path.join(stdout.trimEnd(), "usr", "bin");
            }
            case "win32": {
+                // look up runtime library directory for XCTest alternatively
+                const fallbackPath =


Can we remove this change it is unrelated.

This is just moved here from L207-209

stevapple · 2022-05-02T11:44:15Z

I'm not including this in 0.5.0 though. As I don't want to rush it in.

If you're not including this for 0.5.0, I'll split this pitch into two parts (fixing up target SDK & adding host SDK) and left the former merged first. The current behavior of custom target SDK has some mistakes that need to be addressed.

stevapple · 2022-05-02T11:50:57Z

I'm not sure about the terminology host SDK.

This terminology is derived from "host toolchain" in SwiftPM source code. I don't know where SDKROOT is from but it seems to be a more ancient and platform-specific concept and I would tend to keep the SwiftPM naming.

stevapple · 2022-05-02T12:27:03Z

Given that SwiftPM uses the terminology "destination" to describe the target, I'll update targetSDK into destinationSDK instead.

adam-fowler · 2022-05-02T12:30:50Z

I'm not sure about the terminology host SDK.

This terminology is derived from "host toolchain" in SwiftPM source code. I don't know where SDKROOT is from but it seems to be a more ancient and platform-specific concept and I would tend to keep the SwiftPM naming.

host toolchain is a different thing from host sdk

stevapple · 2022-05-02T12:41:53Z

host toolchain is a different thing from host sdk

There is a clear relationship between destination toolchain — destination SDK — destination target triple. Similarly the logic can be applied to host.

adam-fowler · 2022-05-15T08:36:12Z

Now that #305 is merged, can we close this

stevapple added 3 commits April 21, 2022 23:00

Support custom host SDK (aka. SDKROOT)

dbee7fe

Merge branch 'main' into host-sdk

c00ef36

Add explanation for swift-package

e5da6fe

stevapple added 2 commits April 25, 2022 21:55

Merge branch 'main' into host-sdk

e0e6fac

Try fix CI failure

b2684fb

This was referenced Apr 26, 2022

debugger: pass along SDKROOT for the launch of the debugger #275

Merged

Extension Settings doc.md #240

Merged

Merge branch 'main' into host-sdk

d84f9ae

adam-fowler reviewed May 2, 2022

View reviewed changes

Update naming

fa7ecbf

stevapple mentioned this pull request May 13, 2022

Add setting for environment variables to send to Swift #305

Merged

stevapple closed this May 16, 2022

stevapple deleted the host-sdk branch August 1, 2022 20:23

Support custom host SDK (aka. SDKROOT) #266

Support custom host SDK (aka. SDKROOT) #266

Uh oh!

Conversation

stevapple commented Apr 21, 2022

Uh oh!

stevapple commented Apr 24, 2022

Uh oh!

adam-fowler commented Apr 25, 2022

Uh oh!

stevapple commented Apr 27, 2022

Uh oh!

adam-fowler commented Apr 27, 2022

Uh oh!

adam-fowler commented Apr 27, 2022

Uh oh!

stevapple commented Apr 27, 2022

Uh oh!

stevapple commented Apr 27, 2022

Uh oh!

adam-fowler commented Apr 27, 2022

Uh oh!

stevapple commented Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stevapple commented Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adam-fowler commented Apr 28, 2022

Uh oh!

stevapple commented Apr 28, 2022

Uh oh!

adam-fowler commented Apr 28, 2022

Uh oh!

stevapple commented Apr 28, 2022

Uh oh!

adam-fowler commented Apr 28, 2022

Uh oh!

stevapple commented Apr 28, 2022

Uh oh!

stevapple commented Apr 28, 2022

Uh oh!

adam-fowler commented Apr 28, 2022

Uh oh!

stevapple commented Apr 28, 2022

Uh oh!

stevapple commented May 2, 2022

Uh oh!

stevapple commented May 2, 2022

Uh oh!

adam-fowler left a comment

Choose a reason for hiding this comment

Uh oh!

adam-fowler May 2, 2022

Choose a reason for hiding this comment

Uh oh!

stevapple May 2, 2022

Choose a reason for hiding this comment

Uh oh!

stevapple commented May 2, 2022

Uh oh!

stevapple commented May 2, 2022

Uh oh!

stevapple commented May 2, 2022

Uh oh!

adam-fowler commented May 2, 2022

Uh oh!

stevapple commented May 2, 2022

Uh oh!

adam-fowler commented May 15, 2022

Uh oh!

Uh oh!

Support custom host SDK (aka. `SDKROOT`) #266

Support custom host SDK (aka. `SDKROOT`) #266

stevapple commented Apr 27, 2022 •

edited

Loading

stevapple commented Apr 27, 2022 •

edited

Loading