Target platform selection isn't working with the swiftbuild build system

[cmcgee1024]

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

• Confirmed reproduction steps with SwiftPM CLI. The description text must include reproduction steps with either of command-line SwiftPM commands, swift build, swift test, swift package etc.

Description

When using the native SwiftPM build system you can build without any target/product selector and it will build the top-level products, skipping any irrelevant targets. Also, you can select an executable target and build that. These two things are not working in the same way with the swiftbuild build system. Instead, it should behave in the same way as the native build system with target selection.

This was detected while building Swiftly, which has a Linux target that can only be built on that OS. Dependencies on that target are guarded with .when() selecting only for Linux. Depending on the way that the build is invoked with the swiftbuild build system (no target/product specified) it fails due to compile errors on missing headers. Also, when attempting to target the executable target, it indicates that it can’t find it.

Expected behavior

It is expected that since this is being built on macOS that the Linux-specific dependencies guarded by .when() will not be built and no compile errors will be found.

Actual behavior

Instead, there are compile errors regarding libarchive, a Linux-specific dependency.

Steps to reproduce

1. Using macOS
2. Clone and check out the version of swift-package-manager from github with swiftbuild integration code
3. Clone swiftly from GitHub here: https://github.com/swiftlang/swiftly
4. Run the following command to build swiftly using SwiftPM and the swift build system: swift run swift-build —build-system=swiftbuild —package-path=<path_to_swiftly>

Swift Package Manager version/commit hash

No response

Swift & OS version (output of swift --version ; uname -a)

No response

[cmcgee1024]

This problem can be seen in this PR: #8276

[cmcgee1024]

Swiftly has since adopted plugins, which make this more difficult to reproduce. Luckily, there is a test Fixture package that can be used instead, and it's simpler:

swift run swift-build --build-system=swiftbuild --package-path=Fixtures/Miscellaneous/RootPackageWithConditionals

It needs to be run from macOS though.

[bkhouri]

We can also look at a test added as part of https://github.com/swiftlang/swift-package-manager/pull/8276/files#diff-b1845cad8f64e37500b06818ff71629726c14cf5a8d34add6ef3a9d38236cfe5R590.

[cmcgee1024]

The reason that the native build system doesn't encounter this problem is because of a bit of target selection logic when building without an explicit set of targets. There is a phony command that build everything except the targets that are not "InRootPackages".

    /// Adds a top-level phony command that builds the entire module.
    private func addTargetCmd(_ target: SwiftModuleBuildDescription, cmdOutputs: [Node]) {
        // Create a phony node to represent the entire module.
        let targetName = target.getLLBuildTargetName()
        let targetOutput: Node = .virtual(targetName)

        self.manifest.addNode(targetOutput, toTarget: targetName)
        self.manifest.addPhonyCmd(
            name: targetOutput.name,
            inputs: cmdOutputs,
            outputs: [targetOutput]
        )
        if self.plan.graph.isInRootPackages(target.target, satisfying: target.buildParameters.buildEnvironment) {
            if !target.isTestTarget {
                self.addNode(targetOutput, toTarget: .main)
            }
            self.addNode(targetOutput, toTarget: .test)
        }
    }

Meanwhile, the isInRootPackages() is provided with a satisfying "buildEnvironment", so that it can scan the dependencies of the target in the graph, and eliminate it if there are only dependencies that are filtered out by the environment:

    /// Returns true if a given module is present in root packages and is not excluded for the given build environment.
    public func isInRootPackages(_ module: ResolvedModule, satisfying buildEnvironment: BuildEnvironment) -> Bool {
        // FIXME: This can be easily cached.
        return rootPackages.reduce(
            into: IdentifiableSet<ResolvedModule>()
        ) { (accumulator: inout IdentifiableSet<ResolvedModule>, package: ResolvedPackage) in
            let allDependencies = package.modules.flatMap { $0.dependencies }
            let unsatisfiedDependencies = allDependencies.filter { !$0.satisfies(buildEnvironment) }
            let unsatisfiedDependencyModules = unsatisfiedDependencies.compactMap { (
                dep: ResolvedModule.Dependency
            ) -> ResolvedModule? in
                switch dep {
                case .module(let moduleDependency, _):
                    return moduleDependency
                default:
                    return nil
                }
            }

            accumulator.formUnion(IdentifiableSet(package.modules).subtracting(unsatisfiedDependencyModules))
        }.contains(id: module.id)
    }

For this to work with the swiftbuild build system, the equivalent "AllExcludingTests" and "AllIncludingTests" target would need to exclude similar targets that have unsatisfied dependencies based on the build environment. The change for that would be in the PIF builder.

[jakepetroules]

It's possible we're not properly respecting the "when" conditions by translating those to platform filters in the PIF.

[cmcgee1024]

@jakepetroules I see the platform filters in the PIF. It's just that when the PIF builder assembles these special "All" targets for when the user doesn't specify and explicit target, it includes the targets that only have platform-specific incoming edges, so Swift Build builds them because that's what it's being asked to do.

[cmcgee1024]

This is fixed in #8568