API available to SwiftPM Commands is no longer available to other clients of the SwiftPM

[krzyzanowskim]

Commands is a primary SwiftPM product client. Unfortunately, it uses API that recently became scoped as a package; therefore, it is no longer accessible to third-party clients of the SwiftPM product. It makes it impossible to use the same functionality by other clients (without reimplementing it, I guess). That wasn't an issue until recent changes #7387

For example, it's now impossible to build a tool similar to the swift-api-digester command using SwiftPM product as a dependency, as it lacks the API to build the package.

I see it as an oversight with a chance to revert and make it public again. Ideally, whatever is used by, e.g., Commands should be a public API available to other clients of the SwiftPM product (library).

[MaxDesiatov]

That is correct and is intended. Only stable and versioned API SwiftPM provides is in PackageDescription and PackagePlugin modules. Other modules that unfortunately became exposed with public declarations before package was available in the language are unstable. As SwiftPM doesn't provide SemVer tags, those should not be relied on. They can change or be removed at any moment without warning.

If you intend to build tools on top of SwiftPM, package and build tool plugins or shelling out to CLI commands are the only supported options right now.

SourceKit-LSP is always built with the corresponding SwiftPM source tree with the same snapshot tag, and also doesn't adhere to SemVer. Outside of that specific case SwiftPM should not be used as a library.

[krzyzanowskim]

That is correct and is intended.

that's an unfortunate and unexpected change of the course.

What is the vision of the Build etc. modules in the SwiftPM product and more on the general level SwiftPM product library at this point? One thing is unstable API, and the other thing is no API at all. CLI interface is way too limited for many use cases that API provides for it's own needs.

Doesn't Xcode and Swift Playgrounds use it without shelling out to CLI? (wouldn't that be a bit of an unfair advantage for in-house tools)

[MaxDesiatov]

Doesn't Xcode and Swift Playgrounds use it without shelling out to CLI?

These are closed-source products that are out of scope of the open-source SwiftPM repository. I can't provide any information on that.

[kateinoigakukun]

Saying about API digester related types in SwiftPM, those types themself are pretty trivial and the main functionalities are provided by swift-api-digester command, which is not a part of SwiftPM but a part of Swift toolchain. So to be fair, you can still implement tools similar to diagnose-api-breaking-changes without the type without re-implementing everything.

[krzyzanowskim]

@kateinoigakukun

So to be fair, you can still implement tools similar to diagnose-api-breaking-changes without the type without re-implementing everything.

as long as that tool does not need to build anything like swift-api-digester does. however that was just a quick example of use the API by tools rly.

[kateinoigakukun]

@krzyzanowskim

as long as that tool does not need to build anything like swift-api-digester does.

Hmm, you are right. We can still build packages through CLI but cannot easily collect enough build information required to construct swift-api-digester arguments (e.g. -I path/to/swiftmodules -I path/to/modulemap-dir), so I understand your requests to expose such interfaces to interact with internal information of the build system.

I think the main motivation for hiding those APIs is to increase the opportunities to improve the internal architecture, so exposing whole APIs used by Commands conflicts with the goal. But exposing a part of APIs for a specific use case might be worth considering as far as it does not significantly increase maintenance cost.

So I'd like to understand your specific use cases to have the best interfaces for both SwiftPM maintainers and you.

[xedin]

@krzyzanowskim I've opened a PR to undo the changes - #7568

[bnbarham]

To expand on this a little - we would still very much like to reduce the visibility of various APIs/modules in SwiftPM to make it easier to make changes/refactor/maintain in general. But given how long these have all been public for, we'll revert these changes for now and instead make posts on the forums asking for feedback for the APIs we'd like to hide. Depending on the API it may be that there's another way to retrieve similar information, or that we need to provide a separate (ideally stable) API for it.

Let us know if there's a PR we missed in the revert here 🙇‍♂️.

[bnbarham]

Merged in #7568