Skip to content

Implement swift_build_sdk_interfaces.py using libSwiftDriver #579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 9, 2021

Conversation

nkcsgexi
Copy link
Contributor

@nkcsgexi nkcsgexi commented Apr 2, 2021

This PR is an implementation of swift_build_sdk_interfaces.py using
libSwiftDriver. The tool is used to traverse an entire SDK to find
Swift interface files and to generate prebuilt module cache for each one
of them. Comparing to the python implementation, this implementation generates
the prebuilt modules in the dependency order so that:

(1) It's more performant than the python counterpart. Several preliminary
experiments showed an average of 40% less CPU time spent on generating modules in an SDK.

(2) It fails fast. If an interface is broken in the SDK, this new implementation
won't schedule jobs for building other modules depending on this interface. It should help us avoid
issues like rdar://76201633

@nkcsgexi nkcsgexi requested a review from artemcm April 2, 2021 23:34
@nkcsgexi
Copy link
Contributor Author

nkcsgexi commented Apr 2, 2021

@swift-ci please test

}

func addCommonFlags(commandLine: inout [Job.ArgTemplate]) {
commandLine.appendFlag(.buildModuleFromParseableInterface)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny nitpick: the current version of this is compile-module-from-interface

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch 3 times, most recently from af4afcb to 73d943a Compare April 5, 2021 04:57
@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch 2 times, most recently from 816d5b9 to 8ec2adb Compare April 5, 2021 18:52
@@ -21,6 +21,9 @@ let package = Package(
.executable(
name: "swift-help",
targets: ["swift-help"]),
.executable(
name: "swift-build-sdk-interfaces",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think some changes in build-script-helper.py are required to make sure this gets installed when built with build-script.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, will make those build-script/cmake changes later.

@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch 4 times, most recently from 48326b9 to e0caa90 Compare April 6, 2021 23:47
@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch 4 times, most recently from a9ab8af to 007cb7a Compare April 9, 2021 16:39
@nkcsgexi
Copy link
Contributor Author

nkcsgexi commented Apr 9, 2021

@swift-ci please test

@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch from 007cb7a to 20e570d Compare April 9, 2021 16:58
@nkcsgexi
Copy link
Contributor Author

nkcsgexi commented Apr 9, 2021

@swift-ci please test

@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch 2 times, most recently from 881ebf1 to 2e34250 Compare April 9, 2021 18:03
This PR is an implementation of swift_build_sdk_interfaces.py using
libSwiftDriver. The tool is used to traverse an entire SDK to find
Swift interface files and to generate prebuilt module cache for each one
of them. Comparing to the python implementation, this implementation generates
the prebuilt modules in the dependency order so that:

(1) It's more performant than the python counterpart. Several preliminary
experiments showed an average of 40% less CPU time spent on generating modules in an SDK.

(2) It fails fast. If an interface is broken in the SDK, this new implementation
won't schedule jobs for building other modules depending on this interface. It should help us avoid
issues like rdar://76201633
@nkcsgexi nkcsgexi force-pushed the prebuilt-module-gen branch from 2e34250 to 037aede Compare April 9, 2021 18:06
@nkcsgexi nkcsgexi changed the title [WIP] Teach Driver to generate swift-frontend invocations to build prebuilt module cache Implement swift_build_sdk_interfaces.py using libSwiftDriver Apr 9, 2021
@nkcsgexi
Copy link
Contributor Author

nkcsgexi commented Apr 9, 2021

@swift-ci please test

@nkcsgexi nkcsgexi merged commit 903ae6a into swiftlang:main Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants