Skip to content

PackageLoading: convert most of ManifestLoader to async/await #6624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

MaxDesiatov
Copy link
Contributor

@MaxDesiatov MaxDesiatov commented May 26, 2023

This keeps the public function signatures intact and only converts private or internal functions to async/await.

@MaxDesiatov MaxDesiatov added the WIP Work in progress label May 26, 2023
@MaxDesiatov MaxDesiatov changed the title PackageLoading: convert ManifestLoader to async/await PackageLoading: convert ManifestLoader to async/await May 26, 2023
@MaxDesiatov MaxDesiatov marked this pull request as ready for review May 26, 2023 17:16
@MaxDesiatov
Copy link
Contributor Author

@swift-ci smoke test

@MaxDesiatov MaxDesiatov changed the title PackageLoading: convert ManifestLoader to async/await PackageLoading: convert most of ManifestLoader to async/await May 26, 2023

extension ManifestLoader {
/// Represents behavior that can be deferred until a more appropriate time.
struct DelayableAction<T> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed as unused, we can now use plain defer instead

@MaxDesiatov
Copy link
Contributor Author

@swift-ci smoke test

@MaxDesiatov MaxDesiatov self-assigned this May 26, 2023
@MaxDesiatov MaxDesiatov added concurrency and removed WIP Work in progress labels May 26, 2023
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test windows

1 similar comment
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test windows

@MaxDesiatov
Copy link
Contributor Author

@swift-ci smoke test

@tomerd tomerd added the next waiting for next merge window label Jun 12, 2023
MaxDesiatov added a commit that referenced this pull request Oct 12, 2023
This allows `Workspace.loadPluginImports` to be implemented in a non-blocking way without any uses of `temp_await`.

Also a dependency of #6624.
MaxDesiatov added a commit that referenced this pull request Oct 12, 2023
This allows `Workspace.loadPluginImports` to be implemented in a
non-blocking way without any uses of `temp_await`.

Also a dependency of
#6624.
@MaxDesiatov MaxDesiatov force-pushed the maxd/async-manifest-loader branch from b6d8578 to 818ebd1 Compare October 13, 2023 11:32
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov MaxDesiatov removed the request for review from abertelrud October 13, 2023 12:10
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test windows

@MaxDesiatov
Copy link
Contributor Author

@neonichu CI is green on this now, ready for review

…xd/async-manifest-loader

# Conflicts:
#	Sources/PackageLoading/ManifestLoader.swift
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test windows

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

Copy link
Contributor

@AndrewHoos AndrewHoos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor nits... but otherwise looks good

}

var evaluationResult = EvaluationResult()
return try await self.tokenBucket.withToken {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have the background, but it is unclear why this diff is introducing the withToken indentation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

withToken introduces indentation much earlier, since with stricter concurrency checks withToken has to initialize state like evaluationResult much earlier in its scope than was previously possible with unchecked Dispatch closures. Previously this state was captured with no sendability checks whatsoever. Much broader withToken scope allows us to initialize all state in the same closure, fixing sendability warnings and errors.

"-L", runtimePath.pathString,
"-lPackageDescription",
]
#if !os(Windows)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The OS if statements are now indented in a non-standard way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what "non-standard" means here. AFAIK Swift as a language has no standard formatting guidelines. Formatting used in this PR is consistent with the formatting convention we've codified for SwiftPM in our SwiftFormat config: https://github.com/apple/swift-package-manager/blob/main/.swiftformat. Some parts of SwiftPM written before the config was introduced may not follow it yet, but for new and updated code we try to stick to it, converging on that consistent formatting convention over time.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously the #if was flush left with the file. Now it is one indentation in, but the native level of indentation here is three levels deep. Maybe it is just how GitHub is rendering, but 1 indentation seemed odd because nothing else was at that level

@@ -305,28 +300,91 @@ public final class ManifestLoader: ManifestLoaderProtocol {
dependencyMapper: DependencyMapper,
fileSystem: FileSystem,
observabilityScope: ObservabilityScope,
delegateQueue: DispatchQueue,
callbackQueue: DispatchQueue
delegateQueue: DispatchQueue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the delegateQueue here anymore? there is no way we are FIFO when in swift concurrency so I am I am not sure that forcing a hop to the delegate queue is meaningful, but I haven't thought this all the way through

Copy link
Contributor Author

@MaxDesiatov MaxDesiatov Dec 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're still calling delegate methods from async context that may be executed on some thread from the concurrency pool that we can't know upfront. Until we have strict concurrency checks enabled or have an opportunity to rewrite delegates with AsyncSequence producing events for notifying the caller, I'd like to have delegateQueue passed around explicitly.

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test macos

…xd/async-manifest-loader

# Conflicts:
#	Tests/PackageLoadingTests/PD_4_2_LoadingTests.swift
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

…xd/async-manifest-loader

# Conflicts:
#	Sources/PackageLoading/ManifestLoader.swift
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test windows

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov MaxDesiatov marked this pull request as draft January 15, 2024 10:47
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

MaxDesiatov added a commit that referenced this pull request Feb 23, 2024
This is a stripped down version of
#6624 that modifies
only the test suite and doesn't make any substantial to
`ManifestLoader.swift` other than providing an `async` overload in
`extension ManifestLoaderProtocol` that uses
`withCheckedThrowingContinuation` to bridge into the existing
implementation.
…xd/async-manifest-loader

# Conflicts:
#	Sources/SPMTestSupport/MockManifestLoader.swift
#	Sources/SPMTestSupport/XCTAssertHelpers.swift
#	Tests/PackageLoadingTests/PD_4_2_LoadingTests.swift
@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

@MaxDesiatov
Copy link
Contributor Author

@swift-ci test

furby-tm pushed a commit to wabiverse/swift-package-manager that referenced this pull request May 15, 2024
This is a stripped down version of
swiftlang#6624 that modifies
only the test suite and doesn't make any substantial to
`ManifestLoader.swift` other than providing an `async` overload in
`extension ManifestLoaderProtocol` that uses
`withCheckedThrowingContinuation` to bridge into the existing
implementation.
furby-tm pushed a commit to wabiverse/swift-package-manager that referenced this pull request May 15, 2024
This is a stripped down version of
swiftlang#6624 that modifies
only the test suite and doesn't make any substantial to
`ManifestLoader.swift` other than providing an `async` overload in
`extension ManifestLoaderProtocol` that uses
`withCheckedThrowingContinuation` to bridge into the existing
implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants