Add HashEncoder and QueryEngineTests
#7435
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This ports most of the current `GeneratorEngine` implementation from the Swift SDK Generator to the SwiftPM code base: https://github.com/apple/swift-sdk-generator/blob/main/Sources/GeneratorEngine/Engine.swift. Since Swift NIO is not supported on Windows, references to `AsyncHTTPClient` have been removed. Additionally, we can't use macros in the SwiftPM code base either, thus the `Query` protocol has to conform to `Encodable` instead. We don't have a consistent hashing implementation for `Encodable` yet, and a temporary stub for it is marked with `fatalError` for now.
…xd/query-engine-hashing
Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
…xd/query-engine-scaffolding # Conflicts: # Package.swift
…t-package-manager into maxd/query-engine-hashing # Conflicts: # Package.swift
MaxDesiatov
added
test suite
MaxDesiatov
commented
Apr 2, 2024
MaxDesiatov
commented
Apr 2, 2024
MaxDesiatov
commented
Apr 2, 2024
MaxDesiatov
commented
Apr 2, 2024
MaxDesiatov
commented
Apr 10, 2024
| public let observabilityScope: ObservabilityScope | ||
| package let fileSystem: any AsyncFileSystem | ||
| package let httpClient = HTTPClient() | ||
| package let observabilityScope: ObservabilityScope | ||
| private let resultsCache: SQLiteBackedCache<FileCacheRecord> | ||
| private var isShutDown = false |
There was a problem hiding this comment.
Need to handle circular query dependencies here, probably with a dictionary from Query to query result continuation.
There was a problem hiding this comment.
Also, don't recompute Query hash for deep FS trees.
MaxDesiatov
commented
Apr 10, 2024
MaxDesiatov
commented
Apr 10, 2024
xedin
approved these changes
Apr 10, 2024
…xd/query-engine-hashing # Conflicts: # Package.swift # Sources/Basics/SQLiteBackedCache.swift # Sources/QueryEngine/FileSystem/AsyncFileSystem.swift # Sources/QueryEngine/FileSystem/OpenReadableFile.swift # Sources/QueryEngine/FileSystem/OpenWritableFile.swift # Sources/QueryEngine/FileSystem/VirtualFileSystem.swift # Sources/QueryEngine/Query.swift # Sources/QueryEngine/QueryEngine.swift
|
@swift-ci test |
|
@swift-ci test |
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
Depends on swiftlang#7347. Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
furby-tm
pushed a commit
to wabiverse/swift-package-manager
that referenced
this pull request
May 15, 2024
Depends on swiftlang#7347. Previously, `QueryEngine` couldn't cache anything due to a missing hashing implementation that could be consistent across SwiftPM relaunches. Here `HashEncoder` is implemented on top `Codable`. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping. New `QueryEngine` now works for trivial computations, as verified by newly added `QueryEngineTests`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously,
QueryEnginecouldn't cache anything due to a missing persistable hashing implementation that could be consistent across SwiftPM relaunches. HereHashEncoderis implemented on topCodable. While it probably has certain performance overhead when compared to an implementation with macros, it should be good enough for prototyping.New
QueryEnginenow works for trivial computations, as verified by newly addedQueryEngineTests.