Repository navigation
Conversation
gabotechs
force-pushed
the
gabotechs/datasource-proto-registry
branch
from
October 21, 2026 08:30
86c375a to
8c43f67
Compare
Contributor
Author
|
cc @adriangb I want to tackle this one myself in order to get some hands-on exp with the approach here. Still iterating on this, so not looking for a review. |
gabotechs
force-pushed
the
gabotechs/datasource-proto-registry
branch
from
October 21, 2026 09:19
8c43f67 to
bceb61d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Follow-up to #24631 and #24628. No separate issue.
Rationale for this change
Custom
DataSourceimplementations can encode themselves throughtry_to_proto, but still need aPhysicalExtensionCodecto decode. Applications combining sources from multiple libraries therefore have to compose their codecs.Extend the session-scoped decoder registry to sources so each source can own its decoder and register it by name, following the extension execution-plan and expression APIs.
What changes are included in this PR?
ExtensionDataSourceFromProto,register_data_source, andextension_data_source_node. Constructors receive a typedPhysicalExtensionDataSourceNodeand returnArc<dyn DataSource>.DataSourceExecas a shared extension-plan decoder. Its payload contains the source name and bytes; the wrapper looks up the source decoder and rebuildsDataSourceExecthrough the existing plan registry dispatch. No new top-level plan variant is needed. Source names are independent of plan and expression names.What is the testing strategy for this PR?
Six registry unit tests and fourteen integration tests cover registration, execution after decoding, reading-session state, malformed wrapper and source payloads, codec dispatch boundaries, legacy codec compatibility, shared names across extension kinds, and binary/JSON round trips.
Validation: 541 affected-crate tests and 14 doctests passed; formatting, workspace-wide Clippy (
cargo clippy --all-targets --all-features -- -D warnings), rustdoc with warnings denied, protobuf regeneration, and documentation formatting passed.Are there any user-facing changes?
New source-registry APIs behind the existing
protofeature. Built-in source serialization is unchanged. Readers must register source decoders before writers emit the new wrapper payload. Documented in the 56.0.0 upgrade guide.AI-assisted implementation with Codex.