-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gRPC query extension #15982
gRPC query extension #15982
Conversation
Report stats & close statement
Simple server + query test works
The gRPC shaded jar module causes issues when loaded into an IDE. This PR hides the module from Maven (and hence IDEs) by building it implicitly in the grpc-query module. It is a hack, but it works.
grpc protobufwriter
add empty response and GrpcResponseHandler tests
Allow ITs outside of the 'cases' directory Refactoring
Build the grpc-query extension archive Build a jar for the test Protobuf message Fixes to the extension path mechanism
* Refactor template into cluster directory * Add verify, setup scripts * Forbid snapshots from upstream repos * Basic grpc-query IT test
Retains support for anonymous (test) usage
<parent> | ||
<groupId>org.apache.druid</groupId> | ||
<artifactId>druid</artifactId> | ||
<version>30.0.0-SNAPSHOT</version> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
<version>30.0.0-SNAPSHOT</version> | |
<version>31.0.0-SNAPSHOT</version> |
The master branch has been updated to 31.0.0-SNAPSHOT
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's fix it and then merge it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.
This pull request has been marked as stale due to 60 days of inactivity. |
} | ||
|
||
message QueryRequest { | ||
QueryType queryType = 1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@rishabh Could you make that QueryType is the last field in this proto to preserve the original order to avoid breaking changes and also give QueryType a default value of SQL.
fix proto to be backward compatible
…nto grpc-query-extension
@FrankChen021 could you please take a look at this PR, it has been open for a while |
@findingrish - can you rebase it on master so we can merge it? |
integration-tests-ex/diff.patch
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is this file for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cleaned up.
@findingrish - the test failures look genuine |
Revives #14024 and additionally supports,
This PR doesn't have the shaded module for packaging gRPC and Guava libraries since
grpc-query
module uses the same Guava version as that of Druid.The response is gRPC-specific. It provides the result schema along with the results as a binary "blob". Results can be in CSV, JSON array lines or as an array of Protobuf objects. If using Protobuf, the corresponding class must be installed along with the gRPC query extension so it is available to the Broker at runtime.
Example usage
Sample request,
When using Protobuf response format, bundle up your Protobuf classes
into a jar file, and place that jar file in the
$DRUID_HOME/extensions/grpc-query
directory.Specify the response Protobuf message name in the request.
Release note
This is a "contrib" extension. We don't seem to document such extensions in Druid itself. The README.md can serve as documentation instead.
This PR has: