feat: support FlightSQL in 3.0 #24678
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds the ability to serve FlightSQL queries. gRPC is served over the same address as HTTP. The Arrow Flight service is set up, although some of the underlying traits are not complete.
This commit brings support to the influxdb3 server to handle gRPC requests, i.e., FlightSQL requests. The primary server will route gRPC reqeusts to an underlying FlightService, while other requests will still go through to the standard REST HTTP server. A test was added to check that it was actually working, along with some common test helpers to spawn a server for testing.
This was referenced Feb 21, 2024
hiltontj
force-pushed
the
hiltontj/flight-sql
branch
from
1252841
to
940f1de
Compare
👏 this was never something I was able to figure out when implementing the services in |
mgattozzi
approved these changes
Feb 26, 2024
|
Created #24695 to come back and add some documentation. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #24643
This PR adds support for FlightSQL queries via gRPC to the
influxdb3service. It does so by ensuring theQueryExecutorimplements theQueryNamespaceProvidertrait, and the underlyingQueryDatabaseimplementsQueryNamespace. Satisfying those requirements allows the construction of aFlightServiceServerfrom theservice_grpc_flightcrate.The
FlightServiceServeris a gRPC server that can be served viatonicat the API surface; however, enabling this required sometower::Servicewrangling. Theinfluxdb3_server/src/server.rsmodule was introduced to house this code. The code there is largely inspired by this series of blog posts. The objective is to serve both gRPC (via the newly introducedtonicserver) and standard REST HTTP requests (via the existing HTTP server) on the same port.This is accomplished by the
HybridServicewhich can handle either gRPC or non-gRPC HTTP requests. TheHybridServiceis wrapped in aHybridMakeServicewhich allows us to serve it viahyper::Serveron a single bind address.End-to-end tests were added in
influxdb3/tests/flight.rs. These cover some basic FlightSQL cases. Acommon.rsmodule was added that introduces some fixtures to aid in end-to-end tests ininfluxdb3.Follow-on work
Authorizerimplementation for InfluxDB Edge #24681 to track work on that.influxdb3integration test (auth.rs)