DeltaQuery

DeltaQuery is an open-source SQL query engine based on Arrow FlightSql to support data lakehouse architecture using DeltaLake. It depends on various, powerful open-source engines for bigdata, such as Arrow, Delta, Duckdb, and so on.

Building

DeltaQuery is compiled using Cargo.

To compile, run

cargo build

To execute sample Minio server using Docker, run in the samples directory

docker-compose -f docker-compose-minio.yaml up

To execute server, run

cargo run -- --logfilter deltasync=info --config ./samples/configs/delta.yaml --catalog ./samples/configs/local.yaml

To execute python client using ADBC, run

python3 ./examples/adbc_flightsql_query.py --server "grpc://localhost:32010" --authorization "Basic YWRtaW46YWRtaW4K" --query "select date,hour,name,score from delta.default.test0"

To execute rust client using Arrow FlightSQL, run

cargo run -p flight_sql -- --logfilter flight_sql=info --command statement-query --host 127.0.0.1 --port 32010 --protocol http --authorization "Basic YWRtaW46YWRtaW4K" --query "select date,hour,name,score from delta.default.test0" --output pretty

License

Apache License 2.0, see LICENSE.