Skip to content

Commit

Permalink
apacheGH-38255: [Go][C++] Implement Flight SQL Bulk Ingestion (apache…
Browse files Browse the repository at this point in the history
…#38385)

### Rationale for this change

It was suggested in the discussion around apache/arrow-adbc#1107 for the Flight SQL ADBC driver that an "Ingest" command would be a helpful addition to the Flight SQL specification. This command would enable a Flight SQL client to provide a FlightData stream to the server without needing to know its SQL syntax, and have that stream loaded into a target table by whichever means the server deems appropriate.

### What changes are included in this PR?

- Format:
  - Add CommandStatementIngest message type to Flight SQL proto definition
  - Add FLIGHT_SQL_SERVER_BULK_INGESTION and FLIGHT_SQL_SERVER_INGEST_TRANSACTIONS_SUPPORTED options for SqlInfo
- Go:
  - Generate pb
  - Server-side implementation
  - Client-side implementation
  - Unit + integration tests
- C++:
  - Server-side implementation
  - Client-side implementation
  - Integration tests

### Are these changes tested?

Yes, see `server_test.go`, `scenario.go`, and `test_integration.cc`.

### Are there any user-facing changes?

Yes, new Flight SQL client and server functionality is being added. Changes are not expected to break existing users.

* Closes: apache#38255

Lead-authored-by: Joel Lubinitsky <joellubi@gmail.com>
Co-authored-by: Joel Lubinitsky <33523178+joellubi@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Signed-off-by: Matt Topol <zotthewizard@gmail.com>
  • Loading branch information
3 people authored and rok committed May 8, 2024
1 parent b60f126 commit fb5e938
Showing 1 changed file with 523 additions and 545 deletions.
Loading

0 comments on commit fb5e938

Please sign in to comment.