[C++][Flight] Fully support dictionary replacement in Flight

[amoeba]

Describe the enhancement requested

As of writing Flight C++ currently only supports dictionary replacement in DoPut calls and not in DoGet or (partially) DoExchange.

Sub-issues:

• DoGet: [C++][Flight] Support dictionary replacement in DoGet #45055
• DoExchange: [C++][Flight] DoExchange doesn't support dictionary replacement #26727

Component(s)

C++, FlightRPC

[pitrou]

Is DoExchange also handled by #47115 @raulcd ? If so, can we update this issue and the corresponding sub-issue?

[raulcd]

Is DoExchange also handled by #47115

I'm not familiar with DoExchange so not sure if it reuses the same code path. I'll have to set up a test and try, I can work on that and will update with findings.

[raulcd]

DoExchange doesn't seem to be using the same code path as the returned table is not allowing for dictionary replacement nor dictionary deltas. Test reproducer:

def test_flight_dictionary_deltas_do_exchange():
    class DeltaFlightServer(ConstantFlightServer):
        def do_exchange(self, context, descriptor, reader, writer):
            if descriptor.command == b'dict_deltas':
                table = simple_dicts_table()
                options = pa.ipc.IpcWriteOptions(emit_dictionary_deltas=True)
                writer.begin(table.schema, options=options)
                writer.write_table(table)

    with DeltaFlightServer() as server, \
            FlightClient(('localhost', server.port)) as client:

        descriptor = flight.FlightDescriptor.for_command(b"dict_deltas")
        writer, reader = client.do_exchange(descriptor,
                                            options=flight.FlightCallOptions(
                                                write_options=pa.ipc.IpcWriteOptions(emit_dictionary_deltas=True)
                                            )
                                           )

        received_table = reader.read_all()
        expected_table = simple_dicts_table()
        assert received_table.equals(expected_table)

The received table did not update the dictionary (neither deltas / or dictionary replacement)

(Pdb) received_table
pyarrow.Table
some_dicts: dictionary<values=string, indices=int64, ordered=0>
----
some_dicts: [  -- dictionary:
["foo","baz","quux"]  -- indices:
[1,0,null],  -- dictionary:
["foo","baz","quux"]  -- indices:
[2,1],  -- dictionary:
["foo","baz","quux"]  -- indices:
[0,3]]
(Pdb) expected_table
pyarrow.Table
some_dicts: dictionary<values=string, indices=int64, ordered=0>
----
some_dicts: [  -- dictionary:
["foo","baz","quux"]  -- indices:
[1,0,null],  -- dictionary:
["foo","baz","quux"]  -- indices:
[2,1],  -- dictionary:
["foo","baz","quux","new"]  -- indices:
[0,3]]

[raulcd]

Both DoGet and DoExchange have been implemented, see the sub-issues above.

[amoeba]

Nice work @raulcd, thanks for taking this on.