[C++][FlightRPC] Implement a UCX transport

[asfimport]

Depends on ARROW-15282.

Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm

Related issues:

• [C++][Flight] DoExchange doesn't support dictionary replacement (relates to)
• [C++][FlightRPC] Benchmark in-process Flight performance (relates to)
• [C++][FlightRPC] Refactor auth, middleware into the transport-agnostic layer (relates to)
• [C++][FlightRPC] Refactor remaining methods into transport-agnostic handlers (relates to)
• [C++][FlightRPC] UCX server should be able to shed load (relates to)
• [C++][FlightRPC] Implement shutdown with deadline for UCX (relates to)
• [C++][FlightRPC] Pipeline memory allocation/registration (relates to)
• [C++][FlightRPC] Improve concurrent call implementation in UCX client (relates to)
• [C++][FlightRPC] Investigate TSAN with gRPC/UCX tests (relates to)
• [C++][FlightRPC] Refactor error statuses/codes into the transport-agnostic layer (relates to)
• [Python][FlightRPC] Expose UCX transport to Python (is related to)
• [C++][FlightRPC] Support non-grpc data planes (supercedes)

PRs and other links:

• GitHub Pull Request #12442

_{Note: This issue was originally created as ARROW-15706. Please see the migration documentation for further details.}

[asfimport]

Yibo Cai / @cyb70289:
Hi @lidavidm, I saw you filed some issues about ucx. I have some colleagues working at ucx as core maintainers. Actually, the idea of integrating ucx into flightrpc is from Pavel Shamis in an internal tech meeting. Ping me if you met with some difficulties in ucx and need help. I think Pavel will be okay to give some support (unfortunately I have little knowledge about ucx).

[asfimport]

David Li / @lidavidm:
Thanks for the offer! I actually was introduced to Pavel through a colleague of mine and he has been advising on the approach.

[asfimport]

David Li / @lidavidm:
Issue resolved by pull request 12442
#12442