Auth for Flight RPC/SQL

[prochac]

Hi,

I'm kinda having a confusion how authentication is intended for these two.
As obvious, I'm using Go and Arrow Go module.

I'm implementing Arrow Flight SQL server, and need to use some auth mechanism.

So far, I'm using flight.BasicAuthValidator with combination with flight.CreateServerBasicAuthMiddleware

But I also found flight.ServerAuthHandler that can be set to embedded flight.BaseFlightServer, and then be used by flight.Server that must be exclusively created by flight.NewServerWithMiddleware, as the interceptors aren't otherwise available.

If I'm correct, the authentication is a matter of the Flight RPC layer.
Therefore, it confuses me, that the first method uses Basic "authorization" for /Handshake, and then it's being transformed to Bearer token, returned as "authorization" Trailer from server, and client switches the basic "authorization" to bearer on its side.
Meanwhile, the ServerAuthHandler uses "auth-token-bin" key for metadata, and reads the "login" credentials from the flight.HandshakeRequest payload into flight.BasicAuth, and returns the future "auth-token-bin" as flight.HandshakeResponse Payload

So the BasicAuth and BasicAuthValidator aren't even used together.

What is the unified Auth mechanism for Flight RPC? Does Flight SQL use different?

By my experience, all clients, like Flight SQL JDBC driver, uses only the first method, using "authorization" header and trailer.

[zeroshade]

TL;DR: There isn't a "unified" auth mechanism for Flight RPC or Flight SQL, but most libraries have implemented "Basic" auth for convenience.

Therefore, it confuses me, that the first method uses Basic "authorization" for /Handshake, and then it's being transformed to Bearer token, returned as "authorization" Trailer from server, and client switches the basic "authorization" to bearer on its side.

This is the default way to perform simple authentication with Flight and Flight SQL: using simple "Basic" authentication via the http protocol (grpc uses HTTP/2). You send a message to Handshake with an empty payload using Basic authorization, which validates it via implementing a BasicAuthValidator and returns a auth token for further requests, which gets sent to the Validate method. This is a convenience implementation, as it's the most common, and avoids users having to implement it themselves every time.

Meanwhile, the ServerAuthHandler uses "auth-token-bin" key for metadata, and reads the "login" credentials from the flight.HandshakeRequest payload into flight.BasicAuth, and returns the future "auth-token-bin" as flight.HandshakeResponse Payload

This is for implementing a custom authentication method beyond "Basic" authentication. The BasicAuth message is a convenience for the case where you don't want the client to use grpc headers and instead want them to send a Username and Password as a payload to the /Handshake method. If you need any more information or complexity than that, you need to use your own custom protobuf message as the payload. It also means you'd need to manually implement the /Handshake method instead of just using the default behavior.

The auth-token-bin metadata key goes hand-in-hand with a custom implementation of authentication, if you're not using simple Basic and Bearer authentication then you need a way to ensure that Flight and FlightSQL servers know how to find the authentication token that is returned after the handshake. The ecosystem has stabilized on a canonical metadata key called auth-token-bin i.e. "Authentication Token Binary" which should contain whatever is necessary to validate a particular authentication session etc. This allows the common case ("Basic" Auth + "Bearer" token) to be easily managed via helpers, while also facilitating the complex case where you need to perform custom authentication between languages.

So the BasicAuth and BasicAuthValidator aren't even used together.

Correct, BasicAuth is a convenience message that could be utilized with ServerAuthHandler and ClientAuthHandler to construct a simple Username + Password auth message to send via /Handshake so that everyone doesn't need to constantly create their own version of this type. BasicAuthValidator is for validating the "authorization" header via HTTP Basic Authentication.

By my experience, all clients, like Flight SQL JDBC driver, uses only the first method, using "authorization" header and trailer.

In addition to using the "authorization" header/trailer, the Rust, Java, and C++ (and by extension, many other implementations that bind to the C++ impl like Python and Ruby) implementations also allow using the "auth-token-bin" metadata for custom auth mechanisms just like the Go implementation. Specifically, it was added to the Go implementation so that we would follow suit and be in line with the other Flight RPC implementations.