Allow `auth` property of httpx to be passed to schema registry client

[ericdevries]

Description

We have some custom logic to acquire oauth tokens that need to be sent as a header to the schema registry. Unfortunately, the schema registry client only allows for basic auth and ssl auth. We worked around it like this:

class SchemaRegistryClient(ConfluentKafkaSchemaRegistryClient):
    def __init__(self, config: SchemaRegistryConfig):
        config_dict = {
            "url": config.url
        }
        super().__init__(config_dict)

        if config.use_oauth:
            # override httpx client because we cannot configure a custom auth class
            self._rest_client.session = httpx.Client(
                verify=self._rest_client.verify,
                cert=self._rest_client.cert,
                # This is the property that can't be passed along
                auth=AADAuth(OauthClient(scope=urlsplit(config.url).netloc)),
                proxy=self._rest_client.proxy,
                timeout=self._rest_client.timeout,
            )


class AADAuth(httpx.Auth):
    def __init__(self, oauth_client: OauthClient) -> None:
        self._oauth_client = oauth_client

    def auth_flow(self, request: httpx.Request) -> Generator[httpx.Request, httpx.Response, None]:
        token = self._oauth_client.get_token()
        request.headers["Authorization"] = f"Bearer {token}"

        yield request

Allowing the auth property to be passed to the schema registry client would make it a lot more flexible for anything outside of basic auth and certificate auth.

How to reproduce

Checklist

Please provide the following information:

• confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()):
• Apache Kafka broker version:
• Client configuration: {...}
• Operating system:
• Provide client logs (with 'debug': '..' as necessary)
• Provide broker log excerpts
• Critical issue

[Claimundefine]

https://github.com/confluentinc/confluent-kafka-javascript/blob/master/schemaregistry/rest-service.ts#L27C1-L36C2

Hello, we are looking to implement OAuth authentication by end of Q1, similar to what we have in our Javascript client. It would allow setting a static Bearer token or allow users to input their client credentials, where the client would handle retrieving, caching, and refreshing the Bearer token from your OAuth Identity Provider.

Would this be the functionality you are looking for?

[ericdevries]

In our case we have slightly different ways of acquiring tokens, using a certificate. A “regular” oauth2 solution wouldn’t work in that case. I’d rather have a callable solution like httpx provides, which is what we use now. Perhaps making it a callable and providing a default oauth2 solution would give the flexibility we need while also providing simple to use oauth2 support for more standardized environments

[Claimundefine]

The standardized solution using Client Credentials is currently in progress here, and shortly after we will be adding an additional config field where a user can pass in a function that returns {Bearer Token, Logical Cluster, Identity Pool Id}, to which the fields will be appended to the header.

We are also working on a native jwt_bearer solution sometime in Q2. For now, you can expect custom functions to retrieve Bearer Tokens to be supported soon.

[Claimundefine]

Hi @ericdevries, feel free to check out the new examples for OAuth, currently available with v2.9.0.

[ericdevries]

Thanks, will check it out!