Zilla with Azure EventHubs for Kafka

[yukscck]

Hi,

I manage communication of events among a few services through an Azure EventHubs. Lately, I'm starting to see the need for supporting different protocols, such as MQTT and GRPC to enable more services to join the communications through EventHubs. I came across Zilla and I'm quite interested in exploring if it solves my problem.

EventHubs, on top of AMQP, supports event publishers (producers) and event receivers (consumers) using Kafka protocol. I plan to put a reverse proxy in front of the EventHubs to abstract the specifics away from connected services.

I'd like to ask, if anyone has explored and successfully use EventHubs in place of the Kafka scenarios Zilla supports?

Thanks!

[vordimous]

Hi @yukscck, Zilla implements the Kafka protocol and should work with any project that exposes it. With that Zilla doesn't currently support every Authentication mechanism. Which Auth mechanism will you need when connecting to EventHubs?

Here is the Azure docs link for the available Auth options: https://learn.microsoft.com/en-us/azure/event-hubs/azure-event-hubs-kafka-overview#security-and-authentication

[yukscck]

Hi @vordimous, appreciate your response!

In my environment, services integrate with the Event Hubs using the Oauth 2.0 approach. From what I have tested, the connection is required to use SASL SSL, with the SASL mechanism being OAuthbearer.

On top of being able to accept incoming connections in MQTT, GRPC or AMPQ, I'd like to be able to accept incoming connections in Kafka Protocol as well. Will that be possible?

Thanks!

[vordimous]

In my environment, services integrate with the Event Hubs using the Oauth 2.0 approach. From what I have tested, the connection is required to use SASL SSL, with the SASL mechanism being OAuthbearer.

Zilla doesn't currently support the OAuthbearer SASL mechanism but does support the SASL PLAIN option.

On top of being able to accept incoming connections in MQTT, GRPC or AMPQ, I'd like to be able to accept incoming connections in Kafka Protocol as well. Will that be possible?

Zilla can proxy MQTT, GRPC connections to the Kafka protocol as well as proxy Kafka protocol connections directly. Zilla currently doesn't support the AMPQ to Kafka protocol proxy.