-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kafka protocol support #196
Comments
Hey @davidwmartines, thx for opening this issue. Yes, as you mentioned, the repo is very http-centric, unfortunately :-( I'm looking forward to having some free time to bring Protobuf support, hopefully, this should bring some clarity regarding adding more formatting options. I personally don't have Kafka experience, but curious how this complies with the Cloudevents spec. Need to check. And yeah, PRs are very welcome |
@xSAVIKx I started exploring this and came up with a draft of something that seems to work. I'm not sure if it's the right approach, though. For one thing, the Alternatively, I could simply not create a Your thoughts? (BTW, I researched more and discovered the cloudevents and knative connection, so now the http focus of all this makes more sense.) |
Yeah, I guess |
OK, that makes sense. I have updated my PR to remove the kafa.CloudEvent class, and the kafka conversion functions should work with AnyCloudEvent (both http and pydantic work, at least in my simple testing). I still need to add some unit tests. I think I like this approach. |
Are there any plans for Kafka protocol support in this sdk?
I would primarily be interested in using binary mode with Kafka, with Avro serialization for the data. (Although structured mode for JSON would probably need to be supported, too.)
It seems the current HTTP CloudEvent does most of what is needed, except:
ce_
(notce-
), with the exception ofcontent-type
.I would not expect any specific serialization support, as I would rather be able to pass in any serializer of my choice such as the AvroSerializer from the confluent-kafka-python package, as a callable.
I'd be willing to help with this, but I'm not sure where to begin in this repo as it seems very http-centric at the moment.
The text was updated successfully, but these errors were encountered: