How to submit message offset asynchronously based on kafka protocol

[yanmxa]

There is a scenario base on Kafka protocol binding and I'm not sure how to use the sdk to achieve it:

When the cloudevent client receives an event, it returns the NACK response to tell Kafka not to commit the message offset. And also forwards the event to the backend program, when the backend program has completely consumed the event, then submits the offset of the message partition.


If I can't use cloudevents client to submit message offset directly, how can I get the message partition and offset from the cloud event? so that the other progress can use the kafka client to submit this offset once the message is consumed successfully.

[embano1]

@yanmxa is this still an issue/open question?

[yanmxa]

Yes.

If it is based on this saram protocol, it seems a bit difficult to achieve asynchronous message confirmation. So I referred using the confluent protocol, #918, which can be combined with the cloudevents client to do that.

[embano1]

Another popular one is https://github.com/twmb/franz-go.

@duglin how are we thinking about different implementations of open protocols, such as Kafka? It might be a bit of maintenance headache on the long-term to support multiple implementations of the same transport API. But I can understand the reasons as outlined by @yanmxa. Do we want to just provide one "reference" implementation, add multiple, or switch?

[yanmxa]

Resolved by this PR: #988