Make Kafka AwaitMessageSensor be able to not commit offset #62854
Unanswered
filippzorin
asked this question in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi everyone, I love using Airflow and I've been trying to work with Kafka
AwaitMessageSensor(fromapache-airflow-providers-apache-kafka) to trigger task when message is produced to particular topic. In my case I'd like to not commit offset when sensor polls messages, I'm going to commit offset when message is processed in next task.DAG simplified example:
What do you think about adding option to
AwaitMessageSensorto manage commit behavior while sensor polls messages?Beta Was this translation helpful? Give feedback.
All reactions