Description
Is your feature request related to a problem? Please describe.
I'm building a distributed job queue using a shared subscription. Job processing time range from 1 minute to 12 hours.
I'm looking for a reliable way to redeliver a message when a consumer (job worker) crash with the following requirements:
- Avoid processing the same message several times unless a consumer crashed or unreachable for a long time.
- Redeliver a message in a short time (10 minutes) after a consumer became unreachable without waiting for the maximum possible job processing time (12 hours).
Describe the solution you'd like
I'm looking for a way to let a consumer notify the broker that it is still processing a message to avoid unnecessarily redelivering it. I will set Acknowledgement timeout of 10 minutes on the topic. The consumer will send a 'RESET_ACK_TIMEOUT' or 'WORKING' command for this message to the broker every 5 minutes. The broker will redeliver unacknowledged messages only after ack-timeout has passed since the last reset-ack-timeout was sent by the consumer.
Describe alternatives you've considered
I've considered Negative acknowledgement, Acknowledgement timeout and Dead letter topic but couldn't find a way to reset the ackTimeout of a single message.
Additional context
Disque is a distributed job queue by the creator of redis. You can Add a job and specify a retry timeout.
With the WORKING joibid command you can:
Claims to be still working with the specified job, and asks Disque to postpone the next time it will deliver the job again. The next delivery is postponed for the job retry time.