Question: Why DeDuplicationStrategy class is so specific?

[jmirc]

I developed a filtering feature on top of Kafka using flatbuffer. The filter uses headers to filter or not the message. The goal is to deserialize the payload only when the consumer accepts the message.
I finished yesterday to migrate my old spring-kafka library based on the version 0.8 to use this project and it works.

My question is:
A DeDuplicationStrategy has been created to filter the message is the message is a dupe. This strategy is great for the dedupe use case. However, this feature could be more generic to support many use cases.
My use case could fit perfectly in this strategy, however my use case is not a dedupe is a filter. So I decided to not use the DeDuplicationStrategy interface to handle this use case.

I would like to know if the DeDuplicationStrategy could be replaced by a generic interface that could support different strategies (dedupe, filtering or another use case) to not send the message to the listener.

I am hopping I am clear.

Thanks for your great work.

[artembilan]

Hello, @jmirc !

Thank you for feedback.
Well, the feature has been introduced very recently in the current RC1 milestone: #80 .

Even if you can achieve your filtering goal with the current DeDuplicationStrategy (with some inconvenience around the interface and its method names), I agree that we can rename it into FilterStrategy, when dedup logic ins covered as well.

That is how we can implement Idempotent Receiver with the regular Spring Integration Filter: http://docs.spring.io/spring-integration/reference/html/messaging-endpoints-chapter.html#idempotent-receiver.

@garyrussell , WDYT?

[garyrussell]

Yes, fine with me; thanks for the feedback.

[mbogoevici]

I am wondering if we should perhaps consider making this a Listener implementation or an advice - there are a few other concerns that could be addressed here - including retry. And at this point there is a risk for the container (which is not very simple by itself) to become very complex, IMO.

[artembilan]

@mbogoevici , @garyrussell did that as DeDuplicatingMessageListenerAdapter already, but yes, adviceChain around the listener.onMessage() sounds good for me.
And we could move retryTemplate from puase/resume solution to the Advice too.

[garyrussell]

Right - that's why I decided to do it as listener adapter to keep the concern out of the container itself - we could add an advising listener adapter with an advice chain. Of course, individual adapters can be chained too - avoiding reflection, proxies etc.

Yes, we could move retry to an adapter too.

[garyrussell]

PR issued - I created a separate issue #105 for the retry template extraction.