You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently MergeRollupTask runs in a forward manner, meaning segments that were processed are not processed again even after modifying the watermark in Zookeeper.
Possible use case is that if ingestion of data is not uniform, some days receive more events than others, so rollup for days where events are less, results in let's say 10 segments, but for days where events are much higher, segments are rolled up into 1000 segments.
If MergeRollupTask can be made to run again with new configuration for those affected days it would be ideal.
Druid has this really useful feature called Reindex which does this.
The text was updated successfully, but these errors were encountered:
Currently
MergeRollupTask
runs in a forward manner, meaning segments that were processed are not processed again even after modifying the watermark in Zookeeper.Possible use case is that if ingestion of data is not uniform, some days receive more events than others, so rollup for days where events are less, results in let's say 10 segments, but for days where events are much higher, segments are rolled up into 1000 segments.
If
MergeRollupTask
can be made to run again with new configuration for those affected days it would be ideal.Druid has this really useful feature called Reindex which does this.
The text was updated successfully, but these errors were encountered: