Scenario:

By default, CFP will try to pick up as many as partitions for processing, as the increase of the number of the partitions of the monitored container, with fixed number of CFP instances, the CFP instance will slowly get into high CPU/high memory as more and more worker threads being spinned up.

Even though as of today, customer can use maxScaleCount to help with the high CPU/Memory situation, but it will cause staled partitions. So essentially customer want a way to control the max tasks/workers within each CFP instance, and making sure all the partitions will get processed.

Design challenges:
Possible increased lagging
How to decide which partitions to be processed next - scenarios like some partitions may have more changes than others