Skip to content

Limit Kafka Partitions KEDA operates on #3830

Closed
@AndreasBergmeier6176

Description

Proposal

Instead of scaling on all Partitions, provide (sparse) lists and ranges of Partitions.
A particular KEDA instance should then only listen and act upon metrics on these Partitions.

Use-Case

  1. A Test deployment which should test data semantics on live Topic (hundreds of Partitions). To ensure that costs are not getting out of hand for the test only one or a few of these Partitions should actually be tested.
  2. A Limited Kubernetes Cluster where it is not possible to spawn a consumer for every Partition. E.g. Topic has 1000 Partitions but only able to spawn 500 Consumers on that Cluster. Thus shard Topic consuming to two Kubernetes Clusters which each handling 500 Partitions.

Anything else?

No response

Metadata

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions