You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Kafka requires at least one partition per concurrent consumer, and without a properly set routing key (aka partition key) only one partition will be written too. There is some information suggesting that the partition written to will be changed after 10m, but this would be too long of a windows for most uses.
While sometimes it is possible to use a common tag such as host, and in many respects this is ideal as it maintains ordering of each series, this tag is not always set.
Add a new parameter to the Kafka output plugin to add a random routing key to each message.
Current behavior:
Messages can be sent without a routing key or using the routing_tag option to choose a tag whose value is used as the routing key.
Desired behavior:
Similar to the amqp output, add a new routing_key option to set a static key, or if set to "random" then generate a random key.
[[outputs.kafka]]
routing_key = "random"
Use case:
Without a common tag between all metrics this is required for spreading writes to multiple partitions.
The text was updated successfully, but these errors were encountered:
Feature Request
Proposal:
Kafka requires at least one partition per concurrent consumer, and without a properly set routing key (aka partition key) only one partition will be written too. There is some information suggesting that the partition written to will be changed after 10m, but this would be too long of a windows for most uses.
While sometimes it is possible to use a common tag such as
host
, and in many respects this is ideal as it maintains ordering of each series, this tag is not always set.Add a new parameter to the Kafka output plugin to add a random routing key to each message.
Current behavior:
Messages can be sent without a routing key or using the
routing_tag
option to choose a tag whose value is used as the routing key.Desired behavior:
Similar to the amqp output, add a new routing_key option to set a static key, or if set to "random" then generate a random key.
Use case:
Without a common tag between all metrics this is required for spreading writes to multiple partitions.
The text was updated successfully, but these errors were encountered: