backend: add update partition count dataplane api #1638
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf CI / push-module (pull_request).
|
| "github.com/twmb/franz-go/pkg/kadm" | ||
| ) | ||
|
|
||
| // AddPartitionsToTopics adds partition counts to topics. |
There was a problem hiding this comment.
Comments are somewhat weird for these functions
| }; | ||
| } | ||
|
|
||
| rpc SetTopicPartitions(SetTopicPartitionsRequest) returns (SetTopicPartitionsResponse) { |
There was a problem hiding this comment.
What happens if you set fewer partitions than you currently have? Will partitions be just deleted with the data? Will the data be moved onto the remaining partitions and only then deleted? Etc.
I think we should also clearly point out that changing the partition count is going to impact ordering guarantees - depending on the Partitioner. The most commonly used partitioners determine it based on the hash of the key and modulo it by number of partitions. So the same won't go to the same partition anymore. I think we need to point this out as a warning in all the descriptions.
There was a problem hiding this comment.
What happens if you set fewer partitions than you currently have?
That's an invalid request that's prevented at least at franz-go level (docs). Possibly at protocol / broker level as well.. Can't find conclusively the answer from quick search. UDATE: I have confirmed the enforcement is done at broker.
I have updated docs in proto definitions too.
Co-authored-by: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
Co-authored-by: Martin Schneppenheim <23424570+weeco@users.noreply.github.com>
This adds dataplane API to update partition counts for topic(s).
This has been asked by numerous customers. While working on v1 migration I took a little bit of time to add this in a separate work for a quick follow up PR. This is that PR.
My idea is to merge into release 2.8 branch. And then also do equivalent PR into master when AuthN rework is merged.