[improve][broker] Create non-partitioned system topics in auto topic creation #20397
Conversation
lhotari
added
type/bug
lhotari
requested review from
hangc0276,
Technoboy-,
315157973,
codelipenghui,
poorbarcode,
michaeljmarshall and
mattisonchao
|
/pulsarbot rerun-failure-checks |
| // system topics should be non-partitioned by default regardless of the broker config | ||
| // allowAutoTopicCreationType setting | ||
| if (SystemTopicNames.isSystemTopic(topicName) | ||
| || NamespaceService.isSystemServiceNamespace(topicName.getNamespace())) { | ||
| return false; | ||
| } |
There was a problem hiding this comment.
Why must it be non-partitioned?
Should it be configurable?
Should this be discussed in the mail list?
There was a problem hiding this comment.
Why must it be non-partitioned?
It's not "must", it's more about it being a bad default to make system topics such as the __change_events a partitioned topic. The amount of changes going into these topics isn't something that would justify using partitioned topics by default.
Should it be configurable?
If someone wants to use a specific amount of partitions, it's possible to create the system topics manually. Similarly as for transactions.
Should this be discussed in the mail list?
yes. I guess @michaeljmarshall had plans to open a discussion around this.
There was a problem hiding this comment.
I am pretty sure we want most, if not all, of these to be a single topic to ensure a specific ordering of events. It's worth discussing on the mailing list though. I can start something later today.
| // system topics should be non-partitioned by default regardless of the broker config | ||
| // allowAutoTopicCreationType setting | ||
| if (SystemTopicNames.isSystemTopic(topicName) | ||
| || NamespaceService.isSystemServiceNamespace(topicName.getNamespace())) { | ||
| return false; | ||
| } |
There was a problem hiding this comment.
I am pretty sure we want most, if not all, of these to be a single topic to ensure a specific ordering of events. It's worth discussing on the mailing list though. I can start something later today.
|
The pr had no activity for 30 days, mark with Stale label. |
|
For anyone interested, I did raise this one the mailing list here: https://lists.apache.org/thread/1lndgf1hx821fc12t0pc6j6zdrhkntht. There wasn't much engagement. I am going to leave this open for now. |
|
The pr had no activity for 30 days, mark with Stale label. |
lhotari
added
release/3.0.9
release/4.0.2
and removed
release/3.0.8
release/4.0.1
labels
lhotari
added
release/3.0.10
and removed
release/3.0.9
release/2.10.5
release/2.11.2
release/4.0.2
release/3.0.10
labels
Motivation
Modifications
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: lhotari#147