[improve][build, test] Sync commits from apache/pulsar branch-3.1 into 3.1_ds #227
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…le when using extensible load manager (apache#21213) (apache#21314) (cherry picked from commit 0454410)
…iew construction (apache#21270)" This reverts commit c46ed2d. (cherry picked from commit ceafb3b)
…es from compacted topic (apache#21465)" This reverts commit 80f921a. (cherry picked from commit c616deb)
…ssages from compacted topic (apache#21465) (apache#21571) (cherry picked from commit a303bd1)
… compaction (apache#21578) (apache#21601) (cherry picked from commit 6ddf83d)
(cherry picked from commit d276550)
… was thrown (apache#21810) After the topic policies update, there are many components will be updated one by one, even if the config of components has not been modified. There are the 11 components that need update: - `7` rate limiters(`publish`, `dispatch topic-level`, `dispatch subscription-level`, `dispatch resourceGroup-level`, `subscribe API`, `replication`, `shadow topic replication`) - update ManagedLedger configs(`retention`, `offloader`) - start/stop replication - start/stop compaction - start/stop deduplication Once a component update fails, the following update will be skipped. It would cause a confusing thing: you want to set a retention policy, but it will be skipped due to the `update subscribe rate limiter` failure (you did not edit the `subscribe rate limitation policy`) Since none of the components in the above list have any additional dependencies for individual updates, ensuring success as much as possible is appropriate. - Update topic policies as much as possible even if some component updates fail, all component updates are still in the same thread, and they still update one by one, just throw the error later. - Rename `updatePublishDispatcher` to `updatePublishRateLimiter` (cherry picked from commit ed59967) (cherry picked from commit 19c9e7f)
… (ExtensibleLoadManager) (apache#21855) (cherry picked from commit fe1d9ef)
…che#21869) (cherry picked from commit 11eee3c)
…es (apache#21865) (cherry picked from commit d9029c6)
…pache#21893) ### Motivation When `managedLedgerInactiveLedgerRolloverTimeSeconds` is set, let's say to `300` (5 minutes), the ledger will also get rolled in the case when no new entries (messages) were added to the ledger. This doesn't make sense. Empty ledgers are deleted, but having this extra churn is causing extra load on brokers, bookies, and metadata stores (zookeeper). ### Modifications Skip rolling the ledger if it is empty. (cherry picked from commit 49edc3d) (cherry picked from commit 84a4885)
…ic with a suffix-matched regexp due to a mistake of PIP-145 (apache#21885) (cherry picked from commit 4ebbd2f) (cherry picked from commit ce8c291)
…utOfBoundsException (apache#21856) (cherry picked from commit 434b2c3)
…ispatcher (apache#21765) (cherry picked from commit d3dee51)
…dly (apache#21889) (cherry picked from commit a90e1f1)
… to avoid send task stuck (apache#21804) (cherry picked from commit 4806281)
(cherry picked from commit 2c383a1)
…ache#21867) (cherry picked from commit ea7e978)
…tions (apache#21466) (cherry picked from commit 99cab24)
…t and webServicePortTls are set (apache#21633) Co-authored-by: Jiwe Guo <technoboy@apache.org> (cherry picked from commit f8067b5) (cherry picked from commit 91e073d)
…rvicePort and webServicePortTls are set (apache#21633)" This reverts commit 91e073d. (cherry picked from commit 5e0b424)
…t and webServicePortTls are set (apache#21633) Co-authored-by: Jiwe Guo <technoboy@apache.org> (cherry picked from commit f8067b5) (cherry picked from commit ba1f8a1)
…t and webServicePortTls are set (apache#21842) (cherry picked from commit e10d318) (cherry picked from commit 8620610)
…sed address and advertised listeners are configured (apache#21894) (cherry picked from commit 3158fd3) (cherry picked from commit 358d122)
…ing Pulsar releases (apache#21937) (cherry picked from commit 6347315) (cherry picked from commit 89f722f)
…apache#22163) (cherry picked from commit 6ec473e) (cherry picked from commit 4f2e4c8)
…d client clock skew (apache#21940) (cherry picked from commit 861618a) (cherry picked from commit fee3a33)
…pache#21955) (cherry picked from commit d3b7ca5) (cherry picked from commit d58d598)
… in expiryMon… (apache#21991) Co-authored-by: atomchchen <atomchchen@tencent.com> (cherry picked from commit 220a3d6) (cherry picked from commit 09cb541)
…isabled allowAutoSubscriptionCreation (apache#22078) (cherry picked from commit bbf6ddf) (cherry picked from commit 7eec36e)
apache#22144) (cherry picked from commit e3a081e) (cherry picked from commit 93a4457)
…pache#22149) (cherry picked from commit 74be3fd) (cherry picked from commit c6b4887)
…ache#22153) (cherry picked from commit 72cedb7) (cherry picked from commit bbbd1ef)
… sources (apache#22184) (cherry picked from commit 207335a) (cherry picked from commit b183492)
(cherry picked from commit 7e28e84)
… port if TLS is enabled (apache#21015) (cherry picked from commit 1363777) (cherry picked from commit 628e79d)
This reverts commit 6d379b9.
srinath-ctds
approved these changes
Mar 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
This PR sync all commits apache/branch-3.1 into 3.1_ds.
It cherry-picked the commits of branch-3.1 from 13th Oct 2023 till 4 Mar 2024
Modifications
Describe the modifications you've done.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
doc-required(If you need help on updating docs, create a doc issue)
no-need-doc(Please explain why)
doc(If this PR contains doc changes)