Skip to content

Commit

Permalink
[Docs] Clarify ledger rollover criteria: address size and min time (a…
Browse files Browse the repository at this point in the history
…pache#10911)

* [Docs] Clarify ledger rollover criteria: address size and min time

* Include 2.8.0 docs in update
  • Loading branch information
michaeljmarshall authored Jun 21, 2021
1 parent 2ea1ce6 commit b366ade
Show file tree
Hide file tree
Showing 32 changed files with 64 additions and 55 deletions.
9 changes: 5 additions & 4 deletions conf/broker.conf
Original file line number Diff line number Diff line change
Expand Up @@ -892,10 +892,11 @@ managedLedgerCursorBackloggedThreshold=1000
managedLedgerDefaultMarkDeleteRateLimit=1.0

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
9 changes: 5 additions & 4 deletions conf/standalone.conf
Original file line number Diff line number Diff line change
Expand Up @@ -627,10 +627,11 @@ managedLedgerCursorBackloggedThreshold=1000
managedLedgerDefaultMarkDeleteRateLimit=0.1

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
9 changes: 5 additions & 4 deletions deployment/terraform-ansible/templates/broker.conf
Original file line number Diff line number Diff line change
Expand Up @@ -785,10 +785,11 @@ managedLedgerCursorBackloggedThreshold=1000
managedLedgerDefaultMarkDeleteRateLimit=1.0

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1471,9 +1471,11 @@ public class ServiceConfiguration implements PulsarConfiguration {
@FieldContext(
category = CATEGORY_STORAGE_ML,
doc = "Max number of entries to append to a ledger before triggering a rollover.\n\n"
+ "A ledger rollover is triggered on these conditions Either the max"
+ " rollover time has been reached or max entries have been written to the"
+ " ledger and at least min-time has passed")
+ "A ledger rollover is triggered after the min rollover time has passed"
+ " and one of the following conditions is true:"
+ " the max rollover time has been reached,"
+ " the max entries have been written to the ledger, or"
+ " the max ledger size has been written to the ledger")
private int managedLedgerMaxEntriesPerLedger = 50000;
@FieldContext(
category = CATEGORY_STORAGE_ML,
Expand Down
9 changes: 5 additions & 4 deletions pulsar-client-cpp/test-conf/standalone-ssl.conf
Original file line number Diff line number Diff line change
Expand Up @@ -185,10 +185,11 @@ managedLedgerCacheEvictionWatermark=0.9
managedLedgerDefaultMarkDeleteRateLimit=0.1

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
9 changes: 5 additions & 4 deletions pulsar-client-cpp/test-conf/standalone.conf
Original file line number Diff line number Diff line change
Expand Up @@ -173,10 +173,11 @@ managedLedgerCacheEvictionWatermark=0.9
managedLedgerDefaultMarkDeleteRateLimit=0.1

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
9 changes: 5 additions & 4 deletions pulsar-client-cpp/tests/authentication.conf
Original file line number Diff line number Diff line change
Expand Up @@ -179,10 +179,11 @@ managedLedgerCacheEvictionWatermark=0.9
managedLedgerDefaultMarkDeleteRateLimit=0.1

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
9 changes: 5 additions & 4 deletions pulsar-client-cpp/tests/standalone.conf
Original file line number Diff line number Diff line change
Expand Up @@ -171,10 +171,11 @@ managedLedgerCacheEvictionWatermark=0.9
managedLedgerDefaultMarkDeleteRateLimit=0.1

# Max number of entries to append to a ledger before triggering a rollover
# A ledger rollover is triggered on these conditions
# * Either the max rollover time has been reached
# * or max entries have been written to the ledger and at least min-time
# has passed
# A ledger rollover is triggered after the min rollover time has passed
# and one of the following conditions is true:
# * The max rollover time has been reached
# * The max entries have been written to the ledger
# * The max ledger size has been written to the ledger
managedLedgerMaxEntriesPerLedger=50000

# Minimum time between ledger rollover for a topic
Expand Down
2 changes: 1 addition & 1 deletion site2/docs/reference-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ brokerServiceCompactionThresholdInBytes|If the estimated backlog size is greater
|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed in cache for more than the configured time, will be evicted | 1000 |
|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged' and thus should be set as inactive. | 1000|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker |1024|
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker |1024|
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker |1024|
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger|The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker |1024|
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker. By default, uses 1/5th of available direct memory ||
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheSizeMB| Amount of memory to use for caching data payload in managed ledger. This memory is allocated from JVM direct memory and it’s shared across all the topics running in the same broker. By default, uses 1/5th of available direct memory ||
|managedLedgerCacheEvictionWatermark| Threshold to which bring down the cache level when eviction is triggered |0.9|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ Pulsar brokers are responsible for handling incoming messages from producers, di
|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed in cache for more than the configured time, will be evicted | 1000 |
|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of entries) from where a cursor should be considered 'backlogged' and thus should be set as inactive. | 1000|
|managedLedgerDefaultMarkDeleteRateLimit| Rate limit the amount of writes per second generated by consumer acking the messages |1.0|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered on these conditions: <ul><li>Either the max rollover time has been reached</li><li>or the max entries have been written to the ledger and at least min-time has passed</li></ul>|50000|
|managedLedgerMaxEntriesPerLedger| The max number of entries to append to a ledger before triggering a rollover. A ledger rollover is triggered after the min rollover time has passed and one of the following conditions is true: <ul><li>The max rollover time has been reached</li><li>The max entries have been written to the ledger</li><li>The max ledger size has been written to the ledger</li></ul>|50000|
|managedLedgerMinLedgerRolloverTimeMinutes| Minimum time between ledger rollover for a topic |10|
|managedLedgerMaxLedgerRolloverTimeMinutes| Maximum time before forcing a ledger rollover for a topic |240|
|managedLedgerCursorMaxEntriesPerLedger| Max number of entries to append to a cursor ledger |50000|
Expand Down
Loading

0 comments on commit b366ade

Please sign in to comment.