Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KAFKA-17266: Add dynamic broker config to enable follower fetch using tiered offset #16834

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from
Open

KAFKA-17266: Add dynamic broker config to enable follower fetch using tiered offset #16834

wants to merge 1 commit into from

Conversation

abhijeetk88
Copy link
Contributor

@abhijeetk88 abhijeetk88 commented Aug 8, 2024 •
edited
Loading

In this PR, we add a dynamic broker config to enable/disable follower fetch using tiered offset. This is part of the implementation for KIP-1023

Added tests for the new dynamic broker config.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@abhijeetk88 abhijeetk88 marked this pull request as draft August 8, 2024 11:07
@abhijeetk88 abhijeetk88 changed the title Add follower fetch tiered offset dynamic property Add dynamic broker config to enable follower fetch using tiered offset Aug 8, 2024
@abhijeetk88 abhijeetk88 changed the title Add dynamic broker config to enable follower fetch using tiered offset KAFKA-17266: Add dynamic broker config to enable follower fetch using tiered offset Aug 8, 2024
@abhijeetk88 abhijeetk88 marked this pull request as ready for review August 8, 2024 14:18
@kamalcph kamalcph added the tiered-storage Related to the Tiered Storage feature label Aug 9, 2024
kamalcph
kamalcph reviewed Aug 13, 2024
View reviewed changes
Copy link
Contributor

@kamalcph kamalcph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, left minor comments to address. Thanks for the patch!

server/src/main/java/org/apache/kafka/server/config/ReplicationConfigs.java
@@ -136,6 +136,10 @@ public class ReplicationConfigs {
public static final String REPLICA_SELECTOR_CLASS_CONFIG = "replica.selector.class";
public static final String REPLICA_SELECTOR_CLASS_DOC = "The fully qualified class name that implements ReplicaSelector. This is used by the broker to find the preferred read replica. By default, we use an implementation that returns the leader.";

public static final String FOLLOWER_FETCH_LAST_TIERED_OFFSET_ENABLE_CONFIG = "follower.fetch.last.tiered.offset.enable";
public static final String FOLLOWER_FETCH_LAST_TIERED_OFFSET_ENABLE_DOC = "If enabled, an empty follower will skip replicating offsets up to the last tiered offset and start from the next offset.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc is not clear, can we update it to make it clear for the users. From KIP:

Whether the last tiered offset should be used as the start offset for bootstrapping an empty follower

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to update the KIP as well - The last tiered offset is actually not the start offset for the follower. It is the next offset.

@kamalcph
Copy link
Contributor

kamalcph commented Oct 7, 2024

@abhijeetk88

Please rebase this PR with trunk

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 6, 2025

This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

@github-actions github-actions bot added the stale Stale PRs label Jan 6, 2025
@abhijeetk88
Copy link
Contributor Author

I would like to revive this PR. @kamalcph will you be able to help with reviews. I can rebase the PR with trunk.

@github-actions github-actions bot removed the stale Stale PRs label Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kamalcph kamalcph kamalcph left review comments

Assignees
No one assigned
Labels
tiered-storage Related to the Tiered Storage feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@abhijeetk88 @kamalcph