Skip to content

Conversation

@gbbafna
Copy link
Collaborator

@gbbafna gbbafna commented Jul 1, 2022

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna gbbafna@amazon.com

Description

Correctly updating follower_stats at two places :

  1. At start of replication , reading it from shard itself.
  2. After every write, again updating it from shard itself.

Issues Resolved

431

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@gbbafna gbbafna requested a review from a team July 1, 2022 10:41
@gbbafna gbbafna changed the base branch from lag_stats to main July 1, 2022 11:02
@gbbafna gbbafna changed the title Fixing the followerCheckpoint lag in stats Fixing the follower checkpoint updation in follower stats API Jul 1, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Copy link
Member

@ankitkala ankitkala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gbbafna gbbafna merged commit 6130d6f into opensearch-project:main Jul 4, 2022
@opensearch-trigger-bot
Copy link
Contributor

The backport to 1.1 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.1 1.1
# Navigate to the new working tree
cd .worktrees/backport-1.1
# Create a new branch
git switch --create backport/backport-438-to-1.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 6130d6feda5075a695bc384b2702a76d4db383a6
# Push it to GitHub
git push --set-upstream origin backport/backport-438-to-1.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.1

Then, create a pull request where the base branch is 1.1 and the compare/head branch is backport/backport-438-to-1.1.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 4, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 4, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 4, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 4, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)
gbbafna added a commit that referenced this pull request Jul 7, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)

Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
gbbafna added a commit that referenced this pull request Jul 7, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)

Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Jul 8, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit that referenced this pull request Jul 8, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Jul 8, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Aug 23, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Aug 23, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Aug 23, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit to gbbafna/cross-cluster-replication that referenced this pull request Aug 23, 2022
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit that referenced this pull request Aug 23, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit that referenced this pull request Aug 23, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
gbbafna added a commit that referenced this pull request Aug 26, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
(cherry picked from commit 6130d6f)

Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
gbbafna added a commit that referenced this pull request Aug 26, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
saikaranam-amazon pushed a commit that referenced this pull request Dec 27, 2022
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit to ankitkala/cross-cluster-replication that referenced this pull request Jun 2, 2023
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit that referenced this pull request Jun 2, 2023
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit to ankitkala/cross-cluster-replication that referenced this pull request Jun 2, 2023
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit to ankitkala/cross-cluster-replication that referenced this pull request Jun 2, 2023
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit to ankitkala/cross-cluster-replication that referenced this pull request Jun 2, 2023
…oject#438)

Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit that referenced this pull request Jun 2, 2023
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit that referenced this pull request Jun 2, 2023
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
ankitkala pushed a commit that referenced this pull request Jun 2, 2023
Summary : We need to update followerCheckpoint after writing to the follower index.
Currently, we are not waiting for the writes and updating it with soon-to-be stale values

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants