-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
puller: ignore fallen back resolved ts #2077
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
/merge |
This pull request has been accepted and is ready to merge. Commit hash: d893599
|
Codecov Report
@@ Coverage Diff @@
## master #2077 +/- ##
=================================================
+ Coverage 37.8006% 53.8711% +16.0705%
=================================================
Files 110 164 +54
Lines 11394 17527 +6133
=================================================
+ Hits 4307 9442 +5135
- Misses 6662 7080 +418
- Partials 425 1005 +580 |
In response to a cherrypick label: new pull request created: #2086. |
In response to a cherrypick label: new pull request created: #2087. |
In response to a cherrypick label: new pull request created: #2088. |
What problem does this PR solve?
This is an addition fix after #2038
In #2038, kv client tries to send
resolved ts
to puller after a region information is loaded from PD and dispatched in kv client. However, in extreme scenario,BatchLoadRegionsWithKeyRange
could still miss some regions of intermediate states.What is changed and how it works?
In the #2038 scenario, the fallen back resolved doesn't make any sense, because the
resovled-ts
in puller only guarantees no late data, but doesn't guarantee no repeated data. We have the same behavior in kv client when processing the resovled ts fallen back from TiKV, ref: https://github.com/pingcap/ticdc/blob/43c40f9d368a5f85beaf05080c3f11135648faa5/cdc/kv/client.go#L1473-L1480commit-ts
is smaller thanresolved-ts
.commit-ts
is smaller thanresolved-ts
.TODO, we should add more tests in span frontier module, and make sure the correctness. Some possible tasks:
Check List
Tests
Release note