-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sink(ticdc): fix data loss #9618
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add comments about the reason for the data loss.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hicqu, nongfushanquan The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
/retest |
In response to a cherrypick label: new pull request created to branch |
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
In response to a cherrypick label: new pull request created to branch |
What problem does this PR solve?
Issue Number: close #9592
What is changed and how it works?
Before this pr, asyncCloseAndClearTableSink always returns true, no matter whether the underlying table sink successfully closed. During two-phase scheduling, this error can cause the new node's table sink to write data downstream before the old node's table sink actually exits. And this problem is more likely to be repeated in batch write scenarios (such as storage sink).
Check List
Tests
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note