-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TiCDC support checking if data is entirely replicated to Downstream (#10133) #10336
TiCDC support checking if data is entirely replicated to Downstream (#10133) #10336
Conversation
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
/cherry-pick-invite |
@hongyunyan Please accept the invitation then you can push to the cherry-pick pull requests. |
/test verify |
/retest |
/run-pull-cdc-integration-mysql-test |
/retest |
/retest |
/test cdc-integration-mysql-test |
/retest |
1 similar comment
/retest |
/run-pull-cdc-integration-mysql-test |
SinkCheckpointTs: model.JSONTime(oracle.GetTimeFromTS(status.CheckpointTs)), | ||
PullerResolvedTs: model.JSONTime(oracle.GetTimeFromTS(status.PullerResolvedTs)), | ||
LastSyncedTs: model.JSONTime(oracle.GetTimeFromTS(status.LastSyncedTs)), | ||
NowTs: model.JSONTime(time.Unix(0, 0)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not use the local time here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because now_ts should be the time from pd, not locally. Here we can't get pd time, so we don't provider other local time, which may be misleading.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: CharlesCheung96 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/retest |
1 similar comment
/retest |
/retest |
…cherry-pick-10133-to-release-6.5
/test pull-cdc-integration-mysql-test |
@hongyunyan: The specified target(s) for
Use
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/test cdc-integration-mysql-test |
2 similar comments
/test cdc-integration-mysql-test |
/test cdc-integration-mysql-test |
/retest |
1 similar comment
/retest |
/rebuild |
/build |
/test verify |
3 similar comments
/test verify |
/test verify |
/test verify |
/test verify |
1 similar comment
/test verify |
377734a
to
b33abb5
Compare
b33abb5
to
d93a4ca
Compare
/retest |
/test verify |
/test cdc-integration-kafka-test |
1 similar comment
/test cdc-integration-kafka-test |
This is an automated cherry-pick of #10133
What problem does this PR solve?
Issue Number: close #10289
What is changed and how it works?
lastSyncedTs
to record the latest commitTs received by sink modulelastSyncedTs
from tables and do aggregate at changefeed level to obtain a changefeed levellastSyncedTs
api/v2/changefeeds/<changefeed-id>/synced
to get the changefeed synced status.Check List
Tests
Unit test
Integration test
For integration test, I add the test for both normal cases and failure cases, such as tikv failed etc.
Manual test (add detailed scripts or steps below)
The http api looks like it:
For performance test, we do pressure test on master and ours cdc and get the same sink output, which shows the PR have no impact on the performance.
No code
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note