Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: fix deadlock in new processor #1987

Merged
merged 8 commits into from
Jun 10, 2021
Merged

*: fix deadlock in new processor #1987

merged 8 commits into from
Jun 10, 2021

Conversation

liuzix
Copy link
Contributor

@liuzix liuzix commented Jun 8, 2021 •
edited
Loading

What problem does this PR solve?

  • When the new processor is enabled, deadlock can occur when the Unified Sorter is blocked due to the sink not being able to handle the large amount of data.

What is changed and how it works?

  • Redesigned some details in the Unified Sorter so that the outputCh never blocks AddEntry.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test

Side effects

  • Increased code complexity

Related changes

  • Need to cherry-pick to the release branch

Release note

  • Fix deadlock

@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 8, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 8, 2021 •
edited
Loading

/run-all-tests

@ti-chi-bot ti-chi-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jun 8, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 8, 2021

/run-all-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jun 8, 2021

/run-all-tests

1 similar comment
@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-all-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-all-tests

@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-integration-test

@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-all-tests

1 similar comment
@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-all-tests

@liuzix liuzix marked this pull request as ready for review June 9, 2021 09:08
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 9, 2021
@liuzix liuzix added status/ptal Could you please take a look? needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. labels Jun 9, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 9, 2021

/run-all-tests

@amyangfei amyangfei added this to the v5.1.0 milestone Jun 9, 2021
amyangfei
amyangfei reviewed Jun 9, 2021
View reviewed changes
cdc/puller/sorter/merger.go Show resolved Hide resolved
cdc/puller/sorter/merger.go Outdated Show resolved Hide resolved
cdc/puller/sorter/merger_test.go Outdated Show resolved Hide resolved
@amyangfei amyangfei added the release-blocker This issue blocks a release. Please solve it ASAP. label Jun 10, 2021
@liuzix
Copy link
Contributor Author

liuzix commented Jun 10, 2021

/run-all-tests

@codecov-commenter
Copy link

codecov-commenter commented Jun 10, 2021 •
edited
Loading

Codecov Report

Merging #1987 (eb87468) into master (b6d58ef) will decrease coverage by 0.0791%.
The diff coverage is 64.7016%.

@@               Coverage Diff                @@
##             master      #1987        +/-   ##
================================================
- Coverage   54.0780%   53.9988%   -0.0792%     
================================================
  Files           156        164         +8     
  Lines         16589      17380       +791     
================================================
+ Hits           8971       9385       +414     
- Misses         6667       6996       +329     
- Partials        951        999        +48

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 10, 2021
@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • amyangfei
  • overvenus

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 10, 2021
@overvenus
Copy link
Member

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: eb87468

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 10, 2021
@ti-chi-bot ti-chi-bot merged commit e9f799b into pingcap:master Jun 10, 2021
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 10, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2016.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jun 10, 2021
@liuzix @ti-chi-bot
This is an automated cherry-pick of pingcap#1987
810fd7a 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2017.

@ti-chi-bot ti-chi-bot mentioned this pull request Jun 10, 2021
Merged
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Jun 10, 2021
@liuzix @ti-chi-bot
This is an automated cherry-pick of pingcap#1987
71ac6e2 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #2018.

@ti-chi-bot ti-chi-bot mentioned this pull request Jun 10, 2021
Closed
ti-chi-bot added a commit that referenced this pull request Jun 10, 2021
@ti-chi-bot
*: fix deadlock in new processor (#1987) (#2016)
93e4053
liuzix added a commit to liuzix/ticdc that referenced this pull request Jun 24, 2021
@liuzix
*: fix deadlock in new processor (pingcap#1987)
ce32a13
@liuzix liuzix mentioned this pull request Jun 24, 2021
Merged
@amyangfei amyangfei mentioned this pull request Jun 25, 2021
Closed
ti-chi-bot added a commit that referenced this pull request Aug 11, 2021
@ti-chi-bot
*: fix deadlock in new processor (#1987) (#2017)
75e413e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amyangfei amyangfei amyangfei approved these changes

@overvenus overvenus overvenus approved these changes

@zier-one zier-one Awaiting requested review from zier-one

Assignees
No one assigned
Labels
needs-cherry-pick-release-4.0 Should cherry pick this PR to release-4.0 branch. needs-cherry-pick-release-5.0 Should cherry pick this PR to release-5.0 branch. needs-cherry-pick-release-5.1 Should cherry pick this PR to release-5.1 branch. release-blocker This issue blocks a release. Please solve it ASAP. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look?
Projects
None yet
Milestone
v5.1.0
Development

Successfully merging this pull request may close these issues.
5 participants
@liuzix @codecov-commenter @ti-chi-bot @overvenus @amyangfei