Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement] Refactor lake compaction task cancel checker in BE to ensure it can stop asap #54832

Merged
merged 3 commits into from
Feb 7, 2025
Merged

[Enhancement] Refactor lake compaction task cancel checker in BE to ensure it can stop asap #54832

merged 3 commits into from
Feb 7, 2025

Conversation

wxl24life
Copy link
Contributor

@wxl24life wxl24life commented Jan 8, 2025 •
edited
Loading

Why I'm doing:

What I'm doing:

#54071

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.4
    • 3.3
    • 3.2
    • 3.1
    • 3.0

@wxl24life wxl24life requested a review from a team as a code owner January 8, 2025 05:20
@wxl24life wxl24life force-pushed the xiaolong/github-compaction-abort-checker-refactor branch 4 times, most recently from 47a018f to 0265b28 Compare January 10, 2025 03:52
@wxl24life wxl24life requested a review from a team as a code owner January 10, 2025 03:52
@wxl24life wxl24life changed the title [BugFix] Allow lake compaction task stop-fast while its transaction in fe has aborted [Enhancement] Refactor lake compaction task cancel checker in BE to ensure it can stop asap Jan 10, 2025
@wxl24life wxl24life force-pushed the xiaolong/github-compaction-abort-checker-refactor branch 2 times, most recently from 2f80404 to bc62d54 Compare January 10, 2025 06:11
@wxl24life wxl24life force-pushed the xiaolong/github-compaction-abort-checker-refactor branch 9 times, most recently from 09302e4 to 829b048 Compare January 17, 2025 09:08
@wxl24life
refactor cancel checker for lake compactions in be
5515c73 
Signed-off-by: drake_wang <wxl250059@alibaba-inc.com>
@wxl24life wxl24life force-pushed the xiaolong/github-compaction-abort-checker-refactor branch 2 times, most recently from 978a386 to 08ee90f Compare January 20, 2025 01:32
@wxl24life
add ut
d89c5d9 
Signed-off-by: drake_wang <wxl250059@alibaba-inc.com>
@wxl24life wxl24life force-pushed the xiaolong/github-compaction-abort-checker-refactor branch from 08ee90f to d89c5d9 Compare January 20, 2025 04:55
@kevincai
Copy link
Contributor

This PR just ensure that the txn status can be checked even if the first tablet is done, but not necessary to fix the related issue.

@wxl24life
Copy link
Contributor Author

This PR just ensure that the txn status can be checked even if the first tablet is done, but not necessary to fix the related issue.

The bug issue is more relevant to #54908 。 This pr is an optimization to make txn checking can continue even it's not the first tablet

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

❌ fail : 16 / 37 (43.24%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/storage/lake/compaction_scheduler.cpp 15 36 41.67% [149, 159, 160, 161, 162, 163, 164, 166, 167, 169, 170, 171, 173, 174, 175, 176, 177, 181, 183, 184, 185]
🔵 be/src/storage/lake/compaction_scheduler.h 1 1 100.00% []

@andyziye andyziye enabled auto-merge (squash) February 7, 2025 01:29
@andyziye andyziye disabled auto-merge February 7, 2025 01:29
@andyziye andyziye merged commit 2fd71c7 into StarRocks:main Feb 7, 2025
55 of 56 checks passed
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

@Mergifyio backport branch-3.4

@github-actions github-actions bot removed the 3.4 label Feb 7, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

@Mergifyio backport branch-3.3

@github-actions github-actions bot removed the 3.3 label Feb 7, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025

@Mergifyio backport branch-3.2

@github-actions github-actions bot removed the 3.2 label Feb 7, 2025
@mergify Mergify
Copy link
Contributor

mergify bot commented Feb 7, 2025 •
edited
Loading

backport branch-3.4

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Feb 7, 2025 •
edited
Loading

backport branch-3.3

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Feb 7, 2025 •
edited
Loading

backport branch-3.2

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Feb 7, 2025
@wxl24life @mergify
[Enhancement] Refactor lake compaction task cancel checker in BE to e…
418ce0a 
…nsure it can stop asap (#54832)

(cherry picked from commit 2fd71c7)
@mergify mergify bot mentioned this pull request Feb 7, 2025
Merged
18 tasks
mergify bot pushed a commit that referenced this pull request Feb 7, 2025
@wxl24life @mergify
[Enhancement] Refactor lake compaction task cancel checker in BE to e…
4f10dea 
…nsure it can stop asap (#54832)

(cherry picked from commit 2fd71c7)
mergify bot pushed a commit that referenced this pull request Feb 7, 2025
@wxl24life @mergify
[Enhancement] Refactor lake compaction task cancel checker in BE to e…
b5f1c9b 
…nsure it can stop asap (#54832)

(cherry picked from commit 2fd71c7)

# Conflicts:
#	be/src/storage/lake/compaction_scheduler.cpp
#	be/src/storage/lake/compaction_task_context.h
#	be/test/storage/lake/compaction_scheduler_test.cpp
#	be/test/storage/lake/lake_primary_key_consistency_test.cpp
#	be/test/storage/lake/primary_key_compaction_task_test.cpp
#	be/test/storage/lake/primary_key_publish_test.cpp
This was referenced Feb 7, 2025
Merged
Closed
@wxl24life wxl24life deleted the xiaolong/github-compaction-abort-checker-refactor branch February 7, 2025 01:33
wanpengfei-git pushed a commit that referenced this pull request Feb 7, 2025
@mergify @wxl24life
[Enhancement] Refactor lake compaction task cancel checker in BE to e…
6dcb0f8 
…nsure it can stop asap (backport #54832) (#55633)

Co-authored-by: Drake Wang <wxl24life@gmail.com>
wanpengfei-git pushed a commit that referenced this pull request Feb 7, 2025
@mergify @wxl24life
[Enhancement] Refactor lake compaction task cancel checker in BE to e…
1908ea2 
…nsure it can stop asap (backport #54832) (#55632)

Co-authored-by: Drake Wang <wxl24life@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kevincai kevincai kevincai approved these changes

@caneGuy caneGuy caneGuy approved these changes

@starrocks-xupeng starrocks-xupeng starrocks-xupeng approved these changes

Assignees

@wxl24life wxl24life

Labels
3.3-merged 3.4-merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@wxl24life @kevincai @caneGuy @starrocks-xupeng @andyziye