[BugFix] fix compatibility issue when collecting query statistics #29678
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
silverbullet233
changed the title
silverbullet233
force-pushed
the
fix_compitable_issue
branch
from
02bbbe5 to
6c8647e
Compare
silverbullet233
changed the title
Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
silverbullet233
force-pushed
the
fix_compitable_issue
branch
from
6c8647e to
4beefca
Compare
silverbullet233
changed the title
stdpain
previously approved these changes
Aug 22, 2023
satanson
previously approved these changes
Aug 22, 2023
liuyehcf
previously approved these changes
Aug 22, 2023
trueeyu
reviewed
Aug 22, 2023
Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
silverbullet233
dismissed stale reviews from liuyehcf, satanson, and stdpain
via
0b256db
|
SonarCloud Quality Gate failed.
|
trueeyu
approved these changes
Aug 22, 2023
[FE Incremental Coverage Report]😍 pass : 2 / 2 (100.00%) file detail
|
[BE Incremental Coverage Report]😍 pass : 11 / 13 (84.62%) file detail
|
|
@Mergifyio backport branch-2.4 |
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
mergify bot
pushed a commit
that referenced
this pull request
Aug 22, 2023
…9678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com> (cherry picked from commit be2f71b) # Conflicts: # be/src/exec/pipeline/stream_pipeline_driver.cpp # be/src/runtime/runtime_state.h # fe/fe-core/src/main/java/com/starrocks/qe/SessionVariable.java # gensrc/thrift/InternalService.thrift
mergify bot
pushed a commit
that referenced
this pull request
Aug 22, 2023
…9678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com> (cherry picked from commit be2f71b) # Conflicts: # be/src/exec/pipeline/pipeline_driver.cpp # be/src/exec/pipeline/stream_pipeline_driver.cpp # be/src/runtime/runtime_state.h # fe/fe-core/src/main/java/com/starrocks/qe/SessionVariable.java # gensrc/thrift/InternalService.thrift
silverbullet233
added a commit
to silverbullet233/starrocks
that referenced
this pull request
Aug 23, 2023
…arRocks#29678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
22 tasks
silverbullet233
added a commit
to silverbullet233/starrocks
that referenced
this pull request
Aug 23, 2023
…arRocks#29678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
22 tasks
silverbullet233
added a commit
to silverbullet233/starrocks
that referenced
this pull request
Aug 23, 2023
…arRocks#29678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
22 tasks
silverbullet233
added a commit
to silverbullet233/starrocks
that referenced
this pull request
Aug 23, 2023
…arRocks#29678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
22 tasks
Jay-ju
pushed a commit
to Jay-ju/starrocks
that referenced
this pull request
Sep 7, 2023
…arRocks#29678) Signed-off-by: silverbullet233 <3675229+silverbullet233@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this compatibility issue was introduce by #27779
Before #27779, pipeline engine didn't maintain scan statistics for each table,
QueryStatistics::_stats_itemswas always empty, so _stats_items needn't be merged inQueryStatistics::merge.But after that,
QueryStatistics::_stats_itemsis no longer empty. In the implementation of older version,QueryStatistics::_stats_itemsis a vector and the problem of deduplication is not considered, which may occupy a lot of memory inQueryStatistics::merge. If a cluster is in the process of grayscale upgrade, the memory usage of the old version BE may increase.A new session variable
enable_collect_table_level_scan_statsis introduced here to control whether to collect table level scan stats, which is false by default, so that the behavior of query stats will remain the same as before during the grayscale upgrade. After all nodes are upgraded, we can setenable_collect_table_level_scan_statsback to true to make fe scan metrics work normally.What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: