Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

backport limited stream agg #34368

Merged
merged 4 commits into from
Nov 7, 2023
Merged

backport limited stream agg #34368

merged 4 commits into from
Nov 7, 2023

Conversation

stdpain
Copy link
Contributor

@stdpain stdpain commented Nov 6, 2023

stdpain and others added 4 commits November 2, 2023 19:11
@stdpain
[Enhancement] support limited memory stream aggregate (StarRocks#28402)
0405375 
Signed-off-by: stdpain <34912776+stdpain@users.noreply.github.com>

Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain
[Enhancement] reduce chunk buffer size in streaming aggregate (StarRo…
de66d75 
…cks#28962)

Signed-off-by: stdpain <drfeng08@gmail.com>

Signed-off-by: stdpain <34912776+stdpain@users.noreply.github.com>
@stdpain
[BugFix] Fix mem limited streaming agg may hang (StarRocks#30228)
63d82c8 
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain
Fix conflict
01ac159 
Signed-off-by: stdpain <drfeng08@gmail.com>
@mergify mergify bot assigned stdpain Nov 6, 2023
@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Nov 6, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

warning The version of Java (11.0.21) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 6, 2023

[FE Incremental Coverage Report]

❌ fail : 1 / 2 (50.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/planner/AggregationNode.java 1 2 50.00% [254]

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 6, 2023

[BE Incremental Coverage Report]

❌ fail : 12 / 125 (09.60%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 src/exec/pipeline/aggregate/aggregate_streaming_source_operator.cpp 0 7 00.00% [31, 32, 50, 87, 88, 89, 92]
🔵 src/exec/pipeline/exchange/local_exchange.h 0 6 00.00% [112, 183, 224, 236, 251, 268]
🔵 src/exec/pipeline/exchange/local_exchange.cpp 0 2 00.00% [121, 128]
🔵 src/exec/pipeline/aggregate/aggregate_distinct_streaming_source_operator.cpp 0 8 00.00% [29, 30, 63, 69, 78, 79, 80, 83]
🔵 src/exec/pipeline/aggregate/aggregate_streaming_sink_operator.h 0 1 00.00% [38]
🔵 src/exec/pipeline/aggregate/sorted_aggregate_streaming_sink_operator.cpp 0 1 00.00% [43]
🔵 src/exec/aggregator.h 0 7 00.00% [288, 292, 293, 294, 296, 300, 371]
🔵 src/exec/aggregate/agg_profile.h 0 2 00.00% [38, 39]
🔵 src/exec/pipeline/pipeline_builder.cpp 0 12 00.00% [42, 43, 101, 102, 137, 138, 163, 164, 165, 224, 225, 271]
🔵 src/exec/pipeline/exchange/local_exchange_source_operator.h 0 1 00.00% [189]
🔵 src/exec/pipeline/aggregate/aggregate_distinct_streaming_sink_operator.cpp 0 14 00.00% [26, 27, 60, 61, 63, 78, 79, 105, 107, 108, 109, 110, 112, 114]
🔵 src/exec/pipeline/aggregate/aggregate_streaming_sink_operator.cpp 0 14 00.00% [29, 30, 63, 64, 66, 82, 83, 299, 300, 301, 302, 303, 305, 307]
🔵 src/exec/pipeline/aggregate/aggregate_distinct_streaming_sink_operator.h 0 1 00.00% [38]
🔵 src/exec/aggregate/agg_hash_variant.cpp 0 3 00.00% [242, 243, 244]
🔵 src/exec/chunk_buffer_memory_manager.h 3 15 20.00% [28, 30, 31, 32, 33, 35, 38, 48, 54, 56, 57, 58]
🔵 src/exec/aggregator.cpp 4 19 21.05% [490, 625, 627, 628, 630, 635, 636, 639, 640, 646, 647, 650, 653, 654, 658]
🔵 src/exec/pipeline/exchange/local_exchange_source_operator.cpp 4 11 36.36% [49, 90, 175, 182, 203, 204, 211]
🔵 src/exec/stream/aggregate/stream_aggregator.cpp 1 1 100.00% []

@stdpain stdpain merged commit cf16a4e into StarRocks:branch-3.1 Nov 7, 2023
@zombee0
Copy link
Contributor

zombee0 commented Nov 16, 2023

https://github.com/Mergifyio backport branch-3.1-dla-optimize

@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 16, 2023 •
edited
Loading

backport branch-3.1-dla-optimize

✅ Backports have been created

@mergify mergify bot mentioned this pull request Nov 16, 2023
Merged
Youngwb pushed a commit that referenced this pull request Nov 16, 2023
@mergify @stdpain
backport limited stream agg (backport #34368) (#35207)
7fc677c 
Co-authored-by: stdpain <34912776+stdpain@users.noreply.github.com>
Co-authored-by: stdpain <drfeng08@gmail.com>
GavinMar pushed a commit to GavinMar/starrocks that referenced this pull request Jan 3, 2024
@mergify @stdpain @GavinMar
backport limited stream agg (backport StarRocks#34368) (StarRocks#35207)
78caa2f 
Co-authored-by: stdpain <34912776+stdpain@users.noreply.github.com>
Co-authored-by: stdpain <drfeng08@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@silverbullet233 silverbullet233 silverbullet233 approved these changes

Assignees

@stdpain stdpain

Labels
title needs [type]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@stdpain @zombee0 @silverbullet233