Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE][RUST] Support blockId filter when getting data from memory #1213

Closed
2 of 3 tasks
zuston opened this issue Sep 24, 2023 · 1 comment · Fixed by #1311
Closed
2 of 3 tasks

[FEATURE][RUST] Support blockId filter when getting data from memory #1213

zuston opened this issue Sep 24, 2023 · 1 comment · Fixed by #1311
Assignees
@wenlongbrother
Labels
good first issue Good for newcomers rust

Comments

@zuston
Copy link
Member

zuston commented Sep 24, 2023 •
edited
Loading

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the feature

Support blockId filter when getting data from memory, this will improve the performance for Spark AQE

Motivation

No response

Describe the solution

No response

Additional context

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
@zuston zuston added good first issue Good for newcomers rust labels Sep 24, 2023
@wenlongbrother
Copy link
Contributor

Hi @zuston I am interested in this task, can you assign it to me?

@zuston zuston mentioned this issue Nov 22, 2023
Merged
zuston pushed a commit that referenced this issue Nov 22, 2023
@wenlongbrother
[#1213] feat(rust): Support block filter by taskId when getting memor…
4edd60f 
…y data (#1311)

### What changes were proposed in this pull request?

Support block filter by taskId when getting memory data

### Why are the changes needed?

In AQE, after executing the sub-QueryStages, collect the shuffle data size, So if we can filter block, 
it will improve the performance of AQE.

Fix: #1213

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

1. UTs

---------

Co-authored-by: 蒋文龙 <jiangwenlong@192.168.1.7>
Co-authored-by: 蒋文龙 <jiangwenlong@192.168.1.6>
zuston pushed a commit to zuston/incubator-uniffle that referenced this issue Jan 18, 2024
@wenlongbrother @zuston
[apache#1213] feat(rust): Support block filter by taskId when getting…
9853f5a 
… memory data (apache#1311)

### What changes were proposed in this pull request?

Support block filter by taskId when getting memory data

### Why are the changes needed?

In AQE, after executing the sub-QueryStages, collect the shuffle data size, So if we can filter block, 
it will improve the performance of AQE.

Fix: apache#1213

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

1. UTs

---------

Co-authored-by: 蒋文龙 <jiangwenlong@192.168.1.7>
Co-authored-by: 蒋文龙 <jiangwenlong@192.168.1.6>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wenlongbrother wenlongbrother

Labels
good first issue Good for newcomers rust
Projects
None yet
Milestone
No milestone
2 participants
@zuston @wenlongbrother