Feature: Use array.filter() instead of filter(array) compute#6146
Merged
connortsui20 merged 4 commits intodevelopfrom Jan 27, 2026
Merged
Feature: Use array.filter() instead of filter(array) compute#6146connortsui20 merged 4 commits intodevelopfrom
array.filter() instead of filter(array) compute#6146connortsui20 merged 4 commits intodevelopfrom
Conversation
Contributor
Author
|
Note: it seems like only the bitpacking early filter benchmark suffers, everything else is an improvement |
CodSpeed Performance ReportMerging this PR will degrade performance by 24.32%Comparing
|
connortsui20
force-pushed
the
ct/use-filter-array
branch
from
9b77c0e to
b33edc8
Compare
This was referenced Jan 26, 2026
joseph-isaacs
added
the
action/benchmark-sql
github-actions
bot
removed
the
action/benchmark-sql
Contributor
Benchmarks: TPC-H SF=1 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: FineWeb NVMeSummary
Detailed Results Table
|
Contributor
Benchmarks: TPC-DS SF=1 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=1 on S3Summary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=10 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: FineWeb S3Summary
Detailed Results Table
|
Contributor
Benchmarks: Statistical and Population GeneticsSummary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=10 on S3Summary
Detailed Results Table
|
connortsui20
force-pushed
the
ct/use-filter-array
branch
2 times, most recently
from
47f581b to
b887749
Compare
connortsui20
force-pushed
the
ct/use-filter-array
branch
from
b887749 to
ac91262
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Since in the new world there is no difference between setting up early and late filtering (because of reduce and reduce parent), we really just want to benchmark this in a dedicated benchmark for testing the speed of `.execute(ctx)`. remove pco test since its not measuring anything anymore remove runend filter bench since its not measuring anything anymore Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/use-filter-array
branch
2 times, most recently
from
ac91262 to
48d1a81
Compare
joseph-isaacs
added
the
action/benchmark
github-actions
bot
removed
the
action/benchmark
Contributor
Benchmarks: Random AccessSummary
|
connortsui20
force-pushed
the
ct/use-filter-array
branch
from
581a7e9 to
703fcba
Compare
joseph-isaacs
approved these changes
Jan 27, 2026
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
force-pushed
the
ct/use-filter-array
branch
from
703fcba to
4940e77
Compare
Contributor
Author
|
Whoops looks like this broke the compression throughput and clickbench benchmarks, unfortunately theres no warning |
connortsui20
added a commit
that referenced
this pull request
Jan 27, 2026
…6147) This is based on top of #6146, that needs to merge first. Simply changes the syntax so that we use the new `FilterArray`, so this is purely a cosmetic change after #6146. Note that I only made the change when this gets called inside one of the old compute functions, and the code for all of the old filter compute are going to be moved around soon so this is fine. There are just a few places (around 8) where we call the old `filter` _outside_ of the old compute kernels. I've left those as is so it's easy to search for them later. We still have to figure out what to do with them. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
connortsui20
added a commit
that referenced
this pull request
Jan 28, 2026
This is a mostly cosmetic change to remove the old filter compute kernels from the `FilterArray` implementation, placing the code in `vortex-array/src/arrays/filter/execute` instead of dispersed everywhere. The non-cosmetic changes are trivial, just making some things cleaner to understand. This is based on top of #6147, which is also based on top of #6146 --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
danking
pushed a commit
that referenced
this pull request
Feb 6, 2026
I also removed some benchmarks (3) that now no longer benchmark anything useful. We'll soon want a way to benchmark the execution of arbitrary arrays, and they should all share similar benchmarking logic. In a follow up PR (#6147) I will remove every occurrence of the old `filter` compute function! mwa ha ha --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
danking
pushed a commit
that referenced
this pull request
Feb 6, 2026
…6147) This is based on top of #6146, that needs to merge first. Simply changes the syntax so that we use the new `FilterArray`, so this is purely a cosmetic change after #6146. Note that I only made the change when this gets called inside one of the old compute functions, and the code for all of the old filter compute are going to be moved around soon so this is fine. There are just a few places (around 8) where we call the old `filter` _outside_ of the old compute kernels. I've left those as is so it's easy to search for them later. We still have to figure out what to do with them. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
danking
pushed a commit
that referenced
this pull request
Feb 6, 2026
This is a mostly cosmetic change to remove the old filter compute kernels from the `FilterArray` implementation, placing the code in `vortex-array/src/arrays/filter/execute` instead of dispersed everywhere. The non-cosmetic changes are trivial, just making some things cleaner to understand. This is based on top of #6147, which is also based on top of #6146 --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I also removed some benchmarks (3) that now no longer benchmark anything useful. We'll soon want a way to benchmark the execution of arbitrary arrays, and they should all share similar benchmarking logic.
In a follow up PR (#6147) I will remove every occurrence of the old
filtercompute function! mwa ha ha