vortex-btrblocks: write listviews when elements expand > 50%#6393
Merged
robert3005 merged 1 commit intodevelopfrom Feb 12, 2026
Merged
vortex-btrblocks: write listviews when elements expand > 50%#6393robert3005 merged 1 commit intodevelopfrom
robert3005 merged 1 commit intodevelopfrom
Conversation
Contributor
Author
|
I haven't given much though to the heuristic and just picked 50% so open to discussion here. We could also introduce this slowly via a write option maybe? The motivation for this is that we're starting to do the work of moving our lists to ListViews and would like to keep the "dict-like encoding" + zero copy back to list view. |
asubiotto
force-pushed
the
asubiotto/listview
branch
from
36651d7 to
b44dd33
Compare
Merging this PR will not alter performance
Comparing 
Footnotes
|
asubiotto
force-pushed
the
asubiotto/listview
branch
from
b44dd33 to
25cd045
Compare
joseph-isaacs
added
the
action/benchmark
Deploying vortex-bench with
|
| Latest commit: |
dd0c393
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://ef08e8ed.vortex-93b.pages.dev |
| Branch Preview URL: | https://asubiotto-listview.vortex-93b.pages.dev |
github-actions
bot
removed
the
action/benchmark
Contributor
|
lets runn benchmarks and we can merge if its all good |
Contributor
connortsui20
left a comment
connortsui20
left a comment
There was a problem hiding this comment.
Generally looks good to me! Left a few nits
Contributor
Benchmarks: Random AccessSummary
Detailed Results Table
|
Contributor
Benchmarks: PolarSignals ProfilingSummary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=1 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: FineWeb NVMeSummary
Detailed Results Table
|
Contributor
|
Is the TPC-H regression expected? It seems like quite a lot of queries slow down on the file-compressed benchmark. To be honest, I don't really believe it at all |
Contributor
Benchmarks: TPC-H SF=1 on S3Summary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=10 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: FineWeb S3Summary
Detailed Results Table
|
Contributor
Benchmarks: TPC-DS SF=1 on NVMESummary
Detailed Results Table
|
Contributor
Benchmarks: Statistical and Population GeneticsSummary
Detailed Results Table
|
Contributor
Benchmarks: TPC-H SF=10 on S3Summary
Detailed Results Table
|
Contributor
Benchmarks: Clickbench on NVMESummary
Detailed Results Table
|
Contributor
Author
|
No rush to merge this (I would actually probably prefer to not have this in the release in case there are unexpected regressions). I can look into the benchmark issues. |
Contributor
|
@asubiotto I think the benchmark issues were just bad runs, so no need to worry |
asubiotto
force-pushed
the
asubiotto/listview
branch
from
dd0c393 to
f717fb9
Compare
github-actions
bot
removed
the
action/benchmark
Contributor
Author
|
Updated and addressed the latest comments. Rerunning the benchmarks |
asubiotto
force-pushed
the
asubiotto/listview
branch
from
f717fb9 to
91292c0
Compare
asubiotto
added
action/benchmark
asubiotto
force-pushed
the
asubiotto/listview
branch
from
91292c0 to
c8034df
Compare
asubiotto
added
action/benchmark
github-actions
bot
removed
the
action/benchmark
Contributor
Benchmarks: CompressionSummary
Detailed Results Table
|
Contributor
Author
|
Not sure what happened to the bench comments |
Contributor
|
the comments get updated, last edit seems to be from 1h ago which aligns with the time when labels were added last? |
Contributor
Author
|
Ah, you're right. For some reason I assumed they were new comments every time. |
Previously, listview arrays were always written as lists. This involves materializing the views (i.e. expanding the elements and reorganizing the offsets). This commit chooses to write listviews on compression if the elements would expand > 50% when converting to lists to save on CPU cost and storage space. ZCTL listviews and listviews with low duplication are still encoded as lists, which saves writing the sizes. Signed-off-by: Alfonso Subiotto Marques <alfonso.subiotto@polarsignals.com>
asubiotto
force-pushed
the
asubiotto/listview
branch
from
c8034df to
fce02b8
Compare
robert3005
approved these changes
Feb 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, listview arrays were always written as lists. This involves materializing the views (i.e. expanding the elements and reorganizing the offsets).
This commit chooses to write listviews on compression if the elements would expand > 50% when converting to lists to save on CPU cost and storage space. ZCTL listviews and listviews with low duplication are still encoded as lists, which saves writing the sizes.