Skip to content

Avoid reading fields for empty projections - #9365

Draft
joseph-isaacs wants to merge 1 commit into
ji/vx-plan-cifrom
ji/plan-count-star
Draft

Avoid reading fields for empty projections#9365
joseph-isaacs wants to merge 1 commit into
ji/vx-plan-cifrom
ji/plan-count-star

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • reduce root-independent expressions through RowIdx, allowing an empty pack() projection to reach struct-field pruning
  • treat a childless Pack as a row-count-preserving scan leaf
  • cover both the optimized plan shape and an end-to-end scan that returns every row while issuing zero segment requests

Root cause

COUNT(*) reaches the V2 scanner as an empty pack() projection. Because it references neither the generated row index nor child values, partition_bound returns zero partitions. ExpressionRowIdxRule previously handled only one or two partitions, so the expression stayed above RowIdx and the full struct source remained materialized.

After pruning all fields, the resulting childless Pack also needs to contribute the scan-range endpoint; otherwise natural split planning schedules no task and loses the row count.

Stack

This is the final top-of-stack PR, based on #9341. During this update the complete published chain was restacked onto the current #9142 head and verified after publication:

#9142#9196#9166#9187#9240#9244#9254#9341 → this PR

The benchmark-only full-stack sibling #9342 was also rebuilt on current develop and now contains this change.

Checks

  • cargo +nightly fmt --all -- --check
  • cargo test -p vortex-layout -p vortex-scan-v2 -p vortex-datafusion
  • cargo clippy -p vortex-layout -p vortex-scan-v2 -p vortex-datafusion --all-targets --all-features -- -D warnings
  • cargo clippy --all-targets --all-features -- -D warnings
  • git diff --check

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 19.44%

⚡ 1 improved benchmark
✅ 1961 untouched benchmarks
⏩ 89 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation cold_misaligned[(64, 256)] 5.2 ms 4.4 ms +19.44%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ji/plan-count-star (a836fa0) with ji/vx-plan-ci (cecbc8f)

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant