Skip to content

Revert "SQL style null-on-empty semantics for Sum and Mean" - #9324

Merged
joseph-isaacs merged 1 commit into
developfrom
revert-9113-mk/unify-sums-v2
Aug 10, 2026
Merged

Revert "SQL style null-on-empty semantics for Sum and Mean"#9324
joseph-isaacs merged 1 commit into
developfrom
revert-9113-mk/unify-sums-v2

Conversation

@joseph-isaacs

Copy link
Copy Markdown
Contributor

Reverts #9113

@joseph-isaacs

Copy link
Copy Markdown
Contributor Author

We will revert this and figure out a forward compat way to fix this.

@joseph-isaacs joseph-isaacs added the changelog/fix A bug fix label Aug 10, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 10, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 1.51%

⚡ 11 improved benchmarks
❌ 8 regressed benchmarks
✅ 1921 untouched benchmarks
⏩ 89 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation list_sum_nullable_elements_medium 4.7 ms 11.6 ms -59.25%
Simulation list_sum_nullable_elements_large 464.5 ms 1,093.8 ms -57.54%
Simulation listview_sum_large 150.5 ms 231.7 ms -35.04%
Simulation fsl_sum_medium 1.2 ms 1.7 ms -33.2%
Simulation list_sum_large 185.3 ms 275.4 ms -32.7%
Simulation fsl_sum_large 96.5 ms 140 ms -31.04%
Simulation listview_sum_medium 1.6 ms 2.1 ms -23.35%
Simulation list_sum_medium 2 ms 2.5 ms -21.37%
Simulation sum_f64_all_valid 196.6 µs 88.1 µs ×2.2
Simulation sum_i32_nullable_all_valid 188.5 µs 95.7 µs +96.96%
Simulation fsl_sum_small 213.4 µs 135.5 µs +57.48%
Simulation sum_i32_clustered_nulls 232.6 µs 151 µs +54.05%
Simulation sum_f64_clustered_nulls 241.3 µs 158.9 µs +51.81%
Simulation listview_sum_small 224.3 µs 176.2 µs +27.28%
Simulation list_sum_small 317.6 µs 279.4 µs +13.7%
Simulation sparse_null_count 86.3 µs 76.3 µs +13.06%
Simulation search_index_in_range_chunked 6.4 ms 5.8 ms +10.83%
Simulation sparse_sum 571.5 µs 517.5 µs +10.43%
Simulation null_count_run_end[(10000, 1024, 0.5)] 84.8 µs 77 µs +10.03%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing revert-9113-mk/unify-sums-v2 (00d28df) with develop (4b37237)

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.57088% with 22 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.66%. Comparing base (4b37237) to head (00d28df).

Files with missing lines Patch % Lines
vortex-array/src/aggregate_fn/fns/sum/mod.rs 88.88% 12 Missing ⚠️
vortex-array/src/aggregate_fn/fns/sum/grouped.rs 86.66% 4 Missing ⚠️
vortex-array/src/scalar_fn/fns/list_sum.rs 87.87% 4 Missing ⚠️
...rtex-array/src/arrays/chunked/compute/aggregate.rs 85.71% 1 Missing ⚠️
vortex-array/src/expr/stats/mod.rs 50.00% 1 Missing ⚠️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@joseph-isaacs
joseph-isaacs merged commit 844d8c6 into develop Aug 10, 2026
215 of 223 checks passed
@joseph-isaacs
joseph-isaacs deleted the revert-9113-mk/unify-sums-v2 branch August 10, 2026 13:27
joseph-isaacs pushed a commit that referenced this pull request Aug 11, 2026
Zone maps exist to prune, and a zone sum prunes nothing: `ZoneMap::prune`
lowers predicates, and no predicate is answered by a per-zone total. The
aggregate was recorded because it always had been, and it then had to be
declared in `core2026.08.0` for writes to keep succeeding once the
aggregate filter was armed - a hostage situation, not a justification.

Its semantics are also unsettled: null-on-empty was changed in #9113 and
reverted in #9324 within the same week. That is not a stat to freeze into
an edition whose members are permanent.

Drop `Sum` from the default zoned aggregates and `vortex.sum` from
`core2026.08.0`, so the writer and the declaration stay consistent.
File-level statistics still record `Stat::Sum` via `PRUNING_STATS`; that
mechanism is separate and unaffected, as is the legacy-read path in
`zoned/schema.rs` that still understands a sum in older files.

Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>
joseph-isaacs pushed a commit that referenced this pull request Aug 11, 2026
Zone maps exist to prune, and a zone sum prunes nothing: `ZoneMap::prune`
lowers predicates, and no predicate is answered by a per-zone total. The
aggregate was recorded because it always had been, and it then had to be
declared in `core2026.08.0` for writes to keep succeeding once the
aggregate filter was armed - a hostage situation, not a justification.

Its semantics are also unsettled: null-on-empty was changed in #9113 and
reverted in #9324 within the same week. That is not a stat to freeze into
an edition whose members are permanent.

Drop `Sum` from the default zoned aggregates and `vortex.sum` from
`core2026.08.0`, so the writer and the declaration stay consistent.
File-level statistics still record `Stat::Sum` via `PRUNING_STATS`; that
mechanism is separate and unaffected, as is the legacy-read path in
`zoned/schema.rs` that still understands a sum in older files.

Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>
joseph-isaacs added a commit that referenced this pull request Aug 11, 2026
Zone maps exist to prune, and a zone sum prunes nothing: `ZoneMap::prune`
lowers predicates, and no predicate is answered by a per-zone total. The
aggregate was recorded because it always had been, and it then had to be
declared in `core2026.08.0` for writes to keep succeeding once the
aggregate filter was armed - a hostage situation, not a justification.

Its semantics are also unsettled: null-on-empty was changed in #9113 and
reverted in #9324 within the same week. That is not a stat to freeze into
an edition whose members are permanent.

Drop `Sum` from the default zoned aggregates and `vortex.sum` from
`core2026.08.0`, so the writer and the declaration stay consistent.
File-level statistics still record `Stat::Sum` via `PRUNING_STATS`; that
mechanism is separate and unaffected, as is the legacy-read path in
`zoned/schema.rs` that still understands a sum in older files.

Signed-off-by: "Joe Isaacs" <joe.isaacs@live.co.uk>
joseph-isaacs added a commit that referenced this pull request Aug 11, 2026
Zone maps exist to prune, and a zone sum prunes nothing: `ZoneMap::prune`
lowers predicates, and no predicate is answered by a per-zone total. The
aggregate was recorded because it always had been, and it then had to be
declared in `core2026.08.0` for writes to keep succeeding once the
aggregate filter was armed - a hostage situation, not a justification.

Its semantics are also unsettled: null-on-empty was changed in #9113 and
reverted in #9324 within the same week. That is not a stat to freeze into
an edition whose members are permanent.

Drop `Sum` from the default zoned aggregates and `vortex.sum` from
`core2026.08.0`, so the writer and the declaration stay consistent.
File-level statistics still record `Stat::Sum` via `PRUNING_STATS`; that
mechanism is separate and unaffected, as is the legacy-read path in
`zoned/schema.rs` that still understands a sum in older files.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/fix A bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants