Skip to content

Add the RowFn scalar function framework - #9353

Open
connortsui20 wants to merge 17 commits into
developfrom
ct/row-fn-framework
Open

Add the RowFn scalar function framework#9353
connortsui20 wants to merge 17 commits into
developfrom
ct/row-fn-framework

Conversation

@connortsui20

@connortsui20 connortsui20 commented Aug 11, 2026

Copy link
Copy Markdown
Member

Rationale for this change

Adds shared batch execution for strict scalar functions written as typed row kernels. The framework centralizes input validation, decoding, constants, validity, output construction, and deferred errors.

Progress towards:

For reviewers, read bottom to top (at least on the GitHub UI), as most of the important definitions are in those bottom files.

What changes are included in this PR?

Defines typed row dispatch, prepared inputs, owned outputs, output sinks, validity-aware batch strategies, and encoding-aware reductions. All supported tuple arities get indexed traversal, while unary and binary kernels keep specialized sources. Boundary checks reject invalid lengths, dtypes, and nulls before a kernel output escapes.

This PR deliberately has no production adopter. A paired pre-RowFn benchmark would therefore be meaningless; #9345 is the first performance gate. Focused executor controls and the reproducible comparison runner are isolated in #9351.

What APIs are changed? Are there any user-facing changes?

Adds the public RowFn, RowVisitor, InputElement, OutputElement, and OutputSink<Options> APIs. Every RowFn automatically implements ScalarFnVTable; existing vtables can delegate through row_fn_return_dtype and execute_rows. InputElement and OutputSink are unsafe to implement because their contracts support unchecked reads and uninitialized output.

@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 11.36%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
✅ 1959 untouched benchmarks
⏩ 89 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation slice_primitive_tight_loop[10000] 473.8 µs 423.7 µs +11.82%
Simulation slice_dict_tight_loop[10000] 770.1 µs 691 µs +11.45%
Simulation compress_fsst[(1000, 64, 4)] 1,006.5 µs 908.3 µs +10.81%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/row-fn-framework (58c9e85) with develop (ca7f626)

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@connortsui20
connortsui20 marked this pull request as ready for review August 11, 2026 16:05
@connortsui20
connortsui20 changed the base branch from develop to ct/lane-kernel-sources August 11, 2026 16:07
@connortsui20
connortsui20 marked this pull request as draft August 11, 2026 16:24
Base automatically changed from ct/lane-kernel-sources to develop August 11, 2026 17:12
Introduce typed row execution with self-contained kernel arguments and explicit execution contracts. Validate decoded lengths once per batch while preserving specialized mixed-constant loops.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Cover RowFn executor overhead and strict validity policies with focused microbenchmarks.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20 connortsui20 added the changelog/feature A new feature label Aug 11, 2026
@connortsui20
connortsui20 marked this pull request as ready for review August 11, 2026 18:48
@connortsui20
connortsui20 requested a review from gatesn August 11, 2026 18:59
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant