Skip to content

go: cut Rows' remaining per-call allocations (#456) - #464

Merged
EricAndrechek merged 4 commits into
v1from
v1-go-rows-alloc-456
Oct 6, 2026
Merged

EricAndrechek merged 4 commits into
v1from
v1-go-rows-alloc-456

Conversation

@EricAndrechek

@EricAndrechek EricAndrechek commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

refs #456

#462 made the decoder allocation-light; this cuts what a full Schema.Rows call still allocated around it. Go only; scope is go/chtypes/**, go/internal/abi1/take.go and go/CHANGELOG.md. No exported name changes, no generated file edited, stock encoding/json only.

Measured (darwin-arm64, Go 1.27.1, the reporter's call shape: 100 NDJSON rows, tenant = {p:String} filter with params, WithExport(JSONCompactEachRow), WithDocFlags(0), a fresh four-key settings map each call)

per call before (#462) after
Schema.Rows 412 allocs, 75.2 KB 76 allocs, 45.4 KB
same, settings map built once by the caller 409, 74.9 KB 74, 45.1 KB

Allocation sites, from -memprofile -memprofilerate 1, alloc_space / alloc_objects, per call:

site before after notes
rows result slice ([]RowResult) 23.4 KB / 1 23.4 KB / 1 returned to the caller; only an API change shrinks it
document copy out of the library 16.6 KB / 1 0 / 0 pooled buffer (TakeInto)
export payload copy 3.5 KB / 1 3.5 KB / 1 returned to the caller, stays a fresh Go slice
json.Decoder buffer growth (stock) 11.9 KB / 13 8.1 KB / 7 the stock decoder's growth policy; the 2.4 KB row_spans list is no longer read as one value
per-row Verdict pointers 1.6 KB / 100 2 KB / 4 shared chunks of 32
PartitionID pointer (allocated even when the row has none) 1.6 KB / 100 0 / 0 one chunk when a row has one
per-row InputSpan pointers 2 KB / 4 2 KB / 4 already chunked
row_spans decode ([]*spanDoc, reflect, growslice, spansOf) 7.2 KB / 117 1.9 KB / 3 streamed element by element into the []Span the result carries
decoder locals (top-level fields) 0.4 KB / 17 0.4 KB / 2 one struct
row decoder, sub-decoder 0.5 KB / 3 0 / 0 pooled
settings JSON 0.5 KB / 10 0 / 1 pooled encoder, result cloned
option closures 5 allocs 0 small named types
callConfig, generated Buf handles 1 + 4 1 + 4 generated code and the call itself; not cuttable here
caller's own settings map 0.28 KB / 2 same not the SDK's

Safety

  • Nothing returned aliases C memory or a pooled buffer: TakeInto copies, and TestDecodeBatchOwnsItsMemory overwrites the document buffer after decoding and compares.
  • Rows output is identical to go: allocation-light Rows decoding (#456) #462's across doc_flags 0, 1, 2, 3, 7, 15, 31, 255, body sizes 0, 1, 7, 100, 300 and a malformed body (30 cases, JSON-hashed, run against both trees with the real library; local, not in CI).
  • TestDecodeBatchEquivalence* stay and gain a third real document (a filtered call, with row_spans, rows_passed and rows_cut) and 14 row_spans mutations.
  • go test -race ./chtypes passes (stub-driven concurrent Rows test in CI; a real-library 8-goroutine Rows check under -race run locally).

Not in this PR: anything that needs an ABI or library change (the generated Buf wrappers and their finalizers, the []RowResult shape, the stock decoder's own buffer growth).

🤖 Generated with Claude Code

EricAndrechek and others added 4 commits October 6, 2026 01:03
…nters (refs #456)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ument in the decoder tests (refs #456)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@EricAndrechek
EricAndrechek marked this pull request as ready for review October 6, 2026 05:58
@EricAndrechek
EricAndrechek merged commit 07a74c3 into v1 Oct 6, 2026
50 of 52 checks passed
@EricAndrechek
EricAndrechek deleted the v1-go-rows-alloc-456 branch October 6, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant