Repository navigation
sdk: keyset pagination drops rows tying the page-boundary value (Go + TS) #452
Description
Activity
⚠️ Possible Duplicate Issue(s)🔗 Related PRs
#177 - feat: caching for local table/scope invalidation [closed]
#335 - fix(query): enforce ClickHouse resource caps server-side [closed]
#381 - fix(stream): apply policy row-filter per subscriber on SSE [open]
#434 - feat(sdk): add Go client SDK with full API-tree parity [open]
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:issue_enrichment: auto_enrich: enabled: false
💬 Have feedback or questions? Drop into our discord!
- added 2 commits that reference this issue
on Aug 11, 2026 Consolidating #401 into this issue. Same bug — keyset cursor built from a single column with a strict comparison, so rows tying the boundary value are dropped. This issue is the survivor because it is broader: it covers both SDKs, and it names the wire-format work (tuple comparison or
(a > x) OR (a = x AND b > y)expansion ininternal/query/builder.go) plus the sharedwire_cases.jsonconformance case that keeps the two clients from drifting. #401 was TS-only.But #401 was not a strict subset, so closing it as a plain duplicate would have dropped a real defect. Two things carry over:
1. A second defect on the same line — the cursor predicate accumulates.
clients/ts/src/query-builder.ts:257, in_fetchNext:const nextBuilder = this._clone({ filters: [...this._state.filters, cursorFilter], // <- appends, never replaces orderBy: this._state.orderBy, });
Each
next()keeps the previous page's bound and appends a new one, so page N sends N stackedevent_ts <filters; only the tightest binds. Correctness holds — this one doesn't lose rows — but request body size and server-side parse cost grow linearly with pagination depth. Still live onb2eee9d.This belongs here rather than in its own issue because it is the same edit: the tie-breaker fix has to rewrite exactly this
_clone({filters: …})call. The requirement is that a fix replaces the cursor bound rather than accumulating it — get that wrong while adding the composite cursor and you stack two predicates per page instead of one.2. It has already bitten in production.
From the WaveHouse-Stats dogfooding log (Stats #57, 2026-07-08): Stats skipped a same-second CI event on "Load more" over the live feed. That is the tie-drop failing in a real consumer, not a constructed case. Useful framing from #401 on when it matters: fine for a recent-activity feed, wrong for exhaustive or audit reads — which is the usage that will actually notice.
Also from #401: this is the client-side keyset cursor, distinct from the backend-owned server-side cursor follow-up (#274).
Related: #401 (closed as duplicate of this), #274, #434 (Go SDK, where the Go half of this lands).
Consolidated by the pm-triage routine on 2026-08-25, on Eric's instruction; line refs verified against
b2eee9d.
Keyset pagination in both SDKs builds its cursor from a single column with a strict comparison, so rows that tie on the boundary value are silently dropped.
Problem
Page.Nextfilters on the first.OrderBy()column only, usinggt(asc) orlt(desc) against the last row's value:fetchNextTyped,clients/go/query_builder.go— readsq.state.orderBy[0], emits a strictgt/ltfilter.clients/ts/src/query-builder.ts— same shape.If the last row of a page shares its cursor value with the first rows of the next page, those tied rows are skipped: the filter excludes everything
<=(or>=) the boundary. Paginating on a low-cardinality column (page,status, a truncated timestamp) can drop an arbitrary number of rows with no error and no signal —HasMoreandNextbehave exactly as they do on a clean page.There is also no
Decode-path counterpart: when the cursor column is absent from the projection, pagination ends quietly by design (documented, TS parity). Ties are different — the caller asked for a valid cursor and silently lost rows.Proposed fix
A composite cursor: filter on
(col, tie_breaker)rather thancolalone, using a lexicographic comparison. Requires:.OrderBy()if present, otherwise a server-provided unique column.(a > x) OR (a = x AND b > y)expansion in the structured-query filter grammar (internal/query/builder.go).wire_cases.jsonconformance case, so they can't drift.This is a wire-format change shared by both clients, which is why it isn't being folded into the Go SDK PR.
Interim
Documented as a caveat in
docs/src/content/docs/sdk/go/queries.md(and the TS twin): paginate on a column that is unique per row, or accept that ties at a page edge can be dropped.Related
Surfaced during PR #434 review-thread triage. Same family as the untyped-path
float64precision ceiling already documented on both pages.