Repository navigation
sdk(ts): live-query dedup uses the last fetched row, not the newest — duplicate delivery on desc-ordered fetches #449
Description
Activity
⚠️ Possible Duplicate Issue(s)🔗 Related PRs
#174 - refactor(table names): handle unsafe table names [closed]
#313 - feat(docs): view-transition polish, branded search + 404, chrome pass [closed]
🧪 Issue enrichment is currently in open beta.
You can configure auto-planning by selecting labels in the issue_enrichment configuration.
To disable automatic issue enrichment, add the following to your
.coderabbit.yaml:issue_enrichment: auto_enrich: enabled: false
💬 Have feedback or questions? Drop into our discord!
- added a commit that references this issue
on Aug 11, 2026 - added a commit that references this issue
on Aug 21, 2026 A third failure mode, and it's the one that silently produces no error at all:
received_timestampis not a column every table has.The two in the description are both ordering problems — the boundary is derived from the wrong row, and compared as a string. Both assume the field is there. When it isn't:
const lastRow = rows[rows.length - 1] as Record<string, unknown>; lastTimestamp = lastRow?.received_timestamp as string | undefined; // undefined
received_timestampis part of the ingest envelope and appears in the SSE frame, but a table created by ordinary DDL —CREATE TABLE … (id UInt64, ts DateTime, …)— has no such column, so/v1/querynever returns one. The cast tostring | undefinedmakes that indistinguishable from an empty result, and the consequences are silent in both directions:- Backfill ordering —
.orderBy('received_timestamp', 'desc'), which the docs show as the usage, isn't a valid ordering for such a table. - Dedup — with
lastTimestampundefined, the boundary check has nothing to compare against, so the overlap window is never trimmed.
Nothing throws. You get a live query that appears to work and quietly delivers a different event set than intended, which is worse than the desc-ordering case in the description — that one at least duplicates visibly.
Worth noting for prioritization: this drove a real consumer decision. A first-party browser app building a live-events view chose
.stream()over.liveQuery()specifically because.liveQuery()couldn't be relied on for arbitrary user-defined tables — the feature was avoided rather than worked around.The fix in the description (take the max over parsed values, matching the Go client) doesn't address this on its own — it still assumes the field exists. Whatever lands probably needs to say what
liveQuerydoes on a table withoutreceived_timestamp: require it and fail loudly, let the caller nominate the ordering column, or document the feature as ingest-API tables only. Any of those beats the current silence.- Backfill ordering —
Duplicate of #396.
Same bug, same lines (
clients/ts/src/stream/live-query.ts:63-81). #396 was filed first (repo-wide audit) and is the superset — it carries both failure modes described here plus the missing-received_timestampcase, the sub-second-precision silent drop, and the fetch-error buffer drop. CodeRabbit flagged the overlap on 2026-08-11.Everything unique to this issue has been ported to #396 in this comment — the Go SDK reference implementation on the
go-sdkbranch (with the caveat that it fixes two of the three modes, not the missing-column one), the three live references that need repointing with the fix, and the consumer datapoint about.stream()being chosen over.liveQuery(). Nothing is lost by closing this.Note for the fix PR:
docs/src/content/docs/sdk/streaming.md:115and:188andclients/ts/src/stream/live-query.ts:67all link here. Those links still resolve, but they should be repointed at #396 or removed when the fix lands. The:188prose deliberately documents the current incorrect behavior and reverts to the plain "newest historical timestamp" description at that point.Closing as a duplicate — continue in #396.
Deduplicated by the pm-triage routine on 2026-08-25, on Eric's instruction.
The TypeScript SDK's live query derives its backfill dedup boundary from the last row of the fetch result, not the newest one:
Two problems:
docs/src/content/docs/sdk/streaming.md) orders with.orderBy('received_timestamp', 'desc'), which makes the last row the oldest in the page. The bound is therefore far looser than intended, and events in the fetch/stream overlap window get delivered twice — once viainitial(), again vianext().event.timestamp <= lastTimestampcompares timestamps lexically. That happens to work for fixed-width UTC ISO-8601, but breaks on any format variation (differing sub-second precision, a non-Zoffset).The Go SDK does this correctly —
clients/go/live_query.go:88-100parses the timestamps and takes the maximum across all rows, independent of ordering.Fix: make the TS client take the max over parsed
received_timestampvalues, matching the Go behavior, and add a test covering adesc-ordered fetch.Found during the PR #434 pre-push docs review, which noticed the Go and TS streaming pages asserting contradictory semantics for the same feature. The TS page has been corrected to describe the current (incorrect) behavior and links here; that prose should be reverted to the simple "newest historical timestamp" description once this is fixed.