Skip to content

fix(sdk): stop hardcoding received_timestamp in default query order - #272

Merged
EricAndrechek merged 3 commits into
mainfrom
sdk-orderby
Jun 5, 2026
Merged

EricAndrechek merged 3 commits into
mainfrom
sdk-orderby

Conversation

@EricAndrechek

@EricAndrechek EricAndrechek commented Jun 5, 2026 •

Copy link
Copy Markdown
Member

Summary

wh.from(table).fetch() hardcoded ORDER BY received_timestamp DESC as both the default query order and the cursor-pagination column. On bring-your-own-schema tables without that column, the SDK emitted invalid SQL → ClickHouse Unknown expression identifier received_timestamp → HTTP 500.

This removes the hardcoded default. The SDK now sends ORDER BY only when the caller sets one via .orderBy():

  • _buildAST no longer injects the received_timestamp default — no order_by is sent unless .orderBy() was called.
  • _fetchNext derives the keyset cursor from the explicit order (this._state.orderBy[0]), with no hardcoded fallback; it returns a terminal empty page if there is no cursor.
  • fetch() attaches next() only when there is an explicit order, but still reports hasMore honestly from the row count — so hasMore: true with next: undefined is a valid, honest state.

Net: .fetch() works on any schema (no 500). Deterministic pagination requires an explicit .orderBy(). This also sidesteps the #175 default-cursor tie bug — there is no longer a default cursor to tie on.

Soft behavior change: result.next is now undefined (rather than always defined when hasMore is true) unless the query has an explicit .orderBy(). Callers paginating via await result.next!() should guard on result.next (the sdk.md / README examples already do).

Why no client-side defaultOrderBy option?

An earlier draft of this PR added an opt-in options.defaultOrderBy to the SDK. It was dropped: a client-global default is the wrong granularity (one order for every table — and it re-introduces the invalid-SQL footgun for any queried table lacking that column), and it duplicates per-table sort-key knowledge in every client and language. The right home for a default sort order is the backend, per-table, designed together with pagination (ideally a server-side cursor token that would also lay the #175 tie bug to rest). Tracked as a follow-up.

Tests

  • clients/ts/src/query-builder.test.ts: a bare .fetch() now omits order_by; .limit(n).fetch() with no order → hasMore: true, next: undefined. The explicit-.orderBy() keyset-cursor test still pins the pagination path.
  • tests/e2e/sdk/query.test.ts: regression test — a table created without received_timestamp returns error === null from a bare .fetch(). The previously-skipped feat: embedded ClickHouse strict validation #175 default-cursor test is repurposed to assert the new hasMore-without-next() contract.

Docs

docs/src/content/docs/sdk.md: the Pagination section now states next() requires an explicit .orderBy() (and corrects a pre-existing "exactly limit" → "at least limit" inaccuracy). CHANGELOG.md: ### Fixed entry under ## Unreleased.

Out of scope

clients/ts/src/stream/live-query.ts and stream/sse.ts use received_timestamp for stream dedup — left as-is (the live ingest stream always carries that column).

Closes #270. Relates to #175 (default-cursor tie); builds on #199 (query-builder structure). The backend-owned, per-table default sort order + server-side pagination is tracked as follow-up #274.

Branch history: two commits — eb236e4 (first cut, which added an options.defaultOrderBy) and 6eb4e26 (drops it, per the design discussion). Net diff vs main is the clean 5-file core fix; intended to squash-merge.

🤖 Generated with Claude Code

The query builder hardcoded `ORDER BY received_timestamp DESC` as both the
default query order (_buildAST) and the pagination cursor column (_fetchNext).
On bring-your-own-schema tables lacking that column, wh.from(table).fetch()
emitted invalid SQL -> ClickHouse "Unknown expression identifier
received_timestamp" -> HTTP 500.

Make the fallback order opt-in via a new `options.defaultOrderBy` on
ClientConfig (OrderClause | OrderClause[]), threaded
client -> from() -> TableRef -> QueryBuilder (new 4th ctor arg; _clone carries
it). A new _effectiveOrderBy() resolves the order: an explicit .orderBy() wins,
else the configured default (never for aggregation queries), else none.
_buildAST sends no order_by when empty; _fetchNext derives the cursor from the
effective order with no hardcoded fallback and pins the next page to it;
fetch() attaches next() only when an order exists but still reports hasMore
honestly from the row count.

Net: .fetch() works on any schema (no 500); deterministic pagination via an
explicit .orderBy() or opt-in defaultOrderBy. Also sidesteps the #175
default-cursor tie bug for unconfigured clients. Stream dedup in
stream/live-query.ts and stream/sse.ts still uses received_timestamp and is
intentionally out of scope.

Closes #270. Relates to #175; builds on #199.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 5, 2026 •

Copy link
Copy Markdown

Review Change Stack

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Free

Run ID: 2d2c667a-f706-4ea8-8de4-8412ac244fca

📥 Commits

Reviewing files that changed from the base of the PR and between 6eb4e26 and 5364110.

📒 Files selected for processing (3)
  • clients/ts/src/query-builder.test.ts
  • clients/ts/src/query-builder.ts
  • tests/e2e/sdk/query.test.ts

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Query builder no longer forces a default ordering; pagination now works on tables without assumed sort columns.
    • .fetch() can report hasMore: true while next is undefined unless an explicit order is set; next() is only provided with deterministic ordering.
  • Documentation

    • Clarified pagination docs to reflect hasMore vs next() behavior and ordering requirements.
  • Tests

    • Added/updated unit and e2e tests covering the new pagination semantics and regression cases.

Walkthrough

The TS SDK query builder no longer emits a default ORDER BY; ast.order_by is emitted only when .orderBy() is set. fetch() reports hasMore when limit is hit but attaches next() only if an explicit order exists. Tests, e2e, and docs were updated to match this behavior.

Changes

Remove hardcoded ORDER BY and make cursor pagination optional

Layer / File(s) Summary
Query builder: Remove hardcoded ORDER BY and update pagination
clients/ts/src/query-builder.ts
_buildAST() omits order_by unless .orderBy() was explicitly set; fetch() attaches next() only when hasMore is true and an explicit order exists; _fetchNext() derives cursor from the first explicit orderBy or returns a terminal page.
Unit tests: AST and pagination assertions
clients/ts/src/query-builder.test.ts
Update AST test to expect order_by: undefined for bare queries; adjust pagination tests to allow hasMore: true with next: undefined; add tests for forward pagination with explicit ascending orderBy.
End-to-end tests: Pagination and schema robustness
tests/e2e/sdk/query.test.ts
Add regression test for bare .fetch() (#270) asserting hasMore: true but next: undefined; add temp ClickHouse table test without received_timestamp to verify .fetch() succeeds and cleanup; adjust related comments and a minor assertion.
Documentation: CHANGELOG and SDK pagination guide
CHANGELOG.md, docs/src/content/docs/sdk.md
CHANGELOG documents removal of hardcoded default ORDER BY and ClientConfig.options.defaultOrderBy; SDK.md clarifies that hasMore is accurate when limit is hit and next() is provided only with a deterministic explicit order.

🎯 3 (Moderate) | ⏱️ ~20 minutes


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added documentation Improvements or additions to documentation area/sdk TypeScript SDK (clients/ts/) area/docs Documentation, site/, README labels Jun 5, 2026
@github-actions

github-actions Bot commented Jun 5, 2026 •

Copy link
Copy Markdown

📚 Docs preview is live: https://6f496484-wavehouse-docs.wave-rf.workers.dev

Updated for commit 5364110

EricAndrechek and others added 2 commits June 5, 2026 05:47
The first cut of this branch added an opt-in `options.defaultOrderBy` to the
SDK as a configurable fallback order. Drop it: a client-global default is the
wrong granularity (one order for every table, and it re-trips the invalid-SQL
500 on any queried table that lacks that column) and duplicates each table's
sort key in every client and language.

Revert types.ts / client.ts / table.ts to main; QueryBuilder keeps only the
core fix — no hardcoded received_timestamp default, ORDER BY emitted solely on
an explicit .orderBy(). The right home for a default sort order is the backend,
per-table, designed together with server-side pagination (subsuming #175);
tracked as #274.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…274

Address PR review feedback:
- Re-add ascending-cursor pagination coverage (the gt keyset path lost when
  the defaultOrderBy tests were removed): asserts the cursor filter, the pinned
  order, and the returned next page.
- Add skipped placeholders (unit + e2e) for bare-.fetch() pagination, which
  should expose next() again once a backend per-table default order lands —
  TODO(#274).
- Trim review-flagged verbose comments in query-builder.ts (the no-default
  ORDER BY line is self-evident; the fetch()/_fetchNext notes are now one-liners).

No behavior change; net #270 fix is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@EricAndrechek EricAndrechek left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@EricAndrechek
EricAndrechek marked this pull request as ready for review June 5, 2026 10:21
@EricAndrechek
EricAndrechek merged commit 0975f88 into main Jun 5, 2026
9 of 10 checks passed
@EricAndrechek
EricAndrechek deleted the sdk-orderby branch June 5, 2026 10:21
@github-project-automation github-project-automation Bot moved this from Backlog to Done in WaveHouse Task Board Jun 5, 2026
@github-actions
github-actions Bot requested a review from taitelee June 5, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation, site/, README area/sdk TypeScript SDK (clients/ts/) documentation Improvements or additions to documentation

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

bug(sdk): default ORDER BY / pagination column hardcoded to received_timestamp 500s on BYOS tables

2 participants