Repository navigation
fix(service-analytics): the ObjectQL face applies a query's order, offset and limit, as its echoed sql says - #21363
Conversation
…fset and limit engine.aggregate has no ordering or window grammar, so the ObjectQL face answered every group in the engine's arrival order while its echoed sql and /analytics/sql rendered ORDER BY ... LIMIT ... OFFSET. Every date-bucketed query lands on this face, so a "top N" or ordered trend answered unordered and unlimited. The face now orders the remapped answer by the query's order (the package's one row comparator, applyOrdering) and then applies offset and limit (applyWindow), on the base path and the cross-object path. The dataset door pushes a single query's window down and then re-windowed the answer, applying offset a second time; it now re-sorts only when the window was pushed down. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…mit on SQLite and PostgreSQL The card's rows on both drivers: a bucketed time dimension ordered and limited, a selected dimension and a selected measure equal to the native face, two order keys, limit 0, the cross-object path, the echoed sql run on the same driver, the no-window control, and the dataset door's pushed-down page on both faces. The #21267 controls now assert the ObjectQL face's order. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…ing order does not already give SQLite answers a grouping in ascending key order, so a pin whose expected order matched it stayed green with the face's ordering removed. The fixture's amounts now make every ordered pin, and every window, differ from that order. Claude-Session: https://claude.ai/code/session_01DiCSbmJrkzNhuEAier4VoJ Co-authored-by: Claude <noreply@anthropic.com>
…jectql-face-order-limit
📓 Docs Drift CheckThis PR changes 1 package(s): ⛔ 1 release-owned page(s) name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 10 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 4f6f163a501a15d0e32dfc7a28554c891e7a3870 && git checkout 4f6f163a501a15d0e32dfc7a28554c891e7a3870
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 16eefc6c0e136cd9f21397b0c64229bce0008271 a3179775d3dc3577f6c0d4c3624f5a74a68c6857 && git checkout -B drift-repro 16eefc6c0e136cd9f21397b0c64229bce0008271 && git merge --no-ff a3179775d3dc3577f6c0d4c3624f5a74a68c6857
node scripts/docs-audit/affected-docs.mjs --json 16eefc6c0e136cd9f21397b0c64229bce0008271
|
Fixes #21316
Clause-②: no
What this changes
The ObjectQL face applies
order, thenoffsetandlimit, to the aggregated answer, as its echoedsqlandPOST /api/v1/analytics/sqlsay it does (packages/services/service-analytics/src/strategies/objectql-strategy.ts,orderAndWindow).engine.aggregatedeclares no ordering or window grammar (EngineAggregateOptionsSchema,packages/spec/src/data/data-engine.zod.ts), and neither does theexecuteAggregatecontract (StrategyContext.executeAggregate,packages/spec/src/contracts/analytics-service.ts). So there is nothing to push down, and the strategy applies the three keys itself. It does this on the direct path and on the cross-object (FK-expand) path, after the re-bucket. Before the re-bucket, a limit would cut FK groups that merge into a group it keeps.orderkey names a column exactly as the analytics door's order-key rule (order-key-door.ts, analytics: on the native-SQL strategy an order key that is not a selected member answers 500 (ambiguous, must appear in GROUP BY, or no such column); the ObjectQL face answers 200 for the same queries #21267) admitted it.orderis read verbatim, in its own key order, and no implicit ordering is added. A barelimitslices the engine's order, the same asLIMITwithoutORDER BY.applyOrderingand the window isapplyWindow. Both come fromdataset-executor.ts, the package's one row comparator and window, which the dataset door's post-pass already applies to every grid. Nothing is copied. Other comparators exist in or near the package, and none was reused:preview-evaluator.tshas its owncomparefor draft rows, anddriver-memory'sapplySortis private.The dataset door no longer applies a pushed-down
offsettwice (dataset-executor.ts). For a single-query selection,DatasetExecutorpushesorder,limitandoffsetdown to the strategy and then windowed the answer again. The second slice appliesoffseta second time. This is a shipped defect on the native face, and this change would have extended it to the ObjectQL face, which was right before only because it dropped the window. The post-pass now windows only a grid it could not push down. It still re-sorts every grid.Measured before / after
The measurement went through the real
POST /api/v1/analytics/queryand/analytics/sqlroute (createDispatcherPlugin), usingAnalyticsServicePluginwith its default composition and with capabilities narrowed to the engine aggregate, a realObjectQLengine andSqlDriver. It ran on SQLite (better-sqlite3) and PostgreSQL 16.14 (C.UTF-8 collation).main97239c3, SQLitemain, PostgreSQLtimeDimensions [closed_on, month],order { closed_on: desc },limit 104, 03, 05dimensions [note],order { note: desc }order { amount_sum: desc },limit 2,offset 1On both drivers the echoed
sqlrendered… ORDER BY "closed_on" DESC LIMIT 1(and the others), before and after.Dataset door,
limit 2, offset 1, ordered by a measure over five groups. Before, the native face answered one row (the third) on both drivers, and the ObjectQL face answered the second and third. After, both faces answer the second and third.Pins (
src/__tests__/objectql-face-order-limit.test.ts, SQLite always, PostgreSQL whereOS_TEST_POSTGRES_URLis set)descwithlimit 1answers the newest bucket, on both compositions, and no raw statement runs. Also a bucketed time dimension withoffset 1, limit 2.offset/limit, two order keys,limit 0, and the cross-object path ordered by a measure and by the relationship-path dimension.sql, run on the same driver, answers the rows the ObjectQL face answered.order/offset/limit, the answer is exactly the engine aggregate's rows in the engine's order (a selected dimension and a bucketed one).order-key-selected.test.tsnow assert the ObjectQL face's order, not only its set.Ablations (committed tree, each leg through
scripts/ablation-replace.mjswith the anchor proven to land and the restore proven by blob hash plus an emptygit diff HEAD; the pins import the strategy fromsrc/, so nodist/is involved)Recorded at
c6c4ae9d0. The later merge ofmain(a3179775d) leavespackages/services/service-analytics/byte-identical:git diff c6c4ae9d0 a3179775dover that path is empty. The pin file has 26 cases, 13 per driver.applyWindow(rows, …)): 17 red, 9 green. Every order pin is red on SQLite. The 9 green are the controls,limit 0, and windows that the arrival order happens to satisfy on one driver.applyOrdering(rows, query.order)): 18 red. Everylimit/offsetpin is red on both drivers. The order-only pins and the controls stay green..reverse()): 20 red. Every native-equality pin and both controls are red on both drivers. The dataset-door pins stay green, because the executor re-sorts a middle window that happens to be symmetric. The next leg owns those pins.Acceptance notes
'', numeric text and mixed case. This was measured on this branch at the route:ORDER BYplaces NULL lowest on SQLite and highest on PostgreSQL, and orders text by the column collation (codepoint on SQLite, and on PostgreSQL under C.UTF-8).applyOrderingkeeps NULL and''last in both directions, compares numeric text numerically, and compares other text withlocaleCompare.content/docs/data-modeling/analytics.mdx). It is reused, not changed. The open question is in the report.limit/offsetoutside the non-negative integers.AnalyticsQuerySchemadeclares both asz.number(). Measured on this branch:limit: -1: native SQLite returns every row, native PostgreSQL answers 500. The ObjectQL face now answersapplyWindow's slice (all but the last row), as the dataset door already does for the same value. Before, it answered every row.limit: 1.5: native SQLite answers 500, PostgreSQL rounds to 2 rows, the ObjectQL face answers 1 row.offset: -1: the native face answers 500 on both drivers.offsetwithoutlimiton SQLite. The native face rendersOFFSET nwith noLIMIT, which SQLite refuses (near "OFFSET": syntax error, 500). The ObjectQL face answers it, and its echo renders the same statement. This is in the report as a finding.content/docs/**(outsidereleases/) orskills/**says the ObjectQL face ignoresorder/limit, so no doc sentence becomes false. Code comments that said so, in thedataset-executor.tsheader and post-pass and in theorder-key-door.tsheader, are corrected.Verification (head
a3179775d, after mergingmain)pnpm --filter @objectstack/service-analytics testwith live PostgreSQL 16.14 (OS_TEST_POSTGRES_URL, so no cell skipped): 167 files, 3869 tests passed. Its dependency closure was rebuilt first.pnpm --filter @objectstack/service-analytics typecheck: exit 0. Both touched test files are in its program (tsc --listFiles).node scripts/pm/dispatch-gates.mjs --commands, merge base96b12b589) has 63 commands. All 63 exited 0, and--ranreconciles 63 derived, 63 run, 0 NOT-MEASURED. On the earlier head,pnpm check:dual-build-cjs-loadsfirst answered PREREQUISITE NOT MET (nodist/). It was run again after a fullturbo run buildand answered 0, and it answered 0 again on this head..tsfiles:eslint --no-inline-config --format jsonreads 5 files with 0 errors and 0 warnings.eslint.config.mjsenables no type-aware linting (noparserOptions.project), so this diff cannot change the verdict on any untouched file. The repo-widepnpm lintis left to CI.Generated by Claude Code