fix(service-queue,platform-objects): the claim path's sort keys join the declared index, and due-ness becomes a SQL predicate

[claude]

Part of #17612

Clause-②: no

sys_job_queue's claim path sorted by a key no declared index carried, and decided due-ness after LIMIT had already chosen the rows. Both are fixed; the fork the card wrote is decided below with its costs measured, not asserted.

Premise, re-verified on origin/main

Both readings the card carries still hold, checked at the merge base a26a114d7:

packages/platform-objects/src/audit/sys-job-queue.object.ts
  indexes: [ ['queue','status','scheduled_for'], ['idempotency_key','queue'], ['status'] ]
  ⇒ `priority` appears in NO declared index

packages/services/service-queue/src/db-queue-adapter.ts
  orderBy: [ { field: 'priority', order: 'asc' }, { field: 'scheduled_for', order: 'asc' } ]
  ⇒ the sort's FIRST key is the unindexed one

One thing the card could not have known, and it changes the acceptance criterion. The claim query the planner sees is not the one the adapter writes. SqlDriver.orderKeysFor appends the unique tie-breaker of the deterministic-paging contract (ADR-0053 D-A1 / objectstack#4363) to every paged read, so the real ORDER BY ends …, "id" ASC:

SELECT * FROM "sys_job_queue" WHERE "queue" = ? AND "status" = ?
  ORDER BY "priority" ASC, "scheduled_for" ASC, "id" ASC LIMIT ?

That term is in no index either, and no platform object in this repo bounds its id — which puts the card's acceptance 4 ("EXPLAIN QUERY PLAN no longer shows USE TEMP B-TREE FOR ORDER BY") one step further away than it reads. The statement pinned in the new test is captured from the driver, never retyped, precisely so this term stays visible.

The fork, measured on the real tree

The matrix below is EXPLAIN QUERY PLAN of the driver-emitted claim statement, run through a real TursoDriver on both faces (libsql:// over file::memory: and local :memory:), one index set per row.

declared index	plan	sorter
queue,status,scheduled_for (today)	SEARCH … (queue=? AND status=?)	USE TEMP B-TREE FOR ORDER BY — the whole queue
queue,status,priority,scheduled_for (taken)	SEARCH … (queue=? AND status=?)	… FOR RIGHT PART OF ORDER BY (remote) / … FOR LAST TERM OF ORDER BY (local) — the id term only
queue,status,priority,scheduled_for,id	SEARCH … (queue=? AND status=?)	none
option 2: drop priority from the sort, index unchanged	SEARCH … (queue=? AND status=?)	… FOR RIGHT PART OF ORDER BY — the id term only
option 2 with queue,status,scheduled_for,id	SEARCH … (queue=? AND status=?)	none

Option 1 taken — priority joins a declared index. Three reasons, any one sufficient:

1. Option 2 is not the cheaper half of the fork, it is a strictly worse trade. Rows four and five say so: dropping priority from the sort does not avoid touching the index — with the index left alone it lands on exactly the same partial sorter option 1 reaches, and to do better it needs its own four-column index. So the choice is not "widen the index" against "change behaviour"; it is "widen the index" against "change behaviour and widen a different index".
2. priority is a declared, documented, authorable field whose description is Lower = higher priority, and the claim sort is its only runtime effect. Dropping it leaves the field inert — a capability the runtime advertises and does not deliver, which Prime Directive 10 names directly.
3. It is a behaviour change to every deployment that currently sets priority, taken to avoid a cost measured below at one index column.

What option 1 costs, measured. One index, widened from three key columns to four, on one table. The table still declares three indexes: the new one REPLACES ['queue','status','scheduled_for'] rather than joining it, which is safe because the equality prefix queue, status is unchanged (getQueueSize and purge keep the identical seek) and no reader in this repo uses scheduled_for as an index RANGE. Write cost is one extra key column per row insert/update on sys_job_queue, not an extra index.

Why the sorter is not closed all the way. Row three of the matrix is reachable and it is the shape I first delivered. pnpm check:keyed-text-bounds rejected it, and it is right to:

✗ check:keyed-text-bounds: 1 unbounded keyed text-family column(s)
  • sys_job_queue.id  [text]  no maxLength
A text-family column a declared index keys on must declare a `maxLength` (route A, #11374).
Without one `driver-sql` emits it TEXT; MySQL then refuses `ALTER TABLE ... ADD INDEX` with
ER_BLOB_KEY_WITHOUT_LENGTH, and the object lands REGISTERED-BUT-BROKEN with its declared index
silently absent (measured live on MySQL 8.0.46, #12058).

That gate's allowlist is empty and 147 of 147 keyed text columns are bounded — mine would have been the repo's only exception, and the failure mode is the index vanishing on MySQL, which is the very defect this card is about. Bounding a primary key's column type on provisioned tables is its own piece of work. So the delivery stops at four columns, the declaration says why in as many words, and the residual sorter is bounded to rows tying on the whole indexed prefix rather than covering the queue.

Head-of-line starvation (ruled into this card in its comments)

The due bound was applied in JS to the rows SQL had already chosen:

limit: max * 3,
orderBy: [{ field: 'priority', order: 'asc' }, …],
…
if (sched > now.getTime()) continue;   // ← after LIMIT

So a candidate window full of not-yet-due high-priority rows hid already-due work behind it — not for one tick, indefinitely. It now reads $or: [{ scheduled_for: null }, { scheduled_for: { $lte: now } }] in the where, the same shape SqlOutboxStore.claim uses; null needs its own leg because a NULL column never satisfies a less-than-or-equal comparison in SQL — the comparison answers NULL, never true.

Measured on this package's own fake engine at the default batchSize: 10 (candidate window 30):

queue contents	pollOnce() before	after
30 future-dated priority: 1 + 1 due priority: 100	0	1
control: 29 future-dated priority: 1 + 1 due priority: 100	1	1
candidate rows the engine hands back for case 1	30	1

The second row is the control: it read 1 before the change too, which is what makes the first row a measurement rather than a coincidence.

Reverse verification

Every leg: commit first, mutate on disk, prove the mutation landed by occurrence count, rebuild where the consumer resolves through dist/, run, restore with git checkout HEAD -- path, prove the blob hash returns and the whole tree is clean. trap … EXIT INT TERM on absolute paths throughout.

⚠️ The first attempt at leg 1 was a no-op and is reported rather than quietly re-run. Its --absent marker was the source spelling 'queue', 'status', … with single quotes; esbuild emits double quotes, so the marker was never in dist/ and the pre-flight passed vacuously. scripts/ablation-dist-preflight.mjs caught it on the next leg by failing its present-check. Both legs were redone with the marker that is actually in the built bytes, and a sanity leg now asserts the fixed dist/ carries it before anything is mutated.

leg	mutation	consumer resolves	result
sanity	none	dist/	✓ marker present in 6 built files
1 — shipped index declaration	back to ['queue','status','scheduled_for']	dist/ (rebuilt; ✓ marker absent from all 66 built files)	RED — 1 failed / 69 passed, and the one failure is the drift guard
2 — due-set predicate	$or leg deleted from the where	source	RED — 2 failed / 68 passed: the starvation pin and the candidate-count pin
3 — parity fixture	fixture back to the three-column set	source	RED — 3 failed / 5 passed, the plan pin printing + USE TEMP B-TREE FOR ORDER BY

Why that failing set is the right one. Leg 1 reddens exactly one test, the guard that reads the SHIPPED declaration against the claim the adapter actually emits — nothing else in service-queue depends on the index, which is precisely the drift that let this defect exist. Leg 2 reddens the two pins about LIMIT's input set and nothing about ordering, because the predicate changes which rows are candidates and not their order. Leg 3 reddens the two pre-existing #17609 index-set pins plus the new plan pin, because the fixture is the input to all three.

Restores (whole-tree git status --porcelain empty after each):

sys-job-queue.object.ts                      -> 944483cb… (HEAD 944483cb…)
db-queue-adapter.ts                          -> cb8ef2f6… (HEAD cb8ef2f6…)
turso-local-remote-declared-index-parity.ts  -> 82c2c695… (HEAD 82c2c695…)

Clause-2 re-derivation, from the delivered diff

The claim predicted no. Re-derived: no — no new exported symbol is reachable from a published entry, and no new key lands on an already-published payload. The indexes key already existed on SysJobQueue; only its value changed. Measured, not reasoned:

head surface: platform-objects 98 names · service-queue 15 names
base surface: platform-objects 98 names · service-queue 15 names   (rebuilt at a26a114d7)
DIFF platform-objects (base -> head): identical export surface
DIFF service-queue    (base -> head): identical export surface

POSITIVE  SysJobQueue    in platform-objects dist/*.d.ts : 1 hit
POSITIVE  DbQueueAdapter in service-queue    dist/*.d.ts : 8 hits
NEGATIVE  JOB_QUEUE_PRE_17612  across both dist trees    : 0 hits
NEGATIVE  PAGING_TIE_BREAKER   across both dist trees    : 0 hits
NEGATIVE  capturedJobClaim     across both dist trees    : 0 hits
NEGATIVE  JOB_CLAIM_INDEX      across both dist trees    : 0 hits

control — the surface diff CAN report:  2a3  > C

The four negatives are identifiers this diff introduces; all four are test-local and none reaches a published artifact. The changeset is graded patch accordingly.

Gates — with the denominator

Derived from the change set by node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at d9e2830ba (6 paths vs merge base, three-dot):

	count
derived	65
run and green	63
NOT MEASURED	2
UNRUN	0

The two NOT MEASURED both exit 3, the "PREREQUISITE NOT MET" code both scripts define as explicitly not a pass and not a finding:

• pnpm check:dual-build-cjs-loads — no packages/adapters/hono/dist … and 47 more. Needs a whole-repo pnpm build.
• pnpm check:type-check-debt — Build the closure first, exactly as lint.yml does. Same.

pnpm check:i18n was a third until its named closure was built; it then read OK (9 packages — all bundles in sync, no undeclared authoring keys) and is counted in the 63. pnpm check:keyed-text-bounds is the gate that redirected the fix, and now reads 147 keyed text-family columns judged, 147 bounded. Allowlist: 0 pending, 0 unboundable.

Package-level, at 533e41385:

pnpm --filter @objectstack/service-queue test                     70 passed (70)   [64 before]
pnpm --filter @objectstack/driver-turso test                    1249 passed (1249)
pnpm --filter @objectstack/platform-objects test                 575 passed (575)
pnpm --filter {those three} typecheck                            exit 0
eslint . --no-inline-config --format json                6746 files, 0 errors, 0 warnings

The lint number is the full repo run, not a narrowing — it completed inside the foreground budget, so no scope claim is needed for it.

Operational note for an existing database

The retrofit adds idx_sys_job_queue_queue_status_priority_scheduled_for and does not drop the superseded idx_sys_job_queue_queue_status_scheduled_for (measured: 3 indexes before, 4 after, the row untouched). A provisioned table therefore carries one redundant index until an operator drops it through the migrate-plan path; a freshly created table gets three.

Acceptance notes

• Card scope item 3 (idle polling backoff) is NOT delivered here, deliberately. The card says to reuse service-messaging: NotificationDispatcher issues 32 statements per tick on an EMPTY outbox (reap runs per partition, twice) — idle cost scales with partitions × table size and never backs off #17610's mechanism rather than write a second one. That mechanism landed as DispatchLoop in packages/services/service-messaging/src/dispatch-loop.ts (PR fix(service-messaging): reap once per dispatcher tick, back off while idle, wake on emit #17622) and it is not exported from that package's index.ts, so reusing it means publishing a new symbol from service-messaging and adding a service-queue → service-messaging dependency — a queue service depending on a messaging service. That is an architecture decision this card does not authorize, and writing a second copy is what its own scope item prohibits. Named here for the routing seat; the per-tick cost this card was filed about is gone either way.
• The two Turso faces run different SQLite builds, so EXPLAIN QUERY PLAN renders the same partial-sort plan as RIGHT PART OF ORDER BY (remote, @libsql/client) and LAST TERM OF ORDER BY (local, better-sqlite3). The existing delivery-claim pin compares the two faces by EXPLAIN text equality and passes only because its plan happens to have no sorter line; the new job-claim pin compares by index and by sort class instead, and says why in the file. Noted, not filed — no PR or person is heading for that file with a partial-sort plan.
• countingClient in the parity test now records bound args beside the statement text, additively; the existing pins read .sql and are untouched.
• No row was added to scripts/engine-double-contract.pinned.json: the two fake engines in service-queue were EXTENDED (both learn $or/$lte/IS NULL; every other $ key stays a loud throw) rather than replaced, so the ledger stays at 1493 and no contender needs sequencing.

---

Generated by Claude Code

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

3 anchor(s) derived from 2 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see

• 1 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 60 of 215 client-bound route-ledger rows — the other 155 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 155: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 100 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 5 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 66aa2d98d6d1ea1b12d179e3ec82e9f53697e6b8 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from eccf8f8d585690b1b9b7a894777f1682765f9e12 — the merge of head 499f44a9a01b572b900a1b68af9ef8b17a3d3ae0 into base 66aa2d98d6d1ea1b12d179e3ec82e9f53697e6b8, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin eccf8f8d585690b1b9b7a894777f1682765f9e12 && git checkout eccf8f8d585690b1b9b7a894777f1682765f9e12
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 66aa2d98d6d1ea1b12d179e3ec82e9f53697e6b8 499f44a9a01b572b900a1b68af9ef8b17a3d3ae0 && git checkout -B drift-repro 66aa2d98d6d1ea1b12d179e3ec82e9f53697e6b8 && git merge --no-ff 499f44a9a01b572b900a1b68af9ef8b17a3d3ae0

node scripts/docs-audit/affected-docs.mjs --json 66aa2d98d6d1ea1b12d179e3ec82e9f53697e6b8

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

[claude]

CI status on 533e41385 — one real failure, being root-caused. Recorded here so the state is on the PR rather than only in a seat's chat.

Test Core (2/6) → failure. The failing command, from the check-run annotations:

command (…/packages/plugins/plugin-email) pnpm run test exited (1)
turbo: Failed: @objectstack/plugin-email#test   (32 successful, 38 total)
check-test-completeness: OK (3 of 9 scheduled reported, 6 never reached; 1738 declared, all accounted for)

⚠️ The 6 "never reached" packages are turbo stopping after the first failure — a consequence, ⛔ not six more failures. plugin-email is the only one.

⛔ It is not being dismissed as someone else's. plugin-email has @objectstack/platform-objects as a runtime dependency and @objectstack/service-queue as a devDependency, and seven of its files reference sys_job_queue — so this diff (which replaces that table's declared index and moves the due bound into the claim where) can reach it. The shard is green on origin/main at the merge base a26a114d7, measured, so "red on the base too" does not apply either. It is this PR's until measured otherwise.

⚠️ email-service.queue-delivery.test.ts is the subject of #16506 ("Queue-flake anchor"). A ready-made flake story therefore exists for this exact file — which is the reason not to reach for it. "Flake" is not a root cause, no test will be skipped, disabled, quarantined or weakened to clear this, and no empty commit will be pushed to kick CI. The one permitted re-run is held by this seat and unspent.

Also recorded, since both were repaired from the seat side rather than by the implementer (.claude/agents/os-dev.md:55 puts a PR-body PATCH outside a dev's write budget, so body repairs route back here by construction):

• Clause-②: no now stands on its own line. The earlier ## Clause-2 re-derivation heading read to the gate as "a near miss, not a declaration" — and measured by the implementer, a # prefix breaks that axis whatever numeral follows, so a standalone line was the only remedy. Check Changeset cleared on the edited event with no push and no re-run, exactly as the gate said it would.
• Fixes #17612 → Part of #17612. Card service-queue: DbQueueAdapter.claimBatch 按 priority, scheduled_for 排序,而 sys_job_queue 的声明索引不含 priority —— 排序首键落空,恒定 1s 轮询每跳吃一次临时 B-tree 排序 #17612's ## 范围 carries four items; item 3 (空闲轮询 — reusing service-messaging: NotificationDispatcher issues 32 statements per tick on an EMPTY outbox (reap runs per partition, twice) — idle cost scales with partitions × table size and never backs off #17610's idle backoff rather than writing a second one) is not delivered here, so a closing keyword would shut a card with a quarter of its scope outstanding. The card returns to pm:queue for that item on merge, with the implementer's fork analysis recorded on it.

---

Generated by Claude Code