Repository navigation
service-queue: DbQueueAdapter.claimBatch 按 priority, scheduled_for 排序,而 sys_job_queue 的声明索引不含 priority —— 排序首键落空,恒定 1s 轮询每跳吃一次临时 B-tree 排序 #17612
Description
Activity
补一个本卡正文没有覆盖的正确性问题：队头饥饿
来源：#17610 的开发席位在评估
DbQueueAdapter时顺带复现（PR #17622 报告的范围外发现）。这不是性能问题，是调度正确性问题，和本卡第 1 步「判定priority在领取路径上的地位」直接相关，所以放在这里一起裁，而不是另立卡。机制
claimBatch的「是否到期」过滤是在 SQLLIMIT batchSize × 3之后、在 JS 里做的：const candidates = await this.engine.find(QUEUE_TABLE, { where: { queue, status: 'pending' }, limit: max * 3, orderBy: [{ field: 'priority', order: 'asc' }, { field: 'scheduled_for', order: 'asc' }], }); for (const row of candidates) { const sched = row.scheduled_for ? new Date(row.scheduled_for).getTime() : 0; if (sched > now.getTime()) continue; // ← 到期过滤发生在 LIMIT 之后 … }
按
priority优先排序后取前3 × batchSize行，如果这些行全部是未到期的高优先级任务，JS 过滤会把它们全部跳过，本跳领取 0 个——而排在后面、已经到期的低优先级任务永远进不了候选集，直到那批高优先级任务到期。复现（
service-queue自带的假引擎，遵守orderBy与limit）batchSize = 10（候选上限 30）：队列内容 pollOnce()处理数30 个未来才到期的 priority-1 任务 + 1 个已到期的 priority-100 任务 0（已到期任务一直等） 对照：29 个未来 priority-1 + 1 个已到期 priority-100 1 建议
把到期条件下推进 SQL：
where加scheduled_for: { $lte: now }（NULL视为立即到期需一并处理），使LIMIT作用在「已到期」集合上。这同时让本卡要讨论的索引形状更清楚：等值前缀queue, status+ 范围scheduled_for，再决定priority是排序首键还是让位。Unblocked —
Blocked-by: #17609is satisfied. #17609 closedcompleted, and its delivery PR #17615 is verified onorigin/mainby landing probe ((#17615)= 1 against a(#99999)= 0 negative control), ⛔ not inferred from the blocker card merely being closed.⇒ moved
pm:blocked→pm:queue.⚠️ Grading anddomain:*are unchanged; this stroke moves the queue state only.
Generated by Claude Code
Claim:
domain:servicesexecution PM seat dispatching this card.Session: session_01URLHobLUJB9K1ABV6ofdjj
Branch: claude/issue-17612-job-queue-claim-index
Clause-②: noThe blocker is discharged and the premise re-verified on
origin/main(7010085fa), ⛔ not carried from the card.Blocked-by: #17609is satisfied — #17609 closedcompletedand its delivery PR #17615 reads 1 on theorigin/mainlanding probe against a(#99999)= 0 negative control. So this card's acceptance can now be measured in production shape, which is exactly the condition triage wrote the block for.Premise, measured on
7010085fa:packages/platform-objects/src/audit/sys-job-queue.object.ts:199-203 indexes: [ ['queue','status','scheduled_for'], ['idempotency_key','queue'], ['status'] ] ⇒ priority appears in NO declared index packages/services/service-queue/src/db-queue-adapter.ts:481-483 orderBy: [ { priority, asc }, { scheduled_for, asc } ] ⇒ the sort's FIRST key is the unindexed one⚠️ Clause-②: nois a PREDICTION, to be re-derived from the DELIVERED diff. Neither candidate fix meets the mechanical floor as scoped today — changing the value of an existingindexes:key is not a new key on a published payload, and a sort change exports nothing. ⇒ if the delivery adds ANY new exported symbol reachable from a published entry (index.tsre-exports plus the package'sexports/files), flip toyesand say so; the dist probe with a positive and a negative control is the evidence, ⛔ not the wordexportin the diff.⚠️ Fix direction is a genuine fork and is the dev's to decide and document — triage wrote it into the body and marked it ⛔ not a drive-by:- add
priorityto a declared index (index widens, writes cost more), or - drop
priorityfrom the claim sort (changes claim ORDER — a behaviour change).
⛔ Do not pick by convenience. Measure both costs and state which one the delivery takes and why.
⚠️ Declared indexes are a watched contract surface here (driver-turso/src/turso-local-remote-declared-index-parity.test.ts, and #17609's whole subject). Re-run that parity axis whichever fork is taken. ⓘ If the pins add a row toscripts/engine-double-contract.pinned.json, that is safe right now — the serial relay is complete and no baton is outstanding (mainledger re-measured at 1493) — but report it so the seat can sequence any later contender.
Generated by Claude Code
- add
- added a commit that references this issue
on Sep 14, 2026 6 remaining items
Claim:
domain:servicesexecution PM seat re-dispatching this card for scope item 3 ONLY.Session: session_01URLHobLUJB9K1ABV6ofdjj
Branch: claude/issue-17612-idle-poll-backoff
Clause-②: yesItems 1, 2 and 4 are LANDED and verified on
origin/main(8a017af41, PR #18105) — see the landing record above. ⛔ Do not re-do them. Item 3 (空闲轮询) is the only remaining scope, which is why this card returned topm:queuerather than closing.The direction is REUSE, and the card says so
- 空闲轮询:
DbQueueAdapter默认 1s 恒定轮询…⚠️ 若 service-messaging: NotificationDispatcher issues 32 statements per tick on an EMPTY outbox (reap runs per partition, twice) — idle cost scales with partitions × table size and never backs off #17610 已落地一套退避机制,本卡应复用而非另写一套。
Measured by the previous implementer: #17610's mechanism landed as
DispatchLoopinpackages/services/service-messaging/src/dispatch-loop.ts(PR #17622) and is not exported from that package'sindex.ts.⛔ Option D — writing a second backoff in
service-queue— is prohibited by the card's own scope item.
⛔ Option B — exportingDispatchLoopfromservice-messagingand havingservice-queuedepend on it — is ruled OUT by this seat: it inverts the dependency direction (a queue service depending on a messaging service) and publishes a timing primitive from a package whose domain it is not.⇒ Direction A: lift
DispatchLoopinto a package both can depend on, export it there, and have bothservice-messagingandservice-queueconsume it. One mechanism, no inverted dependency.The escape hatch, and it is a real one
⚠️ The per-tick COST this card was originally filed about is already gone — item 1/2's index change made the claim one indexed,LIMIT-bounded read with no full sort. What remains is statement count while idle. If measurement shows A is disproportionate to that remaining benefit, STOP and report with the numbers rather than proceeding: this seat will then rule C-permanently (leave the constant poll, file the backoff as its own card) and close item 3. ⛔ Do not decide that unilaterally, and ⛔ do not proceed with A just because the order names it.Fences
⚠️ Clause-②: yesis a PREDICTION on direction A publishing a new symbol from a shared package. Re-derive from the DELIVERED diff — reachability from the published entry, ⛔ not the wordexportin a diff. ⭐ And per a finding from this seat's last round: presence in a bundle is NOT reachability — a grep hit indist/index.jsproves nothing on its own; the discriminating reads are the.d.tsexport list and a runtimein await import(entry)check. Ayesrequires the changeset gradedminor, neverpatch.⛔
packages/specis off-limits on this card. If the lift appears to need it, STOP and report.
⛔ Never skip, disable, quarantine or weaken a test. ⛔ Do notPOST /issues; findings go into your report with dedupe words.
⚠️ Read.claude/agents/os-dev.mdfrom the branch you work on and follow THAT over the injected snapshot.⚠️ Sweep fixtures by the rule's consumer radius, ⛔ not by the edited package (os-dev.md:240-241) — last round that exact half cost a CI cycle inplugin-email; enumerate the construction sites and grep their fixtures before pushing.
ⓘscripts/engine-double-contract.pinned.jsonreads 1493 onmainand no relay baton is outstanding; if your pins add a row, that is safe — just report it.
Generated by Claude Code
- 空闲轮询:
os-dev-report
{ "issue": 17612, "status": "done", "branch": "claude/issue-17612-idle-poll-backoff", "head_sha": "95070c921", "pr": "https://github.com/objectstack-ai/objectstack/pull/18134", "premise_still_valid": true, "scope": "item 3 ONLY — items 1/2/4 re-verified as landed on origin/main and left alone", "summary": "Direction A, built after measuring that it was NOT disproportionate. DispatchLoop moved from packages/services/service-messaging/src/dispatch-loop.ts (unexported) to packages/core/src/dispatch-loop.ts and is exported there; service-messaging's two dispatchers re-point their imports, and DbQueueAdapter.start() now runs that loop instead of a flat 1s setInterval. publish() (when due now) and replay() wake the loop, so the backoff ceiling is never on the latency path for work this process was told about. The escape hatch was declined on numbers, not on the order naming A: both packages ALREADY depend on @objectstack/core, so the lift costs zero new dependency edges — one file move, one export, two import re-points — against a measured 29x cut in idle statement count.", "remaining_benefit_measured": { "before_reads_per_idle_hour_one_queue": 3601, "after_reads_per_idle_hour_one_queue": 124, "ratio": "29x fewer, linear in registered queue count", "per_tick_cost": "1 read per registered queue per tick (measured exactly at 1 and 3 queues)", "latency_price": "up to maxIdleIntervalMs (30s default) on work this process was NOT told about only; local publish/replay wake the loop", "cost_of_A": "zero new dependency edges — service-queue and service-messaging both already depend on @objectstack/core; packages/spec untouched" }, "tests": "service-queue 78 passed (70 before + 8 new legs) · service-messaging 460 · core 1316 · downstream consumers plugin-email 468 and plugin-audit 334 — all on final head 95070c921, all via scripts/pm/os-verify-lock.sh (VERDICT command-exit 0). typecheck green for the 3 edited packages and demonstrably reaches *.test.ts (it caught a real TS2345 in the new test). REVERSE VERIFICATION, 2 legs, each mutated on disk (anchor 1 before; injected marker 1 / removed anchor 0 after), then restored with `git checkout HEAD --` and proved byte-identical by blob hash with `git diff HEAD` empty: (a) ablating the backoff in core/src/dispatch-loop.ts -> RED, 1 failed/7 passed, 'expected 3601 to be 124' — independently reproducing the before number AND proving the new vitest alias reaches core SOURCE, since no rebuild happened in that leg; (b) ablating the wake seam in publish() -> RED, 2 failed/6 passed (wake leg + drain leg). Tree clean after both.", "negative_control": "The flat-poll leg is pinned FIRST at 3601 on the same clock/engine/counter, because an upper bound on statements is equally satisfied by a worker that stopped ticking — that control is what makes 124 a reading about the backoff rather than about a dead loop.", "clause_2": { "verdict": "yes — re-derived from the DELIVERED diff, not from the word 'export'", "dts_export_list": "packages/core/dist/index.d.ts (the file exports['.'].types names) carries DispatchLoop, type DispatchLoopOptions and DEFAULT_MAX_IDLE_INTERVAL_MS inside its single export { ... } statement", "runtime_import": "resolved through the package entry to packages/core/dist/index.js via the exports map", "positive_controls": "ObjectKernel, LiteKernel — REACHABLE (published before this diff)", "negative_controls": "DispatchLoopTimer (plausible sibling) and dispatchLoop (wrong case) — both NOT reachable", "type_only_note": "DispatchLoopOptions absent at runtime by design, present in the .d.ts export list", "changeset": "@objectstack/core minor, @objectstack/service-queue minor, @objectstack/service-messaging patch — minor as Clause-② requires, never patch", "label": "needs:contract-review applied via the additive endpoint (HTTP 200), comparative read-back shows no stripping", "pair_exit_code": 0 }, "gate_denominator": "73 derived / 70 run (exit 0) / 3 NOT MEASURED / 0 UNRUN — reconciled with `dispatch-gates.mjs --ran` on final head 95070c921, exit codes recorded per family. The 3 NOT MEASURED are check:dual-build-cjs-loads, check:i18n and check:type-check-debt, each exiting 3, the code those gates use for 'prerequisite not met, nothing was measured' (all three want a whole-repo pnpm build); declared NOT MEASURED, never read as green. Re-derived AFTER merging origin/main because the merge changed dispatch-gates.mjs itself: the family count went 65 -> 73 and the 8 new ones were run.", "gates_i_turned_red_and_fixed": "check:engine-double-contract (new double's update() now opens with assertEngineUpdateDispatch), check:objectql-double-limit (double applies the caller's limit by PRESENCE, so limit:0 is not silently widened to the whole table), check:test-source-alias (package now aliases @objectstack/core to source). All three fixed, none routed around.", "pinned_ledger_delta": "scripts/engine-double-contract.pinned.json +2 rows (one delete, one update, both for the new test file): main 800 entries / 875 pinned-sum -> 802 / 877. NOTE: the dispatch's ⓘ figure of 1493 matches NONE of the readings I can take from this file — entries (800), pinned-sum (875), or the --census member count (2659 across 3687 scanned test files). Reporting the discrepancy rather than claiming the number matched.", "consumer_radius_sweep": "Swept by CONSTRUCTION SITE repo-wide, not by edited package. Every `new DbQueueAdapter` enumerated: all test sites (plugin-email x4, service-queue x8) pass autoStart:false and drive pollOnce() by hand, so the timer change reaches none of them; the one production construction is QueueServicePlugin, whose destroy() already awaits stop(). plugin-email initially failed with 5 file-level load errors — that was 'Failed to resolve entry for @objectstack/service-queue', a missing dist prerequisite and NOT a finding; after building the closure it is 468/468. Also checked: no name collision for the 3 new symbols in core's barrel, and the two `export * from '@objectstack/core'` re-exporters (plugin-hono-server, runtime).", "mcp_calls": "2 — create_pull_request, add_issue_comment. Everything else went through git, the zero-quota issue-page payload, and repo-scoped REST (probed green first).", "open_questions": [], "out_of_scope_findings": [ "noted, not filed: the sibling engine doubles in packages/services/service-queue/src/db-queue-adapter.test.ts and job-queue-retention.test.ts apply `limit` by truthiness (`if (opts.limit)`) — the same shape check:objectql-double-limit rejected in my new file. They sit inside that gate's existing baseline so it already knows about them; a fixture asking for limit:0 would read the whole table. Carrier: the next PR to touch either file. Dedupe words: objectql-double-limit truthiness limit 0 db-queue-adapter.test.ts job-queue-retention.test.ts engine double bound.", "noted, not filed: DbQueueAdapter.start() now runs its first tick immediately (DispatchLoop.start() semantics) where the old setInterval waited one full interval. Strictly lower latency, no test depended on the delay. Recorded in the PR's acceptance notes. Carrier: none needed." ], "contract_deviations": "none — the repo copy of .claude/agents/os-dev.md was read from the branch first and matches the injected snapshot substantively; no action taken that it forbids." }
Generated by Claude Code
Landed — scope item 3, and with it the whole card. PR #18134 merged
2026-09-14T06:55:15Zas920f887dcc43cb6409d5f45c81e74a2626d29b54, enqueued06:25:45Z. All four## 范围items are now delivered (1, 2 and 4 in PR #181058a017af41; 3 here).Verified on
origin/mainwith controls, ⛔ not inferred from the merge:(#18134) = 1 ← the delivery (#18109) = 1 ← positive control, known merged (#18105) = 1 ← positive control, known merged (#99999) = 0 ← negative control content : packages/core/src/dispatch-loop.ts PRESENT; core/src/index.ts re-exports it content : db-queue-adapter.ts maxIdleIntervalMs — 4 occurrences ledger : len(entries) = 802 (was 800), $comment unchanged at 693 charsWhat item 3 delivered
Direction A, and the escape hatch was declined on numbers rather than on the order naming A.
DispatchLoopmoved fromservice-messaging(where it was unexported) into@objectstack/core, is exported there, and bothservice-messaging's dispatchers andDbQueueAdapter.start()now run that one loop. The card's 「复用而非另写一套」 is honoured by a git rename, ⛔ not a copy.The cost measurement that decided it: both packages already depend on
@objectstack/core, so the lift costs zero new dependency edges — one file move, one export, two import re-points. Against idle reads/hour 3601 → 124 (29×), linear in registered queue count. Latency is unchanged for work this process is told about:publish()(when due now) andreplay()wake the loop; the ceiling applies only to rows another node wrote.⭐ The measurement carries its own negative control, which is why 124 means what it says: the flat-poll leg is pinned FIRST at 3601 on the same clock, engine and counter — an upper bound on statement count is equally satisfied by a worker that stopped ticking. The at-tier review then ablated the loop dead and confirmed both legs fail (the control reading
expected 1 to be 3601), ruling out the vacuous reading.Clause-② review of record
Clause-②: yes. At-tier record 5659878791 on PR #18134, head95070c921484eabf579456da80a0d6f7aef84446, verdict PASS,Served-tier: CONTRACT_REVIEW_TIER(the constant's NAME). Dual carrier cleared from this card and the PR in one stroke (provenance5659888702);--pair 18134read 0 after the strip, governed-surface 0 of 11.Published surface:
DispatchLoopandDEFAULT_MAX_IDLE_INTERVAL_MSreachable from@objectstack/core's entry on both runtimes,DispatchLoopOptionstype-only by design, plusDbQueueAdapterOptions.maxIdleIntervalMs— a new key on an already-published payload.⚠️ The review recorded one evidence gap, ⛔ not a wrong claim:DbQueueAdapter.wake()is also a new public method on a published class and the write-up did not name it; theminorgrade already covers it and@objectstack/specis untouched.ⓘ The docs-drift flags on
automation/jobs.mdx,automation/webhooks.mdxandprotocol/kernel/lifecycle.mdxwere homonyms — none namesDispatchLoop; they are the job-schedule trigger field,PluginHealthMonitor, andHttpDispatcher(which still lives inservice-messaging). ⛔ No page was inaccurate and no doc edit was owed.ⓘ Three gates the implementer turned red were fixed, not routed around:
check:engine-double-contract,check:objectql-double-limit,check:test-source-alias— all baselines byte-identical tomain, confirmed independently by the review.ⓘ Carried forward, non-blocking:
subscribe()does notwake(), so a queue registered after the loop has backed off sees already-persisted rows up tomaxIdleIntervalMslate rather than ≤ 1 s. That sits inside the documented latency class. Also noted by the implementer: the sibling engine doubles indb-queue-adapter.test.tsandjob-queue-retention.test.tsapplylimitby truthiness, so a fixture asking forlimit: 0would read the whole table — insidecheck:objectql-double-limit's existing baseline; carrier is the next PR to touch either file.⚠️ This seat is going off shift on the maintainer's ruling 「当前任务处理完就下班」. This was its last item.
Generated by Claude Code
- added 3 commits that reference this issue
on Sep 17, 2026
由分诊席在处理 #17610 时立卡 —— 该卡的维护者明确点名要这个判断:「
service-queue的DbQueueAdapter.claimBatch(默认 1s 轮询)按priority, scheduled_for排序,而声明索引是['queue','status','scheduled_for']——排序首键priority不在索引里;空闲时同样恒定轮询。你若判断应该另立卡,说出来。」⇒ 判断:另立卡。 理由在下面「为什么不并进 #17610」。⛔ 未认领,⛔ 未定级 —— 本卡由分诊席开,
domain:*与优先级见分诊评论。实测(
origin/main@3ef96b47120a989d9333aeeeb6181bc371bc9b52)维护者的两条读数逐条复现,⛔ 不是转述:
排序,
packages/services/service-queue/src/db-queue-adapter.ts:478-487:声明索引,
packages/platform-objects/src/audit/sys-job-queue.object.ts:199-203:QUEUE_TABLE=sys_job_queue(db-queue-adapter.ts:22)。⇒ ⭐ 等值前缀
queue+status能用上索引,但排序首键priority不在索引里 ⇒ 第三列scheduled_for对排序失效,引擎只能落到临时 B-tree 排序。⇒ 与 #17610 同形状:一个恒定轮询的领取查询,拿不到它以为拿得到的索引。⭐ 为什么另立卡而不是并进 #17610
三条,任何一条单独成立即足够:
packages/services/service-messaging(reap 解耦 + 空闲退避);本卡在packages/services/service-queue与packages/platform-objects。同车道(domain:services)⛔ 不等于同一个 worktree、同一个 changeset。priority加进索引(索引变宽,写入变贵),还是把priority从排序里拿掉(改变领取顺序 = 行为变化)。⛔ 这不是能顺手带过的东西。indexes— every declared secondary index is absent on production Turso tenant databases, so hot polling queries full-scan #17609 之后。 维护者在 driver-turso: remote mode never materializes object-levelindexes— every declared secondary index is absent on production Turso tenant databases, so hot polling queries full-scan #17609 里实测:remote Turso 上sys_job_queue声明了 3 个索引、两个生产库里一个都没有。⇒ 今天在生产上,这张表根本没有索引可用不上 —— 「排序首键不在索引里」是 driver-turso: remote mode never materializes object-levelindexes— every declared secondary index is absent on production Turso tenant databases, so hot polling queries full-scan #17609 落地之后才开始有代价的问题。Blocked-by:。Blocked-by: #17609
(⇒ remote 面上
sys_job_queue今天一个声明索引都没有,本卡的验收在 #17609 落地前无法在生产形态上测量。)范围
priority在领取路径上的地位:是真正的排序键(⇒ 索引要覆盖它),还是可以让位给scheduled_for(⇒ 排序要改)。⛔ 这是行为判断,不是索引调优 —— 写清楚再动。orderBy,两者同名同键地对齐。DbQueueAdapter默认 1s 恒定轮询,与 service-messaging: NotificationDispatcher issues 32 statements per tick on an EMPTY outbox (reap runs per partition, twice) — idle cost scales with partitions × table size and never backs off #17610 的第 2 条验收(空闲退避)是同一个形状 ——EXPLAIN QUERY PLAN不再出现USE TEMP B-TREE FOR ORDER BY。⛔ 本卡不主张的
priority应该被删掉,也不主张索引应该加宽 —— 范围第 1 条就是要定这个。SqlDriver上是问题:本地索引齐全,代价是一次 B-tree 排序而不是全表扫描,量级完全不同。sys_job_queue的实际行数与轮询产生的读量。driver-turso: remote mode never materializes object-levelindexes— every declared secondary index is absent on production Turso tenant databases, so hot polling queries full-scan #17609 / service-messaging: NotificationDispatcher issues 32 statements per tick on an EMPTY outbox (reap runs per partition, twice) — idle cost scales with partitions × table size and never backs off #17610 有sys_notification_delivery的实测数字;本表没有,所以本卡⛔ 不引用那些数字。Refs: #17610(同形状的姊妹卡,维护者在其中点名要这个判断)· #17609(remote 索引缺失 —— 本卡的前置)· #17611(同一三件事叠乘里的第三张)
Generated by Claude Code