You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 920f887
Browse filesBrowse the repository at this point in the historyBrowse files
fix(service-queue,core): the idle job-queue poll backs off, on the one shared DispatchLoop (#18134)
Fixes#17612
Scope item 3 (空闲轮询) — the last open item on this card. Items 1, 2 and 4
landed in #18105 and are re-verified untouched on `origin/main` here.
Clause-②: yes
## What the card asked, and what was measured first
The card says item 3 should **reuse** #17610's backoff rather than write
a second one, and the dispatching seat pre-ruled out the two cheap
shapes: option D (a second backoff inside `service-queue`) by the card's
own text, option B (exporting the loop from `service-messaging` and
depending on it from `service-queue`) as an inverted dependency
direction. The remaining direction is A — lift the loop somewhere both
can depend on.
The escape hatch was to stop and report if A turned out disproportionate
to the remaining benefit, since items 1/2 already removed the per-tick
**cost** and only the statement **count** while idle was left. Both
sides were measured before any code moved:
| | reads per idle hour, one registered queue |
|---|---|
| before (flat 1 s `setInterval`) | **3601** |
| after (`DispatchLoop`, 1 s base / 30 s ceiling) | **124** |
That is a **29x** reduction, linear in the number of registered queues,
and on a remote driver every one of those reads is an HTTP round trip.
The cost of A turned out to be small: `service-queue` and
`service-messaging` **already both depend on `@objectstack/core`**, so
the lift adds **zero new dependency edges** — it is one file move, one
export, and two import re-points. A is not disproportionate to a 29x
cut, so A was built rather than reported as a stop.
## What changed
- **`packages/services/service-messaging/src/dispatch-loop.ts` moves to
`packages/core/src/dispatch-loop.ts`** and is exported from core's
index. It is a timing primitive owned by neither the messaging domain
nor the queue domain; core is the package all three consumers already
depend on. `service-messaging` publishes only its `index`, which never
carried the loop, so **nothing published moved** — its two dispatchers
just import from `@objectstack/core` now.
- **`DbQueueAdapter.start()` runs that loop** instead of a bare
`setInterval`. The loop also subsumes the old `running` re-entrancy
flag, which was doing tick coalescing by hand.
- **New option `DbQueueAdapterOptions.maxIdleIntervalMs`** (default 30
s). At or below `pollIntervalMs` it disables the backoff and restores
the flat poll exactly.
- **`publish()` and `replay()` wake the loop**, so the ceiling is never
on the latency path for work this process was told about. A *deferred*
`publish()` deliberately does not wake it — that tick would claim
nothing and would throw the backoff away for free.
## Evidence
Tests: `service-queue` 78 passed (70 before + 8 new legs),
`service-messaging` 460, `core` 1316, and the downstream consumers
`plugin-email` 468 and `plugin-audit` 334 — all on the final head.
`typecheck` green for all three edited packages, and it demonstrably
reaches `*.test.ts` (it caught a real error there during this work).
**The flat-poll leg is a negative control, not decoration.** An upper
bound on statements is satisfied by a worker that stopped ticking
altogether — the one failure a low number cannot distinguish. So the
same clock, engine and counter are run with the backoff disabled and
pinned at 3601; that is what makes 124 a reading about the backoff
rather than about a dead loop.
**Reverse verification** — two legs, each mutated on disk (anchor count
1 before, injected marker 1 / removed anchor 0 after), run, restored
with `git checkout HEAD --`, and proved byte-identical by blob hash with
`git diff HEAD` empty:
| ablation | predicted | observed |
|---|---|---|
| the backoff itself, in `core/src/dispatch-loop.ts` | RED | RED — 1
failed / 7 passed, `expected 3601 to be 124` |
| the wake seam in `DbQueueAdapter.publish()` | RED on the wake leg |
RED — 2 failed / 6 passed (wake leg + the drain leg) |
The first ablation reads **3601** — independently reproducing the before
number from a second direction. It also proves the new vitest alias
really reaches core's **source**: no rebuild happened anywhere in that
leg, and the verdict still moved.
**Clause-② re-derivation, from the delivered diff.** Reachability was
taken as the two discriminating reads, not the word `export` and not a
bundle grep:
- `.d.ts` export list — core's `dist/index.d.ts` (the file
`exports['.'].types` names) carries `DispatchLoop`, `type
DispatchLoopOptions` and `DEFAULT_MAX_IDLE_INTERVAL_MS` inside its
single `export { ... }` statement.
- Runtime `in await import(entry)` through the package entry (resolved
to `packages/core/dist/index.js` via the exports map), with controls
both ways:
| symbol | role | result |
|---|---|---|
| `DispatchLoop` | subject | REACHABLE |
| `DEFAULT_MAX_IDLE_INTERVAL_MS` | subject | REACHABLE |
| `DispatchLoopOptions` | subject, type-only | not reachable at runtime,
by design; present in the `.d.ts` export list |
| `ObjectKernel`, `LiteKernel` | positive controls, published before
this diff | REACHABLE |
| `DispatchLoopTimer` | negative control, plausible sibling | not
reachable |
| `dispatchLoop` | negative control, wrong case | not reachable |
`files` is `["dist", ...]`, so that entry ships. A new symbol is
reachable from a shared package's published entry ⇒ **Clause-②** is yes,
and the changeset grades `@objectstack/core` **minor** accordingly
(never `patch`).
**Gate denominator**, reconciled against `scripts/pm/dispatch-gates.mjs
--ran` on the final head `95070c921`: **73 derived, 70 run, 3 NOT
MEASURED, 0 UNRUN**. The three are `check:dual-build-cjs-loads`,
`check:i18n` and `check:type-check-debt`, each of which exited **3** —
the code those gates use for "prerequisite not met, nothing was
measured", all three wanting a whole-repo `pnpm build`. They are
declared NOT MEASURED rather than read as green; CI builds fresh and
runs them.
Three gates went red against my own diff during this work and were
fixed, not routed around: `check:engine-double-contract` (the new
double's `update()` now opens with `assertEngineUpdateDispatch`),
`check:objectql-double-limit` (the double applies the caller's `limit`
by presence, so `limit: 0` is not silently widened to the whole table),
and `check:test-source-alias` (the package now aliases
`@objectstack/core` to source).
`scripts/engine-double-contract.pinned.json` grew by **2 rows** — one
`delete`, one `update`, both for the new test file — from 800 entries /
875 pinned-sum on `main` to 802 / 877.
## Acceptance notes
- The consumer radius was swept by construction site rather than by
edited package: every `new DbQueueAdapter` in the repo was enumerated,
and every test site passes `autoStart: false` and drives `pollOnce()` by
hand, so the timer change reaches none of them. The one production
construction is `QueueServicePlugin`, whose `destroy()` already awaits
`stop()`.
- `start()` now runs its first tick immediately, where the old
`setInterval` waited one interval. That is strictly lower latency and no
test depended on the delay.
- Observation, not filed: the sibling engine doubles in
`db-queue-adapter.test.ts` and `job-queue-retention.test.ts` apply
`limit` by truthiness (`if (opts.limit)`), the same shape
`check:objectql-double-limit` rejected in the new file. They are inside
that gate's existing baseline, so it already knows; a fixture that asked
for `limit: 0` would read the whole table.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01URLHobLUJB9K1ABV6ofdjj)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
`DbQueueAdapter` backs off while `sys_job_queue` is idle instead of polling flat at 1 s, and the loop that does it is now published from `@objectstack/core` as `DispatchLoop` (#17612).
8
+
9
+
A registered-but-idle queue issued **3600 candidate reads an hour, per queue**, whatever was in the table — on a remote driver, 3600 HTTP round trips an hour of pure idle cost. Measured over one simulated idle hour on the engine boundary the adapter really talks to: **3601 reads before, 124 after**, with the flat-poll number re-measured on the same harness as a control so the new one is a reading about the backoff rather than about a loop that stopped ticking.
10
+
11
+
-**One mechanism, not a third copy.** The idle-backoff loop was written for `NotificationDispatcher` (#17610), shared with `HttpDispatcher` (#17623), and lived unexported inside `@objectstack/service-messaging`. `DbQueueAdapter` was the third polling worker needing it. It moves to `@objectstack/core` — the package all three already depend on — because it is a timing primitive owned by neither the messaging domain nor the queue domain, and having `service-queue` depend on `service-messaging` to reach it would invert the dependency direction. **New export from `@objectstack/core`: `DispatchLoop`, `DispatchLoopOptions`, `DEFAULT_MAX_IDLE_INTERVAL_MS`.**
12
+
-**Nothing published moved.**`@objectstack/service-messaging` exports only its `index`, which never carried the loop; its two dispatchers now import it from `@objectstack/core` and its own surface is byte-unchanged.
13
+
-**New option `DbQueueAdapterOptions.maxIdleIntervalMs`** (default 30 s). Each tick that claims nothing doubles the delay to the next from `pollIntervalMs` up to this ceiling; anything claimed, and every wake, snaps it straight back. **Setting it at or below `pollIntervalMs` restores the flat poll exactly.**
14
+
- ⚠️ **What the backoff costs, and what it does not.** Work published through this adapter now wakes the loop, so a due `publish()` and `replay()` are picked up at the base interval as before — the ceiling is never on their latency path. What it does cost is up to `maxIdleIntervalMs` of extra latency on work this process was never told about: a row another node wrote, a deferred row coming due, a crashed worker's lease expiring. A deferred `publish()` deliberately does **not** wake the loop, since that tick would claim nothing and would throw the backoff away.
0 commit comments