feat(plugin-webhooks): the webhooks service serves the redeliver door from a Request, and the veto no longer waits for realtime (webhooks segment 3 of #22564) (#22797)

Fixes #22756
Clause-②: yes

Segment 3 of the webhooks member of #22564's stage 2, under the
maintainer's ruling 「A + 扫类」 on #22438 (director record `6079645593`,
item 2; triage split `6104808418`). It implements the member that
segment 1 (#22754, landed as `e84aeb36ce`) declares. Dispatched by the
`domain:services` seat 1 (seat post #6021), claim `6105954152`, session
`session_01CBAfsWMSfM3EToQGVStEcp`.

## What changes

All runtime changes are in
`packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts`.

- **The `webhooks` service and its member.** At `kernel:ready` the
plugin registers the `webhooks` slot. It holds an `IWebhookService`
whose `handleRedeliver(request)` serves `POST
/api/v1/webhooks/redeliver` from a web-standard `Request` and answers a
`Response`. It needs no raw app, no Hono context and no `http.server`.
- **One body, two faces.** The redeliver door's body moved, unedited,
out of the mount's inline handler into one private method,
`serveRedeliver(ctx, messaging, c)`. The mount calls it with the real
Hono context. The member calls it with `requestDoorContext(request)`, a
three-member view of the `Request`: its headers, `request.json()`, and a
`json(body, status)` that builds the `Response` the way Hono's `c.json`
does. So both faces share one copy of each rule: the session check, the
active-organization tenant, `MessagingService.redeliverHttp`, and the
status for each outcome. The mount does not call the member, and the
member mounts nothing. `git diff -w` shows the moved block as unchanged.
- **The #8069 veto is installed without realtime or auto-enqueue (seat
decision `6105715713`).** `installRedeliverGuard` used to run only
inside `bootAutoEnqueue`. That function returns early when `autoEnqueue`
is `false`, or when ObjectQL, Realtime or Messaging is missing. A new
step, `bootRedeliverDoor`, now installs the veto with only the engine
and messaging. It runs after the declared-webhook bootstrap and before
both the enqueuer and the mount. It registers the `webhooks` slot **only
after** the veto is installed. If the messaging service cannot take the
veto, the slot stays empty and the existing `error` line now also says
so. The veto still reads `subscriptionsObject` from the auto-enqueue
options.
- **`http.server` before the alias.** `registerAdminRoutes` now reads
`['http.server', 'http-server']` through `tryGetService`, which tries
each name in its own `try`, as `plugin-approvals` does. A host that
registers its server only as `http.server` (`runtime.ts`'s
`config.server` path) now gets the route. A host that registers both
names (`plugin-hono-server`) gets it once.
- **Tests:** `src/webhook-redeliver-member.test.ts` (new, 14 tests).
`hono` joins the package's devDependencies (`^4.13.9`, the same as
`plugin-approvals`; the workspace override resolves it to the existing
4.13.12, +3 lockfile lines), so the mount is driven through a real Hono
app.
- **Ledger:** `scripts/engine-double-contract.pinned.json` gains the one
row that `check:engine-double-contract` asked for. The new test's
`findOne` double opens with `assertEngineFindOnePredicate`, and the
gate's own remedy is `--write`.
- **Changeset:** `.changeset/22756-webhooks-redeliver-member.md`,
`minor`, carrying `Clause-②: yes`.

## Pins (`src/webhook-redeliver-member.test.ts`)

The harness is a real `LiteKernel` and the plugin's real boot. The
plugin's `kernel:ready` hook installs the veto, registers the slot and
mounts the route. Messaging is the real `MessagingService` over the real
`MemoryHttpOutbox`, so each refusal comes from the outbox: the tenant
scope, the row-local refusals and the veto. The member is always read
off the started kernel's `webhooks` slot.

1. **The member, on a kernel with no `http.server` (the hosted shape).**
It answers every arm of the contract with `code` + `status`,
`Content-Type: application/json` and the envelope:
   - `401 UNAUTHENTICATED`;
   - `400 INVALID_REQUEST`;
- `400 MISSING_REQUIRED_FIELD` for a missing, a non-string and a blank
id;
- `404 RESOURCE_NOT_FOUND` for an unknown row and another organization's
row;
   - `409 DELIVERY_NOT_ELIGIBLE` for an unfinished row;
   - `409 DELIVERY_NEVER_SENT` for a parked row;
   - `409 DELIVERY_NOT_ELIGIBLE` from the veto, for a gone subscription;
   - `500 INTERNAL_ERROR` when there is no outbox;
- `200` with `{ id, status: 'pending' }`, after which the row really is
`pending`.

No refusal writes anything. The caller's session is read with
`inProcessSessionReadInput`, so a cookie request reads with
`disableRefresh`. Nothing is mounted.
2. **Parity.** On a kernel with `http.server` and its alias on one Hono
app, the same 11 refusal requests go to the member and then to
`app.fetch`. Status, the full header list and the body bytes are equal
for each. A replay through each face, on twin rows, matches on status
and headers, and on the body once the row id is replaced.
3. **The composition, and no double mount:**
   - both names on one server: exactly one `POST` route;
   - `http.server` alone: one;
   - the alias alone: one;
- two servers under the two names: the route is on `http.server`'s
server and not on the alias's;
- with no `http.server`: nothing is mounted, and the "mounted" line is
never logged.
4. **The veto without realtime.** A redelivery of a delivery whose
subscription is gone is refused with `409 DELIVERY_NOT_ELIGIBLE` by both
faces in two cases: with no realtime service, and with realtime present
but `autoEnqueue: false`. A control row on the same kernel, whose
subscription stands, replays `200`. With a messaging service that has no
`registerRedeliverGuard`, the `webhooks` slot is absent and the `error`
line is logged once.

## The mount on `main`, measured directly

This checks that the route answers the same bytes as the one that ships
on `main`, and not only the same bytes as this branch's member. A
scratch capture test, not committed, booted the plugin with
`http.server` and `http-server` on one Hono app, with realtime present,
so that `main`'s file also installs the veto. It recorded the mount's
status, headers and body for 12 requests, with row ids replaced by fixed
labels, plus the route list. The 12 requests were the 11 refusals above
and one replay.

- **HEAD leg:** plugin blob `d157cbdc6c`, equal to HEAD's.
- **main leg:** the file was replaced with `e84aeb36c`'s blob
`60dc0b99b9` (the hash was checked on disk; `serveRedeliver` hits 0).
- **Result:** the two captures are byte-identical (sha256
`fa158c07f177…` both). The route list is `['POST']` in both.
- **Restore:** `git checkout HEAD -- ABSOLUTE_PATH`. The blob is back to
`d157cbdc6c`, `git diff HEAD` is empty, and the scratch file is removed.
The trap was armed on EXIT, INT and TERM.

## Ablations (each through `scripts/ablation-replace.mjs`)

Each ablation was run against `src/webhook-redeliver-member.test.ts` at
`40a52b4e5`, the plugin blob `d157cbdc6c` that is also HEAD's. Each
mutation was proven on disk (the anchor count 1 to 0, the blob changed),
and each restore was proven (blob equal to HEAD, `git diff HEAD` empty).

| # | Mutation | Result |
|---|---|---|
| A1 | The veto put back behind `main`'s prerequisites (installed only
when `opt !== false` and realtime resolves) | **5 failed.** Both veto
pins (`expected 200 to be 409`), the matrix's veto case, parity's `409
veto` (`expected { status: 200, …} to deeply equal { status: 409, …}`),
and the no-veto-no-slot pin. |
| A2 | The server read as `['http-server']` (`main`'s) | **2 failed:**
`http.server` alone, and the split servers (`expected [] to deeply equal
[ 'POST' ]`). |
| A2b | The alias read first | **1 failed:** the split servers. |
| A3 | The member renamed away | **10 failed.** Every member-reading pin
(`the webhooks slot holds no handleRedeliver`, `expected 'undefined' to
be 'function'`). |
| A4 | The slot registered whether or not the veto is installed | **1
failed:** `expected { Object (handleRedeliver) } to be undefined`. |
| A5 | The route registered twice | **4 failed:** `expected [ 'POST',
'POST' ] to deeply equal [ 'POST' ]`. The first attempt was a no-op: its
replacement contained its anchor, `ablation-replace` refused because the
anchor count did not drop (1 before, 1 after), and nothing ran. The
second attempt used a `for` loop that does not contain the anchor. |
| A6 | The member's `Content-Type` changed to `application/json;
charset=UTF-8` | **4 failed:** the matrix, the replay and both parity
pins. |

## Verification (HEAD `c66f74076`; `origin/main` merged at `a2e94c2a0`)

- `pnpm --filter @objectstack/plugin-webhooks test`: 17 files, 182 tests
passed. The new file passes 14 of 14.
- `pnpm --filter @objectstack/plugin-webhooks typecheck`: exit 0.
`check:test-typecheck` is OK, and the new test is in both programs:
`--listFiles` counts 1 under `tsconfig.json` and 1 under
`tsconfig.test.json`.
- `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derives 87 commands at `c66f74076`, and all
87 ran with exit 0. `--ran` reports 87 derived, 87 run, 0 NOT-MEASURED
and 0 UNRUN. The same 87 had also run green at `04d3756b4`, before the
merge. The first run at `919541c71` had one finding,
`check:engine-double-contract` asking for the ledger row above, and one
`--write` fixed it.
- **Build:** the dependency closure was built first (`pnpm --filter
'@objectstack/plugin-webhooks^...' run build`).
`check:dual-build-cjs-loads` and `check:i18n` first refused with
`PREREQUISITE NOT MET` (exit 3). After `turbo run build
--filter='!@objectstack/docs'` both exit 0. After the merge of
`origin/main` there was one more full build (73 tasks), then the package
test, typecheck and the 87 gates again.
- `origin/main` has since moved to `a8f24b092` (#22781:
`packages/verify` and `packages/qa/dogfood` only, disjoint from this
diff). It is not merged here; the merge queue rebuilds on it.
- **Not run locally:** CI's own job steps and the workflow-valued
families that the derivation lists as such. CI owns those.

## NOT MEASURED: the end-to-end pin through the dispatcher

The pin through the runtime dispatcher's `POST /webhooks/redeliver`
domain is NOT MEASURED here. That domain is #22755 (`domain:cli`), which
has not landed. On `origin/main` `a8f24b092`, `git grep -n -i webhooks`
over `packages/runtime/src/http-dispatcher.ts` and
`packages/runtime/src/domains/` exits 1 with no hits. The control, `git
grep -c -i approvals` on `packages/runtime/src/domains/approvals.ts` at
the same ref, exits 0 with 28. Under the card body, whichever of #22755
and this PR lands second runs that pin. The pins here call the member
through the kernel's `webhooks` slot, as the dispatch directs.

## Acceptance notes

- **File surface.** The claim lists
`packages/plugins/plugin-webhooks/src/**` and the changeset. This PR
also touches:
- `packages/plugins/plugin-webhooks/package.json` (devDependency `hono`)
and `pnpm-lock.yaml` (+3 lines), which the real-Hono pins need;
- `scripts/engine-double-contract.pinned.json` (+5 lines, the
gate-prescribed row).

Neither changes what ships (`files[]` is `dist`, `README.md`,
`CHANGELOG.md`). `packages/spec`, `packages/runtime`, `packages/core`
and the docs are not touched.
- **The mount's source moved but its answers did not.** The card asks
that the raw mount stay byte-unchanged. This PR reads that as the
mount's answers, which the dispatch's pin wording ("byte-equal answers
for the same requests") also does. The handler body moved verbatim into
`serveRedeliver`, the route callback is now one line, and the read order
changed (`http.server` first, which the card asks for). The answers on
`main`'s mount were measured byte-equal above.
- **"Disabled" is not the veto's predicate.** The dispatch's pin wording
said "a delivery whose subscription is disabled". A scratch probe
measured `createWebhookRedeliverGuard` on a subscription with `active:
false` and no secret, and it returned `undefined`, so the delivery is
allowed. The guard refuses a subscription that is **gone**, or one whose
stored secret cannot be recovered (its docblock's cases 1 and 2). So the
pins use a gone subscription. Whether a disabled subscription should
also veto redelivery is outside this card, and nothing here changes it.
- **The veto now applies on more kernels.** On a kernel without
realtime, or with `autoEnqueue: false`, a webhook delivery whose
subscription is gone, or whose secret cannot be recovered, used to be
replayed. It is now refused with `409 DELIVERY_NOT_ELIGIBLE`. That is
the gap the seat decided to close. The changeset states it.
- **Method.** The member does not check the request method. Its one
caller is an exact-`POST` dispatcher domain, and the contract's status
table names no `405`. A forwarded non-`POST` with no body would be told
`400 INVALID_REQUEST` after the session check.
- **No new log line on a kernel without a raw app.** The mount's
existing `debug` line is unchanged. A line naming the dispatcher domain
belongs with #22755, which has not landed.
- `Clause-②: yes` is copied as the claim spells it. The PR owes a
contract-review-tier record before the queue, which the seat arranges.

---

_Generated by [Claude
Code](https://claude.ai/code/session_01CBAfsWMSfM3EToQGVStEcp)_

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: objectstack-fleet[bot]

.changeset/22756-webhooks-redeliver-member.md

@@ -0,0 +1,13 @@
+---
+"@objectstack/plugin-webhooks": minor
+---
+
+feat(plugin-webhooks): the `webhooks` service serves the redeliver door from a web-standard `Request` (`IWebhookService.handleRedeliver`), and the redelivery veto no longer waits for realtime (#22756)
+
+Clause-②: yes
+
+- **What is new.** The plugin registers the `webhooks` service and implements its optional `IWebhookService.handleRedeliver(request)` member: the webhook redeliver door, `POST /api/v1/webhooks/redeliver`, from a `Request` to a `Response`, with no raw app, no Hono context and no `http.server`. A hosted tenant kernel owns no socket, so this is how the door reaches it; its one caller is the runtime HTTP dispatcher's `POST /webhooks/redeliver` domain, which lands separately.
+- **One door, two faces.** The member and the self-hosted route run the same code: the same session check, the same active-organization tenant, the same replay through `MessagingService.redeliverHttp` and the same answer for every outcome. For the same request they answer the same status, the same `Content-Type: application/json` and the same bytes: `401 UNAUTHENTICATED`, `400 INVALID_REQUEST`, `400 MISSING_REQUIRED_FIELD`, `404 RESOURCE_NOT_FOUND`, `409 DELIVERY_NOT_ELIGIBLE`, `409 DELIVERY_NEVER_SENT`, `500 INTERNAL_ERROR`, or `200` with the row's id and new status.
+- **The redelivery veto is installed wherever the door is.** The veto that refuses to replay a webhook delivery whose subscription is gone, or whose signing secret cannot be recovered, now needs only the data engine and the messaging service. It used to be installed only when realtime was present and `autoEnqueue` was not `false`, so on other kernels such a delivery was replayed. It is now refused with `409 DELIVERY_NOT_ELIGIBLE` there too. The `webhooks` service is registered only where the veto is installed.
+- **The route is found under `http.server` too.** The plugin reads `http.server` first, then the `http-server` alias. A host that registers its server only as `http.server` now gets the self-hosted route; one that registers it under both names gets it once.
+- **The self-hosted route is otherwise unchanged.** Where it was mounted before, it answers the same status, headers and bytes for the same request, and it is never mounted twice.

packages/plugins/plugin-webhooks/package.json

@@ -45,6 +45,7 @@
     "@objectstack/metadata-core": "workspace:*",
     "@objectstack/objectql": "workspace:*",
     "@types/node": "^26.6.3",
+    "hono": "^4.13.9",
     "tsx": "^4.23.15",
     "typescript": "^6.0.3",
     "vitest": "^4.1.11"