finding: the QA assertion field path is documented as body.*, but HttpTestAdapter returns the parsed body with no body wrapper — every documented path resolves to undefined

Observation-class finding, measured while running the blast radius for #7256 (PR #7348). Filed unassigned, no pm:queue — for triage to grade. Deliberately not folded into #7256's PR: the fix there is one else branch in packages/core/src/qa/runner.ts, and this is a mismatch between packages/core/src/qa/http-adapter.ts and the describe() text in packages/spec/src/qa/testing.zod.ts — a different file and a different call (change the adapter, or change the convention).

The mismatch

TestAssertionSchema.field documents its own convention, and the generated reference repeats it:

field: z.string().describe('Field path in the result to check (e.g. "body.data.0.status")')

TestRunner.assert resolves that path against whatever the adapter returned. HttpTestAdapter.handleResponse returns the parsed response body itself:

const contentType = response.headers.get('content-type');
if (contentType && contentType.includes('application/json')) {
    return response.json();     // ← the body, not { body: … }
}
return response.text();

There is no body key anywhere in the value the runner sees, and none in the platform's own response envelope either (ResponseEnvelopeConfigSchema wraps into data + meta, never body). So a suite written to the documented convention — field: "body.data.0.status", field: "body.status" — resolves every assertion path to undefined.

Why it surfaced now, and why it is worth a card

Until #7256 this was half-invisible. With equals/not_equals/not_null a body.* path already failed loudly, so an author hit it on their first run and worked the convention out by trial. With contains the undefined fell out of the switch and the assertion passed, so a body.* contains reported ✅ forever and nobody learned anything.

Measured against a stand-in server returning {"ok":true,"data":{"total":0},"name":"acme corp","tags":["acme","globex"]}, same fixture, same HttpTestAdapter:

scenario	base `f3f855ac`	with #7256's fix
`contains` on `body.data.items` (the documented convention)	✅ (asserting nothing)	❌ `got undefined`
`contains` on `data.items` (path absent)	✅ (asserting nothing)	❌ `got undefined`
`contains` on `data.total` (a number)	✅ (asserting nothing)	❌ `got number`
`contains` on `tags` (a real array)	✅	✅
`contains` on `name` (a real string)	✅	✅

So #7256 makes the mismatch visible rather than causing it — the documented-convention row is a test that was never running. That is the right outcome, but the author who now sees red is following the schema's own example, and the error will tell them the path did not resolve without telling them the documentation is what's wrong.

The call, which is a real choice

Fix the convention — correct the describe() example (and the generated reference) to the shape the adapter actually returns (data.0.status). Cheapest, and makes the docs true today. Touches packages/spec (domain:spec-surface by the accept-face test: describe text only, no accept/reject change).
Fix the adapter — have HttpTestAdapter return { status, headers, body } so body.* and status both work, which is the shape the field example implies and what most HTTP test runners hand back. Strictly better for authors (an assertion on the status code is impossible today), but it is a behaviour change to a published adapter and would break any suite written against the current shape.

Option 2 is the better surface and option 1 is the smaller change; picking between them is a protocol call, not a cleanup, which is why this is filed rather than fixed.

Dedup

Searched open issues and PRs for HttpTestAdapter, the field path convention and the body prefix before filing — nothing names this. #7256 names the contains fall-through only.

Findings triage: escalated finding to needs-user-decision (kept domain:engine-core).

The fork is genuine and two-way: (1) fix the documented convention (testing.zod.ts describe text to the adapter's actual shape, e.g. data.0.status — lands domain:spec-surface by the accept-face test), or (2) change HttpTestAdapter to return { status, headers, body } — the better authoring surface (status assertions become possible) but a behavior change to a published adapter that breaks suites written against the current shape. Picking one is a protocol call, as the filer said.
Timeliness (why escalate now rather than hold): qa(core): the contains assertion SILENTLY PASSES when the actual value is neither an array nor a string #7256's fix (PR fix(core): fail the QA contains assertion on a non-evaluable actual instead of silently passing (#7256) #7348, in flight) flips the failure mode from silent-pass to loud-red for every suite written to the documented body.* convention; the ruling is best made before or with that landing, so authors hitting red get docs that tell the truth.
Premise verified on origin/main @ 60f0dd8: HttpTestAdapter.handleResponse returns the parsed body unwrapped; the describe() example says body.data.0.status.
Recommendation: option 1 now (docs true today, zero break), with option 2 as a spec proposal if author demand shows — mirrors the Object-form params on a type:'url' inline action is a THIRD meaning of the key (interpolation scope + newTab) with no authorized spelling (observation) #6828-family "no key without demonstrated pull" pattern.
Dedup: qa(core): the contains assertion SILENTLY PASSES when the actual value is neither an array nor a string #7256 names the contains fall-through only; nothing else names the field path convention.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

Generated by Claude Code

Maintainer ruling — 2026-08-11. From the four-lens decision review (platform long-term coherence / measured business pull / AI-agent error-resistance / startup scope discipline); the maintainer accepted the recommendation set in full.

Ruling: docs follow the adapter. Fix TestAssertionSchema.field's describe() text (and the regenerated reference) to the adapter's real convention: paths resolve against the parsed response body root — no body. wrapper. body.* never worked (equals-class assertions always failed loudly; only contains silently passed), so there is no stored-suite compatibility to preserve.

Optional follow-up, owning lane's choice as a separate card: a loud runner guidance error on body.-prefixed paths.

Routing correction by the triage seat: domain:engine-core → domain:spec-surface — the landing is describe/JSDoc text in packages/spec, acceptance face unchanged (standing instruction ③). State: needs-user-decision → pm:queue.

Recorded by the triage seat Routine (#5474 pilot) on the maintainer's direct instruction — ruling record, not a claim.

Generated by Claude Code

Claim — spec-surface seat (session session_016R9de1FqP7NvwKvqXi92Gh, seat card #6298). pm:queue → pm:dispatched.

Branch: claude/issue-7365-qa-field-path-docs
Worktree: dev-owned per-task worktree ../objectstack-issue-7365
Domain: domain:spec-surface (own lane, per the ruling's routing correction — describe/JSDoc text, acceptance face unchanged)
File surface: packages/spec/src/qa/testing.zod.ts (TestAssertionSchema.field describe text) + the regenerated reference page(s) under content/docs/references/** that render it
Container & model: os-dev subagent (in-process), model claude-sonnet-5 (S-mechanical: the ruling supplies the exact convention; regen is scripted)
Serial constraints: disjoint from everything in flight — The Account app's Notifications entry 403s for every non-admin — no permission set grants sys_inbox_message #7344 (default-permission-sets.ts), objectui: the Unpublished banner and per-app publish read _unpublished; the App Switcher keeps filtering on hidden (#4829 A1 objectui half) #6955 (objectui), A bad key inside a form field's publicPicker reports the CONTAINER branch's diagnostic — the author is told to fix viewKind/config, not the key they typed #7510 (unclaimed, view.zod.ts union error path — different file). qa(core): the contains assertion SILENTLY PASSES when the actual value is neither an array nor a string #7256/PR fix(core): fail the QA contains assertion on a non-evaluable actual instead of silently passing (#7256) #7348 touches packages/core/src/qa/runner.ts, not the spec text.

Executes the 2026-08-11 maintainer ruling (comment 5248467805, recorded by the triage Routine on the maintainer's direct instruction): docs follow the adapter — the field path convention documents resolution against the parsed response body root, no body. wrapper; body.* never worked, so no stored-suite compatibility to preserve. The ruling's optional follow-up (a loud runner guidance error on body.-prefixed paths) is NOT in this card — the seat will file it separately after this lands (dedup first).

Report channel: issue comment here with the os-dev-report HTML-comment marker at draft-PR time.

Generated by Claude Code

Status: done
Branch: claude/issue-7365-qa-field-path-docs
PR: #7565 (draft)
premise_still_valid: true

Summary

Executed the 2026-08-11 maintainer ruling (comment 5248467805) verbatim: "docs follow the adapter." Fixed TestAssertionSchema.field's .describe() text in packages/spec/src/qa/testing.zod.ts to state the real convention — paths resolve against the parsed response body root (the value HttpTestAdapter.handleResponse returns), no body. wrapper. Regenerated content/docs/references/qa/testing.mdx to match. Acceptance face unchanged: field stays z.string().

Before → after describe text

packages/spec/src/qa/testing.zod.ts:45 (TestAssertionSchema.field — the card's named target):

before: 'Field path in the result to check (e.g. "body.data.0.status")'
after: 'Field path in the result to check, resolved against the parsed response body root — no "body." prefix (e.g. "data.0.status")'

Faces measured

Grepped packages/spec/src/qa/testing.zod.ts and packages/spec/src/qa/ for body. path examples. Found 2 faces carrying the claim, both in testing.zod.ts (no separate JSDoc block above either property):

testing.zod.ts:45 — TestAssertionSchema.field describe text. Disposition: fixed (the card's primary target, text above).
testing.zod.ts:58 — TestStepSchema.capture describe text, a sibling doc comment repeating the same body.* claim ('Map result fields to context variables: { "newId": "body.id" }'). Verified against TestRunner.runStep (packages/core/src/qa/runner.ts:152-156): capture paths are resolved with the same getValueByPath(result, path) call as field — same root, same bug. Disposition: fixed — 'Map result fields to context variables, paths resolved against the response body root (e.g. { "newId": "data.id" })'.

No other body. path examples found elsewhere in packages/spec/src/qa/ (index.ts has a module-level JSDoc with no path examples).

Regeneration evidence

$ pnpm --filter @objectstack/spec gen:schema   # required first; json-schema/ is gitignored
✅ Successfully generated 1590 schemas.

$ pnpm --filter @objectstack/spec gen:docs
✅ Generated 231 files

$ git diff --stat content/docs/references/qa/testing.mdx
 content/docs/references/qa/testing.mdx | 4 ++--

$ pnpm --filter @objectstack/spec check:docs
✅ import examples resolve against api-surface/ (62 accepted gap(s) in the baseline)
✅ 231 generated files in sync with packages/spec

Only content/docs/references/qa/testing.mdx changed (the two table rows for field and capture); no other generated file moved.

Hand-written docs sweep

Grepped content/docs/** (excluding references/ and releases/) for the old convention (body.data, body.status, body.id, and broader TestAssertionSchema/assertions:/field: QA-schema mentions) being taught as an example. No hits — no hand-written page teaches the body.* convention, so no finding to file and nothing to fix outside packages/spec.

Verification output

$ pnpm --filter @objectstack/spec typecheck
✓ check:test-typecheck --self-test — 8 semantic case(s) + the parser hold.
check:test-typecheck: OK — 57 file(s) / 265 error(s) held in test-typecheck-debt.json (pre-existing, shrink-only debt ledger — unrelated to this change)
(tsc --noEmit and check:scripts-typecheck both clean, no output = no errors)

$ pnpm --filter @objectstack/spec test
 Test Files  374 passed (374)
      Tests  9805 passed (9805)
   Duration  319.58s

$ node scripts/check-nul-bytes.mjs
check-nul-bytes: OK (scanned 7001 text file(s) -- 7001 tracked, 0 untracked-not-ignored; skipped 5 binary; no raw ASCII control bytes).

Grepped packages/spec/src/qa/testing.test.ts for TestAssertionSchema/body. — it uses 'body.status'/'body.id' as arbitrary string fixture values parsed by field: z.string() (which accepts any string), and never asserts on the .describe() text itself. Confirmed no test pins the old example string as a description — nothing needed updating there for the suite to stay green, and it stayed green.

Changeset decision

Added a patch changeset for @objectstack/spec (.changeset/qa-assertion-field-path-body-root.md), following the #7444 precedent (docs(spec): state the RLS using grammar...) for describe/TSDoc-only spec-docs fixes that ship in the published package and render into the reference.

Labels

PR currently carries only the Auto-Label bot's size/s. A changeset was added, so skip-changeset does not apply — no label action needed from this side.

Out-of-scope findings

None filed. The ruling's optional follow-up (a loud runner guidance error on body.-prefixed paths in packages/core/src/qa/) is explicitly not this card per the dispatch instructions and was not implemented or touched.

Open questions

None.

Generated by Claude Code

docs(spec): fix QA field/capture path convention — no body. prefix (o…

Navigation Menu

Repository navigation

finding: the QA assertion `field` path is documented as `body.*`, but `HttpTestAdapter` returns the parsed body with no `body` wrapper — every documented path resolves to `undefined` #7365

The mismatch

Why it surfaced now, and why it is worth a card

The call, which is a real choice

Dedup

Summary

Before → after describe text

Faces measured

Regeneration evidence

Hand-written docs sweep

Verification output

Changeset decision

Labels

Out-of-scope findings

Open questions

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

finding: the QA assertion field path is documented as body.*, but HttpTestAdapter returns the parsed body with no body wrapper — every documented path resolves to undefined #7365

Description

The mismatch

Why it surfaced now, and why it is worth a card

The call, which is a real choice

Dedup

Activity

claude commented on Aug 10, 2026

claude commented on Aug 11, 2026

os-help commented on Aug 11, 2026

os-help commented on Aug 11, 2026

Summary

Before → after describe text

Faces measured

Regeneration evidence

Hand-written docs sweep

Verification output

Changeset decision

Labels

Out-of-scope findings

Open questions

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

finding: the QA assertion `field` path is documented as `body.*`, but `HttpTestAdapter` returns the parsed body with no `body` wrapper — every documented path resolves to `undefined` #7365