Repository navigation
feat(eval): add local capture API client harness - #233
Conversation
Public real-client evaluation harness for the local Hunter capture API: - adapter.py: AdapterInterface v1 protocol + sanitized Observation/StepResult (transient context never serialized) - http_client.py: direct HTTP client with same-body finalize retries - scenarios/: lifecycle + negative controls; only place PASS/FAIL is decided; every oracle names its public derivation (HouseScanKit @ alpha 263827e) - faultproxy.py: labeled FAKE_HUNTER fault injection (delay/status/abort/ truncate) with harness_* codes that cannot collide with contract codes - receipts.py: run-manifest.json, append-only checks.jsonl, summary.json into fresh never-overwritten run dirs - traces.py + traces/expected-lifecycle-v1.jsonl: frozen expected trace, subset semantics, provisional lines advisory-only, versioned by file - selfcheck.py: in-process plumbing fake (explicitly NOT backend evidence) covering lifecycle, controls, fault round, receipts, trace gate, leak sweep - runner.py: live-endpoint CLI; exit 0 pass / 1 fail / 2 blocked / 3 bad call - fixtures/gen_synthetic_packet.py: deterministic fully synthetic plan - tests: 17 unit tests incl. planted-drift mutation proof that the trace comparator flags wrong expectations Verification at commit: ruff clean; pytest 17/17; selfcheck PASS (20 PASS, 1 GPU NOT_RUN); runner BLOCKED-path probe exit 2 with correct receipts. All receipts synthetic and sanitized; GPU arms named, never run. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ainst a reachable endpoint Verified repair round on the harness (review feedback on #233): - One sink interface: scenarios and vocabulary emitters write through sink.append(line_dict) only; the runner uses the receipt-backed ChecksSink, the self-check a plain list converted after the run. The previous callable/list mismatch crashed every reachable CLI run. - Honest transport labels: the self-check's in-process plumbing fake is FAKE_HUNTER in every row and in the summary — never REAL_MIRROR_CPU. The runner now requires --backend-mode REAL_MIRROR_CPU|FAKE_HUNTER and carries the declared class on every main-round row. - GPU arms: selfcheck emits named gpu-stub.{objects,vlm,dense} NOT_RUN rows; the self-check NOT_RUN count is restart-recovery + 3 GPU arms (the earlier "1 NOT_RUN (GPU arm)" claim in 3be1c2e's message was wrong — that row was the restart control; README/PR correct the record). - Fault-round receipts use fault.-prefixed ids so the second round can never overwrite the first round's lines. - New tests: the actual runner CLI (argparse → receipts → exit codes) driven over real HTTP against a reachable local endpoint (exit 0, all lifecycle rows PASS, no REAL_MIRROR_CPU rows, GPU rows NOT_RUN), plus backend-mode requirement/rejection tests. 20/20 pass. - selfcheck: 20 PASS + 4 NOT_RUN, all FAKE_HUNTER/GPU_STUB; blocked probe still exits 2 with correct receipts; ruff clean. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
|
Note Review skipped Codemetrics is installed on this repository, but it hasn't been added to your team yet. To enable AI code reviews for this repo, go to your dashboard and add BaseScanning to your team's repositories. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
| packet = { | ||
| "schema_version": "0.4", | ||
| "packet_id": packet_id, | ||
| "files": [f["path"] for f in files], | ||
| "note": "synthetic fixture; no homeowner data", | ||
| } | ||
| plan = { | ||
| "plan": { | ||
| "packet_id": packet_id, | ||
| "create_request": { | ||
| "source": "synthetic-fixture", | ||
| "schema_version": "0.4", | ||
| }, |
There was a problem hiding this comment.
Supplied requests get rejected
The generated plan does not match the published upload contract:
create_requestlackspacketId,formatVersion,tier, anddevice.- File descriptions lack
md5andcontentType. - The finalize body uses
schema_versionandpacket_idinstead of the required packet fields.
A backend following that contract rejects these requests before the intended checks run. Build synthetic requests that match CaptureAPI and Packet04, and make the fake check those shapes too.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/fixtures/gen_synthetic_packet.py
Line: 57-69
Comment:
**Supplied requests get rejected**
The generated plan does not match the published upload contract:
- `create_request` lacks `packetId`, `formatVersion`, `tier`, and `device`.
- File descriptions lack `md5` and `contentType`.
- The finalize body uses `schema_version` and `packet_id` instead of the required packet fields.
A backend following that contract rejects these requests before the intended checks run. Build synthetic requests that match `CaptureAPI` and `Packet04`, and make the fake check those shapes too.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
|
||
| # Public CaptureAPI.terminalStatuses (HouseScanKit/CaptureAPI.swift @ alpha | ||
| # 263827ef): polling stops on any of these. | ||
| TERMINAL_STATUSES = frozenset({"complete", "failed", "expired"}) |
There was a problem hiding this comment.
Finished captures keep waiting
TERMINAL_STATUSES leaves out manual_review and needs_views, which are final states in CaptureAPI.terminalStatuses. A capture ending in either state keeps polling until the default 25-minute budget expires and gets marked BLOCKED. Include both states so the harness reads the result when the published client would.
| TERMINAL_STATUSES = frozenset({"complete", "failed", "expired"}) | |
| TERMINAL_STATUSES = frozenset({"complete", "manual_review", "failed", "expired", "needs_views"}) |
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/scenarios/__init__.py
Line: 24
Comment:
**Finished captures keep waiting**
`TERMINAL_STATUSES` leaves out `manual_review` and `needs_views`, which are final states in `CaptureAPI.terminalStatuses`. A capture ending in either state keeps polling until the default 25-minute budget expires and gets marked `BLOCKED`. Include both states so the harness reads the result when the published client would.
```suggestion
TERMINAL_STATUSES = frozenset({"complete", "manual_review", "failed", "expired", "needs_views"})
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| targets = [] | ||
| for entry in entries: | ||
| upload = entry.get("upload") if isinstance(entry, dict) else None | ||
| if isinstance(upload, dict) and isinstance(upload.get("url"), str): | ||
| targets.append( | ||
| {"url": upload["url"], "headers": upload.get("headers") or {}} | ||
| ) | ||
| else: | ||
| targets.append({}) |
There was a problem hiding this comment.
register() drops each answer's path, state, and upload method, then the scenarios pair targets with files by position. The published client matches by path and accepts files already marked committed.
This causes three problems:
- A reordered answer sends bytes to the wrong target.
- A shorter answer silently skips files.
- An already-committed file incorrectly fails upload.
Preserve these fields, match by path, and check that every requested file is accounted for.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/http_client.py
Line: 181-189
Comment:
**Files use the wrong targets**
`register()` drops each answer's `path`, `state`, and upload `method`, then the scenarios pair targets with files by position. The published client matches by `path` and accepts files already marked `committed`.
This causes three problems:
- A reordered answer sends bytes to the wrong target.
- A shorter answer silently skips files.
- An already-committed file incorrectly fails upload.
Preserve these fields, match by path, and check that every requested file is accounted for.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| truncate: bool = False, | ||
| ) -> tuple[Status, str]: | ||
| """Full mini-lifecycle for one tampered file; decision made at commit.""" | ||
| res = adapter.create(plan.create_request, plan.packet_id) |
There was a problem hiding this comment.
Controls change the main capture
Both tampered-upload controls reuse the main capture's packet_id after the lifecycle has finalized it. Create replay therefore returns the same capture instead of a fresh test capture. These controls either overwrite that capture's files or hit its closed state without testing byte checks. Give each integrity control a distinct packet identity and matching request body.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/scenarios/controls.py
Line: 179
Comment:
**Controls change the main capture**
Both tampered-upload controls reuse the main capture's `packet_id` after the lifecycle has finalized it. Create replay therefore returns the same capture instead of a fresh test capture. These controls either overwrite that capture's files or hit its closed state without testing byte checks. Give each integrity control a distinct packet identity and matching request body.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| conn = http.client.HTTPConnection( | ||
| proxy._origin_host, proxy._origin_port, timeout=60 | ||
| ) |
There was a problem hiding this comment.
HTTPS requests lose encryption
FaultProxy accepts an HTTPS origin but always forwards with HTTPConnection. Running --with-faults against an HTTPS endpoint sends the request body and headers without TLS. A TLS-only server then returns an error instead of the intended fault check. Preserve the origin scheme and use HTTPSConnection for HTTPS.
How this was verified: The HTTPS endpoint flows directly into the proxy, which sends the original body and headers through HTTPConnection without a scheme check.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/faultproxy.py
Line: 123-125
Comment:
**HTTPS requests lose encryption**
`FaultProxy` accepts an HTTPS origin but always forwards with `HTTPConnection`. Running `--with-faults` against an HTTPS endpoint sends the request body and headers without TLS. A TLS-only server then returns an error instead of the intended fault check. Preserve the origin scheme and use `HTTPSConnection` for HTTPS.
**How this was verified:** The HTTPS endpoint flows directly into the proxy, which sends the original body and headers through `HTTPConnection` without a scheme check.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
|
||
| # frozen expected trace comparison (REAL_MIRROR_CPU round); advisory | ||
| # (provisional-line) mismatches are reported, never hard failures | ||
| trace_path = Path(__file__).resolve().parents[2] / "traces" / "expected-lifecycle-v1.jsonl" |
There was a problem hiding this comment.
Installed self-check misses its trace
hunter-selfcheck finds its trace relative to the source checkout, but the package includes only packages under src and does not ship the root-level trace. A normal, non-editable install therefore reports “frozen trace missing” even when the checks pass. Ship the trace inside the package and load it as a package resource.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/selfcheck.py
Line: 399
Comment:
**Installed self-check misses its trace**
`hunter-selfcheck` finds its trace relative to the source checkout, but the package includes only packages under `src` and does not ship the root-level trace. A normal, non-editable install therefore reports “frozen trace missing” even when the checks pass. Ship the trace inside the package and load it as a package resource.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| rng_state = rng.getstate() | ||
| seed_int = int.from_bytes(hashlib.sha256(tag.encode()).digest()[:4], "big") | ||
| rng.seed(seed_int) | ||
| payload = bytes(rng.randrange(256) for _ in range(n)) | ||
| rng.setstate(rng_state) |
There was a problem hiding this comment.
Different seeds reuse file bytes
noise_bytes() replaces the supplied random seed with a hash of the path, so changing --seed never changes file contents. The test named test_different_seed_changes_content checks only packet_id, so it misses this. Include the supplied seed when generating bytes and compare actual file contents in that test.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/fixtures/gen_synthetic_packet.py
Line: 25-29
Comment:
**Different seeds reuse file bytes**
`noise_bytes()` replaces the supplied random seed with a hash of the path, so changing `--seed` never changes file contents. The test named `test_different_seed_changes_content` checks only `packet_id`, so it misses this. Include the supplied seed when generating bytes and compare actual file contents in that test.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| out = Path(args.out) | ||
| out.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n", encoding="utf-8") |
There was a problem hiding this comment.
The documented first fixture command writes under runs/fixtures, but that directory does not exist in a fresh checkout and the generator does not create it. write_text() raises FileNotFoundError before users can run the harness. Create the output parent directory or add the required setup command.
| out = Path(args.out) | |
| out.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n", encoding="utf-8") | |
| out = Path(args.out) | |
| out.parent.mkdir(parents=True, exist_ok=True) | |
| out.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n", encoding="utf-8") |
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/fixtures/gen_synthetic_packet.py
Line: 86-87
Comment:
**First fixture command fails**
The documented first fixture command writes under `runs/fixtures`, but that directory does not exist in a fresh checkout and the generator does not create it. `write_text()` raises `FileNotFoundError` before users can run the harness. Create the output parent directory or add the required setup command.
```suggestion
out = Path(args.out)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n", encoding="utf-8")
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| except SystemExit as exc: # argparse + plan errors | ||
| code = exc.code if isinstance(exc.code, int) else 3 | ||
| return code |
There was a problem hiding this comment.
main() returns argparse's usage-error code 2 unchanged, although the documented contract reserves 2 for a blocked run and 3 for bad invocation. Missing required flags or an invalid --backend-mode therefore look like an unavailable endpoint to callers. Map usage errors to 3 while preserving the successful help exit.
| except SystemExit as exc: # argparse + plan errors | |
| code = exc.code if isinstance(exc.code, int) else 3 | |
| return code | |
| except SystemExit as exc: # argparse + plan errors | |
| return 0 if exc.code == 0 else 3 |
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/runner.py
Line: 223-225
Comment:
**Bad commands look blocked**
`main()` returns argparse's usage-error code 2 unchanged, although the documented contract reserves 2 for a blocked run and 3 for bad invocation. Missing required flags or an invalid `--backend-mode` therefore look like an unavailable endpoint to callers. Map usage errors to 3 while preserving the successful help exit.
```suggestion
except SystemExit as exc: # argparse + plan errors
return 0 if exc.code == 0 else 3
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| 1. the runner completes end to end (exit 0) including the FAKE_HUNTER | ||
| fault round through the local proxy; | ||
| 2. every lifecycle.* and control.* check lands PASS except the |
There was a problem hiding this comment.
Self-check overstates its coverage
The self-check says it verifies the runner end to end, but it calls scenarios directly with a list and writes receipts afterward. It never exercises runner.run(), its reachability gate, or ChecksSink. The added runner tests cover some of that separately. Either make the self-check use the runner or narrow its claim so operators know what a green self-check proves.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/hunter-local-ingest/src/hunter_local_ingest/selfcheck.py
Line: 9-11
Comment:
**Self-check overstates its coverage**
The self-check says it verifies the runner end to end, but it calls scenarios directly with a list and writes receipts afterward. It never exercises `runner.run()`, its reachability gate, or `ChecksSink`. The added runner tests cover some of that separately. Either make the self-check use the runner or narrow its claim so operators know what a green self-check proves.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
💡 Codex ReviewEvery plan produced by the documented generator sends only The generated file descriptors omit the nonoptional The published The published BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/http_client.py Lines 210 to 213 in 04e654c When a register response supplies an This condition passes whenever Both corruption controls call create with the lifecycle plan's existing idempotency key. A conforming service therefore returns the already-finalized capture from the main run, after which registration may be refused or may mutate reused state; neither outcome independently tests corrupt or truncated data. Give each control a distinct packet ID and matching create body. When an operator supplies BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/runner.py Lines 113 to 116 in 04e654c The manifest stores the complete command line verbatim. For example, credentials embedded in an endpoint URL or a secret-bearing AGENTS.md reference: AGENTS.md:L30-L32 The documented first-run command writes to BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/runner.py Lines 149 to 157 in 04e654c The main runner executes lifecycle and control scenarios but never calls Any commit reply with an empty or absent BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/faultproxy.py Lines 123 to 125 in 04e654c When BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/runner.py Lines 220 to 225 in 04e654c The CLI help and README promise exit code 3 for a bad invocation, but argparse raises BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/runner.py Lines 123 to 125 in 04e654c The runner records BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/http_client.py Lines 181 to 187 in 04e654c The register parser discards each response entry's The sanitizer copies any string or number found under an allowlisted key without validating its format or origin. A malformed or compromised endpoint can therefore place a credential, signed URL, homeowner text, or private identifier in fields such as AGENTS.md reference: AGENTS.md:L30-L32 BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/http_client.py Lines 158 to 160 in 04e654c Create is issued with zero retries, as are health, register, events, and result, while the published uploader treats transport failures, 408, 429, and almost all 5xx responses from every API step as transient and retries after backoff. A service that returns one recoverable 429 or 500 therefore succeeds for the real client but is recorded as a harness failure, so the harness is not evaluating the client behavior it claims to mirror. BaseScanning/tools/hunter-local-ingest/src/hunter_local_ingest/runner.py Lines 180 to 183 in 04e654c The only runner-configured fault is a 300 ms create delay, but no check asserts elapsed time or even inspects the proxy's hit count, and the delay remains well below the client's timeout. Removing or bypassing the injection produces the same PASS rows and receipt text, so ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed head 04e654ca6af4c32856cfc5e152486654177aa22a against task apps-basescanning-hunter-harness-f1.
tools/hunter-local-ingest/src/hunter_local_ingest/observations.py:57–68accepts arbitrary response strings and nested event values under allowed names.scenarios/__init__.py:160–162writes those strings into public receipts. A credential or private text returned asstatussurvives unchanged.receipts.py:119–126also records the complete endpoint and CLI arguments, including URL credentials/query capabilities. Validate values and screen every emitted artifact, with independent planted-leak tests, before public export.tools/hunter-local-ingest/fixtures/gen_synthetic_packet.py:57–69produces a create body without the publishedpacketId,formatVersion,tieranddevice;scenarios/__init__.py:47–57omits registrationmd5andcontentType. Compare the publicios/HouseScanKit/Sources/HouseScanKit/Upload/CaptureAPI.swift:20–31,48–64. The permissive self-check fake accepts these incomplete requests, so its pass cannot establish real API conformance.tools/hunter-local-ingest/src/hunter_local_ingest/scenarios/lifecycle.py:174passes a partially committed upload whenevercommittedis nonempty. Lines 294–306 pass a result containing onlyschema_version: "0.4", with no shape or capture/run binding; lines 309–321 do not check the claimed null outcome. These oracles can certify a failed or unrelated lifecycle. Add decisive partial-commit, malformed-result and wrong-identity mutations.tools/hunter-local-ingest/src/hunter_local_ingest/scenarios/controls.py:147–168never executes restart/recovery, even with a restart command. Required expiry, changed-body idempotency, body-abort and wrong-run controls and real pinned-backend measurements are also absent. The runner's zero exit permits NOT_RUN required controls (runner.py:197–217). Treat incomplete acceptance as incomplete rather than successful evaluation.
Criteria: met 2/7: owned-path scope and explicit fake/GPU-stub labels. Partial: API seam/lifecycle, fault controls and mutation tests. Missing: safe public export and real pinned-backend comparative evidence. No committed real-service receipts establish the latter; current CI has no harness pytest/self-check job. No local tests were run.
The PR body describes older head 3be1c2e; its callable-sink and fake-label findings are fixed at this head. Refresh Proof/Handoff after correcting the remaining blockers. This review identifies unsafe generated exports, not an observed committed copy of private backend source.
|
Current head The bot findings are folded into the pending recovery and independent verification brief. Direct source inspection confirms that terminal statuses omit The published client-contract, installed-resource, live-trace and CLI cases need decisive sandbox tests. Existing receipt-privacy and signed-upload findings remain. The repaired sink and fake/GPU labels are progress, not proof that these untested paths work. The original parent exhausted its context and still displayed two Working children. A read-only assessment is approved to recover their checkpoints and establish ownership. It does not authorize a competing writer, a nudge to the old parent, or pushes to this branch. Verified findings go to the existing or explicitly released owner after that handoff. No completed review or green metadata check clears those source/runtime blockers. No branch change, paid reviewer setting or proxy-policy workaround was made by this review. |
|
PR description requirement for Every PR body follows Update the body to the current checkpoint and retained limitations. Add one simple How it works diagram for the create/register/upload/commit/finalize/result flow. If this CLI-only change has no visible app change, mark screenshots and recording N/A with that reason. Do not publish private receipts or imply a live backend run occurred. This requests description and evidence corrections from the existing owner. It does not release child writers, authorize a replacement, or accept the deliverable. If the skill or template has not landed on the working base yet, record that gap rather than claiming it exists. |
Template/skills-only sync per recon section 8: alpha's two commits touch .github/pull_request_template.md and .obvious/skills/pr-writing/** with zero overlap with tools/hunter-local-ingest or ios. Keeps the branch on the current PR template and pr-writing skill for body updates. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed 22e3fe7 against current alpha base 57f9d15 and task apps-basescanning-hunter-harness-f1. The complete delta from 04e654c merges only the template and writing skill from alpha. tools/hunter-local-ingest is byte-identical; the real-base diff remains its 19 paths, with no extra template changes attributed to this task.
Blockers remain:
- tools/hunter-local-ingest/src/hunter_local_ingest/observations.py:57-71 and scenarios/init.py:160-162 preserve arbitrary response strings/nested event values in receipts. receipts.py:119-126 records the full endpoint and CLI arguments. Private text or URL credentials/query capabilities can escape into generated exports. Validate values and screen every output artifact with independent planted-leak controls before public export.
- fixtures/gen_synthetic_packet.py:66-69 omits the published create packetId, formatVersion, tier and device; scenarios/init.py:47-57 omits registration md5 and contentType. The current public CaptureAPI.swift:20-31,48-64 still requires these. The permissive fake cannot establish real conformance.
- scenarios/lifecycle.py:174 passes any nonempty committed list, including partial commits. :294-306 passes schema_version alone without required shape or capture/run identity; :309-321 does not check its claimed null outcome. Add partial-commit, malformed-result, wrong-identity and non-null-outcome mutations that fail decisive assertions.
- scenarios/controls.py:147-168 never executes restart, even when configured. Required expiry, changed-body idempotency, body-abort, wrong-run and pinned real-backend measurements remain absent. runner.py:197-217 permits exit zero with required NOT_RUN controls. Keep incomplete acceptance explicitly incomplete.
Criteria remain 2/7 met:
- Met: owned harness paths relative to the current base; explicit fake/GPU-stub labels.
- Partly met: API seam/lifecycle implementation, fault controls, independent mutation coverage, for the source defects above.
- Missing: safe public export and real pinned-backend comparative evidence. No new run artifact or harness test was added.
Lesser: Proof/Handoff still describes 3be1c2e and the old alpha base, including already-fixed sink and fake-label findings. Update it to this checkpoint, retain the live blockers and add the requested simple flow diagram. The writing skill/template now exist on this base; this merge does not complete the description request. CLI media can remain N/A with its reason.
The PR-size label passes; Aikido skips; CodeRabbit explicitly skips draft review. None runs the harness suite or a real endpoint. Source/CI review only, no local test or backend execution. No private source was copied into this review; unsafe generated exports remain a blocker, not evidence of a committed private-source copy.
…nstalls hunter-selfcheck resolves the frozen trace at parents[2] (the tool root in an editable install). In a non-editable install that path lands outside the package, so the trace never ships and the installed selfcheck always fails with "frozen trace missing". Add the package-data glob (traces/*.jsonl) so traces shipped under the package ride every wheel - including future trace versions (v2) without a second packaging edit. The trace file still lives at the tool root; moving it under src/hunter_local_ingest/traces/ plus a package-relative fallback in the selfcheck loader is a coordinated change outside this child's file scope (selfcheck.py is not mine). The installed-CLI tests landing next pin the installed-selfcheck exit 0 as the end state. Verified: pip wheel -> scratch venv (--no-index, non-editable): installed hunter-local-ingest arms (--help); installed selfcheck currently fails at .../lib/python3.13/traces/expected-lifecycle-v1.jsonl (the loader gap above). pytest 20/20, hunter-selfcheck PASS, ruff clean. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…CLIs New tests/test_packaging.py builds the wheel, installs it into a scratch venv (no editable path, --no-index) and drives the installed console scripts over real subprocesses: - installed hunter-selfcheck exits 0 (end state; xfail(strict) until the trace loader resolves package resources - coordinated selfcheck.py change with the writer; XPASS forces marker removal) - selfcheck from the installed package still writes all three receipt artifacts - installed hunter-local-ingest arms a --with-faults round against a reachable local endpoint: lifecycle.* PASS, fault.* rows present and FAKE_HUNTER-labeled, no REAL_MIRROR_CPU row, GPU arms NOT_RUN - exit-code propagation: 2 usage error, 3 schema-invalid plan file, 2 unreachable endpoint (BLOCKED rows named in summary), 1 forced FAIL against a 500-everywhere endpoint - absent --out parents are created, not a crash One wheel build and one venv per pytest session (module-scoped fixtures), tests serial by design (shared sandbox). Observed and pinned: the schema-invalid plan's "3:" diagnostic string is swallowed by main()'s SystemExit -> int conversion (empty stderr, exit 3); the test pins the code only. Flag names and exit codes pinned at 04e654c: pytest 27 passed + 1 xfailed, ruff clean. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…ument install + labels
tests/test_receipts.py additions (single-owner file):
- test_different_seed_changes_file_bytes: decisive recon S3r check -
file content_hex must vary with --seed, not just packet_id. Currently
xfail(strict): noise_bytes reseeds from sha256(path-tag) alone, so all
seeds emit identical bytes and the manifest's fixture_identity seed=N
overstates content variation. Fix belongs to the generator (protocol
child's file; cross-file ask). XPASS forces marker removal once it
lands.
- test_different_seed_keeps_same_paths_and_sizes: seed variation must
not perturb shape (paths, sizes, roles) - hard pin, passes today.
- test_generator_digest_binds_plan_bytes: recomputes the canonical
sha256_of_plan, binding the recorded digest to the emitted plan bytes.
- test_fresh_run_dir_creates_absent_parents: pins parents=True creation
(absent --out parents are created, not a crash - matches the packaging
subprocess arm).
- test_checks_writer_appends_across_instances: a second ChecksWriter
appends, never truncates (append-only receipt contract).
- test_summary_carries_blocked_notes_and_questions: summary carries
blocked notes, lead questions and notes.
README: add Install (editable vs non-editable, with the honest loader
status gate), correct the exit-code map to the source behavior (2 usage
error or BLOCKED; 3 schema-invalid plan; plan-file diagnostics not
printed), document receipt-contract mechanics (three artifacts,
never-overwrite run dirs, append-only checks.jsonl, digest audit trail),
and add the label vocabulary: REAL_MIRROR_CPU is operator-declared and
never verified, FAKE_HUNTER rows are never backend evidence, GPU_STUB
arms are NOT_RUN without explicit authorization.
Verified at this commit: pytest 31 passed + 2 xfailed; hunter-selfcheck
PASS {PASS: 20, NOT_RUN: 4}; ruff clean.
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Join of obv/apps-basescanning-hunter-harness-r3-install (tip 5cf0726, 3 commits by the packaging child): pyproject package-data glob for traces/*.jsonl, tests/test_packaging.py (wheel -> scratch venv -> installed selfcheck/runner subprocess arms, exit-code propagation), test_receipts strengthening (seed-binds-bytes xfail pinned for the protocol child's generator fix), and README install/exit-code/label documentation. Scope verified 04e654c..5cf0726 touches exactly pyproject.toml, README.md, tests/test_packaging.py, tests/test_receipts.py. Open seams tracked for later joins: faults child moves the trace to src/hunter_local_ingest/traces/ (then packaging needs one glob + xfail-removal commit); malformed-plan exit-3 diagnostic belongs to the protocol child's runner.py. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
The single owner of forbidden-pattern screening for harness receipts: credential material (colon/equals and unambiguous CLI space forms), bearer tokens, signed capability URLs, non-loopback endpoints, private hostnames, key material, source references, non-ASCII/control bytes, and JSON-escape smuggling. redact_command renders a manifest-safe command line (credential flags -> ***, URLs classified and redacted, placeholders never shell-quoted). Core matrix: category coverage with independent mutation controls (case toggles, percent-encoding, JSON escapes, renamed signature params), clean honest-receipt controls, shipped-prose controls, and the exact seam signatures for the faults child to import. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
… integrity chain observations.py: allowed keys now validate VALUES by per-key type and shape (id/version/timestamp/vocab patterns, bool/float/int-range gates, 80-char cap), path lists validate every item, events reduce to validated scalars only, and body KEYS are gated like field names — no raw server-controlled string under any key reaches a receipt. receipts.py: the manifest records a loopback-verified host:port label or "<redacted:endpoint>" (never the raw URL, userinfo, or query), commands and all free-text fields pass redact_text, summary creation runs a post-write screen over manifest/checks/summary and records findings in privacy-screen.json (failed evidence is preserved, never rewritten to look clean), every artifact is digest-chained and sealed read-only at close, and verify_run_integrity detects post-chain tamper. sanitize.py: add safe_failure(argv, message) so the runner's CLI failure output can pass through the same redaction. tests/test_privacy.py: canary/mutation matrix over the real emitted containers (manifest, checks.jsonl, summary.json, failure line) with clean-pass controls, planted-regression screening, tamper/missing-chain detection, seal and rewrite-guard checks. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…screening Join of obv/apps-basescanning-hunter-harness-r3-privacy (tip 9a73d2c, 2 commits by the privacy child): artifact-privacy seam (screen_artifact, redact_command in new sanitize.py), strict recursive observation validation with length caps, post-summary screening of all three artifacts plus failure output, digest-chain + read-only seal on emitted receipts, an independent integrity verifier, and the canary/mutation test matrix (tests/test_privacy.py). Scope verified 04e654c..9a73d2c touches exactly observations.py, receipts.py, sanitize.py, test_privacy.py. Known seams: safe_failure wiring into the runner failure path belongs to the protocol child (runner.py owner); installed-selfcheck end state waits on the faults child's package-relative trace loader, then a packaging glob adjustment. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed 1df1f62c44034ced38799a684be513d72430f240 against the original task and its replacing continuation apps-basescanning-hunter-harness-r2. The new packaging tests and privacy screen improve coverage, but the joined client is still incomplete.
Paths below are under tools/hunter-local-ingest/.
src/hunter_local_ingest/observations.py:85-104,159-168,receipts.py:202-222,340-346, andrunner.py:136-142,202-217: privacy remains blocking. Scalar and unknown-key gates accept arbitrary identifier-shaped strings rather than published enums or identities bound to this synthetic run. Free-text manifest fields use pattern redaction, not strict value validation. For example, a short lowercase private marker understatusorrunIdpasses both gates and the final screen. The runner prints the original endpoint, including userinfo, on an unreachable-endpoint failure. Both runner and selfcheck ignore the findings returned bywrite_summary, so a failed privacy screen does not fail execution. Wire the screen and safe failure output into both actual entry points, and test arbitrary-value canaries in emitted artifacts.fixtures/gen_synthetic_packet.py:57-69andsrc/hunter_local_ingest/scenarios/__init__.py:47-57: create and file descriptions still omit required published fields. PublicCaptureAPI.swift:20-31,48-64requires packetId/formatVersion/tier/device and md5/contentType.http_client.py:181-189,209-213also still discards upload path/state/method, pairs targets by position, accepts supplied Authorization, and adds an unsigned Content-Type. A permissive fake passing these requests does not establish public-protocol conformance.src/hunter_local_ingest/scenarios/lifecycle.py:174,294-321: commit passes for any nonempty committed list, result passes for a bare schema_version of 0.4, and the failed/expired branch never checks that outcome is null. Partial commits, unrelated results and invalid final bodies can therefore receive PASS. Require bound file/result identities and isolated mutations that fail these exact judgments.src/hunter_local_ingest/scenarios/controls.py:147-168andrunner.py:172-190,197-217: restart is always NOT_RUN even when a command is supplied; required expiry, changed-body replay, body-abort and wrong-run cases remain absent. The only runner fault is a delay with no assertion that it took effect. Required NOT_RUN checks permit exit 0. No accepted, pinned real-backend run or comparative receipts are present.pyproject.toml:25-26,src/hunter_local_ingest/selfcheck.py:399-407, andtests/test_packaging.py:219-242: installed selfcheck still cannot load the frozen trace. The glob points inside the package while the trace remains outside it; the decisive test is strict xfail.fixtures/gen_synthetic_packet.py:25-29also still replaces the seed with a path-only seed, so different seeds yield identical payload bytes. The new tests acknowledge both defects rather than repair them.
Lesser gaps: receipts.py:296-310 accepts an empty, sealed chain with all three required receipts missing. runner.py:63-74,123-125 records the CLI seed without verifying the supplied plan's seed/digest. The PR body still describes the old 3be1c2e checkpoint and obsolete sink/fake-label defects, and lacks the requested lifecycle diagram.
Criteria: 2/7 met. Owned scope and explicit fake/GPU-stub labels are met. Public lifecycle, fault/recovery coverage, strict artifact privacy, and installed immutable/source-bound receipts are partly met. Actual pinned service acceptance is missing. No out-of-scope or workflow edits; a bounded source-copy comparison found no matching private implementation, which does not clear generated exports.
Verification: exact-head source/diffs and the new tests inspected. PR-size labeling passed; Aikido skipped and CodeRabbit explicitly skipped this draft. There is no harness test CI check. Local probes were not run because the shared heavy lease refused admission with exit 75. No real backend, installed wheel or device execution independently verified.
selfcheck resolved the frozen trace via parents[2] (tool root), so a non-editable install exited 1 with "frozen trace missing". The trace now ships where the code looks first: git-moved byte-identical into src/hunter_local_ingest/traces/, resolved package-relative with the tool-root path kept as a fallback. A compat symlink preserves the old path for tests/test_receipts.py's TRACE_V1 constant until its owner updates it. Wheel proof: installed selfcheck exits 0 with [tool.setuptools.package-data] hunter_local_ingest = ["traces/*.jsonl"] (the glob change belongs to the packaging child's commit). Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…faults faultproxy gains four oracle kinds behind per-step keys, each with after/recover_after gates (prove detection, then prove recovery) and an injected_counts ledger recorded before the faulted response is written — by the time a client can observe a fault, the counter must show it. - truncate_response: declares a plausible Content-Length and closes short of it; a reading client must surface the framing mismatch (http.client IncompleteRead), never parse a short body. Recon 3t. - expired_url: canned 403 harness_url_expired on the storage PUT - the published client's re-register trigger (CaptureUploader 403 path). - capture_not_found: canned 404 with the published code, the one deliberate exception to the harness_* convention (the recovery contract keys on that exact code; FAKE_HUNTER labels keep it from reading as real evidence). Recon 3n. - retry_finalize: injects a failed event carrying data.next == "retry_finalize" into a well-formed events answer (2.5); passes through bodies that cannot carry the signal. Response-mutation helpers are module-level pure functions; unit tests cover each kind's wire behavior, both gates, and the empty-payload framing mismatch. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Four fault rounds now run the full lifecycle through the proxy and are asserted for detection, not just execution: truncated response framing (commit BLOCKED with the IncompleteRead signature), expired signed-URL 403 (upload FAILs and the commit reports empty committed), canned capture_not_found 404 on commit, and a retry_finalize event injection whose failed event must appear in the events observation. A round whose fault never fired, or whose faulted step still reads PASS, is itself a self-check problem (fault_round_problems). control.restart-recovery is no longer a stub: with --restart-command the selfcheck executes the command (60s cap), wipes the fake's in-process state (simulated process memory loss; ids stay monotonic), probes the pre-restart capture, and requires one of the two published outcomes — 2xx for the existing capture, or 404 plus a full lifecycle re-send under the SAME idempotency key. The NOT_RUN stub row is replaced by the real result; without the flag the honest NOT_RUN stays. The fake's storage URLs are now rooted at the request's Host header so proxied rounds receive signed targets that traverse the proxy (direct behavior unchanged); receipt screening is consumed fail-closed from write_summary's post-write screen and verify_run_integrity; the CLI problem output and the manifest's restart command pass redact_text. Verified: pytest 82 passed; ruff clean; hunter-selfcheck exits 0 without the flag (BLOCKED 1, FAIL 3, NOT_RUN 4, PASS 48) and with --restart-command 'true' (NOT_RUN 3, PASS 49; restart row PASS). Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
tests/test_faults.py now covers the restart control end to end: the recovery path (command executes, fake state wiped, pre-restart capture 404s, full lifecycle re-sent under the same idempotency key), the preserved path (2xx probe, replayed idempotency key returns the same capture), and the blocked path (failing command, recovery not attempted). The restart command text itself never reaches a receipt: pattern redaction cannot reliably catch credentials embedded in shell free text (bare -u user:pass args), so selfcheck_manifest_commands and the row observed carry a sha256 fingerprint the operator can match against their own command. The fingerprint is deterministic and auditable. Mutation controls (anti-blind-spot evidence) disable the round evaluator's detections and prove fail-closed behavior: a faulted step recorded PASS, a round whose fault never fired, a wrong failure signature, and a stripped retry_finalize failed event (mutated from a REAL injected observation) are all self-check problems, never silent passes. An honest round evaluates clean. Verified: pytest 91 passed; ruff clean; hunter-selfcheck exits 0 both without the flag (BLOCKED 1, FAIL 3, NOT_RUN 4, PASS 48) and with --restart-command 'true' (NOT_RUN 3, PASS 49); the manifest and the restart-recovery row carry only the fingerprint. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Per the operator's static review of checkpoint 1df1f62: state the limits of the receipts' read-only sealing and digest chain honestly. - Read-only bits are advisory (owner, root, or a later chmod can still write); sealing is a guard rail, not a lock. - The chain detects later modification and cannot prevent it; it binds content after the fact and says nothing about who wrote a file or whether creation was honest - the writer process controls every receipt while producing it, before any seal. - The chain file shares the run dir and owner with the receipts it covers, so a process able to rewrite receipts can rebuild a coherent chain; the machinery assumes the creating process and detects independent, later tampering, it is not a defence against the writer. - Sealing plus a clean screen is not export approval. Also documents the joined integrity surface (privacy-chain.jsonl, privacy-screen.json, verify_run_integrity, summary.json written once and never rewritten) - describes the behavior as joined from the privacy child (9a73d2c), so land this with that join - and the known gap: a never-created receipt is skipped by the screen and unflagged by the verifier today; the pending privacy hardening will make a missing run-manifest.json/checks.jsonl a named, non-green finding in both. Diff confined to README.md. Bar at this commit: pytest 33 passed + 2 xfailed, hunter-selfcheck PASS {PASS: 20, NOT_RUN: 4}, ruff clean. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
The autouse fixture imported observations.reset_expected, which does not exist before the CE1 fix lands, erroring every pre-existing test at setup. The fixture now tolerates the absent registry; binding isolation activates once the fix lands. Red evidence (corrected run) at /home/user/work/hunter-privacy/red-evidence-ce-red.txt: 14 failed, 90 passed, 2 xfailed in 33.49s; ruff clean. All 14 failures are the intended CE1-CE4 controls; the CE3 positive controls (strict loopback hosts, manifest endpoint label) pass and the existing suite is green. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Post-wipe re-application of the lost transport milestone (originally
0a77fb2 on the previous sandbox; new SHA expected). Scope: the direct
HTTP client and its contract tests only.
- adapter.py: interface_version 1.1.0; StepResult.retry_after_s records
the honored/capped Retry-After of transient replies.
- http_client.py: scope-checked base URLs and storage PUT URLs (https
anywhere, plain http loopback-only, no userinfo/query/fragment);
RFC 6750 b64token-validated bearer on the six capture-API steps and
never on storage; storage PUTs copy exactly the signed header set (no
default Content-Type) and refuse signed-header Authorization
(auth_storage_authorization); full transient set {0,408,429,5xx-ex-501}
with exponential 1->30s backoff honoring Retry-After capped at 120s;
commits/finalize keep same-body retries; short-below-Content-Length
responses surface as transport errors (never silently accepted);
redirects never followed; path-keyed register targets with an
in-process step context (capture id, limits, run ids, membership,
event run ids, result views) for scenario oracles.
- modes.py: Transport gains REAL_API and PREPARED labels (docstring
semantics: transport reality only, never a quality/safety claim).
- tests/test_contract.py: 27 client-level contract tests against a
scripted loopback stub origin (scope refusals, credential scoping,
signed-header fidelity, transient classes, Retry-After reporting,
framing, redirect, context views).
Verified at this commit: pytest 47 passed (1:37), ruff clean,
hunter-selfcheck PASS {PASS: 20, NOT_RUN: 4}.
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Post-wipe continuation of the lost milestone-2 work (new SHA expected; the transport checkpoint 3bd71c6 is untouched). - TERMINAL_STATUSES gains manual_review and needs_views (CaptureAPI .terminalStatuses, CaptureAPI.swift:150); ROLE_VOCABULARY is the published seven roles - no mesh (Packet04.swift:235-236). - RegisterFile descriptors now carry base64 16-byte MD5 and contentType (CaptureAPI.swift:48-55, Packet04Producer.swift:13-14). - Register answers match BY PATH (CaptureUploader.swift:460): reordering is irrelevant, committed state short-circuits, PUT+url targets are required else named upload_method_* refusals, answer paths outside the plan are flagged, unacknowledged files re-register bounded at 5 (maxUnacknowledged). - Commit is strict membership: every planned path in committed, notFound/mismatch empty, no unexpected members - a partial commit FAILS (CaptureUploader.swift:578-582). - Storage PUTs: exact signed header set only, 3 concurrent workers, one BadDigest retry on first 400, 403 re-register bounded at maxRefusedPuts=6, expiresAt parsed ISO-8601 with the 60 s requeueExpiring margin before PUT (CaptureUploader.swift:721-726). - Events poll: verdict_ready events bind data.runId nil-or-equal to the finalize runId (CaptureUploader.swift:624); the client view exposes verdict_ready_run_ids. - Result: bounded result_not_ready reads honoring Retry-After, CaptureResult.Response+ResultHead decode, runId binding refuses result_for_another_run, statusesWithoutOutcome null-outcome rule; the invented schema_version demand is gone (the published client never reads it). - traces/expected-lifecycle-v2.jsonl records the corrected frozen expectations (v1 stays byte-frozen); traces.py prefers v2 when present for caller wiring. - tests/test_contract.py: pure-oracle tests for the register round, strict commit verdict, verdict binding, vocabulary, and descriptor fidelity. Scoped gate: pytest 63 passed, ruff clean, selfcheck PASS {PASS: 20, NOT_RUN: 4}. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
… the published client Receipt values now reach disk only through typed schemas (export.py): a string must be a published enum word, a package constant, a digest, or a path or packet id the plan declared. Server ids, unexpected paths and unknown keys become digests. This closes the four reviewed leaks at 1df1f62: shape-valid server ids and paths, separated --token values in recorded commands, the 127.0.0.1.evil.com lookalike, and a summary that screened clean with no manifest or checks file. The client now follows HouseScanKit at alpha 263827e: published create and register bodies, register targets matched by path, exact commit sets, manual_review and needs_views as terminal statuses, retry_finalize, result identity against the finalized run, CaptureUploader.isTransient retries, and signed PUTs with exactly the signed headers and never Authorization. REAL_MIRROR_CPU is now REAL_API, a declaration the harness cannot verify. Frozen traces ship as package data, the generator lives in the package, and the installed self-check exits 0, so both strict xfails are gone. Controls use their own packet ids; the fault round covers retry, 501, abort and truncated responses; restart recovery runs a no-shell command. 189 tests pass, including installed console scripts run from outside the source tree. Co-Authored-By: Claude Code <noreply@anthropic.com>
…3d5fc9 Four runs from the wheel built out of 23d5fc9, installed non-editable and run as console scripts outside the source tree: the self-check (PASS), a declared REAL_API endpoint with nothing listening (BLOCKED, exit 4), a fake that commits tampered bytes (the tamper controls FAIL, exit 1), and a fake that plants a marker in every response (the marker occurs 0 times in the receipts). index.json pins every artifact's sha256, chain heads and the installed source digest, which equals the committed tree's. Co-Authored-By: Claude Code <noreply@anthropic.com>
WIP checkpoint per the 30-minute checkpoint rule. - observations.py: published-enumeration allowlists (status/code/type/next/ decision/schema_version), run-bound expected-value registry (bind_expected), digest reduction for unbound identifiers and paths, bounded digests for unexpected field names; module docs updated. - tests/test_privacy.py: round-one observation tests re-anchored to the new value policy (published raw / bound verbatim / digested otherwise). Control status at this commit: CE1 green; CE2 (3), CE3 (2), CE4 (4) controls still RED by design - the sanitize.py and receipts.py source fixes for those counterexamples are the next commit. Scoped run: 9 failed, 62 passed (all 9 are the intended red CE2-4 controls). Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed bf3bee9 against apps-basescanning-hunter-harness-r2. Module paths below are relative to tools/hunter-local-ingest/src/hunter_local_ingest/.
- sanitize.py:333-334,367-381 and export.py:340-363,383-384,418 still admit arbitrary command text by shape. A synthetic marker such as CANARY-label-a1 survives --label in both manifest and command; an unknown flag such as --CANARY-a1 survives restart metadata and usage-error output. Both pass the pattern screen. The task requires strict privacy for command metadata and failure output, not an implicit public-text exception for labels or foreign flag names. Hash or replace these free-text values and add actual emitted-container controls. runner.py:342 also prints the original output path, outside the privacy boundary.
- contract.py:649-660,695-731 does not decode the whole result as the app does. A valid result with outcome.recommendedPlacement set to 7, or a view with criteria:[7], receives no violation here, although the published CaptureResult.swift requires a placement object and [String]. lifecycle.py:349-359 consequently permits PASS for a result the app cannot decode. Validate known optional nested fields and list elements, with isolated mutations that reach the lifecycle judgment.
- plan.py:233,266-267 performs set membership before checking string types. Recompute a plan's digest after replacing create_request.tier with [] or a file's role with []; membership raises TypeError instead of PlanError. runner.py:284-289 catches only PlanError before the protected execution block, so the installed CLI takes the uncaught traceback path rather than exit 3 with sanitized failure output. Guard types before membership and exercise these inputs through the installed CLI.
- scenarios/controls.py:34-57 still lacks a changed-body idempotency replay control and an exercised expiry control. Faults now cover retry, 501, connection abort and truncated response, but the completed matrix remains narrower than acceptance. scenarios/controls.py:187-190 plus receipts.py:214-220 also let an unconfigured required restart remain NOT_RUN while the run verdict is PASS. Keep partial completion distinct from full fault/recovery acceptance.
Lesser findings: fixturegen.py accepts seeds below 2^31, but plan.py:33 requires exactly six decimal digits, so generated plans for seed 1000000 cannot load. The PR body still describes an older checkpoint and lacks the required lifecycle diagram. There is no successful pinned real-service run; the submitted REAL_API case correctly reports an unreachable endpoint as BLOCKED. That remains an acceptance condition owned by the separate runtime task.
Resolved since 1df1f62: response IDs/unknown keys are reduced to digests; privacy findings affect runner exit; mandatory receipts are checked; create/register fields and path matching are repaired; signed PUT Authorization is refused; commits require the full expected set; result run/status identity is bound; controls have separate capture identities; TLS and truncated-response framing are repaired; trace resources ship; seed changes payload bytes. Existing working behavior should be preserved.
Criteria: 3/7 met. Owned scope, explicit fake/GPU-stub labels, and installed source-bound receipt support are met. Public lifecycle decoding, complete fault/recovery coverage and strict artifact/failure privacy remain partly met. Actual pinned service acceptance is missing.
Verification: focused source/diff and regression-test review; all four committed runs' artifact hashes, chains and counts independently checked. Their package source digest matches this exact head, a3df65c9ef00ded7ec00a6a43d6d5f08d27154d147ee49af0837cb554cfc54bc. The author reports 189 passing tests and installed CLI runs; I did not independently execute them because the heavy lease refused admission with exit 75. Submitted evidence is honestly FAKE_HUNTER or BLOCKED, with GPU stages NOT_RUN. No workflow or collaborator-directory changes. A bounded normalized source comparison found no matching private implementation blocks; no private source is included in this review. CI supplies size labelling success and skipped security/draft reviews, with no harness test check.
CE1 (observations) was fixed at 9f0f83c; this commit closes the rest. CE2 - credential coverage across argument forms: - sanitize.py: the separated-form credential pattern now covers every supported flag (token, secret, client-secret, refresh-token, bearer, authorization, credentials, passphrase, api-key/secret, ...) and requires the literal "--" prefix so harness prose ("which credential form ...") cannot trip the screen; the equals-form kv pattern gains "bearer". The structured argv boundary (redact_command) was already flag-complete; redact_text now matches it. - receipts.py: write_run_manifest accepts structured argv lists and renders each through redact_command (pre-joined strings keep the free-text redaction). commands: list[list[str] | str]. CE3 - one strict parsed-IP loopback policy: - sanitize.py: _is_loopback_host parses the host with ipaddress (is_loopback) or accepts the localhost name; the "127." string-prefix test is gone, so http://127.0.0.1.evil.com:8000/v1 is classified private-endpoint by endpoint, command and failure screening alike. Whole-127/8 positives (127.29.0.4) stay allowed. CE4 - mandatory receipts are never silently absent: - receipts.py write_summary: a missing run-manifest.json/checks.jsonl/ summary.json is a named finding and non-green; an empty checks.jsonl is flagged ("empty (no checks recorded)"). - receipts.py verify_run_integrity: mandatory artifacts are required to exist even when the chain has no entry for them; absence is named, never silently green. "chained but missing" is kept for chain-named files. tests/test_privacy.py: the round-one verifier expectation that an empty run dir reports only the missing chain was the CE4 defect encoded as a test; re-anchored to name the missing mandatory artifacts too. Verification (scoped): pytest tests/test_privacy.py -q -> 71 passed; full suite 104 passed, 2 xfailed; ruff check . clean; hunter-selfcheck PASS {PASS: 20, NOT_RUN: 4}; verify_run_integrity on the fresh selfcheck run dir -> INTACT. All four CE control groups green. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
- gen_synthetic_packet.py emits Packet 0.4-shaped finalize packets
(format "house-capture-packet", formatVersion "0.4", camelCase,
FileEntry path/bytes/sha256/role per Packet04.swift:11-12, :239-244)
and the published CreateRequest body (CaptureAPI.swift:20-27:
packetId, formatVersion, tier, flow, device{model, systemVersion,
appVersion}). Roles come from the published vocabulary - no mesh
(Packet04.swift:235-236).
- Payload bytes derive from seed AND path tag, so a changed seed
changes actual file bytes, not just the packet id (recon §3r);
same seed still reproduces the plan byte for byte.
- The tamper descriptor defaults to a published role (controls.py).
- Malformed plan JSON now exits 3 with the schema message instead of
an unhandled traceback (JSONDecodeError is a ValueError).
- tests: fixture determinism (same seed identical bytes, different
seed different bytes), Packet04 packet shape, published create
body, packetId pattern, published roles. Scoped gate: pytest 69
passed, ruff clean, selfcheck PASS {PASS: 20, NOT_RUN: 4}.
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…its 3
Controls no longer touch the baseline round's identity (recon §3s): a
derived_control_identity helper (scenarios/__init__.py) mints a
bounded, pattern-legal packetId per control and copies the create
request for it. The idempotency control replays its OWN key twice
(idempotency is the control); the corrupt/truncated tamper controls
register and upload tampered bytes onto their own captures; the
runner's fault round derives its "-fh" identity and rewrites the
frozen packet's packetId to match its create.
A malformed plan file now exits 3 with the loader's diagnostic:
json.loads moved inside the loader's try (JSONDecodeError is a
ValueError) and main() prints the swallowed SystemExit message
before mapping it to exit 3.
tests: derived-identity isolation + pattern bound, a scripted
run_controls wiring test proving no control create reuses the
baseline Idempotency-Key or packetId, plan-copy safety, and the
malformed-plan exit-3 path. Scoped gate: pytest 74 passed, ruff
clean, selfcheck PASS {PASS: 20, NOT_RUN: 4}.
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
A REAL_MIRROR_CPU/REAL_API/PREPARED main round now records a
trace.conformance row: compare_trace(sink.checks, preferred_trace_path,
transport) runs after the lifecycle and controls. A hard disagreement
FAILs the run; advisory (provisional-line) disagreements are recorded
without flipping the verdict; a missing frozen trace is an honest
NOT_RUN, never a silent pass. FAKE_HUNTER rounds are excluded — the
selfcheck owns fake-vs-trace comparison (recon §3q).
--backend-mode now accepts every operator-declarable label
(REAL_MIRROR_CPU, FAKE_HUNTER, REAL_API, PREPARED); GPU_STUB stays
reserved for the named not-run arms.
tests: compare_trace hard-vs-advisory semantics, and a runner wiring
test — a REAL_MIRROR_CPU run against a dry endpoint exits 1 with a
FAILing trace.conformance row naming the missing trace ids. Scoped
gate: pytest 77 passed, ruff clean, selfcheck PASS {PASS: 20,
NOT_RUN: 4}.
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
A --with-faults round now records a fault.https-proxy row: BLOCKED until a CONNECT-tunneling client transport and a TLS-capable fault proxy exist. The fault proxy fronts plain HTTP origins only (no TLS termination, no upstream CONNECT) and DirectHttpClient has no proxy support, so an HTTPS origin cannot be fronted and no truncated- response framing fault over TLS is reachable. The row names the exact missing condition so no fault receipt can imply https coverage (recon §3t). Scoped gate: pytest 77 passed, ruff clean. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
- Result decoding now checks every present published field with Swift Codable rules, nested ones included: recommendedPlacement, its box, view criteria, reference crops, reasons and criteria. Required fields reject missing and null; optionals accept both. - Every final API URL passes the published scope rule before any header, token or connection. A server-minted id such as ../outside is refused (auth_destination_refused) with nothing sent. - hunter-verify-run and the integrity check report unreadable, non-UTF-8, non-JSON or wrongly shaped artifacts by file and category with exit 5, never a traceback or the file's content. - The tamper controls register and commit in batches of min(10, upload.maxBatch), as the lifecycle does, and aggregate the answers by path. - Files register reports already committed are recorded explicitly, and trace v3 accepts a missing upload or commit answer only when that set equals the plan's declared paths. Controls and faults open fresh captures per run, so a rerun replays only the baseline. Each finding has a test that failed on the unrepaired code (47 failures in the selection) and passes now; the full suite is 268 passed. Co-Authored-By: Claude Code <noreply@anthropic.com>
Supersession per reconciliation rule: every file privacy child B's CE round touches (observations.py, receipts.py, sanitize.py, tests/test_privacy.py and the template/skill/pyproject/README/test_packaging/test_receipts edits) was also rebuilt in the operator's own commits 23d5fc9/bf3bee97/69a0d4d1, which are the newest operator intent. All four content conflicts are resolved to the operator's versions; no child-only addition survives the overlap, so the merged tree is content-identical to 69a0d4d. The merge consumes B's CE work on the record without dropping any operator change. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Supersession per reconciliation rule: the operator's own commits 23d5fc9/bf3bee97/69a0d4d1 are the newest intent for every file the faults child touches (faultproxy.py, selfcheck.py, tests/test_faults.py — both sides performed the same trace move into src/hunter_local_ingest/traces/). All three content conflicts resolve to the operator's versions; D's re-addition of the legacy repo-path trace is dropped in favor of the operator's package-relative importlib.resources loading only. No child-only addition survives the overlap; the merged tree is content-identical to 2848ffe. The merge consumes D's fault-round and restart-control work on the record without dropping any operator change. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Supersession per reconciliation rule: the operator's full README rewrite (23d5fc9/bf3bee97/69a0d4d1) is the newest intent and already covers the packaging child's honest-limits substance — tamper detection vs. immutability, the same-owner chain caveat, and missing/writable/extra files as named findings through the wired hunter-verify-run. The merge keeps the operator README in full and integrates the two packaging-child clauses the operator text lacked (screen findings never delete evidence; sealing plus a clean screen is tamper-evidence, not publish approval), adapted to the operator wording and recorded here. No other file contributes. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Supersession per reconciliation rule: the operator's own commits 23d5fc9/bf3bee97/69a0d4d1 are the newest intent for every file the protocol child's Phase-1 touches (adapter, http client, modes, runner, scenarios, traces, fixture generator, contract tests, lifecycle trace v2 — git rename detection confirmed the child's repo-path v2 belongs at src/hunter_local_ingest/traces/, where the operator's v2 already lives). All eleven conflicts resolve to the operator's versions; no child-only addition survives the overlap, so the merged tree is content-identical to 5f59edd. The merge consumes C's Phase-1 conformance work on the record without dropping any operator change; Phase-2 wiring lands as its own join when the child reports. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed 5c367ad against apps-basescanning-hunter-harness-r2. This head differs from the notified 92c0dee only in README text; the reviewed source and tests are identical. Module paths below are relative to tools/hunter-local-ingest/src/hunter_local_ingest/; test paths are relative to the tool root.
- sanitize.py:333-334,367-381 and export.py:340-363,383-384,418 retain the command privacy gap. A shape-valid synthetic marker such as CANARY-label-a1 survives --label in the manifest and command; --CANARY-a1 survives as a foreign flag in restart metadata and usage-error output. The pattern screen accepts these strings. tests/test_export.py:151 explicitly treats labels as public text, but the task requires strict privacy for arbitrary arguments and failure output. Hash or replace free-text labels and foreign flag names, and test the actual emitted containers. runner.py:341 also prints the original output path outside that boundary.
- plan.py:244,277-278 still checks set membership before string types. A digest-consistent plan with create_request.tier:[] or a file role:[] raises TypeError. runner.py:284-289 catches only PlanError before protected execution, producing an uncaught traceback instead of sanitized exit 3. Guard types first and exercise these mutations through the installed CLI.
- Fault/recovery acceptance remains partial. scenarios/controls.py:34-57 has no expiry control; :182-184 records an unconfigured restart as NOT_RUN, while receipts.py:218-224 allows the overall verdict to be PASS. The labels disclose the omission, but this does not establish the required recovery behavior. Distinguish completed required controls from optional GPU stubs in the acceptance verdict.
Lesser finding: fixturegen.py accepts seeds below 2^31, while plan.py:30 requires exactly six decimal digits; a generated seed-1000000 plan cannot load.
Resolved since the previous review: known nested result fields now follow the published Swift types; final API URLs are checked before authorization or transport; tamper controls honor the advertised batch limit; damaged-receipt verification has sanitized error paths; committed-capture replay has an explicit, exact-set trace branch and fresh control identities. The PR body now includes the diagram and discloses its runtime and CI limits.
Criteria: 4/7 met: owned scope, published lifecycle at source level, explicit fake/GPU-stub labels, and installed source-bound receipt support. Strict artifact/failure privacy and complete fault/recovery coverage remain partly met. Successful pinned real-service acceptance remains missing and belongs to the separate runtime owner.
Verification: reviewed changed source, tests, public Swift types, PR claims and CI. The author reports 268 passing tests; I did not independently run them because the heavy lease refused admission with exit 75 (count limit reached). Committed receipts still bind the earlier package digest a3df65c9ef00ded7ec00a6a43d6d5f08d27154d147ee49af0837cb554cfc54bc; this head's package digest is 345a406a6d20147a861800e7ff5691487acb2db3bb76512c7a7efe7daf1b5264. The body correctly says no receipts were regenerated. CI covers size labeling and skipped security/draft reviews, with no harness test check. Changes stay within the owned tool. A bounded normalized source comparison found no matching private implementation blocks; no private source is included here.
Phase-2 wiring on top of the protocol Phase-1 checkpoint, based on the merged harness branch at 5f59edd. The merged subtree (typed schema receipts, plan v2, netpolicy scope, contract decoders, steps/oracles, fault round, verify, packaging, evidence) is adopted verbatim: the operator's absorb commits supersede the Phase-1 implementations where they overlapped. Recorded reconciliation: - Strict result required-field decoding (deferred in Phase 1) is covered by contract.decode_result - CaptureResult.Response decoded as strictly as the app's JSONDecoder, every present field checked against its published Swift type, status and outcome kind validated against the published vocabularies, and the public digest omitting homeowner content and signed URLs. Not re-added. - The published URL scope rule on every final API URL is netpolicy's parse_api_base / ApiBase.in_scope / upload_url_problem, which supersedes the Phase-1 client scope code. - Batching is steps.batch_size: min(10, upload.maxBatch or 10). - --backend-mode is the operator's REAL_API | FAKE_HUNTER choice set; GPU arms remain NOT_RUN vocabulary, and the frozen trace is v3. The one wiring the merged tree lacked is added here: --api-token. The runner validates the b64token shape before any run directory exists (EXIT_USAGE, safe_failure; the value is never printed), and client_for passes it only to clients bound for the declared endpoint - the fault round's local proxy runs unauthenticated. redact_argv keeps *** in the manifest command and in failure output; the token never enters any receipt (tests scan every artifact string). New tests/test_auth_wiring.py pins the seam end to end against the served fake: bearer on API routes and no credential on storage PUTs, fail without the flag when the endpoint requires one, redacted malformed-token refusal, endpoint-only dispatch, and redaction in both argv forms. Verification: pytest 273 passed, ruff clean, hunter-selfcheck PASS from the editable install and from a fresh non-editable venv ("receipts: screened clean, chained, sealed"). Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Final join. The tip adopts the operator's absorb commits (23d5fc9/bf3bee97/ 69a0d4d) verbatim per the reconciliation rule and adds exactly the one gap the operator tree lacked: the --api-token credential flag wired through the runner (RFC 6750 b64token validation before any run dir with EXIT_USAGE and safe_failure, FLAG_KINDS 'secret' so receipts keep *** placeholders, and endpoint-only dispatch — the fault round's local proxy stays unauthenticated), plus tests/test_auth_wiring.py. The merged tree therefore carries the operator's content in full, the child's wiring, and drops only the child's stale pre-move trace copies (the operator ships traces package-relative at src/hunter_local_ingest/traces/); the alpha-sync PR template and pr-writing skill files, which the child's branch never carried, remain. Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
SamGu-NRX
left a comment
There was a problem hiding this comment.
Verdict: BLOCK
Reviewed f25f279 against apps-basescanning-hunter-harness-r2. The completed handoff requests review, so this review falls within the brief's 60-minute exception. Module paths below are relative to tools/hunter-local-ingest/src/hunter_local_ingest/.
- sanitize.py:333-334,367-381 and export.py:340-363,383-384,418 still preserve shape-valid free text. CANARY-label-a1 survives --label in the manifest and command; --CANARY-a1 survives foreign flags in restart metadata and usage-error output. The pattern screen accepts both. Hash or replace these values and add actual emitted-container controls. runner.py:364 also prints the raw output path outside the required failure-output privacy boundary.
- plan.py:244,277-278 still performs set membership before string checks. A digest-consistent plan with tier:[] or role:[] raises TypeError; runner.py:304-309 catches only PlanError before protected execution. This takes the uncaught traceback path instead of sanitized exit 3.
- Newly exposed by --api-token: http_client.py:27,103 uses an anchored regex with match(), so a token consisting of synthetic-bearer followed by one newline passes validation. Python's $ also matches before a final newline. runner.py:294-303 consequently does not refuse that malformed token before creating the run directory. The published ScopedCaptureHTTP.swift rejects every line break. Use fullmatch and add newline mutations through the installed CLI, asserting exit 2, no run directory and no request. This finding is from source inspection, not an executed probe.
- Recovery acceptance remains partial. scenarios/controls.py:34-57 has no expiry control; :182-184 emits NOT_RUN for an unconfigured restart while receipts.py:218-224 permits overall PASS. selfcheck.py:58-59 supplies a Python no-op as its restart command, so its PASS establishes re-reading retained fake state, not an actual restart. Required recovery acceptance remains unproved.
Lesser finding: generated seeds of 1000000 or greater remain incompatible with plan.py:30's six-digit packet-id pattern. The handoff also says only real-backend acceptance remains; the public defects above still need repairs.
The new wiring otherwise passes the supplied token only to the declared endpoint's clients, leaves storage PUTs and the fault proxy uncredentialed, and redacts token values in command receipts. Both changed files stay within scope. GPU stages remain explicitly NOT_RUN.
Criteria: 3/7 met: owned scope, explicit fake/GPU-stub labels, and installed source-bound receipt support. Protocol and credential conformance is now partly met because of the newly exposed newline-validation path; strict privacy and complete fault/recovery coverage are also partly met. Pinned real-service acceptance remains missing and separately owned.
Verification: inspected both changed files, their tests, published Swift credential validation, current body and CI. The author reports 273 passing tests and fresh wheel runs; independent local tests were refused by the heavy lease with exit 75, count limit reached. The four committed receipts still bind a3df65c9ef00ded7ec00a6a43d6d5f08d27154d147ee49af0837cb554cfc54bc; this head's package digest is c505cdb10b60fc20f8f7b66c0f254d8d3cff1e309d1397bedfc9de29a74d2062. Claimed fresh receipts are not in the submitted tree. Green CI covers size labeling and skipped security/draft reviews, with no harness test check. No private source is included in this review.
…25f279 - A commit answer that lists one path in two of committed, notFound and mismatch is a violation, and the tamper controls also require intact files to stay out of mismatch and notFound. - A batched register or commit round stops at the first invalid answer as FAIL; a later batch with no answer can no longer lower it to BLOCKED. - Inside the tamper controls, a request the client refused or a malformed answer is FAIL; only a missing answer is BLOCKED. - A Swift Double accepts any JSON number that converts to a finite double (10000000000000000 as well as 1e16), independent of the bounds on integers a receipt may export. Wire-level tests now cover the cloud writer's --api-token flag: the token reaches only direct API requests, never the fault proxy, storage or any receipt or output byte, and never a traversal destination. Fail-before on the unrepaired tree: 12 of 27 selected tests failed. Full suite at the result: 303 passed. Co-Authored-By: Claude Code <noreply@anthropic.com>
…ed fault arm, 64-bit Ints Applies three lead decisions on top of eeb22c4. - The capture-API token comes only from HUNTER_API_TOKEN, and only when --api-token-env is given. --api-token is refused in every spelling before parsing, without echoing its value; abbreviations are off, so it can never resolve to the new flag. The token is checked before any run directory or request, recorded nowhere (the command keeps only the flag name), and kept out of the restart command's and git's environment. - With a token, --with-faults sends no proxied request: every fault check is BLOCKED with auth_fault_proxy_not_authorized and the run exits 4 unless something failed. Before, the unauthenticated proxy's 401s read as endpoint FAILs. Without a token the fault round runs as before. - Published Swift Int fields decode as 64-bit integers. A value past 2**53 is recorded as large-int:<digest> instead of failing the decode or the receipt; a run whose event cursor is 2**62 now polls and passes. Fail-before on the unrepaired tree: 36 of 75 selected tests failed. Full suite: 343 passed. Co-Authored-By: Claude Code <noreply@anthropic.com>
…511668 Seven runs from the wheel built out of 1511668, installed non-editable and run as console scripts outside the source tree: self-check (PASS), an unreachable declared REAL_API endpoint (BLOCKED, exit 4), tampered bytes committed (tamper controls FAIL, exit 1), canaries in every response (0 occurrences), a rerun of a committed capture (PASS), an enforced maxBatch of 2 (PASS), and a token-protected API with --with-faults (authenticated baseline PASS, all 12 fault checks BLOCKED with auth_fault_proxy_not_authorized, exit 4, the synthetic token in no artifact). index.json pins every artifact sha256, the chain heads and the installed source digest, which equals the committed tree's. The evidence/23d5fc9 receipts are unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
…summaries Every summary carried a registered lead question saying no CLI flag supplies an API token, which stopped being true with --api-token-env. New runs now write the fixed policy instead: "--api-token-env opts into HUNTER_API_TOKEN for scoped API routes; token-protected proxy fault coverage remains BLOCKED." The amendment, clock and homeowner-text questions are unchanged. The old sentence is kept verbatim in a separate allowlist used only to validate earlier receipts; nothing writes it and no other text is accepted. Tests show a new summary carries the policy, the legacy text validates only verbatim, and both frozen receipt sets (evidence/23d5fc9, evidence/1511668) verify with the current verifier from re-sealed copies, their files unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
… check, exact Swift Ints - Commit batches are judged by the scenario as they arrive: the lifecycle needs every requested path committed; a tamper control needs its victim kept out and every intact file committed and never refused or missing. The first answer that breaks the rule ends the round as FAIL and no later batch is sent, so a later unanswered batch can no longer turn a decoder-valid but wrong answer into BLOCKED. A purely unanswered batch is still BLOCKED, and an expected tamper still passes across batches. - The bearer token must match RFC 6750 b64token as a whole: re.match with $ accepted a token followed by a newline, which then failed only after a run directory existed. Every anchored validator now uses fullmatch. - Swift Int fields decode by value, as Foundation's JSONDecoder does: 5.0 and 1e3 are integers. The response parser keeps decimal literals as exact Decimals, so 4611686018427387905.0 is polled as exactly that cursor. A decimal spelling is also range-checked through a Double, as Foundation does (9223372036854775807.0 is refused). 1.0000000000000000001, which Foundation rounds to 1, stays refused because it is not an integer. Fail-before on the unrepaired tree: 16 of 59 selected tests failed. Full suite: 387 passed. Foundation JSONDecoder, run locally, agrees on 20 of 21 recorded literals; the difference is the deliberate one above. Co-Authored-By: Claude Code <noreply@anthropic.com>
…eel at aa10463 Three runtime conditions from the wheel built out of aa10463, installed non-editable: a decoder-valid but wrong first commit batch followed by an unanswered one ends FAIL after one batch (exit 1); a token ending in a newline is refused before any directory or request (exit 2, no receipts by design, recorded in index.json); and a decimal-spelled cursor above 2**53 is polled exactly (exit 0). The earlier evidence sets are unchanged. Co-Authored-By: Claude Code <noreply@anthropic.com>
The Int rule is a deliberately conservative oracle, not exact Swift Codable parity: Foundation's JSONDecoder rounds 1.0000000000000000001 to the Int 1, and the harness refuses it because the value is not an integer. Run locally, Foundation agrees with the harness on 20 of 21 recorded literals; this is the 21st. The README now says so next to the claim that Int fields decode as Foundation does, and records that decimal spellings are range-checked through a Double. No code changes. Co-Authored-By: Claude Code <noreply@anthropic.com>
fleet-packet: apps-basescanning-hunter-harness-f1
What changed
tools/hunter-local-ingest/runs the published HouseScanKit upload lifecycle against one capture-API endpoint and writes receipts that are safe to publish. It creates a capture, registers and uploads synthetic files, commits, finalizes with a Packet04-shaped packet, follows events to a terminal status and reads the result. Then it runs controls that tamper with uploads, replay a create and read a capture that does not exist. This head isf326a6c8, targetingalpha; the joins since69a0d4d1are listed in the Handoff below. On top off25f2793,eeb22c41adds the third-review fixes,1511668eapplies the lead's decisions on the credential interface, the authenticated fault arm and SwiftIntdecoding,9f125813adds receipts built from1511668e,84ed64a8replaces one stale sentence that every new summary receipt carries,aa10463afixes three runtime findings from the final review,2bafcb01adds focused receipts built fromaa10463a, andf326a6c8documents the one known SwiftIntcompatibility limit in the README; it changes no code. No iOS, server, recon, workflow or other tool file changes.Receipts are built from typed schemas. At
1df1f62the receipt code copied server values under allowed key names, so a shape-valid id, path or key name went straight into a receipt. Now a value reaches disk only throughexport.py, which accepts published enum words, package constants, digests, bounded numbers, and the paths and packet ids the plan itself declared. The decoders incontract.pyturn server ids intoid:<12 hex>, unknown words intounrecognized:<12 hex>, and unexpected paths and keys into digests, so a disagreement stays visible without its content. Homeowner messages, prompts, signed URLs and raw bodies are never recorded. A value outside its schema stops the run with exit 5 before it is written. The regex screen still runs over the finished files as a second check.This closes the four leaks the review reproduced at
1df1f62:{runId: CANARY-id-…, committed: [CANARY-path-…]}and unexpected key names passed into receipts. They now appear only as digests.hunter-local-ingest run --token CANARY-…was recorded verbatim and screened clean. Commands are now recorded argv element by element (redact_argv). This program's own flags render by type, every other value becomes<value>, and a credential-named flag's value becomes***, in separated, joined and quoted forms. argparse errors no longer echo the offending value.http://127.0.0.1.evil.comas loopback. One policy innetpolicy.pynow serves endpoint validation, upload URLs, manifest labels and the screen. It is the published client's rule: https anywhere, plain http only to exactly127.0.0.1,::1orlocalhost.The client follows the published contract. Create and register bodies are
CaptureAPI.CreateRequestandRegisterFile. Register and commit go in batches ofmin(10, upload.maxBatch), in the lifecycle and the tamper controls alike. Register targets are matched by path, so a reordered answer still sends each file to its own target, and a missing, duplicated or unrequested path fails. Commit must return exactly the uploaded set, and an answer that lists one path in two of committed, notFound and mismatch is a violation. Each commit answer is also judged by the scenario as it arrives: the lifecycle needs every requested path committed, and a tamper control needs its victim kept out and every intact file committed and never refused or missing. The first answer that is invalid or breaks that rule ends the batched round as FAIL and no later batch is sent, so a later batch that gets no answer can no longer lower it to BLOCKED. A batch that only gets no answer is still BLOCKED. A file that register reports already committed, as on a replayed or resumed capture, is neither uploaded nor committed again, and both checks record that set.manual_reviewandneeds_viewsend polling asCaptureAPI.terminalStatusessays.retry_finalizere-sends the packet, a null outcome is final only forfailedandexpired, and the result's run id must be the finalized run. The result decoder checks every published field that is present, nested ones included (placement, box, view criteria, reference crop, reasons, criteria), with JSONDecoder's rules: a required field rejects missing and null, and an optional one accepts both. A SwiftDoubleaccepts any JSON number that converts to a finite double, whether written10000000000000000or1e16; bools, strings, NaN and overflow to infinity are rejected. This is separate from the bounds on integers a receipt may export. SwiftIntfields (eventseqandnext,etaS, upload limits) decode as 64-bit integers by value, as Foundation's JSONDecoder does:5.0and1e3are integers, and the response parser keeps decimal literals exact, so4611686018427387905.0is polled as exactly that cursor. One known compatibility limit sits beside that claim: the rule is a deliberately conservative oracle, not exact Swift Codable parity, so1.0000000000000000001, which Foundation rounds to the Int 1, is refused here because the value is not an integer; one too large for a receipt to hold exactly is recorded aslarge-int:<digest>, never clamped or rejected, and a run whose event cursor is 2^62 polls and passes. Retries followCaptureUploader.isTransient, so 408, 429 and 5xx other than 501 are retried with the same body. A storage PUT carries exactly the signed headers. It never addsContent-Type, and a signed set that holdsAuthorizationis refused before sending. Every API request's final URL passes the published scope rule before any header, token or connection, so a server-minted capture id such as../outsideis refused with nothing sent. That closes a mismatch withScopedCaptureHTTP, which refuses such a URL before it asks for the credential; no backend is known to mint such ids. An API token is read only from theHUNTER_API_TOKENenvironment variable, and only when--api-token-envis given; the earlier--api-token VALUEis refused in every spelling without echoing the value. The whole token is checked (a trailing newline, carriage return, space or other control character is refused) before any run directory or request, goes only on API routes of--endpoint, never on a PUT even when storage shares the origin and never to the fault proxy, is kept out of the restart command's and git's environment, and is recorded nowhere (the command keeps only the flag name).Labels say what the harness can see.
REAL_MIRROR_CPUclaimed CPU processing the harness cannot observe, so the real-endpoint label is nowREAL_API, an operator declaration. Every summary recordsbackend_verified: falseandreal_mirror_cpu: NOT_ESTABLISHED. BLOCKED and NOT_RUN rows never count as passing, and the frozen-trace check reports them as not comparable instead of skipping them.Controls and faults test themselves. Each control and fault check uses its own packet id carrying the run's id, so tampering can never land on the baseline capture and a rerun against the same server opens fresh control captures. The tamper controls flip or truncate one file and require the intact files to be committed and never also reported as mismatch or notFound, so a server that commits nothing, or acknowledges and refuses the same file, fails them. Inside a control, a request the client refused or a malformed answer is FAIL; only a missing answer is BLOCKED. The fault round covers a 503 retried once, a 501 not retried, an aborted connection and a truncated response body. The proxy now speaks TLS to an https origin and re-frames a truncated request body. A restart-recovery check runs an operator's restart command without a shell and reads the baseline capture again.
The installed package works on its own. Frozen traces and the plan generator live inside the package, so the non-editable install's
hunter-selfcheckexits 0 and both strict xfails are gone. Plan file bytes now vary with the seed, and the loader recomputes every digest from the bytes it read. Each manifest binds the package source digest, git HEAD and a digest of any uncommitted diff, the fixture digests, the redacted command and the runtime. Exit codes no longer share 2 between a usage error and a blocked run: 0 pass, 1 fail, 2 usage, 3 plan refused, 4 blocked, 5 receipts failed.hunter-verify-runreports an artifact it cannot read, decode or parse, or one of the wrong JSON shape, by file and category with exit 5, never with a traceback or the file's content.Why
The public client needs repeatable checks against the backend that a separate owner runs, and their receipts have to be publishable. The earlier draft could neither keep server text out of receipts nor tell a real exchange from fake plumbing. A pattern screen cannot tell
CANARY-abc123from an honest id, so the safety has to come from what the code is allowed to write, with the screen as a second check.How it works
Server answers pass through a decoder and a schema check before anything is written; the screen and chain run on the finished files.
The chain and read-only modes detect an edit made after the run. They do not stop the directory's owner from rewriting a file and recomputing the chain; the printed chain head is the only anchor.
Verification
All checks ran locally on macOS arm64 with Python 3.13, under the fleet's admission leases.
pytest -qintools/hunter-local-ingest: 273 passed, no xfail or skip — 268 at69a0d4d1plus 5 in the newtests/test_auth_wiring.pyfrom the wiring join. This includes the installed-wheel suite: one wheel, a scratch venv installed--no-deps --no-index, and every console script run as a subprocess from outside the source tree withPYTHONPATHremoved.ruff checkandruff format --check: clean.1df1f62and against this head. At1df1f62all four leaked or were accepted. At this head all four hold.bf3bee97has new tests: a complete result baseline plus Swift type mutations, hostile capture ids over the real transport with a request log showing nothing sent, a fake that enforcesmaxBatch=2on a five-file plan,hunter-verify-runon damaged files as an installed subprocess, and an installed rerun of a committed capture. Run against the unrepaired harness, 47 of the 79 selected tests failed; all pass now.23d5fc9, before the second-review fixes, are committed inevidence/23d5fc9/, with every artifact's sha256 inindex.json. The installed source digest equals the committed tree's. The self-check is PASS. A declaredREAL_APIendpoint with nothing listening is BLOCKED (exit 4). A fake that commits tampered bytes fails both tamper controls (exit 1). A fake that plants a marker in every response passes with 0 occurrences of the marker in the receipts.hunter-verify-runreported OK on all four sealed originals. Receipts were regenerated at each joined head throughf25f2793by the writer's verification bar below; none were regenerated or committed ateeb22c41, because the run was refused a test lease while another fleet job held the only slot; the committedevidence/23d5fc9/originals are unchanged.1511668e(the source of this head;9f125813adds only receipts):pytest -q -p no:cacheproviderintools/hunter-local-ingest(macOS arm64, Python 3.13.3) gives 343 passed, no xfail or skip, andruff checkandruff format --checkare clean. Credential tests inject a fresh synthetic token into the test process or a subprocess environment only. They show the token reaches only direct API requests, is never read without the flag, stops the run before any directory or request when missing or malformed, never reaches the restart command's environment, and appears in no stdout, stderr or receipt byte, also as an installed console script; the retired--api-tokenexits 2 in separated, joined and bare forms without echoing its value. With a token and--with-faults, all 12 fault checks are BLOCKED withauth_fault_proxy_not_authorizedand the run exits 4; ateeb22c41the same setup sent the proxy's requests unauthenticated and recorded the 401s as FAIL (exit 1), which a test then asserted.aa10463a(the source of this head;2bafcb01adds only receipts):pytest -q -p no:cacheprovidergives 387 passed, no xfail or skip, andruff checkandruff format --checkare clean. Failing before: run against the unrepaired tree, 16 of the 59 selected tests failed, covering a decoder-valid but wrong first commit batch followed by an unanswered one (lifecycle and tamper control), a token ending in a newline, and integral decimal spellings of SwiftIntvalues. Foundation's JSONDecoder, run locally with/usr/bin/swifton 21 literals: on the 20 Int64-parity cases the harness decodes or refuses exactly as Foundation does (5,5.0,1e3,-0.0,4611686018427387905.0and4611686018427387905exactly,9223372036854775807accepted,9223372036854775807.0refused as Foundation range-checks it through a Double, values past 64 bits,5.1,1e400, a bool and a string refused, plus sixDoublecases). The 21st is a separate, stricter oracle refusal and not a Swift type failure: Foundation rounds1.0000000000000000001to 1, and the harness refuses it because the value is not an integer. The lead accepted that as deliberate policy.aa10463aare committed inevidence/aa10463/: the wrong first commit batch ends the run FAIL after one batch (exit 1); a token ending in a newline exits 2 with no directory, no request and the token in no output; and a decimal-spelled cursor above 2^53 is polled exactly (exit 0).hunter-verify-runreported OK on both receipt sets. These are three conditions, not the full seven-run campaign;evidence/1511668/stays the full set for its own source.84ed64a8, the change is the text of one registered constant plus tests, so the check was focused rather than the full suite: all oftests/test_export.pyand the privacy checks on shipped constants and summaries, 36 passed, andruff checkandruff format --checkare clean. The new tests show a fresh summary carries the policy text, the legacy text validates only verbatim, and every run in both frozen receipt sets verifies with the current verifier from a re-sealed copy, with the committed files byte-for-byte unchanged.Inttest, including the end-to-end large-cursor run.1511668eare committed inevidence/1511668/, with every artifact's sha256 inindex.json; the installed source digest equals the committed tree's andhunter-verify-runreported OK on all seven sealed originals. Besides the four earlier kinds of run, they cover a rerun of a committed capture, an enforcedmaxBatchof 2, and a token-protected API with--with-faults(authenticated baseline PASS, the fault arm BLOCKED, 0 unauthenticated API requests, the synthetic token in no output or artifact).5c367adchas new tests: commit answers that list one path in two categories (including an intact file in the victim's two-file batch), an invalid first batch followed by an unavailable one for register and commit, controls that meet a refused request or a malformed register answer, and SwiftDoublespellings with negative controls. Run against the unrepaired harness, 12 of the 27 selected tests failed; all pass now.f25f2793(Linux, Python 3.13.14):pytest -q -p no:cacheprovider→ 273 passed;ruff check .→ All checks passed;hunter-selfcheck --out <fresh dir>from the editable install and from a fresh non-editable wheel install → PASS, counts{PASS: 26, NOT_RUN: 3}, receipts screened clean, chained and sealed;hunter-verify-runon the self-check run directory →verify: OK. The non-editable self-check is the mode that failed at the original draft; it passes here. The same bar passed at each intermediate join (2848ffe4,92c0dee6,5f59edd2,5c367adc, 268 tests each), with the Sam Gu author and bot co-author trailer verified on every commit and the remote guard read immediately before every push.Authorization, points storage at a non-loopback http host, never readies the result, or names another run, and a client that never retries or retries 501.Not run:
Risk
mismatchornotFoundof a 2xx commit, or a refused PUT.evidence/23d5fc9/andevidence/1511668/sets still verify; those files were not rewritten and still carry the old sentence. No receipts were regenerated at84ed64a8, soevidence/1511668/remains the receipt proof for the1511668esource.committedstate, as the published client does, and cannot re-check those bytes. The trace's no-request branch only requires that the reported set equals the declared paths exactly.failedorneeds_more_photos. The lifecycle accepts any terminal status; it does not judge the outcome.Checklist
Draft until the lead and reviewer re-review this head. Merging and real-backend acceptance stay with their owners.
Handoff
Done: all five child scopes and the operator's absorb commits are joined onto this branch — privacy CE round
2848ffe4, faults92c0dee6, install5f59edd2, protocol Phase-15c367adc, and protocol Phase-2 (the--api-tokenrunner wiring andtests/test_auth_wiring.py) merged from66763d6easf25f2793. The verification bar is green atf25f2793(273 tests, decisive self-check,hunter-verify-run, ruff, non-editable-wheel self-check). The PR stays draft.Left: real-backend acceptance (a run against the private mirror's CPU backend) belongs to its separate owner and stays BLOCKED until then; GPU, phone, geometry-quality and electrical-safety arms remain NOT_RUN by design; alpha merge stays with its owners.
How to continue: re-run the bar in
tools/hunter-local-ingest(pytest -q -p no:cacheprovider,hunter-selfcheck --out <fresh dir>,hunter-verify-run <run dir>,ruff check .), keep the branch current withorigin/alphabefore editing this body again, and update it in place — never overwrite the human sections or the footer.🤖 Generated with Claude Code