The effect corpus: golden logs replayed by every interpreter

[gold-silver-copper]

The effect corpus — golden logs replayed by every interpreter (stacked on #2443)

Branch feat/effect-corpus, anchored to 355266ccd (the head of #2443). Executes many_rigs/effect-bus/effect-bus-corpus-pr.md. Five commits, each gated on its own:

commit	title
E0	fix(core)!: minted tool-call ids are deterministic per response
E1	fix(effect-log)!: a replay divergence is its own error kind and fails the run
E2	feat(effect-log)!: the log header names the program
E3	test(verify): the effect corpus
E4	test(verify): goldens and their producers are paired; stale goldens refuse

What this PR proves

An agent program recorded once against a cassette transport (real provider bytes over HTTP) produces an effect log that both interpreters of the program — the bus engine and a hand driver of AgentRun — reproduce from the log alone: no provider, no tool, no memory behind any key. Reproduce means the record as data — every dispatch's kind, outcome and, for a stream recorded with its events, its event sequence — position by position. Six such goldens are committed under crates/rig-verify/fixtures/; a change in what a program asks or is answered fails the replay naming the record and the JSON pointer of the difference, and a golden of another program is refused before the first dispatch.

Anchor-table findings (verified before writing code)

• divergence() on the anchor already compared the whole payload as data for every family except tool calls, which compared name and args only (the prompt's table said "family alone"; that row was stale). E1 compares the tool context too and changes nothing else.
• A tool replay divergence was reachable in test output, but indirectly: the tool replayer's Internal report became a failed tool result the model saw, the history changed, and the model replayer diverged one record later. E1 adds ErrorKind::Divergence and the engine fails the run at the tool record.
• Adapters already minted stream-side tool ids deterministically (SyntheticIds → BlockId::Minted); the random sources were ToolCallId::for_provider(None) (every id-less wire: gemini streaming and unary, ollama, the responses-API bridge), the two hand-built message constructors and rig-agent's streamed-repair diagnostic. E0 replaces all of them.
• The log was already in dispatch order (the recorder opens the slot at dispatch).

Theory, and where each obligation landed

obligation	where
Algebraic handlers: an effect is compared whole (operation + arguments), the handler's answer is the record	divergence compares every family's payload as data, tool context included (E1)
Required effect row (Leijen): a program's effect signature is part of its type	LogHeader::required, compared as a whole and against the served handlers (E2, E4)
Deterministic orchestrator, nondeterministic inputs in the record (Durable Functions, Temporal, RecPlay)	nothing the engine mints is random (E0, guard no_random_ids); hooks are program, tools are record (E2 header names the hooks; E3 replays tools)
Deep handlers: a stream is one operation with many answers	record_effects_with_events() keeps a streamed dispatch's events; the oracle compares the sequence (E3)
Event sourcing: the record is append-only, never edited	goldens re-recorded under RIG_REGENERATE_GOLDEN=1, paired with one producer each, stale goldens refuse (E3, E4)

Rulings (from the prompt, as applied)

1. Tool-call ids: per-response determinism (tool-<index> / the assembling block), not cross-turn uniqueness — providers without ids never echo them and rig matches results to calls within a turn.
2. A divergence is ErrorKind::Divergence, never retryable, and fails the run at its record.
3. The header is program identity: hooks (by short type name), required row, bus policy, run-spec hash. Format 2, no tolerant decode.
4. Hooks are program; tools are record.
5. Streaming goldens keep their events; the oracle compares them byte-for-byte as data.
6. Goldens are recorded from existing cassettes only; where a scenario had no engine-driven cassette, a mock-scripted producer records it (scenario 6, and it is the only one).
7. Every golden has exactly one producer; every producer names a committed golden.

Migration

E0

• ToolCallId::mint() and new_or_mint() are gone. ToolCallId::minted(index) (tool-<index>), ToolCallId::from_block(&BlockId), ToolCallId::new_or_minted(id, index), ToolCallId::for_provider_or(provider, minted). ToolCall::from_wire_indexed(id, index, function) for the index-th call of a response whose wire may omit ids; from_wire is from_wire_indexed(_, 0, _). From<ToolResultContent> for Message is deleted (it minted a random id for a hand-built result; use Message::tool_result(call, name, content)).
• A streamed call whose wire carried no id is named after the block that assembled it (tool-<n>); a unary Gemini response numbers its id-less calls in wire order. Two id-less calls in one response stay distinct; a re-run of the same wire yields the same ids. Cross-turn uniqueness is not promised: a test that scanned a whole history for a result by id alone (generate_sessions.rs) now takes the first result after its call.
• Guard tests/core/no_random_ids.rs: fastrand only in id.rs and the LSH index; id::generate only from the copilot and chatgpt transport headers; no fn mint() in message.rs.

E1

• ErrorKind::Divergence (never retryable). The replayer reports it; ToolDispatchAbort::Failed now covers Divergence alongside BusClosed and HandlerUnavailable, so a diverging tool fails the run with PromptError::Report at the tool record instead of feeding the model a failed result.
• divergence compares a tool call's whole payload (name, args, context).

E2

• EFFECT_LOG_FORMAT = 2. LogHeader gains hooks: Vec<String>, required: BTreeMap<HandlerKey, EffectFamily>, bus: Option<BusConfig>. A format-1 log is refused (check_header), not decoded.
• HookStack::names(); hooks are named by their type's last path segment (generics kept), so the same program in another crate names the same hooks.
• Agent::required_row(), Agent::bus_config(); check_replayable refuses on hook, required-row and bus-policy mismatch with both sides shown. The bus policy is checked by an agent that owns its bus; over a host's bus the policy is the host's and bus_config() is None.
• BusConfig derives Serialize/Deserialize; the rig-effect-log dependency guard admits rig_bus::BusConfig outside replay.rs as the one data type.
• EffectLogRecorder::set_program(hooks, required, bus); ToolServerHandle::retrieval_keys().

E3

• AgentBuilder::record_effects_with_events() — record and keep a streamed dispatch's events on its record.
• AgentBuilder::memory_handler(impl Serve) — serve conversation memory from any memory-family handler (a replayer) under the agent's memory key.
• EffectLogReplayer::for_key / for_log serve the header's required row, not only recorded keys: a tool the program advertised and the model never called is advertised on replay from the definition the recorded requests hold and answers any dispatch with a divergence.
• Recorder::handlers is now called for every handler installed after recording started (every tool the agent's registrar attaches), so LogHeader::handlers lists what actually served. BlockId::from_minted_name.

The corpus

golden	producer (root test)	cassette	scenario
anthropic_completion_smoke	tests/providers/anthropic/cassette/agent.rs completion_smoke_effect_log_is_the_golden_fixture	agent/completion_smoke	one completion
anthropic_memory_conversation	tests/providers/anthropic/cassette/agent.rs memory_conversation_effect_log_is_the_golden_fixture	agent/completion_smoke	load, completion, append — three records
anthropic_streaming_with_events	tests/providers/anthropic/cassette/streaming_tools.rs streaming_tools_effect_log_is_the_golden_fixture	streaming_tools/…	streamed turn, 11 + 4 events kept, one tool call
anthropic_concurrent_tools_serial	tests/providers/anthropic/cassette/streaming_tools.rs concurrent_tools_serial_effect_log_is_the_golden_fixture	streaming_tools/streaming_tool_concurrency_…	two tool calls in one turn, serial_per_handler: true, tool_concurrency(2)
gemini_tool_call_turns	tests/providers/gemini/cassette/hook_stress.rs tool_call_turns_effect_log_is_the_golden_fixture	hook_stress/streaming_lifecycle_…	two tool-call turns on id-less calls (tool-0 each), the scenario E0 makes possible
mock_invalid_tool_call_recovery	tests/core/golden_recovery.rs invalid_tool_call_recovery_effect_log_is_the_golden_fixture	none (mock-scripted)	unknown tool retried once by a hook; the retry is a record, the policy is program

Scenario substitutions from the prompt's list, each because no cassette of the named shape exists and no new HTTP recordings were allowed: the tool-call-turn scenario is gemini's hook_stress program rather than an anthropic one; the concurrent-serial scenario is anthropic's two-tool stream rather than gemini's; the streamed-events scenario is anthropic rather than openai; memory runs over the completion_smoke cassette; the invalid-call recovery has no engine-driven cassette anywhere in the corpus and is scripted from the mock model.

Replayed in crates/rig-verify/tests/golden_replay.rs: twelve tests, each golden by the bus engine and by the hand driver. The hand driver steps AgentRun, prepares each request with prepare_request, streams or completes over the ModelHandle, assembles streamed turns with StreamedTurnAssembler, dispatches tools over ToolHandle::call with the engine's own result shaping (rig_core::transcript::tool_result_output), loads and appends memory over the MemoryHandle, and resolves the invalid call with the same retry the hook returns.

What replaying the corpus found (fixed in E3)

• Late handlers were missing from the header. BusDriver::record_to described the handlers registered at that moment; every tool the agent's registrar attaches arrives later, so LogHeader::handlers held only the model (and memory). The driver now describes each installation to the recorder as it installs it.
• A never-called tool vanished on replay. The replayer only existed for keys with records, so a program advertising two tools of which the model called one advertised one on replay — a different request. The replayer now serves the required row.
• Folding a completion back into a stream re-minted its ids. rig_core::serve re-emits a unary answer as events for a stream consumer; it published every tool call under a wire block keyed by rig's id, so an id-less (Gemini) call came back with a provider id it never had, and an Anthropic single-id call came back with an item_id. The fold now carries the call's ids verbatim (a minted tool-<n> as the minted block it names) and its signature/additional_params, which it also dropped.

The cassette matcher and minted ids (risk 1)

E0 changes what gemini and ollama see on the wire for id-less calls (tool-<n> where a 21-character random id was). The cassette corpus still matches because the matcher never compared those fields literally: tests/common/cassettes.rs replaces the value of every id-bearing JSON field — id, any *_id, any *Id (in_id_field_position), plus the explicit GENERATED_ID_KEYS (call_id, item_id, tool_call_id, tool_use_id, …) — with a numbered placeholder on the recorded body and on the live body before comparing. Byte-identical requests were never possible for those fields; the rule is the proof, and the gemini, ollama and llamacpp cassette suites pass unchanged.

Churn contract (risk 2)

A golden holds every CompletionRequest the program made. A schemars bump, a preamble edit or a tool-definition change re-records the affected scenarios — the same contract as cassettes. Re-recording is one env var (RIG_REGENERATE_GOLDEN=1 cargo test -p rig --test anthropic --test gemini --test core -- golden_fixture), and a stale golden refuses instead of passing.

Remaining risks, stated

• Hook identity by type name (risk 3): two hooks of one type with different state are indistinguishable to the header. Decision recording is the follow-up.
• Required row direction (risk 4): the row must be a subset of the log's handlers (served, whether dispatched or not) and now also equal to the recorded program's row; a retrieval-only tool is in the row and a log that never dispatched it still satisfies it.
• Streaming byte-equality (risk 5): pins adapter minting; a legitimate adapter change re-records anthropic_streaming_with_events.
• Cancellation (risk 6): no producer in this corpus cancels; nothing to report.
• Per-response ids (E0): a history-wide lookup by tool id alone can collide across turns (two id-less first calls are both tool-0). Rig never does one; the one test that did was corrected.

Not in this PR

No new HTTP recordings; no openai golden (no engine-driven openai cassette with events); MIGRATING/CHANGELOG untouched per the repo policy for this series.

Gates

Run on the branch head a7aef7051 (2026-09-03): cargo fmt --all -- --check, cargo clippy --workspace --tests --examples (-D warnings), cargo check -p rig-core --target wasm32-unknown-unknown, cargo test --workspace --no-fail-fast — all clean. E1 and E2 were additionally checked in isolation (cargo check --workspace --tests at each commit).

Follow-up: the corpus records its own scenarios (commits F0–F3 on this PR)

Executes many_rigs/effect-bus/effect-corpus-followup-pr.md on top of E0–E4. New HTTP recordings were allowed for this part; nothing existing was re-recorded.

commit	title
F0	test(core): committed fixtures never hold a key
F1	test(providers): the corpus scenarios, recorded as named
F2	test(verify): the corpus replays its recorded scenarios
F3	docs(verify): the recording loop

Key handling, as followed

Record mode is RIG_PROVIDER_TEST_MODE=record; the recorder reads a key from its env name only in that mode and writes none of it (allow-listed request headers, scrubbed sensitive headers and Gemini's key parameter, placeholdered ids). Every recording ran on the producer's exact test filter. No key value was printed, passed on a command line, or written anywhere. F0's guard (tests/core/fixtures_hold_no_key.rs) scans every committed cassette and golden for the exported secrets (compared in memory, reported by env name and path:line), for the providers' documented key shapes, and for a sensitive header or parameter carrying anything but the placeholder; it was proven to bite on a planted token before the first recording and passes on the tree as committed.

The corpus now

Ten goldens, three providers, each replayed by both interpreters (twenty replay tests):

golden	producer	recorded here
anthropic_completion_smoke	anthropic/cassette/agent.rs	—
anthropic_memory_conversation	anthropic/cassette/agent.rs	—
anthropic_streaming_with_events	anthropic/cassette/streaming_tools.rs	—
anthropic_concurrent_tools_serial	anthropic/cassette/streaming_tools.rs	—
gemini_tool_call_turns	gemini/cassette/hook_stress.rs	—
mock_invalid_tool_call_recovery	core/golden_recovery.rs	—
anthropic_tool_call_turn	anthropic/cassette/effect_corpus.rs	yes: one tool call then the answer, blocking
anthropic_cancelled_stream	anthropic/cassette/effect_corpus.rs	yes: consumer drops the stream after the first text delta
openai_streaming_with_events	openai/cassette/effect_corpus.rs	yes: streamed turn, one tool call, events kept
openai_tool_call_turns	openai/cassette/effect_corpus.rs	yes: two tool-call turns, blocking

All substitutes from the first part stay.

What the recordings proved

• Dual provider ids survive verbatim. The OpenAI Responses wire gives each call a call_id and an item id; both goldens assert every call carries both, and both replay on the bus engine and the hand driver with no change to the E3 fold. Third wire, same three findings holding.
• A consumer cancel is recorded as Cancelled (risk 6 from the corpus prompt, previously without evidence). The first attempt, a ten-line answer, completed before the drop landed; the second, a 600-word essay, records one completion whose outcome is Cancelled with the bus's own message. On replay the replayer answers the record as the cancel it was, so the consumer sees the cancel as its first item and the log still matches.
• Goldens are generated from the replayed cassette, never from the live run. The first tool-call-turn golden, written during the record run, held live toolu_…/msg_…/req_… ids while its cassette held placeholders, and diverged on the first replay. golden_effects now refuses to write in record mode; the loop is record on the filter, then regenerate in replay. This rule was missing from the corpus prompt and is now in the crate docs.

Two scenarios stopped at the two-recording cap (ruling 2)

• Live invalid-call recovery. Claude Sonnet 4.6, told a multiply tool exists while only add is offered, called add on the first attempt and on the second called add then declined to answer a product with an addition tool. It never emits a call to a tool that is not in the request. The mock-scripted recovery golden stays as the scenario; its producer's hook moved to tests/common/goldens.rs so a future live producer names the same type.
• Gemini two calls in one turn. Gemini 2.5 Flash answered the two-lookup program with one call per turn; with functionCallingConfig.mode = ANY and an explicit "in parallel" instruction it called tools until the turn budget ran out and never answered. The Anthropic serial golden stays as the concurrent scenario; the id-less two-call case is not in the corpus, and E0's per-response numbering is pinned by the unit test and the sequential Gemini golden instead.

Gates

Run on 0e8ea42a9 (2026-09-03), in replay mode: cargo fmt --all -- --check, cargo clippy --workspace --tests --examples (-D warnings), cargo check -p rig-core --target wasm32-unknown-unknown, cargo test --workspace --no-fail-fast — all clean.