Repository navigation
fix(agent): pre-action verifier and belief state run nowhere — AgentLoop has no production caller #14031
Description
Activity
- addedbugSomething isn't workingSomething isn't workingnot-wiredImplemented but not connected to appImplemented but not connected to app
on Aug 11, 2026 Full audit context:
docs/research/agent-harness-guard-and-context-audit.md— landing in PR #14032.The doc records what AutoBot already covers (with
file:lineevidence, so this issue is not re-audited later), where our design is deliberately stronger than the source's, and the do-not-re-file list — alongside all five gaps (#14027, #14028, #14029, #14030, #14031) and their wave order.Correction — this is not a new discovery; #11221 already documented it
A
code-reviewerpass caught a material omission in the body above.The dormant loop is known, in-code, under a CLOSED issue
autobot-backend/agent_loop/__init__.py:11-15already says:NOT WIRED IN PRODUCTION (#11221):
AgentLoopis never instantiated by any production
caller — the live tool seam ischat_workflow._dispatch_tool_call. Submodules here
(belief_state, extractors, guards) ARE reused by other subsystems, but the loop
orchestrator itself is dormant, so anything wired only into it is inert.#11221 is CLOSED — "AgentLoop (
agent_loop/) is never instantiated in production — decide
wire vs port vs document", resolved as document. The body above presents the dormancy as
freshly discovered while correcting #13587, and never cites #11221. That is exactly the kind
of omission that makes an issue look bigger than it is.What is actually new — and it is narrower
pre_action_verifier_enableddefaults toTrue(agent_loop/types.py:266) inside that
dormant module. A disabled-by-default dormant component is honest; an enabled-by-default one
reads as an active guard while executing on no request — and is very likely why Umbrella: agent-seam governance is name-scoped, and the best guards are not on the production path #13587
miscounted it.belief_state_enableddefaults toFalse(:258), so it is off and
unreachable, which is the less misleading of the two.- Umbrella: agent-seam governance is name-scoped, and the best guards are not on the production path #13587's "Already covered — NOT filed" list miscounts both, citing
agent_loop/loop.py
for the pre-action verifier and for durable run checkpointing. Documenting a dormant module
under AgentLoop (agent_loop/) is never instantiated in production — decide wire vs port vs document #11221 did not prevent a later audit from reading it as live coverage — which is the
real lesson, and the argument for extraction over documentation.
Grep wording corrected
grep -rn "from agent_loop.loop import\|AgentLoop("returns two non-test hits besides the
tests:agent_loop/__init__.py:38(package re-export) and:28(docstring usage example).
Neither is an instantiation, so the conclusion holds — but "returns only test files" was wrong
as stated.What this changes about the work
Nothing about the scope; something about the framing. #11221 chose document, and the
documentation demonstrably failed to stop a downstream audit from treating the module as live.
That is the case for choosing extract-and-wire this time rather than documenting again —
and for the wiring assertion in the acceptance criteria, which is the only form of
documentation that fails loudly.Full corrected evidence:
docs/research/agent-harness-guard-and-context-audit.md, Gap 5
(PR #14032).- added a commit that references this issue
on Aug 11, 2026 First slice landed: PR #14119 extracts and wires only
PreActionVerifier— belief-state extraction and thebelief_state_enableddefault decision are deliberately left for a follow-up (belief-state's flag defaultsFalse, so it's off and unreachable, a smaller/separable problem than the verifier's default-True-but-inert one).Summary of #14119:
autobot_shared/pre_action_verifier_guard.py— the extracted decision surface (pure: threshold resolution, response parsing,determine_verdict/panel_decision— theHARD_BLOCK-preserving part) plus thePreActionVerifierclass (the LLM call itself can't be made pure).agent_loop/pre_action_verifier.pyis now a thin re-export (thefact_forcingprecedent).chat_workflow/tool_handler.pygets a new_enforce_pre_action_verifierenforcer in the live dispatch chain, scoped toSENSITIVE_TOOLS, gated onpre_action_verifier_enabled(guard-profile resolved, defaultsTrue).- Wiring assertion + mutation test included, per the feat(agent-loop): production seam has no repetition/stagnation guard — the guards exist in a dormant module, unported #13590 template.
Flagged in the PR body as a finding rather than a quiet scope change:
execute_commandis inSENSITIVE_TOOLSand is almost certainly the highest-volume sensitive tool in production, so this adds an LLM round trip in front of its existing approval flow for every shell command. That's the originalAgentLoopscope exactly (never exercised before becauseAgentLoopnever ran in production) — worth a latency/cost look post-merge, not something I narrowed unilaterally.Leaving this issue open for the belief-state slice.
- added a commit that references this issue
on Aug 12, 2026 - addedneeds-decisionBlocked on an owner decision; options and a recommendation are on the issueBlocked on an owner decision; options and a recommendation are on the issue
on Aug 12, 2026 - added 4 commits that reference this issue
on Aug 12, 2026 3 remaining items
Half of this has been fixed since filing — one of the two components named is now live.
PreActionVerifier— no longer dead. It has a real non-test caller:autobot-backend/chat_workflow/tool_dispatch_guards.py:193 await PreActionVerifier().verify(...)imported by
chat_workflow/tool_handler.py:51, plus the extractedautobot_shared/pre_action_verifier_guard.py:387. This landed via the #13590 extraction template.BeliefStateUpdaterandAgentLoop— still dead. Non-test references toBeliefStateareagent_loop/loop.py:28,186(its own definition and use) andbenchmarks/benchmark_belief_state.py:78,384, which is a benchmark script, not production. ForAgentLoop:$ git grep -n -E "from agent_loop.loop import|AgentLoop\(" origin/Dev_new_gui | grep -v test 0 non-test callersThe single hit in
agent_loop/__init__.py:28is inside the module docstring's usage example, not code.So the remaining scope is
BeliefStateUpdater+AgentLoop, and the title's "pre-action verifier and belief state run nowhere" is now half true. Per the never-delete rule these are unfinished work to wire in, not code to remove — andPreActionVerifieris the worked example of how.- addedarea: agent-seamWave 3 · cluster H — Agent seam & tool governanceWave 3 · cluster H — Agent seam & tool governance
on Sep 1, 2026 Verified against merged
main. One of four delivered; the other three all hang off the same missing piece.AC1 — delivered, ticked
28e9df7c8(PR #14119) extracted the verifier and wired it at the live seam:autobot_shared/pre_action_verifier_guard.py564 lines — the decision surface, out of agent_loop/autobot-backend/chat_workflow/tool_handler.py+99 lines — the seam now calls it autobot-backend/agent_loop/pre_action_verifier.py−465 lines — the dead copy shrank to a shim So the verifier no longer "runs nowhere": the tool path invokes it.
AC4 — half delivered
autobot-backend/tests/unit/chat_workflow/test_tool_handler_pre_action_verifier.pycarriestest_the_call_site_still_names_the_enforcer, which is exactly the wiring assertion this criterion asks for — it fails if the seam stops calling the guard.But the criterion says per guard, and there is only one guard wired. The belief-state half has no seam to assert against, so AC4 cannot complete before AC2.
AC2 and AC3 — not delivered
belief_stateappears in noautobot_shared/orchat_workflow/file onmain. Every reference is still insideautobot-backend/agent_loop/:agent_loop/__init__.py agent_loop/loop.py agent_loop/types.py agent_loop/text_utils.py agent_loop/tests/test_belief_cache.py benchmarks/benchmark_belief_state.pyAgentLoopstill has no production caller, so belief state is in exactly the position this issue was filed about — the verifier was lifted out of it and the belief-state path was not.AC3 (decide
belief_state_enabled's default deliberately once it is reachable) is explicitly conditional on that reachability, so it cannot be judged, let alone met, until AC2 lands.One thing worth a look
28e9df7c8also added a tracked entry namedvenv(venv | 1 +in its diffstat). That looks like an accidental commit of a virtualenv symlink rather than intended content. Not this issue's subject and I have not touched it — flagging it because a trackedvenventry tends to cause confusing failures later.Remaining work is one extraction (AC2), which then unblocks AC3 and completes AC4. Not closing.
Correction to my previous comment's last section: the
venventry is already fixed and needs no look.I read it out of
28e9df7c8's diffstat and flagged it without checking the present state. It is not onmain:d74d66875— "fix(repo): untrack the venv symlink that destroys the interpreter on checkout (#14133) (#14138)" — removed it, and it had its own issue.So that paragraph described a commit correctly and the current tree wrongly. The two are different claims and I made the second while only checking the first. Everything above it stands: AC1 delivered and ticked, AC4 half delivered, AC2 and AC3 blocked on the belief-state extraction.
Re-verified against current
main— the defect holds, and re-checking it has a trap in itTriaged under the branch freeze.
The claim still holds
AgentLoophas no production instantiation onmain.PreActionVerifierand
BeliefStateUpdaterremain reachable only throughagent_loop/loop.py, which nothing constructs:PreActionVerifier—loop.py:28(import),loop.py:191-192(construction)BeliefStateUpdater—loop.py:27,loop.py:186; the only other non-test reference is
benchmarks/benchmark_belief_state.py:78,384, a benchmark, not a caller- Defaults unchanged:
types.py:266pre_action_verifier_enabled: bool = True,
types.py:258belief_state_enabled: bool = False
So the verifier still reads as an active guard — enabled by default — while executing on no
request at all. That is the sharpest part of this issue and it is still true.The trap, worth recording before someone re-verifies
The issue's evidence section says the grep returns "only test files". Run today, it also returns:
autobot-backend/agent_loop/__init__.py:28: loop = AgentLoop(That is inside the module docstring — the package's own usage example, not code. A reader
checking whether this issue is stale sees a non-test hit in a production file and can reasonably
conclude a caller now exists and the issue is fixed. It is not; the only other reference at
__init__.py:38is the re-export.This is the same failure this issue is about, one level up: something that reads as reachable while
reaching nothing. Anyone re-verifying should exclude docstrings, or check what the line is inside
before counting it.Branch verdict: do not revive
issue-14031is 4,286 files divergent frommain, 6 weeks old, with two unlanded paths:autobot-backend/chat_workflow/tool_handler.py— its copy predates security(agents): an agent refused an action can get another agent to do it — no per-peer identity #16950 and feat(agents): deliver peer messages at the recipient's next turn, not mid-task #16948, so taking
it would drop both (guards(git): reviving a stale branch can silently reverse a later fix — 75 of 87 revivable branches carry one #17178).venv— a committed symlink (mode120000) pointing at an absolute path inside a
developer's home directory. It is absent frommain, correctly, and must stay that way.
The second one is worth naming: it is invisible in a file-count summary, survives any "only two
files changed" reading, and would plant an absolute machine-specific path in the repository. It is
also a second reason this branch is not a starting point for anything.What the fix still needs
Per the never-delete rule, both components get wired into the live path —
chat_workflow/graph.pyandchat_workflow/tool_handler.py— rather than removed. Until then the
honest interim state would be forpre_action_verifier_enabledto stop defaulting toTrue, since
a default that advertises an inactive guard is the misleading half of the defect.Issue stays open. Nothing here is recoverable from the branch.
Correction: the verifier half of this issue is resolved
Re-checked on
mainduring a comparative research audit
(docs/research/confidence-gated-decision-loop-harness.md).PreActionVerifiernow has a production caller.
chat_workflow/tool_dispatch_guards.py:154enforce_pre_action_verifier, whose own docstring
records it as "its first production caller", wired into the dispatch chain at
chat_workflow/tool_handler.py:3264-3335. Scope matches the original — only tools matching the
canonicalSENSITIVE_TOOLSset — and on a soft block it threadsverifier_rationale,
verifier_refutation_probabilityandverifier_degradationinto thepending_approvalpayload,
so the rationale reaches the human.Still true:
AgentLoopitself has no production caller.grep -rn "AgentLoop("outside
tests returns one hit, the docstring usage example atagent_loop/__init__.py:28.Not re-checked: the belief-state half of this issue.
Adjacent defect found in the same config object and filed separately as #17469 — the three
timeout budgets inagent_loop/types.py:189-192are read by nothing, so wiring this loop would
not bring its budgets with it.v0.9.1 triage (classification only — not a fix proposal)
- Pre-filter — PRs: fix(agent): extract and wire the pre-action verifier at the live tool seam (#14031) #14119 MERGED 2026-08-12 "extract and wire the pre-action verifier at the live tool seam (fix(agent): pre-action verifier and belief state run nowhere — AgentLoop has no production caller #14031)" — targets this (AC1 slice). docs(research): agent-harness guard, context-sizing and ingest-governance audit (#14027) #14032 MERGED "docs(research): agent-harness guard… audit (security(commands): dangerous-command patterns match un-normalized input — homoglyph and ANSI-wrapped commands bypass the gate #14027)" — mentions only.
- Pre-filter — commits since filing (2026-08-11):
--grep '#14031':28e9df7c8(fix(agent): extract and wire the pre-action verifier at the live tool seam (#14031) #14119),47238c619(docs(research): agent-harness guard, context-sizing and ingest-governance audit (#14027) #14032 doc),94c427df3(docs, mention). Path log onagent_loop/{loop,types,pre_action_verifier}.py,autobot_shared/pre_action_verifier_guard.py,chat_workflow/tool_dispatch_guards.py:28e9df7c8,fb7f85f71guards extracted from tool_handler (bug(ci): Dev_new_gui is red — tool_handler.py grew past its #14236 ceiling (4081 vs 4063), failing code-quality on every open PR #14495),c041c120denv casts,3d9f3ba9a(security(agents): an agent refused an action can get another agent to do it — no per-peer identity #16950),de264fd15(feat(agents): deliver peer messages at the recipient's next turn, not mid-task #16948/9),e8c7bf9b5degraded verifier replies (bug(llm): structured_output=True is honoured by exactly one provider — every hosted provider silently drops it #17305/6). None touches belief state. - Files:
autobot-backend/agent_loop/types.py(grep :258,:266),agent_loop/loop.py(grep belief refs),agent_loop/__init__.py(opened :20-40),agent_loop/belief_state.py(exists, not opened),chat_workflow/tool_dispatch_guards.py(grep :154-197),chat_workflow/tool_handler.py(grep :56,:3226-3236,:3266,:3319),tests/unit/chat_workflow/test_tool_handler_pre_action_verifier.py(test names),benchmarks/benchmark_belief_state.py(grep). - Premise check (code read on main 80b6200):
- "Pre-action verifier runs nowhere" — STALE (AC1 met, ticked): real call
tool_dispatch_guards.py:197await PreActionVerifier().verify(...)insideenforce_pre_action_verifier:154, reached fromtool_handler.py:3319inside_dispatch_tool_call(def:3266) via_enforce_pre_action_verifier:3226-3236. - "AgentLoop has no production caller" — TRUE: non-test hits are
agent_loop/__init__.py:28(docstring usage example) and:38(re-export) only. - Belief state reachable only from AgentLoop — TRUE:
loop.py:27import,:186construction,:644,:699,:1350,:1385use; only other non-test reference isbenchmarks/benchmark_belief_state.py:78,384. Nobelief_stateinchat_workflow/orautobot_shared/. - Defaults — TRUE, unchanged:
types.py:258belief_state_enabled = False,:266pre_action_verifier_enabled = True(the latter now governs a live path, so the "reads active while inert" part applies only to the AgentLoop copy). - AC4 (wiring assertion per guard): verifier half present — behavioural
test_the_live_dispatch_seam_reaches_and_awaits_the_verifier(:80) plus rename guard (:63); belief-state half impossible until AC2. - Comment 5751505936 (2026-09-20) says PreActionVerifier is still reachable only via
loop.py— contradicted by the lines above; later comment 5828871088 (2026-09-25) corrects it.
- "Pre-action verifier runs nowhere" — STALE (AC1 met, ticked): real call
- Remaining scope per code: AC2 (belief-state extraction + seam wiring), AC3 (default decision, conditional on AC2), AC4 belief half.
- Blocked by: nothing hard. Issue asks to coordinate with fix(agent): wire read_spilled_output into the production tool seam — execute_tool has no production callers #13919 and fix(agent): the tool-output spill is wired only into AgentLoop, which is never instantiated in production #13997 (one extraction pass over
agent_loop/); adjacent bug(agent-loop): three timeout budgets declared in AgentLoopConfig and read by nothing #17469 (unused timeout budgets intypes.py:189-192, per comment). AC3 is a deliberate owner decision gated on AC2. - Read vs inferred: Read: body + 10 comments; lines cited above. Inferred: that no belief-state seam exists under another name (searched
BeliefState|belief_stateonly, not the concept "assertions tracker"). - Undetermined: state of fix(agent): wire read_spilled_output into the production tool seam — execute_tool has no production callers #13919/fix(agent): the tool-output spill is wired only into AgentLoop, which is never instantiated in production #13997 (not opened); whether the belief-state concept exists in the live graph under different naming.
- Collisions within v0.9.1 (same file; one PR/one agent under the same-file rule):
autobot-backend/chat_workflow/tool_dispatch_guards.pywith feat(agents): deliver peer messages at the recipient's next turn, not mid-task #16948, AI_STACK peer-message drain only wired for the 'chat' role; 'rag'/'system_commands' silently lose messages #16997
Generated by Claude Code
Pre-filter + per-AC pass — partial, and the gap is extraction rather than wiring. Not closed.
Read-only at
origin/mainab05ba3aef. One commit names this issue:28e9df7c84"fix(agent): extract and wire the pre-action verifier at the live tool seam (#14031)" — which is AC1, already ticked.The belief-state half (AC2) is wired but not extracted.
belief_state_enabledis consulted in the live loop atagent_loop/loop.py:644,:699and:1350, so the path is reachable — but the criterion asks to "extract the belief-state update path the same way", i.e. intoautobot_shared/, as the verifier was. A search forbelief_state/BeliefStateunderautobot_shared/finds nothing. So the asymmetry this issue was filed about is half closed: the verifier is shared and wired, the belief state is wired in place.That distinction matters for AC4 (a wiring assertion per guard: a test that fails if the live seam stops). A guard over a shared module can assert the seam from outside; a guard over a path that only exists inside
loop.pyhas to assert against the loop, which is the shape that rots. Extraction is the precondition for the guard, not a tidiness step — which is presumably why the criterion says the same way.AC3 (decide
belief_state_enabled's default deliberately once it is reachable) is now answerable, since the three call sites above make it reachable. It is a decision, not work: the default is currently whatever the config says, and the criterion asks for that to be a recorded choice rather than an inherited one.Verdict: 1 of 4, with AC2 as the blocker for AC4 and AC3 available to decide now. Nothing here needs a host.
B7 spec — #17725 delivers most of this issue's open criteria. The gap is narrower than the batch plan assumed.
Full spec at
~/b7-spec-2026-09-29.md. Read fromorigin/issue-16818-lease-wiring(PR #17725, open) againstorigin/main.I expected to specify the release path. The branch already contains it — its
+28tostalled_run_sweep.pyis:+sweep now releases what the run held: a stalled run hands its workspace back on the +same pass that closes it out. +from llc.services.workspace_lease import release_for_run +STALL_RELEASE_REASON = "run stalled: workspace released by the stalled-run sweep (#16817/#16818)" + released += len(await release_for_run(session, run.id, STALL_RELEASE_REASON))AC After #17725 merges AC2 releases what the run held lease: delivered AC5 two sweepers must not double-release delivered, with the reasoning in the diff — "Inside the same transaction as the status write, and under the same SKIP LOCKED selection: a run that this worker did not claim is not this worker's to release" AC6 release half of the test stalled_run_sweep_test.py+56AC7 reports what it released releasedcounter +logger.infoThe gap: AC2 names three things and the branch releases one
"claims, assignments, workspace leases" — no
claimorassignappears anywhere in those additions.Change: in the same transaction and under the same
SKIP LOCKEDselection, release the run's claims and assignments alongside its lease, each with a reason distinguishable from an expiry the waySTALL_RELEASE_REASONalready is.
Assertion: a swept run afterwards holds no claim, no assignment and no live lease, and the log names all three counts.
Mutation: release the lease only → the claims/assignments assertion reddens while the lease assertion stays green. Without that divergence, "releases what the run held" is satisfied by releasing one of three things.One boundary #17725 records, which this must not cross: "The release frees the slot, not the directory. Whether the directory may be removed is a separate question with its own evidence, and it is asked in
llc.services.workspace_disposal." Directory disposal is out of scope here.
Child of #13587.
Problem
The pre-action adversarial verifier and the belief-state tracker both run in
production nowhere. Their only caller is
AgentLoop, which has no productioninstantiation — the live path is the LangGraph StateGraph in
chat_workflow/graph.pypluschat_workflow/tool_handler.py.Evidence
grep -rn "from agent_loop.loop import\|AgentLoop("across the repo (excludingworktrees and venv) returns only test files:
autobot-backend/agent_loop/test_loop_repetition.py:22,44,212autobot-backend/agent_loop/test_approval_workflow.py:25,53autobot-backend/agent_loop/spill_loop_wiring_test.py:183,185autobot_shared/tool_catalogue_test.py:76,194Both components are reachable only from that dead entry point:
PreActionVerifier— sole non-test caller isagent_loop/loop.py:191-192,1668.pre_action_verifier_enableddefaults toTrue(agent_loop/types.py:266),so it reads as an active guard while executing on no request.
BeliefState/BeliefStateUpdater— sole non-test caller isagent_loop/loop.py:28,186,644,699,1350,1385.belief_state_enableddefaultsto
False(agent_loop/types.py:258): off and unreachable.Correction to this umbrella's own premise
#13587's "Already covered — audited, confirmed present, NOT filed" section lists:
The citation is to a module with no production caller. The capability is real and
well built; it is not covered, because it never executes. The same correction
applies to the "Durable run checkpointing" entry, which also cites
agent_loop/loop.py.By contrast, the entries citing
chat_workflow/tool_handler.pyare accurate andverified still wired today: fact-forcing at
tool_handler.py:3096and configprotection at
tool_handler.py:3061.The pattern that already worked
#13590 (CLOSED) is the template. The repetition/stagnation guard had the identical
problem, and the fix was not to wire
AgentLoopin — it was to extract thedecision logic into
autobot_shared/repetition_guard.pyas a dependency-free purefunction and call it from the live seam (
tool_handler.py:3163). Its own moduledocstring records the diagnosis: the machinery "existed in
agent_loop/and rannowhere".
fact_forcingtook the same route and is now a thin re-export ofautobot_shared.fact_forcing_guard.Keep the decision logic pure and side-effect-free; let the graph node own the side
effect. That is what makes it both testable and wireable.
Scope
PreActionVerifier's decision surface intoautobot_shared/,preserving the
HARD_BLOCKverdict, and call it from the live approval gate.where tool results are already collected.
belief_state_enabled's default deliberately once it is reachable —flipping it while unreachable changes nothing and would read as a fix.
calling it. Without this the next refactor silently un-wires it again, and
"no failures" reads as "working".
Related, same root cause, already filed: #13919 (
read_spilled_outputhas noproduction callers) and #13997 (tool-output spill wired only into
AgentLoop).Those two plus this one are the full inventory of
agent_loop/capability thatnever reaches production.
Order
Wave B of #13587, after #13590 (CLOSED — it establishes the extraction
pattern). No dependency on #14027/#14028/#14029. Coordinate with #13919/#13997:
one extraction pass over
agent_loop/is cheaper than three.Found during a comparative audit of an external agent-harness architecture, where
guards are pure decision classes plugged into graph nodes so there is only one
runtime and no second harness to fall out of date.