Make submit-tool call a successCriteria gate, not a substrate enforcement

[legreffier]

Context

Follow-up to #1158 (proposer/propose/claim rename) and the discussion on PR #1151.

Today the substrate treats "did the agent call submit_<task>_output?" as a runtime-side concern: the executor checks for a captured payload, and if none is present, fails the attempt with output_missing. Recent discussion landed on adding an in-session "feedback" message to inform the agent that its session ended without a submit call.

But that whole machinery is a category error. The submit-tool call is part of what the agent promises when claiming the task — it is not a separate substrate-level invariant. Adding a re-prompt layer accumulates substrate complexity for something that already has a clean home in the promise model.

Proposal

Make the submit-tool gate a successCriteria entry on every task type that has a registered output schema. The agent then self-verifies it like any other criterion.

Concretely:

1. At proposal time (or task-type validation), every task whose type has a registered output schema gets input.successCriteria auto-augmented with a gate criterion:

   {
     "id": "submit-output",
     "kind": "gate",
     "description": "Call `submit_<task_type>_output` exactly once with valid structured output."
   }

   The criterion is not optional and is not a per-proposal choice. It is part of what claiming any task with a registered output schema means.

2. The self-verification flow (buildSelfVerificationBlock → agent fetches criteria → agent emits verification.results[]) already handles this. The agent answers honestly: status: "pass" if it called the tool, status: "fail" if it did not. The verification block carries the agent's own acknowledgement of whether it kept its promise.

3. Failure recording: when verification.results[id="submit-output"].status === "fail", the attempt is recorded as the agent reporting an unkept promise. Same observable outcome as today's output_missing, but framed as the agent's own self-assessment rather than the substrate inferring failure.

Why this matters beyond cosmetics

Collapses runtime complexity

The proposed in-session "feedback nudge" in PR #1151 becomes unnecessary. The agent is already prompted by the existing self-verification block to fetch criteria and check them; "did I submit?" is one more bullet in that check. No new prompt path, no new counter label (captured_via_feedback), no coercive re-prompt copy.

Resolves the framing tension noted in PR #1151

In the same discussion thread I had to repeatedly walk back imperative framings ("Call it now") to informational ones. That tension exists because the runtime was reaching for tools to enforce something that promises theory says it cannot enforce. Putting the check inside successCriteria puts it on the side of the relationship where enforcement makes sense: the agent's self-binding.

Generalises to other "did the agent do the basic thing?" gates

Other criteria belong in this bucket: did the agent emit a verification block at all when successCriteria was non-empty? did the agent produce non-empty response? did outputCid round-trip? Today these are scattered across finalize.ts cross-field rules and submit-output-tool.ts validation. Lifting them into successCriteria gives the agent one place to look for "what am I promising," and the substrate one place to record "did the agent keep it."

The current verification required iff successCriteria rule in finalize.ts is the right shape but the wrong framing — it reads as "server enforcing cross-field rule" rather than "the agent promised to self-check and didn't." Same code, different conceptual home.

Aligns with #1158

#1158 renamed the relationship to proposer/propose/claim. This issue follows the same arc: stop treating substrate-level mechanisms as enforcement and start treating them as proposal contracts that the agent voluntarily binds to on claim.

Scope

• libs/tasks/src/ — helper that augments input.successCriteria with the submit-output gate at validation time (or proposer-side helper, depending on where the rule should live).
• libs/agent-runtime/src/prompts/self-verification.ts — keep the self-verification block exactly as it is. The agent's behaviour does not change; only the criteria it fetches do.
• libs/pi-extension/src/runtime/execute-pi-task.ts:939 — replace the output_missing synthesised failure with logic that consults the agent's verification.results[] for the submit-output criterion. If the agent reported fail (or didn't emit verification at all), record the attempt as a self-reported failure.
• apps/agent-daemon/src/lib/finalize.ts — the existing verification required iff successCriteria rule may become redundant once the criterion is universal; revisit.
• Tests for every task type that has a registered output schema (fulfill_brief, run_eval, judge_eval_attempt, judge_pack, curate_pack, render_pack, pr_review, assess_brief).

Out of scope

• The rename in Rename task proposer/impose terminology to proposer/propose #1158. This issue assumes that vocabulary.
• Cross-attempt retry semantics. Per the PR feat(#943): Phase 0 eval substrate dogfood — SDK, pi-extension, prompts, daemon, imposers #1151 discussion, retry is voluntary re-claim by an agent on the network, not a substrate concern.
• Per-task-type criteria beyond the submit-output gate. Each task type can extend successCriteria with its own checks; this issue is only about the universal gate.

Expected outcome

• Every task-type prompt asks the agent to fetch criteria and self-verify, including "did I call the submit tool" as a first-class criterion.
• Substrate stops inferring "agent forgot to submit"; instead it reads the agent's own verification.results[] and records the agent's self-reported pass/fail.
• The current output_missing failure code is reframed as a self-reported failure (or replaced with a code that says so: submit_criterion_failed, verification_missing, etc.).
• Future per-task-type completion checks have a single, well-defined home rather than being scattered across runtime, finalize, and validation.

Related

• Rename task proposer/impose terminology to proposer/propose #1158 — terminology rename (proposer/propose/claim). Sets the vocabulary this issue uses.
• PR feat(#943): Phase 0 eval substrate dogfood — SDK, pi-extension, prompts, daemon, imposers #1151 — eval substrate dogfood; surfaced the submit-output reliability problem that motivated this issue.