test(routing): lock HTTP/SSE echo of unchecked conduct - #683
Closed
cursor[bot] wants to merge 16 commits into
Closed
test(routing): lock HTTP/SSE echo of unchecked conduct#683cursor[bot] wants to merge 16 commits into
cursor[bot] wants to merge 16 commits into
Conversation
Closes the test-time-compute-allocation gap between what Fugu, Conductor,
and TRINITY (docs/architecture.md, arXiv:2512.04695, arXiv:2512.04388) call
for and what main actually does: reasoning_effort was accepted at the HTTP
edge but silently dropped on the orchestrated route/conduct path, and there
was no way to get a single checked judgment without paying for the full
four-step conduct() workflow.
- ModelClient.chat/stream_chat forward reasoning_effort (OpenAI-compatible
minimal/low/medium/high) to the provider payload when set, omitted
otherwise -- unaffected for providers/callers that never opt in.
- reasoning_effort threads through the whole call chain (server body ->
CostRoutingCoordinator.complete -> TaskOrchestrator.run/complete/_dispatch
-> route_once/conduct/route_and_verify -> _invoke -> ModelClient.chat),
including the plan-generation and model-judge calls, and is folded into
the response cache key so a cached low-effort answer can't be served for
a high-effort request. Batch-channel requests intentionally drop it today
(BatchRequest has no such field) -- documented, not silently wrong.
- New mode="verify" (TaskOrchestrator.route_and_verify): one worker call
plus one checked verifier judgment, for adjudication-shaped requests
("does B follow from A?") that need a verified verdict without the
thinker/worker/verifier/synthesizer workflow's cost.
- _client_chat() call-site helper keeps every existing ModelClient-shaped
test double/subclass in this repo working unchanged when reasoning_effort
is unset (the default) -- no test double needed touching.
Tests: tests/test_paper_contracts.py (reasoning_effort reaches every
provider call in a conduct() run; omitted by default; verify mode's trace
shape and reasoning_effort propagation) and a new
tests/test_reasoning_effort_and_verify_mode.py (HTTP-level: verify mode,
invalid/valid reasoning_effort validation). Full suite: 307 passed.
Does not touch any of the ~20 other open PRs' surface (OpenAI-compat
headers, security/session hardening, pricing/routing) -- verified no
existing open PR claims reasoning_effort or a partial-conduct mode before
starting this.
…s diff This PR's Semgrep check failed on 5 findings, none introduced by this change (line numbers only shifted because earlier edits in this branch added lines above them): - cost_ledger.py:586,605,625 (sqlalchemy-execute-raw-query): already bandit-suppressed (# nosec B608) with the same rationale -- the interpolated pieces are a DB-API placeholder character and fixed internal column-name constants, never request data; actual values always go through the parameterized second argument. Semgrep doesn't read bandit's nosec syntax, so it re-flags what bandit already accepted. Added the matching # nosemgrep suppression alongside the existing nosec comment -- no SQL construction logic changed. - orchestrator.py (unverified-ssl-context, dynamic-urllib-use-detected): same pattern -- both already carry a bandit nosec with an accepted rationale (verify_tls=False is an explicit opt-in dev-only argument, not a default; the request URL is validated by _provider_url()/ _validate_provider() -- https-only, path-injection-safe, private/ loopback/link-local/reserved-IP-rejecting -- before urlopen is ever reached). Added the matching # nosemgrep suppression with the same rationale spelled out for the urllib case. Verified locally: 'semgrep --config auto --severity WARNING --severity ERROR --error' now reports 0 findings on both files (was 5). Full test suite still 307 passed (comment-only change, no behavior touched).
The later quality-cost staging scripts were collected by pytest because stage_quality_cost_policy_test.py matches *_test.py. Importing that module wrote tests/test_quality_cost_adaptive_default.py during collection, which then failed the Full unit suite. The apply workflow also used contents:write (Scorecard Token-Permissions) and regex-patched orchestrator.py into a U+0001 SyntaxError on red-green-verify. Keep the already-landed reasoning_effort + verify mode and adaptive route/verify/conduct dispatch. Ignore scripts/ during collection so helper modules cannot inject tests again. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
A scripts/*_test.py helper was collected as a test and wrote a failing file into tests/ during import. Keep collect_ignore covering scripts/ and fuzz/, and document why so the Full unit suite cannot pick up staging helpers again. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
mode=verify no longer fallback-accepts a neutral verifier report or returns a rejected worker answer as a normal completion. Auto verify hints drop ambiguous check/review/confirm tokens and match ASCII terms on word boundaries. The chat surface echoes routing_decision and applied-or-dropped reasoning_effort; batch 202 reports the drop. Architecture notes now say request-level only and leave issue #568 open. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
The ledger counted only the public completion text, so a two-call verify invoice looked like a single route. Sum worker and verifier outputs (and any multi-step trace) before recording usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
mode=verify still accepted password/looks good/not accepted via substring terms, run() dropped reasoning_effort so HTTP never echoed it, and SSE shipped raw verification. Invoice every trace step, including an empty verifier plus a long worker. Conduct no longer serves the worker answer on reject. Everyday validate/judge/확인/평가 stay on the single-worker route. Per-role profiles remain issue #568. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
#618 still accepted "the password was accepted" as a verdict and served the synthesizer after a rejected conduct run. Require first-line or whole-report ACCEPT, echo answer_status on HTTP/SSE, and keep worker or synthesizer text on the trace only. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
#634 still stamped answer_status=accepted when a generated plan omitted a verifier, and treated an empty verifier step as omitted. Serve the synthesizer only when no verifier step exists, fail-close a present-but-empty verifier, and persist the status from complete(). Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
#664 persisted omitted-verifier status and locked rejected-verify framing, but a later accepted|rejected allowlist could drop unchecked from the buyer envelope while those tests stayed green. Assert complete() HTTP and SSE echo answer_status=unchecked and still serve the synthesizer. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
3 tasks
seonghobae
marked this pull request as ready for review
August 16, 2026 17:05
seonghobae
enabled auto-merge (squash)
August 16, 2026 17:05
Author
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
auto-merge was automatically disabled
August 16, 2026 18:23
Pull request was closed
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Successor to #664. Keep omitted-verifier
answer_status=unchecked, empty-verifier fail-closedrejected, and rejected-envelope gating. Close the buyer-visible hole #664 left green.#664 at
a448732persisteduncheckedfromrun()and locked rejected-verify HTTP/SSE framing. A lateraccepted|rejectedallowlist on the buyer envelope could stay green while dropping the only field that distinguishes “served but not checked” from a verified accept.This head:
complete(mode="conduct")on an omitted-verifier generated plan echoesorchestration.answer_status == uncheckedon HTTP and the final SSE chunk.verification.accepted is not Trueon both surfaces.unchecked.reasoning_effort_profileand equal-budget ablation.Do not merge #149 at
e3f7588, #612 at95393a0, #618 ataa188cb, #622 at1ec6a76, #634 at62cb671, or #664 ata448732. Prefer this successor. Independent non-author approval is still required.Test plan
python3 tests/test_generated_workflow.py— omitted verifier isuncheckedonconduct(),run(), andcomplete()HTTP/SSE; empty verifier step fails closed.python3 tests/test_verify_mode_honesty.py— architecture names persist/HTTP/SSEunchecked; rejected verify persist and framing stay locked.python3 tests/test_paper_contracts.pytest_model_judge.pytest_reasoning_effort_and_verify_mode.pytest_streaming.pytest_conventions.pytest_api_contract.pytest_self_check.pyCloses the leftover #664 honesty gap. Does not close #568.