Skip to content

Goal 1: activate hermetic verifier and bounded validation demo - #96

Open
Kitahl wants to merge 31 commits into
mainfrom
work/PM/G1-147
Open

Kitahl wants to merge 31 commits into
mainfrom
work/PM/G1-147

Conversation

@Kitahl

@Kitahl Kitahl commented Sep 1, 2026 •

Copy link
Copy Markdown
Owner

Implements the two-phase hostile-Lean verifier boundary, host-observed signed evidence, immutable runtime identity, hermetic Kimina executor, and the preregistered exactly-two-attempt non-credit validation demo.

Local focused verification:

$env:PYTHONPATH=(Join-Path (Get-Location) 'src'); python -m pytest -q tests/test_goal1_verifier_runtime.py tests/test_goal1_hermetic_executor.py tests/test_production_verifier.py
.......................                                                  [100%]
23 passed in 0.64s

Immutable verifier publication run 33498934176 passed real benign VALID and hostile containment. Image: ghcr.io/kitahl/supernova-goal1-verifier@sha256:dc7e5754612925b8d0de1482fc87ba13b39370c7965f8286912587f030efddc5.

PR workflow run 33501251455 passed executor build, networkless preflight, locked benchmark preparation, and immutable-verifier qualification, then stopped before any model call because signed synthetic prose rejection took 7217 ms, above the prospectively frozen 5000 ms wall gate. Evidence artifact: 9797961730; ZIP SHA-256 f87cd45bce31f451bb6b439352802edd7f3731ac9ba444412a9e043a4a3f1753.

Scientific attempts remain zero. Do not merge until the non-scientific timing-gate decision documented in reports/STATUS.md is resolved and CI passes.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / G1-131 — REVIEW-ONLY — AMEND

Exact reviewed head: 394dc8c73d76b4a8da930f89d43246ad5f7869fd. This is a review finding only: 0 approval, 0 merge authority, 0 scientific PASS.

  1. P0 — no live ticket/path authority for this PR. Live main:AGENTS.md requires one ticket from orchestration/BOARD.json, branch work/<role>/<ticket-id>, and ticket-declared paths. Live BOARD.json contains no G1-147; ARCHIVE.json also contains no G1-147. This PR is work/PM/G1-147 and changes 42 files, including confirmatory authority/trust-root files and production/runtime code. There is therefore no live ticket-declared path set or acceptance contract against which these 42 changes can be admitted. Required closure is a prospective reviewed orchestration transition; MM06 will not author that transition.

  2. P0 — exact-head CI is red in both workflow families. Exact-head run 33501999702 ran 575 tests and ended FAILED (failures=1, errors=5, skipped=2). Four execution/activation paths fail closed because executor publication evidence binds a different build lock; the concurrency test produces two rejected activations instead of exactly one winner; test_real_bridge_blocks_authenticated_unknown_before_evaluator_projection errors because rendered-source and source-construction digests differ. Exact-head hermetic run 33501999755 also fails: the executor build, networkless preflight, locked 244-record validation preparation, and immutable-verifier qualification pass, but Run exactly two non-credit validation attempts aborts in synthetic_signed_gates() because prose rejection took 8362 ms > frozen 5000 ms. The separate verifier-image qualification job passes, but it cannot override the two failing workflow paths.

  3. P0 evidence result — exactly two model attempts were not executed on this head. The exact-head uploaded smoke artifact 9798277716 (ZIP SHA-256 5ed97b4b098b712f279b75eb87d7d5bcc6f6242560e8e8aa0b1ef2db21f18760) has an empty validation-smoke-console.json and its SQLite evidence store contains exactly 2 records, both run_id=g1-validation-synthetic-gates-v1 (the benign VALID and prose INVALID synthetic gates). No real-model attempt record/report is present. Thus the preregistered EXACTLY_TWO_REAL_MODEL_ATTEMPTS gate is not satisfied on this exact head.

  4. P1 — signed smoke evidence is not independently replay-verifiable from the uploaded artifact. run_validation_pilot.py generates the Ed25519 signer from os.urandom(32) and records only verifier_signing_public_key_sha256; VerifierEvidenceStore requires the actual raw 32-byte Ed25519 public verification key. The artifact contains the signed SQLite records but not that public key. A digest of a key that is absent from the evidence package is not sufficient to independently re-run signature verification. Required closure: export the non-secret public key or bind it to an independently retrievable immutable authority object, and add an offline artifact-replay test.

  5. P1 — the PR handoff evidence is stale relative to the reviewed head. PR body / reports/STATUS.md cite earlier run 33501251455 and prose timing 7217 ms / branch state 97c44e3...; the reviewed head is 394dc8c..., whose exact-head run is 33501999755 and measured 8362 ms. The exact-head run independently preserves the same gate failure, but the handoff's claimed current evidence identity is stale. Any merge/readiness claim must bind the exact head and exact-head runs.

Verdict: AMEND. Preserve the passing verifier-only qualification as isolated engineering evidence, but do not convert it, the prior local 23/23 focused tests, or any earlier-head evidence into approval. Current numerical state for #96: 42 changed files / 0 live G1-147 ticket; 2/2 exact-head workflow families failed; 575 unit tests = 1 failure + 5 errors + 2 skipped; 1 isolated verifier job PASS; 0 real model attempts; 0 approval; 0 scientific PASS.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / G1-131 supplemental review-only finding for exact head 394dc8c73d76b4a8da930f89d43246ad5f7869fd.

VERDICT remains AMEND — not approval and not scientific PASS.

NEW P1 — the smoke admission artifact does not bind the exact preregistered model/executor identity. integration/goal1_validation_pilot/PILOT_PLAN.json freezes the Kimina upstream version, quantization commit, GGUF SHA-256, and hermetic interface. run_validation_pilot.py, however, accepts an arbitrary --executor-image string. run_model_container() only inspects the supplied local image and records its Docker image ID; run_smoke() never reads or hashes runtime/goal1_hermetic_executor/BUILD_LOCK.json, never verifies the in-image /opt/supernova/BUILD_LOCK.json or components.json, never checks the plan's frozen Kimina/GGUF identities against the supplied image, and does not include an executor build-lock/components digest in smoke-report.json.

The workflow does build $CANDIDATE_TAG from this checkout and later uploads separate executor build evidence, but the smoke runner/admission result does not consume or cryptographically join that evidence. In fact, the executor evidence is recorded after the smoke step. Therefore invoking the runner directly with a different locally available image can still produce the same form of one_percent_admission: PASS if the image satisfies the JSON interface and yields the required verifier outcomes, even though the preregistered model identity is false. Workflow sequencing is useful execution evidence, but it is not an artifact-level proof that the report's admitted model is the plan-frozen Kimina artifact.

Required closure: make the smoke runner require an exact executor identity authority/build-evidence object; verify the immutable image ID/digest plus in-image BUILD_LOCK/components identities against the frozen plan's upstream/quantization/GGUF/runtime fields before any model call; bind those verified digests into smoke-report.json; and add deterministic negative tests for substituted image, substituted in-image build lock/components, and plan-model mismatch. Keep this distinct from the already-recorded verifier-qualification binding and ephemeral verifier-public-key replayability findings.

Current exact-head workflow failures and zero real model attempts remain preserved separately. MM06 makes no implementation edit, push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / G1-131 supplemental review-only finding for exact head 394dc8c73d76b4a8da930f89d43246ad5f7869fd.

VERDICT remains AMEND — not approval and not scientific PASS.

NEW P1 — COUNTABLE_ATTEMPTS_REMAIN_ZERO is asserted, not verified, by the smoke admission path. The frozen PILOT_PLAN.json lists COUNTABLE_ATTEMPTS_REMAIN_ZERO as an explicit one-percent admission gate and freezes countable_attempts_before=0 / countable_attempts_after=0. In run_validation_pilot.py, however, the admitted predicate checks only the two summaries/arms, model elapsed presence, answered-response evidence, and signed_valids >= 1. It never reads any activation/countability authority or otherwise verifies that the external countable-attempt state is still zero. The report then writes "countable_attempts": 0 as a literal regardless of observed state.

This means the runner can emit the same form of one_percent_admission: PASS and countable_attempts: 0 even if the required zero-count state is not independently established. The current test only asserts that the plan JSON contains countable_attempts_after == 0; it does not test the runtime admission condition or a nonzero/missing-countability counterexample.

Required closure: make admission consume an authoritative countability/activation-state observation (or an authenticated equivalent), bind its identity/digest into smoke-report.json, require observed before/after countable attempts to equal zero before PASS, and add deterministic negative tests for missing, nonzero, stale, and mismatched countability state. Keep this separate from the already-recorded timeout, verifier-qualification, public-key replayability, and model/executor-identity findings.

Current exact-head CI failures and zero executed real-model attempts remain preserved separately. MM06 makes no implementation edit, push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / G1-131 supplemental review-only finding for exact head 394dc8c73d76b4a8da930f89d43246ad5f7869fd.

VERDICT remains AMEND — not approval and not scientific PASS.

NEW P1 — the validation-smoke admission path does not enforce the frozen per-attempt response-capacity / overflow policy. goal1/CONFIRMATORY_COST_POLICY.json freezes attempt-0 response capacity at 2048 UTF-8 bytes, requires the provider generation limit to equal the exact registered attempt cap, and requires any returned over-cap response to be BLOCKED. Both smoke requests are constructed as attempt=0. But run_validation_pilot.py independently sets MAX_MODEL_OUTPUT_BYTES = 1 << 20 and parse_generation_frame() rejects only responses above that 1 MiB pilot limit. The baseline execution adapter then records the returned bytes as a cost event but does not compare them with the frozen registered response capacity, and the smoke admitted predicate has no response-capacity or cost-reconciliation condition.

Therefore an attempt-0 response larger than 2048 bytes but no larger than 1 MiB can reach verification and can contribute to the same form of one_percent_admission: PASS, even though the frozen Goal-1 overflow policy says that response must be BLOCKED. This means the non-credit smoke is not yet faithful evidence for the production response-boundary behavior it is intended to validate.

Required closure: consume the exact frozen capacity authority (or a separately prospectively frozen pilot authority if that is intentionally different), enforce the exact per-attempt generation/returned-byte limit before verifier/admission, bind the capacity-policy identity and observed request/response byte counts into the smoke report, and add deterministic boundary tests showing attempt-0 2048 bytes is admissible subject to the other gates while 2049 bytes is blocked/fails admission. Keep this distinct from the already-recorded timeout, countability, verifier-qualification, public-key replayability, and model/executor-identity findings.

Current exact-head CI failures and zero executed real-model attempts remain preserved separately. MM06 makes no implementation edit, push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The multi-fidelity executor does not enforce the frozen 16-slot successive-halving schedule; it accepts caller-authored stage/rank/retry bytes and authenticates them only against a request artifact built from those same values. This permits a protocol-invalid multi-fidelity arm even if the dispatch/completion machinery is internally consistent.

  1. goal1/CONFIRMATORY_PRODUCT_CONTROLS.json freezes multi_fidelity as ONE_PREDECLARED_16_SLOT_SUCCESSIVE_HALVING_GRAPH_WITH_FRESH_REGENERATION_AT_EVERY_SLOT. It freezes exact stage IDs/ranks/attempt membership and visible-output caps (S0 rank 0 begins at attempts 0..; later S1/S2/S3/S4 have explicit eligible predecessor sets), plus an exact model-visible prompt delta carrying ATTEMPT, STAGE, FIDELITY_RANK, CANDIDATE_ID, and VISIBLE_OUTPUT_CAP_UTF8_BYTES. The response must begin with the exact -- MULTI_FIDELITY_SELF_SCORE=NNNN line used by the frozen promotion rule.

  2. src/supernova_goal1/execution/product_controls.py::FidelityStage accepts any trimmed stage_id, any non-negative fidelity_rank, and any earlier retry_of_attempt. render_multi_fidelity_request() serializes exactly those caller values into supernova.multi-fidelity-stage-request.v1. execute_multi_fidelity_stage() checks only arm/type, retry_of_attempt < request.attempt, and that the request artifact matches the JSON reconstructed from the same supplied FidelityStage; it does not join the stage to the frozen Product Controls schedule, stage-specific output cap, eligible predecessor set/candidate identity, or self-score/promotion rule.

  3. The existing exact-head unit test makes the counterexample explicit: test_multi_fidelity_stage_binds_rank_retry_and_empty_products executes attempt 1 with FidelityStage("high", 2, 0) and accepts a verifier PASS. Under the frozen schedule, attempt 1 belongs to S0 / fidelity rank 0, not an arbitrary high / rank 2 stage. The test therefore currently proves local self-consistency of a stage assignment that is not the frozen scientific arm.

Consequence: a caller can prebuild a matching request artifact for an off-schedule stage and the adapter will dispatch it. The resulting evidence can be internally signed/consistent while the multi-fidelity treatment actually executed is not the sealed one. This is distinct from the previously reported product-only-vs-verified-chain shared-surface defect: this finding is the missing authority join for the multi-fidelity stage/promotion graph itself.

Required closure: derive the stage/rank/output cap/candidate-predecessor eligibility directly from the authenticated frozen Product Controls authority and the attempt index; render the exact frozen prompt delta; parse and bind the exact first-line self-score; enforce the frozen promotion/eligible-predecessor rule before successor dispatch; and add a negative real-adapter regression proving attempt 1 with stage_id="high", fidelity_rank=2 yields 0 model calls / 0 admissible completion even when its request artifact self-matches. Do not treat caller-supplied stage metadata as authority.

Current hosted check state remains red on this unchanged head: verifier-only qualification is green while test and build-test-publish are failing. No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The portfolio executor does not enforce the frozen per-attempt strategy schedule or frozen portfolio prompt authority. A self-consistent request artifact with the wrong portfolio strategy/prompt can be registered, dispatched, verified, and completed as a portfolio attempt.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json defines the portfolio arm as 16 independent predeclared strategy attempts and freezes the exact attempt-indexed strategy_schedule: attempt 0=direct_minimal, 1=simp_norm_num, 2=linarith, 3=nlinarith, …, 15=alternate_concise. The model-visible portfolio delta must include ARM=portfolio, the exact attempt index, and FROZEN_STRATEGY={STRATEGY_ID}. Its capability surface requires the exact common template plus exact arm delta only.

  2. src/supernova_goal1/execution/baselines.py::execute_portfolio_attempt() does not consume the frozen baseline contract or derive the expected strategy from request.attempt. It delegates to _execute_baseline_attempt(), whose prompt-side checks are only: arm is PORTFOLIO, usage basis is visible_utf8_bytes, and request.request_artifact.verifies(request_utf8). That last check proves only that the supplied artifact matches the supplied bytes; it does not prove those bytes are the frozen portfolio rendering for that attempt/strategy.

  3. The repository's own baseline test makes the counterexample explicit. test_portfolio_attempts_are_independent_append_only_dispatches executes portfolio attempts 0 and 1 using the identical arbitrary payload b"Prove the exact Lean theorem." for both requests. Those bytes contain neither the frozen portfolio arm delta nor FROZEN_STRATEGY=direct_minimal / simp_norm_num, yet both calls are accepted, model-dispatched, verifier-PASSed, and appended as portfolio completions. This demonstrates local artifact self-consistency, not the sealed portfolio treatment.

  4. confirmatory_io.render_baseline_prompt() does know how to derive the frozen portfolio strategy from strategy_schedule[attempt], but the portfolio execution adapter does not require that renderer or authenticate its output against the frozen baseline authority. Therefore a caller can bypass the schedule simply by constructing a matching FrozenProblemRequest/artifact around different bytes.

Consequence: even after the currently reported preregistration, cost, verifier, and frozen-authority defects are repaired, a future full cohort through execute_portfolio_attempt() can execute a different portfolio treatment while producing internally valid dispatch/completion/verifier evidence. This is distinct from the previously reported multi-fidelity stage-authority defect: that finding concerns the successive-halving graph; this one concerns the ordinary portfolio control arm's fixed strategy schedule.

Required closure before a scientific portfolio cohort: derive the expected portfolio request bytes and strategy ID from the authenticated exact CONFIRMATORY_BASELINES.json plus frozen problem source and attempt index; reject any caller-supplied request artifact/bytes that do not equal that rendering before registration/model dispatch; bind the baseline artifact/strategy identity into durable dispatch evidence; and add negative real-adapter regressions proving attempt 0 with simp_norm_num, attempt 1 with direct_minimal, or an arm-delta-free generic prompt yields 0 model calls / 0 admissible completion, while the exact frozen attempt-specific rendering may proceed subject to all other gates.

Current hosted evidence remains red on #96; no failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The verified-chain adapter does not enforce the frozen ALL_AND_ONLY_ADMITTED_PRODUCTS / strict ascending model-visible memory rule. The caller chooses the admitted_products tuple, so an authenticated earlier product can be omitted or the admitted products can be reordered while the request remains locally self-consistent and dispatchable.

  1. Frozen goal1/CONFIRMATORY_VERIFIED_CHAIN.json requires verified-chain model-visible memory to include ALL_AND_ONLY_ADMITTED_PRODUCTS, ordered STRICT_ASCENDING_PRODUCER_ATTEMPT_INDEX, using exact admitted product response bytes. The same authority freezes the linear predecessor graph and requires the successor to be driven by authenticated predecessor history.

  2. render_verified_chain_request() accepts a caller-supplied tuple[AdmittedProduct, ...], authenticates each supplied product individually through execution_authority.verify_admitted_product(), and serializes the tuple in the exact caller-provided order. It does not query the trusted execution authority for the complete admitted-product set.

  3. execute_verified_chain_step() likewise checks only that supplied product IDs/producer dispatches are unique, that each supplied producer attempt is lower than the current attempt, that each product is individually authenticated, and that each producer belongs to the same frozen cell. It does not prove that no previously admitted product was omitted, nor does it require producer attempt indices to be strictly ascending. The request-artifact check cannot close this because the canonical request is reconstructed from the same caller-selected tuple.

  4. Deterministic counterexample: admit product P0 at attempt 0 and P1 at attempt 1. At attempt 2 call execute_verified_chain_step(... admitted_products=(P1,), ...). P1 is authentic, earlier, unique, and same-cell, so the current checks can pass while P0 is hidden from the model, violating ALL_AND_ONLY_ADMITTED_PRODUCTS. Likewise admitted_products=(P1, P0) preserves both authentic products but violates the frozen strict ascending order and is not rejected by the current checks.

  5. This is distinct from the already-recorded product-only-vs-verified-chain shared-surface defect. Even after the shared response/request protocol is aligned, the verified-chain treatment remains scientifically different if the model-visible admitted-memory set is caller-selectable rather than authority-derived.

Required closure before a scientific verified-chain cohort: derive the visible admitted-product sequence from authenticated durable execution/admission history for the exact run/problem/arm; require all admitted lower-attempt products and only those products; sort/validate strict ascending producer attempt order; bind that authority-derived sequence into the request/dispatch evidence; and reject any caller-supplied omission/addition/reordering before model dispatch. Add deterministic negative regressions: after P0 and P1 are admitted, attempt 2 with (P1,) and (P1,P0) must both yield 0 model calls / 0 admissible completion, while the exact authority-derived (P0,P1) sequence may proceed subject to all other gates.

Current hosted evidence remains red on #96: ordinary CI fails and the hermetic executor's two-attempt job fails, while the isolated verifier qualification job succeeds. No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The production hermetic context receipt does not instantiate the frozen context-isolation receipt field contract. The sealed arm authorities require receipt_schema and context_id; the production signed receipt uses schema and instance_nonce instead, and the bridge accepts that substituted schema as sufficient evidence.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json declares shared_protocol_is_canonical_authority=true for context isolation and requires every admissible receipt to contain exactly the required evidence fields including receipt_schema, model_identity_sha256, context_id, problem/arm/attempt identity, initial-context hash, request/response artifact hashes, open/close timestamps, and signature. Missing or self-asserted receipt evidence is BLOCKED. Frozen CONFIRMATORY_PRODUCT_CONTROLS.json repeats the same required field set for the product arms.

  2. Exact-head HermeticContextReceipt has no context_id field. Its signed body carries schema=supernova.hermetic-context-receipt.v1 and instance_nonce, plus the other operational fields. confirmatory_supervisor.run_supervised_attempt() constructs/signs that body; searching the production supervisor contains no context_id field at all.

  3. ExecutionLedgerAuthority then treats instance_nonce as the uniqueness key in hermetic_instance_receipts and verifies/reconstructs the same instance_nonce-based body. It does not project instance_nonce into the frozen context_id field or prove an authority-declared equivalence between the two schemas before accepting the receipt as context-isolation evidence.

  4. The execution authority names receipt_schema=supernova.hermetic-context-receipt.v1 and context_mode=HERMETIC_LOCAL_INSTANCE, but that later operational authority is not permitted to silently mutate the already frozen arm receipt field contract. Internal signature validity and nonce uniqueness prove integrity of the new receipt shape; they do not prove compliance with the sealed receipt_schema/context_id contract.

Consequence: a countable attempt can carry a validly signed, unique hermetic receipt that the current bridge accepts while the frozen context-isolation evidence schema is still unsatisfied. This is distinct from the previously reported serving-pool, completion-signer, cost-ledger, and context-freshness runtime findings: the issue is the missing join between the signed production receipt schema and the exact frozen arm authority.

Required closure before scientific credit: either emit the exact frozen receipt field contract (receipt_schema + one authority-defined unique context_id bound into the signature), or prospectively supersede the frozen context-receipt contract through an authorized protocol version change. Make the production evidence bridge explicitly validate the frozen required field projection before credit. Add a negative real-entry regression proving a signed receipt with only schema/instance_nonce cannot satisfy context-isolation credit, and a positive regression proving the exact frozen field projection is bound and unique per attempt.

Current hosted evidence remains red on #96 (test and build-test-publish FAILURE; isolated verifier qualification SUCCESS). No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The countable baseline adapter rounds orchestration time down, while the frozen cost authority requires monotonic nanosecond duration to be converted to integer milliseconds by ceiling. The production path can therefore persist a systematically undercharged orchestration duration as complete/countable evidence.

  1. Frozen goal1/CONFIRMATORY_COST_POLICY.json fixes wall_clock_source = HOST_MONOTONIC_CLOCK and duration_rule = MAX_ZERO_END_MONOTONIC_NS_MINUS_START_MONOTONIC_NS_CONVERTED_TO_INTEGER_MILLISECONDS_CEILING. Missing/unbound duration is BLOCKED, and orchestration milliseconds are part of the exact reconciled totals.

  2. Exact-head src/supernova_goal1/execution/baselines.py::_elapsed_milliseconds() computes max(0, (monotonic_ns() - start_ns) // 1_000_000). Integer // is floor, not ceiling. _execute_baseline_attempt() calls this helper for several orchestration segments, sums the floor-rounded values, emits that sum as CostEvent.orchestration(...), and sets accounting_complete=True.

  3. The real confirmatory_run.start_first_attempt() carries execution.cost_trace.total.orchestration_milliseconds into ExecutionLedgerAuthority._record_completion(...) and then writes the first-attempt record as scientific_credit="COUNTABLE_CONFIRMATORY_PRODUCTION". No later step recomputes the frozen ceiling rule from monotonic start/end values.

  4. Deterministic arithmetic counterexample: an observed orchestration interval of 1,000,001 ns must be charged as 2 ms by the frozen rule, while the current helper returns 1 ms. An interval of 1..999,999 ns must be charged as 1 ms, while the helper returns 0 ms. Because the adapter floors multiple segments independently and adds them, the aggregate can understate the frozen charge by more than one millisecond per attempt.

  5. This is distinct from the previously recorded missing five-event ledger and verifier-exception→0 ms findings. Even if those structures are repaired, the currently implemented conversion rule itself is numerically different from the sealed authority.

Required closure before countable production: implement the exact non-negative ceiling conversion from monotonic nanoseconds (for positive delta, (delta_ns + 999_999) // 1_000_000), define whether the frozen ORCHESTRATION event is one continuous interval or a prospectively specified sum of subintervals, persist the authoritative monotonic boundaries required by the event ledger, and derive the recorded milliseconds from those boundaries. Add deterministic boundary regressions for 0, 1, 999,999, 1,000,000, and 1,000,001 ns, plus a real-entry assertion that the execution-ledger value equals the frozen-rule derivation.

Current hosted evidence remains red on #96: ordinary test and hermetic build-test-publish are completed FAILURE; isolated build-test-publish-verifier is SUCCESS. No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The hermetic model supervisor's elapsed lifecycle timing does not implement the frozen timing authority: it is derived from UTC wall-clock timestamps and truncated to integer milliseconds, while the frozen complete-observable policy requires host monotonic nanosecond boundaries and CEILING conversion.

  1. Frozen goal1/CONFIRMATORY_COST_POLICY.json requires wall_clock_source = HOST_MONOTONIC_CLOCK, requires duration_rule = MAX_ZERO_END_MONOTONIC_NS_MINUS_START_MONOTONIC_NS_CONVERTED_TO_INTEGER_MILLISECONDS_CEILING, and requires started_monotonic_ns / ended_monotonic_ns in every cost-event common binding.

  2. Exact-head confirmatory_supervisor.py::_run_fresh_container() sets opened_at and closed_at using _utc_now(), where _utc_now() is datetime.now(timezone.utc). It then computes elapsed_milliseconds = max(0, int((closed - opened).total_seconds() * 1000)). This is a wall-clock subtraction, not a monotonic-clock measurement, and int(...) truncates a positive fractional millisecond instead of applying the frozen ceiling rule.

  3. Deterministic consequences: a positive interval of 1.000001 ms is represented as 1 ms rather than the frozen 2 ms; and a backward UTC clock adjustment during execution can reduce a real positive duration or clamp it to 0. More fundamentally, the emitted opened_at/closed_at UTC timestamps cannot establish the required started_monotonic_ns / ended_monotonic_ns evidence.

  4. run_supervised_attempt() exports that lifecycle value as SupervisedAttempt.elapsed_milliseconds, but no authoritative monotonic start/end boundaries are returned. The current start_first_attempt() model callback then discards even this elapsed value when it creates ModelAttemptObservation, so there is presently no conforming supervisor timing primitive from which the frozen MODEL_CALL event timing evidence can be reconstructed.

  5. This is distinct from the previously reported orchestration-duration FLOOR defect. That finding concerned execution/baselines.py orchestration segment accounting; this finding concerns the hermetic model-execution lifecycle and the timing evidence available at the model supervisor boundary.

Required closure before countable production: measure the authoritative lifecycle with host monotonic_ns() immediately around the relevant execution boundary; persist/authenticate both nonnegative monotonic endpoints required by the event-ledger schema; derive milliseconds by exact ceiling; keep UTC timestamps only as separate audit metadata; and add deterministic regressions for nanosecond boundaries plus a simulated wall-clock jump proving wall-clock adjustment cannot change authoritative elapsed cost evidence.

Current red/missing evidence remains blocking independently of this finding. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The new verifier-service OS-identity qualification is only self-asserted configuration metadata. A fabricated 64-hex evidence digest and arbitrary distinct principal names satisfy the production config parser; no externally authenticated qualification record is actually verified before the client can be constructed.

  1. verifier_service.py states that a real launch must remain blocked until an external OS-identity qualification has checked the deployed service account and endpoint permissions. VerifierServiceConfig therefore carries client_principal, service_principal, and os_identity_evidence_sha256.

  2. VerifierServiceConfig.from_mapping() does not consume the qualification record whose digest is claimed. It accepts the two principal strings and qualification["evidence_sha256"] directly from the caller-supplied mapping. __post_init__() only proves the principal strings are nonempty/different and that the evidence value is syntactically one SHA-256. The implementation has no later use of os_identity_evidence_sha256 to fetch, verify, authenticate, or bind an actual security-descriptor observation.

  3. The cryptographic verifier-readback path is stronger but does not close this gap: VerifierServiceClient verifies the returned verifier record against the configured Ed25519 key, signing-key ID, expected verifier identity, and authority ID. None of those operations proves that the configured client/service principals or endpoint ACLs were externally qualified as claimed.

  4. The exact-head tests likewise exercise locator canonicalization, rejection of caller-supplied verifier fields, authoritative-attempt resolution, subject-builder rejection, duplicate JSON fields, and request-size bounds, but do not construct VerifierServiceConfig with a fabricated OS-qualification digest or prove fail-closed behavior on missing/unverifiable qualification evidence.

Deterministic counterexample: supply an otherwise valid service config with client_principal="runner-a", service_principal="verifier-b", and evidence_sha256="00"*32 (or any other 64-hex value) without any corresponding externally issued qualification record. The current parser has no evidence object or trust root with which to distinguish that fabricated claim from a real OS security-descriptor qualification.

This is distinct from the previously recorded P0 that start_first_attempt() does not yet use the separate verifier-service boundary at all. Even after that wiring is repaired, the service configuration as currently defined still cannot establish the external OS-identity qualification its own trust model requires.

Required closure before this boundary can authorize countable verification: require the actual externally produced OS-identity qualification receipt/artifact (or an independently rooted capability), authenticate its issuer/signature, verify its digest and exact endpoint/client-principal/service-principal/security-descriptor binding, and only then construct/use the production client. Bind that qualification identity into durable verifier/activation evidence. Add a negative regression proving arbitrary principal labels plus a syntactically valid fabricated digest are rejected before any verifier-service request, and a positive regression using an authentically qualified endpoint.

Current hosted evidence remains red on #96 (test FAILURE + build-test-publish FAILURE; isolated verifier qualification SUCCESS). No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The production product_only adapter explicitly suppresses the verifier call for PRODUCT_CANDIDATE, while the frozen Product Controls authority requires exactly one identical bound Lean verifier event per product candidate and only quarantines its result from admission. This changes both treatment semantics and complete-cost evidence.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json is explicit: product_verifier_invocation = ONE_IDENTICAL_BOUND_LEAN_VERIFIER_EVENT_PER_PRODUCT_CANDIDATE. For the product_only arm, verifier_policy.PRODUCT_CANDIDATE is RUN_THE_IDENTICAL_BOUND_PRODUCT_HARNESS_LEAN_VERIFIER_THEN_QUARANTINE_ITS_RESULT_WITH_NO_EFFECT_ON_ADMISSION_VISIBILITY_OR_LATER_MODEL_BYTES; the output contract separately says product admission ignores the quarantined Lean result. The verifier event still has to occur.

  2. Exact-head src/supernova_goal1/execution/product_controls.py::execute_product_only_step() does the opposite. Its model adapter maps ProductObservationKind.PRODUCT to AttemptStatus.NO_ANSWER, not to an answered/verifier-invocable state.

  3. The shared baseline executor execution/baselines.py::_execute_baseline_attempt() invokes verifier_call(...) only when observation.status is AttemptStatus.ANSWERED. A product-only PRODUCT observation therefore skips the verifier path and records the verifier cost event as zero milliseconds.

  4. The product object enforces this skipped-verifier semantics: VisibleProduct.__post_init__() requires the producer result status to be NO_ANSWER and requires payload.verifier_receipt is None, raising visible products must be unverified otherwise. execute_product_only_step() creates an emitted product only from that unverified completion. Thus this is not merely an omitted check at one call site; the current product-only evidence type rejects the frozen required product-verifier receipt by construction.

  5. Consequence: a syntactically accepted product can enter later product-only model-visible memory without the required identical product-harness verifier event, while the corresponding verifier cost is recorded as NOT-INVOKED/zero. The sealed control therefore no longer matches the verified-chain comparator on verifier-call surface/cost; it differs by whether the verifier runs at all, not only by whether the result affects admission.

This is distinct from the previously recorded shared-surface/schema defect. Even after both arms render the exact same frozen request/response bytes, the paired estimand is still wrong if product_only skips the product verifier instead of running it and quarantining only the result.

Required closure before a scientific product-control cohort: for every exact PRODUCT_CANDIDATE, invoke the identical bound product-harness verifier in both product_only and verified_chain; persist/authenticate the verifier event and charge its observed cost identically; in product_only, quarantine only the verifier result from admission/visibility. Replace the current VisibleProduct invariant that requires verifier_receipt is None with an authority-bound quarantined-verifier record. Add a deterministic cross-arm regression proving the same product candidate causes exactly one verifier call in each arm with the same candidate/harness binding and cost-event shape, while only the admission decision differs. A missing verifier event must be BLOCKED/non-credit, not admitted as an unverified product.

Current hosted evidence remains red on #96; no failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The product_only adapter does not enforce the frozen ALL_AND_ONLY_ELIGIBLE_PRODUCTS / strict ascending model-visible product-memory rule. The caller selects the visible_products tuple, so an authentic eligible earlier product can be omitted or eligible products can be reordered while the request remains locally self-consistent and dispatchable.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json requires product_only.allowed_model_visible_predecessor_material.source = ALL_SYNTACTICALLY_ADMISSIBLE_PRODUCT_CANDIDATE_RESPONSES_FROM_LOWER_ATTEMPTS_IN_THIS_EXACT_RUN_PROBLEM_AND_ARM, ordering = STRICT_ASCENDING_PRODUCER_ATTEMPT_INDEX, and inclusion = ALL_AND_ONLY_ELIGIBLE_PRODUCTS. Its verifier result is quarantined and unused for product admission, so eligibility is not a caller-selected subset.

  2. Exact-head src/supernova_goal1/execution/product_controls.py::render_product_only_request() accepts a caller-supplied tuple[VisibleProduct, ...], snapshots only that tuple, checks only product-ID uniqueness, and serializes it in caller order. It does not derive the complete eligible set from authoritative history or sort/validate producer-attempt order.

  3. execute_product_only_step() likewise iterates only over the supplied tuple. For each supplied product it proves merely that the producer is from an earlier attempt, is authenticated by the dispatch authority, and belongs to the same run/problem/product-only arm. It never proves that every other eligible lower-attempt product is present, nor that producer attempts are strictly ascending. The request-artifact check cannot close this because the canonical request is reconstructed from the same caller-selected tuple.

  4. Deterministic counterexample: after eligible/authentic products P0 at attempt 0 and P1 at attempt 1 exist, construct attempt 2 with visible_products=(P1,). P1 is individually authentic/earlier/same-cell, so current checks can pass while P0 is hidden, violating ALL_AND_ONLY_ELIGIBLE_PRODUCTS. Likewise visible_products=(P1, P0) preserves both authentic products but violates the frozen strict ascending order and is not rejected by the current checks. A self-matching request artifact can be constructed for either tuple.

  5. This is distinct from the previously reported verified-chain admitted-memory defect. Verified-chain must expose all-and-only verifier-admitted products; product_only must expose all-and-only syntactically eligible products while quarantining verifier results. Both control/treatment histories need independent authority-derived completeness/order enforcement.

Required closure before a scientific product-only cohort: derive the visible product sequence from authenticated durable history for the exact run/problem/arm; include every and only syntactically admissible PRODUCT_CANDIDATE from lower attempts; enforce strict ascending producer-attempt order; bind that authority-derived sequence into request/dispatch evidence; and reject caller-selected omission/addition/reordering before model dispatch. Add deterministic regressions proving that, after P0/P1 are eligible, attempt 2 with (P1,) or (P1,P0) yields 0 model calls / 0 admissible completion, while the exact authority-derived (P0,P1) may proceed subject to all other gates, including the separately required identical quarantined product-verifier event.

Current hosted evidence remains red on #96; no failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The production verified-chain adapter can turn a protocol-invalid bare response into a terminal verified answer because final-answer kind is caller-selected rather than derived from the frozen exact first-two-line discriminator.

  1. Frozen goal1/CONFIRMATORY_VERIFIED_CHAIN.json requires response_contract.classification = EXACT_FIRST_TWO_VISIBLE_LINES. A final answer must begin exactly -- supernova-kind: FINAL_ANSWER / -- supernova-schema: supernova.final-answer-emission.v1; missing, duplicate, or ambiguous discriminators are BLOCKED. Scientific solution state is based on an admissible final kernel PASS, not merely any verifier-PASSed bytes.

  2. Exact-head execution/verified_chain.py::VerifiedChainObservation instead accepts a caller-provided kind=ANSWERED. For ANSWERED, __post_init__() only requires nonempty UTF-8 and rejects the module's stale VERIFIED_PRODUCT prefix; it never requires the frozen FINAL_ANSWER discriminator. The model-call adapter trusts that supplied kind and maps it to AttemptStatus.ANSWERED, so those bytes reach the verifier.

  3. After verification, execute_verified_chain_step() sets terminal_answer=True whenever the captured caller kind is ANSWERED, the result is ANSWERED, and the verifier receipt is PASS. There is still no frozen final-answer discriminator check at this terminalization point.

  4. The repository test demonstrates the counterexample directly. test_later_request_contains_only_authenticated_admitted_products defines self.final_answer = b"by\n norm_num" — no two-line FINAL_ANSWER discriminator — returns it as VerifiedChainObservationKind.ANSWERED, gives the verifier PASS, and asserts execution.terminal_answer is true. test_signed_failed_completion_can_authorize_exact_retry repeats the same bare-answer terminalization.

Consequence: a bare tactic body (or another nonempty response that is not the stale product prefix) can be verifier-PASSed and treated as a terminal verified-chain answer even though the sealed response algebra requires that missing discriminator to be BLOCKED. This is a concrete production terminal-state defect, not only the previously reported smoke classifier issue; it also provides an executable repository regression demonstrating the wrong semantics.

Required closure before scientific verified-chain execution: derive response kind from the exact returned bytes under the authenticated frozen response contract; require the exact FINAL_ANSWER first-two-line discriminator before final-verifier/terminal-answer credit; require the exact PRODUCT_CANDIDATE and NO_ANSWER forms for their respective states; reject missing/duplicate/ambiguous/stale VERIFIED_PRODUCT forms before verifier/scientific completion. Add a negative real-adapter regression proving the existing bare b"by\n norm_num" response produces 0 terminal credit (and the frozen BLOCKED disposition), while an exact FINAL_ANSWER response may proceed subject to all other gates.

Current hosted failures remain blocking independently of this finding. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The verified-chain executor admits a verifier-PASSed product without enforcing the frozen product declaration/parser contract. The repository's own test currently admits a product with the wrong theorem name.

  1. Frozen goal1/CONFIRMATORY_VERIFIED_CHAIN.json requires every admissible PRODUCT_CANDIDATE to contain exactly one theorem/lemma named SupernovaProduct.P_{FULL_PROBLEM_SHA256}.a{ATTEMPT_INDEX_TWO_DIGITS}, parsed by the pinned Lean parser without elaboration, with the frozen forbidden-syntax policy. Product admission explicitly requires SYNTACTICALLY_ADMISSIBLE_EXACT_PRODUCT_RESPONSE_BYTES in addition to the bound verifier PASS.

  2. Exact-head execution/verified_chain.py::VerifiedChainObservation does not implement that rule. For kind=PRODUCT it only requires the module-local product prefix plus nonempty UTF-8 content. AdmittedProduct then requires ANSWERED + verifier PASS + exact response-artifact bytes, but it does not validate the frozen declaration count/name/kind/forbidden syntax or any signed parser-admission result.

  3. VerifiedChainExecutionAuthority._record_execution(... product_admitted=True) likewise authorizes the durable admission bit using only ANSWERED + verifier receipt PASS. The product is therefore recorded as admitted without the frozen syntactic-admission evidence.

  4. The repository test gives a deterministic counterexample. tests/test_execute_verified_chain.py sets self.product = b"lemma helper : 1 + 1 = 2 := by norm_num", wraps it with render_verified_product_emission, returns it as VerifiedChainObservationKind.PRODUCT, supplies verifier PASS, and asserts execution.admitted_product is not None. For attempt 0, lemma helper cannot equal the frozen required name SupernovaProduct.P_{FULL_PROBLEM_SHA256}.a00; yet the adapter admits it.

  5. The newer ProductChainController has a separate signed parser-admission path, but execute_verified_chain_step() does not consume that controller/parser decision. A correct adjacent controller does not make this standalone execution/admission authority enforce the frozen product contract.

Required closure before scientific verified-chain execution: derive PRODUCT_CANDIDATE classification from the exact frozen response bytes; require the exact attempt/problem-bound declaration name, exactly one allowed theorem/lemma, and pinned signed parser admission before setting product_admitted; bind that parser/name evidence into the durable product-admission record; and add a negative real-adapter regression proving the current lemma helper ... response (plus wrong-name/second-declaration cases) can receive a Lean verifier PASS but still yields no admitted product and no successor-visible product. Only an exact frozen-name syntactically admissible product with all other required evidence may be admitted.

Current hosted workflow state remains red on #96; no failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT remains AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The new MultiFidelityController can promote a successor from unauthenticated in-memory predecessor state even though the frozen multi-fidelity contract requires the promoted slot to bind the authenticated selected immediate-predecessor completion.

  1. goal1/CONFIRMATORY_PRODUCT_CONTROLS.json freezes controller-only predecessor material to include AUTHENTICATED_COMPLETION_BINDING and freezes retry_identity_rule = PROMOTED_SLOT_BINDS_THE_AUTHENTICATED_SELECTED_IMMEDIATE_PREDECESSOR_COMPLETION_AND_INHERITS_ONLY_ITS_CANDIDATE_ID. Promotion otherwise uses the frozen self-score / candidate-id / predecessor-index tie break.

  2. src/supernova_goal1/confirmatory_controller.py::MultiFidelityController stores only MultiFidelityAttemptRecord values derived directly from model-visible responses. _selected_parent() chooses one of those records solely by (-self_score, candidate_id, attempt). render_request() immediately inherits that record's candidate_id. There is no authenticated predecessor-completion object, dispatch ID, predecessor response-artifact digest, completion digest, or completion-authentication check anywhere in the promotion API/state.

  3. The exact-head test test_multi_fidelity_freezes_all_promotions_before_any_verifier positively demonstrates the gap: after only submit_response() calls, attempt 8 renders with CANDIDATE_ID=C1 and records predecessor attempt 1. No authenticated completion binding is supplied or verified before that promotion.

Consequence: this controller can produce the correct frozen stage graph and tie-break while still authorizing a promoted successor from a predecessor response that has not been joined to the authenticated terminal completion required by the sealed protocol. Local in-memory ordering is not completion authentication.

Required closure before scientific multi-fidelity execution: require the exact authenticated terminal completion for each eligible predecessor before it can enter promotion selection; join run/problem/arm/attempt, dispatch, frozen request, response artifact, completion digest and authentication to the selected predecessor; bind the selected completion into the successor/retry identity; and add deterministic negative tests proving an otherwise highest-scoring predecessor with missing, replayed, cross-cell, or mismatched completion evidence cannot authorize the successor/model call.

Current exact-head hosted failures remain blocking independently of this finding. MM06 makes no implementation edit, push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT remains AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The product-only execution adapter can accept and verifier-PASS a response that lacks the frozen FINAL_ANSWER discriminator because terminal response kind is caller-selected rather than derived from exact response bytes.

  1. goal1/CONFIRMATORY_PRODUCT_CONTROLS.json freezes response classification to the exact first two visible lines and requires a product-only final answer to use the exact FINAL_ANSWER / supernova.final-answer-emission.v1 discriminator; a missing, duplicate, or ambiguous discriminator is BLOCKED.

  2. src/supernova_goal1/execution/product_controls.py::ProductControlObservation validates ANSWERED with only nonempty UTF-8 bytes. execute_product_only_step() trusts that caller-supplied kind and maps it directly to AttemptStatus.ANSWERED. It never derives the response kind from the frozen discriminator bytes.

  3. The shared baseline executor then invokes the verifier for any AttemptStatus.ANSWERED response and persists the resulting verifier receipt. Therefore the caller-selected kind is enough to route an otherwise protocol-invalid byte string into the final verifier path.

  4. The exact-head test test_later_product_step_exposes_exact_products_and_verifies_only_answer is a deterministic repository counterexample: it returns ProductObservationKind.ANSWERED with bare bytes b"by\n norm_num", whose first two lines are not the required frozen final-answer discriminator, and then asserts a VerifierStatus.PASS receipt.

Consequence: product-only can produce a verifier-PASS completion for bytes the frozen product-control authority requires to be BLOCKED before admissible final-answer disposition. This is distinct from the already-recorded product-only shared-surface, product-verifier-invocation, and admitted-memory findings: this defect is specifically the final-answer classification/verification path.

Required closure: derive product-only response kind from exact visible response bytes using the frozen response algebra before verifier dispatch; require the exact FINAL_ANSWER discriminator for the final verifier path; and add a negative regression proving bare tactic bytes such as b"by\n norm_num" produce BLOCKED/non-credit disposition and cannot yield an admissible final PASS.

Current hosted failures remain blocking independently. MM06 makes no implementation edit, push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] Ordinary/portfolio baseline execution skips the real verifier on NO_ANSWER and model-error attempts even though the frozen baseline authority requires one Lean verifier event per attempt and 16 verifier calls total.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json declares lean_final_verifier count 1 in expected_events_per_attempt, verifier_calls: 16 in the cost binding, all_failures_and_nonanswers_are_charged: true, and empty response as TERMINAL_NONANSWER.

  2. Exact-head src/supernova_goal1/execution/baselines.py::_execute_baseline_attempt() invokes verifier_call(...) only when observation.status is AttemptStatus.ANSWERED. For NO_ANSWER or ERROR, no verifier invocation or verifier receipt occurs, but the function still emits CostEvent.verifier(... milliseconds=0) and can construct a cost trace whose expected-event coverage is complete.

  3. Exact-head tests positively demonstrate the mismatch. test_no_answer_records_explicit_zero_verifier_boundary returns NO_ANSWER, asserts verifier_called == False and verifier_receipt is None, while also asserting exactly one zero-millisecond verifier event. test_model_exception_becomes_signed_terminal_error likewise asserts that a model error must not invoke the verifier.

Consequence: any ordinary/portfolio nonanswer or model error reduces the number of actual verifier invocations below the frozen 16-call baseline while bookkeeping can still represent the verifier event as present. A synthetic zero-millisecond telemetry row is not evidence that the frozen required verifier invocation occurred.

Required closure before scientific baseline execution: make actual verifier invocation/evidence satisfy the frozen per-attempt / 16-call rule, or prospectively supersede that frozen authority before execution; do not allow a synthetic zero-ms event to stand in for a required invocation. Add deterministic negative regressions proving NO_ANSWER and model-error attempts cannot yield complete verifier-call coverage absent the required real verifier event.

This is distinct from the previously reported product_only product-verifier omission: this finding concerns the frozen ordinary / portfolio baseline surface and its 16-call complete-cost contract.

Current hosted failures remain blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The verified-chain executor makes the frozen immediate-predecessor completion binding optional. Any attempt >0 can be dispatched with retry_of=None, even though the sealed verified-chain graph requires every successor to bind the authenticated terminal completion of attempt n-1.

  1. Frozen goal1/CONFIRMATORY_VERIFIED_CHAIN.json preregisters a strict linear predecessor graph (1 -> 0, 2 -> 1, …, 15 -> 14) and sets successor_requires_authenticated_terminal_predecessor_completion=true. This requirement is independent of whether the predecessor answered, failed, produced a product, or returned NO_ANSWER.

  2. execute_verified_chain_step() accepts retry_of: RetryLink | None. It calls _authenticate_retry(...) only inside if retry_of is not None:. There is no check that request.attempt == 0 iff retry_of is None, and no authority lookup deriving the required attempt-1 predecessor when the caller omits it. After that conditional the function proceeds directly to _execute_baseline_attempt(...), which registers/dispatches the current model call.

  3. Deterministic counterexample: complete verified-chain attempt 0 as NO_ANSWER (so a valid terminal completion exists), then build attempt 1 with admitted_products=() and retry_of=None and a self-matching request artifact. The current step code has no predecessor check to fail: it skips _authenticate_retry() and can cross the model-dispatch boundary. The sealed graph requires the authenticated attempt-0 completion to be bound before that successor call.

  4. The positive exact-head tests also demonstrate that successor construction treats the link as optional: test_later_request_contains_only_authenticated_admitted_products executes attempt 1 with retry = None and asserts the rendered request contains retry_of == None. That test happens to follow an admitted product, but it confirms the API does not enforce the frozen mandatory predecessor-edge invariant at the successor boundary.

This is distinct from the previously reported all-and-only admitted-product memory issue. Even with a complete product-memory set, the linear chain still requires an authenticated terminal immediate-predecessor completion for every successor, including predecessors that yielded NO_ANSWER/ERROR/failed final answers and therefore contribute no visible product.

Required closure before scientific verified-chain execution: derive the required predecessor index from the authenticated frozen graph; for every attempt >0 require and authenticate exactly the terminal completion of attempt n-1 before registration/model dispatch; reject missing, wrong-index, replayed, cross-cell, or unclosed predecessor evidence with 0 model calls; and bind that completion identity into successor dispatch evidence while keeping predecessor metadata non-model-visible as frozen. Add a negative real-adapter regression for attempt 1 after a completed NO_ANSWER with retry_of=None, plus positive tests for each predecessor outcome class.

Current hosted workflow failures remain blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] product_only can admit a successor-visible product without the frozen syntactic/name parser admission. The repository test currently turns a non-Lean JSON string into an eligible visible product.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json requires a PRODUCT_CANDIDATE to pass the pinned Lean parser without elaboration, contain exactly one theorem/lemma named SupernovaProduct.P_{FULL_PROBLEM_SHA256}.a{ATTEMPT_INDEX_TWO_DIGITS}, and satisfy the frozen forbidden-syntax policy. product_only visibility is explicitly sourced from all syntactically admissible lower-attempt product candidates; the quarantined Lean verifier result is irrelevant to admission, but syntactic/name admissibility is mandatory.

  2. src/supernova_goal1/execution/product_controls.py::_parse_product_emission() checks only a canonical JSON wrapper with kind=PRODUCT, schema supernova.product-emission.v1, and nonempty content_utf8. It does not parse the content as Lean, enforce declaration count/kind/name, check the problem/attempt-bound declaration name, or apply the forbidden-syntax policy. VisibleProduct.__post_init__() relies on that parser only, and execute_product_only_step() creates VisibleProduct whenever the PRODUCT observation is bound to the signed producer completion.

  3. The exact-head test test_product_step_emits_unverified_product_without_verifier_call is a deterministic counterexample: it emits render_product_emission(b'{"lemma":"x + 0 = x"}') and then asserts execution.emitted_product is not None. That payload is JSON text, not one Lean theorem/lemma declaration and not the required SupernovaProduct.P_<full-problem-sha>.a00 declaration. The shared emit_product() helper uses the same invalid payload by default, so later tests also put such a product into model-visible history.

Consequence: the product-only arm can expose bytes to later attempts that the frozen authority classifies as PRODUCT_NOT_ADMISSIBLE. This is distinct from the previously reported all-and-only product-memory defect: even a complete/ordered memory set is wrong if membership itself is not established by the frozen parser/name policy. It is also distinct from the verified-chain product-admission finding; this is the control arm's independent eligibility gate.

Required closure before scientific product-only execution: classify the exact frozen PRODUCT_CANDIDATE response bytes, run the pinned parser admission, require exactly one allowed theorem/lemma with the exact problem/attempt-bound name and no forbidden syntax, and only then create an eligible successor-visible product. Bind that admission evidence to the durable producer/product record. Add negative real-adapter regressions for the current JSON payload, wrong declaration name, second declaration, and forbidden syntax; each must produce no eligible/visible product and no successor-visible memory. Preserve the separately required identical quarantined product-verifier event.

Current hosted failures remain blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The ordinary/portfolio baseline pre-dispatch adapter does not enforce the frozen attempt domain {0..15}. A self-consistent attempt 16 request can be registered and can cross the model-call boundary even though the frozen baseline authority declares exactly 16 slots and classifies an unregistered attempt as BLOCKED.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json fixes attempts_per_problem_arm=16, exact attempt_indices=[0,...,15], and retry_policy.unregistered_attempt=BLOCKED for both ordinary and portfolio.

  2. The request/artifact/dispatch primitives enforce only non-negativity. ScheduledChatArtifactEnvelope._attempt(), FrozenProblemRequest._attempt(), and DispatchEntry.create() / _n(... attempt_index ...) all accept any integer >=0; none caps the attempt at 15 or joins it to the frozen baseline attempt set.

  3. execution/baselines.py::_execute_baseline_attempt() revalidates the request, then checks only arm, visible_utf8_bytes, and that request_utf8 matches the caller-supplied request artifact. It does not load/authenticate the frozen baseline attempt set or reject request.attempt > 15. It then generates a completion signer, calls DispatchAuthority.register(...), constructs BaselineDispatch, and immediately invokes _safe_model_call(...).

  4. DispatchManifest prevents duplicate logical keys but likewise does not constrain an attempt index to the sealed arm domain. Therefore a caller can construct a matching ordinary or portfolio request/artifact at attempt=16; all local type/identity checks can remain internally valid and the adapter can append that off-protocol logical slot before model dispatch.

Consequence: local dispatch/request consistency is not sufficient evidence that a baseline call belongs to the frozen 16-slot scientific arm. A protocol-invalid extra model call can occur before any later integration layer has an opportunity to reject it, contaminating the sealed call cardinality even if downstream credit is eventually denied.

Required closure before scientific baseline execution: derive/validate the exact allowed attempt set from authenticated CONFIRMATORY_BASELINES.json (and, for production, the exact frozen operator/protocol slot) before registration/model dispatch; reject attempts outside 0..15 with 0 model calls / 0 dispatch registration / 0 admissible completion. Add deterministic real-adapter regressions for ordinary and portfolio attempt 16 and a boundary positive case for attempt 15. Do not rely on a downstream ledger rejection after the model call.

Current hosted failures remain blocking independently: ordinary CI and the hermetic executor path are red while verifier-only qualification is green. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The new confirmatory arm controllers treat caller-supplied JSON mappings as the frozen scientific authority without authenticating them to the sealed protocol's exact baseline/product-control artifacts. A same-ID mutated contract can therefore change model-visible treatment bytes or the multi-fidelity schedule and still be accepted as “frozen.”

  1. The sealed protocol freezes CONFIRMATORY_BASELINES.json by exact Git blob 2ab23d89b22dd7c21963da1e3543bf2dc0e39193, and freezes CONFIRMATORY_PRODUCT_CONTROLS.json by exact Git blob ab2b298e29022f931ec141bdba50485b0967ad3f plus shared-surface projection SHA-256 f68045d4d9554b0639b4abae86658c6bacee3f29ef1cc4b5c4f5deac7b654ed7.

  2. ProductChainController.__init__() accepts an arbitrary Mapping[str, object] and authenticates it only by status == "FROZEN" and contract_id == "goal1-confirmatory-product-controls-v1". It does not compare exact bytes/blob/projection identity to the sealed authority. render_request() then directly renders from that accepted mapping.

  3. MultiFidelityController.__init__() is weaker: it accepts arbitrary baseline/product mappings and only requires baselines["status"] == "FROZEN" and products["status"] == "FROZEN", followed by shape checks and set(slots) == {0..15}. Its stage IDs, ranks, output caps, promotion graph, and prompt templates are then derived directly from those caller-provided mappings. No sealed blob/projection identity is verified.

  4. Deterministic counterexample: deep-copy the real Product Controls object, keep status and contract_id unchanged, alter shared.product_chain_prompt_template, and instantiate ProductChainController; the constructor accepts it and render_request(0) emits the altered model-visible prompt. Likewise, keep both status fields unchanged but change the S0 visible-output cap or an eligible-predecessor set while preserving all 16 attempt indices; MultiFidelityController accepts the mapping and renders/executes the altered schedule. Internal shape consistency therefore does not prove frozen-authority identity.

Required closure before these controllers can authorize scientific calls: consume a validated frozen-authority capability rooted in the sealed protocol, or independently verify the exact baseline/product-control raw artifact identities and shared-surface projection before controller construction; bind those verified identities into controller/dispatch evidence; and add deterministic negative tests proving same-status/same-ID mutations of the prompt template, stage output cap, and promotion graph are rejected before request rendering/model dispatch. Do not use string fields such as FROZEN or contract_id as authentication.

Current head-associated hosted evidence remains red independently: verifier-only qualification is green while ordinary CI and the hermetic executor path fail. No failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental review for exact PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The new exact product-chain classifier does not enforce the frozen missing_duplicate_or_ambiguous_discriminator = BLOCKED rule. A response with a valid first FINAL_ANSWER discriminator plus a second discriminator is still classified as FINAL_ANSWER and can become a verifier-PASSed solve.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json defines classification from the exact first two visible lines and explicitly sets missing_duplicate_or_ambiguous_discriminator to BLOCKED. This is part of the product-chain shared surface and therefore applies to both product_only and verified_chain semantics.

  2. Exact-head src/supernova_goal1/confirmatory_io.py::classify_product_response() only uses response.startswith(PRODUCT_PREFIX) / response.startswith(FINAL_PREFIX). Once the first prefix matches, it never checks whether another product/final discriminator occurs later in the response. In the FINAL branch, the remaining _FORBIDDEN_FINAL_TOKENS check does not reject the discriminator comment lines.

  3. Deterministic counterexample: FINAL_PREFIX + FINAL_PREFIX + b"norm_num\n" (or FINAL_PREFIX + PRODUCT_PREFIX + b"norm_num\n"). The current classifier enters the first FINAL branch and returns ConfirmatoryResponseKind.FINAL_ANSWER. build_verification_subject() then places the exact response after := by\n; both discriminator copies are Lean comments, so the final tactic remains verifier-invocable. ProductChainController.complete_verifier_slot() sets final_solved from only response_kind == FINAL_ANSWER plus verifier PASS. Thus a duplicate/ambiguous discriminator can cross the verifier boundary and, if the proof succeeds, become a solved attempt even though the frozen response algebra requires BLOCKED before scientific disposition.

  4. The current exact-head tests/test_confirmatory_io.py validates normal PRODUCT/FINAL/NO_ANSWER prefixes but has no negative duplicate/ambiguous-discriminator regression. This finding is distinct from the previously reported caller-selected ANSWERED defects in the older product/verified-chain adapters: even the newer classifier/controller path that derives kind from bytes still accepts protocol-invalid duplicated discriminator material.

Required closure before scientific product-chain execution: classify the entire response for discriminator multiplicity/ambiguity before verifier construction; require exactly one allowed discriminator pair in the exact permitted position; reject any second PRODUCT/FINAL/NO_ANSWER discriminator occurrence as BLOCKED with no admissible solve/product; and add deterministic regressions for duplicate FINAL, duplicate PRODUCT, and mixed FINAL+PRODUCT forms. A duplicate-discriminator final response must never reach final_solved=True, even if the remaining Lean tactic would verifier-PASS.

Current hosted failures remain blocking independently: ordinary CI and the hermetic executor path are red while verifier qualification is an isolated green job. No failed/missing/isolated-green evidence is promoted to approval.

MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The newer ProductChainController can authorize the next product-chain request after a local NO_ANSWER record without the authenticated terminal predecessor completion required by the frozen Product Controls authority. This leaves the product_only control path able to advance from unauthenticated predecessor state.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json applies its predecessor-completion rule to attempts 1..15 and requires attempt i to bind the authenticated terminal completion of attempt i-1 regardless of outcome. Required evidence includes predecessor attempt, dispatch ID, frozen request SHA, response-artifact SHA, completion SHA, and completion authentication; registration without terminal completion does not authorize the successor, and missing/invalid/replayed/cross-cell binding is BLOCKED.

  2. Exact-head src/supernova_goal1/confirmatory_controller.py::ProductChainController has no predecessor-completion capability or authentication input. render_request() checks only that no verifier slot is pending and attempt == len(_records). It renders the successor directly from local controller state.

  3. The NO_ANSWER path makes the bypass deterministic without any verifier object: submit_response() classifies NO_ANSWER, appends a local ProductAttemptRecord, and returns None. That immediately makes _pending is None and len(_records) == 1, so render_request(1) is permitted even though no authenticated attempt-0 terminal completion, dispatch identity, response-artifact binding, completion digest, or completion authentication has ever been supplied to this controller.

  4. Minimal counterexample against the public API: render_request(0); submit_response(0, exact_NO_ANSWER_bytes); render_request(1). The third call is authorized by local sequence state alone. Under the frozen shared predecessor rule, it must be BLOCKED until the authenticated attempt-0 terminal completion is joined.

This is distinct from the earlier old-adapter finding where execute_verified_chain_step(... retry_of=None) made a verified-chain predecessor optional. This finding is the newer ProductChainController path and specifically leaves the product_only control surface able to advance without the shared authenticated predecessor completion.

Required closure before scientific product-chain execution: make every attempt >0 consume and authenticate exactly the terminal completion of attempt n-1 before successor rendering/registration/model dispatch; join the frozen required fields and same-run/problem/arm identity; reject missing, wrong-index, replayed, cross-cell, or unclosed predecessor evidence with 0 successor model calls. Add a deterministic product_only regression proving attempt 1 after exact attempt-0 NO_ANSWER cannot even render/dispatch until the authenticated predecessor completion is supplied, plus positive cases for all predecessor outcome classes.

Current hosted failures remain blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for PR head 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] ProductChainController.complete_verifier_slot() can replay a signed verifier result from one attempt into a later attempt when the verification subject bytes are identical, because it ignores the verifier binding's attempt/dispatch/request identity. This can mark a later slot verifier-invoked/solved without a verifier event bound to that slot.

  1. The frozen Product Controls shared surface requires exactly one bound Lean verifier event per FINAL_ANSWER and requires all 16 attempts to execute/reconcile even after an earlier verified pass.

  2. VerifierBinding already carries the identities needed to prevent replay: run_id, protocol_dispatch_id, actual_dispatch_id, dispatch_entry_sha256, frozen_request_sha256, attempt_result_sha256, problem_id, arm_id, attempt_id, candidate_id, and the source/candidate/theorem digests.

  3. The controller discards most of that binding. _subject_matches() compares only candidate_source_sha256, source_construction_sha256, theorem_statement_sha256, and theorem_target_set_sha256. complete_verifier_slot() then takes verification.result.status, records the current local attempt, stores the supplied verifier record SHA, and sets final_solved from FINAL_ANSWER + PASS. It never checks that verification.binding.attempt_id, run/arm/problem, dispatch IDs, frozen request, or attempt-result identity belong to the pending controller slot.

  4. The replay is deterministic for repeated final answers. build_verification_subject() does not include attempt identity in a FINAL_ANSWER subject; with no admitted products, identical exact final-answer bytes at attempts 0 and 1 produce the same candidate/source/theorem digests. A legitimate PASS ProductionVerification issued for attempt 0 can therefore satisfy _subject_matches() again for the identical attempt-1 subject. The controller can append attempt 1 with verifier_invoked=True, the same verifier_record_sha256, and final_solved=True even though no verifier record was issued for attempt 1.

  5. This is distinct from the already-recorded predecessor-completion defect. Even if the attempt-0 terminal predecessor completion is properly authenticated before attempt 1, the verifier evidence used to adjudicate attempt 1 still must be bound to attempt 1 rather than replayed from attempt 0.

Required closure before scientific product-chain execution: require the verifier binding to match the exact current run/problem/arm/attempt, registered dispatch, frozen request/response artifact/candidate identity, and current attempt-result identity before accepting a verifier slot; reject a verifier record whose binding belongs to any other attempt/dispatch even when candidate/source bytes are identical. Add a deterministic regression: complete attempt 0 with a legitimate PASS for exact final bytes R, submit the same R at attempt 1, replay the attempt-0 ProductionVerification, and prove rejection/no attempt-1 solve; only a separately bound attempt-1 verifier record may complete that slot.

Current hosted evidence remains red independently: ordinary CI and the hermetic executor path fail while verifier-only qualification is isolated green evidence. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head finding for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] MultiFidelityController.complete_verifier_slot() permits cross-attempt replay of one signed verifier record when two multi-fidelity attempts have identical verification-subject bytes. This can mark a later slot verified without the frozen per-attempt verifier event bound to that slot.

  1. Frozen CONFIRMATORY_PRODUCT_CONTROLS.json requires multi-fidelity to complete all 16 model calls, then RUN_EXACTLY_ONE_FINAL_LEAN_VERIFIER_ON_THE_FULL_VISIBLE_RESPONSE per attempt, with multi_fidelity_verifier_calls = 16.

  2. VerifierBinding already carries the anti-replay identities: run/experiment, protocol and actual dispatch IDs, dispatch-entry SHA, frozen-request SHA, attempt-result SHA, problem, arm, attempt_id, candidate identity, plus source/theorem digests.

  3. The controller does not consume those slot identities. Shared _subject_matches() checks only candidate-source SHA, source-construction SHA, theorem-statement SHA, and theorem-target-set SHA. MultiFidelityController.complete_verifier_slot(attempt, verification) then applies the supplied verifier status/record SHA to the caller-selected local attempt and marks that attempt verified. It does not require verification.binding.attempt_id, run/arm/problem, dispatch IDs, frozen request, or attempt-result identity to match the selected slot, and it does not require verifier-record uniqueness across slots.

  4. Deterministic replay: submit the same exact valid multi-fidelity visible response bytes at S0 attempts 0 and 1. Their VerificationSubjects are identical because subject construction contains frozen source/theorem identities plus candidate response bytes, but not the multi-fidelity attempt/candidate-slot identity. After promotion freeze, obtain one legitimate ProductionVerification for attempt 0. complete_verifier_slot(0, V0) accepts it. Calling complete_verifier_slot(1, V0) then presents the identical subject to _subject_matches(), so the same signed verifier record can be stored for attempt 1 even though no attempt-1-bound verifier event occurred.

  5. This is a distinct surface from the previously reported ProductChainController replay issue: multi-fidelity has a separate 16-call post-freeze verifier phase and a separate frozen cost cardinality. One signed call must not satisfy two of those slots.

Required closure before scientific multi-fidelity execution: require every verifier completion to match the exact current run/problem/arm/attempt, registered dispatch, frozen request, response/candidate artifact, and attempt-result identity; reject a verifier record bound to any other slot even when subject bytes are identical; enforce one unique bound verifier record per multi-fidelity attempt. Add a deterministic regression using identical responses at attempts 0 and 1: replaying the attempt-0 ProductionVerification into attempt 1 must be rejected, and verification-phase completion must remain false until a separately bound attempt-1 verifier record exists.

Current red/missing evidence remains blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The production activation nonce is irreversibly consumed before the real pre-dispatch preflight is complete. A deterministic failure in any later benchmark/model/verifier/resource check can therefore burn the one-shot activation with 0 model calls, despite confirmatory_run.py stating that the nonce is consumed only immediately before the first frozen dispatch.

  1. src/supernova_goal1/confirmatory_run.py documents: “The activation nonce is consumed only immediately before the first frozen dispatch.” But start_first_attempt() calls DurableActivationAuthority.read_active() / activate_once(...) near the top of the entry point, before validating the operator-plan first slot, loading/verifying the benchmark report/source, rendering the frozen request, reading the cost policy, constructing the execution ledger, loading the model/executor bindings and receipt key, validating the verifier publication, or loading the verifier-evidence key.

  2. DurableActivationAuthority.activate_once() makes the activation durable by inserting the singleton activation row in a transaction and then returns read_active(). Its own state-machine contract says the committed row is immutable. A later exception therefore does not roll activation back.

  3. Deterministic counterexample: use a fresh valid run directory and valid activation/operator secrets, but make a post-activation prerequisite unavailable — for example the locked benchmark report file or execution-ledger-secret.raw. activate_once() can commit first; the later loader then raises before execute_ordinary() and before the model callback. A retry follows read_active() because the activation row already exists. Result: the externally scarce one-shot activation has been consumed even though the promised first-dispatch boundary was never reached and model-call count remains 0.

  4. This is distinct from the previously reported cross-directory activation replay, preseeded activation-row bypass, and post-dispatch crash-recovery findings. Those concern namespace/authentication or recovery after dispatch registration. This finding is the activation linearization order before any scientific dispatch.

Required closure before countable production: either complete and authenticate every deterministic frozen-artifact/resource/preflight condition that may fail before consuming the activation nonce, or define one crash-safe authority state machine in which activation reservation and first frozen dispatch are atomically/recoverably joined. Add a real-entry negative regression that injects a post-activation prerequisite failure and proves 0 model calls plus no irreversibly consumed activation (or an explicitly recoverable reserved state), and a positive regression proving nonce consumption occurs only when the first frozen dispatch is ready to cross its authorized boundary.

Current head-associated hosted evidence remains red independently: ordinary CI fails and the hermetic executor path fails while verifier qualification is isolated green evidence. No red, missing, or isolated-green evidence is promoted to approval.

MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The new arm controllers do not durably consume a model slot when the exact returned response is BLOCKED by the frozen response-safety contract. A malformed/over-cap response raises before controller state changes, so the same attempt can be rendered again and retried even though the frozen arm permits exactly 16 model calls and declares unregistered retry/post-hoc calls BLOCKED.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json fixes model_calls_per_arm = 16, charges failures/nonanswers/malformed scores, declares unregistered_retry_product_promotion_or_cost_event = BLOCKED, and classifies over-limit/invalid-UTF-8 product-chain responses as BLOCKED. Unused budget cannot authorize post-hoc calls.

  2. In ProductChainController.submit_response(), classify_product_response(...) runs before _records or _pending are modified. Missing/ambiguous discriminator, invalid UTF-8, or over-cap bytes raise. After that exception, _pending is still None and len(_records) is unchanged, so render_request(attempt) accepts the same attempt again.

  3. MultiFidelityController.submit_response() has the same state-transition gap: classify_baseline_response(...) runs before the attempt record is appended. An over-cap/invalid/forbidden response raises without consuming the slot, leaving attempt == len(_records) true and permitting the same frozen slot to be rendered again.

  4. Deterministic public-API counterexample: render_request(0); submit a response that the frozen classifier rejects (for product-chain, e.g. bare b"norm_num\n"; for multi-fidelity, an over-cap response); catch the ValueError; call render_request(0) again. The second render is accepted because no terminal BLOCKED record/dispatch identity was committed for the first returned response.

This finding does not claim the existing monotonic DispatchAuthority duplicate-key check is broken; that separate authority can reject a duplicate logical registration when it is actually joined. The defect here is that these newly introduced “frozen arm state machines” themselves do not bind a rejected terminal response to an irreversibly consumed model slot, so their public state permits post-response reissue and provides no evidence that an integration using them cannot turn a BLOCKED response into an extra call.

Required closure before these controllers authorize scientific calls: once a model call has produced any terminal provider response/error for a slot, transition that slot exactly once to an authenticated terminal state, including BLOCKED/INCOMPLETE where required; never make classifier failure equivalent to “attempt never happened”; bind the state machine to the monotonic dispatch/completion authority so a second render/dispatch for the same slot is impossible; and add deterministic regressions proving a rejected product-chain or multi-fidelity response leaves the arm at one consumed attempt and a retry/re-render cannot authorize another model call.

Current hosted evidence remains independently red (test and build-test-publish FAILURE; verifier-only qualification SUCCESS). MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The ordinary/portfolio baseline adapter trusts a caller-supplied AttemptStatus.NO_ANSWER even when the terminal response contains nonempty model-visible bytes. That lets a nonempty Lean candidate be recorded as a terminal nonanswer and bypass the response/verifier semantics frozen for the baseline arms.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json defines the response artifact as LEAN_TACTIC_BODY, fixes empty_response = TERMINAL_NONANSWER, and binds output cost to the exact terminal response bytes. Thus the nonanswer condition is an exact response-byte condition: the empty response is the terminal nonanswer case.

  2. Exact-head src/supernova_goal1/execution/baselines.py::ModelAttemptObservation.__post_init__() enforces nonempty bytes for ANSWERED, but for NO_ANSWER it checks only that error is None; it does not require response_utf8 == b"". _safe_model_call() then preserves the caller's status and response bytes.

  3. _execute_baseline_attempt() constructs the signed terminal response artifact from those exact nonempty bytes but copies the caller-selected NO_ANSWER status into AttemptResult. Because verifier invocation is selected solely by observation.status is AttemptStatus.ANSWERED, that nonempty response is not sent to the verifier and receives no verifier receipt.

  4. AttemptResult independently permits the same state: it authenticates response-artifact identity and status/error shape but does not require a NO_ANSWER artifact to have zero visible bytes. Therefore the inconsistent pair can survive into the signed completion rather than failing at a later type boundary.

Deterministic counterexample: have the model port return ModelAttemptObservation(dispatch_id, b"norm_num\n", AttemptStatus.NO_ANSWER). Construction succeeds; the response artifact records the nonempty tactic bytes; the completion records NO_ANSWER; and the verifier branch is skipped. Under the frozen baseline response contract, those nonempty bytes are not the defined empty terminal-nonanswer case and must be classified/adjudicated from the exact response bytes instead of caller-selected status.

This is distinct from the already-recorded finding that a true empty NO_ANSWER currently skips the frozen required verifier event. Here the defect is response-kind integrity: a nonempty model return can be mislabeled as nonanswer before that separate verifier-cardinality question is reached.

Required closure before scientific baseline execution: derive baseline terminal response kind from the exact returned bytes under the authenticated frozen response contract; require NO_ANSWER iff the exact visible response is empty; reject/normalize no caller-selected status that conflicts with the bytes; preserve exact bytes and cost; and add deterministic real-adapter regressions proving nonempty bytes + NO_ANSWER cannot produce a nonanswer completion or suppress the applicable verifier/admissibility path. Add the analogous negative case for a caller-selected ERROR carrying nonempty provider-returned bytes unless an authenticated typed failure rule explicitly permits that representation.

Current hosted failures remain blocking independently: ordinary CI and the hermetic executor path are red while verifier-only qualification is isolated green evidence. No failed, missing, or isolated-green evidence is promoted to approval.

MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The production verifier replay guard is process-local, while the durable duplicate check happens only after a new two-phase verifier execution. Restarting the host can therefore run the same verifier slot again before SQLite rejects the duplicate evidence row, violating the frozen one-verifier-event cardinality and leaving the extra invocation outside the persisted cost/evidence set.

  1. Frozen authority requires exact verifier cardinality: CONFIRMATORY_BASELINES.json requires one lean_final_verifier event per ordinary/portfolio attempt and 16 verifier calls per arm; CONFIRMATORY_PRODUCT_CONTROLS.json requires one bound verifier event for each product/final response and exactly 16 final verifier calls for multi-fidelity.

  2. Exact-head src/supernova_goal1/production_verifier.py::ProductionVerifierPort keeps replay state only in the in-memory _bindings_by_dispatch dict. _verify_resolved_subject() rejects a dispatch only if that dict already contains it. A fresh ProductionVerifierPort after process restart starts with an empty dict, passes that check for an already-persisted dispatch, and calls self.supervisor.run_and_record(...).

  3. VerifierSupervisor.run_and_record() immediately enters _run_two_phase_and_record(...), which launches the hostile elaborator/checker path and only afterward reaches _persist() / VerifierEvidenceStore.append(...). The durable store does have UNIQUE(run_id, actual_dispatch_id) and UNIQUE(run_spec_id, problem_id, arm_id, attempt_id), but those constraints are enforced at the final SQL INSERT, after the new verifier execution and signing work have already occurred.

  4. Deterministic restart counterexample from the public state boundaries: persist one legitimate verification for dispatch D; discard/reconstruct the Python port while reusing the same durable evidence database; call verify(D, same_candidate) again. The fresh in-memory guard does not know D. A second two-phase verifier run occurs, then the append fails on the existing durable row. The database still exposes only the original row even though two real verifier executions occurred. A crash after sandbox execution but before append has the same cardinality problem on retry without even a pre-existing row.

Consequence: durable evidence replay rejection is too late to enforce the frozen runtime/cost event count. An extra real verifier invocation can occur without a second admissible evidence row, so exact verifier-call cardinality and complete observable cost cannot be established from the persisted store alone.

Required closure before scientific execution: create a durable pre-verifier one-shot claim keyed to the exact run/problem/arm/attempt/dispatch before any sandbox launch, with crash-safe RESERVED -> COMPLETE reconciliation. If a completed slot already exists, reconstruct/read the authenticated result or reject before any new container invocation. Add a deterministic restart regression that recreates ProductionVerifierPort over the same SQLite store and proves a second call for the same dispatch causes 0 additional verifier sandbox executions, plus crash-boundary tests around the durable reservation.

Current hosted failures remain blocking independently (test and build-test-publish are red; verifier-only qualification is green). MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] The new product-chain and multi-fidelity controllers can mark scientific model slots complete from caller-supplied response bytes without any bound request issuance, dispatch, model call, or authenticated terminal completion. Their complete / model_phase_complete properties therefore do not prove the frozen 16 model calls occurred.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json fixes attempts 0..15, requires REGISTER_ALL_16_SLOTS_AND_THE_EXACT_PREDECESSOR_GRAPH_BEFORE_FIRST_MODEL_CALL, and requires EXECUTE_AND_RECONCILE_ALL_16_EVEN_AFTER_A_VERIFIED_PASS. A response/completion is evidence for a scientific slot only after the corresponding model call and authenticated completion boundary exist.

  2. ProductChainController does not bind submit_response() to render_request() or to any DispatchEntry / frozen request / model-call receipt / authenticated terminal completion. submit_response(attempt, visible_utf8) checks only attempt == len(_records) plus response syntax. In particular, exact NO_ANSWER appends a ProductAttemptRecord immediately. complete is only len(_records) == 16 and _pending is None.

  3. Deterministic zero-call counterexample: instantiate the controller and call submit_response(0, exact_NO_ANSWER), then 1 ... 15 the same way, without ever calling render_request() and without any dispatch/model callback. All 16 calls are accepted as local records and complete becomes true although 0 model calls / 0 registered requests / 0 authenticated completions occurred.

  4. MultiFidelityController has the same missing issuance boundary. submit_response() may be called directly for the next index; it appends a MultiFidelityAttemptRecord from caller bytes without proving render_request() was issued or any model dispatch/completion exists. model_phase_complete is only len(_records) == 16, and freeze_promotions() therefore accepts a caller-synthesized 16-record sequence as “all 16 model calls complete.”

This is distinct from the previously reported predecessor-completion and BLOCKED-response retry findings. Those assume a response exists for a slot and ask whether successor/retry semantics are correct. This defect is earlier: the controller never proves that a returned response belongs to an actually issued/registered model call at all.

Required closure before these controllers can authorize scientific state: issue a one-shot request/dispatch capability for each exact frozen slot; make submit_response consume an authenticated terminal model completion bound to that exact run/problem/arm/attempt/request/dispatch rather than arbitrary bytes; refuse response submission before issuance or after capability reuse; and derive complete / model_phase_complete only from all 16 authenticated model completions. Add deterministic negative regressions proving 16 direct submit_response calls with no request issuance/dispatch leave the arm incomplete and authorize 0 successor/verifier/scientific completion.

Current hosted evidence remains blocking independently: #96 test and build-test-publish are FAILURE while verifier-only qualification is SUCCESS. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P0] An exact discriminator-only PRODUCT_CANDIDATE is classified as a product, then crashes before the frozen required product-verifier/parser event. The slot is left unconsumed and can be retried.

  1. Frozen goal1/CONFIRMATORY_PRODUCT_CONTROLS.json classifies product-chain responses from the exact first two visible discriminator lines, requires product syntax/name admission separately, sets parse/name-policy failure to PRODUCT_NOT_ADMISSIBLE, and requires ONE_IDENTICAL_BOUND_LEAN_VERIFIER_EVENT_PER_PRODUCT_CANDIDATE. For product_only, that verifier result is quarantined from admission; the event itself is still mandatory.

  2. Exact-head confirmatory_io.classify_product_response() accepts any response beginning with PRODUCT_PREFIX. For the exact bytes PRODUCT_PREFIX and nothing after it, body = b"", but the function still returns ConfirmatoryResponseKind.PRODUCT_CANDIDATE with product_parser_source_utf8=b"".

  3. ProductChainController.submit_response() then calls build_verification_subject() before setting _pending. VerificationSubject.__post_init__() rejects an empty non-null product_parser_source with TypeError("product_parser_source must be non-empty exact bytes"). Therefore the exact discriminator-only product response never becomes a pending verifier slot, never receives the mandatory product verifier event/parser-admission result, and never becomes a terminal PRODUCT_NOT_ADMISSIBLE record.

  4. Deterministic public-API counterexample: render_request(0) → submit_response(0, PRODUCT_PREFIX) raises → render_request(0) succeeds again because _records is still empty and _pending is still None. This is distinct from the previously recorded generic BLOCKED-response retry finding: the frozen first-two-line classifier makes these bytes a PRODUCT_CANDIDATE; the defect is that an invalid/empty declaration body bypasses the required product verifier/parser event instead of becoming a charged, non-admissible product attempt.

Required closure before scientific product-chain execution: represent discriminator-valid but parser-invalid product candidates all the way through the bound product-verifier event; record signed parser admission false (including the empty-declaration case), preserve the verifier event/cost, consume the model slot exactly once, and forbid re-render/retry. Add a deterministic regression using exactly PRODUCT_PREFIX with no body: expected outcome is one consumed attempt, one bound product-verifier event, syntax_admissible=false, product_admitted=false, and no second render/dispatch for that slot.

Current hosted failures remain blocking independently; no failed, missing, or isolated-green evidence is promoted to approval. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

@Kitahl Kitahl left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MM06 / live G1-131 — REVIEW-ONLY supplemental exact-head review for 22ca5ecd0f74da2312b5863917b0826b762c421d.

VERDICT: AMEND — 0 approval, 0 merge authority, 0 scientific PASS.

[P1] The new exact response classifiers have a command-grammar hole: Lean #... commands such as #check are not rejected by _FORBIDDEN_FINAL_TOKENS, so protocol-invalid non-tactic commands can survive classification and reach the verifier.

  1. Frozen goal1/CONFIRMATORY_BASELINES.json defines the response artifact as LEAN_TACTIC_BODY; its exact common prompt explicitly says to return only the tactic body and forbids commands. The product-chain final-answer surface likewise requires only the final tactic body under the frozen response contract.

  2. Exact-head src/supernova_goal1/confirmatory_io.py enforces final-response syntax with _FORBIDDEN_FINAL_TOKENS, but that regex enumerates imports/declarations/namespaces/options/sorry/admit and does not include Lean command forms beginning with #. classify_baseline_response() and the FINAL branch of classify_product_response() both rely on this regex.

  3. Lean's language reference defines #check term as a top-level command. The production verifier does not independently enforce the frozen tactic-body grammar: VerifierSupervisor._run_two_phase_and_record() forms solution_source = source + candidate, runtime/goal1_verifier/entrypoint.py::compile_module() compiles that full module, and export/checking is target-scoped to the configured theorem names/allowed axioms rather than proving that no extra informational command was present.

  4. Deterministic classifier counterexamples are therefore:

    • product final: FINAL_PREFIX + b"norm_num\n#check Nat\n";
    • multi-fidelity: b"-- MULTI_FIDELITY_SELF_SCORE=9999\nnorm_num\n#check Nat\n".
      Both evade the current forbidden-token regex. The product form is classified FINAL_ANSWER; the multi-fidelity form remains a verifier candidate. The extra #check is not a tactic-body response even though the classifier does not reject it.
  5. Current tests/test_confirmatory_io.py covers normal PRODUCT/FINAL/NO_ANSWER forms and score parsing but has no negative #check/#eval/other command regression.

This is distinct from the already-recorded old baseline-adapter bypass: this finding is in the newer byte-derived classifier/controller path itself, including ProductChainController and MultiFidelityController users.

Required closure before these classifiers authorize scientific state: parse/enforce the exact allowed tactic-body grammar rather than a partial denylist, or otherwise prove the entire candidate contains only the permitted tactic body/comment surface; reject top-level #check, #eval, #print, and any other non-permitted command before verifier/scientific disposition; add deterministic negative regressions for product-final and multi-fidelity responses carrying a valid proof followed by such a command. Preserve exact returned bytes/cost when blocked.

Current red/missing evidence remains blocking independently. MM06 authors no implementation edit, implementation push, merge, approval, or scientific adjudication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant