[ops]: refactor Pester into a readiness/execution/evidence service model

[svelderrainruiz]

Technical debt surfaced during local iteration

The service-model pilot exposed concrete debt that should stay visible on this standing issue.

Debt now captured locally on the pilot branch

• Receipt coupling was too weak: execution only trusted readiness_status and did not validate the actual readiness receipt artifact.
  • Local fix is now on codex/pester-service-model-pilot at 265593ac.
• Execution uploaded raw artifacts before the execution receipt existed.
  • This made evidence classification depend on heuristics instead of a first-class execution contract.
  • Local fix is also on 265593ac.
• Evidence classification was still overly summary-centric.
  • It now needs the execution receipt to distinguish seam-defect from ordinary test failures.
  • Local fix is on 265593ac.
• The pilot had no trusted PR entrypoint.
  • Local fix is now on codex/pester-service-model-pilot at 5d133cd3 via .github/workflows/pester-service-model-on-label.yml.
  • The new entrypoint uses pull_request_target plus same-owner gating so self-hosted execution never runs untrusted cross-owner fork code.
• The service-model workflows did not accept explicit checkout repository/ref inputs, so the pilot could not prove PR head truth safely.
  • Local fix is on 5d133cd3 via explicit checkout_repository / checkout_ref threading through readiness and execution.

Debt still open

• .github/workflows/pester-reusable.yml remains the active monolith and still couples:
  • preflight
  • host/bootstrap debt
  • execution
  • evidence generation
• .github/workflows/test-pester.yml still routes directly to the monolith, so the required gate has not been promoted to the service model yet.
• Host-plane/bootstrap debt is still being paid in the monolith path instead of being fully owned by readiness receipts.
• The current pilot does not yet define named execution packs strongly enough to separate:
  • merge-gate deterministic coverage
  • release certification coverage
  • diagnostics/characterization coverage
• The trusted pilot router is not proven remotely yet; the next job is to exercise it on PRs without changing the required gate.

Standing direction

• Keep the issue focused on replacing the monolith with a readiness/execution/evidence service model.
• Keep surfacing technical debt here as it is discovered locally.
• Do not switch the required gate until the additive path proves equivalent or better behavior on a trusted trigger path.

Evidence

• pilot PR: Introduce Pester service-model pilot #2068
• local receipt-driven execution cut: 265593ac
• local trusted PR entrypoint cut: 5d133cd3
• latest monolith evidence sample: full self-hosted dispatcher can remain live without pester-summary.json while event logging stops at execution start

Remote proof on 2026-03-31

• Direct pester-gate.yml dispatch on upstream branch issue/2069-pester-service-model produced a real additive proof run: 23802276898.
• The split itself worked remotely:
  • pester-run / normalize completed successfully
  • pester-evidence / evidence downloaded and classified outputs successfully before failing closed at Propagate gate outcome
• The concrete failure class was not generic anymore. Evidence classified the run as seam-defect because the gate/evidence handoff lost execution context on the skip path:
  • readiness_status arrived empty
  • execution_job_result arrived as success
  • raw execution artifact download failed because no execution artifact existed
  • execution receipt was missing
• That exposed a real contract flaw in the pilot: reusable-workflow outputs were collapsing when execution never started.
• Local fix is now on codex/pester-service-model-pilot at c2df2843.
  • pester-run.yml now has an always-on finalize job that emits execution_status, dispatcher_exit_code, and raw_artifact_name even when the execution job is skipped.
  • pester-gate.yml now passes fallback-bearing readiness and execution status into evidence.
  • pester-evidence.yml now records raw artifact download outcome explicitly instead of hiding that context.
• Remaining open debt after this slice:
  • pester-service-model-on-label.yml still is not proven remotely as a trusted PR router; direct gate dispatch is the only proven remote path so far.
  • vi-history-scenarios-windows-plan instability on #2068 is still unrelated background noise on the pilot PR surface.

[svelderrainruiz]

Remote proof update on 2026-03-31:

• Run 23802276898 on head ed299fae proved the additive service model remotely, but exposed a skip-path output-collapse defect.
  • readiness_status arrived empty
  • execution_job_result arrived as success
  • evidence had to fail closed as seam-defect
• Local fix c2df2843 added an always-on finalize contract so skipped execution no longer collapses reusable-workflow outputs.
• Run 23802588431 on head c2df2843 proved that fix.
  • readiness_status arrived as failure
  • execution_job_result arrived as skipped
  • evidence no longer inferred from empty workflow outputs
• That proof exposed the next architectural debt: execution receipt was still coupled to the raw execution artifact path.
• Local fix b6b415ec separated the execution contract from raw outputs.
  • pester-run/finalize now uploads a dedicated pester-execution-contract artifact
  • pester-evidence downloads that contract first-class and only downloads raw output when execution actually ran
• Run 23802755645 on head b6b415ec proved the new shape remotely.
  • finalize uploaded the execution contract artifact successfully
  • evidence downloaded the execution contract artifact successfully
  • raw artifact download was correctly skipped
  • classification receipt is now truthful and compact:
    • readiness-status=failure
    • execution-job-skipped
    • summary-missing
  • execution-receipt-missing is no longer part of the skip-path failure

Current remaining debt after this slice:

• pester-service-model-on-label.yml is still not proven remotely as the trusted PR router; direct pester-gate.yml dispatch remains the only proven remote pilot path.
• The additive service model is now proving the right failure class, but test-pester.yml and pester-reusable.yml still own the required gate.

[svelderrainruiz]

Trusted router proof update on 2026-03-31:

• Same-owner develop PR proof is now successful.
• PR #2072 (probe/pester-service-model-router-develop) triggered Pester service-model pilot on trusted PR label via pull_request_target.
• Run 23803961995 proved the trusted router path itself:
  • trust-context completed successfully
  • pilot executed instead of collapsing to the skip path
  • the run then failed only at the expected evidence propagation seam in the older develop-hosted service-model slice
• This narrows the standing debt:
  • trusted same-owner PR routing on develop is proven
  • fork PRs like #2070 are correctly rejected as untrusted-cross-owner-fork
  • integration-branch reachability is still separate debt because the router branch filter had to be extended for integration/**

This means the next step is not to keep questioning the trusted router on develop; it is to carry the integration-branch reachability fix into the main service-model hardening branch and keep reducing router/dispatch drift there.

[svelderrainruiz]

Service-model update from local integration proof work:

• The trusted router is now proven on a same-owner develop PR via #2072 / run 23803961995.
• The remaining router debt is not admission on develop; it is integration-branch reachability.
• I had already proven that the integration pilot branch needed explicit integration/** coverage in .github/workflows/pester-service-model-on-label.yml.
• I have now carried that reachability fix into the main hardening branch for #2070 so the standing-priority path owns the debt directly.

Technical debt status:

• #2070 now includes skip-path hardening plus integration-branch router reachability.
• The next debt after this is router/dispatch drift and eventual promotion away from the monolithic test-pester.yml / pester-reusable.yml gate.

[svelderrainruiz]

Opened follow-up PR #2073 to carry the remaining integration-branch router reachability debt that missed #2070.

• PR: [ops]: allow trusted Pester router on integration branches #2073
• Scope: allow .github/workflows/pester-service-model-on-label.yml to target integration/**
• Reason: the trusted router is already proven on same-owner develop PRs; integration-mounted upstream proofs were still filtered out by branch policy.

Current debt split:

• #2070 merged skip-path hardening into develop
• #2073 carries integration reachability
• remaining standing debt stays router/dispatch drift and eventual promotion away from the monolith