Skip to content
ci

Stop 25: correct two counts in the boundary-2 state block (#141) #325

Sign in to view logs

Stop 25: correct two counts in the boundary-2 state block (#141)

Stop 25: correct two counts in the boundary-2 state block (#141) #325

Workflow file for this run

.github/workflows/ci.yml at f4a88c7
# This repository's artifacts are prose, shell tools and YAML contracts, so CI checks those
# rather than compiling anything.
#
# Link rot is the real risk. During the 2026-08-09 verification, 7 of 43 sources redirected,
# including the entire Codex documentation host. A curriculum whose "verified reading" list
# 404s has quietly stopped being verified, and nobody notices until a cohort hits it.
name: ci
on:
pull_request:
push:
branches: [main]
schedule:
# Vendor agent docs move constantly. Weekly, so rot is found before a cohort finds it.
- cron: '17 6 * * 1'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
shell:
name: shell tools
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
# fetch-depth: 0 so this checkout carries `origin/main`. The review hook test resolves
# its glob-liveness case against the TRUNK tree, never the checkout, and under the
# default shallow fetch there is no trunk ref to read — so that case SKIPS, and a skip
# is not a pass, so the suite exits non-zero. The ref makes the check RUN.
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Syntax check every tool
run: for f in tools/*.sh .claude/hooks/*.sh; do bash -n "$f" || exit 1; done
- name: ShellCheck
uses: ludeeus/action-shellcheck@00b27aa7cb85167568cb48a3838b75f4265f2bca # master as of 2026-09-21, pinned
with:
scandir: ./tools
severity: warning
- name: ShellCheck the hooks
uses: ludeeus/action-shellcheck@00b27aa7cb85167568cb48a3838b75f4265f2bca # master as of 2026-09-21, pinned
with:
scandir: ./.claude/hooks
severity: warning
# opencode, git and the reviewer are all stubbed — no network, no tokens, no model
# calls. A hook that silently does nothing is worse than no hook, so the cases
# asserting the reviewer was NOT called run here too.
- name: The review hook behaves
run: ./.claude/hooks/opencode-review.test.sh
- name: The spine status renderer still refuses
run: ./tools/verify-render-spine-status.sh
# Offline fixture sets for the Track B checkers: no model, network, docker or sibling
# checkout. Run by hand only, they were words a human chose to run; here they gate.
- name: The agent overlay checker still refuses
run: ./tools/verify-agent-overlay-checker.sh
- name: The overlay parity checker still refuses
run: ./tools/verify-overlay-parity-checker.sh
- name: The skill activation classifier still refuses
run: ./tools/verify-skill-activation.sh
- name: The state re-read counter still refuses
run: ./tools/verify-count-state-reread.sh
- name: The completion contract checker still refuses
run: ./tools/verify-completion-contract-checker.sh
- name: The policy gate still refuses
run: ./tools/verify-policy-gate.sh
- name: The repair limit still refuses
run: ./tools/verify-repair-limit.sh
- name: The run state checker still refuses
run: ./tools/verify-run-state-checker.sh
# A script nothing ShellChecks looks exactly like one that lints clean: both are a
# green run with no output. This compares every tracked *.sh against the directories
# the two steps above scan — pinned in tools/shellcheck-scanned.txt, never parsed out
# of this file — and against the exempt table's written reasons.
- name: Every tracked script is ShellChecked or exempt
run: ./tools/verify-shellcheck-coverage.sh
verify-sh:
name: verify-sh — every exit code is produced
runs-on: ubuntu-latest
timeout-minutes: 5
# Its own job rather than a step of `shell tools`: the fixture set resolves
# ../agent-observatory-benchmarks, and actions/checkout cannot write outside the
# workspace, so both repositories are checked out side by side underneath it.
steps:
- uses: actions/checkout@v7
with:
path: agent-learning-lab
# The sibling is pinned to `main`, which moves: a benchmarks commit can turn this job red.
- uses: actions/checkout@v7
with:
repository: UnityInFlow/agent-observatory-benchmarks
ref: main
path: agent-observatory-benchmarks
- name: Every exit code verify-sh can return is produced
run: bash tools/verify-verify-sh.sh
working-directory: agent-learning-lab
yaml:
name: contracts parse
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: '3.12'
- name: Install PyYAML
run: pip install --quiet pyyaml
# A rubric or run-record template that does not parse is a contract nobody can apply.
- name: Parse every YAML contract
run: |
python3 - <<'PY'
import glob, sys, yaml
bad = 0
for f in glob.glob('templates/*.yaml') + glob.glob('benchmark/rubrics/*.yaml') + glob.glob('.opencode/fixtures/*.yaml'):
try:
yaml.safe_load(open(f))
print('ok ', f)
except Exception as e:
print('FAIL', f, e); bad = 1
sys.exit(bad)
PY
run-record:
name: a bare token count is rejected
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: '3.12'
- name: Install PyYAML
run: pip install --quiet pyyaml
# The template holding the provenance shape is Layer 3 on its own. This job is what
# makes it Layer 2, and the fixtures are what prove the job still rejects anything.
- name: Every fixture produces its registered exit code
run: ./tools/verify-run-record-validator.sh
score-output:
name: a run that found something is not discarded
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
# The scorer producing no sheet is E-001's most informative possible result. Until
# 2026-08-27 the harness gave it the same exit code as a crash, and the experiment's
# Exclusions then said to re-run it as infrastructure. This job is what keeps the
# five outcomes distinguishable across all three agents; the fixtures are what prove
# it still tells them apart.
- name: Every model-output fixture produces its registered class
run: ./tools/verify-model-output-classifier.sh
sheet-categories:
name: a sheet missing a category is rejected
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
# A sheet carrying three of the rubric's four categories used to pass everything: the
# JSON schema said minItems 1 with no maxItems, and the classifier only asks whether a
# `categories:` key exists. A missing cell is not a null cell, and E-001's dependent
# variable is a ratio whose denominator is the cell count. This job is what keeps
# check-sheet-categories.sh discriminating; the cases are what prove it.
- name: The category-set check still rejects a short sheet
run: ./tools/verify-sheet-category-checker.sh
# The category check admits a sheet by SHAPE; this admits the set of sheets by IDENTITY.
# Thirty-four well-formed sheets could still be two for one run and none for another, so
# every owed run id (plus the probe id) must have exactly one admitted 2026-09-07 sheet.
- name: Every owed second-reader run id has exactly one admitted sheet
run: bash evidence/second-reader/verify-sheet-coverage.sh
# A coverage check that cannot fail is a list, not a check. Feed it one run id that has
# no sheet and require a non-zero exit, so a verifier that stops reading its ids file
# (or always exits 0) turns this job red instead of staying quietly green.
- name: The coverage check refuses a run id with no sheet
run: |
ids="$RUNNER_TEMP/made-up-run-ids.txt"
printf 'deadbeef-0000\n' > "$ids"
if bash evidence/second-reader/verify-sheet-coverage.sh "$ids"; then
echo "verify-sheet-coverage.sh admitted made-up run id deadbeef-0000; it must refuse" >&2
exit 1
fi
board-freshness:
name: a published board does not outlive its source
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0 # the check reads HANDOFF.md's own commit history, not just HEAD
# HANDOFF.md cites artifact URLs as "the readable view of this file". A view is a COPY,
# and the guard against a stale copy was a sentence asking the reader to trust the file
# instead — Layer 3, which failed on 2026-08-30 when the source moved eleven times in a
# session and the board went on describing superseded work. This makes staleness fail
# loudly instead of silently. It cannot keep a board correct; only republishing does.
- name: Every declared board still matches HANDOFF.md
run: ./tools/check-board-freshness.sh
# The check itself gets checked, for the reason check-sheet-categories.sh exists as a
# cautionary tale: it shipped ShellCheck-clean with nine passing fixtures and a green
# job, and it silently admitted what it was built to reject. One of these nine cases
# found a real parse bug in the checker on the day it was written.
- name: The freshness check still rejects
run: ./tools/verify-board-freshness-checker.sh
phase-contract:
name: a narrated phase is not a phase
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
# B5's gate is "phase markers observable in the transcript, no code written before
# DESIGN". The first half is greppable and the second half is not: an agent can edit
# the repository in turn two and print all six markers in its closing message, and a
# checker that reads the final text reports six of six and passes it. That run is the
# plain baseline wearing phase markers, and it is the arm this job exists to make
# impossible to score as a treatment. Case C is that exact transcript.
- name: The phase-contract check still rejects a narrated run
run: ./tools/verify-phase-contract-checker.sh
run-gate:
name: a gate-failing run is not scored
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v7
# Decision D admits a B2 run on the evaluator's recorded verdict instead of a fixture
# name. The invariant it protects — only gate-passing submissions get scored — is worth
# exactly as much as this check's willingness to refuse. A version that stopped refusing
# would look identical in every passing run.
- name: The run-gate check still refuses
run: ./tools/verify-run-gate-checker.sh
links:
name: verified reading is still verified
runs-on: ubuntu-latest
timeout-minutes: 15
# Advisory on PRs — a vendor moving a doc is not the contributor's fault and must not
# block their change. It still fails the scheduled run, which is where rot gets noticed.
continue-on-error: ${{ github.event_name == 'pull_request' }}
steps:
- uses: actions/checkout@v7
- name: Check every source link
run: ./tools/check-links.sh