roadmap(#231 Phase 1): delete monthly-research.yml by BaseInfinity · Pull Request #235 · BaseInfinity/claude-sdlc-harness

BaseInfinity · 2026-04-24T20:26:07Z

Summary

Delete .github/workflows/monthly-research.yml (519 lines, 2 API blocks, zero merged artifacts in 30d while burning $11-23/month)
Stub all 17 test_monthly_* assertions in tests/test-workflow-triggers.sh with #231 Phase 1 pattern
Update live docs (CI_CD.md, ARCHITECTURE.md, plans/AUTO_SELF_UPDATE.md) to mark monthly-research REMOVED

Scope is narrow: ONLY deletes the monthly workflow. Weekly-update migration is Phase 2+, gated on ROADMAP #230.

Why

"Perplexity-as-CI" antipattern. Scheduled LLM research calls produced nothing mergeable because research questions are better asked inline in a Claude Code session with full repo context. Phase 1 extracts the obvious kill while deferring the prove-it-runner + local shepherd comparison work.

Explicitly Deferred (per Codex plan review)

.github/last-community-scan.txt kept — still consumed by scan-community job + ci.yml:66 test. Phase 3 handles.
prove-it-test job kept — tests/test-prove-it.sh is a test suite, NOT an <overlap-paths> runner. Phase 2 builds that runner.
No /monthly-research skill shipped — research happens inline in a Claude Code session.

Codex cross-model review

Round 1: 7/10 NOT CERTIFIED (2 findings: live docs + research-update/ literal)
Round 2: 8/10 NOT CERTIFIED (my doc fix broke a test assertion)
Round 3: 9/10 CERTIFIED

Test plan

bash tests/test-workflow-triggers.sh → 165/165 pass
bash tests/test-self-update.sh → 153/153 pass
bash tests/test-doc-consistency.sh → 22/22 pass
bash tests/test-compliance.sh → 10/10 pass
All 7 remaining workflow YAMLs parse
grep -rn 'research-update/' → 0 matches
Validate CI job green

Zero merged artifacts in 30d while burning $11-23/month in Anthropic API. 'Perplexity-as-CI' was a poor fit — research happens inline in a Claude Code session now, not as a scheduled LLM call. Scope (Codex-hardened, 9/10 CERTIFIED round 3): - git rm .github/workflows/monthly-research.yml (519 lines, 2 API blocks) - Stub all 17 test_monthly_* assertions in tests/test-workflow-triggers.sh with 'n/a per #231 Phase 1 (monthly-research.yml deleted)' pattern - Update live docs to mark monthly-research REMOVED: - CI_CD.md (workflow table, overview section, secrets table) - ARCHITECTURE.md (file tree) - plans/AUTO_SELF_UPDATE.md (5 sections: banner, unified pattern, deep-dive, summary table, files structure, cost table, design row) - ROADMAP.md #231 Phase 1 marked DONE 2026-04-24 - Historical audit tables left intact (past incidents, not live docs) Explicitly deferred per Codex plan review: - .github/last-community-scan.txt kept (still consumed by scan-community) - prove-it-test job kept (no <overlap-paths> runner exists yet; Phase 2) - No /monthly-research skill shipped Tests: 165/165 workflow-trigger tests pass. YAML parse green on all 7 remaining workflows. Codex review: round 1 (7/10, 2 findings — docs + research-update/ literal), round 2 (8/10, test broke due to my doc-wording change), round 3 CERTIFIED 9/10.

github-actions · 2026-04-24T20:27:24Z

Claude encountered an error —— View job

I'll analyze this and get back to you.

* release: v1.37.0 Bundles two Codex-certified PRs: - PR #235 (ROADMAP #231 Phase 1): delete monthly-research.yml — 519 lines + 4 claude-code-action steps removed, zero merged artifacts in 30d, $11-23/mo API burn eliminated. Stubbed 17 test_monthly_* assertions (#231 Phase 1 pattern), marked 5 live-doc sections REMOVED/historical. Codex 3-round: 9/10. - PR #236 (ROADMAP #217): loud WARNING below xhigh effort — max preferred, xhigh floor, single source of truth. Dupe effort check removed from instructions-loaded-check.sh. 2 TDD tests + 1 regression test. Codex 3-round: 10/10. Version parity: - package.json, SDLC.md metadata + table, CLAUDE_CODE_SDLC_WIZARD.md (2 refs), .claude-plugin/plugin.json, .claude-plugin/marketplace.json → 1.37.0 - skills/update/SKILL.md Step 3 example refreshed - CHANGELOG entry "[1.37.0] - 2026-04-24" ROADMAP: - #232: /update-wizard should mimic 'claude update' UX - #233: mixed-mode coder+reviewer tier (Sonnet coder + Opus reviewer for simple repos, inspired by codex-sdlc-wizard) - #234: GitHub About populated (description, homepage, 8 topics) Release review (Codex round 1 → 2): - REL-001 fixed: commit now exists on release branch (was empty) - REL-002 fixed: '2 claude-code-action blocks' corrected to '4 steps' - REL-003 fixed: .reviews/handoff.json + untracked .claude/hooks excluded * release(v1.37.0): fix CHANGELOG line 26 count (2 → 4 claude-code-action steps) Codex round-2 recheck caught that the 'Removed' section bullet still said '2 API blocks' — line 11 was fixed, line 26 was not. Align with reality: pre-delete workflow had 4 uses: anthropics/claude-code-action@v1 steps.

#301) User call-out 2026-05-01. Research-only row asking whether AI Evals methodology (pre-deploy validation + post-deploy runtime monitoring, quality gates) adds anything beyond what we already have: Tier 1/2 E2E + score-history + CUSUM + token-spike. Likely NO-GO since we already implement the methodology, just under different naming. ~1-2 hr write-up when picked up.

Verdict: NO-GO. The Thoughtworks Decoder article is methodology-only; every layer it describes (pre-deployment validation, post-deployment monitoring, quality gates, continuous oversight, performance consistency, output accuracy, error-mode catching, model-evolution tracking) already has a working analog in this repo's pipeline. Only candidate gap: bias/alignment evaluation. Out of scope — the wizard is SDLC enforcement, not LLM ethics; bias checks belong to model providers and domain-specific tools. Pattern continues with prior NO-GO research items #76 (Promptfoo) and #77 (constrain-to-playbook prompt). External-methodology audits keep validating our pipeline is mature; revisit only when an external source points to a specific technique we don't have. Closes ROADMAP #235.

* release: v1.65.0 — paperwork closes for #210 + #235 Two roadmap-only closures bundled into a single release: - #210 Node 24 false-green test: already shipped in PR #217 (045c938) on 2026-04-23 with the defensive guard + committed negative control. Audit 2026-05-04 confirms zero workflows use oven-sh/setup-bun, all 15 tests green, full action surface Node 24-compatible. Hard deadline June 2 met. - #235 Thoughtworks AI Evals methodology audit: NO-GO verdict at .reviews/research-235-ai-evals.md. Article is methodology-only; every layer already has an analog in our pipeline (Tier 1/2 evaluator + score-history + CUSUM + token-spike + cross-model review + SDP). Only candidate gap is bias/alignment eval — out of scope. Pattern continues with #76 (Promptfoo) and #77 (constrain-to-playbook) NO-GOs. Zero code changes. All version files synced 1.64.0 → 1.65.0. * fix(release-1.65.0): untrack runtime-state files + gitignore Two files were accidentally included in the v1.65.0 release commit: - .claude/scheduled_tasks.lock (CC scheduler runtime state, per-machine) - .metrics/token-history.jsonl (local token-spike-check.sh metrics) Both are runtime-state files that should never be in version control. git rm --cached removes them from tracking; .gitignore additions prevent future accidental commits via 'git add -A'.

* release: v1.66.0 — paperwork close for #95 + roll-up of PR #309 / issue #308 Closes ROADMAP #95 (Nous Research competitive audit) with NO-GO verdict — different layer of the stack (Nous makes models + agent frameworks; we enforce SDLC process). Fourth external-product audit to land NO-GO (after #76 Promptfoo, #77 constrain-to-playbook, #235 Thoughtworks AI Evals). Pattern: the wizard's niche keeps re-validating against external benchmarks. Also rolls up paperwork for in-flight items already shipped this turn: - PR #309 (codex sibling callout near top of README + wizard doc) - gh issue close 308 (4/4 API entries audited, zero wizard changes — Sonnet 1M retirement only affects 4.5/4, our mixed-mode tier pins to 4.6) Zero code changes. * fix(release-1.66.0): trim update SKILL.md changelog list to stay under 5K-token threshold

#311) Closes ROADMAP #99 with NO-GO verdict. AutoGPT pivoted to "agent platform" (184k stars, 102 releases, latest 2026-04-29) — same layer as Claude Code / Codex CLI / OpenCode, not a target for SDLC enforcement. Three blockers: (a) layer mismatch, (b) no hook primitive (blocks ≠ pre-tool-call hooks), (c) audience mismatch (agent builders, not interactive SWE workflows). 5/5 external-product audits NO-GO (continues #76 Promptfoo, #77 constrain- to-playbook, #235 Thoughtworks AI Evals, #95 Nous Research). Pattern: external-product audits keep validating the wizard's niche. Also collapses #231 Phase 2-4 entries in skills/update/SKILL.md changelog list to a single summary line, reclaiming ~100 tokens to stay under the 5K-token session-load threshold. Zero code changes.

…243 follow-up (#312) Closes ROADMAP #97 with NO-GO + one validating parallel: April 2026 "Automated Alignment Researchers" paper conceptually parallels our cross- model review pattern (LLM-as-reviewer-of-LLM via "scalable oversight"). Our implementation predates the paper and already mitigates its noted weaknesses (reward hacking, limited generalization) via vendor-diverse adversarial framing + verification checklist. RSP, Transparency, Constitution, Economic Futures audited or skipped as off-topic for third-party tooling. Also closes #243 follow-up: token-spike-check.sh has accumulated 8 rows in .metrics/token-history.jsonl on maintainer machine, well above the 5-record rolling-baseline threshold. SessionStart filter + transcript-dir resolution working as designed. 6/6 external audits NO-GO (continues #76, #77, #95, #99, #235). Trim: condensed 1.65.0 + 1.66.0 changelog entries in skills/update/SKILL.md to keep the file under the 5K-token session-load threshold. Zero code changes.