Skip to content

docs: GPT-5.5 preferred for cross-model review + README quickstart - #230

Merged
BaseInfinity merged 1 commit into
mainfrom
roadmap-gpt55-wizard-update
Apr 24, 2026
Merged

docs: GPT-5.5 preferred for cross-model review + README quickstart#230
BaseInfinity merged 1 commit into
mainfrom
roadmap-gpt55-wizard-update

Conversation

@BaseInfinity

@BaseInfinity BaseInfinity commented Apr 24, 2026

Copy link
Copy Markdown
Owner

Summary

Two doc updates:

  1. Cross-model review section in CLAUDE_CODE_SDLC_WIZARD.md now says GPT-5.5 preferred, GPT-5.4 fallback. Codex CLI auto-picks best available — no config change needed.
  2. README.md gets a new Optional: Cross-Model Review (Codex) section with 3-command setup and one-liner example. Frames it as easy to adopt.

xhigh reasoning remains non-negotiable (existing guidance, reinforced).

Why

Per memory research_gpt_5_5_release.md: GPT-5.5 shipped April 2026. OpenAI positions it as "noticeably stronger than Claude Opus 4.7 at reasoning/autonomy." Relevant to our reviewer-tier strategy — no executor-tier change yet (vendor quote ≠ evidence, wait for independent benchmark).

User callout (2026-04-24): "chat gpt5.5 dropped we need to update the sdlc-wizard to include that for cross model review if people have it on" + "prefer xhigh for cross model review... mention in README how easy it is to use."

Test plan

  • bash tests/test-doc-consistency.sh — 22/0 green
  • bash tests/test-workflow-triggers.sh — green
  • No code changes, no test impact expected in CI validate.
  • e2e-* will red on API credit cap as usual (known canary).

Scope excluded

  • No executor-tier change (Opus 4.7 stays as default builder — wait for independent 5.5 benchmarks)
  • Not updating historical "cross-model audit (Codex GPT-5.4)" reference at CLAUDE_CODE_SDLC_WIZARD.md:104 — that's accurate historical record of what happened

@BaseInfinity
BaseInfinity merged commit 2fb12ea into main Apr 24, 2026
4 of 5 checks passed
@BaseInfinity BaseInfinity mentioned this pull request Apr 24, 2026
Merged
10 tasks
BaseInfinity added a commit that referenced this pull request Apr 24, 2026
@BaseInfinity
roadmap(#212): Option 1 — de-require e2e, shepherd becomes advisory
156f07d 
Massive cleanup: 1530 lines deleted from ci.yml, cron disabled on weekly/monthly API-burning workflows, pr-review.yml simplified to wait on validate only, obsolete tests stubbed with clear #212-traceable messages.

Codex: 10/10 CERTIFIED (2 rounds: 2/10 plan → 3/10 code → 10/10 code).
CI: validate green. e2e checks no longer exist.
Branch protection: patched to require validate only (done just now).
Follow-ups: ROADMAP #230 (shepherd baseline/candidate), #231 (weekly/monthly cleanup).
This was referenced Apr 24, 2026
Merged
Merged
@BaseInfinity
docs: GPT-5.5 preferred for cross-model review + README quickstart
693baf4 
OpenAI shipped GPT-5.5 April 2026. Codex CLI picks up best available
model automatically — if user has 5.5 access they get 5.5, otherwise
5.4. No config change on user side.

Changes:
- CLAUDE_CODE_SDLC_WIZARD.md cross-model section: "GPT-5.5 preferred,
  GPT-5.4 fallback" + clarification that Codex CLI auto-picks best
  available.
- README.md: new "Optional: Cross-Model Review (Codex)" section with
  3-command setup (npm i / export key / codex --version), one-liner
  example invocation, and link to the full protocol. Framed as easy
  to adopt — that was the ask.
- Reinforces xhigh is non-negotiable (existing memory lesson).

No code changes. No test impact. Doc consistency + workflow triggers
both green.
BaseInfinity added a commit that referenced this pull request Apr 27, 2026
@BaseInfinity
roadmap: mark 5 shipped items DONE (#218 #221 #224 #225 #226)
c64a50a 
All 5 entries had been implemented but their title cells still read OPEN.
Adding DONE markers + commit/PR/version references for archaeology:

- #218 MCP-tool hook audit → PR #250, v1.41.1
- #221 post-mortem wizard integration → PR #249, v1.41.0
- #224 prompt-hook-fires-once test → PR #243, v1.38.0
  (tests/test-prompt-hook-fires-once.sh)
- #225 cleanupPeriodDays guidance → PR #248, v1.40.1
- #226 P1 fake-CI honest labeling → commit 13dccc4
  (Option B: judge-consistency caveat; Option A true N-trial deferred to
  ROADMAP #230 / #212 Option 1 follow-up)

Same closure-gap pattern just hit on #209 (PR #255). Five stale markers
is enough signal that a roadmap-hygiene check would be worth a small
piece of automation — track separately.
BaseInfinity added a commit that referenced this pull request Apr 27, 2026
@BaseInfinity
roadmap: mark 5 shipped items DONE (#218 #221 #224 #225 #226) (#256)
97db941 
All 5 entries had been implemented but their title cells still read OPEN.
Adding DONE markers + commit/PR/version references for archaeology:

- #218 MCP-tool hook audit → PR #250, v1.41.1
- #221 post-mortem wizard integration → PR #249, v1.41.0
- #224 prompt-hook-fires-once test → PR #243, v1.38.0
  (tests/test-prompt-hook-fires-once.sh)
- #225 cleanupPeriodDays guidance → PR #248, v1.40.1
- #226 P1 fake-CI honest labeling → commit 13dccc4
  (Option B: judge-consistency caveat; Option A true N-trial deferred to
  ROADMAP #230 / #212 Option 1 follow-up)

Same closure-gap pattern just hit on #209 (PR #255). Five stale markers
is enough signal that a roadmap-hygiene check would be worth a small
piece of automation — track separately.
@BaseInfinity BaseInfinity mentioned this pull request Apr 28, 2026
Merged
6 tasks
BaseInfinity added a commit that referenced this pull request Apr 28, 2026
@BaseInfinity
feat(shepherd): --compare-baseline flag for delta comparison (v1.49.0…
3378e82 
…, ROADMAP #230) (#274)

* feat(shepherd): --compare-baseline flag for delta comparison (v1.49.0)

ROADMAP #230 — local-shepherd.sh --compare-baseline runs the same scenario
on main (via git worktree) AND the current branch, computes score delta,
posts comparison check-run + PR comment. Unblocks #231 Phase 2 weekly-update
migration ($25-55/week saved). Single-run mode unchanged.

- Atomic dual-row history append (Codex P1: deferred from baseline block;
  candidate failure leaves zero comparison rows, no orphan baselines)
- BASELINE_TMPRUN nested under TMPRUN so existing trap covers cleanup
  (Codex P1: previously leaked on early failure paths)
- Provenance fields computed once, reused for both rows
- Same scenario for both runs (apples-to-apples; selected by PR number)

9 new quality tests (22/22 total in test-local-shepherd.sh). Codex round 2
CERTIFIED 9/10 (round 1 found 2 P1s, both fixed with regression tests).

* fix(test): mock evaluator state file uses explicit tmpdir, not $TMPDIR

CI failure on Linux: $TMPDIR is unset on GHA ubuntu runners, so the mock
evaluator's state file resolved to /_compare_eval_state (root path, no
write permission). Both calls fell through to the "not-first" branch
returning 9/10 instead of 7 then 9, masking the delta and breaking
test_compare_baseline_posts_delta_summary.

Fix: state file path is the test's own tmpdir, captured at heredoc-
expansion time. Reproducible regardless of $TMPDIR availability.
BaseInfinity added a commit that referenced this pull request Apr 28, 2026
@BaseInfinity
roadmap: mark #230 DONE, #231 Phase 2 ready to start, token bloat aud…
472ca81 
…it fully closed

- #230 (shepherd compare-baseline) shipped v1.49.0 PR #274
- #231 Phase 2 unblocked — was "gated on #230", now ready to pick up
- Token bloat audit Phases 1-3 all done (dedupe + audit tool + SKILL trim);
  audit reports 0 trim candidates after v1.48.0 PR #273
@BaseInfinity BaseInfinity mentioned this pull request Apr 28, 2026
Merged
9 tasks
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap: prune low-value items based on 2026-04-24 user review
d7326ed 
User audit: "with all the opus fixes i dont think we need to do
adaptive thinking test anymore... im confused why does that [5.5
calibration] need API and replay harness huh this... we should audit
[weekly/monthly] when replacing them if we even need it."

Closed:
- #214 adaptive thinking A/B: moot given xhigh/max floor mandate.
  Saves $12 + spares API burn.
- #213 ship DISABLE_ADAPTIVE_THINKING default: same logic. xhigh
  floor > default band-aid env var. Keeps as opt-in for power users.
- #223 GPT-5.5 in review tier: already shipped via Codex config
  (model = "gpt-5.5" default). Formal calibration was academic.

Updated:
- #230 shepherd baseline/candidate: marked P3 nice-to-have with user
  quote "just get feature parity once." Not urgent.
- #231 weekly/monthly cleanup: audit outcome embedded. weekly-update
  actively useful (migrate its 9 API blocks). monthly-research mostly
  fails on cron + rarely run manually (last success 2026-03-27) +
  519 lines of low-value research-issue creation → proposed deletion
  instead of migration.

No code changes. doc-consistency green.
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap(#212): Option 1 — de-require e2e, shepherd becomes advisory
7444bef 
Kill the Anthropic API burn in CI by removing the e2e simulation jobs
entirely. Shepherd (shipped in #229) stays as the local advisory tool.

Rationale (per Codex architect review .reviews/architect-proposal.md):
Original plan (pending check-run + local overwrite) got 2/10 NOT
CERTIFIED because branch protection is pinned to GitHub Actions
app_id:15368 — a local gh OAuth token cannot satisfy a required check.
Option 1 matches reality (we admin-merge around red e2e every time
anyway) and cleanly unblocks fork PRs.

Changes:
- ci.yml: remove e2e-quick-check + e2e-full-evaluation jobs (1534 lines
  deleted). Keep validate + cleanup-old-comments.
- weekly-update.yml: disable cron (9 claude-code-action blocks still
  present, tracked for removal in ROADMAP #231). Manual via
  workflow_dispatch only.
- monthly-research.yml: same — disable cron.
- pr-review.yml: wait on validate only (e2e check no longer exists).
- Tests: stub obsolete test-degradation + test-workflow-triggers tests
  that asserted deleted ci.yml structure. Shepherd parity test reworked
  to assert signatures exist in shepherd (ci.yml parity is moot now).
- ROADMAP: #212 updated to "OPTION 1 SHIPPING"; added #230 (shepherd
  baseline/candidate extension) + #231 (weekly/monthly workflow cleanup).

What this PR does NOT change (intentional, follow-up items):
- Shepherd stays candidate-only (no baseline comparison) — ROADMAP #230
- Weekly/monthly workflows still have claude-code-action blocks —
  ROADMAP #231
- Branch protection is NOT patched in this PR — that PATCH happens
  separately via `gh api` just before admin-merge to avoid a window
  where new PRs have no e2e check producer but still require it

Signal loss accepted: CI no longer auto-scores PRs. Shepherd must be
run manually on each PR the maintainer wants to score. Matches today-
reality of admin-merge-every-time.

Tests passing locally:
- test-workflow-triggers: 54/0
- test-local-shepherd: 13/0
- test-hooks: 115/1 (pre-existing failure)
- test-degradation-detection: 14/0
- test-cusum: pass
- test-doc-consistency: 22/0

Branch protection will be patched separately via:
  gh api repos/.../branches/main/protection/required_status_checks \\
    --method PATCH --field checks=[{context=validate,app_id=15368}]
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap(#212): Option 1 — de-require e2e, shepherd becomes advisory
8d6bfaa 
Massive cleanup: 1530 lines deleted from ci.yml, cron disabled on weekly/monthly API-burning workflows, pr-review.yml simplified to wait on validate only, obsolete tests stubbed with clear #212-traceable messages.

Codex: 10/10 CERTIFIED (2 rounds: 2/10 plan → 3/10 code → 10/10 code).
CI: validate green. e2e checks no longer exist.
Branch protection: patched to require validate only (done just now).
Follow-ups: ROADMAP #230 (shepherd baseline/candidate), #231 (weekly/monthly cleanup).
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap(#231): full migration plan + Codex P1/P2 revisions
055e6d8 
4-phase kill-or-port plan for weekly-update.yml + monthly-research.yml
after full audit (7 jobs, ~\$120/month API burn, ~0 merged artifacts in 30d).

Phase 1: ONLY delete monthly-research.yml (revised per Codex 5/10).
Phase 2: Ship #230 compare-baseline + --strip-paths; then delete prove-it-test.
Phase 3: Port check-updates + scan-community + version-test to local shepherd.
Phase 4: Shrink weekly-update.yml to 0-100 lines.

Codex P1 catches incorporated:
- .github/last-community-scan.txt CANNOT be deleted in Phase 1 (still
  consumed by scan-community + ci.yml:66)
- tests/test-prove-it.sh is a test suite, NOT <overlap-paths> runner —
  deferred prove-it-test deletion to Phase 2 after building a real runner
- Test cleanup scope expanded — Codex found 8+ monthly assertions
  (lines 56, 882, 1233, 1530, 1730, 1861, 2647, 2668)
- Docs cleanup list expanded: CI_CD.md, ARCHITECTURE.md, AUTO_SELF_UPDATE.md,
  CLAUDE_CODE_SDLC_WIZARD.md:445
- Dropped the /monthly-research skill proposal — research-in-conversation
  is fine, no shipping replacement needed

Plan CERTIFIED after revisions. Codex P1 count: 3 → 0. Score 5/10 → ready
to execute Phase 1 as follow-up PR.

No code changes this PR. Just ROADMAP update.
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap: mark 5 shipped items DONE (#218 #221 #224 #225 #226) (#256)
9891f4f 
All 5 entries had been implemented but their title cells still read OPEN.
Adding DONE markers + commit/PR/version references for archaeology:

- #218 MCP-tool hook audit → PR #250, v1.41.1
- #221 post-mortem wizard integration → PR #249, v1.41.0
- #224 prompt-hook-fires-once test → PR #243, v1.38.0
  (tests/test-prompt-hook-fires-once.sh)
- #225 cleanupPeriodDays guidance → PR #248, v1.40.1
- #226 P1 fake-CI honest labeling → commit 13dccc4
  (Option B: judge-consistency caveat; Option A true N-trial deferred to
  ROADMAP #230 / #212 Option 1 follow-up)

Same closure-gap pattern just hit on #209 (PR #255). Five stale markers
is enough signal that a roadmap-hygiene check would be worth a small
piece of automation — track separately.
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
feat(shepherd): --compare-baseline flag for delta comparison (v1.49.0…
8c0a18d 
…, ROADMAP #230) (#274)

* feat(shepherd): --compare-baseline flag for delta comparison (v1.49.0)

ROADMAP #230 — local-shepherd.sh --compare-baseline runs the same scenario
on main (via git worktree) AND the current branch, computes score delta,
posts comparison check-run + PR comment. Unblocks #231 Phase 2 weekly-update
migration ($25-55/week saved). Single-run mode unchanged.

- Atomic dual-row history append (Codex P1: deferred from baseline block;
  candidate failure leaves zero comparison rows, no orphan baselines)
- BASELINE_TMPRUN nested under TMPRUN so existing trap covers cleanup
  (Codex P1: previously leaked on early failure paths)
- Provenance fields computed once, reused for both rows
- Same scenario for both runs (apples-to-apples; selected by PR number)

9 new quality tests (22/22 total in test-local-shepherd.sh). Codex round 2
CERTIFIED 9/10 (round 1 found 2 P1s, both fixed with regression tests).

* fix(test): mock evaluator state file uses explicit tmpdir, not $TMPDIR

CI failure on Linux: $TMPDIR is unset on GHA ubuntu runners, so the mock
evaluator's state file resolved to /_compare_eval_state (root path, no
write permission). Both calls fell through to the "not-first" branch
returning 9/10 instead of 7 then 9, masking the delta and breaking
test_compare_baseline_posts_delta_summary.

Fix: state file path is the test's own tmpdir, captured at heredoc-
expansion time. Reproducible regardless of $TMPDIR availability.
BaseInfinity added a commit that referenced this pull request May 5, 2026
@BaseInfinity
roadmap: mark #230 DONE, #231 Phase 2 ready to start, token bloat aud…
bf33bae 
…it fully closed

- #230 (shepherd compare-baseline) shipped v1.49.0 PR #274
- #231 Phase 2 unblocked — was "gated on #230", now ready to pick up
- Token bloat audit Phases 1-3 all done (dedupe + audit tool + SKILL trim);
  audit reports 0 trim candidates after v1.48.0 PR #273
@BaseInfinity BaseInfinity mentioned this pull request May 5, 2026
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@BaseInfinity