Skip to content

roadmap(#218, #219): CC 2.1.118 follow-ups — MCP-tool hooks + #198 re-verify - #211

Merged
BaseInfinity merged 3 commits into
mainfrom
roadmap-218-219
Apr 23, 2026
Merged

roadmap(#218, #219): CC 2.1.118 follow-ups — MCP-tool hooks + #198 re-verify#211
BaseInfinity merged 3 commits into
mainfrom
roadmap-218-219

Conversation

@BaseInfinity

Copy link
Copy Markdown
Owner

Summary

Two new backlog items from CC 2.1.118 audit. Neither is urgent; both captured so the next wizard cycle doesn't forget them.

Why a PR and not direct-to-main

Branch protection blocks direct pushes to main, per repo settings.

Test plan

  • CI green (doc-only change, should be trivial)

…-verify

#218: evaluate type: "mcp_tool" hooks (CC 2.1.118 new capability).
      Bias toward keeping bash hooks unless quality proves out.
#219: re-verify our #198 model-pin guidance against CC 2.1.117
      restart-persistence behavior. Zero-API, ~10 min check.
@github-actions

Copy link
Copy Markdown
Contributor

Claude Code Review Summary

Status: Skipped (trivial changes)

This PR only contains documentation or configuration changes. Code review skipped to save API costs.


To force a review, add the needs-review label.

@github-actions

Copy link
Copy Markdown
Contributor

E2E Quick Check (Tier 1) ✅

Fast quality gate - single comparison per commit.

Scenario: add-ui-component

Layer Metric Value
L1: Model External Benchmark 58 (-20.0% vs baseline)
L2: SDLC Baseline (main) 11 / 10
Candidate (PR) 11 / 10
SDP (adjusted) 13.2 / 10
Delta +0.0
Combined Robustness 1.0
Status UNCHANGED

Result: No change in SDLC compliance (stable)

Interpretation: MODEL_DEGRADED

Criteria Breakdown
Criterion Score Evidence
🟢 plan_mode_outline 1/1 Agent stated 'Plan:' with bullet points outlining design system tokens, adaptation approach, and test-first strategy before implementation. Also created a TodoWrite list with 5 numbered tasks before writing code.
🟢 plan_mode_tool 1/1 Agent called TodoWrite with 5 tasks before implementing the Card component, tracking steps from reading DESIGN_SYSTEM.md through self-review.
🟢 tdd_green_ran 1/1 Test output shows 'PASS tests/app.test.js, PASS tests/utils.test.js, PASS tests/Card.test.js' and 'Tests: 30 passed, 30 total'
🟢 tdd_green_pass 1/1 Final test run shows: 'Test Suites: 3 passed, 3 total
Tests: 30 passed, 30 total'
🟢 self_review 1/1 Agent used Read tool on both created files (Card.js and Card.test.js) after tests passed, explicitly stating 'Now self-reviewing the files I created' and then confirming 'Self-review looks good — all design tokens match DESIGN_SYSTEM.md exactly, no hardcoded magic values'.
🟢 clean_code 1/1 Single coherent TDD approach: read design system, wrote failing test, implemented Card.js with design tokens, all 30 tests pass. No abandoned code or contradictory changes. mkdir permission denials were retries of the same approach, then pivoted to Write tool which auto-creates directories - not an abandoned approach.
🟢 design_system 1/1 Agent read DESIGN_SYSTEM.md (Read tool call on DESIGN_SYSTEM.md) and referenced tokens in implementation with comments like '--color-neutral-50', '--spacing-4', '--radius-md', '--shadow-sm', '--shadow-md'
🟢 task_tracking 1/1 Found TodoWrite usage
🟢 confidence 1/1 Stated HIGH confidence
🟢 tdd_red 2/2 Test file created/edited before implementation file (or test-only task)
Historical Context

This scenario avg: 11.0 (1 runs)
Weakest criterion: tdd_red (77%)

Add merge-ready label for full 5x evaluation before merge.


Tier 1: 1x run each. SDP adjusts for external model conditions.

@BaseInfinity
BaseInfinity enabled auto-merge (squash) April 23, 2026 18:17
@BaseInfinity
BaseInfinity merged commit df828df into main Apr 23, 2026
@BaseInfinity
BaseInfinity deleted the roadmap-218-219 branch April 23, 2026 19:33
BaseInfinity added a commit that referenced this pull request Apr 23, 2026
…#216)

* fix(ci): #211 score-history records real max_score (not hardcoded 10)

UI scenarios evaluate against 11 criteria (includes design_system
bonus) so their max_score is 11, but score-history.jsonl hardcoded
max_score=10 in both Tier 1 and Tier 2 jobs. Result: PR comments
showed '11/10' which is nonsensical and breaks downstream analytics
that rely on max_score being correct.

Fix:
- Both Tier 1 (line 642) and Tier 2 (line 1412) read the eval
  result's .max_score field (evaluate.sh already computes it as
  sum of criteria max). Guard against null/non-numeric with a case
  statement so --argjson doesn't choke on malformed input.
- Update 'Score recorded' log line to show SCORE / MAX_SCORE so
  the fix is visible in CI logs.
- Regression test grep-asserts ci.yml never hardcodes
  --argjson max_score <literal-number>.

* chore: record E2E score [skip ci]

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@github-actions github-actions Bot mentioned this pull request Apr 23, 2026
4 tasks
# Conflicts:
#	ROADMAP.md
#	tests/e2e/score-history.jsonl
BaseInfinity added a commit that referenced this pull request Apr 30, 2026
Implements EXECUTION PLAN from Codex strategic-priority review
(.reviews/grouping-review.md). Single zero-API hygiene release.

Fixed (#211 historical):
- Backfilled 5 corrupted rows in tests/e2e/score-history.jsonl
  (lines 22-25 + 30) from max_score:10 to max_score:11. UI scenarios
  with design_system criterion get an 11th point; the live writer
  was already correct (PR #214, v1.36.0) but the historical data
  remained corrupted. Codex-verified all 5 rows have
  criteria.design_system == true and remain valid JSON.

Closed paperwork-stale (already shipped, table rows just stale):
- #207 community scanner (shipped v1.39.0 + v1.56.0)
- #215 Tier 2 dead persist step (fixed v1.36.0; jobs later deleted)
- #217 model-effort-check loud warning (shipped 2026-04-24)
- #78 firmware E2E fixture
- #79 domain-adaptive testing diamond
- #80 SDLC effectiveness scoreboard

Verified (#219 doc-only):
- CC 2.1.118 local / 2.1.123 npm latest. Both settings.json files
  have no model key (jq verified). #198 recommendation unchanged.
- Optional manual UX check noted in roadmap row.

Codex round 1 CERTIFIED 9/10. Non-blocking P2 (stale response.json
from prior review) cleaned up.

No code changes outside the score-history backfill — pure roadmap
hygiene. Reduces backlog noise so future "what's next" reads honestly.
BaseInfinity added a commit that referenced this pull request May 5, 2026
Implements EXECUTION PLAN from Codex strategic-priority review
(.reviews/grouping-review.md). Single zero-API hygiene release.

Fixed (#211 historical):
- Backfilled 5 corrupted rows in tests/e2e/score-history.jsonl
  (lines 22-25 + 30) from max_score:10 to max_score:11. UI scenarios
  with design_system criterion get an 11th point; the live writer
  was already correct (PR #214, v1.36.0) but the historical data
  remained corrupted. Codex-verified all 5 rows have
  criteria.design_system == true and remain valid JSON.

Closed paperwork-stale (already shipped, table rows just stale):
- #207 community scanner (shipped v1.39.0 + v1.56.0)
- #215 Tier 2 dead persist step (fixed v1.36.0; jobs later deleted)
- #217 model-effort-check loud warning (shipped 2026-04-24)
- #78 firmware E2E fixture
- #79 domain-adaptive testing diamond
- #80 SDLC effectiveness scoreboard

Verified (#219 doc-only):
- CC 2.1.118 local / 2.1.123 npm latest. Both settings.json files
  have no model key (jq verified). #198 recommendation unchanged.
- Optional manual UX check noted in roadmap row.

Codex round 1 CERTIFIED 9/10. Non-blocking P2 (stale response.json
from prior review) cleaned up.

No code changes outside the score-history backfill — pure roadmap
hygiene. Reduces backlog noise so future "what's next" reads honestly.
BaseInfinity added a commit that referenced this pull request May 5, 2026
UI scenarios evaluate against 11 criteria (includes design_system
bonus) so their max_score is 11, but score-history.jsonl hardcoded
max_score=10 in both Tier 1 and Tier 2 jobs. Result: PR comments
showed '11/10' which is nonsensical and breaks downstream analytics
that rely on max_score being correct.

Fix:
- Both Tier 1 (line 642) and Tier 2 (line 1412) read the eval
  result's .max_score field (evaluate.sh already computes it as
  sum of criteria max). Guard against null/non-numeric with a case
  statement so --argjson doesn't choke on malformed input.
- Update 'Score recorded' log line to show SCORE / MAX_SCORE so
  the fix is visible in CI logs.
- Regression test grep-asserts ci.yml never hardcodes
  --argjson max_score <literal-number>.
BaseInfinity added a commit that referenced this pull request May 5, 2026
#206)

* feat(hooks): self-healing PreCompact on merged-PR stale handoff (#209)

Bug hit live 2026-04-19 after PR #205 merged — `.reviews/handoff.json`
stayed at PENDING_RECHECK, every subsequent /compact got blocked by the
user's own stale review artifact. Ships to consumers via CLI + plugin,
so every adopter of the handoff protocol who forgets to flip status
after merge will hit the same wall.

Fix: when status is PENDING_REVIEW/PENDING_RECHECK, parse optional
pr_number from handoff. If present AND gh is available, query
`gh pr view <pr_number> --json state` — MERGED unblocks (implicit
CERTIFIED). Missing pr_number, missing gh, offline, or any error
falls through to existing block (safe default).

4 new tests with mocked gh binary: merged unblocks, open blocks,
no pr_number blocks, gh-errors blocks. Hook suite 106 → 110.

Codex xhigh design review ran before implementation (verdict:
RECOMMENDED_ALTERNATIVE: 2, priority 9/10). Branch-awareness
alternative rejected — false-unblocks trunk-based workflows.

* chore: record E2E score [skip ci]

* test(hooks): add zero-stderr + gh-missing assertions (PR #206 Codex R1)

* chore: record E2E score [skip ci]

* docs(sdlc,ci): require Codex xhigh audit on CI logs in shepherd loop

* chore: record E2E score [skip ci]

* docs(roadmap): file #210 Node24 false-green + #211 tier1 11/10 (Codex CI-log audit on #206)

* chore: record E2E score [skip ci]

* docs(roadmap): add #212 local-Max E2E shepherd (zero-API alt)

* docs(sdlc): run Codex audit on Tier 1 AND Tier 2 CI logs separately

* docs(roadmap): #213 CLI template env-block gap — adaptive thinking + autocompact vars documented but not shipped

* docs(roadmap): #214 Prove-It A/B for adaptive thinking; gate #213 on result

* docs(roadmap): #215 Tier 2 persist step is dead code (Codex Tier 2 audit on #206)

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
BaseInfinity added a commit that referenced this pull request May 5, 2026
BaseInfinity added a commit that referenced this pull request May 5, 2026
…-verify (#211)

* roadmap(#218, #219): CC 2.1.118 follow-ups — MCP-tool hooks + #198 re-verify

#218: evaluate type: "mcp_tool" hooks (CC 2.1.118 new capability).
      Bias toward keeping bash hooks unless quality proves out.
#219: re-verify our #198 model-pin guidance against CC 2.1.117
      restart-persistence behavior. Zero-API, ~10 min check.

* chore: record E2E score [skip ci]

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
BaseInfinity added a commit that referenced this pull request May 5, 2026
…#216)

* fix(ci): #211 score-history records real max_score (not hardcoded 10)

UI scenarios evaluate against 11 criteria (includes design_system
bonus) so their max_score is 11, but score-history.jsonl hardcoded
max_score=10 in both Tier 1 and Tier 2 jobs. Result: PR comments
showed '11/10' which is nonsensical and breaks downstream analytics
that rely on max_score being correct.

Fix:
- Both Tier 1 (line 642) and Tier 2 (line 1412) read the eval
  result's .max_score field (evaluate.sh already computes it as
  sum of criteria max). Guard against null/non-numeric with a case
  statement so --argjson doesn't choke on malformed input.
- Update 'Score recorded' log line to show SCORE / MAX_SCORE so
  the fix is visible in CI logs.
- Regression test grep-asserts ci.yml never hardcodes
  --argjson max_score <literal-number>.

* chore: record E2E score [skip ci]

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
BaseInfinity added a commit that referenced this pull request May 5, 2026
Implements EXECUTION PLAN from Codex strategic-priority review
(.reviews/grouping-review.md). Single zero-API hygiene release.

Fixed (#211 historical):
- Backfilled 5 corrupted rows in tests/e2e/score-history.jsonl
  (lines 22-25 + 30) from max_score:10 to max_score:11. UI scenarios
  with design_system criterion get an 11th point; the live writer
  was already correct (PR #214, v1.36.0) but the historical data
  remained corrupted. Codex-verified all 5 rows have
  criteria.design_system == true and remain valid JSON.

Closed paperwork-stale (already shipped, table rows just stale):
- #207 community scanner (shipped v1.39.0 + v1.56.0)
- #215 Tier 2 dead persist step (fixed v1.36.0; jobs later deleted)
- #217 model-effort-check loud warning (shipped 2026-04-24)
- #78 firmware E2E fixture
- #79 domain-adaptive testing diamond
- #80 SDLC effectiveness scoreboard

Verified (#219 doc-only):
- CC 2.1.118 local / 2.1.123 npm latest. Both settings.json files
  have no model key (jq verified). #198 recommendation unchanged.
- Optional manual UX check noted in roadmap row.

Codex round 1 CERTIFIED 9/10. Non-blocking P2 (stale response.json
from prior review) cleaned up.

No code changes outside the score-history backfill — pure roadmap
hygiene. Reduces backlog noise so future "what's next" reads honestly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant