feat(setup): CI shepherd opt-in + workflow analyzer (#48)

[BaseInfinity]

Summary

• Added CI shepherd as explicit top-level opt-in question (Q18) in setup wizard, gating CI monitoring/review/bot detail questions behind it (Codex review finding feat: add auto-update system for Claude Code releases #1)
• Created /ci-analyzer skill for GitHub Actions workflow gap analysis — covers linting gaps, review hooks, E2E suggestions (roadmap scope, Codex finding test: validate trivial detection and needs-review bypass #4)
• Added ci-analyzer to CLI distribution (8→9 template files), updated wizard Step 0.3 comparison table and Complementary Tools section
• Shepherd choice persists in SDLC.md metadata <!-- CI Shepherd: enabled/disabled/not applicable --> (Codex finding test: validate PR workflows after PR #1 merge #2)
• Added test: self-heal live validation (intentional CI failure) #52 Smart Setup Questions and [LOW] Auto-Update: Claude Code v2.1.44 #53 Plan Auto-Approval Gate to roadmap (user feedback)

Test plan

• 8 new tests in test-self-update.sh (shepherd opt-in, question gating, Q18 count, skill existence, template parity, roadmap categories, wizard reference)
• File count bumps in test-cli.sh (8→9) and test-setup-path.sh (8→9, all 9 fixtures)
• Full test suite: 421 passed, 0 failed
• Cross-model review (Codex xhigh): 4 findings addressed (3 P1 ACCEPTED, 1 P2 ACCEPTED)
• CI validates on PR

[github-actions]

E2E Quick Check (Tier 1) ⚠️

Fast quality gate - single comparison per commit.

Scenario: hard-refactor

Layer	Metric	Value
L1: Model	External Benchmark	59 (-20.0% vs baseline)
L2: SDLC	Baseline (main)	8 / 10
	Candidate (PR)	7 / 10
	SDP (adjusted)	8.4 / 10
	Delta	-1.0
Combined	Robustness	1.0
	Status	MINOR_DIP

Result: Minor variance detected (within acceptable range)

Interpretation: MODEL_DEGRADED

Criteria Breakdown

	Criterion	Score	Evidence
🟢	plan_mode_outline	1/1	The agent outlined a numbered plan before writing code: 'Confidence: MEDIUM

Plan:

1. Update tests/app.test.js to import and test Calculator class (TDD RED)
2. Run tests — should fail (no Calculator class yet)
3. Add Calculator class to src/app.js with greet, add, calculate (fixing typo)
4. Run tests — should pass
5. Self-review' |
   | 🟢 | plan_mode_tool | 1/1 | Agent called TodoWrite with 5 tasks before any Edit tool calls. The first TodoWrite call created tasks: 'Update tests for Calculator class (TDD RED)', 'Run tests to verify they fail', 'Implement Calculator class in src/app.js', 'Run tests to verify they pass', 'Self-review changes'. The agent then continued to update the todo list throughout execution to track progress. |
   | 🟢 | tdd_green_ran | 1/1 | Two test runs shown: RED phase: 'Test Suites: 1 failed, 1 passed, 2 total
   Tests: 3 failed, 21 passed, 24 total'; GREEN phase: 'PASS tests/app.test.js
   PASS tests/utils.test.js
   Test Suites: 2 passed, 2 total
   Tests: 24 passed, 24 total' |
   | 🟢 | tdd_green_pass | 1/1 | The final test run shows: 'Test Suites: 2 passed, 2 total
   Tests: 24 passed, 24 total' — all tests pass in the GREEN phase run. |
   | 🔴 | self_review | 0/1 | The agent stated 'Self-review:' and listed bullet points summarizing what it did, but never actually used Read, Bash (git diff), or any other tool to read back the modified files. It did not inspect the final state of app.js or app.test.js after modifications. The self-review was purely from memory without re-reading any files. |
   | 🟢 | clean_code | 1/1 | The agent followed a single coherent approach: (1) read current files, (2) stated plan and confidence, (3) created todo list, (4) updated tests for Calculator class (TDD RED), (5) ran tests confirming failure, (6) implemented Calculator class in app.js, (7) ran tests confirming all pass (GREEN), (8) self-reviewed. No abandoned implementations, no commented-out code, no contradictory changes. The legacy functions were intentionally retained for backward compatibility, not as dead code from an abandoned approach. |
   | 🟢 | task_tracking | 1/1 | Found TodoWrite usage |
   | 🟢 | confidence | 1/1 | Stated MEDIUM confidence |
   | 🔴 | tdd_red | 0/2 | Not found |

Historical Context

This scenario avg: 7.0 (1 runs)
Weakest criterion: tdd_red (0%)

Add merge-ready label for full 5x evaluation before merge.

---

Tier 1: 1x run each. SDP adjusts for external model conditions.

[github-actions]

Claude finished @BaseInfinity's task in 1m 42s —— View job

---

PR Code Review

• Read CLAUDE.md and CODE_REVIEW_EXCEPTIONS.md for repo guidelines
• Fetch PR diff and review all changed files
• Check for security issues, SDLC compliance, and test coverage
• Check E2E scenario coverage for affected areas
• Write and post final review

Summary

Adds CI shepherd as an explicit opt-in question (Q18) gating CI sub-questions, creates a new /ci-analyzer skill for GitHub Actions workflow gap analysis, updates the CLI distribution (8→9 template files), and adds two roadmap items (#52, #53).

Recommendation: APPROVE

Findings

P0 — Critical (must fix before merge)

None.

P1 — Must Fix (should fix before merge)

None.

P2 — Suggestions (nice to have)

• tests/test-self-update.sh:876-887 — test_setup_skill_template_parity_questions uses grep -c "Q[0-9]" to count question references, but since it's a substring match, "Q13" matches the Q[0-9] pattern via "Q1". The test name says "question count" but it's really testing parity between live and template files (which is already covered by the general parity tests). Consider either renaming to clarify intent or using a stricter pattern like \bQ[0-9]\+\b if actually counting distinct questions. Fix this →

• .claude/skills/setup/SKILL.md:1 — The description field previously said "asks 16 config questions" while the body said "Ask ALL 17 Questions" — this PR jumps both to 18, which is correct but worth noting: the old description was already out of sync (description said 16, body said 17). The PR silently fixes this pre-existing drift. Might be worth a note in the PR description for traceability.

Strengths

• Clean separation of concerns — The /ci-analyzer skill is well-scoped with explicit boundaries: only GitHub Actions (v1), no overlap with /claude-automation-recommender, and clear fallback messaging for non-GitHub CI backends.
• Proper gating pattern — The CI shepherd opt-in (Q18) cleanly gates sub-questions behind the top-level answer. The wizard doc restructures the flow so "CI monitoring detail" and "review feedback level" only appear after shepherd is enabled. This addresses the Codex review finding well.
• Metadata persistence — `` in SDLC.md is a clean pattern for persisting setup choices without polluting user-visible content.
• Template parity enforced — Both .claude/skills/ci-analyzer/SKILL.md and cli/templates/skills/ci-analyzer/SKILL.md are identical, and the test suite (test_ci_analyzer_template_parity) validates this with diff -q. Same for the setup SKILL.md.
• Comprehensive test coverage — 8 new focused tests covering: opt-in question existence, question gating order, question count, template parity (×2), roadmap category coverage, and wizard reference. File count bumps in test-cli.sh and test-setup-path.sh are consistent.
• Good comparison table update — The Step 0.3 table now clearly shows the three-tool separation (Wizard / Recommender / CI Analyzer) with per-category coverage markers.
• Roadmap items (test: self-heal live validation (intentional CI failure) #52, [LOW] Auto-Update: Claude Code v2.1.44 #53) — Well-described with clear context and trigger conditions for each.

E2E Coverage Note

This PR affects .claude/skills/setup/SKILL.md (SDLC guidance) and CLAUDE_CODE_SDLC_WIZARD.md (wizard behavior). No E2E scenarios exist specifically for the setup wizard flow, but this is expected — the wizard is an interactive skill that's difficult to test end-to-end. The structural bash tests in test-self-update.sh provide appropriate coverage for the changes made here.

Files Reviewed

• .claude/skills/ci-analyzer/SKILL.md
• .claude/skills/setup/SKILL.md
• CLAUDE_CODE_SDLC_WIZARD.md
• ROADMAP.md
• cli/init.js
• cli/templates/skills/ci-analyzer/SKILL.md
• cli/templates/skills/setup/SKILL.md
• tests/test-cli.sh
• tests/test-self-update.sh
• tests/test-setup-path.sh

---

[github-actions]

The review is posted. Here's a summary of what was done:

• Read CLAUDE.md and CODE_REVIEW_EXCEPTIONS.md for repo-specific guidelines
• Analyzed the full PR diff across all 10 changed files
• Checked for security issues (none found), SDLC compliance (follows patterns well), and test coverage (8 new tests + file count bumps)
• Verified E2E coverage implications — changes affect skills and wizard doc but structural bash tests provide appropriate coverage
• Posted the review recommending APPROVE with two minor P2 suggestions (test naming clarity and a note about a pre-existing question count discrepancy that was silently fixed)