Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"name": "sdlc-wizard",
"source": ".",
"description": "SDLC enforcement for AI agents — TDD, planning, self-review, CI shepherd",
"version": "1.65.0",
"version": "1.66.0",
"author": {
"name": "Stefan Ayala"
},
Expand Down
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sdlc-wizard",
"version": "1.65.0",
"version": "1.66.0",
"description": "SDLC enforcement for AI agents — TDD, planning, self-review, CI shepherd",
"author": {
"name": "Stefan Ayala",
Expand Down
69 changes: 69 additions & 0 deletions .reviews/research-95-nous.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# ROADMAP #95 Research: Nous Research vs SDLC Wizard

**Verdict: NO-GO. Different layer of the stack.**

**Date:** 2026-05-04
**Source:** [nousresearch.com](https://nousresearch.com), [github.com/NousResearch](https://github.com/NousResearch)

## Original question

> Evaluate `nousresearch.com` — what are they building, how does it compare to SDLC wizard's approach? Are they doing agent enforcement, testing methodology, or something orthogonal? Competitive analysis.

## What Nous Research actually builds

From their own copy: *"We train world-class open source language models and build infrastructure to coordinate distributed, unbiased training."*

Product surface:

| Product | What it is |
|---------|-----------|
| **Hermes** (Hermes 4 et al.) | Open-weights base / instruct LLMs |
| **Hermes Agent** | Generic agent framework (autonomous task execution) |
| **hermes-agent-self-evolution** | DSPy + GEPA pipeline that evolves the agent's own skills / prompts / code |
| **atropos** | RL environments for collecting + evaluating LLM trajectories |
| **Hermes-Function-Calling** | Function-calling tooling for the Hermes models |
| **Psyche** | Distributed-training network infrastructure |
| **Nous Chat / API Portal / Simulators** | End-user + developer access surfaces |

## Layer-comparison

| Layer | Nous Research owns | SDLC Wizard owns |
|-------|--------------------|------------------|
| Pre-training / fine-tuning | ✓ (Hermes models, Psyche) | — |
| RL eval environments | ✓ (atropos) | — |
| Agent framework (model executes tasks) | ✓ (Hermes Agent) | — |
| Agent self-evolution (model rewrites its own prompts/skills) | ✓ (hermes-agent-self-evolution) | — |
| **SDLC process enforcement on a human-driven coding agent** | — | ✓ |
| TDD red-before-green hook | — | ✓ |
| Planning gate / confidence stating | — | ✓ |
| Cross-model adversarial review | — | ✓ |
| CI shepherd loop | — | ✓ |

**They build the engines. We enforce the build pipeline that uses an engine.** No surface overlap.

## "But what about Hermes Agent — isn't that an agent like Claude Code?"

Hermes Agent is an autonomous agent framework — the model loop itself, plus function-calling, plus a self-evolution layer. That's the same layer Claude Code, Codex CLI, and OpenCode occupy. The wizard doesn't compete with any of those — the wizard runs **inside** an agent host and enforces SDLC discipline on what the host does.

If anything, Hermes Agent is a *target* for the wizard, not a competitor: the OpenCode port (ROADMAP #9) is exactly the play to make the wizard agent-host-agnostic, which would let it run on any OpenAI-compat backend including Hermes via Nous's API portal. That work is already tracked.

`hermes-agent-self-evolution` is interesting (model-driven self-improvement of skills/prompts/code) but it operates at the agent's *own* layer — the agent rewrites itself. Our self-evolution loop operates at the *process* layer — the harness rewrites the wizard. Conceptually parallel, mechanically separate.

## Pattern repeat — what this looks like next to prior NO-GOs

This is the fourth external-methodology audit to land NO-GO:

| # | Audit | Verdict | Reason |
|---|-------|---------|--------|
| #76 | Promptfoo | NO-GO | Already implements Promptfoo's best patterns; no statistical layer |
| #77 | Constrain-to-Playbook | NO-GO | Already constrained on the prompts where it helps; PR review is open-ended on purpose |
| #235 | Thoughtworks AI Evals | NO-GO | Methodology-only article; every layer already has an analog in our pipeline |
| #95 | Nous Research | NO-GO (this) | Different layer of the stack — model R&D vs process enforcement |

Pattern: external audits keep validating that the wizard is mature in its niche. Revisit only when an external source surfaces a *specific technique* we don't already have, not when it surfaces a different *product*.

## Action

- Mark ROADMAP #95 as DONE pointing at this research doc.
- No code changes.
- Keep an eye on the OpenCode port (#9) — that's the right vehicle if anyone ever asks "can the wizard run against Hermes / a self-hosted Nous endpoint?" The answer becomes "yes, via OpenCode."
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,28 @@ All notable changes to the SDLC Wizard.

> **Note:** This changelog is for humans to read. Don't manually apply these changes - just run the wizard ("Check for SDLC wizard updates") and it handles everything automatically.

## [1.66.0] - 2026-05-04

### Closed (paperwork-stale roadmap rows)

- **ROADMAP #95 — Nous Research competitive audit** ✅ DONE 2026-05-04 with NO-GO verdict. Research write-up at `.reviews/research-95-nous.md`. Nous Research builds open-weights LLMs (Hermes), agent frameworks (Hermes Agent), RL environments (atropos), and distributed-training infra (Psyche) — different layer of the stack from SDLC enforcement. Hermes Agent is the same layer as Claude Code / Codex CLI / OpenCode (a *target* for the wizard, not a competitor); the OpenCode port (#9) is the right vehicle if anyone ever wants the wizard to run against a self-hosted Nous endpoint. Pattern continues with #76 (Promptfoo NO-GO), #77 (constrain-to-playbook NO-GO), #235 (Thoughtworks AI Evals NO-GO): external-product audits keep validating the wizard's niche.

- **`docs/codex-near-top` cross-reference** ✅ DONE 2026-05-04 in PR #309. Surfaced `codex-sdlc-wizard` sibling at the top of README.md (after the tagline, before Install) and `CLAUDE_CODE_SDLC_WIZARD.md` (after the "What This Is" intro), so users on OpenAI's Codex CLI find the alternative without scrolling 250+/500+ lines to the Ecosystem section. Two new doc-consistency tests (head -30 / head -50 grep) keep the callout from drifting out of the top fold. OpenCode sibling intentionally not mentioned yet (per maintainer — bootstrap shipping in a different session).

- **GitHub issue #308 (API features review)** ✅ CLOSED 2026-05-04. 4/4 entries audited (`gh issue view 308#issuecomment-4375055751`): zero wizard changes needed. Sonnet 1M-beta retirement (2026-04-30) only affects Sonnet 4.5 and Sonnet 4 — wizard's `sonnet[1m]` mixed-mode tier resolves to Sonnet 4.6 which keeps 1M GA per [API release notes](https://platform.claude.com/docs/en/release-notes/api.md). Rate Limits API + Memory for Managed Agents are different products. Haiku 3 retirement is irrelevant — `grep -ri "haiku.3\|claude-3-haiku"` returned zero hits across the wizard.

### Files

- `.reviews/research-95-nous.md` (new — research write-up, force-added past `.reviews/` gitignore matching #206 + #235 precedent)
- `ROADMAP.md` (#95 marked DONE with verdict reference)
- `README.md` + `CLAUDE_CODE_SDLC_WIZARD.md` (codex sibling callout near top, shipped in PR #309)
- `tests/test-doc-consistency.sh` (2 new top-of-doc grep tests for codex callout, shipped in PR #309)
- `CHANGELOG.md`, `SDLC.md`, `skills/update/SKILL.md`, `package.json`, `.claude-plugin/plugin.json` + `marketplace.json`, `CLAUDE_CODE_SDLC_WIZARD.md` (1.65.0 → 1.66.0)

### Notes

Zero code changes (research + cross-reference docs only). Backlog after this release: 1 open issue (#302 user-level setup skill — design-blocked) + ROADMAP top items #212 (multi-day), #9 OpenCode (separate session per maintainer).

## [1.65.0] - 2026-05-04

### Closed (paperwork-stale roadmap rows)
Expand Down
4 changes: 2 additions & 2 deletions CLAUDE_CODE_SDLC_WIZARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -2976,7 +2976,7 @@ If deployment fails or post-deploy verification catches issues:

**SDLC.md:**
```markdown
<!-- SDLC Wizard Version: 1.65.0 -->
<!-- SDLC Wizard Version: 1.66.0 -->
<!-- Setup Date: [DATE] -->
<!-- Completed Steps: step-0.1, step-0.2, step-0.4, step-1, step-2, step-3, step-4, step-5, step-6, step-7, step-8, step-9 -->
<!-- Git Workflow: [PRs or Solo] -->
Expand Down Expand Up @@ -4055,7 +4055,7 @@ Walk through updates? (y/n)
Store wizard state in `SDLC.md` as metadata comments (invisible to readers, parseable by Claude):

```markdown
<!-- SDLC Wizard Version: 1.65.0 -->
<!-- SDLC Wizard Version: 1.66.0 -->
<!-- Setup Date: 2026-01-24 -->
<!-- Completed Steps: step-0.1, step-0.2, step-1, step-2, step-3, step-4, step-5, step-6, step-7, step-8, step-9 -->
<!-- Git Workflow: PRs -->
Expand Down
2 changes: 1 addition & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ Living tracker of projects shipped using this wizard. **Rule:** only list projec
| 92 | Research: Rigorous Autocompact Benchmarking | Nobody has published controlled benchmarks testing different autocompact thresholds. Opportunity to be THE authority. Test: quality metrics (task completion accuracy, code correctness) at 50/60/70/75/80/83% thresholds across 200K and 1M models. Measure: pre/post compaction context preservation, cost per session, degradation curves. Low priority but high differentiation potential |
| 93 | ~~Node.js 20 EOL~~ DONE | Workflows already migrated to Node 24-native action versions: `actions/checkout@v5`, `actions/setup-node@v5`, `actions/upload-artifact@v6`. No `oven-sh/setup-bun` usage in repo. Verified by `tests/test-node24-compliance.sh` (13 tests, all green). Shipped pre-deadline (deadline was June 2, 2026) |
| 183 | Advisor Tool A/B — Light Tier Validation (supersedes #94 research) | **RESEARCH COMPLETE.** aistupidlevel.info independent benchmarks (7-axis, CUSUM, 95% CI) show: Opus 4.6=47/VOLA, Sonnet 4.6=37/DEGR, Codex GPT-5.3=27/DEGR. Degradation confirmed by Anthropic staff (Boris Cherny, GH #42796, 274 comments). Root causes: adaptive thinking under-allocation, medium effort default, "simplest approach" system prompt (v2.1.64). Gap is multi-turn/planning specific — SWE-bench (single-shot) looks fine. **Tiered model strategy decided:** Critical (Opus executor + Codex reviewer), Standard (Opus + optional Codex), Light (Sonnet executor + Opus advisor — advisor compensates for weaker executor), Vibe (Sonnet only). Advisor tool (`advisor-tool-2026-03-01`) is API-only beta, not in CC CLI yet. Opus-advising-Opus is waste — advisor only valuable when executor is weaker. **Next:** build A/B workflow to validate Light tier (Sonnet+advisor vs Opus) on our E2E, integrate aistupidlevel monitoring. Blog: `claude.com/blog/the-advisor-strategy` |
| 95 | Research: Nous Research Comparison | Evaluate `nousresearch.com` — what are they building, how does it compare to SDLC wizard's approach? Are they doing agent enforcement, testing methodology, or something orthogonal? Competitive analysis |
| 95 | ~~Research: Nous Research Comparison~~ ✅ DONE 2026-05-04 — see [`.reviews/research-95-nous.md`](.reviews/research-95-nous.md) | **Verdict: NO-GO.** Different layer of the stack — Nous Research builds open-weights LLMs (Hermes), agent frameworks (Hermes Agent), RL environments (atropos), and distributed-training infra (Psyche). No surface overlap with SDLC enforcement: they build the engines, we enforce the pipeline that uses an engine. Hermes Agent is the same layer as Claude Code / Codex CLI / OpenCode — a *target* for the wizard, not a competitor. The OpenCode port (#9) is the right vehicle if anyone ever wants the wizard to run against a self-hosted Nous endpoint. Pattern continues with #76 (Promptfoo NO-GO), #77 (constrain-to-playbook NO-GO), #235 (Thoughtworks AI Evals NO-GO): external-product audits keep validating our niche; revisit only when an external source surfaces a *specific technique* we don't have, not when it surfaces a different *product*. **Original question:** evaluate `nousresearch.com` — what are they building, how does it compare to SDLC wizard's approach? |
| 96 | ~~Audit: CC Degradation Detection~~ ✅ ALL PHASES DONE | Degradation infra shipped earlier. **Phase 1 DONE v1.57.0 (PR #290):** de-coached benchmark prompt. **Phase 2 DONE v1.58.0 (PR #293):** ground-truth gate. **Phase 3 PR 1 DONE v1.60.0:** wizard-installation lift-proof harness. **Phase 3 PR 2 DONE v1.61.0:** calibration scenario suite — `tests/e2e/scenarios/calibration-careful-read.md` is the first in a `calibration-*` family designed to reward self-review (parsePrice with 5 edge-case formats; rushed agent silently corrupts `'$1,000.00'` to `1`, a thousand-fold pricing bug). 6 new tests in `tests/test-calibration-scenarios.sh` (scenario format validator). End-to-end calibration verification (does low-effort agent actually score lower?) deferred to ROADMAP #212(i) Prove-It Gate paired runs. **The #96 series is now complete:** prompt de-coached → tests required → wizard contribution measurable → calibration signals embedded in scenario suite. |
| 97 | Research: Anthropic Policy & Research Alignment | Audit Anthropic's policy/research pages for SDLC relevance: (a) Responsible Scaling Policy (`anthropic.com/responsible-scaling-policy`) — does our enforcement model align with their safety levels? (b) Transparency reports (`anthropic.com/transparency`) — can we reference/align? (c) Constitution (`anthropic.com/constitution`) — SDLC as constitutional AI for dev process? (d) Economic Futures (`anthropic.com/economic-futures`) — positioning for AI-assisted development economy (e) Research page (`anthropic.com/research`) — any papers relevant to agent enforcement, tool use safety, or evaluation methodology. Goal: align wizard's philosophy with Anthropic's public positions where genuine overlap exists |
| 98 | ~~Audit: Community Feedback & Contribution Paths~~ DONE | PR #191. Added `.github/ISSUE_TEMPLATE/` (bug_report, feature_request, question) with GitHub-valid frontmatter + `config.yml` (blank issues disabled, contact links to Discussions + in-session `/feedback`). Added `.github/PULL_REQUEST_TEMPLATE.md` with Prove-It Gate checklist + test plan section. Enabled GitHub Discussions via `gh api ... -f has_discussions=true`. Added README "Feedback" section with live links to all three paths. `tests/test-community-paths.sh` — 14 tests, mutation-verified (deleting config.yml or stripping frontmatter `about:` trips CI loudly). Codex xhigh 2-round code review: 5/10 → 10/10 CERTIFIED |
Expand Down
6 changes: 3 additions & 3 deletions SDLC.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<!-- SDLC Wizard Version: 1.65.0 -->
<!-- SDLC Wizard Version: 1.66.0 -->
<!-- Setup Date: 2026-01-24 -->
<!-- Completed Steps: step-0.1, step-0.2, step-1, step-2, step-3, step-4, step-5, step-6, step-7, step-8, step-9 -->
# SDLC Configuration
Expand All @@ -7,8 +7,8 @@

| Property | Value |
|----------|-------|
| Wizard Version | 1.65.0 |
| Last Updated | 2026-04-30 |
| Wizard Version | 1.66.0 |
| Last Updated | 2026-05-04 |
| Claude Code Baseline | v2.1.111+ (required for Opus 4.7 / `opus[1m]`) |
| Recommended Model | `opus[1m]` (Opus 4.7, 1M context) — run `/model opus[1m]` |
| Recommended Effort | `max` (preferred) / `xhigh` (floor) — run `/effort max` |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "agentic-sdlc-wizard",
"version": "1.65.0",
"version": "1.66.0",
"description": "SDLC enforcement for Claude Code — hooks, skills, and wizard setup in one command",
"bin": {
"sdlc-wizard": "cli/bin/sdlc-wizard.js"
Expand Down
13 changes: 3 additions & 10 deletions skills/update/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,10 @@ Parse CHANGELOG entries between the user's installed version and latest. Present

```
Installed: 1.42.0
Latest: 1.65.0
Latest: 1.66.0

What changed:
- [1.66.0] roadmap hygiene — closed #95 (Nous Research competitive audit — NO-GO; different layer of the stack, Nous builds open-weights LLMs + agent frameworks, we enforce SDLC process; pattern continues with #76 + #77 + #235 NO-GOs). Research write-up at `.reviews/research-95-nous.md`. Also rolls in PR #309 (codex sibling callout near top of README + wizard doc) and `gh issue close 308` (4/4 API entries audited, zero wizard changes — Sonnet 1M retirement only affects 4.5/4, our mixed-mode tier pins to 4.6).
- [1.65.0] roadmap hygiene — closed paperwork-stale rows #210 (Node 24 false-green test, already shipped in PR #217) and #235 (Thoughtworks AI Evals research — NO-GO verdict, methodology already implemented under different naming; pattern continues with #76 + #77 NO-GOs). Research write-up at `.reviews/research-235-ai-evals.md`. No code changes.
- [1.64.0] XDLC ecosystem cross-references — README, wizard doc, and ROADMAP now cross-reference all three sibling packages (`agentic-sdlc-wizard`, `codex-sdlc-wizard`, `claude-gdlc-wizard`). New "Ecosystem (Sibling Projects)" section in README. 3 new doc-consistency tests prevent drift.
- [1.63.0] cache-cost observability closeout (#204 absorbed by #220) — `tests/test-token-spike.sh` gains explicit cache-miss regression test + negative-control test. SDLC skill + wizard doc gain "Cache-Cost Surprises" sections covering 10-20× silent cost blowups (mid-session CLAUDE.md edits, idle pruning, upstream cache bugs) and detection via `hooks/token-spike-check.sh`'s `costly_tokens` metric.
Expand All @@ -111,15 +112,7 @@ What changed:
- [1.53.0] delete scan-community cron (#231 Phase 3c) — manual `claude --print` invocation of analyze-community.md
- [1.52.0] delete community-e2e-test cron (#231 Phase 3b) — manual local-shepherd review of scan-community digest
- [1.51.0] delete version-test cron (#231 Phase 3a) — manual local-Max replacement via npm i + local-shepherd
- [1.50.0] local-shepherd.sh --strip-paths flag (#231 Phase 2 — replaces deleted prove-it-test cron)
- [1.49.0] local-shepherd.sh --compare-baseline flag (#230)
- [1.48.0] SKILL.md trim — token bloat audit phase 2 follow-up
- [1.47.0] Codex review progress wrapper (#259)
- [1.46.1] npx check surfaces dangling+enabled plugin state (#266)
- [1.46.0] PreCompact dry-run env vars (#240)
- [1.45.0] PreCompact path (c) — SHA-ancestry self-heal (#257)
- [1.44.1] Autocompact compound-misconfig detection (#207)
... (full entries from fetched CHANGELOG)
... (older entries omitted — read the full CHANGELOG.md for anything pre-1.51.0)
```

Read the actual entries from the fetched CHANGELOG; don't paraphrase. The user wants to see exactly what shipped.
Expand Down