v1.39.0: community feature-discovery scanner (#207) - #244
Closed
BaseInfinity wants to merge 4 commits into
Closed
Conversation
Codex round 1 review of v1.39.0 surfaced that v1.38.0's new tests weren't wired into ci.yml/CONTRIBUTING.md, so test-workflow-triggers fails on this branch (164/1). - ci.yml: append test-repo-complexity.sh (#233) + test-prompt-hook- fires-once.sh (#224) after test-local-shepherd.sh - CONTRIBUTING.md: add same two entries to the local-test command list (must mirror ci.yml validate steps; test-workflow-triggers enforces this) Verified: bash tests/test-workflow-triggers.sh → 165/165 PASS.
Adds repo-complexity heuristic + 3-way setup prompt so simple repos can opt into Sonnet 4.6 in-session while keeping cross-model review at the flagship tier. - New cli/lib/repo-complexity.js + `npx agentic-sdlc-wizard complexity` subcommand. Classifies via filesystem signals (LOC, tests, hooks, workflows) plus stakes flag (.env / secrets/ / credentials/ at any depth forces complex regardless of size). - Setup Step 9.5 expanded from binary y/N to 3-way (no-pin / mixed / flagship), with the heuristic suggesting a default per repo. - Cross-model review section in skills/sdlc/SKILL.md explicitly notes reviewer always at flagship regardless of coder pin. - Update skill Step 7.5 recognizes sonnet[1m] as a valid mixed-mode pin (no migration prompt). - Reconciles with #198: opt-in per-project; no-pin remains default. Tests: 11/11 in tests/test-repo-complexity.sh including boundary (29 tests/4 hooks/4 workflows/~9K LOC → simple, 30 tests = high threshold → complex) and nested stakes (config/.env, app/secrets/). 128/128 hook regression, 22/22 doc consistency, 70/70 cli, 25/25 plugin, 10/10 compliance. Codex 2-round review: round 1 5/10 NOT CERTIFIED (4 findings — un- runnable doc command, root-only stakes, threshold mismatch with docs, stale 1.37.1 example), round 2 9/10 CERTIFIED.
…8.0) CC 2.1.118 shipped a fix for prompt hooks double-firing when an agent-hook verifier subagent itself made tool calls. We can't unit- test CC's behavior, but we can ship instrumentation so the maintainer can verify the fix holds in real sessions. - hooks/sdlc-prompt-check.sh: opt-in via SDLC_HOOK_FIRE_LOG env var. Each post-dedupe invocation appends one tab-separated record (<ts>\t<pid>\tsdlc-prompt-check). Unwritable paths fail silently. - tests/test-prompt-hook-fires-once.sh: 6 tests asserting counter increment, opt-in semantics, log shape, byte-identical output diff (instrumented vs uninstrumented), and unwritable-path tolerance. Test harness cd's into an isolated workspace so the result is independent of caller cwd. - CLAUDE_CODE_SDLC_WIZARD.md: new "Verifying Prompt-Hook-Fires-Once" subsection with maintainer procedure (set env, run session, count log lines, compare to prompt count). Codex 2-round (rounds 3+4 of the v1.38.0 review): round 3 8/10 (test harness false-green from caller cwd, weak output assertion), round 4 10/10 CERTIFIED.
Contributor
|
Claude encountered an error —— View job I'll analyze this and get back to you. |
Replaces the deleted CI scan-community job (per #231 Phase 3 plan) with a maintainer-runnable offline scan. Extracts /[a-z][a-z0-9-]* slash-command mentions from transcript text (Reddit / HN / Discord exports), dedupes against tests/e2e/known-slash-commands.txt allowlist, emits JSON digest of unknown candidates. - tests/e2e/scan-community.sh: scanner script. Lowercases line before matching (case-insensitive); sample window is centered on the matched slash so long-line context isn't truncated; reads files via 'cat < "$arg"' so dash-leading filenames work. - tests/e2e/known-slash-commands.txt: allowlist seeded with wizard skills, CC native commands, and common URL-path false positives. Plain text format with # comments; maintainer appends as new commands ship. - tests/test-community-scanner.sh: 14 tests covering detection, allowlist filter (CC native + wizard skills), case-insensitive extraction, dedup + count, empty-input edge case, JSON shape, stdin input, multi-file aggregation, sample-context inclusion, long-line sample window, dash-leading filename. - CLAUDE_CODE_SDLC_WIZARD.md: new "Community Feature-Discovery Scanner" subsection with maintainer procedure (capture transcripts → run scanner → triage `jq .candidates`). - CHANGELOG, version bump 1.38.0 → 1.39.0 across package.json, plugin.json, marketplace.json, SDLC.md, CLAUDE_CODE_SDLC_WIZARD.md version comments, and skills/update/SKILL.md Step 3 example. Codex 3-round review: round 1 6/10 NOT CERTIFIED (orphan tests not in CI, case-sensitive extraction, sample truncation can drop slash, stale 1.38.0 in update skill, dash-leading filename), round 2 8/10 NOT CERTIFIED (5 findings FIXED but git index UU pending), round 3 10/10 CERTIFIED.
4 tasks
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/e2e/scan-community.shextracts unknown/[a-z][a-z0-9-]*slash-command mentions from transcript text (Reddit / HN / Discord exports), dedupes againsttests/e2e/known-slash-commands.txtallowlist, emits JSON digest of candidates with count + sample. Replaces the deleted CI scan-community job (per ROADMAP roadmap(#212): Option 1 — de-require e2e, shepherd becomes advisory #231 Phase 3) with a maintainer-runnable offline scan.What's in the allowlist
/sdlc,/setup,/update,/feedback,/code-review,/less-permission-prompts,/claude-automation-recommender,/schedule,/ultrareview/help,/clear,/model,/effort,/usage,/cost,/stats,/compact,/resume,/init,/mcp,/plugin,/agents,/hooks,/permissions,/sandbox,/fast,/exit,/login,/logout,/doctor,/install,/uninstall,/settings/dev,/usr,/var,/tmp,/etc,/bin,/lib,/opt,/home,/root,/proc,/sys,/run,/mnt,/media,/srvLength-≥4 filter drops
/a//abstyle noise.Tests
14/14
tests/test-community-scanner.sh(NEW): detection, allowlist filter (CC native + wizard skills), case-insensitive extraction, dedup + count, empty-input, JSON shape, stdin input, multi-file aggregation, sample-context inclusion, long-line sample window, dash-leading filename. All other test suites green.Cross-model review
Codex 3-round: round 1 6/10 NOT CERTIFIED (5 findings — orphan tests not in CI [fixed in parent commit], case-sensitive extraction, sample truncation, stale 1.38.0 in update skill, dash-leading filename), round 2 8/10 NOT CERTIFIED (5 findings FIXED but git index UU pending), round 3 10/10 CERTIFIED.
Stacked on top of v1.38.0
This PR depends on #243 (v1.38.0). Merge #243 first; rebase this PR if needed.
Test plan
validatejob passes (depends on v1.38.0: mixed-mode tier (#233) + prompt-hook-fires-once instrumentation (#224) #243's CI fix landing)echo "Try /unknownthing in this prompt" | bash tests/e2e/scan-community.sh -→ expect/unknownthingin candidates