feat: implement issue #615 — [Phase 2] Register skill_candidate artifact type (skill rubric + pass/score channel)

[don-petry]

Closes #615

Implemented by dev-lead agent. Please review.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Warning

Review limit reached

@don-petry, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 28 minutes and 17 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 097bd479-bc32-41fe-a50f-fe53bd5eb185

📥 Commits

Reviewing files that changed from the base of the PR and between 2f97b2c and a818174.

⛔ Files ignored due to path filters (1)

• scripts/lib/review-registry.tsv is excluded by !**/*.tsv

📒 Files selected for processing (6)

• .github/workflows/lint.yml
• prompts/skill-review.md
• scripts/evals/review-skill.sh
• scripts/lib/README.md
• scripts/post-skill-score.sh
• tests/test_skill_review.bats

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch dev-lead/issue-615-20260617-1840

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

[gemini-code-assist]

Code Review

This pull request registers a new skill_candidate artifact type to the review registry, enabling automated strict-improvement reviews of prompt-skill edits. It introduces the prompts/skill-review.md rubric, the scripts/evals/review-skill.sh driver, the scripts/post-skill-score.sh output channel, and comprehensive BATS tests. The feedback highlights two key improvements: ensuring mktemp portability on macOS/BSD by moving the XXXXXX placeholder to the end of the template, and robustly extracting JSON from LLM output using a Python fallback to handle conversational preambles.

[don-petry]

Dev-Lead — fix-bot-comment (applied)

Changes committed and pushed.

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

[donpetry-bot]

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-06-17T19:56:23Z.

[don-petry]

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning

Issues addressed: 0
- No new issues detected (quality gate passing)
- No security hotspots identified
- No open review threads to resolve
Files changed: N/A
Skipped (informational): 0
```
**Status:** ✅ **No action required**
The SonarCloud bot reports a passing quality gate with zero issues. All CI checks pass and there are no reviewer blockers. The PR is in good shape.

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

[donpetry-bot]

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-06-17T20:00:22Z.

[don-petry]

Auto-rebase failed — merge conflict — this branch has conflicts with main that must be resolved.

dev-lead will attempt to resolve this automatically. If it cannot, a follow-up comment will explain what needs manual attention.

To resolve manually instead:

git fetch origin
git merge origin/main
# resolve conflicts, then:
git add .
git commit
git push

[don-petry]

Dev-Lead — rebase (no-changes)

Agent reasoning

PR: #770
Rebased onto: main
Conflicts resolved: 1 file
- .github/workflows/lint.yml: both sides added a new bats test entry after test_plan_review.bats — kept both (test_plan_critic.bats from main, test_skill_review.bats from PR)
Push: success
```

[donpetry-bot]

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-06-17T22:19:00Z.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[don-petry]

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

[donpetry-bot]

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: a818174ed00a66c1c8aecf31b169f86ba305f1ca
Cascade: triage → deep (triage: haiku 4.5 → deep: opus 4.8 + duck: o4-mini → audit: fable 5)

Summary

PR #770 additively registers the skill_candidate review artifact type (skill rubric + machine-readable pass/score channel) per issue #615, with all 5 acceptance criteria met and full bats coverage. The two advisory-bot findings triage flagged as unaddressed (Gemini HIGH mktemp BSD portability and MEDIUM JSON-preamble robustness) are in fact already fixed at head SHA a818174 by the 'fix(bot): address bot feedback' commit. CI is fully green (CodeQL, gitleaks, AgentShield, SonarCloud, shellcheck, bats) and the change is automation tooling with no auth/secret/crypto/DB surface.

Findings

• INFO: Gemini HIGH finding resolved: mktemp template is now 'skill-score.json.XXXXXX' (trailing 3+ X's), which is portable across GNU and BSD/macOS mktemp. No action needed.
• INFO: Gemini MEDIUM finding resolved: a python3 raw_decode-based extractor now scans the agent's stdout for the valid {artifact_type:'skill_candidate'} JSON object as a fallback when the model emits a conversational preamble instead of writing the output file. No action needed.
• INFO: Output channel correctly validates verdict (pass|fail), score type (number), and range [0,1], rejecting garbage signals before they reach the gate. Bats suite exercises registry resolution, the additive guard over pr_diff/plan_json, driver dispatch wiring, and all output-channel error paths.

---

Reviewed by the PR-review cascade (triage: haiku 4.5 → deep: opus 4.8 + duck: o4-mini → audit: fable 5). Reply if you need a human review.

[donpetry-bot]

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: a818174ed00a66c1c8aecf31b169f86ba305f1ca
Review mode: triage-approved (single reviewer)

Summary

PR #770 additively registers a third review artifact type, skill_candidate (Epic #581 strict-improvement skill reviewer), per issue #615: a skill rubric (prompts/skill-review.md), a content_ref driver (scripts/evals/review-skill.sh), a machine-readable pass/score output channel (scripts/post-skill-score.sh), a registry TSV row, README docs, and 268 lines of bats coverage. The triage-approved assessment holds: pure automation tooling with no auth/secret/crypto/DB surface, CI fully green, and the two Gemini advisory findings already resolved at this head SHA.

Linked issue analysis

Closes #615 ([Phase 2] Register skill_candidate artifact type). The registry now resolves skill_candidate -> {rubric: prompts/skill-review.md, output_channel: scripts/post-skill-score.sh}, the driver dispatches via the registry and reuses engine.sh deep-tier routing (no new model selector), and the output channel validates verdict (pass|fail) and a numeric score in [0,1] before handing the signal to the gate. Acceptance criteria are exercised by the bats suite (registry resolution, additive guard over pr_diff/plan_json, driver wiring, all output-channel error paths). Substantively addressed.

Findings

• INFO: Shell scripts are clean — set -euo pipefail, quoted expansions, and a BSD/macOS-portable mktemp template (skill-score.json.XXXXXX, trailing X's). Confirmed shellcheck/SonarCloud green.
• INFO: Gemini's two advisory findings (mktemp portability HIGH, JSON-preamble robustness MEDIUM) are already fixed at head a818174 via the 'fix(bot): address bot feedback' commit; a python3 raw_decode fallback recovers the JSON when the model emits a conversational preamble. No action needed.
• INFO: Output channel correctly rejects garbage signals (missing/non-JSON file, unknown verdict, non-numeric/out-of-range score) before they reach the strict-improvement gate, and statically guards against ever posting a GitHub PR review. CodeRabbit/Codex were rate-limited, not blocking; the change was already APPROVED by the prior full cascade at this same SHA.

CI status

All checks green or appropriately skipped. SUCCESS: CodeQL (actions/python), Secret scan (gitleaks), AgentShield, SonarCloud (Quality Gate passed — 0 new issues, 0 hotspots), shellcheck, bats, validate-agent-profiles, and the full Dev-Lead/Lint/Test suites. No failing checks.

---

Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.