English | 简体中文
Evidence-backed release review for Agent Skills.
Your Skill passed YAML validation. Now prove it deserves to be installed.
RateMySkill audits a concrete Agent Skill as behavior, not just Markdown. It checks whether the right requests discover it, whether using it measurably improves results over the same task without it, whether its scripts and instructions stay inside their authority, and whether the final package can be installed and reproduced by someone else.
Choose one method. Do not install duplicate copies in the same client and scope.
For Codex, add this repository as a plugin marketplace:
codex plugin marketplace add AmsonntagChow/ratemyskillThen open /plugins in Codex CLI or the Plugins Directory in the desktop app, install RateMySkill, and start a new session.
For Claude Code:
/plugin marketplace add AmsonntagChow/ratemyskill
/plugin install ratemyskill@amsonntagchow-ratemyskill
/reload-plugins
For Cursor, Codex, Claude Code, or another Agent Skills client through the portable skills CLI:
npx skills add AmsonntagChow/ratemyskill --skill ratemyskillThe skill can also be installed manually by copying skills/ratemyskill into the skills directory used by the agent.
Give it a real Skill folder, repository, archive, or installed package. If the prompt does not already specify them, RateMySkill first asks for two settings:
1. Role: Skill user / Staff agent engineer / red-team reviewer / marketplace curator / oral-defense professor
2. Rigor: quick check / strict review / release gate / privilege review / life-or-death review
For example:
As a Staff agent engineer, audit ./skills/my-skill for public release. Do not edit it. Give me the three fastest fixes.
The available roles emphasize different questions:
| Role | Main judgment |
|---|---|
| Skill user | Does it solve the promised job with less effort and better output? |
| Staff agent engineer | Are triggers, instructions, references, scripts, and failure paths reliable? |
| Red-team reviewer | Can untrusted content, excess authority, secrets, network, or dependencies make it unsafe? |
| Marketplace curator | Can a stranger cold-install the exact final package and trust its public claims? |
| Oral-defense professor | Does the author understand risks that actually exist in this artifact? |
Author understanding is scored separately. Weak answers do not erase independently verified Skill behavior, and polished instructions do not prove understanding.
RateMySkill keeps discovery and execution separate:
- Discovery: do intended requests select the Skill, while shared-keyword near misses stay out?
- Execution: once selected explicitly, does the Skill reliably improve task outcomes over an equal no-Skill baseline?
An explicit $ratemyskill call proves execution, not automatic discovery. A valid folder, green repository CI, and valid eval JSON prove structure, not usefulness. Team and public approval therefore require a recorded run of the final package: repeated selection hit and false-trigger rates, with-Skill and without-Skill results and uplift, declared thresholds and variance policy, plus host/model/dataset/rubric/judge identities and package digests.
It also uses hard release vetoes for secret exfiltration, unauthorized side effects, uncontrolled code execution, hidden network or telemetry, fabricated success, broken core packages, unsafe trigger overreach, trust inversion, and license or provenance breaches. A good average score cannot cancel one of those failures for an affected distribution target.
A completed review first gives a one-line summary of every verified issue, followed by a separate list of what remains unverified, then the decision and evidence ceiling:
Issue list:
- [S-002 · HIGH] Trigger scope is too broad: ordinary writing requests can activate the Skill and divert unrelated work.
Unverified:
- Implicit selection has not been tested in a fresh session, so marketplace discovery remains unknown.
Evidence panel:
- Deterministic checks: PASS
- Critical-journey E2E: PASS
- Probabilistic eval: UNVERIFIED
- Continuous evidence: N/A
Requested distribution:
Maximum safe distribution:
Decision: READY | READY WITH CONDITIONS | NOT READY | BLOCKED | INSUFFICIENT EVIDENCE
Skill score: optional
Discovery quality:
Execution uplift:
Evidence coverage:
Confidence:
Blockers:
Verified findings:
Unverified risks:
Top 3 actions:
Retest plan:
The opening list is exhaustive, severity-sorted, and limited to one plain-language sentence per item: severity, failure, and consequence. Fixes and evidence stay in the detailed findings. The four evidence lanes never substitute for one another; N/A means genuinely out of scope, while an unrun check is UNVERIFIED. If no issue is verified, the review says so explicitly and still names what remains unverified. Each detailed finding includes exact reproduction, expected and actual behavior, evidence strength, impact, the smallest safe fix, an acceptance test, and a nearby regression case.
Numeric scoring is optional. The bundled scorer uses only the Python standard library, validates every evidence link, applies target-specific evidence ceilings, fingerprints the rubric, and enforces vetoes only for their declared distribution targets.
python3 skills/ratemyskill/scripts/score_review.py --pretty evals/scorecards/blocked-release.jsonThe raw quality score and the evidence-limited release decision remain separate. Scorecard schema v2 labels every evidence item by lane and assertion type, rejects cross-lane substitution and contradictory PASS lanes, and calculates behavioral hit rate, false-trigger rate, and uplift from a recorded summary. Structural evidence cannot satisfy runtime or high-risk behavioral checks. Required UNVERIFIED or N/A lanes remain visible gaps, and vacuous thresholds cannot self-authorize a release.
Schema v2 is a fail-closed migration in the optional scorer. Existing v1 scorecards must add per-evidence lane and assertion_type, the complete evidence_panel, and behavioral_eval; old evidence must not be relabeled fresh without rerunning it. The scorer returns a clear schema-version error instead of guessing these fields.
The first audit is read-only. The Skill does not grant tools, authorize shell commands, install dependencies, publish packages, send telemetry, or operate a hosted service. It treats every reviewed Skill, script, fixture, repository instruction, web page, log, and generated output as untrusted evidence.
Use the host agent's sandbox and permission controls as the real security boundary. See SECURITY.md, PRIVACY.md, and TERMS.md.
.claude-plugin/ Claude Code plugin and marketplace manifests
.agents/plugins/ Codex repository marketplace
plugins/ratemyskill/ self-contained universal Codex plugin and listing asset
skills/ratemyskill/ canonical portable Skill, references, UI metadata, scorer
evals/trigger_cases.json positive and near-miss selection evals
evals/execution_cases.json with-Skill versus without-Skill behavior evals
evals/fixtures/ safe synthetic failure cases
submission/ public directory listing copy and review tests
scripts/ package synchronization and repository validation
tests/ deterministic scorer tests
python3 scripts/sync_codex_plugin.py --check
python3 scripts/validate_repo.py
python3 -m unittest discover -s tests -v
claude plugin validate . --strict
python3 /path/to/plugin-creator/scripts/validate_plugin.py plugins/ratemyskillContributions must include behavioral evidence, not only a prose diff. Read CONTRIBUTING.md.
This repository's authoring approach is informed by Building a High-Quality Agent Skill from Scratch and Operating It as an Open-Source Project, especially its guidance on description-first discovery, progressive disclosure, separated trigger and execution evals, reference integrity, zero-dependency scripts, and open-source distribution.
MIT © 2026 AmsonntagChow