You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Build the CI validator and index.json generator the docs already specify #48
docs/bot-schema.md and CONTRIBUTING.md both describe a CI validator as though it exists — "a required field turns that silent runtime blank into a loud CI failure on the pull request, before you ever look at it", "Validation catches the mechanical things. Review is for the two questions a schema cannot answer." None of it is built yet. cc @anildukkipatty
Missing today:
.github/workflows/ — does not exist, so no checks run on any PR
schema/bot.schema.json — does not exist, yet all 12 bots point $schema at ../../schema/bot.schema.json, so every bot.json has a dangling reference and no editor completion
index.json — described as "generated by CI ... the only file the app fetches". Not generated, so nothing consumes the library yet
verified.json + CODEOWNERS — the docs put the verified badge in a maintainer-only root file. Neither exists, so the badge has no source
The practical effect is that all twelve mechanical checks fall on whoever reviews the PR, by hand, every time. #46 and #47 are both partly a consequence of that.
Rules the docs already commit to
Fail the PR when:
a required field is missing or empty
slug != folder name, or collides with another bot
name collides with another bot
features is not exactly 3, or any length limit is exceeded (name ≤24, description ≤80, about 120–400, features ≤60 each, examplePrompt ≤200, slug matching ^[a-z0-9]+(-[a-z0-9]+)*$)
category, mascot.body, mascot.color, mascot.activity, agent or permissionMode is outside its enum
agent is opencode and model is absent
instructions.md is missing, empty, or over ~600 words
author.github does not match the PR author
the PR touches more than one bot folder, or edits index.json / verified.json
the diff contains an absolute home-directory path or anything shaped like a credential
Warn (not fail) on a missing mascot block — the docs are explicit that this is a nudge in review, not a break.
On merge to main, regenerate index.json: every bot.json field, plus instructions and setupInstructions inlined from the Markdown, plus derived authorPhoto, the resolved colour variable, and verified from verified.json.
Two bots on main would fail rule 7 today
I dry-ran the list above against current main:
Bot
Result
shipguard
FAIL — instructions.md is 1392 words vs the ~600 limit
npm-publisher
FAIL — 650 words
lumie, merge-safety-reviewer, npm-publisher
auto-approve with no allowedTools fence
everything else
passes
So the validator needs a decision before it can go green: either grandfather what's already merged, or fix those two. Worth noting shipguard's length is mostly its very thorough disallowedTools list, which is a good thing — the word limit applies to instructions.md only, so it's the prose that needs trimming, and it may be that 600 is simply the wrong number.
The three auto-approve-without-a-fence bots aren't a documented failure, but CONTRIBUTING.md says "pair it with a tool allowlist ... or send it back", so surfacing them as a PR warning would put the right pressure on new submissions.
These aren't in the docs but each would have caught something a human reviewer nearly missed:
Tool names must match the agent's namespace.Add Design Doc Diff Checker #38 shipped "agent": "opencode" with "allowedTools": ["Read","Grep","Glob","Bash"] — Claude Code's capitalisation. opencode uses lowercase tool names, so that fence likely matches nothing or is ignored outright. A cross-check of tool names against the declared agent is cheap and catches a class of silently-broken fences.
model shape, and ideally resolvability.provider/model shape is trivial to assert. A wrong slug means the bot fails at install for every user, so even a shape check beats nothing.
Flag setup.md for reviewer attention. Per the docs a setup.md makes the bot refuse all work until setup succeeds, so an unnecessary step disables the bot rather than degrading it. 7 of 12 bots ship one. A warning prompting "is this genuinely required?" is in the spirit of the existing guidance.
Soft/style warnings, low priority: description should be one sentence (standout's is three, within the char limit).
Sequencing
There are 6 open PRs (#40–#45), all one contributor's "Ghost of <maintainer>" series with near-identical shape. Landing the validator before that batch means it gets exercised on six similar PRs at once, and they get checked consistently rather than by reviewer stamina. That's the argument for doing this next rather than after.
docs/bot-schema.mdandCONTRIBUTING.mdboth describe a CI validator as though it exists — "a required field turns that silent runtime blank into a loud CI failure on the pull request, before you ever look at it", "Validation catches the mechanical things. Review is for the two questions a schema cannot answer." None of it is built yet. cc @anildukkipattyMissing today:
.github/workflows/— does not exist, so no checks run on any PRschema/bot.schema.json— does not exist, yet all 12 bots point$schemaat../../schema/bot.schema.json, so everybot.jsonhas a dangling reference and no editor completionindex.json— described as "generated by CI ... the only file the app fetches". Not generated, so nothing consumes the library yetverified.json+ CODEOWNERS — the docs put the verified badge in a maintainer-only root file. Neither exists, so the badge has no sourceThe practical effect is that all twelve mechanical checks fall on whoever reviews the PR, by hand, every time. #46 and #47 are both partly a consequence of that.
Rules the docs already commit to
Fail the PR when:
slug!= folder name, or collides with another botnamecollides with another botfeaturesis not exactly 3, or any length limit is exceeded (name≤24,description≤80,about120–400,features≤60 each,examplePrompt≤200,slugmatching^[a-z0-9]+(-[a-z0-9]+)*$)category,mascot.body,mascot.color,mascot.activity,agentorpermissionModeis outside its enumagentisopencodeandmodelis absentinstructions.mdis missing, empty, or over ~600 wordsauthor.githubdoes not match the PR authorindex.json/verified.jsonWarn (not fail) on a missing
mascotblock — the docs are explicit that this is a nudge in review, not a break.On merge to
main, regenerateindex.json: everybot.jsonfield, plusinstructionsandsetupInstructionsinlined from the Markdown, plus derivedauthorPhoto, the resolved colour variable, andverifiedfromverified.json.Two bots on main would fail rule 7 today
I dry-ran the list above against current
main:shipguardinstructions.mdis 1392 words vs the ~600 limitnpm-publisherlumie,merge-safety-reviewer,npm-publisherauto-approvewith noallowedToolsfenceSo the validator needs a decision before it can go green: either grandfather what's already merged, or fix those two. Worth noting
shipguard's length is mostly its very thoroughdisallowedToolslist, which is a good thing — the word limit applies toinstructions.mdonly, so it's the prose that needs trimming, and it may be that 600 is simply the wrong number.The three
auto-approve-without-a-fence bots aren't a documented failure, butCONTRIBUTING.mdsays "pair it with a tool allowlist ... or send it back", so surfacing them as a PR warning would put the right pressure on new submissions.Three checks worth adding, from #46 and #47
These aren't in the docs but each would have caught something a human reviewer nearly missed:
"agent": "opencode"with"allowedTools": ["Read","Grep","Glob","Bash"]— Claude Code's capitalisation. opencode uses lowercase tool names, so that fence likely matches nothing or is ignored outright. A cross-check of tool names against the declared agent is cheap and catches a class of silently-broken fences.modelshape, and ideally resolvability.provider/modelshape is trivial to assert. A wrong slug means the bot fails at install for every user, so even a shape check beats nothing.setup.mdfor reviewer attention. Per the docs asetup.mdmakes the bot refuse all work until setup succeeds, so an unnecessary step disables the bot rather than degrading it. 7 of 12 bots ship one. A warning prompting "is this genuinely required?" is in the spirit of the existing guidance.Soft/style warnings, low priority:
descriptionshould be one sentence (standout's is three, within the char limit).Sequencing
There are 6 open PRs (#40–#45), all one contributor's "Ghost of <maintainer>" series with near-identical shape. Landing the validator before that batch means it gets exercised on six similar PRs at once, and they get checked consistently rather than by reviewer stamina. That's the argument for doing this next rather than after.