Repository navigation
docs(research): prompts.chat source analysis and AutoBot comparison (#16108) - #16119
Conversation
…16108) Carrying a research session's output into a PR: it holds no worktree by design, and the two files were sitting uncommitted in the main tree, which is supposed to stay clean on base. 370-line audit of prompts.chat and f/awesome-chatgpt-prompts, fetched 2026-09-09, plus the _index.md row. Both land together because the row references the umbrella and its children. SOURCES ARE NAMED, DELIBERATELY. The research skill anonymizes a studied source by default and exempts one the operator asks to have cited, which is the case here. The doc says so in its opening paragraph. Keeping the citation rather than genericising it back is the point -- an anonymized source cannot be re-checked by the next reader, and the audit's conclusions are only as good as the ability to go and look. The finding that matters: of the 156-row corpus and 68 skills, only three carry a mechanism rather than a persona, and the corpus has ZERO self-correcting entries. The three mechanisms became #16112 (itemized re-audit rather than a single scalar), #16111 (a reconstruction test proving a compression is recoverable) and #16113 (declared invalid action combinations). Each adoption names the skill it came from. Nothing here proposes importing external code, data or dependencies. Seven issues were filed from this audit as umbrella #16108 with native sub-issues, and one blocked_by edge -- #16111 depends on #16110, because filing them together would have let the fix land as "add a reconstruction test" while the blind 200-character score stayed underneath it.
|
Warning Review limit reachedNext included review available in 58 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This PR's red is the bug #16107 fixes, and I am deliberately not rewording it
An ATX heading needs The obvious fix is to reword until green, and that is exactly the wrong move. It is what every previous author did, and it is why the defect survived #16050 — which was aimed straight at this gate — and then looked like a first occurrence when another session hit it this morning. A reword leaves no trace, so the gate has taught the same wrong lesson at least three times with no record of any of them. So this stays red until #16107 lands. Two reasons:
Nothing about this PR's own content is in question — the rationale is there, it is filled, and it says why this issue stands alone. |
…16104, #16122) (#16129) * fix(ci): clear the three defects every open PR is inheriting (#16089, #16104, #16122) Eight red check-runs across nine PRs reduce to three causes, none of which any of those PRs introduced. Fixing them one per PR means three CI suites and three review passes to unblock a queue that is already blocked; fixing them together means one. 1. DEPENDENCY SECURITY SCAN -- js-yaml ~4.3.1 -> ~4.3.2. The advisory range is 4.0.0-4.3.1, so 4.3.2 is outside it and IS the fix. With the redocly allowance already on base, this is the half that clears the scan. Affects #15980, #16088, #16123. 2. PYTHON-SUITE SHARD 7/12 -- a comment citing a line number. comment_line_number_citations_test forbids it, and security.yml carried `security_scan_gate.py:100` from a merge earlier today. Mine. A line number is invalidated by edits to code it does not describe and fails silently into whatever later occupies it -- and it was written inside a paragraph warning that the allowance is coarser than it looks. Symbol name only now. Affects #15980, #16106, #16107, #16117, #16125. 3. CHECK SAME-SCOPE BATCHING -- an ATX heading needs `#` followed by a space. check_pr_issue_batching.py treated any line starting with `#` as the next heading, so a rationale opening `#15961 alone...` made a FILLED section parse as empty. The message now also distinguishes "no section found" from "section found but empty", quoting the line that ended the parse -- because a red only self-corrects when it names its real cause, and every previous author here reworded until green, leaving no trace. Affects #16117, #16119, #16123, #16127. TAKING DEPENDABOT'S WHOLE package.json WOULD HAVE REGRESSED SIX PACKAGES, and that is the reason this is assembled by hand rather than merged from its branch. #16088 was cut before several upgrades landed, so adopting the file wholesale downgrades apexcharts 7.1.0->6.10.0, cytoscape 3.34.2->3.34.1, onnxruntime-web 1.29.0->1.26.0, vega-embed 7.2.0->7.1.0, vue-i18n 11.4.10->11.4.8 and vue-router 5.3.1->5.2.0. Caught by reading the diff rather than the intent. So the frontend change is exactly one line of package.json and three lines of the lockfile -- version, resolved and integrity for node_modules/js-yaml, copied verbatim from dependabot's verified entry rather than invented. The nine remaining `4.3.1` strings in the lockfile are debug, @types/esrecurse, ansis and polished; dependabot's own lockfile has the same nine. 44 tests pass: the batching parser's full suite including the four reproduction cases, and the citation guard that shard 7 was failing on. * security: drop the redocly allowance -- the scanner stopped reporting it (#16089) The gate failed on the allowance itself: allowance(s) ['@redocly/openapi-core'] name findings the scanner no longer reports. Remove them; an allowance that matches nothing tolerates whatever appears next. Not the advisory returning. The advisory GOING. This is the self-cleaning property that the allowance was justified against, arriving and doing exactly what it was described as doing -- and it is the third time today that a check I verified has then been right about my own work. Worth being precise about why an empty match is a failure rather than a shrug. An allowance keyed on a package name admits ANY future advisory in that package, so one that currently matches nothing is not inert -- it is a standing exemption for whatever appears there next, with nobody watching. Nothing else in this system asks whether an allowance is still needed, so without this check the entry sits until someone happens to reread the block. The npm gate now carries NO allowances, which is the state to defend. Adding one back needs the evidence the last one had: no reachable fix, checked against the REGISTRY rather than the advisory range -- a fix at a major behind a third party's dependency range is still a fix, which is the distinction that nearly made the js-yaml allowance permanent -- and the package does not ship. #16122's subject was pointing the allowance at a removal tracker. There is no allowance to track, so #16123 is now moot as well as superseded.
|
Review as a non-author: approve on content, one finding, and it is in the index row rather than the doc. Docs-only, two files, 374 added lines. I verified every claim in the body that could be checked rather than read. Structural claims — both hold. All nine issue references in the diff resolve, all are issues rather than PRs, and the three mechanism-to-issue mappings match the target titles — Codebase evidence — spot-checked four of six, all accurate. No internal identifiers. Scanned for IPs, The method note is the strongest section and I would resist any trim to it. It records that two findings originally rested on greps returning nothing, and that positive controls falsified part of both — the fidelity check turned out not absent but 200-characters-wide. It then tracks the same finding across four instruments and states the conclusion precisely: "what moved the number was always a new instrument, never a closer reading." That is the distinction between nothing found and did not look, written down with its own failures attached. The findingThe
The doc says:
Those are different claims. The doc's is bounded — six named axes, each with It also sits oddly beside the rest of its own sentence, which reports four adoptions and three filed defects. A reader who takes the row at face value has been told we are ahead everywhere and simultaneously that we have seven things to fix. The index row is the surface that gets read. Suggested: No objection to the named sources — the exemption is stated in the doc's opening paragraph, and the reasoning that an anonymised source cannot be re-checked by the next reader is right. Ready to merge from my side once CI reports, with or without the row reworded; I would rather it were reworded. |
The row said "our loop design is ahead on every axis"; the doc says "nothing in the source's loop design is ahead of ours", bounded to six named axes with file:line evidence against one source. The row dropped the comparand, leaving an unbounded claim beside its own report of four adoptions and three filed defects. The index is the surface people scan to decide whether to open the document, so the careful scoping ended up in the file they then do not read. Wording is the author's own, moved up rather than invented.
1 participant
Refs #16108
Thinking Path
Carrying a research session's output into a PR. That session holds no worktree by design — research files issues, not code — and its two files were sitting uncommitted in the main tree, which is supposed to stay clean on base. This is the handoff, not new work.
370-line audit of
prompts.chatandf/awesome-chatgpt-prompts, fetched 2026-09-09, plus onedocs/research/_index.mdrow. They land together because the row references the umbrella and its children.What Changed
docs/research/community-prompt-and-skill-registry.mddocs/research/_index.mdSources are named, deliberately
The research skill anonymizes a studied source by default and exempts one the operator asks to have cited. That exemption was invoked here, and the doc states it in its opening paragraph rather than leaving the reader to wonder.
Worth preserving rather than genericising back on review: an anonymised source cannot be re-checked by the next reader, and an audit's conclusions are worth exactly as much as the ability to go and look.
The finding
Of a 156-row corpus and 68 skills, only three carry a mechanism rather than a persona, and the corpus has zero self-correcting entries. Each of the three became an issue that names the skill it came from:
Nothing here proposes importing external code, data or dependencies.
Verification
Docs-only. The claim I can verify and did: the
_index.mdrow's issue references resolve — #16108 exists as an umbrella with #16110–#16116 attached as native sub-issues, and #16111 carries ablocked_byedge on #16110.That edge is the part worth keeping. Filing those two as one issue would have let the fix land as "add a reconstruction test" while the blind 200-character score stayed underneath it — the failure mode reproducing itself one layer up.
Model Used
Claude Opus 5 (1M context).
Single-issue rationale
#16108's doc alone. The two files are one deliverable — the index row is meaningless without the doc — and the seven children are separate implementation work owned by other sessions.
What I have not verified: the doc's claims about the external sources. I did not re-fetch
prompts.chator the 156-row CSV, so the audit's external observations rest on the research session's reading, not on mine. The sources are named precisely so that a reviewer who wants to can check them — which is the argument for the citation exemption, stated here rather than implied.