Repository navigation
Put the hallucination-hunter agent where it can actually be found - #3
Merged
Merged
Conversation
The agent has existed since 1.1.1 and was almost invisible. It appeared in two prose sentences and nowhere else: not in either flowchart, not on the Pages site, not in the repository description. A subagent nobody can see is a subagent nobody runs. The interesting part is why the map did not show it. skill_mapper.py builds its graph from resolved markdown links in SKILL.md, and SKILL.md named the path in backticks -- which is text, not a link. So the only route to the file was through README.md, and README.md is not on the path Claude follows under progressive disclosure. The file was reachable by a human browsing GitHub and not by the agent that was supposed to use it. Making it a real link takes the map from 35 resolved links to 36 and puts SKILL.md one hop from the agent. That is the same class of defect skill_mapper exists to surface, found in this repository by running this repository's own tool on itself. The conceptual flowchart in the README was also wrong in a smaller way: it claimed six independent inspections and drew five. claim_auditor.py was missing from it entirely. Both it and the agent are now in the diagram, the agent shown where it actually sits -- downstream of claim_auditor, taking its UNVERIFIED list as a worklist rather than starting from prose. The "genuine output" mermaid block was regenerated from the current tool rather than hand-patched. It had drifted: the node-id scheme changed at some point and the committed block still used the old one. Adds a features table, which the README did not have, covering all seven scripts and the agent, and says plainly which of them is not a script. The Pages site gains a "Where the scripts stop" section explaining the boundary the agent exists to cross: a deterministic check can prove a flag does not exist, but not that a citation fails to support the claim it is attached to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The hallucination-hunter agent has shipped since 1.1.1 and was almost invisible — two prose sentences, and nothing else. Not in either flowchart, not on the Pages site, not in the repository description. A subagent nobody can see is a subagent nobody runs.
Why the map never showed it
skill_mapper.pybuilds its graph from resolved markdown links in SKILL.md. SKILL.md named the path in backticks — which is text, not a link. So the only route to the file was throughREADME.md, and README.md is not on the path Claude follows under progressive disclosure.The file was reachable by a human browsing GitHub, and not by the agent that was supposed to use it.
Making it a real link takes the map from 35 resolved links to 36 and puts SKILL.md one hop from the agent. Verified with the tool, before and after:
That is the same class of defect
skill_mapper.pyexists to surface, found in this repository by running this repository's own tool on itself.The conceptual flowchart was also wrong
It said "six independent inspections" and drew five —
claim_auditor.pywas missing from the diagram entirely. Both it and the agent are now in it, with the agent placed where it actually sits: downstream ofclaim_auditor, taking itsUNVERIFIEDlist as a worklist rather than starting from prose.The "genuine output" block had drifted
Regenerated from the current tool rather than hand-patched — the node-id scheme changed at some point (
SKILL_md→n_SKILL_md) and the committed block still used the old one. It stays trimmed to README size, and says so.New: a features table
The README had no at-a-glance summary. Added one covering all seven scripts and the agent, with a column saying what each one answers rather than what it is called — and marking plainly which one is not a script.
Pages site
Gains a "Where the scripts stop" section explaining the boundary the agent exists to cross: a deterministic check can prove a documented flag does not exist, but not that a cited paper fails to say what it is cited for, that a version went stale, or that a benchmark number has no source. Those need judgement, and judgement is where a model hallucinates — so the agent is built to make guessing expensive: tools first, evidence per finding, UNVERIFIED rather than a guess, and a closing statement of what it did not check.
Also adds the agent to the resources list and to
.github/repository-metadata.yml.Checks
claim_auditor.py→ TRUTHFUL (0 contradictions)script_tester.py→ exit 0pytest tests→ 196 passedscripts/andtests/are untouched — this is documentation and one link.🤖 Generated with Claude Code