You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[LOW] Decision: does an evidence-support lint belong in the CLI, or upstream at staging? (PR #59) #63
Does an evidence-support lint belong in the CLI at all, or upstream at staging time?
PR [LOW] feat(review): lint findings whose confidence outruns their evidence #59 adds vhir review --findings --lint, flagging findings whose stated confidence outruns their cited evidence: HIGH confidence with no audit_ids, an interpretation asserting intent or attribution with nothing cited, an interpretation far longer than its observation, HIGH confidence alongside hedging.
The underlying problem is real and is not in dispute:confidence and confidence_justification are written by the same generative model that wrote the finding, and the examiner is shown a self-report as though it were calibrated.
The honest limitation is the decision. This is a lint over surface features of the text. It does not and cannot judge whether the cited evidence supports the claim — that needs reading the artifacts. A flagged finding may be perfectly sound; an unflagged one has not been verified. Both caveats are printed with the output, but a reader can still over-trust it.
Options
Ship as-is, in review. Cheap, opt-in, honest about its limits. Risk: an examiner reads "nothing flagged" as "checked".
Move it upstream to staging. The MCP backend writes these findings; flagging there means the model gets told its confidence is unsupported before a human ever sees the finding. Arguably the right place — the CLI is then reviewing better input rather than annotating bad input.
Don't ship it. If you judge that a surface lint invites more false confidence than it removes, that is a defensible call and I would not argue hard against it. This is the weakest of the four proposals in this series and I would rather say so than oversell it.
The word lists (_INTENT_TERMS, _ATTRIBUTION_TERMS, _HEDGE_TERMS) encode English, Windows-centric IR prose. They are plain tuples at the top of the module.
attribution-claim fires whether or not evidence is cited, on the grounds that a single host's artifacts essentially never support attribution. Tell me if that is too aggressive.
Related: #56, #57, #58 — same design review, each independently cherry-pickable.
Activity
scriptedstatement commentedon Oct 13, 2026
Closed with #59; see the comment there. Thank you.