Skip to content

Promote data-driven AI-tells + first-class severity into prose.mjs - #9

Merged
bdelanghe merged 1 commit into
mainfrom
claude/prose-data-driven-severity
Jun 23, 2026
Merged

Promote data-driven AI-tells + first-class severity into prose.mjs#9
bdelanghe merged 1 commit into
mainfrom
claude/prose-data-driven-severity

Conversation

@bdelanghe

Copy link
Copy Markdown
Contributor

Promotes the spike from #8 into production, closing proposals 1 & 3 of #6. Behavior-preserving — the deterministic audit output is byte-identical; the change is structural.

What changed

  • ai-tells.json (root, canonical) — AI-tell patterns + lexicon as data, each rule carrying its own level. Promoted from spikes/; seeded from the vale-signs-of-ai-writing corpus. aiIsms() loads it; the structural tells that aren't simple regex (em-dash count, anaphora, tricolons) stay in code.
  • prose.mjs — every check now returns { level, msg } (error | warn | suggestion) instead of bare strings, so severity is first-class rather than re-derived downstream.
  • audit.mjs — renders the glyph from level (and folds the still-string, cached type-audit findings into the same model via a small classifier) instead of regex-matching message text for tiering.
  • spikes/vale-rules.mjs now reads the promoted root ai-tells.json (single source of truth — the spike can't drift from production); spikes/ai-tells.json removed; spike README notes the promotion. The optional vale-provider.mjs shell-out spike stays (proposal 2, still open).
  • test.mjs — assert on finding.msg; add coverage for the { level, msg } contract and an error-level data rule (chatbot artifact).
  • README.md — document the data-driven rules file and first-class severity.

Why

Before: patterns/lexicon hard-coded in prose.mjs, and tiers were pattern-matched from message text in audit.mjs (/^(ai-ism:|proofread:)/ etc.) — brittle and duplicated. Now the AI-tell rules are editable data that tracks the upstream corpus, and severity rides on each finding.

Verification

node test.mjs passes. Stubbed full node audit.mjs produces identical output to main, now driven by level:

  ✦ pdp.body.aiism       [body    ] 10/10
       ⚠ ai-ism: "isn't just a frame — it's" — antithesis "it isn't X — it's Y" cadence
       ⚠ ai-ism: "the easy part" — "the [easy/hard] part" framing
       ⚠ ai-ism: "Fast, simple, and beautiful" — rule-of-three triad; vary it
       · grammar/style: "just" — "just" can weaken meaning

New data rules (e.g. chatbot artifacts) now fire at error:

$ node spikes/vale-rules.mjs "As an AI language model, I cannot provide that."
       ✗ [error] chatbot meta-artifact — "As an AI language model"

(Full audit.mjs verified locally with stubs for the network-blocked JSR deps; CI has JSR access.)

Closes #6 proposals 1 & 3 (proposal 2 — the vale shell-out — remains an opt-in spike).

🤖 Generated with Claude Code


Generated by Claude Code

Migrates the spike (issue #6) into production:

- ai-tells.json (root, canonical) — AI-tell patterns + lexicon as data, each
  rule carrying its own severity level. Promoted from spikes/; seeded from the
  vale-signs-of-ai-writing corpus. aiIsms() loads it; the structural tells
  (em-dash count, anaphora, tricolons) stay in code.
- prose.mjs — every check now returns { level, msg } (error | warn |
  suggestion) instead of bare strings, so severity is first-class.
- audit.mjs — derives the render glyph from level (and classifies the still-
  string, cached type-audit findings into the same model) instead of
  regex-matching message text for tiering. Output is unchanged.
- spikes/ — vale-rules.mjs now reads the promoted root ai-tells.json (single
  source of truth, can't drift); spikes/ai-tells.json removed; README notes
  the promotion. The optional vale-provider.mjs spike stays (proposal 2).
- test.mjs — assert on finding.msg; add coverage for the { level, msg }
  contract and an error-level data rule.
- README — document the data-driven rules file and first-class severity.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011GNDVjvsbTPsbWV9fEyjLt
@bdelanghe
bdelanghe marked this pull request as ready for review June 23, 2026 03:00
@bdelanghe
bdelanghe merged commit 6eb6fd9 into main Jun 23, 2026
1 check passed
bdelanghe added a commit that referenced this pull request Jun 23, 2026
Bump 0.2.1 → 0.3.0. Since v0.2.1: AI-ism + overclaim checks (#5), proofread + readability + severity tiers (#7), Vale prior-art spikes (#8), and the data-driven ai-tells.json with first-class { level, msg } severity (#9).
@bdelanghe
bdelanghe deleted the claude/prose-data-driven-severity branch June 23, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adopt Vale's AI-tell rules instead of hand-maintaining the lexicon

2 participants