Skip to content

feat(holmes): make PR comments plain-English and glossary-backed #466

Closed
Closed
feat(holmes): make PR comments plain-English and glossary-backed#466
Labels
holmesHOLMES scoring + workflowslegend:EVIDENCEEvidence/Holmes/CI/proof workv0.3.0Scheduled work for the v0.3.0 release.work:productDirectly improves a sponsor actor outcome
Milestone
Goalpost: Make It Truthful

Description

@flyingrobots
flyingrobots
opened on Mar 30, 2026

Sponsor actor

A pull-request reviewer who needs to decide whether a change is safe to merge without already knowing Wesley/HOLMES jargon.

Hill

A reviewer can read the HOLMES GitHub comment and understand the ship recommendation, the top reasons, the next actions, and the jargon from the comment alone.

Playback

Open a PR with a HOLMES comment and understand, without opening source code or docs:

  • safe / investigate / do not ship
  • why
  • what to do next
  • what SCS / TCI / MRI / evidence trust mean

Type

product

Invariants

  • keep the underlying report truthful
  • do not hide failing gates or weak evidence
  • keep the GitHub comment machine-generated and reproducible
  • do not break existing JSON report contracts

Non-goals

  • redesign the CLI markdown report wholesale
  • remove the Holmes/Watson/Moriarty flavor
  • change SHIPME or certificate policy

Acceptance

  • the PR comment starts with a plain-English summary
  • the visible summary avoids unexplained acronyms
  • the comment includes actionable next steps
  • a collapsed glossary defines the major HOLMES terms
  • tests cover the comment-builder behavior

Activity

  1. flyingrobots
    added
    holmesHOLMES scoring + workflows
    work:productDirectly improves a sponsor actor outcome
    on Mar 30, 2026
  2. flyingrobots
    added
    lane:inbox
    legend:EVIDENCEEvidence/Holmes/CI/proof work
    on Jun 4, 2026
  3. flyingrobots
    added this to the Goalpost: Make It Truthful milestone on Jun 24, 2026
  4. flyingrobots
    removed
    lane:inbox
    on Jun 24, 2026
  5. flyingrobots
    added this to Wesley Roadmapon Jun 24, 2026
  6. github-project-automation
    moved this to Todo in Wesley Roadmapon Jun 24, 2026
  7. flyingrobots
    added
    triage:bad-codeDebt intake that still needs scheduling, closure, or a repo move
    on Jun 24, 2026

  8. flyingrobots commented on Jun 25, 2026

    @flyingrobots
    flyingrobots
    on Jun 25, 2026
    OwnerAuthor

    Triage decision: keep this in Wesley and move it out of the inbox into lane:v0.3.0 / Goalpost: Make It Truthful.

    Rationale: Plain-English, glossary-backed HOLMES comments are still current reviewer-facing evidence work.

    This is scheduled core Wesley work now, not an inbox item. The triage label is being removed as part of this state change.

  9. flyingrobots
    added
    v0.3.0Scheduled work for the v0.3.0 release.
    and removed
    triage:bad-codeDebt intake that still needs scheduling, closure, or a repo move
    on Jun 25, 2026

  10. flyingrobots commented on Jul 6, 2026

    @flyingrobots
    flyingrobots
    on Jul 6, 2026
    OwnerAuthor

    Bookkeeping closeout: this is complete in shipped code.

    Evidence:

    • PR [codex] Clarify Holmes output and move Wesley onto METHOD #467 explicitly included this issue and shipped the Holmes PR comment readability work.
    • Current code exposes buildHolmesSuiteComment from packages/wesley-holmes/src/pr-comment.mjs.
    • Current regression coverage in packages/wesley-holmes/test/pr-comment.test.mjs asserts the issue acceptance criteria: plain-English summary, actionable next steps, glossary, and no unexplained SCS/TCI/MRI before the glossary.
    • The live PR test: cover Holmes weights JSON output #721 Holmes comment also shows the shipped behavior: plain-English readout, suggested next actions, and collapsed glossary.

    Closing as completed. Future refinements should be filed as narrower follow-up issues rather than keeping this broad acceptance issue open.

  11. flyingrobots
    closed this as completedon Jul 6, 2026
  12. github-project-automation
    moved this from Todo to Done in Wesley Roadmapon Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    holmesHOLMES scoring + workflowslegend:EVIDENCEEvidence/Holmes/CI/proof workv0.3.0Scheduled work for the v0.3.0 release.work:productDirectly improves a sponsor actor outcome

    Projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions