Skip to content

ci(l1): prevent AI review comments from using #N notation that GitHub auto-links as issue references#6187

Merged
avilagaston9 merged 1 commit intomainfrom
fix/ai-review-enumeration
Feb 12, 2026
Merged

ci(l1): prevent AI review comments from using #N notation that GitHub auto-links as issue references#6187
avilagaston9 merged 1 commit intomainfrom
fix/ai-review-enumeration

Conversation

@avilagaston9
Copy link
Contributor

Motivation

The Claude AI code reviewer (.github/workflows/pr_ai_review.yaml) enumerates findings using #1, #2, etc., which GitHub auto-links as references to issues/PRs. This clutters PR activity feeds and confuses readers. Example: #6186 (comment)

Description

Add formatting rules to the AI review prompt (.github/prompts/ai-review.md) instructing the model to use 1., 2. or bullet points instead of #N, and to refer back to items as "Item 1", "Point 2" rather than "Issue #1".

Checklist

  • Updated STORE_SCHEMA_VERSION (crates/storage/lib.rs) if the PR includes breaking changes to the Store requiring a re-sync.

enumerated items as issue/PR references. The model now uses `1.`, `2.`
instead of `#1`, `#2`, and refers back to items as "Item 1" rather than
"Issue #1".
@github-actions
Copy link

🤖 Kimi Code Review

The change adds formatting rules to the AI review prompt. This is a documentation-only change that doesn't affect any code. The new rules are clear and will help maintain consistent formatting in AI-generated reviews. No issues found.


Automated review by Kimi (Moonshot AI) · custom prompt

@github-actions
Copy link

🤖 Codex Code Review

No code changes to review; this PR only updates reviewer prompt text in .github/prompts/ai-review.md. No correctness, security, performance, or Ethereum-specific concerns.


Automated review by OpenAI Codex · custom prompt

@github-actions
Copy link

🤖 Claude Code Review

That's fine - I have enough context from the PR description. Let me now provide the code review.


Code Review - PR #6187

Summary

This PR adds formatting rules to the AI review prompt to prevent GitHub from auto-linking enumerated items as issue/PR references. The change is small, well-motivated, and correctly addresses the problem described.

Findings

✅ Overall Assessment: LGTM

The changes are minimal, focused, and solve a real UX problem with the AI code reviewer workflow.

Detailed Review

File: .github/prompts/ai-review.md (lines 21-23)

Positives:

  1. Clear and specific instructions: The formatting rules are explicit about what to avoid (#N) and what to use instead (1., 2., bullet points).

  2. Good examples: Including concrete examples like #1, #2 and "Item 1", "Point 2" makes the instruction unambiguous for the AI model.

  3. Proper placement: The formatting rules are added at the end of the prompt, which is appropriate since they're meta-instructions about output format rather than review criteria.

  4. Consistent with existing tone: The addition maintains the concise, directive style of the original prompt.

Minor Suggestions:

  1. Wording clarity (line 22): Consider making the instruction even more explicit about referring back to previously mentioned items:

    - When cross-referencing previously mentioned items, use "Item 1", "Point 2", etc. — never "Issue #1" or "#1".

    This clarifies that the second rule is about cross-references, not just general enumeration.

  2. Consider alternative phrasing (line 22): The current wording NEVER use '#N' (e.g. #1, #2) might be slightly ambiguous. You could consider:

    - NEVER use hash-number notation (e.g., #1, #2) for enumeration — GitHub renders those as issue/PR references.

Non-issues:

  • The markdown formatting is correct
  • No trailing whitespace issues
  • The rules don't conflict with existing prompt instructions
  • The change is backwards-compatible (won't break existing workflow)

Testing Considerations

While this is a prompt change that doesn't have traditional unit tests, you may want to:

  • Monitor the next few AI-generated reviews to ensure the formatting rules are being followed
  • If the AI models still occasionally use #N notation, consider making the instruction more emphatic or moving it to the top of the prompt

Conclusion

This is a well-executed fix for an annoying UX issue. The change is minimal, clear, and should effectively prevent the GitHub auto-linking problem. Approved for merge.


Automated review by Claude (Anthropic) · custom prompt

@avilagaston9 avilagaston9 changed the title Fix AI review comments using #N notation that GitHub auto-links as issue references Prevent AI review comments from using #N notation that GitHub auto-links as issue references Feb 11, 2026
@avilagaston9 avilagaston9 changed the title Prevent AI review comments from using #N notation that GitHub auto-links as issue references ci(l1): prevent AI review comments from using #N notation that GitHub auto-links as issue references Feb 11, 2026
@github-actions github-actions bot added the L1 Ethereum client label Feb 11, 2026
@avilagaston9 avilagaston9 marked this pull request as ready for review February 11, 2026 21:00
@avilagaston9 avilagaston9 requested a review from a team as a code owner February 11, 2026 21:00
Copilot AI review requested due to automatic review settings February 11, 2026 21:00
@ethrex-project-sync ethrex-project-sync bot moved this to In Review in ethrex_l1 Feb 11, 2026
@github-actions
Copy link

🤖 Kimi Code Review

This PR adds formatting rules to the AI review prompt template. The changes are straightforward and correct:

  • Line 20-21: Adds clear formatting rules about not using #N syntax for enumeration
  • Line 22-23: Specifies how to refer back to items using "Item 1", "Point 2", etc.

The changes are appropriate and will help ensure consistent formatting in AI-generated reviews. No issues found.


Automated review by Kimi (Moonshot AI) · custom prompt

@github-actions
Copy link

🤖 Codex Code Review

Reviewed the diff. It only updates .github/prompts/ai-review.md to add formatting rules. No code changes, so there are no correctness, security, performance, EVM, or consensus concerns to flag.

Looks good.


Automated review by OpenAI Codex · custom prompt

@greptile-apps
Copy link

greptile-apps bot commented Feb 11, 2026

Greptile Overview

Greptile Summary

This PR adds formatting instructions to the AI code review prompt to prevent GitHub from auto-linking enumerated items as issue/PR references. The change instructs AI reviewers to use 1., 2. or bullet points instead of #1, #2, and to refer back to items as "Item 1" or "Point 2" rather than "Issue #1".

  • Directly addresses the problem described in the motivation (cluttered PR activity feeds from auto-linked references)
  • Simple, focused change that only modifies the prompt file
  • Clear instructions that should be easy for AI models to follow
  • No code logic changes, minimal risk

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The change only modifies documentation (AI review prompt) with clear formatting instructions and does not affect any code logic, build processes, or runtime behavior
  • No files require special attention

Important Files Changed

Filename Overview
.github/prompts/ai-review.md Added formatting rules to prevent GitHub from auto-linking #N notation as issue references in AI review comments

@github-actions
Copy link

🤖 Claude Code Review

Perfect! Now I have all the context I need to review this PR. Let me provide my code review.


Code Review: PR 6187

Summary

This PR adds formatting rules to the AI review prompt to prevent GitHub from auto-linking enumerated items (like #1, #2) as issue/PR references. The change is straightforward and addresses a legitimate UX issue.

Review Findings

✅ Correctness & Implementation

The implementation is correct and minimal. The added formatting rules (.github/prompts/ai-review.md:21-23) clearly instruct AI reviewers to:

  1. Use 1., 2. or bullet points instead of #N notation
  2. Refer back to items using "Item 1", "Point 2" rather than "Issue docs: add milestones #1"

✅ Documentation & Clarity

The instructions are clear, provide concrete examples (#1, #2), and explain the alternative formats. The placement at the end of the prompt under a "Formatting rules:" section is appropriate.

✅ Impact & Risk

  • Low risk: This is a non-code change that only affects AI-generated review comments
  • No breaking changes: No impact on the actual codebase or storage schema
  • Immediate benefit: Prevents GitHub activity feed pollution and confusion

Observations

Minor suggestion (optional): The formatting rules could optionally include an instruction to add "Output your review findings in markdown format" to ensure structured output, but this may already be implicit in the AI models' behavior.

Note on effectiveness: The success of this change depends on:

  • How well each AI model (Kimi, Codex, Claude) follows the instructions
  • Whether the reusable workflows at lambdaclass/actions properly inject this custom prompt

The PR author should verify that the .github/prompts/ai-review.md file is actually being used by the reusable workflows referenced in .github/workflows/pr_ai_review.yaml:30,35,40.

Verdict

✅ Approved - This is a good, targeted fix for a real UX issue. The implementation is minimal and correct.


Automated review by Claude (Anthropic) · custom prompt

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the AI review prompt to avoid using #1, #2, etc. when enumerating findings, preventing GitHub from auto-linking them as issue/PR references in PR activity feeds.

Changes:

  • Adds explicit formatting rules to the AI reviewer prompt to avoid #N enumeration.
  • Instructs the model to refer back to enumerated items as “Item N” / “Point N” instead of “Issue #N”.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.


Formatting rules:
- NEVER use `#N` (e.g. #1, #2) for enumeration — GitHub renders those as issue/PR references. Use `1.`, `2.`, etc. or bullet points instead.
- When referring back to items, use "Item 1", "Point 2", etc. — never "Issue #1" or "#1".
Copy link

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new "NEVER" wording is broad enough to be read as forbidding legitimate references to GitHub issues/PRs (e.g., #6186) in review text, not just enumeration. Consider scoping this to "when enumerating findings" and (if needed) documenting how to reference real issues/PRs without triggering accidental autolinks (e.g., wrap in backticks or use "PR 6186").

Suggested change
- When referring back to items, use "Item 1", "Point 2", etc. — never "Issue #1" or "#1".
- When referring back to enumerated review items, use "Item 1", "Point 2", etc., and avoid using bare "#1", "#2", etc. as enumeration labels. Legitimate GitHub issues/PRs may still be referenced (for example, "PR 6186" or `#6186`).

Copilot uses AI. Check for mistakes.
MegaRedHand added a commit to lambdaclass/ethlambda that referenced this pull request Feb 11, 2026
pablodeymo pushed a commit to lambdaclass/ethlambda that referenced this pull request Feb 11, 2026
See lambdaclass/ethrex#6187 for more context.

Review agents make use of `#N` for numbering comments, which triggers
GitHub references, linking all reviewed PRs with links to the first PRs
of the repo (see #1 for an example).
@avilagaston9 avilagaston9 added this pull request to the merge queue Feb 12, 2026
Merged via the queue into main with commit 728dc7d Feb 12, 2026
80 of 82 checks passed
@avilagaston9 avilagaston9 deleted the fix/ai-review-enumeration branch February 12, 2026 13:00
@github-project-automation github-project-automation bot moved this from In Review to Done in ethrex_l1 Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

L1 Ethereum client

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants