Use human-readable names in CLI output by ntBre · Pull Request #25937 · astral-sh/ruff

ntBre · 2026-06-12T22:56:55Z

Summary

This PR updates all of our CLI output formats to use human-readable names instead of rule codes in preview.

Test Plan

The first commit adds a new CLI test and then each following commit updates one output format and the associated snapshots.

The last commit also updates the ecosystem check to normalize rule names back to codes to avoid a huge diff that causes the check to time out. Alternatively, we may just want to accept the time out on this PR so that we can see the un-normalized report in the future.

astral-sh-bot · 2026-06-12T22:59:34Z

Memory usage report

Memory usage unchanged ✅

astral-sh-bot · 2026-06-12T23:01:15Z

`ecosystem-analyzer` results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

codspeed-hq · 2026-06-15T17:20:30Z

Merging this PR will not alter performance

✅ 131 untouched benchmarks

_{Comparing brent/human-readable-diagnostics (79b2600) with main (c540576)}

astral-sh-bot · 2026-06-15T17:30:31Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

ntBre · 2026-06-15T18:05:37Z

+        fib.py:1:8: unused-import: [*] `os` imported but unused
+        fib.py:6:5: unused-variable: [*] Local variable `x` is assigned to but never used


I think these cases look kind of bad, but they should not be reachable in Ruff. We currently disable hide_severity in preview too, so all of the real Ruff snapshots should look like error[rule-code] [*], as seen above.

I wonder if we should revert the severity change. Given that its incomplete and we don't plan on shipping it anytime soon. I'd also say that it's important that the rendering without severity looks good, because that's most likely what we'll ship with human readable names.

Hmm, I guess I wasn't thinking that we'd stabilize human-readable names before severities, but I guess it could make sense. I also like how error[rule-code] looks, potentially even if the severity is always error. But I'm happy to revert it if you think that's best.

For this specific case, I think the main options are something like:

fib.py:1:8: unused-import: [*] `os` imported but unused fib.py:1:8: unused-import [*] `os` imported but unused fib.py:1:8: unused-import [*]: `os` imported but unused fib.py:1:8: unused-import[*]: `os` imported but unused # no fix indicator fib.py:1:4: undefined-name: Undefined name `a`

Of those, the current one is probably my favorite, despite calling it "kind of bad," because it aligns with the no-fix case, at least.

MichaReiser · 2026-06-16T08:26:21Z

+        } else if config.preview {
+            // In preview, Ruff shows both the severity and the human-readable name, so we don't
+            // need a colon.
+            diag.id().to_string()
        } else {


I think we should revert the severity change for now and re-apply it if we decide to ship severities. For now, we should assume that we ship human readable names without warning severities (which is what we defined in our plan)

MichaReiser

Thanks for splitting the PR into many different commits. I found it very useful when reviewing.

I think we should revert the severity change, either before or after this PR, because the most likely outcome is that we ship human readable names without severity.

We also need to review the warn and warn_user_once calls (e.g. warn_incompatible_formatter_settings). But we can do this in a separate PR

MichaReiser · 2026-06-16T08:27:29Z

 [
  {
    "cell": null,
-    "code": "F401",


What's the reason for removing code? Can't we emit both and make code optional instead?

No real reason. I also considered emitting both but narrowly opted for this version. I'll switch to keeping both and allow code to be missing in preview.

MichaReiser · 2026-06-16T09:27:21Z

+@cache
+def rule_name_to_code(executable: Path) -> dict[str, str]:
+    rules = json.loads(
+        check_output(
+            [executable, "rule", "--all", "--output-format", "json"],
+            encoding="utf8",
+        )
+    )
+    return {rule["name"]: rule["code"] for rule in rules}
+
+
+def normalize_rule_name(line: str, rule_names: dict[str, str]) -> str:
+    match = CHECK_DIAGNOSTIC_LINE_RE.match(line)
+    if match is None or (code := rule_names.get(match["code"])) is None:
+        return line
+
+    start, end = match.span("code")
+    return f"{line[:start]}{code}{line[end:]}"


Given that there will likely be rules without code in the future, it's probably better if the normalization here goes into the opposite direction. That is, we normalize codes to human readable names.

MichaReiser · 2026-06-16T09:28:05Z

        comparison_task.result(),
    )

+    if options.preview:


Can we always run the normalization and make normalize_rule_name forgiving in that it returns the original code if there's no match (which I think it already does?)

Yeah, it's just a bit slower always to run it, so I was trying to keep it narrowly scoped. When we stabilize this, we could remove the normalization and just match on human-readable names again.

We could also just not normalize at all and accept the ecosystem check failing on this PR. Once it's the baseline this won't be necessary either. (Until the stabilization PR at which point we'll hit the same one-time issue on the stable ecosystem check)

Maybe keep for now and remove it in a follow up PR? That gives us the guarantee that Ruff still emits the same errors (and the ecosystem check doesn't filter them out)

Sounds good! I'll just leave the script as-is then and follow up immediately dropping the normalization.

Summary -- Reverts the name normalization part of #25937 now that human-readable names are also present in the baseline for future ecosystem check runs. Test Plan -- CI on this PR

Summary -- This reverts #23845 since we're now prioritizing the human-readable names and category work before getting back to severity. Test Plan -- Existing tests updated following #25937 I said I didn't love diagnostics like this: ``` fib.py:1:8: unused-import: [*] `os` imported but unused ``` with both a human-readable name and a fix indicator on that PR, but I think it's not actually that bad, and I haven't thought of a better alternative, so I think it's fine for now.

Summary -- This reverts #23845 since we're now prioritizing the human-readable names and category work before getting back to severity. I had to revert (the revert of) the changes to the ecosystem check script and then add another normalization pass again to get CI to pass. I can follow up immediately to revert these changes once this is the baseline. Test Plan -- Existing tests updated following #25937 I said I didn't love diagnostics like this: ``` fib.py:1:8: unused-import: [*] `os` imported but unused ``` with both a human-readable name and a fix indicator on that PR, but I think it's not actually that bad, and I haven't thought of a better alternative, so I think it's fine for now.

## Summary This PR follows up on #25937 to display human-readable names in the LSP and playground too, when preview is enabled. ## Test Plan In VS Code with `preview = true` in my `ruff.toml`: <img width="788" height="352" alt="Screenshot 2026-06-16 at 16 28 50" src="https://github.com/user-attachments/assets/9352d76c-a87a-4721-9ca0-c9b130c98452" /> with `preview = false`: <img width="794" height="318" alt="Screenshot 2026-06-16 at 16 29 13" src="https://github.com/user-attachments/assets/88a3346e-63a2-4e51-8d3b-fa71b36388b2" /> In the playground with preview enabled: <img width="730" height="333" alt="Screenshot 2026-06-16 at 16 58 15" src="https://github.com/user-attachments/assets/3df5e40b-6b7e-4832-b0ba-6b3cb3b685b0" /> and with preview disabled: <img width="769" height="323" alt="Screenshot 2026-06-16 at 16 51 30" src="https://github.com/user-attachments/assets/b7261c3b-512d-43bb-9dd7-ddf825773734" />

spaceone · 2026-07-09T22:29:41Z

What is the config option to revert this, so that the number based rule names are shown again?
none of the output-formats concise, full, json, json-lines, junit, grouped, github, gitlab, pylint, rdjson, azure, sarif seem to work.

ntBre · 2026-07-09T22:34:35Z

The config option to revert this is currently preview. The stable behavior continues to be showing rule codes. However, we're also tracking this kind of feedback in #26320.

ntBre added preview Related to preview mode features diagnostics Related to reporting of diagnostics. labels Jun 12, 2026

ntBre added 3 commits June 15, 2026 12:02

add baseline test cases

9aa7f80

full

884784e

concise

1bcc872

ntBre force-pushed the brent/human-readable-diagnostics branch 2 times, most recently from 6dcd77d to 5ccbee9 Compare June 15, 2026 17:19

ntBre commented Jun 15, 2026

View reviewed changes

ntBre added 11 commits June 15, 2026 14:28

json and json_lines

d6be154

junit

167f884

grouped

8f3771d

github

aef2faf

gitlab

2b81c85

pylint

49fedd6

rdjson

92a37dc

azure

cbb219a

sarif

4f0dc72

delete allow(dead_code)

42b02a0

normalize preview diagnostics in ecosystem check

0d34aae

ntBre force-pushed the brent/human-readable-diagnostics branch from 5ccbee9 to 0d34aae Compare June 15, 2026 18:28

ntBre marked this pull request as ready for review June 15, 2026 19:03

ntBre requested a review from a team as a code owner June 15, 2026 19:03

astral-sh-bot Bot requested a review from charliermarsh June 15, 2026 19:03

ntBre requested review from MichaReiser and removed request for a team and charliermarsh June 15, 2026 19:03

MichaReiser reviewed Jun 16, 2026

View reviewed changes

MichaReiser approved these changes Jun 16, 2026

View reviewed changes

json: preserve code field

79b2600

ntBre merged commit a9dce57 into main Jun 16, 2026
59 checks passed

ntBre deleted the brent/human-readable-diagnostics branch June 16, 2026 14:38

ntBre mentioned this pull request Jun 16, 2026

Revert rule name normalization changes in preview #26044

Merged

ntBre mentioned this pull request Jun 16, 2026

Revert "Display output severity in preview" #26050

Merged

ntBre mentioned this pull request Jun 16, 2026

Use human-readable names in LSP and playground diagnostics #26058

Merged

BrewTestBot mentioned this pull request Jun 18, 2026

ruff 0.15.18 Homebrew/homebrew-core#288694

Merged

NorthLake mentioned this pull request Jun 24, 2026

Update JSON schema for ruff LSP InSyncWithFoo/ryecharm#236

Closed

		fib.py:1:8: unused-import: [*] `os` imported but unused
		fib.py:6:5: unused-variable: [*] Local variable `x` is assigned to but never used

Uh oh!

Conversation

ntBre commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

astral-sh-bot Bot commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Memory usage report

Uh oh!

astral-sh-bot Bot commented Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ecosystem-analyzer results

Uh oh!

codspeed-hq Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

astral-sh-bot Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaReiser left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaReiser Jun 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

spaceone commented Jul 9, 2026

Uh oh!

ntBre commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ntBre commented Jun 12, 2026 •

edited

Loading

astral-sh-bot Bot commented Jun 12, 2026 •

edited

Loading

astral-sh-bot Bot commented Jun 12, 2026 •

edited

Loading

`ecosystem-analyzer` results

codspeed-hq Bot commented Jun 15, 2026 •

edited

Loading

astral-sh-bot Bot commented Jun 15, 2026 •

edited

Loading

`ruff-ecosystem` results

MichaReiser left a comment •

edited

Loading

MichaReiser Jun 16, 2026 •

edited

Loading