CI Python formatting is confusing

[ehuss]

When there is a python formatting error in CI, it prints the following confusing message:

python formatting does not match! Printing diff:
error: unexpected argument '--check' found

  tip: to pass '--check' as a value, use '-- --check'

Usage: ruff check <FILES|--fix|--no-fix|--unsafe-fixes|--no-unsafe-fixes|--show-source|--no-show-source|--show-fixes|--no-show-fixes|--diff|--watch|--fix-only|--no-fix-only|--ignore-noqa|--output-format <OUTPUT_FORMAT>|--output-file <OUTPUT_FILE>|--target-version <TARGET_VERSION>|--preview|--no-preview|--select <RULE_CODE>|--ignore <RULE_CODE>|--extend-select <RULE_CODE>|--extend-ignore <RULE_CODE>|--per-file-ignores <PER_FILE_IGNORES>|--extend-per-file-ignores <EXTEND_PER_FILE_IGNORES>|--exclude <FILE_PATTERN>|--extend-exclude <FILE_PATTERN>|--fixable <RULE_CODE>|--unfixable <RULE_CODE>|--extend-fixable <RULE_CODE>|--extend-unfixable <RULE_CODE>|--respect-gitignore|--no-respect-gitignore|--force-exclude|--no-force-exclude|--line-length <LINE_LENGTH>|--dummy-variable-rgx <DUMMY_VARIABLE_RGX>|--no-cache|--cache-dir <CACHE_DIR>|--stdin-filename <STDIN_FILENAME>|--extension <EXTENSION>|--exit-zero|--exit-non-zero-on-fix|--statistics|--add-noqa|--show-files|--show-settings|--ecosystem-ci>

Some issues:

• The diff is obviously not displayed.
• There's no information on how to check this locally.
• I can't find any documentation in the rustc-dev-guide about this.
• Setting this up locally seems hard:
  • Dig through CI scripts until you find the magic incantation
  • ./x test tidy --extra-checks=py returns an error about virtualenv not installed
  • The error message is wrong for my environment, it says python3.11 but I have python3.12
  • python3 -m pip install virtualenv returns an error about an externally-managed-environment
  • brew install virtualenv seemed to do something
  • Wait, no, tidy is right about python3.11, but that's not what I use. python3.11 -m pip install virtualenv does something
  • ./x test tidy --extra-checks=py fails with something about ruff not being installed
  • brew install ruff seemed to do something
  • ./x test tidy --extra-checks=py now seems to work 🎉 but it does not show me the diff 😦
  • ./x test tidy --extra-checks=py:fmt --bless updated the file

[onur-ozkan]

cc @Kobzol

[Kobzol]

Could you please send a link to the workflow where the formatting error was reported? Locally it shows me this:

checking python file formatting
Would reformat: src/ci/github-actions/ci.py
rerun tidy with `--extra-checks=py:fmt --bless` to reformat Python code

[Kobzol]

Ah, TIDY_PRINT_DIFF=1 was needed to replicate. I will investigate and send a PR with a fix.

[Kobzol]

#135950 should fix the mentioned issues. I documented the Python checks in the rustc dev guide, fixed the diff problem, and also improved the creation of virtual environments (and its error messages :) ).