(🎁) Preserve crlf line endings in blackd #3257

KotlinIsland · 2022-09-05T08:29:04Z

Description

Added CRLF line ending preservation to blackd, perfect for gorgeous looks 🚀

Checklist - did you ... (know?)

Add a CHANGELOG entry if necessary?
Add / update tests if necessary?
Add new / update outdated documentation?

resolves (🐞) Windows line endings not preserved with blackd #3255

github-actions · 2022-09-05T08:51:27Z

diff-shades reports zero changes comparing this PR (1c1647a) to main (383b228).

What is this? | Workflow run | diff-shades documentation

cooperlees

Thanks - LGTM, but just asking some questions for my understanding (prob my missing knowledge here).

cooperlees · 2022-09-05T12:38:26Z

src/blackd/__init__.py

+        # Preserve CRLF newlines
+        if req_str[req_str.find("\n") - 1] == "\r":
+            formatted_str = formatted_str.replace("\n", "\r\n")


So, am I right in understanding you're only checking the end of the entire posted data? Are intermediate CRLF (\r\n) are already being preserved? Was blackd only stripping at the end of the returned data?

If so, maybe make the comment say that # Preserve CRLF newline at the end of blackd returned str

I'm checking the first \n to see if it was preceded by an \r. If so, replace all \n with \r\n

cooperlees · 2022-09-05T12:59:40Z

tests/test_blackd.py

+
+        @unittest_run_loop
+        async def test_preserve_newline(self) -> None:
+            for data in (b"c\r\nc\r\n", b"l\nl\n"):


Why doesn't this need to be AST parsable data? Do we mock that somewhere?

This code looks syntactically valid to me.

JelleZijlstra · 2022-09-05T13:29:17Z

src/blackd/__init__.py

@@ -133,6 +133,13 @@ async def handle(request: web.Request, executor: Executor) -> web.Response:
            executor, partial(black.format_file_contents, req_str, fast=fast, mode=mode)
        )

+        # Preserve CRLF newlines
+        if req_str[req_str.find("\n") - 1] == "\r":


Is it possible to mix \n and \r\n newlines in a single file? If so, this could change the contents of string literals with \n newlines in them.

I think this is what normal black does currently to files, although I'm not certain.

Yeah, I just checked with a mixed file in normal black, and it uses the first newline to rewrite the rest of the file.

I think formatting while preserving mixed newlines would be extremely difficult and probably a giant waste of effort.

felix-hilden · 2022-09-11T14:01:58Z

tests/test_blackd.py

+
+    @unittest_run_loop
+    async def test_preserve_newline(self) -> None:
+        for data in (b"c\r\nc\r\n", b"l\nl\n"):


Could we also test:

first is only n, second is rn, and the corresponding expectation

the other way around

I'm not sure what you mean, could you provide an example please.

"a\na\r\n"

Like this?

If so, should the same test be added to normal black(tbh it might already), and should we document the behaviour as well?

Yep exactly, sorry for the confusion 😅 and a\r\na\n At a quick glance, I couldn't find a test with mixed line endings. Only ones like the one you wrote here. But it would be nice, and a short mention on documentation couldn't hurt either, agreed 👍 Just to be explicit.

I have made a sus discovery, black will not normalize newlines if the rest of the file is unchanged. Should I make blackd behave exactly the same? (I think always normalizing is arguably better).

I added tests for both blackd and black, and updated the docs.

zsol

lgtm.
It's unfortunate that it's not practical to share the newline preservation with black

* Drop upper version bounds on dependencies (GH-2718) They mostly cause unnecessary trouble. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Primer: exclude crashing sqlalchemy file for now (GH-2735) Until we can properly look into and fix it. -> https://github.com/psf/black/issues/2734 * Documentation: include Wing IDE 8 integrations (GH-2733) Wing IDE 8 now supports autoformatting w/ Black natively 🎉 Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Improve CLI reference wording (#2753) * Stubs: preserve blank line between attributes and methods (#2736) * Action: Support running in a docker container (#2748) see: https://github.com/actions/runner/issues/716 * Fix call patterns that contain as-expression on the kwargs (#2749) * Remove Python 2 support (#2740) *blib2to3's support was left untouched because: 1) I don't want to touch parsing machinery, and 2) it'll allow us to provide a more useful error message if someone does try to format Python 2 code. * Enhance `--verbose` (#2526) Black would now echo the location that it determined as the root path for the project if `--verbose` is enabled by the user, according to which it chooses the SRC paths, i.e. the absolute path of the project is `{root}/{src}`. Closes #1880 * Speed up new backtracking parser (#2728) * Fix handling of standalone match/case with newlines/comments (#2760) Resolves #2759 * Change git url for pip installation in README (#2761) * Change git url for pip installation in README Unauthenticated git protocol was disabled recently by Github and should not be used anymore. https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git * Update CHANGES.md * Change installation url to comply with git security change (#2765) Co-authored-by: Jeffrey Lazar <jlazar@MacBook-Pro-2.local> * don't expect changes on poetry (#2769) They just made themselves ESP-compliant in https://github.com/python-poetry/poetry/commit/ecb030e1f0b7c13cc11971f00ee5012e82a892bc * Normalise string prefix order (#2297) Closes #2171 * Don't make redundant copies of the DFA (#2763) * Added decent coloring (#2712) * CI: add diff-shades integration (#2725) Hopefully this makes it much easier to gauge the impacts of future changes! * Dont require typing-extensions in 3.10 (GH-2772) 3.10 ships with TypeGuard which is the newest feature we need. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * [trivial] Use proper test cases on `unittest` (#2775) * Bump sphinx from 4.3.2 to 4.4.0 in /docs (#2776) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.2 to 4.4.0. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.3.2...v4.4.0) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix typo in diff_shades.yml workflow (#2778) * Create --preview CLI flag (#2752) * Fix and speedup diff-shades integration (#2773) * Deprecate ESP and move the functionality under --preview (#2789) * Hint at likely cause of ast parsing failure in error message (#2786) Co-authored-by: Batuhan Taskaya <isidentical@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Add support for custom python cell magics (#2744) Fixes #2742. This PR adds the ability to configure additional python cell magics. This will allow formatting cells in Jupyter Notebooks that are using custom (python) magics. * Set `click` lower bound to `8.0.0` (#2791) Closes #2774 * add wind technology software projects using black (#2792) * Switch to Furo (#2793) - Add Furo dependency to docs/requirements.txt - Drop a fair bit of theme configuration - Fix the toctree declarations in index.rst - Move stuff around as Furo isn't 100% compatible with Alabaster Furo was chosen as it provides excellent mobile support, user controllable light/dark theming, and is overall easier to read * Allow setting custom cache directory on all platforms (#2739) Fixes #2506 ``XDG_CACHE_HOME`` does not work on Windows. To allow for users to set a custom cache directory on all systems I added a new environment variable ``BLACK_CACHE_DIR`` to set the cache directory. The default remains the same so users will only notice a change if that environment variable is set. The specific use case I have for this is I need to run black on in different processes at the same time. There is a race condition with the cache pickle file that made this rather difficult. A custom cache directory will remove the race condition. I created ``get_cache_dir`` function in order to test the logic. This is only used to set the ``CACHE_DIR`` constant. * Mark Felix and Batuhan as maintainers (#2794) Y'all deserve it :) * Refactor logic for stub empty lines (#2796) This PR is intended to have no change to semantics. This is in preparation for #2784 which will likely introduce more logic that depends on `current_line.depth`. Inlining the subtraction gets rid of offsetting and makes it much easier to see what the result will be. * Mention "skip news" label in CHANGELOG action (#2797) Co-authored-by: hauntsaninja <> * Enable pattern matching by default (#2758) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Allow blackd to be run as a package (#2800) * Remove Beta mentions in README + Docs (#2801) - State we're now stable and that we'll uphold our formatting changes as per policy - Link to The Black Style doc. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Use `magic_trailing_comma` and `preview` for `FileMode` in `fuzz` (#2802) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Make SRC or code mandatory and mutually exclusive (#2360) (#2804) Closes #2360: I'd like to make passing SRC or `--code` mandatory and the arguments mutually exclusive. This will change our (partially already broken) promises of CLI behavior, but I'll comment below. * Hug power operators if its operands are "simple" (#2726) Since power operators almost always have the highest binding power in expressions, it's often more readable to hug it with its operands. The main exception to this is when its operands are non-trivial in which case the power operator will not hug, the rule for this is the following: > For power ops, an operand is considered "simple" if it's only a NAME, numeric CONSTANT, or attribute access (chained attribute access is allowed), with or without a preceding unary operator. Fixes GH-538. Closes GH-2095. diff-shades results: https://gist.github.com/ichard26/ca6c6ad4bd1de5152d95418c8645354b Co-authored-by: Diego <dpalma@evernote.com> Co-authored-by: Felix Hildén <felix.hilden@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * properly run ourselves twice (#2807) The previous run-twice logic only affected the stability checks but not the output. Now, we actually output the twice-formatted code. * Fix crash on some power hugging cases (#2806) Found by the fuzzer. Repro case: python -m black -c 'importA;()<<0**0#' * black-primer: stop running it (#2809) At the moment, it's just a source of spurious CI failures and busywork updating the configuration file. Unlike diff-shades, it is run across many different platforms and Python versions, but that doesn't seem essential. We already run unit tests across platforms and versions. I chose to leave the code around for now in case somebody is using it, but CI will no longer run it. * more trailing comma tests (#2810) * Use parentheses on method access on float and int literals (#2799) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Felix Hildén <felix.hilden@gmail.com> * Tests for unicode identifiers (#2816) * reorganize release notes for 22.1.0 (#2790) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Elaborate on Python support policy (#2819) * Treat blank lines in stubs the same inside top-level `if` statements (#2820) * torture test (#2815) Fixes #2651. Fixes #2754. Fixes #2518. Fixes #2321. This adds a test that lists a number of cases of unstable formatting that we have seen in the issue tracker. Checking it in will ensure that we don't regress on these cases. * Formalise style preference description (#2818) Closes #1256: I reworded our style docs to be more explicit about the style we're aiming for and how it is changed (or isn't). * Fix arithmetic stability issue (#2817) It turns out "simple_stmt" isn't that simple: it can contain multiple statements separated by semicolons. Invisible parenthesis logic for arithmetic expressions only looked at the first child of simple_stmt. This causes instability in the presence of semicolons, since the next run through the statement following the semicolon will be the first child of another simple_stmt. I believe this along with #2572 fix the known stability issues. * Fix instability due to trailing comma logic (#2572) It was causing stability issues because the first pass could cause a "magic trailing comma" to appear, meaning that the second pass might get a different result. It's not critical. Some things format differently (with extra parens) * Add a test case to torture.py (#2822) Co-authored-by: hauntsaninja <> * Update classifiers to reflect stable (#2823) * Remove test suite from setup.py (#2824) We no longer use it * Fix changelog entries in the wrong release (#2825) * Fix changelog entries in the wrong release (#2825) * Prepare docs for release 22.1.0 (GH-2826) * Adjust `--preview` documentation (#2833) * Exclude __pypackages__ by default (GH-2836) PDM uses this as part of not-accepted-yet PEP 582. * Soft comparison of --required-version (#2832) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Felix Hildén <felix.hilden@gmail.com> * release process: formalize the changelog template (#2837) I did this manually for the last few releases and I think it's going to be helpful in the future too. Unfortunately this adds a little more work during the release (sorry @cooperlees). This change will also improve the merge conflict situation a bit, because changes to different sections won't merge conflict. For the last release, the sections were in a kind of random order. In the template I put highlights and "Style" first because they're most important to users, and alphabetized the rest. * Surface links to Stability Policy (GH-2848) * Isolate command line tests from user-level config (#2851) * Update description for GitHub Action `options:` argument (GH-2858) It was missing --diff as one of the default arguments passed. * Create indentation FAQ entry (#2855) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Bump sphinx-copybutton from 0.4.0 to 0.5.0 in /docs (#2871) Bumps [sphinx-copybutton](https://github.com/executablebooks/sphinx-copybutton) from 0.4.0 to 0.5.0. - [Release notes](https://github.com/executablebooks/sphinx-copybutton/releases) - [Changelog](https://github.com/executablebooks/sphinx-copybutton/blob/master/CHANGELOG.md) - [Commits](https://github.com/executablebooks/sphinx-copybutton/compare/v0.4.0...v0.5.0) --- updated-dependencies: - dependency-name: sphinx-copybutton dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add Django in 'used by' section in Readme (#2875) * Add Django in 'used by' section in Readme * Fix Readme issue * Avoid crashing when the user has no homedir (#2814) * Order the disabled error codes for pylint (GH-2870) Just make them alphabetical. * Fix typo in file_collection_and_discovery.md (GH-2860) "you your" -> "your" Co-authored-by: Felix Hildén <felix.hilden@gmail.com> * Isolate command line tests for notebooks from user-level config (#2854) * correct Vim integration code (#2853) - use `Black` directly: the commands an autocommand runs are Ex commands, so no execute or colon is necessary. - use an `augroup` (best practice) to prevent duplicate autocommands from hindering performance. * Add special config verbose log case when black is using user-level config (#2861) * Bump furo from 2022.1.2 to 2022.2.14.1 in /docs (GH-2892) Bumps [furo](https://github.com/pradyunsg/furo) from 2022.1.2 to 2022.2.14.1. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2022.01.02...2022.02.14.1) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Format ourselves in preview mode (#2889) * separate CHANGELOG section for preview style (#2890) * fix new formatting issue (#2895) Race between #2889 and another PR. * README: fix "Pragmatism" link target (#2901) Fixes #2897 * replace md5 with sha256 (#2905) MD5 is unavailable on systems with active FIPS mode. That makes black crash when run on such systems. * Move test for g:load_black to improve plugin performance (GH-2896) If a vim/neovim user wishes to suppress loading the vim plugin by setting g:load_black in their VIMRC (for me, Arch linux automatically adds the plugin to Neovim's RTP, even though I'm not using it), the current location of the test comes after a call to has('python3'). This adds, in my tests, between 35 and 45 ms to Vim load time (which I know isn't a lot but it's also unnecessary). Moving the call to `exists('g:load_black')` to before the call to `has('python3')` removes this unnecessary test and speeds up loading. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Bump furo from 2022.2.14.1 to 2022.3.4 in /docs (#2906) Bumps [furo](https://github.com/pradyunsg/furo) from 2022.2.14.1 to 2022.3.4. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2022.02.14.1...2022.03.04) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Allow `for`'s target expression to be starred (#2879) Fixes #2878 * Bump actions/checkout from 2 to 3 (#2909) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 2 to 3 (#2908) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 3. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cooper Lees <me@cooperlees.com> * Fix handling of Windows junctions in normalize_path_maybe_ignore (#2904) Fixes #2569 * Use tomllib on Python 3.11 (#2903) * Bump mypy, flake8, and pre-commit-hooks in pre-commit (GH-2922) * Farewell black-primer, it was nice knowing you (#2924) Enjoy your retirement at https://github.com/cooperlees/black-primer * Remove power hugging formatting from preview (#2928) It is falsely placed in preview features and always formats the power operators, it was added in #2789 but there is no check for formatting added along with it. * Update pylint config docs (#2931) * dont skip formatting #%% (#2919) Fixes #2588 * stub style: remove some possible future changes (#2940) Fixes #2938. All of these suggested future changes are out of scope for an autoformatter and should be handled by a linter instead. * Github now supports .git-blame-ignore-revs (GH-2948) It's in beta. https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view * Avoid magic-trailing-comma in single-element subscripts (#2942) Closes #2918. * Remove unnecessary parentheses from tuple unpacking in `for` loops (#2945) * Resolve new flake8-bugbear errors (B020) (GH-2950) Fixes a couple places where we were using the same variable name as we are iterating over. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Remove unnecessary parentheses from `except` clauses (#2939) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Enforce no formatting changes for PRs via CI (GH-2951) Now PRs will run two diff-shades jobs, "preview-changes" which formats all projects with preview=True, and "assert-no-changes" which formats all projects with preview=False. The latter also fails if any changes were made. Pushes to main will only run "preview-changes" Also the workflow_dispatch feature was dropped since it was complicating everything for little gain. * Bump sphinx from 4.4.0 to 4.5.0 in /docs (GH-2959) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.4.0 to 4.5.0. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.4.0...v4.5.0) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix _unicodefun patch code for Click 8.1.0 (#2966) Fixes #2964 * Prepare release 22.3.0 (#2968) * Bump actions/cache from 2.1.7 to 3 (GH-2962) Bumps [actions/cache](https://github.com/actions/cache) from 2.1.7 to 3. - [Release notes](https://github.com/actions/cache/releases) - [Commits](https://github.com/actions/cache/compare/v2.1.7...v3) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Keep tests working w/ upcoming aiohttp 4.0.0 (#2974) aiohttp.test_utils.unittest_run_loop was deprecated since aiohttp 3.8 and aiohttp 4 (which isn't a thing quite yet) removes it. To maintain compatibility with the full range of versions we declare to support, test_blackd.py will now define a no-op replacement if it can't be imported. Also, mypy is painfully slow to use without a cache, let's reenable it. * Convert `index.rst` and `license.rst` to markdown (#2852) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Add # type: ignore for click._unicodefun import (#2981) * Remove click pin in diff-shades workflow (#2979) Click 8.1.1 was released with a fix for pallets/click#2227. * Bump peter-evans/find-comment from 1.3.0 to 2 (#2960) Bumps [peter-evans/find-comment](https://github.com/peter-evans/find-comment) from 1.3.0 to 2. - [Release notes](https://github.com/peter-evans/find-comment/releases) - [Commits](https://github.com/peter-evans/find-comment/compare/d2dae40ed151c634e4189471272b57e76ec19ba8...1769778a0c5bd330272d749d12c036d65e70d39d) --- updated-dependencies: - dependency-name: peter-evans/find-comment dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump peter-evans/create-or-update-comment from 1.4.5 to 2 (#2961) Bumps [peter-evans/create-or-update-comment](https://github.com/peter-evans/create-or-update-comment) from 1.4.5 to 2. - [Release notes](https://github.com/peter-evans/create-or-update-comment/releases) - [Commits](https://github.com/peter-evans/create-or-update-comment/compare/a35cf36e5301d70b76f316e867e7788a55a31dae...c9fcb64660bc90ec1cc535646af190c992007c32) --- updated-dependencies: - dependency-name: peter-evans/create-or-update-comment dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * try-except tomllib import (#2987) See #2965 I left the version check in place because mypy doesn't generally like try-excepted imports. * Fix broken link in README.md (#2989) Broken when we converted some more RST docs to MyST * Remove unnecessary parentheses from `with` statements (#2926) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Update test_black.shhh_click test for click 8+ (#2993) The 8.0.x series renamed its "die on LANG=C" function and the 8.1.x series straight up deleted it. Unfortunately this makes this test type check cleanly hard, so we'll just lint with click 8.1+ (the pre-commit hook configuration was changed mostly to just evict any now unsupported mypy environments) * Update FAQ: Mention formatting of custom jupyter cell magic (#2982) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Top PyPI Packages: Use 30-days data, 365 is no longer available (#2995) * Output python version and implementation as part of `--version` flag (#2997) Example: black, 22.1.1.dev56+g421383d.d20220405 (compiled: no) Python (CPython) 3.9.12 Co-authored-by: Batuhan Taskaya <isidentical@gmail.com> * Better manage return annotation brackets (#2990) Allows us to better control placement of return annotations by: a) removing redundant parens b) moves very long type annotations onto their own line Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Remove redundant parentheses around awaited coroutines/tasks (#2991) This is a tricky one as await is technically an expression and therefore in certain situations requires brackets for operator precedence. However, the vast majority of await usage is just await some_coroutine(...) and similar in format to return statements. Therefore this PR removes redundant parens around these await expressions. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Correctly handle fmt: skip comments without internal spaces (#2970) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Explain our use of mypyc in the FAQ (#3002) I realized we don't have a FAQ entry about this, let's change that so compiled: yes/no doesn't surprise as many people :) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Bump actions/upload-artifact from 2 to 3 (#3004) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Quote "black[jupyter]" in README.md (#3007) * Bump furo from 2022.3.4 to 2022.4.7 in /docs (#3003) Bumps [furo](https://github.com/pradyunsg/furo) from 2022.3.4 to 2022.4.7. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2022.03.04...2022.04.07) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Quote black[jupyter] and black[d] in installation docs (#3006) We just got someone on Discord who was confused because the command as written caused their shell to try to do command expansion. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Make ipynb tests compatible with ipython 8.3.0+ (#3008) * Support 3.11 / PEP 654 syntax (#3016) * Updated Black Docker Hub link in docs (#3023) Fixes #3022 * Fix strtobool function (#3025) * Fix strtobool function for vim plugin * Update CHANGES.md Co-authored-by: Cooper Lees <me@cooperlees.com> * Stop pinning lark-parser (#3041) - Latest version works more Test: `tox -e fuzz` * Bump myst-parser from 0.16.1 to 0.17.2 in /docs (#3019) Bumps [myst-parser](https://github.com/executablebooks/MyST-Parser) from 0.16.1 to 0.17.2. - [Release notes](https://github.com/executablebooks/MyST-Parser/releases) - [Changelog](https://github.com/executablebooks/MyST-Parser/blob/master/CHANGELOG.md) - [Commits](https://github.com/executablebooks/MyST-Parser/compare/v0.16.1...v0.17.2) --- updated-dependencies: - dependency-name: myst-parser dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Set permissions for GitHub actions (#3043) Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much. - Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs [Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/) Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com> * Move imports of `ThreadPoolExecutor` into `reformat_many()`, allowing Black-in-the-browser (#3046) This is a slight perf win for use-cases that don't invoke `reformat_many()`, but more importantly to me today it means I can use Black in pyscript. * Docs: clarify fmt:on/off requirements (#2985) (#3048) * Put closing quote on a separate line if docstring is too long (#3044) Fixes #1632 Co-authored-by: Felix Hildén <felix.hilden@gmail.com> * Read simple data cases automatically (#3034) Co-authored-by: Felix Hildén <felix.hilden@gmail.com> * Bump docker/setup-qemu-action from 1 to 2 (#3056) Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 1 to 2. - [Release notes](https://github.com/docker/setup-qemu-action/releases) - [Commits](https://github.com/docker/setup-qemu-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/setup-qemu-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/build-push-action from 2 to 3 (#3057) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2 to 3. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v2...v3) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/login-action from 1 to 2 (#3059) Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/setup-buildx-action from 1 to 2 (#3058) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 1 to 2. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Remove hard coded test cases (#3062) * Document new Microsoft Black Formatter extension for VSCode (#3063) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Add more examples to exclude files in addition to the defaults (#3070) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Implement support for PEP 646 (#3071) * Add script to ease migration to black (#3038) * Add script to ease migration to black * Update CHANGES.md Co-authored-by: Cooper Lees <me@cooperlees.com> * Fix minor typo (#3096) * Bump pre-commit/action from 2.0.3 to 3.0.0 (#3108) Bumps [pre-commit/action](https://github.com/pre-commit/action) from 2.0.3 to 3.0.0. - [Release notes](https://github.com/pre-commit/action/releases) - [Commits](https://github.com/pre-commit/action/compare/v2.0.3...v3.0.0) --- updated-dependencies: - dependency-name: pre-commit/action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Remove newline after code block open (#3035) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Update documentation dependencies (#3118) Furo, myst-parser, and Sphinx (had to pin docutils due to sphinx breakage) * Use is_number_token instead of assertion (#3069) * Bump actions/setup-python from 3 to 4 (#3121) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v3...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Test run black on self (#3114) * Add run_self environment in tox * Add run_self task as part of the lint CI flow * Remove hard coded sources list * Remove black from pre-commit Co-authored-by: Cooper Lees <me@cooperlees.com> * Replace link to Requests documentation (#3125) * Bump sphinx from 5.0.1 to 5.0.2 in /docs (#3128) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.0.1 to 5.0.2. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/5.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v5.0.1...v5.0.2) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Only call get_future_imports when needed (#3135) * Bump furo from 2022.6.4.1 to 2022.6.21 in /docs (#3138) Bumps [furo](https://github.com/pradyunsg/furo) from 2022.6.4.1 to 2022.6.21. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2022.06.04.1...2022.06.21) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update preview style docs to include recent changes (#3136) Covers GH-2926, GH-2990, GH-2991, and GH-3035. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Prepare docs for release 22.6.0 (#3139) * Fix typo in CHANGES.md (#3142) * Use RTD's new build process and config (#3149) See the deprecation notice: https://docs.readthedocs.io/en/stable/config-file/v2.html#python-version * Stability policy: permit exceptional changes for unformatted code (#3155) * Recommend using BlackConnect in IntelliJ IDEs (#3150) * Recommend using BlackConnect in IntelliJ IDEs * IntelliJ IDEs integration docs: improve formatting * Add changelog for recommending BlackConnect * IntelliJ IDEs integration docs: improve formatting * Apply suggestions from code review Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Fix indentation * Apply italic to Black name Consequently with other places in the document * Move CHANGELOG entry to Unreleased section * IntelliJ IDEs integration docs: bring back a point with formatting a file * IntelliJ IDEs integration docs: fix extra whitespace and linebreak Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Copy over comments when hugging power ops (#2874) Otherwise they'd be deleted which was a regression in 22.1.0 (oops! my bad!). Also type comments are now tracked in the AST safety check on all compatible platforms to error out if this happens again. Overall the line rewriting code has been rewritten to do "the right thing (tm)", I hope this fixes other potential bugs in the code (fwiw I got to drop the bugfix in blib2to3.pytree.Leaf.clone since now bracket metadata is properly copied over). Fixes #2873 * Don't (ever) put a single-char closing docstring quote on a new line (#3166) Doing so is invalid. Note this only fixes the preview style since the logic putting closing docstring quotes on their own line if they violate the line length limit is quite new. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Add warning to not run blackd publicly in docs (#3167) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Move to explicitly creating a new loop (#3164) * Move to explicitly creating a new loop - >= 3.10 add a warning that `get_event_loop` will not automatically create a loop - Move to explicit API Test: - `python3.11 -m venv --upgrade-deps /tmp/tb` - `/tmp/tb/bin/pip install -e .` - Install deps and no blackd as aiohttp + yarl can't build still with 3.11 - https://github.com/aio-libs/aiohttp/issues/6600 - `export PYTHONWARNINGS=error` ``` cooper@l33t:~/repos/black$ /tmp/tb/bin/black . All done! ✨ 🍰 ✨ 44 files left unchanged. ``` Fixes #3110 * Add to CHANGES.md * Fix a cooper typo yet again * Set default asyncio loop to our explicitly created one + unset on exit * Update CHANGES.md Fix my silly typo. Co-authored-by: Thomas Grainger <tagrain@gmail.com> Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Thomas Grainger <tagrain@gmail.com> * Actually disable docstring prefix normalization with -S + fix instability (#3168) The former was a regression I introduced a long time ago. To avoid changing the stable style too much, the regression is only fixed if --preview is enabled Annoyingly enough, as we currently always enforce a second format pass if changes were made, there's no good way to prove the existence of the docstring quote normalization instability issue. For posterity, here's one failing example: --- source +++ first pass @@ -1,7 +1,7 @@ def some_function(self): - ''''<text here> + """ '<text here> <text here, since without another non-empty line black is stable> - ''' + """ pass --- first pass +++ second pass @@ -1,7 +1,7 @@ def some_function(self): - """ '<text here> + """'<text here> <text here, since without another non-empty line black is stable> """ pass Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Fix typo in config docs for --extend-exclude (#3170) The old regex in the example was invalid and caused an error on startup. * configure strict pytest and filterwarnings=['error', ... (#3173) * configure strict pytest * ignore current warnings * Add pypy-3.8 to test matrix (#3174) * Improve warning filtering in tests (#3175) * Fix the handling of `# fmt: skip` when it's at a colon line (#3148) When the Leaf node with `# fmt: skip` is a NEWLINE inside a `suite` Node, the nodes to ignore should be from the siblings of the parent `suite` Node. There is a also a special case for the ASYNC token, where it expands to the grandparent Node where the ASYNC token is. This fixes GH-2646, GH-3126, GH-2680, GH-2421, GH-2339, and GH-2138. * Fix an infinite loop when using `# fmt: on/off` ... (#3158) ... in the middle of an expression or code block by adding a missing return. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Use underscores instead of a space in a test file's name (#3180) ... for *consistency* * Bump sphinx from 5.0.2 to 5.1.0 in /docs (#3183) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/5.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v5.0.2...v5.1.0) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add isort to linting toolchain Co-authored-by: Shivansh-007 <shivansh-007@outlook.com> * Reformat codebase with isort * Bump pre-commit hooks (#3191) * Consolidate test CI and add concurrency limits (#3189) * Vim plugin: prefix messages with "Black: " (#3194) As mentioned in GH-3185, when using Black as a Vim plugin, especially automatically on save, the plugin's messages can be confusing, as nothing indicates that they come from Black. * Remove blib2to3 grammar cache logging (#3193) As error logs are emitted often (they happen when Black's cache directory is created after blib2to3 tries to write its cache) and cause issues to be filed by users who think Black isn't working correctly. These errors are expected for now and aren't a cause for concern so let's remove them to stop worrying users (and new issues from being opened). We can improve the blib2to3 caching mechanism to write its cache at the end of a successful command line invocation later. * Add sanity check to executable CD + more (#3190) Building executables without any testing is quite sketchy, let's at least verify they won't crash on startup and format Black's own codebase. Also replaced "binaries" with "executables" since it's clearer and won't be confused with mypyc. Finally, I added colorama so all Windows users can get colour. * Move fuzz.py to scripts/ (#3199) * Bump sphinx from 5.1.0 to 5.1.1 in /docs (#3201) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.1.0 to 5.1.1. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/5.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v5.1.0...v5.1.1) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * makes install available for all users in docker image (#3202) * makes install available for all users in docker image moves the installation path from /root/.local to a virtualenv. this way we still get the lightweight multistage build without excluding non-root users. * adds changelog entry for docker-image fix A changelog entry has been added under the Integration subheader * changes dockerfile to use the venv activate script we are now using the inbuilt venv activate script, as well as explicitly mentioning the binary location in the entrypoint cmd. Co-authored-by: Nicolò <nicolo.intrieri@spinforward.it> Co-authored-by: Cooper Lees <me@cooperlees.com> * Remove invalid syntax in docstrings -S --preview test (#3205) uR is not a legal string prefix, so this test breaks (AssertionError: cannot use --safe with this file; failed to parse source file AST: invalid syntax) if changed to one in which the file is changed. I've changed the last test to have u alone, and added an R to the test above instead. * Use debug f-strings for feature detection (#3215) Fixes GH-2907. * Use --no-implicit-optional for type checking (#3220) This makes type checking PEP 484 compliant (as of 2018). mypy will change its defaults soon. See: https://github.com/python/mypy/issues/9091 https://github.com/python/mypy/pull/13401 * add preview option support for blackd (#3217) Fixes #3195 Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Port & upstream mypyc wheel build workflow (#3197) * Strip trailing commas in subscripts with -C (#3209) Fixes #2296, #3204 * Update email (#3235) This file gets scraped a lot, so create a distinct email for potential spam. * Add passing 3.11 CI by exempting blackd tests (#3234) - Had to exempt blackd tests for now due to aiohttp - Skip by using `sys.version_info` tuple - aiohttp does not compile in 3.11 yet - refer to #3230 - Add a deadsnakes ubuntu workflow to run 3.11-dev to ensure we don't regress - Have it also format ourselves Test: - `tox -e 311` Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Fix a string merging/split issue caused by standalone comments. (#3227) Fixes #2734: a standalone comment causes strings to be merged into one far too long (and requiring two passes to do so). Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Remove hacky subprocess call in action.yml (#3226) Updates action.yml to use the alternative $GITHUB_ACTION_PATH variable instead of the original ${{ github.action_path }} which caused issues with bash on the Windows runners. This removes the need for a Python subprocess to call the main.py script. * Fix misdetection of project root with `--stdin-filename` (#3216) There are a number of places this behaviour could be patched, for instance, it's quite tempting to patch it in `get_sources`. However I believe we generally have the invariant that project root contains all files we want to format, in which case it seems prudent to keep that invariant. This also improves the accuracy of the "sources to be formatted" log message with --stdin-filename. Fixes GH-3207. * Lazily import parallelized format modules `black.reformat_many` depends on a lot of slow-to-import modules. When formatting simply a single file, the time paid to import those modules is totally wasted. So I moved `black.reformat_many` and its helpers to `black.concurrency` which is now *only* imported if there's more than one file to reformat. This way, running Black over a single file is snappier Here are the numbers before and after this patch running `python -m black --version`: - interpreted: 411 ms +- 9 ms -> 342 ms +- 7 ms: 1.20x faster - compiled: 365 ms +- 15 ms -> 304 ms +- 7 ms: 1.20x faster Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com> * Load .gitignore and exclude regex at time of use Loading .gitignore and compiling the exclude regex can take more than 15ms. We shouldn't and don't need to pay this cost if we're simply formatting files given on the command line directly. I would've loved to lazily import pathspec, but the patch won't be clean until the file collection and discovery logic is refactored first. Co-authored-by: Fabio Zadrozny <fabiofz@gmail.com> * Delay worker count determination os.cpu_count() can return None (sounds like a super arcane edge case though) so the type annotation for the `workers` parameter of `black.main` is wrong. This *could* technically cause a runtime TypeError since it'd trip one of mypyc's runtime type checks so we might as well fix it. Reading the documentation (and cross-checking with the source code), you are actually allowed to pass None as `max_workers` to `concurrent.futures.ProcessPoolExecutor`. If it is None, the pool initializer will simply call os.cpu_count() [^1] (defaulting to 1 if it returns None [^2]). It'll even round down the worker count to a level that's safe for Windows. ... so theoretically we don't even need to call os.cpu_count() ourselves, but our Windows limit is 60 (unlike the stdlib's 61) and I'd prefer not accidentally reintroducing a crash on machines with many, many CPU cores. [^1]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor [^2]: https://github.com/python/cpython/blob/a372a7d65320396d44e8beb976e3a6c382963d4e/Lib/concurrent/futures/process.py#L600 * Add parens around implicit string concatenations where it increases readability (#3162) Adds parentheses around implicit string concatenations when it's inside a list, set, or tuple. Except when it's only element and there's no trailing comma. Looking at the order of the transformers here, we need to "wrap in parens" before string_split runs. So my solution is to introduce a "collaboration" between StringSplitter and StringParenWrapper where the splitter "skips" the split until the wrapper adds the parens (and then the line after the paren is split by StringSplitter) in another pass. I have also considered an alternative approach, where I tried to add a different "string paren wrapper" class, and it runs before string_split. Then I found out it requires a different do_transform implementation than StringParenWrapper.do_transform, since the later assumes it runs after the delimiter_split transform. So I stopped researching that route. Originally function calls were also included in this change, but given missing commas should usually result in a runtime error and the scary amount of changes this cause on downstream code, they were removed in later revisions. * Use strict mypy checking (#3222) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Use .gitignore files in the initial source directories (#3237) Solves https://github.com/psf/black/issues/2598 where Black wouldn't use .gitignore at folder/.gitignore if you ran `black folder` for example. Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Improve & update release process to reflect recent changes (#3242) - Formalise release cadence guidelines - Overhaul release steps to be easier to follow and more thorough - Reorder changelog template to something more sensible - Update release automation docs to reflect recent improvements (notably the addition of in-repo mypyc wheel builds) Co-authored-by: Felix Hildén <felix.hilden@gmail.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Update stable branch after publishing to PyPI (#3223) We've decided to a) convert stable back into a branch and b) to update it immediately as part of the release process. We may as well automate it. And about going back to a branch ... Git tags are not the right tool, at all[^1]. They come with the expectation that they will never change. Things will not work as expected if they do change, doubly so if they change regularly. Once you pull stable from the remote and it's copied in your local repository, no matter how many times you run git pull you'll never see it get updated automatically. Your only recourse is to delete the tag via `git tag -d stable` before pulling. This gets annoying really quickly since stable is supposed to be the solution for folks "who want to move along as Black developers deem the newest version reliable."[^2] See this comment for how this impacts users using our Vim plugin[^3]. It also affects us developers[^4]. If you have stable locally, once we cut a new release and update the stable tag, a simple `git pull` / `git fetch` will not pull down the updated stable tag. Unless you remember to delete stable before pulling, stable will become stale and useless. You can argue this is a good thing ("people should explicitly opt into updating stable"), but IMO it does not match user expectations nor developer expectations[^5]. Especially since not all our integrations that use stable are bound by this security measure, for example our GitHub Action (since it does a clean fetch of the repository every time it's used). I believe consistency would be good here. Finally, ever since we switched to a tag, we've been facing issues with ReadTheDocs not picking up updates to stable unless we force a rebuild. The initial rebuild on the stable update just pulls the commit the tag previously pointed to. I'm not sure if switching back to a branch will fix this, but I'd wager it will. [^1]: https://git-scm.com/docs/git-tag#_on_re_tagging [^2]: https://black.readthedocs.io/en/stable/contributing/release_process.html#moving-the-stable-tag [^3]: https://github.com/psf/black/issues/2503#issuecomment-1196357379 [^4]: In fairness, most folks working on Black probably don't use the `stable` ref anyway, especially us maintainers who'd know what is the latest version by heart, but it'd still be nice to make it usable for local dev though. [^5]: Also what benefit does a `stable` ref have over explicit version tags like `22.6.0`? If you're going to opt into some odd pin mechanism, might as well use explicit version tags for clarity and consistency. * Prepare docs for release 22.8.0 (#3248) * docs: adds ExitStack alternative to future_style.md (#3247) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Add preview flag to Vim plugin (#3246) This allows the configuration of the --preview flag in the Vim plugin. * Mitigate deprecation of aiohttp's `@middleware` decorator (#3259) This is deprecated since aiohttp 4.0. If it doesn't exist just define a no-op decorator that does nothing (after the other aiohttp imports though!). By doing this, it's safe to ignore the DeprecationWarning without needing to require the latest aiohttp once they remove `@middleware`. * Move 3.11 tests to install aiohttp without C extensions (#3258) * Move 311 tests to install aiohttp without C extensions - Configure tox to install aiohttp without extensions - i.e. use `AIOHTTP_NO_EXTENSIONS=1` for pip install - This allows us to reenable blackd tests that use aiohttp testing helpers etc. - Had to ignore `cgi` module deprecation warning - Filed issue for aiohttp to fix: https://github.com/aio-libs/aiohttp/issues/6905 Test: - `/tmp/tb/bin/tox -e 311` * Fix formatting + linting * Add latest aiohttp for loop fix + Try to exempt deprecation warning but failed - will ask for help * Remove unnecessary warning ignore Co-authored-by: Cooper Ry Lees <me@wcooperlees.com> Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * [FIX] migrate-black.py: don't fail on binary files (#3266) * Fix a crash on dicts with paren-wrapped long string keys (#3262) Fix a crash when formatting some dicts with parenthesis-wrapped long string keys. When LL[0] is an atom string, we need to check the atom node's siblings instead of LL[0] itself, e.g.: dictsetmaker atom STRING '"This is a really long string that can\'t be expected to fit in one line and is used as a nested dict\'s key"' /atom COLON ':' atom LSQB ' ' '[' listmaker STRING '"value"' COMMA ',' STRING ' ' '"value"' /listmaker RSQB ']' /atom COMMA ',' /dictsetmaker * Improve order of paragraphs on line splitting (#3270) These two paragraphs were tucked away at the end of the section, after the diversion on backslashes. I nearly missed the first paragraph and opened a nonsense issue, and I think the second belongs higher up with it too. * Fix mypyc build errors on newer manylinux2014_x86_64 images (#3272) Make sure `gcc` is installed in the build env The mypyc build requires `gcc` to be installed even if it's being built with `clang`, otherwise `clang` fails to find `libgcc`. * Bump furo from 2022.6.21 to 2022.9.15 in /docs (#3277) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Build mypyc wheels for CPython 3.11 (#3276) Bumps cibuildwheel from 2.8.1 to 2.10.0 which has 3.11 building enabled by default. Unfortunately mypyc errors out on 3.11: src/black/files.py:29:9: error: Name "tomllib" already defined (by an import) [no-redef] ... so we have to also hide the fallback import of tomli on older 3.11 alphas from mypy[c]. * Make context manager examples in future style docs consistent (#3274) * Support version specifiers in GH action (#3265) Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> * Fix a crash when `# fmt: on` is used on a different block level than `# fmt: off` (#3281) Previously _Black_ produces invalid code because the `# fmt: on` is used on a different block level. While _Black_ requires `# fmt: off` and `# fmt: on` to be used at the same block level, incorrect usage shouldn't cause crashes. The formatting behavior this PR introduces is, the code below the initial `# fmt: off` block level will be turned off for formatting, when `# fmt: on` is used on a different level or there is no `# fmt: on`. This also matches the current behavior when `# fmt: off` is used at the top-level without a matching `# fmt: on`, it turns off formatting for everything below `# fmt: off`. - Fixes #2567 - Fixes #3184 - Fixes #2985 - Fixes #2882 - Fixes #2232 - Fixes #2140 - Fixes #1817 - Fixes #569 * Make README logo link to docs (#3285) docs: Make README logo link to docs * Switch build backend to Hatchling (#3233) This implements PEP 621, obviating the need for `setup.py`, `setup.cfg`, and `MANIFEST.in`. The build backend Hatchling (of which I am a maintainer in the PyPA) is now used as that is the default in the official Python packaging tutorial. Hatchling is available on all the major distribution channels such as Debian, Fedora, and many more. ## Python support The earliest supported Python 3 version of Hatchling is 3.7, therefore I've also set that as the minimum here. Python 3.6 is EOL and other build backends like flit-core and setuptools also dropped support. Python 3.6 accounted for 3-4% of downloads in the last month. ## Plugins Configuration is now completely static with the help of 3 plugins: ### Readme hynek's hatch-fancy-pypi-readme allows for the dynamic construction of the readme which was previously coded up in `setup.py`. Now it's simply: ```toml [tool.hatch.metadata.hooks.fancy-pypi-readme] content-type = "text/markdown" fragments = [ { path = "README.md" }, { path = "CHANGES.md" }, ] ``` ### Versioning hatch-vcs is currently just a wrapper around setuptools-scm (which despite the legacy naming is actually now decoupled from setuptools): ```toml [tool.hatch.version] source = "vcs" [tool.hatch.build.hooks.vcs] version-file = "src/_black_version.py" template = ''' version = "{version}" ''' ``` ### mypyc hatch-mypyc offers many benefits over the existing approach: - No need to manually select files for inclusion - Avoids the need for the current CI workaround for https://github.com/mypyc/mypyc/issues/946 - Intermediate artifacts (like `build/`) from setuptools and mypyc itself no longer clutter the project directory - Runtime dependencies required at build time no longer need to be manually redeclared as this is a built-in option of Hatchling Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Fix outdated references to 3.6 and run pyupgrade (#3286) I also missed the accidental removal of the 3.11 classifier in the PR. * Always call freeze_support() if sys.frozen is True (#3275) * Bump actions/upload-artifact from 2 to 3 (#3289) updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-major Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump sphinx from 5.1.1 to 5.2.1 in /docs (#3288) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.1.1 to 5.2.1. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/5.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v5.1.1...v5.2.1) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add option to format Jupyter Notebooks in GitHub Action (#3282) To run the formatter on Jupyter Notebooks, Black must be installed with an extra dependency (`black[jupyter]`). This commit adds an optional argument to install Black with this dependency when using the official GitHub Action [1]. To enable the formatter on Jupyter Notebooks, just add `jupyter: true` as an argument. Feature requested at [2]. [1]: https://black.readthedocs.io/en/stable/integrations/github_actions.html [2]: https://github.com/psf/black/issues/3280 Signed-off-by: Antonio Ossa Guerra <aaossa@uc.cl> * Mention CHANGES.md in PR template explicitly (#3295) This makes the location more explicit which hopefully makes the PR process smoother for other first time contributors. Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> * Bump pypa/cibuildwheel from 2.10.0 to 2.10.2 (#3290) updated-dependencies: - dependency-name: pypa/cibuildwheel dependency-type: direct:production update-type: version-update:semver-patch Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Enable build isolation under CIWB (#3297) No idea how this is still here after the Hatchling PR, but it is no longer useful and is breaking the build. * Add .ipynb_checkpoints to DEFAULT_EXCLUDES (#3293) Jupyter creates a checkpoint file every single time you create an .ipynb file, and then it updates the checkpoint file every single time you manually save your progress for the initial .ipynb. These checkpoints are stored in a directory named `.ipynb_checkpoints`. Co-authored-by: Batuhan Taskaya <isidentical@gmail.com> * Bump myst-parser from 0.18.0 to 0.18.1 in /docs (#3303) Bumps [myst-parser](https://github.com/executablebooks/MyST-Parser) from 0.18.0 to 0.18.1. - [Release notes](https://github.com/executablebooks/MyST-Parser/releases) - [Changelog](https://github.com/executablebooks/MyST-Parser/blob/master/CHANGELOG.md) - [Commits](https://github.com/executablebooks/MyST-Parser/compare/v0.18.0...v0.18.1) --- updated-dependencies: - dependency-name: myst-parser dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump furo from 2022.9.15 to 2022.9.29 in /docs (#3304) Bumps [furo](https://github.com/pradyunsg/furo) from 2022.9.15 to 2022.9.29. - [Release notes](https://github.com/pradyunsg/furo/releases) - [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md) - [Commits](https://github.com/pradyunsg/furo/compare/2022.09.15...2022.09.29) --- updated-dependencies: - dependency-name: furo dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cooper Lees <cooper@fb.com> * Bump sphinx from 5.2.1 to 5.2.3 in /docs (#3305) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 5.2.1 to 5.2.3. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/5.x/CHANGES) - [Commits](https://github.com/sphinx-doc/sphinx/compare/v5.2.1...v5.2.3) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docutils from 0.18.1 to 0.19 in /docs (#3161) Bumps [docutils](https://docutils.sourceforge.io/) from 0.18.1 to 0.19. --- updated-dependencies: - dependency-name: docutils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Preserve crlf line endings in blackd (#3257) Co-authored-by: KotlinIsland <kotlinisland@users.noreply.github.com> * Add option to skip the first line of source code (#3299) * Add option to skip the first line in source file This commit adds a CLi option to skip the first line in the source files, just like the Cpython command line allows [1]. By enabling the flag, using `-x` or `--skip-source-first-line`…

DetachHead approved these changes Sep 5, 2022

View reviewed changes

cooperlees reviewed Sep 5, 2022

View reviewed changes

JelleZijlstra reviewed Sep 5, 2022

View reviewed changes

KotlinIsland force-pushed the feat/blackd_crlf branch from 3ce63e1 to 23c6427 Compare September 6, 2022 11:12

KotlinIsland requested review from JelleZijlstra, DetachHead and cooperlees and removed request for JelleZijlstra, DetachHead and cooperlees September 8, 2022 04:43

felix-hilden reviewed Sep 11, 2022

View reviewed changes

KotlinIsland force-pushed the feat/blackd_crlf branch from 23c6427 to 91c5705 Compare September 12, 2022 10:06

KotlinIsland changed the title ~~(🎁) Preserve crlf newlines in blackd~~ (🎁) Preserve crlf line endings in blackd Sep 12, 2022

KotlinIsland force-pushed the feat/blackd_crlf branch from 91c5705 to 14cc0e4 Compare September 12, 2022 10:09

Preserve crlf line endings in blackd

1c1647a

KotlinIsland force-pushed the feat/blackd_crlf branch from 14cc0e4 to 1c1647a Compare September 12, 2022 10:12

JelleZijlstra approved these changes Sep 23, 2022

View reviewed changes

KotlinIsland requested a review from cooperlees September 23, 2022 03:50

aaossa mentioned this pull request Oct 4, 2022

Add option to skip the first line of source code #3299

Merged

3 tasks

zsol approved these changes Oct 4, 2022

View reviewed changes

JelleZijlstra merged commit 0359b85 into psf:main Oct 4, 2022

KotlinIsland mentioned this pull request Oct 5, 2022

Use project / system line endings lensvol/intellij-blackconnect#61

Closed

KotlinIsland deleted the feat/blackd_crlf branch October 5, 2022 04:44

KotlinIsland mentioned this pull request Feb 10, 2023

(🐞) blackd: fix issue for mishandling single character input #3558

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(🎁) Preserve crlf line endings in blackd #3257

(🎁) Preserve crlf line endings in blackd #3257

KotlinIsland commented Sep 5, 2022 •

edited

Loading

github-actions bot commented Sep 5, 2022 •

edited

Loading

cooperlees left a comment

cooperlees Sep 5, 2022

KotlinIsland Sep 5, 2022 •

edited

Loading

cooperlees Sep 5, 2022

JelleZijlstra Sep 5, 2022

JelleZijlstra Sep 5, 2022

KotlinIsland Sep 5, 2022 •

edited

Loading

KotlinIsland Sep 5, 2022

felix-hilden Sep 11, 2022

KotlinIsland Sep 11, 2022 •

edited

Loading

felix-hilden Sep 11, 2022 •

edited

Loading

KotlinIsland Sep 12, 2022

KotlinIsland Sep 12, 2022

zsol left a comment

(🎁) Preserve crlf line endings in blackd #3257

(🎁) Preserve crlf line endings in blackd #3257

Conversation

KotlinIsland commented Sep 5, 2022 • edited Loading

Description

Checklist - did you ... (know?)

github-actions bot commented Sep 5, 2022 • edited Loading

cooperlees left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KotlinIsland Sep 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KotlinIsland Sep 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KotlinIsland Sep 11, 2022 • edited Loading

Choose a reason for hiding this comment

felix-hilden Sep 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zsol left a comment

Choose a reason for hiding this comment

KotlinIsland commented Sep 5, 2022 •

edited

Loading

github-actions bot commented Sep 5, 2022 •

edited

Loading

KotlinIsland Sep 5, 2022 •

edited

Loading

KotlinIsland Sep 5, 2022 •

edited

Loading

KotlinIsland Sep 11, 2022 •

edited

Loading

felix-hilden Sep 11, 2022 •

edited

Loading