Skip to content

Commit

Permalink
[red-knot] Vendor typeshed's stdlib (#11340)
Browse files Browse the repository at this point in the history
This PR vendors typeshed!

-  The first commit vendors the stdlib directory from typeshed into a new crates/red_knot/vendored_typeshed directory.
-  The second commit adjusts various linting config files to make sure that the vendored code is excluded from typo checks, formatting checks, etc.
-  The LICENSE and README.md files are also vendored, but all other directories and files (stubs, scripts, tests, test_cases, etc.) are excluded. We should have no need for them (except possibly stubs/, discussed in more depth below).
-  Similar to the way pyright has a commit.txt file in its vendored copy of typeshed, to indicate which typeshed commit the vendored code corresponds to, I've also added a crates/red_knot/vendored_typeshed/source_commit.txt file in the third commit of this PR.

One open question is: should we vendor the stdlib and stubs directories, or just the stdlib directory? The stubs/ directory contains stubs for 162 third-party packages outside the stdlib. Mypy and typeshed_client1 only vendor the stdlib directory; pyright and pyre vendor both the stdlib and stubs directories; pytype vendors the entire typeshed repo (scripts/, tests/ and all).

In this PR, I've chosen to copy mypy and typeshed_client. Unlike vendoring the stdlib, which is unavoidable if we want to do typechecking of the stdlib, it's not strictly necessary to vendor the stubs directory: each subdirectory in stubs is published to PyPI as a standalone stubs distribution that can be (uv)-pip-installed into a virtual environment. It might be useful for our users if we vendored those stubs anyway, but there are costs as well as benefits to doing so (apart from just the sheer amount of vendored code in the ruff repository), so I'd rather consider it separately.
  • Loading branch information
AlexWaygood authored and dhruvmanila committed May 17, 2024
1 parent ef223ec commit f07ea65
Show file tree
Hide file tree
Showing 16 changed files with 3,372 additions and 0 deletions.
14 changes: 14 additions & 0 deletions crates/red_knot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,18 @@ The Red Knot crate contains code working towards multifile analysis, type infere

Red Knot vendors [typeshed](https://github.com/python/typeshed)'s stubs for the standard library. The vendored stubs can be found in `crates/red_knot/vendor/typeshed`. The file `crates/red_knot/vendor/typeshed/source_commit.txt` tells you the typeshed commit that our vendored stdlib stubs currently correspond to.

<<<<<<< HEAD
The typeshed stubs are updated every two weeks via an automated PR using the `sync_typeshed.yaml` workflow in the `.github/workflows` directory. This workflow can also be triggered at any time via [workflow dispatch](https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow#running-a-workflow).
||||||| parent of c80c1712f ([red-knot] Vendor typeshed's stdlib (#11340))
=======
Updating the vendored stubs is currently done manually. On a Unix machine, follow the following steps (if you have a typeshed clone in a `typeshed` directory, and a Ruff clone in a `ruff` directory):

```shell
rm -rf ruff/crates/red_knot/vendor/typeshed
mkdir ruff/crates/red_knot/vendor/typeshed
cp typeshed/README.md ruff/crates/red_knot/vendor/typeshed
cp typeshed/LICENSE ruff/crates/red_knot/vendor/typeshed
cp -r typeshed/stdlib ruff/crates/red_knot/vendor/typeshed/stdlib
git -C typeshed rev-parse HEAD > ruff/crates/red_knot/vendor/typeshed/source_commit.txt
```
>>>>>>> c80c1712f ([red-knot] Vendor typeshed's stdlib (#11340))
5 changes: 5 additions & 0 deletions crates/red_knot/vendor/typeshed/source_commit.txt
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
<<<<<<< HEAD
a9d7e861f7a46ae7acd56569326adef302e10f29
||||||| parent of c80c1712f ([red-knot] Vendor typeshed's stdlib (#11340))
=======
2d33fe212221a05661c0db5215a91cf3d7b7f072
>>>>>>> c80c1712f ([red-knot] Vendor typeshed's stdlib (#11340))
Loading

0 comments on commit f07ea65

Please sign in to comment.