Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv pip install -e and its PEP-660-style editable installs break vscode import resolution (and probably static type checkers) #3898

Closed
ThiefMaster opened this issue May 29, 2024 · 5 comments
Labels
question Asking for clarification or support

Comments

@ThiefMaster
Copy link

My context is that I have a main package and some related packages. Everything is installed in editable mode for development.

When using standard pip with setuptools installed, this creates an egg-link file inside site-packages which is looked up just fine by type checkers including Pyright (which powers the vscode Python/Pylance extension).

However, when using uv pip install -e, the package is installed in PEP-660 mode with a __editable__.<pkgname+version>.pth and __editable___<pkgname+version>_finder.py, ie using a dynamic import hook.

Of course static analyzers generally do not understand those, and don't even try to extract the information from the "dynamic" code even though it would be trivial to do (by simply parsing the MAPPING dict from the ..._finder.py).

While this is not directly uv's fault, and I generally agree with not outputting legacy stuff, this is causing a problem when IDEs can no longer properly resolve imports. And at least the pyright/pylance developers don't seem to be willing to add any workarounds on their side, so they'd rather wait for the Python ecosystem to agree on something that does not require dynamic import lookups. But that probably needs someone to be willing to write a PEP and get it accepted which all takes some time.

Some related issues/comments:

Is there any chance uv can do something about this, such as providing an option to generate an egg-link instead of a PEP660-style __editable__*.pth, even though it's slightly more legacy?

@h-mayorquin
Copy link

To be honest I am bit confused. Pip passess the implementation responsability of editable installs to the backend:

https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs

Doesn't uv does the same?

For packages built with setuptools there are a coupe of configuration options used at installation time. That said, I am not sure if the latest version of setuptools works the way that OP proposes here anymore.

For reference, when building with hatch works with the default that allows static analysis tools to work by default:
pypa/hatch#1629 (comment)
I am not sure what uv pip install -e would do if the build backend was hatchling.

Any chance a mantainer could chime in, clarify the state of affairs and say whether this is:

  • Important but not resources to addres it at the moment.
  • Not important, unlikely to be addressed.

@zanieb
Copy link
Member

zanieb commented Jul 19, 2024

I agree this issue is not clear. I'm not the expert on this area, but a hatch build backend doesn't use a dynamic .pth

❯ uv venv
Using Python 3.12.3 interpreter at: /opt/homebrew/opt/python@3.12/bin/python3.12
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
❯ grep "build-system" -A 2 ../packse/pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
❯ uv pip install -e ../packse -q                                                                                      
❯ cat .venv/lib/python3.12/site-packages/_packse.pth
/Users/zb/workspace/packse/src%     

This seems like a problem with setuptools? In their documentation they note:

Setuptools offers no guarantee of which technique will be used to perform an editable installation. This will vary from project to project and may change depending on the specific version of setuptools being used.

It seems possible that you're getting different versions of setuptools depending on if you're using uv or pip? Having you tried using --config-settings to configure this?

@zanieb zanieb added the question Asking for clarification or support label Jul 19, 2024
@konstin
Copy link
Member

konstin commented Jul 19, 2024

This is a known problem with setuptools: pypa/setuptools#3518, it also affects e.g. mypy: python/mypy#13392. It's unfortunately not something we can do anything about, i recommend using one of the workarounds in the linked issues or switching to a different backend such as hatchling or flit.

@konstin konstin closed this as completed Jul 19, 2024
@ThiefMaster
Copy link
Author

ThiefMaster commented Jul 19, 2024

It's unfortunately not something we can do anything about

Why not offer a way to do an editable install using the classic static resolution (egg-link) instead of the dynamic one? A while ago I noticed that pip does this when setuptools is installed, while it uses the dynamic resolve when setuptools is not installed.

I never really looked into alternative build backends, but I assume this also means changing all the related configs (setup.cfg etc.) in the project, which is a mess if you have many projects and you either end up with one using something different than all the others, or having to update all of them for consistency...

@konstin
Copy link
Member

konstin commented Jul 19, 2024

The old egg mechanism was never standardized and is basically pip-setuptools special casing; in uv, we support PEP 660, the standard editable installs that work across build backends. Pip has also deprecated its current behavior and will remove it in the 25.0 release (pypa/pip#11457).

benpankow added a commit to dagster-io/dagster that referenced this issue Sep 4, 2024
…all (#24029)

## Summary & Motivation

After wiping my pyright & base virtual envs, I was unable to get pyright
working, hitting [strange errors that dagster imports were not
found](https://dagsterlabs.slack.com/archives/C03A0D72A6T/p1724874346772169):

```python
/Users/ben/repos/dagster/python_modules/libraries/dagster-shell/dagster_shell/__init__.py:
  1:5: Import "dagster._core.libraries" could not be resolved (reportMissingImports)
```

Some Googling turned up that there are [multiple ways to install
editable packages](astral-sh/uv#3898) & the
default behavior of `uv pip install` is to not install packages in
compat mode, which pyright requires.

We [explicitly opt in in our pyright venv building
code](https://github.com/dagster-io/dagster/blob/master/scripts/run-pyright.py#L304),
but I'm wondering if my base editable installs not using compat mode has
some bleed-over to my pyright venv (e.g. the editable install is
shared)? Either way, making this change, rerunning `make dev_install`
and then rerunning `make rebuild_pyright` seemed to fix my issues.


## Changelog [Bug]

`NOCHANGELOG`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Asking for clarification or support
Projects
None yet
Development

No branches or pull requests

4 participants