-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix --target not working with --editable #9636
Conversation
… understand --home but instead requires the --install-dir option. Parially fixes pypa#4390
--target
not working with --editable
--target
not working with --editable
--target
not working with --editable
This would also fix pypa/setuptools#392 |
Thanks @webknjaz Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At first blush, it's not working for me:
I tried without pip-run (for the target install), but it still doesn't produce a usable install:
It seems there's no This patch does correct the "error: option --home not recognized" but it still doesn't provide a viable install. Making that work may require changes to Setuptools. I haven't yet looked into what behaviors affect making an editable package visibly installed. |
Aha. So the reason This patch seems fine, but I'd be curious - does this patch provide any benefit without improved |
I do not claim to fully understand the internals of setuptools, but it is my understanding that --target only installs the package to the folder in question. You still need to add the directory / the packages inside it to the install path to get python to be able to import them. The project that prompted me to do the patch uses code like this to achieve that: def add_site(path):
"""This adds a path to as proper site packages to all associated import
systems. Primarily it invokes `site.addsitedir` and also configures
pkg_resources' metadata accordingly.
"""
site.addsitedir(path)
ws = pkg_resources.working_set
ws.entry_keys.setdefault(path, [])
ws.entries.append(path)
for dist in pkg_resources.find_distributions(path, False):
ws.add(dist, path, insert=True) (Note to self: The code above doesn't seem quite right, as it seems to add the path twice to |
I think it serves as the baseline to make the install work, and then build upon it to make discovery of installed packages simpler in the relevant packages (not sure yet what package that should be.) |
IMO we should
|
In the abstract, Setuptools is interested in solving this issue. My preference would be to solve the 'editable' problem in the packaging space and not just patch the Setuptools implementation, though I'd also accept a patch to Setuptools to address this specific weakness. |
Me too, but the interest around this is just insufficient to move this forward. A part of me want to just break editables entirely to force people’s hands into making something happen. In the same vein, I don’t have any interest in merging this until someone to actually work on a setuptools patch. (Note that this is for me personally; other pip maintainers may feel differently, and I would feel most happy if this can fix itself without me spending any effort.) |
Well then, what are the possible steps to get this fixed? In Setuptools? In the package space? For Setuptools I would imagine that it needs to recognise .egg-link files and activate those packages if the directory is on the python path. Or is it something else that should happen? |
I think these files are relic of ancient times when people used easy_install to install Python packages and are not and should not be supported by any modern tool.
.pth files are established way of extending PYTHONPATH and I think the right way to go here. Either just add editables to this file or come up with some different .pth file for this purpose. Or maybe it should be a separate .pth file per editable package installed? I wonder if issue #9471 won't have similar problem? |
I may be wrong here, but my understanding of .pth files is that they basically add directories to the python path, while .egg-link files basically act as symlinks to link a package in a folder which (hopefully) is already part of the python path. Also, I think handling of metadata is slightly special, because it is taken from within the project folder, instead of being installed alongside the package in the site directory. Since Symlinks seem to be supported even in Windows nowadays Maybe just using an actual symlink to the project is the right way to go? To support working on editable installed packages it's metadata should really be in the project instead of the site folder, so either that needs / should be symlinked too, or should be taken from inside the project initially? I'd love some thoughts on these points. |
Using symlinks for install is what flit does – https://flit.readthedocs.io/en/latest/cmdline.html?highlight=symlink#cmdoption-flit-install-s |
There's plenty of context here. We have a proof of concept implementation in the form of my editables package, plus an initial implementation PR in pip (which I can't recall the link for, and don't know the status of - it's in that Discourse thread, though). Every time we discuss standardising editables, though, we get a certain way and then everyone loses motivation/focus and things stall again, as @uranusjr noted. I don't know if this change wants to get sucked into the whole "standardise editables" debate, though. But if @jaraco feels it's better to do that than some sort of tactical hack to setuptools, I can understand that. |
BTW, my personal opinion is similar to @uranusjr - I feel that we shouldn't keep trying to prop up the existing editable mechanism. If it doesn't work, the solution is to sort out standardising a replacement. If the community isn't motivated to do that, fine, but in that case we should put up with what we have, rough edges and breakages and all. |
I am not sure why feel that lately the sentiment always seems to be that all or nothing solutions are required to accept any patch. What happened to small incremental improvements? This patch doesn't break anything that wasn't broken before, enables a use-case that wasn't available before, and allows further incremental improvements in other places, all without blocking a standardisation attempt to unify / rework editable installs. I would like to work on a patch to setuptools next that makes it easier to work with |
I'm committed to addressing this in Setuptools, just not on any particular timeline. The argument from dwt is compelling and I recommend to accept this change to facilitate a tactical fix in Setuptools. |
pip flag `-e` and `--target` cannot be used at the same time, they are trying to resolve this but could take a while. See pypa/pip#9636. So I decide not to install rez with `--target` for all venv to access, just re-install it in every venv.
as setup.py develop does not understand
--home
but instead requires the--install-dir
option.Partially fixes #4390