[FR] Implement PEP 660 -- Editable Installations

[pradyunsg]

What's the problem this feature will solve?

PEP 660 standardises the mechanism that setup.py develop uses to make "editable" packages.

Currently, pip needs to implement fairly horrible hacks to make legacy setup.py develop builds work. It'd be great if setuptools could implement PEP 660, which would be the last item from setuptools' end before pip can start the process of removing the legacy directly-call-setup.py build system.

Describe the solution you'd like

PEP 660's build_editable hook gets implemented in aetuptools.

Alternative Solutions

Not doing this.

I imagine this will mean that either (a) pip continues to maintain editable installs for setuptools or (b) pip drops the legacy build system entirely and users of setuptools projects will no longer be able to make editable installations.

I don't think (b) is happening any time soon, and it'd likely be best to implement the hook here.

Additional context

IIUC, this has been implemented in setuptools-pep660 already. It might not be a lot of work to migrate the work done there into seriptools directly.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

See the discussion and dholth/setuptools_pep660#1.

[ichard26]

I took a stab at this but I can't seem to find a reliable way to query the importable base of the packages / modules (i.e. the parent directory of {name}.dist-info generated from an in-tree build). For what it's worth this is my first time working in the setuptools codebase so there's probably something I'm missing :)

[ichard26]

Status update: after adding some more runtime patching of setuptools internals to query all of the information I need I got an implementation that can do an editable install for at least a simple single pure-Python module distribution and a src-style pure-Python multi-package distribution (psf/black, duh 🙂) 🎉

main...ichard26:pep-660

I need to test other situations and packaging configurations, but I've ran out of time for today :)

[tacaswell]

I think that preserving editable installs for packages with c / c++ / cython installations via some mechanism is cirtical (I say with my Matplotlib and h5py hats on).

I would be perfectly happy to absorb changes to our setup.py / pyproject.toml / built incantation, but there needs to be some path forward.

[ichard26]

I think that preserving editable installs for packages with c / c++ / cython installations via some mechanism is cirtical (I say with my Matplotlib and h5py hats on).

Right, unfortunately I don't really have much experience in this domain¹. It is my understanding that PEP 660's way of handling extensions is by mandating a in-tree and in-place extension build (effectively python setup.py build_ext -i) before the editable wheel is produced. This obviously means the editable install / the in-place extension build will have to be redone to see changes in the extensions. Other than that I don't see how using a root .pth file (pointing to the importable base which by now should have the built extensions in-tree)² changes things as it's basically how python setup.py develop worked anyway, just codified ^^

I'd love input and insight from people like yourself regarding extensions for both my own understanding and implementation testing!

Footnotes

1. I can only speak with my psf/black (and maybe pypa/bandersnatch) hats on 🙂 ↩

2. This is the implementation strategy I chose as it seems the simplest haha ↩