Adopt distutils

Originally reported by: jaraco (Bitbucket: jaraco, GitHub: jaraco)

---

Setuptools builds on the distutils framework of the Python Stdlib. Doing so has benefits of depending on a standard, extensible framework, but also has downsides, such as limiting the advancement of distutils (across many Python versions) but simultaneously adding monkey patches to patch bugs in distutils.

I'd like to consider if Setuptools should adopt distutils (or a version of it). Doing so would decouple Setuptools from distutils, freeing Setuptools from the constraints of distutils and freeing distutils from the expectations of Setuptools.

Doing so would allow distutils to adapt at the same pace as Setuptools, addressing issues and vetting patches that could be ported back to distutils in CPython.

I've once proposed this idea and it was promptly dismissed by @RonnyPfannschmidt. Why is vendoring distutils in Setuptools not an option?

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/417

Original comment by ncoghlan (Bitbucket: ncoghlan, GitHub: ncoghlan):

---

I think this could work well with an eventual approach like the following:

1. By default, importing setuptools would leave the standard distutils namespace alone
2. setuptools would gain a "setuptools.override_distutils()" command that injected the vendored distutils into sys.modules (and complained if the standard library one had already been imported)
3. pip was updated to both import setuptools and tell it to explicitly override distutils

There'd need to be a transition period where setuptools did the injection by default, as the monkeypatching of distutils is currently implicit in importing setuptools

Longer term, we could eventually aim to move distutils out of the normal standard library update cycle, and instead install it via ensurepip (such that it evolves at the pace of the Python packaging ecosystem, rather than at the pace of the underlying language runtime)

Original comment by dstufft (Bitbucket: dstufft, GitHub: dstufft):

---

@ncoghlan Would that work? Doesn't Python depend on distutils to build itself?

Original comment by qwcode (Bitbucket: qwcode, GitHub: qwcode):

---

I love the possibility. btw, I recall this being discussed a couple years back. At the time, I guess it was too radical to gain any traction.

as for Nick's 1/2/3 points, wouldn't vendoring it, effectively give it a new namespace, and avoid any concerns with conflicts?

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

My thinking was more along the lines with what Marcus is suggesting.

Thanks to Nick's comments, I'm reminded that pip install (of non-binary distributions) and also setup.py install (where setuptools is somehow imported) are expected to run with setuptools' overrides. Maintaining that expectation could be tricky, but is probably achievable.

Original comment by ncoghlan (Bitbucket: ncoghlan, GitHub: ncoghlan):

---

Python does depend on distutils to build some extension modules, but not the core interpreter itself.

However, unbundling per se probably isn't worth the hassle, as implementing the 1/2/3 approach I suggested above based on a "setuptool._distutils" vendored copy should provide most of the benefits with a fraction of the pain. The key will be ensuring the stdlib distutils is kept in sync with the setuptools vendored copy for bug fixes (all versions) and new features (future versions).

Original comment by RonnyPfannschmidt (Bitbucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt):

---

i fear this will result in a certain hell, there is no reasonable way to get rid of the current legacy within setuptools without creating many lines of compatibility breaks

Original comment by ralfgommers (Bitbucket: ralfgommers, GitHub: Unknown):

---

I don't quite understand how the "vendoring and put in a new namespace" idea would help. There are other packages that extend (and monkeypatch) distutils. So they'd then all have to change to instead extend/monkeypatch setuptools.distutils?

Original comment by andi0661 (Bitbucket: andi0661, GitHub: Unknown):

---

I like the idea of separating distutils and setuptools, but given @RonnyPfannschmidt 's and @ralfgommers 's comments, it seems to me that there is so much new legacy on top of setuptools that it would require a mechanism that allows for incompatible changes. Maybe a new major version is sufficient, or a different package name. That would allow packages to transition to it as they have a need for the better functions, less bugs, etc.

[jaraco]

@xoviat : To answer your question from the other ticket, I'd very much appreciate a PR.

Okay, I'll send one your way sometime soon.

@jaraco Please create a new distutils branch:

Ref #1191.

[jaraco]

Distutils branch now exists, created against the latest master. Is that what you had in mind?