Discuss how to update the "Tool recommendations" page

[jeanas]

Quoting from https://gregoryszorc.com/blog/2023/10/30/my-user-experience-porting-off-setup.py/:

Outdated Tool Recommendations from the PyPA

Finally I find Tool recommendations in the PyPA User Guide. Under Packaging tool recommendations it says:

• Use setuptools to define projects.
• Use build to create Source Distributions and wheels.
• If you have binary extensions and want to distribute wheels for multiple platforms, use cibuildwheel as part of your CI setup to build distributable wheels.
• Use twine for uploading distributions to PyPI.

Finally, some canonical documentation from the PyPA that comes out and suggests what to use!

But my relief immediately turns to questioning whether this tooling recommendations documentation is up to date:

1. If setuptools is recommended, why does the Packaging Python Projects tutorial use Hatch?
2. How exactly should I be using setuptools to define projects? Is this referring to setuptools as a [build-system] backend? The existence of define seemingly implies using setup.py or setup.cfg to define metadata. But I thought these distutils/setuptools specific mechanisms were deprecated in favor of the more generic pyproject.toml?
3. Why aren't other tools like Hatch, pip, poetry, flit, and pdm mentioned on this page? Where's the guidance on when to use these alternative tools?
4. There are footnotes referencing distutils as if it is still a modern practice. No mention that it was removed from the standard library in Python 3.12.
5. But the build tool is referenced and that tool is relatively new. So the docs have to be somewhat up-to-date, right?

These are all valid concerns.

I'm willing to do the work of updating that page, but I think this is a potentially very delicate/controversial topic, so I figured I'd open an issue to agree on a plan first, so as not to mix this discussion with PR review details.

Some thoughts by tool category:

• I think we can agree that pip should be recommended. But this could/should also mention pipx for the specific case of CLI tools. I don't think pipx has much competition right now (though it may have in the future).
• The recommendation of virtualenv/venv is fine. But we could mention that there many (a great many) tools out there to manage environments more automatically (virtualenvwrapper, Poetry/PDM/Hatch, etc.)
• The recommendation of pip-tools, Pipenv or poetry for lock files could be expanded to include PDM.
• I think we should drop buildout and Hashdist, both of these are dead judging from Git commit history, especially Hashdist.
• I do not think the PyPA should officially recommend setuptools as a build backend. After all, the explicit purpose of PEP 517 was to allow other backends. Here, I think we should simply list the different alternatives without making anything that would look like an official choice, since there is no consensus on such a choice. The backends could be listed in the same order as the tabs in the "Packaging projects" tutorial.
• The cibuildwheel recommendation is fine. I don't think there are any serious alternatives.
• I'm not sure about recommending build and twine. Certainly, they're better than python setup.py bdist_wheel and python setup.py upload, but the equivalent Hatch/Poetry/PDM commands are fine too.

By and large, I think we should make less recommendations on that page, be more objective, and especially avoid recommending a given tool when there is no consensus among PyPA members that this should be the blessed tool, since the officialness of the page makes users believe that a consensus exists. For better or worse, there is more good advice to be given on what not to use — e.g., this could be a good page to remind things like "don't use python setup.py upload", "don't use distutils", and "whichever backend you choose, declare it in the [build-system] table".

One other thing I'm not sure about is the status of Pipenv (how well-maintained is it and are many people using it after all the controversy that there was around it?).

[webknjaz]

• I'm not sure about recommending build and twine. Certainly, they're better than python setup.py bdist_wheel and python setup.py upload, but the equivalent Hatch/Poetry/PDM commands are fine too.

These are purpose-specific tools, related to specific tasks in packaging and so we recommend them. The latter are workflow tools, not necessarily packaging tools, only one of them being under PyPA, some having a track record of serious problems. PyPUG should recommend tools for tasks. Workflow tools can be in a separate category, like managing virtualenvs (next to things like tox/nox).

[webknjaz]

• I think we should drop buildout and Hashdist, both of these are dead judging from Git commit history, especially Hashdist.

Yes.

[webknjaz]

• I do not think the PyPA should officially recommend setuptools as a build backend. After all, the explicit purpose of PEP 517 was to allow other backends. Here, I think we should simply list the different alternatives without making anything that would look like an official choice, since there is no consensus on such a choice. The backends could be listed in the same order as the tabs in the "Packaging projects" tutorial.

It's a PyPA project that works. And in some cases, there's no (easily) usable replacements. So I'm -1 on this.

[webknjaz]

the officialness of the page makes users believe that a consensus exists.

I think we might need a document "What PyPA is not." explaining that there can't be consensus in some areas, by design.

[webknjaz]

One other thing I'm not sure about is the status of Pipenv (how well-maintained is it and are many people using it after all the controversy that there was around it?).

I'm hearing that the new maintainers are better. Same with Poetry — people said that the current maintainers aren't as problematic. But I'm still not a fan of either because of their history and bad experiences.

[jeanas]

• I do not think the PyPA should officially recommend setuptools as a build backend. After all, the explicit purpose of PEP 517 was to allow other backends. Here, I think we should simply list the different alternatives without making anything that would look like an official choice, since there is no consensus on such a choice. The backends could be listed in the same order as the tabs in the "Packaging projects" tutorial.

It's a PyPA project that works. And in some cases, there's no (easily) usable replacements. So I'm -1 on this.

TBH, I was expecting this to be the controversial item 😄

I completely admit that I am biased, since I personally find setuptools hard to understand and use.

With that being said, I don't really understand your argument with setuptools being a PyPA project, since Flit and Hatch are also PyPA projects. Since there are three build backends in the PyPA, how do we justify the existence of the other two if PyPA "officially recommends" setuptools?

I also think it's worth remembering that when this recommendation was added long ago, it was a recommendation to use setuptools as opposed to distutils, not as opposed to other build backends which only became possible later, and at that time there was consensus on using setuptools rather than distutils. Today, there is clearly no consensus — you like setuptools, I like Hatch, the Flit maintainers like Flit, ... So, by the evolution of the situation, the page has now become objectively misleading, since it presents setuptools as the official recommendation as if this were the PyPA consensus, whereas there is no PyPA consensus. Quoting @pfmoore on https://discuss.python.org/t/removing-setup-cfg-and-setup-py-from-the-packaging-tutorial/16096/19?u=jeanas: “We’ve worked hard to open up the ecosystem to allow users to have a choice of backends. Promoting and encouraging that choice is part of that work.”

Finally, it's kind of contradictory if the official recommendation is setuptools but the "Packaging projects" tutorial uses Hatchling by default.

To be extra clear, I'm absolutely not advocating that we change this page to recommend a build backend other than setuptools, but that we say something like

“PyPA does not recommend a single build backend for all cases. Popular build backends include Hatchling, setuptools, Flit-core, PDM-backend and Poetry-core. For compiled extension modules, common build backends are meson-python, scikit-build and maturin.”

[sinoroc]

the officialness of the page makes users believe that a consensus exists.

I think we might need a document "What PyPA is not." explaining that there can't be consensus in some areas, by design.

I like this idea.

[jeanas]

I've opened #1474, which implements basically what I said regarding build backends, along with other updates of the same page.

[webknjaz]

“PyPA does not recommend a single build backend for all cases. Popular build backends include Hatchling, setuptools, Flit-core, PDM-backend and Poetry-core. For compiled extension modules, common build backends are meson-python, scikit-build and maturin.”

Yeah, I see your points and having read what you wrote and linked earlier got me thinking that it would be useful to have some sort of an admonition/banner at the top of all (many?) pages with this (similar?) explanation.

I remember the PyPA got a lot of blame for using one tool or the other in different guides with people reading it as favoritism. And having more explicit disclaimers would hopefully improve the perception. The individual documents will still use a single tool per document in most cases due to the nature of those text types, of course.

[webknjaz]

Also, I have a feeling that there's confusion in some places when we document using tools as build front-ends. This is because many of those are workflow tools that compare more with tox/nox, not pip — they often manage virtualenvs and something that resembles lockfile ideas. They are also arbitrary command runners. And strictly speaking, I don't think something called a packaging guide should focus on those. In this context, I'd mention hatchling over Hatch, for example, and be explicit about their components that implement cross-compatible standards rather than unique inventions that cause vendor-locking.

[eli-schwartz]

If setuptools is recommended, why does the Packaging Python Projects tutorial use Hatch?

The tutorial used Hatch because the tutorial was improved to start using pyproject.toml metadata as defined by PEP 621, and at the time the PR was opened by @henryiii, setuptools didn't support that, but 6 months later when the tutorial was merged, setuptools had supported it for quite some time. I got the impression at the time that it was unofficially chosen because "hatch good", but I could be wrong.

It was the wrong move either way, since flit provides a much better tutorial for getting people started quick-'n-easy which is kind of what a tutorial is for.

To be extra clear, I'm absolutely not advocating that we change this page to recommend a build backend other than setuptools, but that we say something like

“PyPA does not recommend a single build backend for all cases. Popular build backends include Hatchling, setuptools, Flit-core, PDM-backend and Poetry-core. For compiled extension modules, common build backends are meson-python, scikit-build and maturin.”

Setuptools remains an adequate build backend for cases where you have no external dependencies, don't use fortran, and single-threaded compilation isn't a bottleneck for you. This is a huge percentage of packages.

Maturin is a special-case backend, which is very useful if you use rust (very nearly the only game in town) and very useless if you don't. To an extent it feels a bit to me like "if you use rust, you know maturin exists and you also probably wanted to use it anyway". I'm not sure whether it would be more confusing to mention it for something as generic as "compiled extension modules" -- in comparison, meson-python and scikit-build are powered by actual omni-language build systems that can handle pretty much anything you might try throwing at it (meson can even build rust code, just FYI ;)) and thus make a lot of sense to advertise in that slot.