Moved distutils' doc for setup-keywords into our doc to document all supported keywords

[venthur]

Summary of changes

1. Moved keywords into own document and linked it from index.txt (as suggested in Document *all* supported keywords, including those from distutils #1700)
2. Fixed the now broken section-cross-references (only works in same document) by making them explicit
3. Migrated docs from distutils: https://docs.python.org/3/distutils/setupscript.html

Closes #1700

Pull Request Checklist

• News fragment added in changelog.d. See documentation for details

[benoit-pierre]

I think some of those keywords should be marked as deprecated:

• data_files: does not work with wheels, so should be avoided
• dependency_links: not supported anymore by pip

[benoit-pierre]

And for setup_requires, we should point people towards PEP 518 instead.

[venthur]

Should setup_requires also marked as deprecated then, or just a link to PEP-518?

[pganssle]

@venthur It's not quite deprecated, more discouraged. As long as it's possible to invoke the setup.py file without your setup_requires dependencies installed, they're actually fine to use, since they're added on to the requirements specified in pyproject.toml as part of the PEP 517 build, but it's better to specify the requirements in pyproject.toml.

[jwodder]

You're missing the requires, obsoletes, and provides keywords (mentioned here in the Distutils docs), the ext_package keyword (mentioned here), and use_2to3_exclude_fixers (mentioned here in the Setuptools docs). Also missing is test_runner, a setuptools keyword that was apparently never documented.

[benoit-pierre]

requires is superseded by install_requires, and as obsoletes/provides are not supported by pip, I'm not sure we want to mention those.

[jwodder]

requires is superseded by install_requires, and as obsoletes/provides are not supported by pip, I'm not sure we want to mention those.

They are still valid setup() arguments and should be documented. At the very least, the documentation should state why you're not supposed to use them so that the few people who currently do use them have an authoritative source to be directed to.

Another keyword missing from this PR: long_description_content_type (currently only documented for use in setup.cfg, though it can be used in setup() as well)

[pganssle]

I agree with @jwodder about what should and should not be documented, but given that those things are already undocumented, I don't think we need to block @venthur's PR on getting them added. I tend to think that we should merge a PR as long as it's an improvement on the current state of affairs even if it doesn't perfectly fix the problem it's attempting to solve.

That said, I have three concerns here:

1. The way "Supported setup.py keywords" sits in the table of contents doesn't seem right to me. It seems like it should be a sub-page of setuptools.txt that is just broken out into its own page.
2. I'd like to reduce the separation between setup.cfg and setup.py keywords, because they are just two ways to specify the same data. Moving the setup.py keywords without also moving the setup.cfg table (or otherwise specifying the setup.cfg equivalent for each keyword) cuts against this effort.
3. If a bunch of this information is copied from distutils' documentation, what do we need to do to comply with the license the text is released under? I think the PSF license is similar to a BSD-style license, so we may have some obligation to note somewhere that some of this text is at least a derivative work of the distutils documentation.

It is possible we're already doing the third thing and I just don't know about it, I just thought I'd bring it up in this ticket so we don't accidentally fall out of compliance.

[venthur]

I've added the missing keywords (also the ones that are not supported anymore for completeness.

@jwodder How exactly you want me to re-structure the file, heading, etc? I'm open to suggestions but if you have a formulation I can copy-paste -- that would be the easiest :) You can also push to my branch directly if that's the fastest way to proceed.

Regarding the licensing, I wouldn't bother too much. First of all, I didn't just copy paste from distutils, I reformulated the explanation so they match with what you had before. But if you want to be bullet proof, you could add a file somewhere in this repo, explaining that some parts of setuptools are based/copied from distutils which is licensed under ...

[venthur]

So I've re-created the old structure. The keywords are in a separate file but includeed in the main document.

[benoit-pierre]

We actually use the deprecated by PEP 314 fields Provides / Obsoletes when writing metadata, instead of Provides-Dist / Obsoletes-Dist, the core difference being:

Each entry contains a string describing a package or module...

The XXX-Dist version of the fields only allow a project name.

So we should deprecate the use of provides / obsoletes and mark them at such in the documentation (in fact we already have a ticket for that: #1569). And if we do support the PEP 314 fields at some points, we should do so with the addition of 2 new setup keywords (provides_dists / obsoletes_dists?).

[pganssle]

@benoit-pierre Is there any way for us to detect whether something is a package or module when constructing the metadata? If so, I think it might be easier for us to detect modules and just deprecate the use of provides/obsoletes for modules, rather than renaming them.

If not, it seems somewhat excessive to add new keywords and drop the old ones just to notify people not to put module names there (since most people probably aren't doing that anyway).

@venthur I was originally thinking we should put these keywords on their own page, just not with the top-level link to them, but I think it might be better to do a total re-org of the docs at some point, which would mean additional links would have a pretty short lifetime so maybe you are right that having it directly in-line is better?

I'll take a closer look at this soon-ish and see what I think. Sorry for the delay.

[benoit-pierre]

I'd prefer if we used 2 new dedicated keywords, instead of trying to be clever about it. Be explicit. This will also simplify the documentation, as we can just link to the relevant PEP section, instead of copy pasting and merging to different PEPs.

[venthur]

Any updates on this? This was quite a lot of work and it would be a shame to let it rot.

[jaraco]

I've re-synced the PR. Unless there's objection, I recommend to merge this change and iterate from there.

[venthur]

this branch gets more and more difficult to maintain. can we please merge?

[layday]

This is also missing libraries which I believe is used by build_clib but I can't find where this was ever documented (see https://stackoverflow.com/a/49277244) and I've no idea how setup options are derived in the source code.