Mention alternatives to discover plugins via entry_points

[ashwinvis]

In "Creating and discovering plugins" guide, the recommendation is to use pkg_resources.iter_entry_points. In https://github.com/takluyver/entrypoints it is noted that:

The pkg_resources module distributed with setuptools provides a way to discover entrypoints as well, but it contains other functionality unrelated to entrypoint discovery, and it does a lot of work at import time. Merely importing pkg_resources causes it to scan the files of all installed packages. Thus, in environments where a large number of packages are installed, importing pkg_resources can be very slow (several seconds).

I agree with this observation too. Now with the standard library importlib.metadata feature landing in Python 3.10+ and its backport, this recommendation can be updated with more lighter:

import sys
if sys.version_info[:2] < (3, 10):
    from backports.entry_points_selectable import entry_points
else:
    from importlib.metadata import entry_points

...
entry_points(group='myapp.plugins')

[webknjaz]

@ashwinvis do you want to draft a PR for this?

[henryiii]

Shouldn't you just require importlib_metadata>3.6 for < Python 3.10? Ah, maybe it's because the normal backport dropped support for Python 2.7 and 3.5?

[layday]

There's a new entry point interface coming in Python 3.10. backports.entry_points_selectable backports this to Python < 3.10 without requiring the whole of the importlib_metadata, uh... backport under Python 3.8 and 3.9. The 3.6 version of importlib_metadata and the version that'll ship with 3.10 deprecate the older entry point mapping interface - that is, key access, iteration, etc. will emit a deprecation warning.

[henryiii]

TLDR, for modern entry points:

• Python 2.7 and 3.5 are supported by backports.entry_points_selectable and not by any version of importlib_metadata
• Python 3.6-3.7 it internally requires importlib_metadata 3.6+ anyway
• Python 3.8-3.9 you can avoid including importlib_metadata (though it's just a pure Python package, so likely not a huge deal, but a minor savings?)

Obviously, after that you can just use the stdlib. Did I summarize correctly? Such a summary might be useful on the main page of backports.entry_points_selectable, @jaraco ? The current readme doesn't really clarify why you'd use it if you are okay with requiring importlib_metadata>=3.6, which I'd assume anyone adding this requirement would also be fine with doing.

[layday]

You'd use it if you support Python 3.5 and earlier and want to use the new interface or avoid deprecation warnings since importlib_metadata 3.6 requires Python 3.6 or later.

[layday]

This is a lot of unnecessary complexity and the guides and the tutorials should not recommend the entry points backport.

[jaraco]

Did I summarize correctly?

* Python 2.7 and 3.5 are supported by `backports.entry_points_selectable` and not by any version of `importlib_metadata`

I don't understand "and not by any version of importlib_metadata". importlib_metadata<3 supports Python 2.7 and 3.5. importlib_metadata < 1.2 supports Python 3.4.

• Python 3.6-3.7 it internally requires importlib_metadata 3.6+ anyway

By "it", I presume you mean backports.entry_points_selectable? That's not entirely true. Yes, in a clean environment, loading from PyPI, pip will pick up the latest release of importlib_metadata, but it only requires importlib_metadata (any version). If the environment already has importlib_metadata 1.2 installed, backports.entry_points_selectable will be satisfied by that older implementation.

* Python 3.8-3.9 you can avoid including `importlib_metadata` (though it's just a pure Python package, so likely not a huge deal, but a minor savings?)

I agree with this sentiment, but some package maintainers have pushed back on this requirement. They don't want to impose an additional dependency and constraint if they don't have to.

Obviously, after that you can just use the stdlib.

Almost. At Python 3.10a7 (pending release) you can. 3.10a6 still requires importlib_metadata>=3.6 or backports.entry_point_selectable.

Such a summary might be useful on the main page of backports.entry_points_selectable.

Agreed. I attempted to summarize the concerns in the readme, but it's clear I could explain better, maybe with examples to clarify.

The current readme doesn't really clarify why you'd use it if you are okay with requiring importlib_metadata>=3.6, which I'd assume anyone adding this requirement would also be fine with doing.

Agreed, I expect the vast majority of consumers to simply adopt importlib_metadata >= 3.6 irrespective of environment or at least on Python 3.6-3.10a6. Some libraries still support Python 2.7 and 3.5 and those need the backport.

But there's another concern, something I also hadn't considered until very recently. Some packages are not applications, but libraries, and libraries need to be somewhat careful about the dependencies they include, because if they include a constraint on a dependency that can't be met in some environment (perhaps because another dependency depends more strictly on a lower version or because the dependency is satisfied through some other constraint in the environment such as a distro package that's not upgradable), the library becomes incompatible with that environment.

backports.entry_points_selectable exists to allow libraries to avoid constraining their dependency on importlib_metadata and continue to avoid requiring it at all on Python 3.8 and 3.9 if that's what they wish.

This is a lot of unnecessary complexity and the guides and the tutorials should not recommend the entry points backport.

Agreed. Focus should be on the simple, broad use-case of importlib_metadata>=3.6 on non-sunset versions of Python (or maybe native on Python 3.10a7+).

[henryiii]

Some packages are not applications, but libraries, and libraries need to be somewhat careful about the dependencies they include, because if they include a constraint on a dependency that can't be met in some environment

I need you to help me argue with Poetry's devspython-poetry/poetry#3757 that they should not be pushing everyone to set upper limits on everything, a library should not be setting something like click>=7, click<8. :)

I would clarify the above points in the readme. Also, it should be quite rare that a library would set an upper limit on importlib_metadata (unless they are a Poetry user and don't know better that go against the docs and defaults 😠), so having the clarification should help. On my Python 3.6+ projects (and I may about to go 3.7+ in one! Woohoo, annotations!), I'd generally be happy to set importlib_metadata 3.6 as the minimum. I still have some 2.7/3.5 projects, so there it would be useful; I should be able to gather the info to make that desision from the README. (Not collecting entry points in that many projects, though, just a general observation).

For the guide here, giving the recommendation for 3.6+ (Python and importlib_metadata ;) ) sounds just fine to me.