bpo-38159: Clarify documentation of PyState_AddModule

[encukou]

This was never intented to be called manually from PyInit_*.

Also, clarify PyState_RemoveModule return value.

https://bugs.python.org/issue38159

Automerge-Triggered-By: @encukou

Automerge-Triggered-By: @encukou

[eduardo-elizondo]

Just to clarify "implementing an alternative import mechanism" meant someone changing __import__ right? Otherwise, could you expand a bit more on what you meant by that? :)

Btw for context, these were added since the documentation currently specifies that PyState_FindModule needs to be paired up with PyState_AddModule to work. Since the import documentation does not offer a guarantee that your module will be attached to the interpreter state (through PyState_FindModule), it seemed like the right call to have that in the init functions.

As a matter of fact, in our system we do rely on this signal to correctly work with PyState_FindModule. Our import mechanism does not do it by default.

Maybe an alternative can be to update the import documentation that it should offer this guarantee? On the other hand, it seems like that should be an implementation detail and it's best to let the module themselves call the right APIs. Thoughts?

[encukou]

Changing __import__, creating module objects dynamically, things like that. I'm not 100% sure on this expected behavior myself, unfortunately. But it's currently redundant in the common case.

[encukou]

I asked over at at capi-sig.

[matrixise]

@encukou maybe you could add "DO-NOT-MERGE"?

[encukou]

Note to reviewer: See the capi-sig discussion

[encukou]

cc @ncoghlan

[eduardo-elizondo]

LGTM with a small comment

[eduardo-elizondo]

An alternate implementation might not necessarily call PyState_AddModule but still achieve the same end result. Perhaps just say that Python attaches the module to the interpreter state and should be accessible through PyState_FindModule. How they achieve that should be up to the implementation

[encukou]

All ways I can find of wording this more precisely make it significantly harder to understand.

I think it's understood that alternate implementations are free to do an equivalent of calling PyState_AddModule, instead of actually calling it. They should still implement the PyState_AddModule semantics (unless they decide to differ, of course).

Another thing that's implied here is that Python only calls this on single-phase-init modules. And another is that it's idempotent.

Should we document all that? How much hand-holding do authors of alternative import mechanisms need? (This is a real question, which you are in a good position to answer.)

[encukou]

But my main issue is:

All ways I can find of wording this more precisely make it significantly harder to understand.

If you can think of a better way, let me know! Otherwise I can merge this.

[ncoghlan]

Added a couple of specific suggestions that I think improve precision without making it too much harder to read.

[encukou]

Azure Pipelines didn't start, I'll try closing & reopening.

[miss-islington]

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8.
🐍🍒⛏🤖

[bedevere-bot]

GH-17026 is a backport of this pull request to the 3.8 branch.

[bedevere-bot]

GH-17027 is a backport of this pull request to the 3.7 branch.