Isolate the _datetime extension module

[neonene]

Feature or enhancement

Proposal:

I hope this issue will complete _datetime isolation.

My concerns (and answers)

• Py_MOD_PER_INTERPRETER_GIL_SUPPORTED should be applied in sync with _zoninfo?

  • YES: Possible.

• Can a module state have a C-API structure, keeping a capsule just for comatibility?

  • NO: Possible, but a user should not touch the module state.

• C-API supports only the main interpreter? Otherwise, PyInterpreterState is acceptable to point each structure?

  • NO: PyDateTimeAPI cannot emit an error. Also, no PyInterpreterState member is accessible from datetime.h. UPDATE: Seems to be possible by using a global function pointer instead of a function.

Specific issue:

• Initialize/Finalize Python multiple time and import datetime each time lead to a memory corruption #118608

Links to previous discussion of this feature:

• Isolate Stdlib Extension Modules #103092
• gh-71587: Isolate _datetime #102995
• gh-71587: Establish global state in _datetime #110475

Linked PRs (closed)

• gh-117398: datetime: Support per-interpreter C-API #117399
• gh-117398: datetime: Make use of C-API capsules for sub-interpreters #117413
• gh-117398: datetime: Use wrappers for C-API functions #118115
• gh-117398: datetime: Access C-API via PyInterpreterState #118357

Linked PRs

• gh-117398: Isolate _datetime (3.13.0a6 PoC) #117498
• gh-117398: Isolate _datetime (3.13.0a6 PoC) with AC #118337
• gh-117398: Move types to datetime state #118606
• gh-117398: Add multiphase support to _datetime #119373
• gh-117398: Statically Allocate the Datetime C-API #119472
• gh-117398: Add datetime C-API type check test for subinterpreters #119604
• [3.13] gh-117398: Add multiphase support to _datetime (gh-119373) #119636
• gh-117398: Convert datetime.IsoCalendarDate To A Heap Type #119637
• [3.13] gh-117398: Revert gh-119636, Add multiphase support to _datetime #119639
• gh-117398: Add datetime Module State #119640
• [3.13] gh-117398: Statically Allocate the Datetime C-API (GH-119472) #119641
• gh-117398: gh-119655: datetime: Init static state once & don't free it #119662
• [3.13] gh-117398: Add multiphase support to _datetime #119694
• [3.13] gh-117398: Convert datetime.IsoCalendarDate To A Heap Type (gh-119637) #119695
• gh-117398: Add datetime Module State #119810
• gh-117398: Use Per-Interpreter State for the _datetime Static Types #119929
• [3.13] gh-117398: Add datetime Module State (gh-119810) #120004
• [3.13] gh-117398: Use Per-Interpreter State for the _datetime Static Types (gh-119929) #120009
• gh-71587: Drop local reference cache to _strptime module in _datetime #120224
• [3.13] gh-117398: Add datetime C-API type check test for subinterpreters (gh-119604) #120463
• gh-117398: Use the correct module loader for iOS in datetime CAPI test. #120477
• [3.13] gh-117398: Use the correct module loader for iOS in datetime CAPI test (GH-120477) #121561

[neonene]

PR #117399 fails except Windows. Is there an easy way to access a PyInterpreterState member? I'll try later allowing C-API only for the main-interpreter with a global variable.

[erlend-aasland]

I briefly discussed the C API capsule with @pganssle on the 2023 (or was it 2022?) language summit. I also asked the Steering Council to comment about backwards compatibility concerns regarding datetime.h (which is not included by Python.h):

• Request for comment: datetime.h and PEP-387 steering-council#208

Putting it in the PyInterpreterState struct is an option, but I'm not sure how much we want to bloat that struct. We probably want to deprecate the current capsulated API and introduce a new capsulated API.