[C API] The internal C API <pycore_dict.h> cannot be used in C++

[P403n1x87]

Bug report

Checklist

• I am confident this is a bug in CPython, not a bug in a third-party project
• I have searched the CPython issue tracker,
  and am confident this bug has not been reported before

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

3.12.0rc1

A clear and concise description of the bug:

This looks like a new instance of #67832.

@vstinner This seems to occur in 3.12 when importing e.g. pycore_dict.h in a C++ module https://github.com/P403n1x87/echion/actions/runs/5775912109/job/16065738879

In file included from /opt/hostedtoolcache/Python/3.12.0-rc.1/x64/include/python3.12/internal/pycore_atomic.h:15,
                       from /opt/hostedtoolcache/Python/3.12.0-rc.1/x64/include/python3.12/internal/pycore_runtime.h:12,
                       from /opt/hostedtoolcache/Python/3.12.0-rc.1/x64/include/python3.12/internal/pycore_dict.h:13,
                       from ./echion/mirrors.h:14,
                       from echion/coremodule.cc:29:
      /usr/lib/gcc/x86_64-linux-gnu/9/include/stdatomic.h: At global scope:
      /usr/lib/gcc/x86_64-linux-gnu/9/include/stdatomic.h:40:9: error: ‘_Atomic’ does not name a type
         40 | typedef _Atomic _Bool atomic_bool;
            |         ^~~~~~~

https://github.com/P403n1x87/echion/blob/29275a13aea67a8d008471523f38bc4477499a98/echion/mirrors.h#L14

The same job with clang on OSX doesn't show the same issue: https://github.com/P403n1x87/echion/actions/runs/5775912109/job/16065739714.

Linked PRs

• gh-108216: Cleanup #include in internal header files #108228
• gh-108216: Add pycore_dict_struct.h internal header file #108235

[vstinner]

This looks like a new instance of #67832.

This old issue was about including the public C API <Python.h>. This issue is about using the internal C API in C++.

[vstinner]

What is your use case for using the internal C API?

https://github.com/P403n1x87/echion/ description:

Near-zero-overhead, in-process CPython frame stack sampler with async support

It looks like a Python profiler.

[vstinner]

In Python 3.11, I added test_cppext to check that the Python C API is usable in C++, but only the public C API.

[P403n1x87]

It looks like a Python profiler.

Indeed it is a profiler (actually just a frame stack sample fwiw). The internal API is used to access internal fields of structures from copies of Python objects so that it can all be done without the GIL

[vstinner]

With Python 3.11, it's possible to include <pycore_dict.h> in C++? In 3.11, there is no include.

Python 3.12 added two includes:

#include "pycore_dict_state.h"
#include "pycore_runtime.h"         // _PyRuntime

PEP 684: A Per-Interpreter GIL modified DICT_NEXT_VERSION() to access interp->dict_state.global_version: the counter is now "per-interpreter". The problem is that accessing PyInterprepter members requires to include pycore_interp.h is a giant beast: pycore_interp.h includes the problematic pycore_atomic.h.