Soft-deprecate parts of the undocumented C API

[ZeroIntensity]

Hi,

I've begun working on getting the C API documentation up to date: python/cpython#141004. Unfortunately, several parts of the undocumented C API aren't very helpful. For example, there's a bunch of undocumented Py*Iter_Type for the container classes. We don't really want users to start using these, but we can't just remove them either. @encukou and I believe that the best option is to soft-deprecate them.

The list of APIs is available below.

[skirpichev]

Please consider also

1. PyLong_BASE, PyLong_MASK and PyLong_SHIFT
2. Py_INFINITY/NAN
3. Py_MATH_PIl/Py_MATH_El could be just deprecated. Just one example I found on the Github was misuse of these macros.

Edit:
4. Py_RETURN_NAN/INF

[vstinner]

I ran a code searchs on PyPI top 15,000 projects (at 2025-09-22).

Py*Iter types

Regex: (PyODictIter_Type|PyByteArrayIter_Type|PyRangeIter_Type|PyLongRangeIter_Type|PySetIter_Type|PyTupleIter_Type|PyBytesIter_Type|PySeqIter_Type|PyCallIter_Type|PyListIter_Type|PyListRevIter_Type|PyUnicodeIter_Type)

It found a single project using 2 types: Nuitka (2.7.16) uses PySeqIter_Type and PyCallIter_Type.

Type objects for some builtin classes (PyEnum_Type, PyMap_Type, PyReversed_Type, PyZip_Type).

Regex: PyEnum_Type|PyMap_Type|PyReversed_Type|PyZip_Type

4 projects are using PyReversed_Type and PyEnum_Type:

• Nuitka (2.7.16)
• lazy_object_proxy (1.12.0)
• newrelic (10.17.0)
• wrapt (1.17.3)

Details

PYPI-2025-09-22/lazy_object_proxy-1.12.0.tar.gz: lazy_object_proxy-1.12.0/src/lazy_object_proxy/cext.c: return PyObject_CallFunctionObjArgs((PyObject *)&PyReversed_Type,
PYPI-2025-09-22/newrelic-10.17.0.tar.gz: newrelic-10.17.0/newrelic/packages/wrapt/_wrappers.c: return PyObject_CallFunctionObjArgs((PyObject *)&PyReversed_Type,
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/Serialization.py: return "(PyObject *)&PyEnum_Type"
PYPI-2025-09-22/wrapt-1.17.3.tar.gz: wrapt-1.17.3/src/wrapt/_wrappers.c: return PyObject_CallFunctionObjArgs((PyObject *)&PyReversed_Type,

PyRange* APIs and PyLongRangeIter_Type.

Regex: PyRange_Type|PyRangeIter_Type|PyRange_Check|PyLongRangeIter_Type.

4 projects are using PyRange_Type and PyRange_Check()

• Nuitka (2.7.16)
• datatable (1.1.0)
• mypy (1.18.2)
• mypy_dev-1.19.0a2

Details

PYPI-2025-09-22/datatable-1.1.0.tar.gz: datatable-1.1.0/src/core/python/obj.cc: bool _obj::is_range()         const noexcept { return v && PyRange_Check(v); }
PYPI-2025-09-22/datatable-1.1.0.tar.gz: datatable-1.1.0/src/core/python/range.cc: v = PyObject_CallObject(reinterpret_cast<PyObject*>(&PyRange_Type),
PYPI-2025-09-22/mypy-1.18.2.tar.gz: mypy-1.18.2/mypyc/primitives/misc_ops.py: load_address_op(name="builtins.range", type=object_rprimitive, src="PyRange_Type")
PYPI-2025-09-22/mypy-1.18.2.tar.gz: mypy-1.18.2/mypyc/test-data/irbuild-basic.test: r0 = load_address PyRange_Type
PYPI-2025-09-22/mypy_dev-1.19.0a2.tar.gz: mypy_dev-1.19.0a2/mypyc/primitives/misc_ops.py: load_address_op(name="builtins.range", type=object_rprimitive, src="PyRange_Type")
PYPI-2025-09-22/mypy_dev-1.19.0a2.tar.gz: mypy_dev-1.19.0a2/mypyc/test-data/irbuild-basic.test: r0 = load_address PyRange_Type
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/Serialization.py: return "(PyObject *)&PyRange_Type"
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/CompiledCodeHelpers.c: struct _rangeobject2 *result = (struct _rangeobject2 *)PyObject_New(struct _rangeobject2, &PyRange_Type);
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/CompiledCodeHelpers.c: struct _rangeobject3 *result = (struct _rangeobject3 *)PyObject_New(struct _rangeobject3, &PyRange_Type);
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/CompiledCodeHelpers.c: struct _rangeobject3 *result = (struct _rangeobject3 *)PyObject_New(struct _rangeobject3, &PyRange_Type);
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/HelpersDeepcopy.c: PyDict_SetItem(_deep_copy_dispatch, (PyObject *)&PyRange_Type, _deep_noop);
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/HelpersDeepcopy.c: PyBytes_CheckExact(value) || PyRange_Check(value) || PyType_Check(value) || PySlice_Check(value) ||
PYPI-2025-09-22/Nuitka-2.7.16.tar.gz: Nuitka-2.7.16/nuitka/build/static_src/HelpersDeepcopy.c: PyBool_Check(value) || PyRange_Check(value) || PySlice_Check(value) || PyCFunction_Check(value)) {

[vstinner]

Soft-deprecate parts of the undocumented C API

If an API is unused and we don't want to document it, I would prefer to hard deprecate it, rather than soft deprecating it.

Py*Iter types.

We can deprecate PyODictIter_Type, PyByteArrayIter_Type, PyRangeIter_Type, PyLongRangeIter_Type, PySetIter_Type, PyTupleIter_Type, PyBytesIter_Type, PyListIter_Type, PyListRevIter_Type, PyUnicodeIter_Type.

(I would prefer to not deprecate PySeqIter_Type and PyCallIter_Type which are used by Nuitka.)

Type objects for some builtin classes (PyEnum_Type, PyMap_Type, PyReversed_Type, PyZip_Type).

We can deprecate PyMap_Type and PyZip_Type.

PyRange* APIs and PyLongRangeIter_Type.

We can deprecate PyRangeIter_Type and PyLongRangeIter_Type.

Anything under pyport.h.

Would you mind to elaborate? Many pyport.h macros are needed to implement the C API (in header files). For example, _Py_TYPEOF() is used by Py_CLEAR(), and _Py_NO_RETURN is used to defined Py_FatalError().

Internal type flags (Py_TPFLAGS_HAVE_VERSION_TAG, Py_TPFLAGS_INLINE_VALUES, Py_TPFLAGS_IS_ABSTRACT).

I don't think that we should deprecate them.

Macros exposing implementation details (there are a lot of these).

Please elaborate.

PyLong_FromPid & PyLong_AsPid, in favour of PyLong_AsNativeBytes with sizeof(pid_t)

Is it really an issue to keep these 2 functions?

[vstinner]

PyLong_BASE, PyLong_MASK and PyLong_SHIFT

Python 3.14 just added PyLong_GetNativeLayout(). It might be early to deprecate these 3 macros.

Py_INFINITY/NAN

Py_NAN looks special, it has special code for Solaris.

For Py_INFINITY, it looks reasonable to use INFINITY instead.

Py_MATH_PIl/Py_MATH_El could be just deprecated. Just one example I found on the Github was misuse of these macros.

Ok to deprecate these two macros.

[ZeroIntensity]

Nuitka copies our internal APIs for optimization's sake, so they're probably just copying the code for things like range.__new__ and similar. And, as long as it's a soft deprecation, Nuitka won't care anyway.

Would you mind to elaborate? Many pyport.h macros are needed to implement the C API (in header files). For example, _Py_TYPEOF() is used by Py_CLEAR(), and _Py_NO_RETURN is used to defined Py_FatalError().

Sorry, I meant deprecating anything under pyport.h with a Py prefix (implying they're public). For example, Py_ALIGNED, Py_MEMCPY, Py_CAN_START_THREADS.

I don't think that we should deprecate them.

Why not? Users shouldn't touch these.

Please elaborate.

Things such as PySet_MINSIZE expose implementation details.

[vstinner]

Sorry, I meant deprecating anything under pyport.h with a Py prefix (implying they're public). For example, Py_ALIGNED, Py_MEMCPY, Py_CAN_START_THREADS.

Py_MEMCPY() is already soft deprecated.

Py_ALIGNED() doesn't seem to be used by Python anymore: _Py_ALIGNED_DEF() is used instead. @encukou may have an opinion on this macro.

Py_CAN_START_THREADS looks useful.

I don't think that we should deprecate them.

Why not? Users shouldn't touch these.

Debuggers and profilers might want to inspect these flags (Py_TPFLAGS_HAVE_VERSION_TAG, Py_TPFLAGS_INLINE_VALUES, Py_TPFLAGS_IS_ABSTRACT). IMO it's too late to move them to the internal C API.

Things such as PySet_MINSIZE expose implementation details.

It's needed to define PySetObject structure. Since PySetObject is part of the public C API, I don't think that exposing PySet_MINSIZE is an issue.

[skirpichev]

Python 3.14 just added PyLong_GetNativeLayout(). It might be early to deprecate these 3 macros.

It depends. Now PEP 387 recommends 5 years before removal, i.e. something like 3.20. Though, I suggest do soft-deprecation.

Py_NAN looks special, it has special code for Solaris.

It seems this is for system, which doesn't fit in build requirements. See python/cpython#141004 for details. Do we want to relax build requirements?

[ZeroIntensity]

I've updated the list based on Victor's feedback.

[encukou]

Soft-deprecate parts of the undocumented C API

If an API is unused and we don't want to document it, I would prefer to hard deprecate it, rather than soft deprecating it.

Soft-deprecation requires writing docs, so, yes, we'd need to do that.
Though I'd still generally prefer documenting APIs as part of normal deprecation, especially if we know use cases of the deprecated API. When people get deprecation warnings they should go search the docs for advice.

PyLong_FromPid & PyLong_AsPid, in favour of PyLong_AsNativeBytes with sizeof(pid_t)

Is it really an issue to keep these 2 functions?

Definitely keep them, but I think soft-deprecation is a good way to document that we don't want to add similar new API, and that there's a replacement. Basically, avoid people pointing to it as a precedent.

Py_ALIGNED() doesn't seem to be used by Python anymore: _Py_ALIGNED_DEF() is used instead. @encukou may have an opinion on this macro.

Soft-deprecation is the way to go here. Alignment control is in C now; you should use that directly if you (unlike Python) can.

Things such as PySet_MINSIZE expose implementation details.

It's needed to define PySetObject structure. Since PySetObject is part of the public C API, I don't think that exposing PySet_MINSIZE is an issue.

We can use the _PySet_MINSIZE name for that.

[StanFromIreland]

Dear C-API WG,

please consider soft deprecating the following macros defined in cpython/pyctype.h:

Py_ISALNUM
Py_ISALPHA
Py_ISDIGIT
Py_ISLOWER
Py_ISSPACE
Py_ISUPPER
Py_ISUPPER
Py_TOLOWER
Py_TOUPPER

[malemburg]

please consider soft deprecating the following macros defined in cpython/pyctype.h:

Py_ISALNUM ...

Why do you think these should be soft deprecated ?

Those macros provide a good way to get locale independent Latin-1 character type information. The C lib variants usually are locale aware, which is not always what you want when parsing text based on a fixed set of rules.

[StanFromIreland]

It does not seem like something for CPython to provide, it should be a private internal (we actually call it „internal” in the docs already) utility IMO.