Merge branch 'main' into interp-call-expansion

Author: ericsnowcurrently

.devcontainer/devcontainer.json

@@ -1,5 +1,5 @@
 {
-    "image": "ghcr.io/python/devcontainer:2024.09.25.11038928730",
+    "image": "ghcr.io/python/devcontainer:2025.05.25.15232270922",
     "onCreateCommand": [
         // Install common tooling.
         "dnf",

Doc/c-api/init_config.rst

@@ -2111,7 +2111,7 @@ initialization::
 
         /* Specify sys.path explicitly */
         /* If you want to modify the default set of paths, finish
-           initialization first and then use PySys_GetObject("path") */
+           initialization first and then use PySys_GetAttrString("path") */
         config.module_search_paths_set = 1;
         status = PyWideStringList_Append(&config.module_search_paths,
                                          L"/path/to/stdlib");

Doc/c-api/intro.rst

@@ -838,3 +838,41 @@ after every statement run by the interpreter.)
 
 Please refer to :file:`Misc/SpecialBuilds.txt` in the Python source distribution
 for more detailed information.
+
+
+.. _c-api-tools:
+
+Recommended third party tools
+=============================
+
+The following third party tools offer both simpler and more sophisticated
+approaches to creating C, C++ and Rust extensions for Python:
+
+* `Cython <https://cython.org/>`_
+* `cffi <https://cffi.readthedocs.io>`_
+* `HPy <https://hpyproject.org/>`_
+* `nanobind <https://github.com/wjakob/nanobind>`_ (C++)
+* `Numba <https://numba.pydata.org/>`_
+* `pybind11 <https://pybind11.readthedocs.io/>`_ (C++)
+* `PyO3 <https://pyo3.rs/>`_ (Rust)
+* `SWIG <https://www.swig.org>`_
+
+Using tools such as these can help avoid writing code that is tightly bound to
+a particular version of CPython, avoid reference counting errors, and focus
+more on your own code than on using the CPython API. In general, new versions
+of Python can be supported by updating the tool, and your code will often use
+newer and more efficient APIs automatically. Some tools also support compiling
+for other implementations of Python from a single set of sources.
+
+These projects are not supported by the same people who maintain Python, and
+issues need to be raised with the projects directly. Remember to check that the
+project is still maintained and supported, as the list above may become
+outdated.
+
+.. seealso::
+
+   `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/guides/packaging-binary-extensions/>`_
+      The Python Packaging User Guide not only covers several available
+      tools that simplify the creation of binary extensions, but also
+      discusses the various reasons why creating an extension module may be
+      desirable in the first place.

Doc/c-api/sys.rst

@@ -258,10 +258,57 @@ These are utility functions that make functionality from the :mod:`sys` module
 accessible to C code.  They all work with the current interpreter thread's
 :mod:`sys` module's dict, which is contained in the internal thread state structure.
 
+.. c:function:: PyObject *PySys_GetAttr(PyObject *name)
+
+   Get the attribute *name* of the :mod:`sys` module.
+   Return a :term:`strong reference`.
+   Raise :exc:`RuntimeError` and return ``NULL`` if it does not exist or
+   if the :mod:`sys` module cannot be found.
+
+   If the non-existing object should not be treated as a failure, you can use
+   :c:func:`PySys_GetOptionalAttr` instead.
+
+   .. versionadded:: next
+
+.. c:function:: PyObject *PySys_GetAttrString(const char *name)
+
+   This is the same as :c:func:`PySys_GetAttr`, but *name* is
+   specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
+   rather than a :c:expr:`PyObject*`.
+
+   If the non-existing object should not be treated as a failure, you can use
+   :c:func:`PySys_GetOptionalAttrString` instead.
+
+   .. versionadded:: next
+
+.. c:function:: int PySys_GetOptionalAttr(PyObject *name, PyObject **result)
+
+   Variant of :c:func:`PySys_GetAttr` which doesn't raise
+   exception if the object does not exist.
+
+   * Set *\*result* to a new :term:`strong reference` to the object and
+     return ``1`` if the object exists.
+   * Set *\*result* to ``NULL`` and return ``0`` without setting an exception
+     if the object does not exist.
+   * Set an exception, set *\*result* to ``NULL``, and return ``-1``,
+     if an error occurred.
+
+   .. versionadded:: next
+
+.. c:function:: int PySys_GetOptionalAttrString(const char *name, PyObject **result)
+
+   This is the same as :c:func:`PySys_GetOptionalAttr`, but *name* is
+   specified as a :c:expr:`const char*` UTF-8 encoded bytes string,
+   rather than a :c:expr:`PyObject*`.
+
+   .. versionadded:: next
+
 .. c:function:: PyObject *PySys_GetObject(const char *name)
 
-   Return the object *name* from the :mod:`sys` module or ``NULL`` if it does
-   not exist, without setting an exception.
+   Similar to :c:func:`PySys_GetAttrString`, but return a :term:`borrowed
+   reference` and return ``NULL`` *without* setting exception on failure.
+
+   Preserves exception that was set before the call.
 
 .. c:function:: int PySys_SetObject(const char *name, PyObject *v)
 

Doc/c-api/unicode.rst

@@ -1726,10 +1726,6 @@ They all return ``NULL`` or ``-1`` if an exception occurs.
    from user input, prefer calling :c:func:`PyUnicode_FromString` and
    :c:func:`PyUnicode_InternInPlace` directly.
 
-   .. impl-detail::
-
-      Strings interned this way are made :term:`immortal`.
-
 
 .. c:function:: unsigned int PyUnicode_CHECK_INTERNED(PyObject *str)
 
@@ -1806,9 +1802,24 @@ object.
 
    See also :c:func:`PyUnicodeWriter_DecodeUTF8Stateful`.
 
+.. c:function:: int PyUnicodeWriter_WriteASCII(PyUnicodeWriter *writer, const char *str, Py_ssize_t size)
+
+   Write the ASCII string *str* into *writer*.
+
+   *size* is the string length in bytes. If *size* is equal to ``-1``, call
+   ``strlen(str)`` to get the string length.
+
+   *str* must only contain ASCII characters. The behavior is undefined if
+   *str* contains non-ASCII characters.
+
+   On success, return ``0``.
+   On error, set an exception, leave the writer unchanged, and return ``-1``.
+
+   .. versionadded:: next
+
 .. c:function:: int PyUnicodeWriter_WriteWideChar(PyUnicodeWriter *writer, const wchar_t *str, Py_ssize_t size)
 
-   Writer the wide string *str* into *writer*.
+   Write the wide string *str* into *writer*.
 
    *size* is a number of wide characters. If *size* is equal to ``-1``, call
    ``wcslen(str)`` to get the string length.

Doc/data/stable_abi.dat

@@ -204,17 +204,32 @@ value must be in a particular range or must satisfy other conditions,
 :c:data:`PyExc_ValueError` is appropriate.
 
 You can also define a new exception that is unique to your module.
-For this, you can declare a static global object variable at the beginning
-of the file::
+The simplest way to do this is to declare a static global object variable at
+the beginning of the file::
 
-   static PyObject *SpamError;
+   static PyObject *SpamError = NULL;
 
-and initialize it with an exception object in the module's
+and initialize it by calling :c:func:`PyErr_NewException` in the module's
 :c:data:`Py_mod_exec` function (:c:func:`!spam_module_exec`)::
 
+   SpamError = PyErr_NewException("spam.error", NULL, NULL);
+
+Since :c:data:`!SpamError` is a global variable, it will be overwitten every time
+the module is reinitialized, when the :c:data:`Py_mod_exec` function is called.
+
+For now, let's avoid the issue: we will block repeated initialization by raising an
+:py:exc:`ImportError`::
+
+   static PyObject *SpamError = NULL;
+
    static int
    spam_module_exec(PyObject *m)
    {
+       if (SpamError != NULL) {
+           PyErr_SetString(PyExc_ImportError,
+                           "cannot initialize spam module more than once");
+           return -1;
+       }
        SpamError = PyErr_NewException("spam.error", NULL, NULL);
        if (PyModule_AddObjectRef(m, "SpamError", SpamError) < 0) {
            return -1;
@@ -253,6 +268,11 @@ needed to ensure that it will not be discarded, causing :c:data:`!SpamError` to
 become a dangling pointer. Should it become a dangling pointer, C code which
 raises the exception could cause a core dump or other unintended side effects.
 
+For now, the :c:func:`Py_DECREF` call to remove this reference is missing.
+Even when the Python interpreter shuts down, the global :c:data:`!SpamError`
+variable will not be garbage-collected. It will "leak".
+We did, however, ensure that this will happen at most once per process.
+
 We discuss the use of :c:macro:`PyMODINIT_FUNC` as a function return type later in this
 sample.
 

Doc/extending/extending.rst

@@ -26,19 +26,9 @@ Recommended third party tools
 =============================
 
 This guide only covers the basic tools for creating extensions provided
-as part of this version of CPython. Third party tools like
-`Cython <https://cython.org/>`_, `cffi <https://cffi.readthedocs.io>`_,
-`SWIG <https://www.swig.org>`_ and `Numba <https://numba.pydata.org/>`_
-offer both simpler and more sophisticated approaches to creating C and C++
-extensions for Python.
-
-.. seealso::
-
-   `Python Packaging User Guide: Binary Extensions <https://packaging.python.org/guides/packaging-binary-extensions/>`_
-      The Python Packaging User Guide not only covers several available
-      tools that simplify the creation of binary extensions, but also
-      discusses the various reasons why creating an extension module may be
-      desirable in the first place.
+as part of this version of CPython. Some :ref:`third party tools
+<c-api-tools>` offer both simpler and more sophisticated approaches to creating
+C and C++ extensions for Python.
 
 
 Creating extensions without third party tools

Doc/extending/index.rst

@@ -37,24 +37,9 @@ Writing C is hard; are there any alternatives?
 ----------------------------------------------
 
 There are a number of alternatives to writing your own C extensions, depending
-on what you're trying to do.
-
-.. XXX make sure these all work
-
-`Cython <https://cython.org>`_ and its relative `Pyrex
-<https://www.csse.canterbury.ac.nz/greg.ewing/python/Pyrex/>`_ are compilers
-that accept a slightly modified form of Python and generate the corresponding
-C code.  Cython and Pyrex make it possible to write an extension without having
-to learn Python's C API.
-
-If you need to interface to some C or C++ library for which no Python extension
-currently exists, you can try wrapping the library's data types and functions
-with a tool such as `SWIG <https://www.swig.org>`_.  `SIP
-<https://github.com/Python-SIP/sip>`__, `CXX
-<https://cxx.sourceforge.net/>`_ `Boost
-<https://www.boost.org/libs/python/doc/index.html>`_, or `Weave
-<https://github.com/scipy/weave>`_ are also
-alternatives for wrapping C++ libraries.
+on what you're trying to do. :ref:`Recommended third party tools <c-api-tools>`
+offer both simpler and more sophisticated approaches to creating C and C++
+extensions for Python.
 
 
 How can I execute arbitrary Python statements from C?

Doc/faq/extending.rst

@@ -14,13 +14,11 @@ We recommend the following resources for porting extension modules to Python 3:
   module.
 * The `Porting guide`_ from the *py3c* project provides opinionated
   suggestions with supporting code.
-* The `Cython`_ and `CFFI`_ libraries offer abstractions over
-  Python's C API.
+* :ref:`Recommended third party tools <c-api-tools>` offer abstractions over
+  the Python's C API.
   Extensions generally need to be re-written to use one of them,
   but the library then handles differences between various Python
   versions and implementations.
 
 .. _Migrating C extensions: http://python3porting.com/cextensions.html
 .. _Porting guide: https://py3c.readthedocs.io/en/latest/guide.html
-.. _Cython: https://cython.org/
-.. _CFFI: https://cffi.readthedocs.io/en/latest/

Doc/howto/cporting.rst

@@ -161,6 +161,8 @@ your terminal won't be left in a funny state on exception and you'll be
 able to read the exception's message and traceback.
 
 
+.. _windows-and-pads:
+
 Windows and Pads
 ================
 

Doc/howto/curses.rst

@@ -171,13 +171,17 @@ and work with streams:
 .. function:: start_unix_server(client_connected_cb, path=None, \
                  *, limit=None, sock=None, backlog=100, ssl=None, \
                  ssl_handshake_timeout=None, \
-                 ssl_shutdown_timeout=None, start_serving=True)
+                 ssl_shutdown_timeout=None, start_serving=True, cleanup_socket=True)
    :async:
 
    Start a Unix socket server.
 
    Similar to :func:`start_server` but works with Unix sockets.
 
+   If *cleanup_socket* is true then the Unix socket will automatically
+   be removed from the filesystem when the server is closed, unless the
+   socket has been replaced after the server has been created.
+
    See also the documentation of :meth:`loop.create_unix_server`.
 
    .. note::
@@ -198,6 +202,9 @@ and work with streams:
    .. versionchanged:: 3.11
       Added the *ssl_shutdown_timeout* parameter.
 
+   .. versionchanged:: 3.13
+      Added the *cleanup_socket* parameter.
+
 
 StreamReader
 ============

Doc/library/asyncio-stream.rst

@@ -2031,35 +2031,55 @@ Utility functions
    pointer.
 
 
-.. function:: create_string_buffer(init_or_size, size=None)
+.. function:: create_string_buffer(init, size=None)
+              create_string_buffer(size)
 
    This function creates a mutable character buffer. The returned object is a
    ctypes array of :class:`c_char`.
 
-   *init_or_size* must be an integer which specifies the size of the array, or a
-   bytes object which will be used to initialize the array items.
+   If *size* is given (and not ``None``), it must be an :class:`int`.
+   It specifies the size of the returned array.
 
-   If a bytes object is specified as first argument, the buffer is made one item
-   larger than its length so that the last element in the array is a NUL
-   termination character. An integer can be passed as second argument which allows
-   specifying the size of the array if the length of the bytes should not be used.
+   If the *init* argument is given, it must be :class:`bytes`. It is used
+   to initialize the array items. Bytes not initialized this way are
+   set to zero (NUL).
+
+   If *size* is not given (or if it is ``None``), the buffer is made one element
+   larger than *init*, effectively adding a NUL terminator.
+
+   If both arguments are given, *size* must not be less than ``len(init)``.
+
+   .. warning::
+
+      If *size* is equal to ``len(init)``, a NUL terminator is
+      not added. Do not treat such a buffer as a C string.
+
+   For example::
+
+      >>> bytes(create_string_buffer(2))
+      b'\x00\x00'
+      >>> bytes(create_string_buffer(b'ab'))
+      b'ab\x00'
+      >>> bytes(create_string_buffer(b'ab', 2))
+      b'ab'
+      >>> bytes(create_string_buffer(b'ab', 4))
+      b'ab\x00\x00'
+      >>> bytes(create_string_buffer(b'abcdef', 2))
+      Traceback (most recent call last):
+         ...
+      ValueError: byte string too long
 
    .. audit-event:: ctypes.create_string_buffer init,size ctypes.create_string_buffer
 
 
-.. function:: create_unicode_buffer(init_or_size, size=None)
+.. function:: create_unicode_buffer(init, size=None)
+              create_unicode_buffer(size)
 
    This function creates a mutable unicode character buffer. The returned object is
    a ctypes array of :class:`c_wchar`.
 
-   *init_or_size* must be an integer which specifies the size of the array, or a
-   string which will be used to initialize the array items.
-
-   If a string is specified as first argument, the buffer is made one item
-   larger than the length of the string so that the last element in the array is a
-   NUL termination character. An integer can be passed as second argument which
-   allows specifying the size of the array if the length of the string should not
-   be used.
+   The function takes the same arguments as :func:`~create_string_buffer` except
+   *init* must be a string and *size* counts :class:`c_wchar`.
 
    .. audit-event:: ctypes.create_unicode_buffer init,size ctypes.create_unicode_buffer