Merge branch 'main' into bitfields

Author: matthiasgoergens

Doc/conf.py

@@ -249,3 +249,18 @@
 # bpo-40204: Disable warnings on Sphinx 2 syntax of the C domain since the
 # documentation is built with -W (warnings treated as errors).
 c_warn_on_allowed_pre_v3 = False
+
+# Fix '!' not working with C domain when pre_v3 is enabled
+import sphinx
+
+if sphinx.version_info[:2] < (5, 3):
+    from sphinx.domains.c import CXRefRole
+
+    original_run = CXRefRole.run
+
+    def new_run(self):
+        if self.disabled:
+            return super(CXRefRole, self).run()
+        return original_run(self)
+
+    CXRefRole.run = new_run

Doc/howto/logging-cookbook.rst

@@ -3428,6 +3428,82 @@ the above handler, you'd pass structured data using something like this::
     i = 1
     logger.debug('Message %d', i, extra=extra)
 
+How to treat a logger like an output stream
+-------------------------------------------
+
+Sometimes, you need to interface to a third-party API which expects a file-like
+object to write to, but you want to direct the API's output to a logger. You
+can do this using a class which wraps a logger with a file-like API.
+Here's a short script illustrating such a class:
+
+.. code-block:: python
+
+    import logging
+
+    class LoggerWriter:
+        def __init__(self, logger, level):
+            self.logger = logger
+            self.level = level
+
+        def write(self, message):
+            if message != '\n':  # avoid printing bare newlines, if you like
+                self.logger.log(self.level, message)
+
+        def flush(self):
+            # doesn't actually do anything, but might be expected of a file-like
+            # object - so optional depending on your situation
+            pass
+
+        def close(self):
+            # doesn't actually do anything, but might be expected of a file-like
+            # object - so optional depending on your situation. You might want
+            # to set a flag so that later calls to write raise an exception
+            pass
+
+    def main():
+        logging.basicConfig(level=logging.DEBUG)
+        logger = logging.getLogger('demo')
+        info_fp = LoggerWriter(logger, logging.INFO)
+        debug_fp = LoggerWriter(logger, logging.DEBUG)
+        print('An INFO message', file=info_fp)
+        print('A DEBUG message', file=debug_fp)
+
+    if __name__ == "__main__":
+        main()
+
+When this script is run, it prints
+
+.. code-block:: text
+
+    INFO:demo:An INFO message
+    DEBUG:demo:A DEBUG message
+
+You could also use ``LoggerWriter`` to redirect ``sys.stdout`` and
+``sys.stderr`` by doing something like this:
+
+.. code-block:: python
+
+    import sys
+
+    sys.stdout = LoggerWriter(logger, logging.INFO)
+    sys.stderr = LoggerWriter(logger, logging.WARNING)
+
+You should do this *after* configuring logging for your needs. In the above
+example, the :func:`~logging.basicConfig` call does this (using the
+``sys.stderr`` value *before* it is overwritten by a ``LoggerWriter``
+instance). Then, you'd get this kind of result:
+
+.. code-block:: pycon
+
+    >>> print('Foo')
+    INFO:demo:Foo
+    >>> print('Bar', file=sys.stderr)
+    WARNING:demo:Bar
+    >>>
+
+Of course, these above examples show output according to the format used by
+:func:`~logging.basicConfig`, but you can use a different formatter when you
+configure logging.
 
 .. patterns-to-avoid:
 

Doc/library/ast.rst

@@ -1991,20 +1991,28 @@ and classes for traversing abstract syntax trees:
 
 .. function:: literal_eval(node_or_string)
 
-   Safely evaluate an expression node or a string containing a Python literal or
+   Evaluate an expression node or a string containing only a Python literal or
    container display.  The string or node provided may only consist of the
    following Python literal structures: strings, bytes, numbers, tuples, lists,
    dicts, sets, booleans, ``None`` and ``Ellipsis``.
 
-   This can be used for safely evaluating strings containing Python values from
-   untrusted sources without the need to parse the values oneself.  It is not
-   capable of evaluating arbitrarily complex expressions, for example involving
-   operators or indexing.
+   This can be used for evaluating strings containing Python values without the
+   need to parse the values oneself.  It is not capable of evaluating
+   arbitrarily complex expressions, for example involving operators or
+   indexing.
+
+   This function had been documented as "safe" in the past without defining
+   what that meant. That was misleading. This is specifically designed not to
+   execute Python code, unlike the more general :func:`eval`. There is no
+   namespace, no name lookups, or ability to call out. But it is not free from
+   attack: A relatively small input can lead to memory exhaustion or to C stack
+   exhaustion, crashing the process. There is also the possibility for
+   excessive CPU consumption denial of service on some inputs. Calling it on
+   untrusted data is thus not recommended.
 
    .. warning::
-      It is possible to crash the Python interpreter with a
-      sufficiently large/complex string due to stack depth limitations
-      in Python's AST compiler.
+      It is possible to crash the Python interpreter due to stack depth
+      limitations in Python's AST compiler.
 
       It can raise :exc:`ValueError`, :exc:`TypeError`, :exc:`SyntaxError`,
       :exc:`MemoryError` and :exc:`RecursionError` depending on the malformed

Doc/library/asyncio-future.rst

@@ -85,7 +85,8 @@ Future Object
 
    Future is an :term:`awaitable` object.  Coroutines can await on
    Future objects until they either have a result or an exception
-   set, or until they are cancelled.
+   set, or until they are cancelled. A Future can be awaited multiple
+   times and the result is same.
 
    Typically Futures are used to enable low-level
    callback-based code (e.g. in protocols implemented using asyncio

Doc/library/asyncio-task.rst

@@ -294,11 +294,13 @@ perform clean-up logic. In case :exc:`asyncio.CancelledError`
 is explicitly caught, it should generally be propagated when
 clean-up is complete. Most code can safely ignore :exc:`asyncio.CancelledError`.
 
-Important asyncio components, like :class:`asyncio.TaskGroup` and the
-:func:`asyncio.timeout` context manager, are implemented using cancellation
-internally and might misbehave if a coroutine swallows
-:exc:`asyncio.CancelledError`.
+The asyncio components that enable structured concurrency, like
+:class:`asyncio.TaskGroup` and :func:`asyncio.timeout`,
+are implemented using cancellation internally and might misbehave if
+a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code
+should not call :meth:`uncancel <asyncio.Task.uncancel>`.
 
+.. _taskgroups:
 
 Task Groups
 ===========
@@ -1003,76 +1005,6 @@ Task Object
       Deprecation warning is emitted if *loop* is not specified
       and there is no running event loop.
 
-   .. method:: cancel(msg=None)
-
-      Request the Task to be cancelled.
-
-      This arranges for a :exc:`CancelledError` exception to be thrown
-      into the wrapped coroutine on the next cycle of the event loop.
-
-      The coroutine then has a chance to clean up or even deny the
-      request by suppressing the exception with a :keyword:`try` ...
-      ... ``except CancelledError`` ... :keyword:`finally` block.
-      Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
-      not guarantee that the Task will be cancelled, although
-      suppressing cancellation completely is not common and is actively
-      discouraged.
-
-      .. versionchanged:: 3.9
-         Added the *msg* parameter.
-
-      .. deprecated-removed:: 3.11 3.14
-         *msg* parameter is ambiguous when multiple :meth:`cancel`
-         are called with different cancellation messages.
-         The argument will be removed.
-
-      .. _asyncio_example_task_cancel:
-
-      The following example illustrates how coroutines can intercept
-      the cancellation request::
-
-          async def cancel_me():
-              print('cancel_me(): before sleep')
-
-              try:
-                  # Wait for 1 hour
-                  await asyncio.sleep(3600)
-              except asyncio.CancelledError:
-                  print('cancel_me(): cancel sleep')
-                  raise
-              finally:
-                  print('cancel_me(): after sleep')
-
-          async def main():
-              # Create a "cancel_me" Task
-              task = asyncio.create_task(cancel_me())
-
-              # Wait for 1 second
-              await asyncio.sleep(1)
-
-              task.cancel()
-              try:
-                  await task
-              except asyncio.CancelledError:
-                  print("main(): cancel_me is cancelled now")
-
-          asyncio.run(main())
-
-          # Expected output:
-          #
-          #     cancel_me(): before sleep
-          #     cancel_me(): cancel sleep
-          #     cancel_me(): after sleep
-          #     main(): cancel_me is cancelled now
-
-   .. method:: cancelled()
-
-      Return ``True`` if the Task is *cancelled*.
-
-      The Task is *cancelled* when the cancellation was requested with
-      :meth:`cancel` and the wrapped coroutine propagated the
-      :exc:`CancelledError` exception thrown into it.
-
    .. method:: done()
 
       Return ``True`` if the Task is *done*.
@@ -1186,3 +1118,125 @@ Task Object
       in the :func:`repr` output of a task object.
 
       .. versionadded:: 3.8
+
+   .. method:: cancel(msg=None)
+
+      Request the Task to be cancelled.
+
+      This arranges for a :exc:`CancelledError` exception to be thrown
+      into the wrapped coroutine on the next cycle of the event loop.
+
+      The coroutine then has a chance to clean up or even deny the
+      request by suppressing the exception with a :keyword:`try` ...
+      ... ``except CancelledError`` ... :keyword:`finally` block.
+      Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
+      not guarantee that the Task will be cancelled, although
+      suppressing cancellation completely is not common and is actively
+      discouraged.
+
+      .. versionchanged:: 3.9
+         Added the *msg* parameter.
+
+      .. deprecated-removed:: 3.11 3.14
+         *msg* parameter is ambiguous when multiple :meth:`cancel`
+         are called with different cancellation messages.
+         The argument will be removed.
+
+      .. _asyncio_example_task_cancel:
+
+      The following example illustrates how coroutines can intercept
+      the cancellation request::
+
+          async def cancel_me():
+              print('cancel_me(): before sleep')
+
+              try:
+                  # Wait for 1 hour
+                  await asyncio.sleep(3600)
+              except asyncio.CancelledError:
+                  print('cancel_me(): cancel sleep')
+                  raise
+              finally:
+                  print('cancel_me(): after sleep')
+
+          async def main():
+              # Create a "cancel_me" Task
+              task = asyncio.create_task(cancel_me())
+
+              # Wait for 1 second
+              await asyncio.sleep(1)
+
+              task.cancel()
+              try:
+                  await task
+              except asyncio.CancelledError:
+                  print("main(): cancel_me is cancelled now")
+
+          asyncio.run(main())
+
+          # Expected output:
+          #
+          #     cancel_me(): before sleep
+          #     cancel_me(): cancel sleep
+          #     cancel_me(): after sleep
+          #     main(): cancel_me is cancelled now
+
+   .. method:: cancelled()
+
+      Return ``True`` if the Task is *cancelled*.
+
+      The Task is *cancelled* when the cancellation was requested with
+      :meth:`cancel` and the wrapped coroutine propagated the
+      :exc:`CancelledError` exception thrown into it.
+
+   .. method:: uncancel()
+
+      Decrement the count of cancellation requests to this Task.
+
+      Returns the remaining number of cancellation requests.
+
+      Note that once execution of a cancelled task completed, further
+      calls to :meth:`uncancel` are ineffective.
+
+      .. versionadded:: 3.11
+
+      This method is used by asyncio's internals and isn't expected to be
+      used by end-user code.  In particular, if a Task gets successfully
+      uncancelled, this allows for elements of structured concurrency like
+      :ref:`taskgroups` and :func:`asyncio.timeout` to continue running,
+      isolating cancellation to the respective structured block.
+      For example::
+
+        async def make_request_with_timeout():
+            try:
+                async with asyncio.timeout(1):
+                    # Structured block affected by the timeout:
+                    await make_request()
+                    await make_another_request()
+            except TimeoutError:
+                log("There was a timeout")
+            # Outer code not affected by the timeout:
+            await unrelated_code()
+
+      While the block with ``make_request()`` and ``make_another_request()``
+      might get cancelled due to the timeout, ``unrelated_code()`` should
+      continue running even in case of the timeout.  This is implemented
+      with :meth:`uncancel`.  :class:`TaskGroup` context managers use
+      :func:`uncancel` in a similar fashion.
+
+   .. method:: cancelling()
+
+      Return the number of pending cancellation requests to this Task, i.e.,
+      the number of calls to :meth:`cancel` less the number of
+      :meth:`uncancel` calls.
+
+      Note that if this number is greater than zero but the Task is
+      still executing, :meth:`cancelled` will still return ``False``.
+      This is because this number can be lowered by calling :meth:`uncancel`,
+      which can lead to the task not being cancelled after all if the
+      cancellation requests go down to zero.
+
+      This method is used by asyncio's internals and isn't expected to be
+      used by end-user code.  See :meth:`uncancel` for more details.
+
+      .. versionadded:: 3.11

Doc/library/email.compat32-message.rst

@@ -83,7 +83,7 @@ Here are the methods of the :class:`Message` class:
       Note that this method is provided as a convenience and may not always
       format the message the way you want.  For example, by default it does
       not do the mangling of lines that begin with ``From`` that is
-      required by the unix mbox format.  For more flexibility, instantiate a
+      required by the Unix mbox format.  For more flexibility, instantiate a
       :class:`~email.generator.Generator` instance and use its
       :meth:`~email.generator.Generator.flatten` method directly.  For example::
 
@@ -125,7 +125,7 @@ Here are the methods of the :class:`Message` class:
       Note that this method is provided as a convenience and may not always
       format the message the way you want.  For example, by default it does
       not do the mangling of lines that begin with ``From`` that is
-      required by the unix mbox format.  For more flexibility, instantiate a
+      required by the Unix mbox format.  For more flexibility, instantiate a
       :class:`~email.generator.BytesGenerator` instance and use its
       :meth:`~email.generator.BytesGenerator.flatten` method directly.
       For example::