diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst index 6ae6bfe4aa6ab4..6eae24b38fae48 100644 --- a/Doc/c-api/code.rst +++ b/Doc/c-api/code.rst @@ -32,11 +32,13 @@ bound into a function. .. c:function:: Py_ssize_t PyCode_GetNumFree(PyCodeObject *co) - Return the number of free variables in a code object. + Return the number of :term:`free (closure) variables ` + in a code object. .. c:function:: int PyUnstable_Code_GetFirstFree(PyCodeObject *co) - Return the position of the first free variable in a code object. + Return the position of the first :term:`free (closure) variable ` + in a code object. .. versionchanged:: 3.13 @@ -144,7 +146,8 @@ bound into a function. Equivalent to the Python code ``getattr(co, 'co_freevars')``. Returns a new reference to a :c:type:`PyTupleObject` containing the names of - the free variables. On error, ``NULL`` is returned and an exception is raised. + the :term:`free (closure) variables `. On error, ``NULL`` is returned + and an exception is raised. .. versionadded:: 3.11 diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 6d16e04ac3d864..8e0cf7bb0fc088 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -7,7 +7,8 @@ Initialization, Finalization, and Threads ***************************************** -See also the :ref:`Python Initialization Configuration `. +See :ref:`Python Initialization Configuration ` for details +on how to configure the interpreter prior to initialization. .. _pre-init-safe: @@ -21,6 +22,15 @@ a few functions and the :ref:`global configuration variables The following functions can be safely called before Python is initialized: +* Functions that initialize the interpreter: + + * :c:func:`Py_Initialize` + * :c:func:`Py_InitializeEx` + * :c:func:`Py_InitializeFromConfig` + * :c:func:`Py_BytesMain` + * :c:func:`Py_Main` + * the runtime pre-initialization functions covered in :ref:`init-config` + * Configuration functions: * :c:func:`PyImport_AppendInittab` @@ -32,6 +42,7 @@ The following functions can be safely called before Python is initialized: * :c:func:`Py_SetProgramName` * :c:func:`Py_SetPythonHome` * :c:func:`PySys_ResetWarnOptions` + * the configuration functions covered in :ref:`init-config` * Informative functions: @@ -43,10 +54,12 @@ The following functions can be safely called before Python is initialized: * :c:func:`Py_GetCopyright` * :c:func:`Py_GetPlatform` * :c:func:`Py_GetVersion` + * :c:func:`Py_IsInitialized` * Utilities: * :c:func:`Py_DecodeLocale` + * the status reporting and utility functions covered in :ref:`init-config` * Memory allocators: @@ -62,11 +75,13 @@ The following functions can be safely called before Python is initialized: .. note:: - The following functions **should not be called** before - :c:func:`Py_Initialize`: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`, + Despite their apparent similarity to some of the functions listed above, + the following functions **should not be called** before the interpreter has + been initialized: :c:func:`Py_EncodeLocale`, :c:func:`Py_GetPath`, :c:func:`Py_GetPrefix`, :c:func:`Py_GetExecPrefix`, :c:func:`Py_GetProgramFullPath`, :c:func:`Py_GetPythonHome`, - :c:func:`Py_GetProgramName` and :c:func:`PyEval_InitThreads`. + :c:func:`Py_GetProgramName`, :c:func:`PyEval_InitThreads`, and + :c:func:`Py_RunMain`. .. _global-conf-vars: @@ -346,34 +361,42 @@ Initializing and finalizing the interpreter this should be called before using any other Python/C API functions; see :ref:`Before Python Initialization ` for the few exceptions. - This initializes - the table of loaded modules (``sys.modules``), and creates the fundamental - modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. It also initializes - the module search path (``sys.path``). It does not set ``sys.argv``; use - the new :c:type:`PyConfig` API of the :ref:`Python Initialization - Configuration ` for that. This is a no-op when called for a - second time - (without calling :c:func:`Py_FinalizeEx` first). There is no return value; it is a - fatal error if the initialization fails. - - Use the :c:func:`Py_InitializeFromConfig` function to customize the + This initializes the table of loaded modules (``sys.modules``), and creates + the fundamental modules :mod:`builtins`, :mod:`__main__` and :mod:`sys`. + It also initializes the module search path (``sys.path``). It does not set + ``sys.argv``; use the :ref:`Python Initialization Configuration ` + API for that. This is a no-op when called for a second time (without calling + :c:func:`Py_FinalizeEx` first). There is no return value; it is a fatal + error if the initialization fails. + + Use :c:func:`Py_InitializeFromConfig` to customize the :ref:`Python Initialization Configuration `. .. note:: - On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, which will - also affect non-Python uses of the console using the C Runtime. + On Windows, changes the console mode from ``O_TEXT`` to ``O_BINARY``, + which will also affect non-Python uses of the console using the C Runtime. .. c:function:: void Py_InitializeEx(int initsigs) This function works like :c:func:`Py_Initialize` if *initsigs* is ``1``. If - *initsigs* is ``0``, it skips initialization registration of signal handlers, which - might be useful when Python is embedded. + *initsigs* is ``0``, it skips initialization registration of signal handlers, + which may be useful when CPython is embedded as part of a larger application. - Use the :c:func:`Py_InitializeFromConfig` function to customize the + Use :c:func:`Py_InitializeFromConfig` to customize the :ref:`Python Initialization Configuration `. +.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config) + + Initialize Python from *config* configuration, as described in + :ref:`init-from-config`. + + See the :ref:`init-config` section for details on pre-initializing the + interpreter, populating the runtime configuration structure, and querying + the returned status structure. + + .. c:function:: int Py_IsInitialized() Return true (nonzero) when the Python interpreter has been initialized, false @@ -440,12 +463,111 @@ Initializing and finalizing the interpreter .. versionadded:: 3.6 + .. c:function:: void Py_Finalize() This is a backwards-compatible version of :c:func:`Py_FinalizeEx` that disregards the return value. +.. c:function:: int Py_BytesMain(int argc, char **argv) + + Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings, + allowing the calling application to delegate the text decoding step to + the CPython runtime. + + .. versionadded:: 3.8 + + +.. c:function:: int Py_Main(int argc, wchar_t **argv) + + The main program for the standard interpreter, encapsulating a full + initialization/finalization cycle, as well as additional + behaviour to implement reading configurations settings from the environment + and command line, and then executing ``__main__`` in accordance with + :ref:`using-on-cmdline`. + + This is made available for programs which wish to support the full CPython + command line interface, rather than just embedding a Python runtime in a + larger application. + + The *argc* and *argv* parameters are similar to those which are passed to a + C program's :c:func:`main` function, except that the *argv* entries are first + converted to ``wchar_t`` using :c:func:`Py_DecodeLocale`. It is also + important to note that the argument list entries may be modified to point to + strings other than those passed in (however, the contents of the strings + pointed to by the argument list are not modified). + + The return value will be ``0`` if the interpreter exits normally (i.e., + without an exception), ``1`` if the interpreter exits due to an exception, + or ``2`` if the argument list does not represent a valid Python command + line. + + Note that if an otherwise unhandled :exc:`SystemExit` is raised, this + function will not return ``1``, but exit the process, as long as + ``Py_InspectFlag`` is not set. If ``Py_InspectFlag`` is set, execution will + drop into the interactive Python prompt, at which point a second otherwise + unhandled :exc:`SystemExit` will still exit the process, while any other + means of exiting will set the return value as described above. + + In terms of the CPython runtime configuration APIs documented in the + :ref:`runtime configuration ` section (and without accounting + for error handling), ``Py_Main`` is approximately equivalent to:: + + PyConfig config; + PyConfig_InitPythonConfig(&config); + PyConfig_SetArgv(&config, argc, argv); + Py_InitializeFromConfig(&config); + PyConfig_Clear(&config); + + Py_RunMain(); + + In normal usage, an embedding application will call this function + *instead* of calling :c:func:`Py_Initialize`, :c:func:`Py_InitializeEx` or + :c:func:`Py_InitializeFromConfig` directly, and all settings will be applied + as described elsewhere in this documentation. If this function is instead + called *after* a preceding runtime initialization API call, then exactly + which environmental and command line configuration settings will be updated + is version dependent (as it depends on which settings correctly support + being modified after they have already been set once when the runtime was + first initialized). + + +.. c:function:: int Py_RunMain(void) + + Executes the main module in a fully configured CPython runtime. + + Executes the command (:c:member:`PyConfig.run_command`), the script + (:c:member:`PyConfig.run_filename`) or the module + (:c:member:`PyConfig.run_module`) specified on the command line or in the + configuration. If none of these values are set, runs the interactive Python + prompt (REPL) using the ``__main__`` module's global namespace. + + If :c:member:`PyConfig.inspect` is not set (the default), the return value + will be ``0`` if the interpreter exits normally (that is, without raising + an exception), or ``1`` if the interpreter exits due to an exception. If an + otherwise unhandled :exc:`SystemExit` is raised, the function will immediately + exit the process instead of returning ``1``. + + If :c:member:`PyConfig.inspect` is set (such as when the :option:`-i` option + is used), rather than returning when the interpreter exits, execution will + instead resume in an interactive Python prompt (REPL) using the ``__main__`` + module's global namespace. If the interpreter exited with an exception, it + is immediately raised in the REPL session. The function return value is + then determined by the way the *REPL session* terminates: returning ``0`` + if the session terminates without raising an unhandled exception, exiting + immediately for an unhandled :exc:`SystemExit`, and returning ``1`` for + any other unhandled exception. + + This function always finalizes the Python interpreter regardless of whether + it returns a value or immediately exits the process due to an unhandled + :exc:`SystemExit` exception. + + See :ref:`Python Configuration ` for an example of a + customized Python that always runs in isolated mode using + :c:func:`Py_RunMain`. + + Process-wide parameters ======================= diff --git a/Doc/c-api/init_config.rst b/Doc/c-api/init_config.rst index 9dc9ba61e7a60f..6f8962afc7af0d 100644 --- a/Doc/c-api/init_config.rst +++ b/Doc/c-api/init_config.rst @@ -1356,14 +1356,13 @@ the :option:`-X` command line option. The ``show_alloc_count`` field has been removed. +.. _init-from-config: + Initialization with PyConfig ---------------------------- -Function to initialize Python: - -.. c:function:: PyStatus Py_InitializeFromConfig(const PyConfig *config) - - Initialize Python from *config* configuration. +Initializing the interpreter from a populated configuration struct is handled +by calling :c:func:`Py_InitializeFromConfig`. The caller is responsible to handle exceptions (error or exit) using :c:func:`PyStatus_Exception` and :c:func:`Py_ExitStatusException`. @@ -1835,26 +1834,6 @@ return ``-1`` on error: } -Py_RunMain() -============ - -.. c:function:: int Py_RunMain(void) - - Execute the command (:c:member:`PyConfig.run_command`), the script - (:c:member:`PyConfig.run_filename`) or the module - (:c:member:`PyConfig.run_module`) specified on the command line or in the - configuration. - - By default and when if :option:`-i` option is used, run the REPL. - - Finally, finalizes Python and returns an exit status that can be passed to - the ``exit()`` function. - -See :ref:`Python Configuration ` for an example of -customized Python always running in isolated mode using -:c:func:`Py_RunMain`. - - Runtime Python configuration API ================================ diff --git a/Doc/c-api/tuple.rst b/Doc/c-api/tuple.rst index 7a8a6134282ade..815afddad19df1 100644 --- a/Doc/c-api/tuple.rst +++ b/Doc/c-api/tuple.rst @@ -167,7 +167,8 @@ type. .. c:member:: const char *name - Name of the struct sequence type. + Fully qualified name of the type; null-terminated UTF-8 encoded. + The name must contain the module name. .. c:member:: const char *doc diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst index b2ac0c903c2bd7..f5704cffa199a5 100644 --- a/Doc/c-api/unicode.rst +++ b/Doc/c-api/unicode.rst @@ -1438,6 +1438,31 @@ They all return ``NULL`` or ``-1`` if an exception occurs. This function returns ``-1`` upon failure, so one should call :c:func:`PyErr_Occurred` to check for errors. + .. seealso:: + + The :c:func:`PyUnicode_Equal` function. + + +.. c:function:: int PyUnicode_Equal(PyObject *a, PyObject *b) + + Test if two strings are equal: + + * Return ``1`` if *a* is equal to *b*. + * Return ``0`` if *a* is not equal to *b*. + * Set a :exc:`TypeError` exception and return ``-1`` if *a* or *b* is not a + :class:`str` object. + + The function always succeeds if *a* and *b* are :class:`str` objects. + + The function works for :class:`str` subclasses, but does not honor custom + ``__eq__()`` method. + + .. seealso:: + + The :c:func:`PyUnicode_Compare` function. + + .. versionadded:: 3.14 + .. c:function:: int PyUnicode_EqualToUTF8AndSize(PyObject *unicode, const char *string, Py_ssize_t size) diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst index 67167444d0a685..9f02bdb5896563 100644 --- a/Doc/c-api/veryhigh.rst +++ b/Doc/c-api/veryhigh.rst @@ -25,30 +25,6 @@ are only passed to these functions if it is certain that they were created by the same library that the Python runtime is using. -.. c:function:: int Py_Main(int argc, wchar_t **argv) - - The main program for the standard interpreter. This is made available for - programs which embed Python. The *argc* and *argv* parameters should be - prepared exactly as those which are passed to a C program's :c:func:`main` - function (converted to wchar_t according to the user's locale). It is - important to note that the argument list may be modified (but the contents of - the strings pointed to by the argument list are not). The return value will - be ``0`` if the interpreter exits normally (i.e., without an exception), - ``1`` if the interpreter exits due to an exception, or ``2`` if the parameter - list does not represent a valid Python command line. - - Note that if an otherwise unhandled :exc:`SystemExit` is raised, this - function will not return ``1``, but exit the process, as long as - :c:member:`PyConfig.inspect` is zero. - - -.. c:function:: int Py_BytesMain(int argc, char **argv) - - Similar to :c:func:`Py_Main` but *argv* is an array of bytes strings. - - .. versionadded:: 3.8 - - .. c:function:: int PyRun_AnyFile(FILE *fp, const char *filename) This is a simplified interface to :c:func:`PyRun_AnyFileExFlags` below, leaving diff --git a/Doc/data/stable_abi.dat b/Doc/data/stable_abi.dat index 19dc71a345b474..9314facd2ad873 100644 --- a/Doc/data/stable_abi.dat +++ b/Doc/data/stable_abi.dat @@ -783,6 +783,7 @@ func,PyUnicode_DecodeUnicodeEscape,3.2,, func,PyUnicode_EncodeCodePage,3.7,on Windows, func,PyUnicode_EncodeFSDefault,3.2,, func,PyUnicode_EncodeLocale,3.7,, +func,PyUnicode_Equal,3.14,, func,PyUnicode_EqualToUTF8,3.13,, func,PyUnicode_EqualToUTF8AndSize,3.13,, func,PyUnicode_FSConverter,3.2,, diff --git a/Doc/glossary.rst b/Doc/glossary.rst index e72a8d002d507d..933fb0319452a6 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -231,6 +231,28 @@ Glossary A variable defined in a class and intended to be modified only at class level (i.e., not in an instance of the class). + closure variable + A :term:`free variable` referenced from a :term:`nested scope` that is defined in an outer + scope rather than being resolved at runtime from the globals or builtin namespaces. + May be explicitly defined with the :keyword:`nonlocal` keyword to allow write access, + or implicitly defined if the variable is only being read. + + For example, in the ``inner`` function in the following code, both ``x`` and ``print`` are + :term:`free variables `, but only ``x`` is a *closure variable*:: + + def outer(): + x = 0 + def inner(): + nonlocal x + x += 1 + print(x) + return inner + + Due to the :attr:`codeobject.co_freevars` attribute (which, despite its name, only + includes the names of closure variables rather than listing all referenced free + variables), the more general :term:`free variable` term is sometimes used even + when the intended meaning is to refer specifically to closure variables. + complex number An extension of the familiar real number system in which all numbers are expressed as a sum of a real part and an imaginary part. Imaginary @@ -454,6 +476,13 @@ Glossary the :term:`global interpreter lock` which allows only one thread to execute Python bytecode at a time. See :pep:`703`. + free variable + Formally, as defined in the :ref:`language execution model `, a free + variable is any variable used in a namespace which is not a local variable in that + namespace. See :term:`closure variable` for an example. + Pragmatically, due to the name of the :attr:`codeobject.co_freevars` attribute, + the term is also sometimes used as a synonym for :term:`closure variable`. + function A series of statements which returns some value to a caller. It can also be passed zero or more :term:`arguments ` which may be used in diff --git a/Doc/howto/sockets.rst b/Doc/howto/sockets.rst index 0bbf97da39768d..cbc49d15a0771b 100644 --- a/Doc/howto/sockets.rst +++ b/Doc/howto/sockets.rst @@ -100,8 +100,8 @@ mainloop of the web server:: (clientsocket, address) = serversocket.accept() # now do something with the clientsocket # in this case, we'll pretend this is a threaded server - ct = client_thread(clientsocket) - ct.run() + ct = make_client_thread(clientsocket) + ct.start() There's actually 3 general ways in which this loop could work - dispatching a thread to handle ``clientsocket``, create a new process to handle diff --git a/Doc/library/ctypes.rst b/Doc/library/ctypes.rst index 535c5173be50de..d76b8d4809c078 100644 --- a/Doc/library/ctypes.rst +++ b/Doc/library/ctypes.rst @@ -2589,6 +2589,8 @@ fields, or any other data types containing pointer type fields. the structure when being packed or unpacked to/from memory. Setting this attribute to 0 is the same as not setting it at all. + .. versionadded:: 3.13 + .. attribute:: _layout_ An optional string naming the struct/union layout. It can currently diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst index 64510a77c67c11..f0b465bc9ce39c 100644 --- a/Doc/library/datetime.rst +++ b/Doc/library/datetime.rst @@ -948,6 +948,10 @@ Other constructors, all class methods: This function is preferred over :meth:`today` and :meth:`utcnow`. + .. note:: + + Subsequent calls to :meth:`!datetime.now` may return the same + instant depending on the precision of the underlying clock. .. classmethod:: datetime.utcnow() diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index e3919c2ffad84c..1d084a8bf38d98 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -969,7 +969,8 @@ iterations of the loop. .. opcode:: GET_LEN - Perform ``STACK.append(len(STACK[-1]))``. + Perform ``STACK.append(len(STACK[-1]))``. Used in :keyword:`match` statements where + comparison with structure of pattern is needed. .. versionadded:: 3.10 @@ -1434,7 +1435,7 @@ iterations of the loop. slot ``i`` of the "fast locals" storage in this mapping. If the name is not found there, loads it from the cell contained in slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading - free variables in class bodies (which previously used + :term:`closure variables ` in class bodies (which previously used :opcode:`!LOAD_CLASSDEREF`) and in :ref:`annotation scopes ` within class bodies. @@ -1463,8 +1464,8 @@ iterations of the loop. .. opcode:: COPY_FREE_VARS (n) - Copies the ``n`` free variables from the closure into the frame. - Removes the need for special code on the caller's side when calling + Copies the ``n`` :term:`free (closure) variables ` from the closure + into the frame. Removes the need for special code on the caller's side when calling closures. .. versionadded:: 3.11 @@ -1937,10 +1938,10 @@ instructions: .. data:: hasfree - Sequence of bytecodes that access a free variable. 'free' in this - context refers to names in the current scope that are referenced by inner - scopes or names in outer scopes that are referenced from this scope. It does - *not* include references to global or builtin scopes. + Sequence of bytecodes that access a :term:`free (closure) variable `. + 'free' in this context refers to names in the current scope that are + referenced by inner scopes or names in outer scopes that are referenced + from this scope. It does *not* include references to global or builtin scopes. .. data:: hasname diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index a96f69e6170f00..7f8df704a33327 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -684,9 +684,10 @@ are always available. They are listed here in alphabetical order. ``__builtins__`` dictionary into *globals* before passing it to :func:`exec`. The *closure* argument specifies a closure--a tuple of cellvars. - It's only valid when the *object* is a code object containing free variables. - The length of the tuple must exactly match the number of free variables - referenced by the code object. + It's only valid when the *object* is a code object containing + :term:`free (closure) variables `. + The length of the tuple must exactly match the length of the code object'S + :attr:`~codeobject.co_freevars` attribute. .. audit-event:: exec code_object exec diff --git a/Doc/library/importlib.metadata.rst b/Doc/library/importlib.metadata.rst index ddb72784694fd9..85d5a2d684d6eb 100644 --- a/Doc/library/importlib.metadata.rst +++ b/Doc/library/importlib.metadata.rst @@ -122,7 +122,7 @@ Entry points Returns a :class:`EntryPoints` instance describing entry points for the current environment. Any given keyword parameters are passed to the - :meth:`!~EntryPoints.select` method for comparison to the attributes of + :meth:`!select` method for comparison to the attributes of the individual entry point definitions. Note: it is not currently possible to query for entry points based on @@ -158,7 +158,7 @@ attributes for convenience:: >>> sorted(eps.groups) # doctest: +SKIP ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation'] -:class:`!EntryPoints` has a :meth:`!~EntryPoints.select` method to select entry points +:class:`!EntryPoints` has a :meth:`!select` method to select entry points matching specific properties. Select entry points in the ``console_scripts`` group:: @@ -232,7 +232,7 @@ Distribution metadata `PackageMetadata protocol `_. In addition to providing the defined protocol methods and attributes, subscripting - the instance is equivalent to calling the :meth:`!~PackageMetadata.get` method. + the instance is equivalent to calling the :meth:`!get` method. Every `Distribution Package `_ includes some metadata, which you can extract using the :func:`!metadata` function:: @@ -245,7 +245,7 @@ the values are returned unparsed from the distribution metadata:: >>> wheel_metadata['Requires-Python'] # doctest: +SKIP '>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*' -:class:`PackageMetadata` also presents a :attr:`!~PackageMetadata.json` attribute that returns +:class:`PackageMetadata` also presents a :attr:`!json` attribute that returns all the metadata in a JSON-compatible form per :PEP:`566`:: >>> wheel_metadata.json['requires_python'] @@ -331,7 +331,7 @@ Once you have the file, you can also read its contents:: return s.encode('utf-8') return s -You can also use the :meth:`!~PackagePath.locate` method to get the absolute +You can also use the :meth:`!locate` method to get the absolute path to the file:: >>> util.locate() # doctest: +SKIP diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 9a62249816c9bf..c138e903fa5a0f 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -134,7 +134,7 @@ loops that truncate the stream. To compute a running minimum, set *function* to :func:`min`. For a running maximum, set *function* to :func:`max`. Or for a running product, set *function* to :func:`operator.mul`. - To build an `Amortization table + To build an `amortization table `_, accumulate the interest and apply payments: @@ -736,6 +736,26 @@ loops that truncate the stream. allows nested :func:`tee` calls to share the same underlying data chain and to have a single update step rather than a chain of calls. + The flattening property makes tee iterators efficiently peekable: + + .. testcode:: + + def lookahead(tee_iterator): + "Return the next value without moving the input forward" + [forked_iterator] = tee(tee_iterator, 1) + return next(forked_iterator) + + .. doctest:: + + >>> iterator = iter('abcdef') + >>> [iterator] = tee(iterator, 1) # Make the input peekable + >>> next(iterator) # Move the iterator forward + 'a' + >>> lookahead(iterator) # Check next value + 'b' + >>> next(iterator) # Continue moving forward + 'b' + ``tee`` iterators are not threadsafe. A :exc:`RuntimeError` may be raised when simultaneously using iterators returned by the same :func:`tee` call, even if the original *iterable* is threadsafe. @@ -952,15 +972,6 @@ and :term:`generators ` which incur interpreter overhead. iterators = cycle(islice(iterators, num_active)) yield from map(next, iterators) - def partition(predicate, iterable): - """Partition entries into false entries and true entries. - - If *predicate* is slow, consider wrapping it with functools.lru_cache(). - """ - # partition(is_odd, range(10)) → 0 2 4 6 8 and 1 3 5 7 9 - t1, t2 = tee(iterable) - return filterfalse(predicate, t1), filter(predicate, t2) - def subslices(seq): "Return all contiguous non-empty subslices of a sequence." # subslices('ABCD') → A AB ABC ABCD B BC BCD C CD D @@ -1178,15 +1189,19 @@ The following recipes have a more mathematical flavor: >>> list(it) ['d', 'e', 'f'] + >>> list(prepend(1, [2, 3, 4])) [1, 2, 3, 4] + >>> list(enumerate('abc')) [(0, 'a'), (1, 'b'), (2, 'c')] + >>> list(islice(tabulate(lambda x: 2*x), 4)) [0, 2, 4, 6] + >>> list(tail(3, 'ABCDEFG')) ['E', 'F', 'G'] >>> # Verify the input is consumed greedily @@ -1195,6 +1210,7 @@ The following recipes have a more mathematical flavor: >>> list(input_iterator) [] + >>> it = iter(range(10)) >>> consume(it, 3) >>> # Verify the input is consumed lazily @@ -1205,6 +1221,7 @@ The following recipes have a more mathematical flavor: >>> next(it, 'Done') 'Done' + >>> nth('abcde', 3) 'd' >>> nth('abcde', 9) is None @@ -1216,6 +1233,7 @@ The following recipes have a more mathematical flavor: >>> list(it) ['d', 'e'] + >>> [all_equal(s) for s in ('', 'A', 'AAAA', 'AAAB', 'AAABA')] [True, True, True, False, False] >>> [all_equal(s, key=str.casefold) for s in ('', 'A', 'AaAa', 'AAAB', 'AAABA')] @@ -1229,24 +1247,19 @@ The following recipes have a more mathematical flavor: >>> ''.join(it) 'bbccc' + >>> quantify(range(99), lambda x: x%2==0) 50 - >>> quantify([True, False, False, True, True]) 3 - >>> quantify(range(12), predicate=lambda x: x%2==1) 6 + >>> a = [[1, 2, 3], [4, 5, 6]] >>> list(flatten(a)) [1, 2, 3, 4, 5, 6] - >>> list(repeatfunc(pow, 5, 2, 3)) - [8, 8, 8, 8, 8] - - >>> take(5, map(int, repeatfunc(random.random))) - [0, 0, 0, 0, 0] >>> list(ncycles('abc', 3)) ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b', 'c'] @@ -1256,9 +1269,11 @@ The following recipes have a more mathematical flavor: >>> list(input_iterator) [] + >>> sum_of_squares([10, 20, 30]) 1400 + >>> list(reshape([(0, 1), (2, 3), (4, 5)], 3)) [(0, 1, 2), (3, 4, 5)] >>> M = [(0, 1, 2, 3), (4, 5, 6, 7), (8, 9, 10, 11)] @@ -1279,6 +1294,7 @@ The following recipes have a more mathematical flavor: >>> list(reshape(M, 12)) [(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11)] + >>> list(transpose([(1, 2, 3), (11, 22, 33)])) [(1, 11), (2, 22), (3, 33)] >>> # Verify that the inputs are consumed lazily @@ -1290,11 +1306,13 @@ The following recipes have a more mathematical flavor: >>> list(zip(input1, input2)) [(2, 22), (3, 33)] + >>> list(matmul([(7, 5), (3, 5)], [[2, 5], [7, 9]])) [(49, 80), (41, 60)] >>> list(matmul([[2, 5], [7, 9], [3, 4]], [[7, 11, 5, 4, 9], [3, 5, 2, 6, 3]])) [(29, 47, 20, 38, 33), (76, 122, 53, 82, 90), (33, 53, 23, 36, 39)] + >>> list(convolve([1, -1, -20], [1, -3])) == [1, -4, -17, 60] True >>> data = [20, 40, 24, 32, 20, 28, 16] @@ -1317,6 +1335,7 @@ The following recipes have a more mathematical flavor: >>> list(signal_iterator) [30, 40, 50] + >>> from fractions import Fraction >>> from decimal import Decimal >>> polynomial_eval([1, -4, -17, 60], x=5) @@ -1348,6 +1367,7 @@ The following recipes have a more mathematical flavor: >>> polynomial_eval([11, 2], 7) == 11 * 7 + 2 True + >>> polynomial_from_roots([5, -4, 3]) [1, -4, -17, 60] >>> factored = lambda x: (x - 5) * (x + 4) * (x - 3) @@ -1355,9 +1375,11 @@ The following recipes have a more mathematical flavor: >>> all(factored(x) == expanded(x) for x in range(-10, 11)) True + >>> polynomial_derivative([1, -4, -17, 60]) [3, -8, -17] + >>> list(iter_index('AABCADEAF', 'A')) [0, 1, 4, 7] >>> list(iter_index('AABCADEAF', 'B')) @@ -1415,12 +1437,14 @@ The following recipes have a more mathematical flavor: >>> ''.join(input_iterator) 'DEAF' + >>> # Verify that the target value can be a sequence. >>> seq = [[10, 20], [30, 40], 30, 40, [30, 40], 50] >>> target = [30, 40] >>> list(iter_index(seq, target)) [1, 4] + >>> # Verify faithfulness to type specific index() method behaviors. >>> # For example, bytes and str perform continuous-subsequence searches >>> # that do not match the general behavior specified @@ -1450,6 +1474,7 @@ The following recipes have a more mathematical flavor: >>> set(sieve(10_000)).isdisjoint(carmichael) True + >>> list(factor(99)) # Code example 1 [3, 3, 11] >>> list(factor(1_000_000_000_000_007)) # Code example 2 @@ -1495,6 +1520,7 @@ The following recipes have a more mathematical flavor: >>> all(list(factor(n)) == sorted(factor(n)) for n in range(2_000)) True + >>> totient(0) # https://www.wolframalpha.com/input?i=totient+0 0 >>> first_totients = [1, 1, 2, 2, 4, 2, 6, 4, 6, 4, 10, 4, 12, 6, 8, 8, 16, 6, @@ -1514,9 +1540,15 @@ The following recipes have a more mathematical flavor: >>> totient(6 ** 20) == 1 * 2**19 * 2 * 3**19 # repeated primes True + >>> list(flatten([('a', 'b'), (), ('c', 'd', 'e'), ('f',), ('g', 'h', 'i')])) ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i'] + + >>> list(repeatfunc(pow, 5, 2, 3)) + [8, 8, 8, 8, 8] + >>> take(5, map(int, repeatfunc(random.random))) + [0, 0, 0, 0, 0] >>> random.seed(85753098575309) >>> list(repeatfunc(random.random, 3)) [0.16370491282496968, 0.45889608687313455, 0.3747076837820118] @@ -1525,9 +1557,11 @@ The following recipes have a more mathematical flavor: >>> list(repeatfunc(pow, 3, 2, 5)) [32, 32, 32] + >>> list(grouper('abcdefg', 3, fillvalue='x')) [('a', 'b', 'c'), ('d', 'e', 'f'), ('g', 'x', 'x')] + >>> it = grouper('abcdefg', 3, incomplete='strict') >>> next(it) ('a', 'b', 'c') @@ -1541,6 +1575,7 @@ The following recipes have a more mathematical flavor: >>> list(grouper('abcdefg', n=3, incomplete='ignore')) [('a', 'b', 'c'), ('d', 'e', 'f')] + >>> list(sliding_window('ABCDEFG', 1)) [('A',), ('B',), ('C',), ('D',), ('E',), ('F',), ('G',)] >>> list(sliding_window('ABCDEFG', 2)) @@ -1570,6 +1605,7 @@ The following recipes have a more mathematical flavor: ... 'zero or negative n not supported' + >>> list(roundrobin('abc', 'd', 'ef')) ['a', 'd', 'e', 'b', 'f', 'c'] >>> ranges = [range(5, 1000), range(4, 3000), range(0), range(3, 2000), range(2, 5000), range(1, 3500)] @@ -1583,38 +1619,19 @@ The following recipes have a more mathematical flavor: >>> ''.join(chain(*input_iterators)) 'dijkopqr' - >>> def is_odd(x): - ... return x % 2 == 1 - - >>> evens, odds = partition(is_odd, range(10)) - >>> list(evens) - [0, 2, 4, 6, 8] - >>> list(odds) - [1, 3, 5, 7, 9] - >>> # Verify that the input is consumed lazily - >>> input_iterator = iter(range(10)) - >>> evens, odds = partition(is_odd, input_iterator) - >>> next(odds) - 1 - >>> next(odds) - 3 - >>> next(evens) - 0 - >>> list(input_iterator) - [4, 5, 6, 7, 8, 9] >>> list(subslices('ABCD')) ['A', 'AB', 'ABC', 'ABCD', 'B', 'BC', 'BCD', 'C', 'CD', 'D'] + >>> list(powerset([1,2,3])) [(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)] - >>> all(len(list(powerset(range(n)))) == 2**n for n in range(18)) True - >>> list(powerset('abcde')) == sorted(sorted(set(powerset('abcde'))), key=len) True + >>> list(unique_everseen('AAAABBBCCDAABBB')) ['A', 'B', 'C', 'D'] >>> list(unique_everseen('ABBCcAD', str.casefold)) @@ -1629,6 +1646,7 @@ The following recipes have a more mathematical flavor: >>> ''.join(input_iterator) 'AAABBBCCDAABBB' + >>> list(unique_justseen('AAAABBBCCDAABBB')) ['A', 'B', 'C', 'D', 'A', 'B'] >>> list(unique_justseen('ABBCcAD', str.casefold)) @@ -1643,6 +1661,7 @@ The following recipes have a more mathematical flavor: >>> ''.join(input_iterator) 'AAABBBCCDAABBB' + >>> list(unique([[1, 2], [3, 4], [1, 2]])) [[1, 2], [3, 4]] >>> list(unique('ABBcCAD', str.casefold)) @@ -1650,6 +1669,7 @@ The following recipes have a more mathematical flavor: >>> list(unique('ABBcCAD', str.casefold, reverse=True)) ['D', 'c', 'B', 'A'] + >>> d = dict(a=1, b=2, c=3) >>> it = iter_except(d.popitem, KeyError) >>> d['d'] = 4 @@ -1667,6 +1687,7 @@ The following recipes have a more mathematical flavor: >>> next(it, 'empty') 'empty' + >>> first_true('ABC0DEF1', '9', str.isdigit) '0' >>> # Verify that inputs are consumed lazily @@ -1747,21 +1768,36 @@ The following recipes have a more mathematical flavor: return true_iterator(), chain(transition, it) + def partition(predicate, iterable): + """Partition entries into false entries and true entries. + + If *predicate* is slow, consider wrapping it with functools.lru_cache(). + """ + # partition(is_odd, range(10)) → 0 2 4 6 8 and 1 3 5 7 9 + t1, t2 = tee(iterable) + return filterfalse(predicate, t1), filter(predicate, t2) + + + .. doctest:: :hide: >>> dotproduct([1,2,3], [4,5,6]) 32 + >>> sumprod([1,2,3], [4,5,6]) 32 + >>> list(islice(pad_none('abc'), 0, 6)) ['a', 'b', 'c', None, None, None] + >>> list(triplewise('ABCDEFG')) [('A', 'B', 'C'), ('B', 'C', 'D'), ('C', 'D', 'E'), ('D', 'E', 'F'), ('E', 'F', 'G')] + >>> population = 'ABCDEFGH' >>> for r in range(len(population) + 1): ... seq = list(combinations(population, r)) @@ -1769,16 +1805,38 @@ The following recipes have a more mathematical flavor: ... assert nth_combination(population, r, i) == seq[i] ... for i in range(-len(seq), 0): ... assert nth_combination(population, r, i) == seq[i] - + ... >>> iterable = 'abcde' >>> r = 3 >>> combos = list(combinations(iterable, r)) >>> all(nth_combination(iterable, r, i) == comb for i, comb in enumerate(combos)) True + >>> it = iter('ABCdEfGhI') >>> all_upper, remainder = before_and_after(str.isupper, it) >>> ''.join(all_upper) 'ABC' >>> ''.join(remainder) 'dEfGhI' + + + >>> def is_odd(x): + ... return x % 2 == 1 + ... + >>> evens, odds = partition(is_odd, range(10)) + >>> list(evens) + [0, 2, 4, 6, 8] + >>> list(odds) + [1, 3, 5, 7, 9] + >>> # Verify that the input is consumed lazily + >>> input_iterator = iter(range(10)) + >>> evens, odds = partition(is_odd, input_iterator) + >>> next(odds) + 1 + >>> next(odds) + 3 + >>> next(evens) + 0 + >>> list(input_iterator) + [4, 5, 6, 7, 8, 9] diff --git a/Doc/library/locale.rst b/Doc/library/locale.rst index 0246f99157024a..04035b33d0ed48 100644 --- a/Doc/library/locale.rst +++ b/Doc/library/locale.rst @@ -314,6 +314,15 @@ The :mod:`locale` module defines the following exception and functions: Get a representation of up to 100 values used to represent the values 0 to 99. + The function temporarily sets the ``LC_CTYPE`` locale to the locale + of the category that determines the requested value (``LC_TIME``, + ``LC_NUMERIC``, ``LC_MONETARY`` or ``LC_MESSAGES``) if locales are + different and the resulting string is non-ASCII. + This temporary change affects other threads. + + .. versionchanged:: 3.14 + The function now temporarily sets the ``LC_CTYPE`` locale in some cases. + .. function:: getdefaultlocale([envvars]) diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst index 317ca8728248c8..0e9dc33ae2123a 100644 --- a/Doc/library/logging.config.rst +++ b/Doc/library/logging.config.rst @@ -753,16 +753,17 @@ The ``queue`` and ``listener`` keys are optional. If the ``queue`` key is present, the corresponding value can be one of the following: -* An object implementing the :class:`queue.Queue` public API. For instance, - this may be an actual instance of :class:`queue.Queue` or a subclass thereof, - or a proxy obtained by :meth:`multiprocessing.managers.SyncManager.Queue`. +* An object implementing the :meth:`Queue.put_nowait ` + and :meth:`Queue.get ` public API. For instance, this may be + an actual instance of :class:`queue.Queue` or a subclass thereof, or a proxy + obtained by :meth:`multiprocessing.managers.SyncManager.Queue`. This is of course only possible if you are constructing or modifying the configuration dictionary in code. * A string that resolves to a callable which, when called with no arguments, returns - the :class:`queue.Queue` instance to use. That callable could be a - :class:`queue.Queue` subclass or a function which returns a suitable queue instance, + the queue instance to use. That callable could be a :class:`queue.Queue` subclass + or a function which returns a suitable queue instance, such as ``my.module.queue_factory()``. * A dict with a ``'()'`` key which is constructed in the usual way as discussed in diff --git a/Doc/library/symtable.rst b/Doc/library/symtable.rst index 15e0b23aa12bf0..56cd6b8afaa73e 100644 --- a/Doc/library/symtable.rst +++ b/Doc/library/symtable.rst @@ -167,11 +167,12 @@ Examining Symbol Tables .. method:: get_nonlocals() - Return a tuple containing names of nonlocals in this function. + Return a tuple containing names of explicitly declared nonlocals in this function. .. method:: get_frees() - Return a tuple containing names of free variables in this function. + Return a tuple containing names of :term:`free (closure) variables ` + in this function. .. class:: Class diff --git a/Doc/library/types.rst b/Doc/library/types.rst index 3c3c760c206ff2..84b80ec6efd59f 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -199,7 +199,7 @@ Standard names are defined for the following types: .. data:: CellType The type for cell objects: such objects are used as containers for - a function's free variables. + a function's :term:`closure variables `. .. versionadded:: 3.8 diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 513199d21456bf..d059a660548c7e 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -564,8 +564,9 @@ Special read-only attributes in which the function was defined. * - .. attribute:: function.__closure__ - - ``None`` or a :class:`tuple` of cells that contain bindings for the - function's free variables. + - ``None`` or a :class:`tuple` of cells that contain bindings for the names specified + in the :attr:`~codeobject.co_freevars` attribute of the function's + :attr:`code object `. A cell object has the attribute ``cell_contents``. This can be used to get the value of the cell, as well as set the value. @@ -1285,10 +1286,14 @@ Special read-only attributes * - .. attribute:: codeobject.co_cellvars - A :class:`tuple` containing the names of :ref:`local variables ` - that are referenced by nested functions inside the function + that are referenced from at least one :term:`nested scope` inside the function * - .. attribute:: codeobject.co_freevars - - A :class:`tuple` containing the names of free variables in the function + - A :class:`tuple` containing the names of + :term:`free (closure) variables ` that a :term:`nested scope` + references in an outer scope. See also :attr:`function.__closure__`. + + Note: references to global and builtin names are *not* included. * - .. attribute:: codeobject.co_code - A string representing the sequence of :term:`bytecode` instructions in diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst index 99cb09d09331d8..cb6c524dd97a30 100644 --- a/Doc/reference/executionmodel.rst +++ b/Doc/reference/executionmodel.rst @@ -90,7 +90,7 @@ If a name is bound in a block, it is a local variable of that block, unless declared as :keyword:`nonlocal` or :keyword:`global`. If a name is bound at the module level, it is a global variable. (The variables of the module code block are local and global.) If a variable is used in a code block but not -defined there, it is a :dfn:`free variable`. +defined there, it is a :term:`free variable`. Each occurrence of a name in the program text refers to the :dfn:`binding` of that name established by the following name resolution rules. @@ -337,6 +337,9 @@ enclosing namespace, but in the global namespace. [#]_ The :func:`exec` and :func:`eval` functions have optional arguments to override the global and local namespace. If only one namespace is specified, it is used for both. +.. XXX(ncoghlan) above is only accurate for string execution. When executing code objects, + closure cells may now be passed explicitly to resolve co_freevars references. + Docs issue: https://github.com/python/cpython/issues/122826 .. _exceptions: diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index ab72ad49d041e1..decde0d297cf59 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1809,6 +1809,8 @@ returns a boolean value regardless of the type of its argument single: named expression pair: assignment; expression +.. _assignment-expressions: + Assignment expressions ====================== diff --git a/Doc/tools/extensions/pyspecific.py b/Doc/tools/extensions/pyspecific.py index c89b1693343b4e..b6623a2b8e01f1 100644 --- a/Doc/tools/extensions/pyspecific.py +++ b/Doc/tools/extensions/pyspecific.py @@ -353,8 +353,8 @@ def run(self): # Support for building "topic help" for pydoc pydoc_topic_labels = [ - 'assert', 'assignment', 'async', 'atom-identifiers', 'atom-literals', - 'attribute-access', 'attribute-references', 'augassign', 'await', + 'assert', 'assignment', 'assignment-expressions', 'async', 'atom-identifiers', + 'atom-literals', 'attribute-access', 'attribute-references', 'augassign', 'await', 'binary', 'bitwise', 'bltin-code-objects', 'bltin-ellipsis-object', 'bltin-null-object', 'bltin-type-objects', 'booleans', 'break', 'callable-types', 'calls', 'class', 'comparisons', 'compound', diff --git a/Doc/tutorial/introduction.rst b/Doc/tutorial/introduction.rst index 054bac59c955d5..65e3b1938bca9c 100644 --- a/Doc/tutorial/introduction.rst +++ b/Doc/tutorial/introduction.rst @@ -197,21 +197,19 @@ and workarounds. String literals can span multiple lines. One way is using triple-quotes: ``"""..."""`` or ``'''...'''``. End of lines are automatically included in the string, but it's possible to prevent this by adding a ``\`` at -the end of the line. The following example:: - - print("""\ +the end of the line. In the following example, the initial newline is not +included:: + + >>> print("""\ + ... Usage: thingy [OPTIONS] + ... -h Display this usage message + ... -H hostname Hostname to connect to + ... """) Usage: thingy [OPTIONS] -h Display this usage message -H hostname Hostname to connect to - """) - -produces the following output (note that the initial newline is not included): -.. code-block:: text - - Usage: thingy [OPTIONS] - -h Display this usage message - -H hostname Hostname to connect to + >>> Strings can be concatenated (glued together) with the ``+`` operator, and repeated with ``*``:: diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index a47d5e077a357b..565f74149725d5 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -814,6 +814,10 @@ ctypes See :gh:`124520` for discussion and links to changes in some affected projects. +* :class:`ctypes.Structure` objects have a new :attr:`~ctypes.Structure._align_` + attribute which allows the alignment of the structure being packed to/from + memory to be specified explicitly. + (Contributed by Matt Sanderson in :gh:`112433`) dbm --- diff --git a/Doc/whatsnew/3.14.rst b/Doc/whatsnew/3.14.rst index 67d8d389b58082..4d71a24e9cc9ca 100644 --- a/Doc/whatsnew/3.14.rst +++ b/Doc/whatsnew/3.14.rst @@ -587,6 +587,12 @@ Changes in the Python API Wrap it in :func:`staticmethod` if you want to preserve the old behavior. (Contributed by Serhiy Storchaka and Dominykas Grigonis in :gh:`121027`.) +* The :func:`locale.nl_langinfo` function now sets temporarily the ``LC_CTYPE`` + locale in some cases. + This temporary change affects other threads. + (Contributed by Serhiy Storchaka in :gh:`69998`.) + + Build Changes ============= @@ -687,6 +693,10 @@ New Features `__ mentioned in :pep:`630` (:gh:`124153`). +* Add :c:func:`PyUnicode_Equal` function to the limited C API: + test if two strings are equal. + (Contributed by Victor Stinner in :gh:`124502`.) + Porting to Python 3.14 ---------------------- diff --git a/Include/internal/pycore_function.h b/Include/internal/pycore_function.h index 6d44e933e8a8cb..c45d281125febb 100644 --- a/Include/internal/pycore_function.h +++ b/Include/internal/pycore_function.h @@ -18,6 +18,10 @@ extern PyObject* _PyFunction_Vectorcall( #define FUNC_MAX_WATCHERS 8 +#define FUNC_VERSION_UNSET 0 +#define FUNC_VERSION_CLEARED 1 +#define FUNC_VERSION_FIRST_VALID 2 + #define FUNC_VERSION_CACHE_SIZE (1<<12) /* Must be a power of 2 */ struct _func_version_cache_item { @@ -41,6 +45,12 @@ struct _py_func_state { extern PyFunctionObject* _PyFunction_FromConstructor(PyFrameConstructor *constr); +static inline int +_PyFunction_IsVersionValid(uint32_t version) +{ + return version >= FUNC_VERSION_FIRST_VALID; +} + extern uint32_t _PyFunction_GetVersionForCurrentState(PyFunctionObject *func); PyAPI_FUNC(void) _PyFunction_SetVersion(PyFunctionObject *func, uint32_t version); void _PyFunction_ClearCodeByVersion(uint32_t version); diff --git a/Include/internal/pycore_global_objects_fini_generated.h b/Include/internal/pycore_global_objects_fini_generated.h index 28a76c36801b4b..3140a75a47c5ee 100644 --- a/Include/internal/pycore_global_objects_fini_generated.h +++ b/Include/internal/pycore_global_objects_fini_generated.h @@ -757,6 +757,7 @@ _PyStaticObjects_CheckRefcnt(PyInterpreterState *interp) { _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_initializing)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_io)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_is_text_encoding)); + _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_isatty_open_only)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_length_)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_limbo)); _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(_lock_unlock_module)); diff --git a/Include/internal/pycore_global_strings.h b/Include/internal/pycore_global_strings.h index ac789b06fb8a61..1591cb0a3f114f 100644 --- a/Include/internal/pycore_global_strings.h +++ b/Include/internal/pycore_global_strings.h @@ -246,6 +246,7 @@ struct _Py_global_strings { STRUCT_FOR_ID(_initializing) STRUCT_FOR_ID(_io) STRUCT_FOR_ID(_is_text_encoding) + STRUCT_FOR_ID(_isatty_open_only) STRUCT_FOR_ID(_length_) STRUCT_FOR_ID(_limbo) STRUCT_FOR_ID(_lock_unlock_module) diff --git a/Include/internal/pycore_magic_number.h b/Include/internal/pycore_magic_number.h index 2414d25d41bfbf..a88ff2deeba941 100644 --- a/Include/internal/pycore_magic_number.h +++ b/Include/internal/pycore_magic_number.h @@ -259,6 +259,7 @@ Known values: Python 3.14a1 3605 (Move ENTER_EXECUTOR to opcode 255) Python 3.14a1 3606 (Specialize CALL_KW) Python 3.14a1 3607 (Add pseudo instructions JUMP_IF_TRUE/FALSE) + Python 3.14a1 3608 (Add support for slices) Python 3.15 will start with 3650 @@ -271,7 +272,7 @@ PC/launcher.c must also be updated. */ -#define PYC_MAGIC_NUMBER 3607 +#define PYC_MAGIC_NUMBER 3608 /* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes (little-endian) and then appending b'\r\n'. */ #define PYC_MAGIC_NUMBER_TOKEN \ diff --git a/Include/internal/pycore_runtime_init.h b/Include/internal/pycore_runtime_init.h index e6adb98eb19130..a17ba46966daa1 100644 --- a/Include/internal/pycore_runtime_init.h +++ b/Include/internal/pycore_runtime_init.h @@ -11,6 +11,7 @@ extern "C" { #include "pycore_ceval_state.h" // _PyEval_RUNTIME_PERF_INIT #include "pycore_faulthandler.h" // _faulthandler_runtime_state_INIT #include "pycore_floatobject.h" // _py_float_format_unknown +#include "pycore_function.h" #include "pycore_object.h" // _PyObject_HEAD_INIT #include "pycore_obmalloc_init.h" // _obmalloc_global_state_INIT #include "pycore_parser.h" // _parser_runtime_state_INIT @@ -243,7 +244,7 @@ extern PyTypeObject _PyExc_MemoryError; .dict_state = _dict_state_INIT, \ .mem_free_queue = _Py_mem_free_queue_INIT(INTERP.mem_free_queue), \ .func_state = { \ - .next_version = 1, \ + .next_version = FUNC_VERSION_FIRST_VALID, \ }, \ .types = { \ .next_version_tag = _Py_TYPE_BASE_VERSION_TAG, \ diff --git a/Include/internal/pycore_runtime_init_generated.h b/Include/internal/pycore_runtime_init_generated.h index 7847a5c63ebf3f..c9d20d0b5aacdb 100644 --- a/Include/internal/pycore_runtime_init_generated.h +++ b/Include/internal/pycore_runtime_init_generated.h @@ -755,6 +755,7 @@ extern "C" { INIT_ID(_initializing), \ INIT_ID(_io), \ INIT_ID(_is_text_encoding), \ + INIT_ID(_isatty_open_only), \ INIT_ID(_length_), \ INIT_ID(_limbo), \ INIT_ID(_lock_unlock_module), \ diff --git a/Include/internal/pycore_stackref.h b/Include/internal/pycore_stackref.h index b5b6993812057d..cf6dd22cfb18d1 100644 --- a/Include/internal/pycore_stackref.h +++ b/Include/internal/pycore_stackref.h @@ -111,7 +111,8 @@ PyStackRef_AsPyObjectBorrow(_PyStackRef stackref) static inline PyObject * PyStackRef_AsPyObjectSteal(_PyStackRef stackref) { - if (!PyStackRef_IsNull(stackref) && PyStackRef_IsDeferred(stackref)) { + assert(!PyStackRef_IsNull(stackref)); + if (PyStackRef_IsDeferred(stackref)) { return Py_NewRef(PyStackRef_AsPyObjectBorrow(stackref)); } return PyStackRef_AsPyObjectBorrow(stackref); @@ -131,9 +132,10 @@ PyStackRef_AsPyObjectSteal(_PyStackRef stackref) static inline _PyStackRef _PyStackRef_FromPyObjectSteal(PyObject *obj) { + assert(obj != NULL); // Make sure we don't take an already tagged value. assert(((uintptr_t)obj & Py_TAG_BITS) == 0); - unsigned int tag = (obj == NULL || _Py_IsImmortal(obj)) ? (Py_TAG_DEFERRED) : Py_TAG_PTR; + unsigned int tag = _Py_IsImmortal(obj) ? (Py_TAG_DEFERRED) : Py_TAG_PTR; return ((_PyStackRef){.bits = ((uintptr_t)(obj)) | tag}); } # define PyStackRef_FromPyObjectSteal(obj) _PyStackRef_FromPyObjectSteal(_PyObject_CAST(obj)) @@ -193,6 +195,7 @@ PyStackRef_FromPyObjectImmortal(PyObject *obj) # define PyStackRef_CLOSE(REF) \ do { \ _PyStackRef _close_tmp = (REF); \ + assert(!PyStackRef_IsNull(_close_tmp)); \ if (!PyStackRef_IsDeferred(_close_tmp)) { \ Py_DECREF(PyStackRef_AsPyObjectBorrow(_close_tmp)); \ } \ @@ -214,10 +217,11 @@ PyStackRef_FromPyObjectImmortal(PyObject *obj) static inline _PyStackRef PyStackRef_DUP(_PyStackRef stackref) { + assert(!PyStackRef_IsNull(stackref)); if (PyStackRef_IsDeferred(stackref)) { - assert(PyStackRef_IsNull(stackref) || - _Py_IsImmortal(PyStackRef_AsPyObjectBorrow(stackref)) || - _PyObject_HasDeferredRefcount(PyStackRef_AsPyObjectBorrow(stackref))); + assert(_Py_IsImmortal(PyStackRef_AsPyObjectBorrow(stackref)) || + _PyObject_HasDeferredRefcount(PyStackRef_AsPyObjectBorrow(stackref)) + ); return stackref; } Py_INCREF(PyStackRef_AsPyObjectBorrow(stackref)); diff --git a/Include/internal/pycore_unicodeobject_generated.h b/Include/internal/pycore_unicodeobject_generated.h index a688f70a2ba36f..d335373e88ee74 100644 --- a/Include/internal/pycore_unicodeobject_generated.h +++ b/Include/internal/pycore_unicodeobject_generated.h @@ -784,6 +784,10 @@ _PyUnicode_InitStaticStrings(PyInterpreterState *interp) { _PyUnicode_InternStatic(interp, &string); assert(_PyUnicode_CheckConsistency(string, 1)); assert(PyUnicode_GET_LENGTH(string) != 1); + string = &_Py_ID(_isatty_open_only); + _PyUnicode_InternStatic(interp, &string); + assert(_PyUnicode_CheckConsistency(string, 1)); + assert(PyUnicode_GET_LENGTH(string) != 1); string = &_Py_ID(_length_); _PyUnicode_InternStatic(interp, &string); assert(_PyUnicode_CheckConsistency(string, 1)); diff --git a/Include/unicodeobject.h b/Include/unicodeobject.h index dee00715b3c51d..2ce3a008b7129e 100644 --- a/Include/unicodeobject.h +++ b/Include/unicodeobject.h @@ -966,6 +966,10 @@ PyAPI_FUNC(int) PyUnicode_EqualToUTF8(PyObject *, const char *); PyAPI_FUNC(int) PyUnicode_EqualToUTF8AndSize(PyObject *, const char *, Py_ssize_t); #endif +#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030e0000 +PyAPI_FUNC(int) PyUnicode_Equal(PyObject *str1, PyObject *str2); +#endif + /* Rich compare two strings and return one of the following: - NULL in case an exception was raised diff --git a/Lib/_pyio.py b/Lib/_pyio.py index 18849b309b8605..7b6d10c008d3cb 100644 --- a/Lib/_pyio.py +++ b/Lib/_pyio.py @@ -238,7 +238,7 @@ def open(file, mode="r", buffering=-1, encoding=None, errors=None, result = raw try: line_buffering = False - if buffering == 1 or buffering < 0 and raw.isatty(): + if buffering == 1 or buffering < 0 and raw._isatty_open_only(): buffering = -1 line_buffering = True if buffering < 0: @@ -1794,6 +1794,21 @@ def isatty(self): self._checkClosed() return os.isatty(self._fd) + def _isatty_open_only(self): + """Checks whether the file is a TTY using an open-only optimization. + + TTYs are always character devices. If the interpreter knows a file is + not a character device when it would call ``isatty``, can skip that + call. Inside ``open()`` there is a fresh stat result that contains that + information. Use the stat result to skip a system call. Outside of that + context TOCTOU issues (the fd could be arbitrarily modified by + surrounding code). + """ + if (self._stat_atopen is not None + and not stat.S_ISCHR(self._stat_atopen.st_mode)): + return False + return os.isatty(self._fd) + @property def closefd(self): """True if the file descriptor will be closed by close().""" diff --git a/Lib/enum.py b/Lib/enum.py index 9d53eb86bc2116..17d72738792982 100644 --- a/Lib/enum.py +++ b/Lib/enum.py @@ -1092,6 +1092,21 @@ def _add_member_(cls, name, member): # now add to _member_map_ (even aliases) cls._member_map_[name] = member + @property + def __signature__(cls): + from inspect import Parameter, Signature + if cls._member_names_: + return Signature([Parameter('values', Parameter.VAR_POSITIONAL)]) + else: + return Signature([Parameter('new_class_name', Parameter.POSITIONAL_ONLY), + Parameter('names', Parameter.POSITIONAL_OR_KEYWORD), + Parameter('module', Parameter.KEYWORD_ONLY, default=None), + Parameter('qualname', Parameter.KEYWORD_ONLY, default=None), + Parameter('type', Parameter.KEYWORD_ONLY, default=None), + Parameter('start', Parameter.KEYWORD_ONLY, default=1), + Parameter('boundary', Parameter.KEYWORD_ONLY, default=None)]) + + EnumMeta = EnumType # keep EnumMeta name for backwards compatibility @@ -1135,13 +1150,6 @@ class Enum(metaclass=EnumType): attributes -- see the documentation for details. """ - @classmethod - def __signature__(cls): - if cls._member_names_: - return '(*values)' - else: - return '(new_class_name, /, names, *, module=None, qualname=None, type=None, start=1, boundary=None)' - def __new__(cls, value): # all enum instances are actually created during class construction # without calling this method; this method is called by the metaclass' diff --git a/Lib/idlelib/tree.py b/Lib/idlelib/tree.py index 0726d7e23660f6..182ce7189614da 100644 --- a/Lib/idlelib/tree.py +++ b/Lib/idlelib/tree.py @@ -83,6 +83,8 @@ def wheel_event(event, widget=None): class TreeNode: + dy = 0 + def __init__(self, canvas, parent, item): self.canvas = canvas self.parent = parent @@ -199,23 +201,22 @@ def update(self): def draw(self, x, y): # XXX This hard-codes too many geometry constants! - dy = 20 self.x, self.y = x, y self.drawicon() self.drawtext() if self.state != 'expanded': - return y + dy + return y + TreeNode.dy # draw children if not self.children: sublist = self.item._GetSubList() if not sublist: # _IsExpandable() was mistaken; that's allowed - return y+17 + return y + TreeNode.dy for item in sublist: child = self.__class__(self.canvas, self, item) self.children.append(child) cx = x+20 - cy = y + dy + cy = y + TreeNode.dy cylast = 0 for child in self.children: cylast = cy @@ -289,6 +290,11 @@ def drawtext(self): self.label.bind("", lambda e: wheel_event(e, self.canvas)) self.label.bind("", lambda e: wheel_event(e, self.canvas)) self.text_id = id + if TreeNode.dy == 0: + # The first row doesn't matter what the dy is, just measure its + # size to get the value of the subsequent dy + coords = self.canvas.bbox(id) + TreeNode.dy = max(20, coords[3] - coords[1] - 3) def select_or_edit(self, event=None): if self.selected and self.item.IsEditable(): diff --git a/Lib/inspect.py b/Lib/inspect.py index 17314564f35397..1763ef640bbe04 100644 --- a/Lib/inspect.py +++ b/Lib/inspect.py @@ -2424,18 +2424,10 @@ def _signature_from_callable(obj, *, pass else: if sig is not None: - # since __text_signature__ is not writable on classes, __signature__ - # may contain text (or be a callable that returns text); - # if so, convert it - o_sig = sig - if not isinstance(sig, (Signature, str)) and callable(sig): - sig = sig() - if isinstance(sig, str): - sig = _signature_fromstr(sigcls, obj, sig) if not isinstance(sig, Signature): raise TypeError( 'unexpected object {!r} in __signature__ ' - 'attribute'.format(o_sig)) + 'attribute'.format(sig)) return sig try: diff --git a/Lib/logging/config.py b/Lib/logging/config.py index 3781cb1aeb9ae2..6a6a7f726f7e0c 100644 --- a/Lib/logging/config.py +++ b/Lib/logging/config.py @@ -499,7 +499,7 @@ def as_tuple(self, value): def _is_queue_like_object(obj): """Check that *obj* implements the Queue API.""" - if isinstance(obj, queue.Queue): + if isinstance(obj, (queue.Queue, queue.SimpleQueue)): return True # defer importing multiprocessing as much as possible from multiprocessing.queues import Queue as MPQueue @@ -516,13 +516,13 @@ def _is_queue_like_object(obj): # Ideally, we would have wanted to simply use strict type checking # instead of a protocol-based type checking since the latter does # not check the method signatures. - queue_interface = [ - 'empty', 'full', 'get', 'get_nowait', - 'put', 'put_nowait', 'join', 'qsize', - 'task_done', - ] + # + # Note that only 'put_nowait' and 'get' are required by the logging + # queue handler and queue listener (see gh-124653) and that other + # methods are either optional or unused. + minimal_queue_interface = ['put_nowait', 'get'] return all(callable(getattr(obj, method, None)) - for method in queue_interface) + for method in minimal_queue_interface) class DictConfigurator(BaseConfigurator): """ diff --git a/Lib/pydoc_data/topics.py b/Lib/pydoc_data/topics.py index 97bb4eb52f4386..ae56c136608472 100644 --- a/Lib/pydoc_data/topics.py +++ b/Lib/pydoc_data/topics.py @@ -417,33 +417,43 @@ 'caused a\n' 'syntax error.\n', 'assignment-expressions': 'Assignment expressions\n' - '**********************\n' - '\n' - 'An assignment expression (sometimes also called a “named expression”' - '\nor “walrus”) assigns an expression to an identifier, while also\n' - 'returning the value of the expression.\n' - '\n' - 'One common use case is when handling matched regular expressions:\n' - '\n' - ' if matching := pattern.search(data):\n' - ' do_something(matching)\n' - '\n' - 'Or, when processing a file stream in chunks:\n' - '\n' - ' while chunk := file.read(9000):\n' - ' process(chunk)\n' - '\n' - 'Assignment expressions must be surrounded by parentheses when used as\n' - 'expression statements and when used as sub-expressions in slicing,\n' - 'conditional, lambda, keyword-argument, and comprehension-if\n' - 'expressions and in assert, with, and assignment statements. In all\n' - 'other places where they can be used, parentheses are not required,\n' - 'including in if and while statements.\n' - '\n' - 'Added in version 3.8.\n' - 'See also:\n' - '\n' - ' **PEP 572** - Assignment Expressions\n', + '**********************\n' + '\n' + ' assignment_expression ::= [identifier ":="] ' + 'expression\n' + '\n' + 'An assignment expression (sometimes also called a ' + '“named expression”\n' + 'or “walrus”) assigns an "expression" to an ' + '"identifier", while also\n' + 'returning the value of the "expression".\n' + '\n' + 'One common use case is when handling matched ' + 'regular expressions:\n' + '\n' + ' if matching := pattern.search(data):\n' + ' do_something(matching)\n' + '\n' + 'Or, when processing a file stream in chunks:\n' + '\n' + ' while chunk := file.read(9000):\n' + ' process(chunk)\n' + '\n' + 'Assignment expressions must be surrounded by ' + 'parentheses when used as\n' + 'expression statements and when used as ' + 'sub-expressions in slicing,\n' + 'conditional, lambda, keyword-argument, and ' + 'comprehension-if\n' + 'expressions and in "assert", "with", and ' + '"assignment" statements. In\n' + 'all other places where they can be used, ' + 'parentheses are not required,\n' + 'including in "if" and "while" statements.\n' + '\n' + 'Added in version 3.8: See **PEP 572** for more ' + 'details about\n' + 'assignment expressions.\n', 'async': 'Coroutines\n' '**********\n' '\n' diff --git a/Lib/site.py b/Lib/site.py index cafd3ab70b2cac..b3194d79fb5ab8 100644 --- a/Lib/site.py +++ b/Lib/site.py @@ -491,12 +491,21 @@ def register_readline(): This can be overridden in the sitecustomize or usercustomize module, or in a PYTHONSTARTUP file. """ + if not sys.flags.ignore_environment: + PYTHON_BASIC_REPL = os.getenv("PYTHON_BASIC_REPL") + else: + PYTHON_BASIC_REPL = False + import atexit try: import readline import rlcompleter # noqa: F401 - import _pyrepl.readline - import _pyrepl.unix_console + if PYTHON_BASIC_REPL: + CAN_USE_PYREPL = False + else: + import _pyrepl.readline + import _pyrepl.unix_console + from _pyrepl.main import CAN_USE_PYREPL except ImportError: return @@ -517,7 +526,6 @@ def register_readline(): pass if readline.get_current_history_length() == 0: - from _pyrepl.main import CAN_USE_PYREPL # If no history was loaded, default to .python_history, # or PYTHON_HISTORY. # The guard is necessary to avoid doubling history size at @@ -525,13 +533,17 @@ def register_readline(): # through a PYTHONSTARTUP hook, see: # http://bugs.python.org/issue5845#msg198636 history = gethistoryfile() - if os.getenv("PYTHON_BASIC_REPL") or not CAN_USE_PYREPL: - readline_module = readline - else: + + if CAN_USE_PYREPL: readline_module = _pyrepl.readline + exceptions = (OSError, *_pyrepl.unix_console._error) + else: + readline_module = readline + exceptions = OSError + try: readline_module.read_history_file(history) - except (OSError,* _pyrepl.unix_console._error): + except exceptions: pass def write_history(): diff --git a/Lib/test/certdata/make_ssl_certs.py b/Lib/test/certdata/make_ssl_certs.py index 48f980124e1198..198c64035c5044 100644 --- a/Lib/test/certdata/make_ssl_certs.py +++ b/Lib/test/certdata/make_ssl_certs.py @@ -139,7 +139,6 @@ def make_cert_key(cmdlineargs, hostname, sign=False, extra_san='', f.write(req) args = ['req', '-new', '-nodes', '-days', cmdlineargs.days, '-newkey', key, '-keyout', key_file, - '-extensions', ext, '-config', req_file] if sign: with tempfile.NamedTemporaryFile(delete=False) as f: @@ -148,7 +147,7 @@ def make_cert_key(cmdlineargs, hostname, sign=False, extra_san='', args += ['-out', reqfile ] else: - args += ['-x509', '-out', cert_file ] + args += ['-extensions', ext, '-x509', '-out', cert_file ] check_call(['openssl'] + args) if sign: diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py index 1722cc8612ca6b..8ef8fae44f1d25 100644 --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -26,7 +26,7 @@ from test import support from test.support import os_helper from test.support import ( - TestFailed, run_with_locale, no_tracing, + TestFailed, run_with_locales, no_tracing, _2G, _4G, bigmemtest ) from test.support.import_helper import forget @@ -2895,7 +2895,7 @@ def test_float(self): got = self.loads(pickle) self.assert_is_copy(value, got) - @run_with_locale('LC_ALL', 'de_DE', 'fr_FR') + @run_with_locales('LC_ALL', 'de_DE', 'fr_FR', '') def test_float_format(self): # make sure that floats are formatted locale independent with proto 0 self.assertEqual(self.dumps(1.2, 0)[0:3], b'F1.') diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py index 1a44cc638b5714..72ce5dacd1be4c 100644 --- a/Lib/test/support/__init__.py +++ b/Lib/test/support/__init__.py @@ -930,8 +930,8 @@ def check_sizeof(test, o, size): test.assertEqual(result, size, msg) #======================================================================= -# Decorator for running a function in a different locale, correctly resetting -# it afterwards. +# Decorator/context manager for running a code in a different locale, +# correctly resetting it afterwards. @contextlib.contextmanager def run_with_locale(catstr, *locales): @@ -942,16 +942,21 @@ def run_with_locale(catstr, *locales): except AttributeError: # if the test author gives us an invalid category string raise - except: + except Exception: # cannot retrieve original locale, so do nothing locale = orig_locale = None + if '' not in locales: + raise unittest.SkipTest('no locales') else: for loc in locales: try: locale.setlocale(category, loc) break - except: + except locale.Error: pass + else: + if '' not in locales: + raise unittest.SkipTest(f'no locales {locales}') try: yield @@ -959,6 +964,46 @@ def run_with_locale(catstr, *locales): if locale and orig_locale: locale.setlocale(category, orig_locale) +#======================================================================= +# Decorator for running a function in multiple locales (if they are +# availasble) and resetting the original locale afterwards. + +def run_with_locales(catstr, *locales): + def deco(func): + @functools.wraps(func) + def wrapper(self, /, *args, **kwargs): + dry_run = '' in locales + try: + import locale + category = getattr(locale, catstr) + orig_locale = locale.setlocale(category) + except AttributeError: + # if the test author gives us an invalid category string + raise + except Exception: + # cannot retrieve original locale, so do nothing + pass + else: + try: + for loc in locales: + with self.subTest(locale=loc): + try: + locale.setlocale(category, loc) + except locale.Error: + self.skipTest(f'no locale {loc!r}') + else: + dry_run = False + func(self, *args, **kwargs) + finally: + locale.setlocale(category, orig_locale) + if dry_run: + # no locales available, so just run the test + # with the current locale + with self.subTest(locale=None): + func(self, *args, **kwargs) + return wrapper + return deco + #======================================================================= # Decorator for running a function in a specific timezone, correctly # resetting it afterwards. diff --git a/Lib/test/test__locale.py b/Lib/test/test__locale.py index 0947464bb8c04e..ba2d31f9c1ee9d 100644 --- a/Lib/test/test__locale.py +++ b/Lib/test/test__locale.py @@ -115,16 +115,17 @@ def numeric_tester(self, calc_type, calc_value, data_type, used_locale): def test_lc_numeric_nl_langinfo(self): # Test nl_langinfo against known values tested = False + oldloc = setlocale(LC_CTYPE) for loc in candidate_locales: try: setlocale(LC_NUMERIC, loc) - setlocale(LC_CTYPE, loc) except Error: continue for li, lc in ((RADIXCHAR, "decimal_point"), (THOUSEP, "thousands_sep")): if self.numeric_tester('nl_langinfo', nl_langinfo(li), lc, loc): tested = True + self.assertEqual(setlocale(LC_CTYPE), oldloc) if not tested: self.skipTest('no suitable locales') @@ -135,10 +136,10 @@ def test_lc_numeric_nl_langinfo(self): def test_lc_numeric_localeconv(self): # Test localeconv against known values tested = False + oldloc = setlocale(LC_CTYPE) for loc in candidate_locales: try: setlocale(LC_NUMERIC, loc) - setlocale(LC_CTYPE, loc) except Error: continue formatting = localeconv() @@ -146,6 +147,7 @@ def test_lc_numeric_localeconv(self): "thousands_sep"): if self.numeric_tester('localeconv', formatting[lc], lc, loc): tested = True + self.assertEqual(setlocale(LC_CTYPE), oldloc) if not tested: self.skipTest('no suitable locales') @@ -153,10 +155,10 @@ def test_lc_numeric_localeconv(self): def test_lc_numeric_basic(self): # Test nl_langinfo against localeconv tested = False + oldloc = setlocale(LC_CTYPE) for loc in candidate_locales: try: setlocale(LC_NUMERIC, loc) - setlocale(LC_CTYPE, loc) except Error: continue for li, lc in ((RADIXCHAR, "decimal_point"), @@ -173,6 +175,7 @@ def test_lc_numeric_basic(self): nl_radixchar, li_radixchar, loc, set_locale)) tested = True + self.assertEqual(setlocale(LC_CTYPE), oldloc) if not tested: self.skipTest('no suitable locales') @@ -180,10 +183,10 @@ def test_float_parsing(self): # Bug #1391872: Test whether float parsing is okay on European # locales. tested = False + oldloc = setlocale(LC_CTYPE) for loc in candidate_locales: try: setlocale(LC_NUMERIC, loc) - setlocale(LC_CTYPE, loc) except Error: continue @@ -199,6 +202,7 @@ def test_float_parsing(self): self.assertRaises(ValueError, float, localeconv()['decimal_point'].join(['1', '23'])) tested = True + self.assertEqual(setlocale(LC_CTYPE), oldloc) if not tested: self.skipTest('no suitable locales') diff --git a/Lib/test/test_capi/test_unicode.py b/Lib/test/test_capi/test_unicode.py index e6f85427214958..65d8242ad3fc60 100644 --- a/Lib/test/test_capi/test_unicode.py +++ b/Lib/test/test_capi/test_unicode.py @@ -1903,6 +1903,39 @@ def test_recover_error(self): self.assertEqual(writer.finish(), 'Hello World.') + def test_unicode_equal(self): + unicode_equal = _testlimitedcapi.unicode_equal + + def copy(text): + return text.encode().decode() + + self.assertTrue(unicode_equal("", "")) + self.assertTrue(unicode_equal("abc", "abc")) + self.assertTrue(unicode_equal("abc", copy("abc"))) + self.assertTrue(unicode_equal("\u20ac", copy("\u20ac"))) + self.assertTrue(unicode_equal("\U0010ffff", copy("\U0010ffff"))) + + self.assertFalse(unicode_equal("abc", "abcd")) + self.assertFalse(unicode_equal("\u20ac", "\u20ad")) + self.assertFalse(unicode_equal("\U0010ffff", "\U0010fffe")) + + # str subclass + self.assertTrue(unicode_equal("abc", Str("abc"))) + self.assertTrue(unicode_equal(Str("abc"), "abc")) + self.assertFalse(unicode_equal("abc", Str("abcd"))) + self.assertFalse(unicode_equal(Str("abc"), "abcd")) + + # invalid type + for invalid_type in (b'bytes', 123, ("tuple",)): + with self.subTest(invalid_type=invalid_type): + with self.assertRaises(TypeError): + unicode_equal("abc", invalid_type) + with self.assertRaises(TypeError): + unicode_equal(invalid_type, "abc") + + # CRASHES unicode_equal("abc", NULL) + # CRASHES unicode_equal(NULL, "abc") + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index 428036e1765b8f..290656f070503a 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -2,7 +2,6 @@ import contextlib import copy import io -import locale import pickle import sys import unittest @@ -1812,16 +1811,10 @@ def test_getwriter(self): self.assertRaises(TypeError, codecs.getwriter) self.assertRaises(LookupError, codecs.getwriter, "__spam__") + @support.run_with_locale('LC_CTYPE', 'tr_TR') def test_lookup_issue1813(self): # Issue #1813: under Turkish locales, lookup of some codecs failed # because 'I' is lowercased as "ı" (dotless i) - oldlocale = locale.setlocale(locale.LC_CTYPE) - self.addCleanup(locale.setlocale, locale.LC_CTYPE, oldlocale) - try: - locale.setlocale(locale.LC_CTYPE, 'tr_TR') - except locale.Error: - # Unsupported locale on this system - self.skipTest('test needs Turkish locale') c = codecs.lookup('ASCII') self.assertEqual(c.name, 'ascii') diff --git a/Lib/test/test_compile.py b/Lib/test/test_compile.py index e6dc7a53189e12..6f838da6018741 100644 --- a/Lib/test/test_compile.py +++ b/Lib/test/test_compile.py @@ -1386,6 +1386,14 @@ def check_op_count(func, op, expected): actual += 1 self.assertEqual(actual, expected) + def check_consts(func, typ, expected): + slice_consts = 0 + consts = func.__code__.co_consts + for instr in dis.Bytecode(func): + if instr.opname == "LOAD_CONST" and isinstance(consts[instr.oparg], typ): + slice_consts += 1 + self.assertEqual(slice_consts, expected) + def load(): return x[a:b] + x [a:] + x[:b] + x[:] @@ -1401,15 +1409,30 @@ def long_slice(): def aug(): x[a:b] += y - check_op_count(load, "BINARY_SLICE", 4) + def aug_const(): + x[1:2] += y + + def compound_const_slice(): + x[1:2:3, 4:5:6] = y + + check_op_count(load, "BINARY_SLICE", 3) check_op_count(load, "BUILD_SLICE", 0) - check_op_count(store, "STORE_SLICE", 4) + check_consts(load, slice, 1) + check_op_count(store, "STORE_SLICE", 3) check_op_count(store, "BUILD_SLICE", 0) + check_consts(store, slice, 1) check_op_count(long_slice, "BUILD_SLICE", 1) check_op_count(long_slice, "BINARY_SLICE", 0) check_op_count(aug, "BINARY_SLICE", 1) check_op_count(aug, "STORE_SLICE", 1) check_op_count(aug, "BUILD_SLICE", 0) + check_op_count(aug_const, "BINARY_SLICE", 0) + check_op_count(aug_const, "STORE_SLICE", 0) + check_consts(aug_const, slice, 1) + check_op_count(compound_const_slice, "BINARY_SLICE", 0) + check_op_count(compound_const_slice, "BUILD_SLICE", 0) + check_consts(compound_const_slice, slice, 0) + check_consts(compound_const_slice, tuple, 1) def test_compare_positions(self): for opname_prefix, op in [ diff --git a/Lib/test/test_decimal.py b/Lib/test/test_decimal.py index c591fd54430b18..d1e7e69e7e951b 100644 --- a/Lib/test/test_decimal.py +++ b/Lib/test/test_decimal.py @@ -1253,7 +1253,7 @@ def test_deprecated_N_format(self): self.assertRaises(ValueError, format, h, '10Nf') self.assertRaises(ValueError, format, h, 'Nx') - @run_with_locale('LC_ALL', 'ps_AF') + @run_with_locale('LC_ALL', 'ps_AF', '') def test_wide_char_separator_decimal_point(self): # locale with wide char separator and decimal point Decimal = self.decimal.Decimal diff --git a/Lib/test/test_embed.py b/Lib/test/test_embed.py index 3edc19d8254754..035d4418b15c51 100644 --- a/Lib/test/test_embed.py +++ b/Lib/test/test_embed.py @@ -168,7 +168,8 @@ def run_repeated_init_and_subinterpreters(self): # Parse the line from the loop. The first line is the main # interpreter and the 3 afterward are subinterpreters. interp = Interp(*match.groups()) - if support.verbose > 1: + if support.verbose > 2: + # 5 lines per pass is super-spammy, so limit that to -vvv print(interp) self.assertTrue(interp.interp) self.assertTrue(interp.tstate) @@ -279,6 +280,10 @@ def test_pre_initialization_api(self): """ env = dict(os.environ, PYTHONPATH=os.pathsep.join(sys.path)) out, err = self.run_embedded_interpreter("test_pre_initialization_api", env=env) + if support.verbose > 1: + print() + print(out) + print(err) if MS_WINDOWS: expected_path = self.test_exe else: @@ -296,6 +301,10 @@ def test_pre_initialization_sys_options(self): env['PYTHONPATH'] = os.pathsep.join(sys.path) out, err = self.run_embedded_interpreter( "test_pre_initialization_sys_options", env=env) + if support.verbose > 1: + print() + print(out) + print(err) expected_output = ( "sys.warnoptions: ['once', 'module', 'default']\n" "sys._xoptions: {'not_an_option': '1', 'also_not_an_option': '2'}\n" diff --git a/Lib/test/test_exceptions.py b/Lib/test/test_exceptions.py index ba858c49400911..b3c21cd4f3d585 100644 --- a/Lib/test/test_exceptions.py +++ b/Lib/test/test_exceptions.py @@ -8,6 +8,7 @@ import weakref import errno from codecs import BOM_UTF8 +from itertools import product from textwrap import dedent from test.support import (captured_stderr, check_impl_detail, @@ -1336,6 +1337,29 @@ def test_unicode_errors_no_object(self): for klass in klasses: self.assertEqual(str(klass.__new__(klass)), "") + def test_unicode_error_str_does_not_crash(self): + # Test that str(UnicodeError(...)) does not crash. + # See https://github.com/python/cpython/issues/123378. + + for start, end, objlen in product( + range(-5, 5), + range(-5, 5), + range(7), + ): + obj = 'a' * objlen + with self.subTest('encode', objlen=objlen, start=start, end=end): + exc = UnicodeEncodeError('utf-8', obj, start, end, '') + self.assertIsInstance(str(exc), str) + + with self.subTest('translate', objlen=objlen, start=start, end=end): + exc = UnicodeTranslateError(obj, start, end, '') + self.assertIsInstance(str(exc), str) + + encoded = obj.encode() + with self.subTest('decode', objlen=objlen, start=start, end=end): + exc = UnicodeDecodeError('utf-8', encoded, start, end, '') + self.assertIsInstance(str(exc), str) + @no_tracing def test_badisinstance(self): # Bug #2542: if issubclass(e, MyException) raises an exception, diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py index 048bb14509064b..f588e16b70123a 100644 --- a/Lib/test/test_float.py +++ b/Lib/test/test_float.py @@ -175,7 +175,7 @@ def check(s): # non-UTF-8 byte string check(b'123\xa0') - @support.run_with_locale('LC_NUMERIC', 'fr_FR', 'de_DE') + @support.run_with_locale('LC_NUMERIC', 'fr_FR', 'de_DE', '') def test_float_with_comma(self): # set locale to something that doesn't use '.' for the decimal point # float must not accept the locale specific decimal point but diff --git a/Lib/test/test_imaplib.py b/Lib/test/test_imaplib.py index 1fd75d0a3f4c7b..a6509fc3ba0eae 100644 --- a/Lib/test/test_imaplib.py +++ b/Lib/test/test_imaplib.py @@ -57,7 +57,7 @@ def timevalues(self): timezone(timedelta(0, 2 * 60 * 60))), '"18-May-2033 05:33:20 +0200"'] - @run_with_locale('LC_ALL', 'de_DE', 'fr_FR') + @run_with_locale('LC_ALL', 'de_DE', 'fr_FR', '') # DST rules included to work around quirk where the Gnu C library may not # otherwise restore the previous time zone @run_with_tz('STD-1DST,M3.2.0,M11.1.0') diff --git a/Lib/test/test_inspect/test_inspect.py b/Lib/test/test_inspect/test_inspect.py index d2dc9e147d29c2..2ecb7ec1e26e0e 100644 --- a/Lib/test/test_inspect/test_inspect.py +++ b/Lib/test/test_inspect/test_inspect.py @@ -4879,38 +4879,6 @@ def foo(): pass self.assertEqual(signature_func(foo), inspect.Signature()) self.assertEqual(inspect.get_annotations(foo), {}) - def test_signature_as_str(self): - self.maxDiff = None - class S: - __signature__ = '(a, b=2)' - - self.assertEqual(self.signature(S), - ((('a', ..., ..., 'positional_or_keyword'), - ('b', 2, ..., 'positional_or_keyword')), - ...)) - - def test_signature_as_callable(self): - # __signature__ should be either a staticmethod or a bound classmethod - class S: - @classmethod - def __signature__(cls): - return '(a, b=2)' - - self.assertEqual(self.signature(S), - ((('a', ..., ..., 'positional_or_keyword'), - ('b', 2, ..., 'positional_or_keyword')), - ...)) - - class S: - @staticmethod - def __signature__(): - return '(a, b=2)' - - self.assertEqual(self.signature(S), - ((('a', ..., ..., 'positional_or_keyword'), - ('b', 2, ..., 'positional_or_keyword')), - ...)) - def test_signature_on_derived_classes(self): # gh-105080: Make sure that signatures are consistent on derived classes diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index 230ba954cd286d..d4ceb7c8dc0b41 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -2377,16 +2377,22 @@ def __getattr__(self, attribute): return getattr(queue, attribute) class CustomQueueFakeProtocol(CustomQueueProtocol): - # An object implementing the Queue API (incorrect signatures). + # An object implementing the minimial Queue API for + # the logging module but with incorrect signatures. + # # The object will be considered a valid queue class since we # do not check the signatures (only callability of methods) # but will NOT be usable in production since a TypeError will - # be raised due to a missing argument. - def empty(self, x): + # be raised due to the extra argument in 'put_nowait'. + def put_nowait(self): pass class CustomQueueWrongProtocol(CustomQueueProtocol): - empty = None + put_nowait = None + +class MinimalQueueProtocol: + def put_nowait(self, x): pass + def get(self): pass def queueMaker(): return queue.Queue() @@ -3946,56 +3952,70 @@ def test_config_queue_handler(self): msg = str(ctx.exception) self.assertEqual(msg, "Unable to configure handler 'ah'") + def _apply_simple_queue_listener_configuration(self, qspec): + self.apply_config({ + "version": 1, + "handlers": { + "queue_listener": { + "class": "logging.handlers.QueueHandler", + "queue": qspec, + }, + }, + }) + @threading_helper.requires_working_threading() @support.requires_subprocess() @patch("multiprocessing.Manager") def test_config_queue_handler_does_not_create_multiprocessing_manager(self, manager): - # gh-120868, gh-121723 - - from multiprocessing import Queue as MQ - - q1 = {"()": "queue.Queue", "maxsize": -1} - q2 = MQ() - q3 = queue.Queue() - # CustomQueueFakeProtocol passes the checks but will not be usable - # since the signatures are incompatible. Checking the Queue API - # without testing the type of the actual queue is a trade-off - # between usability and the work we need to do in order to safely - # check that the queue object correctly implements the API. - q4 = CustomQueueFakeProtocol() - - for qspec in (q1, q2, q3, q4): - self.apply_config( - { - "version": 1, - "handlers": { - "queue_listener": { - "class": "logging.handlers.QueueHandler", - "queue": qspec, - }, - }, - } - ) - manager.assert_not_called() + # gh-120868, gh-121723, gh-124653 + + for qspec in [ + {"()": "queue.Queue", "maxsize": -1}, + queue.Queue(), + # queue.SimpleQueue does not inherit from queue.Queue + queue.SimpleQueue(), + # CustomQueueFakeProtocol passes the checks but will not be usable + # since the signatures are incompatible. Checking the Queue API + # without testing the type of the actual queue is a trade-off + # between usability and the work we need to do in order to safely + # check that the queue object correctly implements the API. + CustomQueueFakeProtocol(), + MinimalQueueProtocol(), + ]: + with self.subTest(qspec=qspec): + self._apply_simple_queue_listener_configuration(qspec) + manager.assert_not_called() @patch("multiprocessing.Manager") def test_config_queue_handler_invalid_config_does_not_create_multiprocessing_manager(self, manager): # gh-120868, gh-121723 for qspec in [object(), CustomQueueWrongProtocol()]: - with self.assertRaises(ValueError): - self.apply_config( - { - "version": 1, - "handlers": { - "queue_listener": { - "class": "logging.handlers.QueueHandler", - "queue": qspec, - }, - }, - } - ) - manager.assert_not_called() + with self.subTest(qspec=qspec), self.assertRaises(ValueError): + self._apply_simple_queue_listener_configuration(qspec) + manager.assert_not_called() + + @skip_if_tsan_fork + @support.requires_subprocess() + @unittest.skipUnless(support.Py_DEBUG, "requires a debug build for testing" + " assertions in multiprocessing") + def test_config_reject_simple_queue_handler_multiprocessing_context(self): + # multiprocessing.SimpleQueue does not implement 'put_nowait' + # and thus cannot be used as a queue-like object (gh-124653) + + import multiprocessing + + if support.MS_WINDOWS: + start_methods = ['spawn'] + else: + start_methods = ['spawn', 'fork', 'forkserver'] + + for start_method in start_methods: + with self.subTest(start_method=start_method): + ctx = multiprocessing.get_context(start_method) + qspec = ctx.SimpleQueue() + with self.assertRaises(ValueError): + self._apply_simple_queue_listener_configuration(qspec) @skip_if_tsan_fork @support.requires_subprocess() diff --git a/Lib/test/test_pyrepl/test_pyrepl.py b/Lib/test/test_pyrepl/test_pyrepl.py index 36f940eaea4eac..1a76832386bf1d 100644 --- a/Lib/test/test_pyrepl/test_pyrepl.py +++ b/Lib/test/test_pyrepl/test_pyrepl.py @@ -1204,6 +1204,18 @@ def test_python_basic_repl(self): self.assertNotIn("Exception", output) self.assertNotIn("Traceback", output) + # The site module must not load _pyrepl if PYTHON_BASIC_REPL is set + commands = ("import sys\n" + "print('_pyrepl' in sys.modules)\n" + "exit()\n") + env["PYTHON_BASIC_REPL"] = "1" + output, exit_code = self.run_repl(commands, env=env) + self.assertEqual(exit_code, 0) + self.assertIn("False", output) + self.assertNotIn("True", output) + self.assertNotIn("Exception", output) + self.assertNotIn("Traceback", output) + @force_not_colorized def test_bad_sys_excepthook_doesnt_crash_pyrepl(self): env = os.environ.copy() diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py index d16ad7ef5d4328..b14d500a9c6e97 100644 --- a/Lib/test/test_stable_abi_ctypes.py +++ b/Lib/test/test_stable_abi_ctypes.py @@ -805,6 +805,7 @@ def test_windows_feature_macros(self): "PyUnicode_DecodeUnicodeEscape", "PyUnicode_EncodeFSDefault", "PyUnicode_EncodeLocale", + "PyUnicode_Equal", "PyUnicode_EqualToUTF8", "PyUnicode_EqualToUTF8AndSize", "PyUnicode_FSConverter", diff --git a/Lib/test/test_str.py b/Lib/test/test_str.py index 6600dcf9157971..4de6c1cba152bd 100644 --- a/Lib/test/test_str.py +++ b/Lib/test/test_str.py @@ -1665,7 +1665,7 @@ def test_startswith_endswith_errors(self): self.assertIn('str', exc) self.assertIn('tuple', exc) - @support.run_with_locale('LC_ALL', 'de_DE', 'fr_FR') + @support.run_with_locale('LC_ALL', 'de_DE', 'fr_FR', '') def test_format_float(self): # should not format with a comma, but always with C locale self.assertEqual('1.0', '%.1f' % 1.0) diff --git a/Lib/test/test_strptime.py b/Lib/test/test_strptime.py index 038746e26c24ad..37f6b08db28b3e 100644 --- a/Lib/test/test_strptime.py +++ b/Lib/test/test_strptime.py @@ -7,7 +7,8 @@ import os import sys from test import support -from test.support import skip_if_buggy_ucrt_strfptime, warnings_helper +from test.support import warnings_helper +from test.support import skip_if_buggy_ucrt_strfptime, run_with_locales from datetime import date as datetime_date import _strptime @@ -207,8 +208,8 @@ class StrptimeTests(unittest.TestCase): """Tests for _strptime.strptime.""" def setUp(self): - """Create testing time tuple.""" - self.time_tuple = time.gmtime() + """Create testing time tuples.""" + self.time_tuple = time.localtime() def test_ValueError(self): # Make sure ValueError is raised when match fails or format is bad @@ -289,54 +290,67 @@ def test_unconverteddata(self): # Check ValueError is raised when there is unconverted data self.assertRaises(ValueError, _strptime._strptime_time, "10 12", "%m") - def helper(self, directive, position): + def roundtrip(self, fmt, position, time_tuple=None): """Helper fxn in testing.""" - fmt = "%d %Y" if directive == 'd' else "%" + directive - strf_output = time.strftime(fmt, self.time_tuple) + if time_tuple is None: + time_tuple = self.time_tuple + strf_output = time.strftime(fmt, time_tuple) strp_output = _strptime._strptime_time(strf_output, fmt) - self.assertTrue(strp_output[position] == self.time_tuple[position], - "testing of '%s' directive failed; '%s' -> %s != %s" % - (directive, strf_output, strp_output[position], - self.time_tuple[position])) + self.assertEqual(strp_output[position], time_tuple[position], + "testing of %r format failed; %r -> %r != %r" % + (fmt, strf_output, strp_output[position], + time_tuple[position])) + if support.verbose >= 3: + print("testing of %r format: %r -> %r" % + (fmt, strf_output, strp_output[position])) def test_year(self): # Test that the year is handled properly - for directive in ('y', 'Y'): - self.helper(directive, 0) + self.roundtrip('%Y', 0) + self.roundtrip('%y', 0) + self.roundtrip('%Y', 0, (1900, 1, 1, 0, 0, 0, 0, 1, 0)) + # Must also make sure %y values are correct for bounds set by Open Group - for century, bounds in ((1900, ('69', '99')), (2000, ('00', '68'))): - for bound in bounds: - strp_output = _strptime._strptime_time(bound, '%y') - expected_result = century + int(bound) - self.assertTrue(strp_output[0] == expected_result, - "'y' test failed; passed in '%s' " - "and returned '%s'" % (bound, strp_output[0])) + strptime = _strptime._strptime_time + self.assertEqual(strptime('00', '%y')[0], 2000) + self.assertEqual(strptime('68', '%y')[0], 2068) + self.assertEqual(strptime('69', '%y')[0], 1969) + self.assertEqual(strptime('99', '%y')[0], 1999) def test_month(self): # Test for month directives - for directive in ('B', 'b', 'm'): - self.helper(directive, 1) + self.roundtrip('%m', 1) + + @run_with_locales('LC_TIME', 'C', 'en_US', 'fr_FR', 'de_DE', 'ja_JP', 'he_IL', '') + def test_month_locale(self): + # Test for month directives + self.roundtrip('%B', 1) + self.roundtrip('%b', 1) + for m in range(1, 13): + self.roundtrip('%B', 1, (1900, m, 1, 0, 0, 0, 0, 1, 0)) + self.roundtrip('%b', 1, (1900, m, 1, 0, 0, 0, 0, 1, 0)) def test_day(self): # Test for day directives - self.helper('d', 2) + self.roundtrip('%d %Y', 2) def test_hour(self): # Test hour directives - self.helper('H', 3) - strf_output = time.strftime("%I %p", self.time_tuple) - strp_output = _strptime._strptime_time(strf_output, "%I %p") - self.assertTrue(strp_output[3] == self.time_tuple[3], - "testing of '%%I %%p' directive failed; '%s' -> %s != %s" % - (strf_output, strp_output[3], self.time_tuple[3])) + self.roundtrip('%H', 3) + + # NB: Only works on locales with AM/PM + @run_with_locales('LC_TIME', 'C', 'en_US', 'ja_JP') + def test_hour_locale(self): + # Test hour directives + self.roundtrip('%I %p', 3) def test_minute(self): # Test minute directives - self.helper('M', 4) + self.roundtrip('%M', 4) def test_second(self): # Test second directives - self.helper('S', 5) + self.roundtrip('%S', 5) def test_fraction(self): # Test microseconds @@ -347,12 +361,18 @@ def test_fraction(self): def test_weekday(self): # Test weekday directives - for directive in ('A', 'a', 'w', 'u'): - self.helper(directive,6) + self.roundtrip('%w', 6) + self.roundtrip('%u', 6) + + @run_with_locales('LC_TIME', 'C', 'en_US', 'fr_FR', 'de_DE', 'ja_JP', '') + def test_weekday_locale(self): + # Test weekday directives + self.roundtrip('%A', 6) + self.roundtrip('%a', 6) def test_julian(self): # Test julian directives - self.helper('j', 7) + self.roundtrip('%j', 7) def test_offset(self): one_hour = 60 * 60 @@ -449,20 +469,96 @@ def test_bad_timezone(self): "time.daylight set to %s and passing in %s" % (time.tzname, tz_value, time.daylight, tz_name)) - def test_date_time(self): + # NB: Does not roundtrip in some locales due to the ambiguity of + # the date and time representation (bugs in locales?): + # * Seconds are not included: bem_ZM, bokmal, ff_SN, nb_NO, nn_NO, + # no_NO, norwegian, nynorsk. + # * Hours are in 12-hour notation without AM/PM indication: hy_AM, + # id_ID, ms_MY. + # * Year is not included: ha_NG. + # * Use non-Gregorian calendar: lo_LA, thai, th_TH. + # + # BUG: Generates invalid regexp for br_FR, csb_PL, Arabic. + # BUG: Generates regexp that does not match the current date and time + # for fa_IR, gez_ER, gez_ET, lzh_TW, my_MM, or_IN, shn_MM, yo_NG. + # BUG: Generates regexp that does not match the current date and time + # for fa_IR, gez_ER, gez_ET, lzh_TW, my_MM, or_IN, shn_MM, yo_NG, + # fr_FR, ja_JP, he_IL, ko_KR, zh_CN, etc. + @run_with_locales('LC_TIME', 'C', 'en_US', 'de_DE', + 'eu_ES', 'mfe_MU') + def test_date_time_locale(self): # Test %c directive - for position in range(6): - self.helper('c', position) - - def test_date(self): + now = time.time() + self.roundtrip('%c', slice(0, 6), time.localtime(now)) + # 1 hour 20 minutes 30 seconds ago + self.roundtrip('%c', slice(0, 6), time.localtime(now - 4830)) + # 12 hours ago + self.roundtrip('%c', slice(0, 6), time.localtime(now - 12*3600)) + # different days of the week + for i in range(1, 7): + self.roundtrip('%c', slice(0, 6), time.localtime(now - i*24*3600)) + # different months + for i in range(1, 12): + self.roundtrip('%c', slice(0, 6), time.localtime(now - i*30*24*3600)) + # different year + self.roundtrip('%c', slice(0, 6), time.localtime(now - 366*24*3600)) + + # NB: Dates before 1969 do not roundtrip on some locales: + # bo_CN, bo_IN, dz_BT, eu_ES, eu_FR. + @run_with_locales('LC_TIME', 'C', 'en_US', 'de_DE', 'ja_JP') + def test_date_time_locale2(self): + # Test %c directive + self.roundtrip('%c', slice(0, 6), (1900, 1, 1, 0, 0, 0, 0, 1, 0)) + + # NB: Does not roundtrip because use non-Gregorian calendar: + # lo_LA, thai, th_TH. + # BUG: Generates regexp that does not match the current date + # for az_IR, fa_IR, lzh_TW, my_MM, or_IN, shn_MM, + # Arabic, ja_JP, ko_KR, zh_CN, etc. + @run_with_locales('LC_TIME', 'C', 'en_US', 'fr_FR', 'de_DE', + 'he_IL', 'eu_ES') + def test_date_locale(self): # Test %x directive - for position in range(0,3): - self.helper('x', position) - - def test_time(self): + now = time.time() + self.roundtrip('%x', slice(0, 3), time.localtime(now)) + # different days of the week + for i in range(1, 7): + self.roundtrip('%x', slice(0, 3), time.localtime(now - i*24*3600)) + # different months + for i in range(1, 12): + self.roundtrip('%x', slice(0, 3), time.localtime(now - i*30*24*3600)) + # different year + self.roundtrip('%x', slice(0, 3), time.localtime(now - 366*24*3600)) + + # NB: Dates before 1969 do not roundtrip on many locales, including C. + @unittest.skipIf( + support.is_emscripten or support.is_wasi, + "musl libc issue on Emscripten, bpo-46390" + ) + @run_with_locales('LC_TIME', 'en_US', 'fr_FR', 'de_DE', 'ja_JP') + def test_date_locale2(self): + # Test %x directive + self.roundtrip('%x', slice(0, 3), (1900, 1, 1, 0, 0, 0, 0, 1, 0)) + + # NB: Does not roundtrip in some locales due to the ambiguity of + # the time representation (bugs in locales?): + # * Seconds are not included: bokmal, ff_SN, nb_NO, nn_NO, no_NO, + # norwegian, nynorsk. + # * Hours are in 12-hour notation without AM/PM indication: hy_AM, + # ms_MY, sm_WS. + # BUG: Generates regexp that does not match the current time for + # aa_DJ, aa_ER, aa_ET, am_ET, az_IR, byn_ER, fa_IR, gez_ER, gez_ET, + # lzh_TW, my_MM, om_ET, om_KE, or_IN, shn_MM, sid_ET, so_DJ, so_ET, + # so_SO, ti_ER, ti_ET, tig_ER, wal_ET. + @run_with_locales('LC_TIME', 'C', 'en_US', 'fr_FR', 'de_DE', 'ja_JP') + def test_time_locale(self): # Test %X directive - for position in range(3,6): - self.helper('X', position) + now = time.time() + self.roundtrip('%X', slice(3, 6), time.localtime(now)) + # 1 hour 20 minutes 30 seconds ago + self.roundtrip('%X', slice(3, 6), time.localtime(now - 4830)) + # 12 hours ago + self.roundtrip('%X', slice(3, 6), time.localtime(now - 12*3600)) def test_percent(self): # Make sure % signs are handled properly @@ -714,12 +810,7 @@ def test_new_localetime(self): def test_TimeRE_recreation_locale(self): # The TimeRE instance should be recreated upon changing the locale. - locale_info = locale.getlocale(locale.LC_TIME) - try: - locale.setlocale(locale.LC_TIME, ('en_US', 'UTF8')) - except locale.Error: - self.skipTest('test needs en_US.UTF8 locale') - try: + with support.run_with_locale('LC_TIME', 'en_US.UTF8'): _strptime._strptime_time('10 2004', '%d %Y') # Get id of current cache object. first_time_re = _strptime._TimeRE_cache @@ -736,10 +827,6 @@ def test_TimeRE_recreation_locale(self): # to the resetting to the original locale. except locale.Error: self.skipTest('test needs de_DE.UTF8 locale') - # Make sure we don't trample on the locale setting once we leave the - # test. - finally: - locale.setlocale(locale.LC_TIME, locale_info) @support.run_with_tz('STD-1DST,M4.1.0,M10.1.0') def test_TimeRE_recreation_timezone(self): diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index 530c317a852e77..5b5779231f06ce 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -2,7 +2,6 @@ from test.support import warnings_helper import decimal import enum -import locale import math import platform import sys @@ -600,17 +599,8 @@ def test_get_clock_info(self): class TestLocale(unittest.TestCase): - def setUp(self): - self.oldloc = locale.setlocale(locale.LC_ALL) - - def tearDown(self): - locale.setlocale(locale.LC_ALL, self.oldloc) - + @support.run_with_locale('LC_ALL', 'fr_FR', '') def test_bug_3061(self): - try: - tmp = locale.setlocale(locale.LC_ALL, "fr_FR") - except locale.Error: - self.skipTest('could not set locale.LC_ALL to fr_FR') # This should not cause an exception time.strftime("%B", (2009,2,1,0,0,0,0,0,0)) diff --git a/Lib/test/test_types.py b/Lib/test/test_types.py index 3c9e33e3c9dbfc..d1161719d98040 100644 --- a/Lib/test/test_types.py +++ b/Lib/test/test_types.py @@ -398,7 +398,7 @@ def test(i, format_spec, result): test(123456, "1=20", '11111111111111123456') test(123456, "*=20", '**************123456') - @run_with_locale('LC_NUMERIC', 'en_US.UTF8') + @run_with_locale('LC_NUMERIC', 'en_US.UTF8', '') def test_float__format__locale(self): # test locale support for __format__ code 'n' @@ -407,7 +407,7 @@ def test_float__format__locale(self): self.assertEqual(locale.format_string('%g', x, grouping=True), format(x, 'n')) self.assertEqual(locale.format_string('%.10g', x, grouping=True), format(x, '.10n')) - @run_with_locale('LC_NUMERIC', 'en_US.UTF8') + @run_with_locale('LC_NUMERIC', 'en_US.UTF8', '') def test_int__format__locale(self): # test locale support for __format__ code 'n' for integers diff --git a/Misc/NEWS.d/next/C API/2018-06-30-21-48-16.bpo-34008.2Wjtm0.rst b/Misc/NEWS.d/next/C API/2018-06-30-21-48-16.bpo-34008.2Wjtm0.rst new file mode 100644 index 00000000000000..1a01dafc758004 --- /dev/null +++ b/Misc/NEWS.d/next/C API/2018-06-30-21-48-16.bpo-34008.2Wjtm0.rst @@ -0,0 +1,2 @@ +Added ``Py_IsInitialized`` to the list of APIs that are safe to call before +the interpreter is initialized, and updated the embedding tests to cover it. diff --git a/Misc/NEWS.d/next/C_API/2024-09-25-11-44-02.gh-issue-124502.qWuDjT.rst b/Misc/NEWS.d/next/C_API/2024-09-25-11-44-02.gh-issue-124502.qWuDjT.rst new file mode 100644 index 00000000000000..f515619328b359 --- /dev/null +++ b/Misc/NEWS.d/next/C_API/2024-09-25-11-44-02.gh-issue-124502.qWuDjT.rst @@ -0,0 +1,2 @@ +Add :c:func:`PyUnicode_Equal` function to the limited C API: test if two +strings are equal. Patch by Victor Stinner. diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2024-10-03-14-39-41.gh-issue-123378.dCxANf.rst b/Misc/NEWS.d/next/Core_and_Builtins/2024-10-03-14-39-41.gh-issue-123378.dCxANf.rst new file mode 100644 index 00000000000000..5cd34535d674d3 --- /dev/null +++ b/Misc/NEWS.d/next/Core_and_Builtins/2024-10-03-14-39-41.gh-issue-123378.dCxANf.rst @@ -0,0 +1,3 @@ +Fix a crash in the :meth:`~object.__str__` method of :exc:`UnicodeError` +objects when the :attr:`UnicodeError.start` and :attr:`UnicodeError.end` +values are invalid or out-of-range. Patch by Bénédikt Tran. diff --git a/Misc/NEWS.d/next/Documentation/2018-07-04-20-35-25.bpo-34008.bqecIb.rst b/Misc/NEWS.d/next/Documentation/2018-07-04-20-35-25.bpo-34008.bqecIb.rst new file mode 100644 index 00000000000000..a89086af35bfc1 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-07-04-20-35-25.bpo-34008.bqecIb.rst @@ -0,0 +1,8 @@ +The :c:func:`Py_Main` documentation moved from the "Very High Level API" section to the +"Initialization and Finalization" section. + +Also make it explicit that we expect ``Py_Main`` to typically be called instead +of ``Py_Initialize`` rather than after it (since ``Py_Main`` makes its own +call to ``Py_Initialize``). Document that calling both is +supported but is version dependent on which settings +will be applied correctly. diff --git a/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst b/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst new file mode 100644 index 00000000000000..ba607bf7a42cd9 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst @@ -0,0 +1,3 @@ +Clarified the dual usage of the term "free variable" (both the formal +meaning of any reference to names defined outside the local scope, and the +narrower pragmatic meaning of nonlocal variables named in ``co_freevars``). diff --git a/Misc/NEWS.d/next/IDLE/2024-10-04-15-34-34.gh-issue-122392.V8K3w2.rst b/Misc/NEWS.d/next/IDLE/2024-10-04-15-34-34.gh-issue-122392.V8K3w2.rst new file mode 100644 index 00000000000000..541f6212794ef2 --- /dev/null +++ b/Misc/NEWS.d/next/IDLE/2024-10-04-15-34-34.gh-issue-122392.V8K3w2.rst @@ -0,0 +1,2 @@ +Increase currently inadequate vertical spacing for the IDLE browsers (path, +module, and stack) on high-resolution monitors. diff --git a/Misc/NEWS.d/next/Library/2024-02-27-10-22-15.gh-issue-115937.0cVNur.rst b/Misc/NEWS.d/next/Library/2024-02-27-10-22-15.gh-issue-115937.0cVNur.rst new file mode 100644 index 00000000000000..f9dae0c9b8e2b2 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-02-27-10-22-15.gh-issue-115937.0cVNur.rst @@ -0,0 +1,3 @@ +Removed extra preprocessing for the ``__signature__`` attribute: the code +just check if it's a :class:`inspect.Signature` instance. Patch by Sergey B +Kirpichev. diff --git a/Misc/NEWS.d/next/Library/2024-10-02-15-05-45.gh-issue-124653.tqsTu9.rst b/Misc/NEWS.d/next/Library/2024-10-02-15-05-45.gh-issue-124653.tqsTu9.rst new file mode 100644 index 00000000000000..6f5ad12d2c2981 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-02-15-05-45.gh-issue-124653.tqsTu9.rst @@ -0,0 +1,2 @@ +Fix detection of the minimal Queue API needed by the :mod:`logging` module. +Patch by Bénédikt Tran. diff --git a/Misc/NEWS.d/next/Library/2024-10-02-22-53-48.gh-issue-90102.4qX52R.rst b/Misc/NEWS.d/next/Library/2024-10-02-22-53-48.gh-issue-90102.4qX52R.rst new file mode 100644 index 00000000000000..0e708ed11d21b5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-02-22-53-48.gh-issue-90102.4qX52R.rst @@ -0,0 +1,3 @@ +Skip the ``isatty`` system call during open() when the file is known to not +be a character device. This provides a slight performance improvement when +reading whole files. diff --git a/Misc/NEWS.d/next/Library/2024-10-03-19-16-38.gh-issue-123961.ik1Dgs.rst b/Misc/NEWS.d/next/Library/2024-10-03-19-16-38.gh-issue-123961.ik1Dgs.rst new file mode 100644 index 00000000000000..b637b895d0b803 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-03-19-16-38.gh-issue-123961.ik1Dgs.rst @@ -0,0 +1,2 @@ +Convert :mod:`curses` to multi-phase initialization (:pep:`489`), thereby +fixing reference leaks at interpreter shutdown. Patch by Bénédikt Tran. diff --git a/Misc/NEWS.d/next/Library/2024-10-04-12-43-03.gh-issue-69998.DVqOXX.rst b/Misc/NEWS.d/next/Library/2024-10-04-12-43-03.gh-issue-69998.DVqOXX.rst new file mode 100644 index 00000000000000..65388e0b4e7ee4 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-04-12-43-03.gh-issue-69998.DVqOXX.rst @@ -0,0 +1,3 @@ +Fix :func:`locale.nl_langinfo` in case when different categories have +different locales. The function now sets temporarily the ``LC_CTYPE`` locale +in some cases. This temporary change affects other threads. diff --git a/Misc/NEWS.d/next/Library/2024-10-08-13-28-22.gh-issue-125096.Vz0W5g.rst b/Misc/NEWS.d/next/Library/2024-10-08-13-28-22.gh-issue-125096.Vz0W5g.rst new file mode 100644 index 00000000000000..c582a2dfe7243c --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-10-08-13-28-22.gh-issue-125096.Vz0W5g.rst @@ -0,0 +1,5 @@ +If the :envvar:`PYTHON_BASIC_REPL` environment variable is set, the +:mod:`site` module no longer imports the :mod:`!_pyrepl` module. Moreover, +the :mod:`site` module now respects :option:`-E` and :option:`-I` command +line options: ignore :envvar:`PYTHON_BASIC_REPL` in this case. Patch by +Victor Stinner. diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml index fe0a5e44f8fb15..62978261745d79 100644 --- a/Misc/stable_abi.toml +++ b/Misc/stable_abi.toml @@ -2536,3 +2536,5 @@ added = '3.14' [const.Py_TP_USE_SPEC] added = '3.14' +[function.PyUnicode_Equal] + added = '3.14' diff --git a/Modules/_csv.c b/Modules/_csv.c index a623ea449da779..913560ce4a0ee3 100644 --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -367,6 +367,8 @@ static struct PyMemberDef Dialect_memberlist[] = { { NULL } }; +#undef D_OFF + static PyGetSetDef Dialect_getsetlist[] = { { "delimiter", (getter)Dialect_get_delimiter}, { "escapechar", (getter)Dialect_get_escapechar}, @@ -502,6 +504,7 @@ dialect_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) DIALECT_GETATTR(skipinitialspace, "skipinitialspace"); DIALECT_GETATTR(strict, "strict"); } +#undef DIALECT_GETATTR /* check types and convert to C values */ #define DIASET(meth, name, target, src, dflt) \ @@ -515,6 +518,7 @@ dialect_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) DIASET(_set_int, "quoting", &self->quoting, quoting, QUOTE_MINIMAL); DIASET(_set_bool, "skipinitialspace", &self->skipinitialspace, skipinitialspace, false); DIASET(_set_bool, "strict", &self->strict, strict, false); +#undef DIASET /* validate options */ if (dialect_check_quoting(self->quoting)) @@ -1026,6 +1030,8 @@ static struct PyMemberDef Reader_memberlist[] = { { NULL } }; +#undef R_OFF + static PyType_Slot Reader_Type_slots[] = { {Py_tp_doc, (char*)Reader_Type_doc}, @@ -1441,6 +1447,8 @@ static struct PyMemberDef Writer_memberlist[] = { { NULL } }; +#undef W_OFF + static int Writer_traverse(WriterObj *self, visitproc visit, void *arg) { diff --git a/Modules/_cursesmodule.c b/Modules/_cursesmodule.c index 61b65675375547..27d5df08de933e 100644 --- a/Modules/_cursesmodule.c +++ b/Modules/_cursesmodule.c @@ -160,30 +160,31 @@ typedef chtype attr_t; /* No attr_t type is available */ #define _CURSES_PAIR_CONTENT_FUNC pair_content #endif /* _NCURSES_EXTENDED_COLOR_FUNCS */ -typedef struct _cursesmodule_state { - PyObject *error; // PyCursesError - PyTypeObject *window_type; // PyCursesWindow_Type -} _cursesmodule_state; +typedef struct { + PyObject *error; // curses exception type + PyTypeObject *window_type; // exposed by PyCursesWindow_Type +} cursesmodule_state; -// For now, we keep a global state variable to prepare for PEP 489. -static _cursesmodule_state curses_global_state; - -static inline _cursesmodule_state * -get_cursesmodule_state(PyObject *Py_UNUSED(module)) +static inline cursesmodule_state * +get_cursesmodule_state(PyObject *module) { - return &curses_global_state; + void *state = PyModule_GetState(module); + assert(state != NULL); + return (cursesmodule_state *)state; } -static inline _cursesmodule_state * -get_cursesmodule_state_by_cls(PyTypeObject *Py_UNUSED(cls)) +static inline cursesmodule_state * +get_cursesmodule_state_by_cls(PyTypeObject *cls) { - return &curses_global_state; + void *state = PyType_GetModuleState(cls); + assert(state != NULL); + return (cursesmodule_state *)state; } -static inline _cursesmodule_state * -get_cursesmodule_state_by_win(PyCursesWindowObject *Py_UNUSED(win)) +static inline cursesmodule_state * +get_cursesmodule_state_by_win(PyCursesWindowObject *win) { - return &curses_global_state; + return get_cursesmodule_state_by_cls(Py_TYPE(win)); } /*[clinic input] @@ -192,6 +193,9 @@ class _curses.window "PyCursesWindowObject *" "clinic_state()->window_type" [clinic start generated code]*/ /*[clinic end generated code: output=da39a3ee5e6b4b0d input=ae6cb623018f2cbc]*/ +/* Indicate whether the module has already been loaded or not. */ +static int curses_module_loaded = 0; + /* Tells whether setupterm() has been called to initialise terminfo. */ static int curses_setupterm_called = FALSE; @@ -211,8 +215,8 @@ static const char *curses_screen_encoding = NULL; * set and this returns 0. Otherwise, this returns 1. * * Since this function can be called in functions that do not - * have a direct access to the module's state, the exception - * type is directly taken from the global state for now. + * have a direct access to the module's state, '_curses.error' + * is imported on demand. */ static inline int _PyCursesCheckFunction(int called, const char *funcname) @@ -220,7 +224,12 @@ _PyCursesCheckFunction(int called, const char *funcname) if (called == TRUE) { return 1; } - PyErr_Format(curses_global_state.error, "must call %s() first", funcname); + PyObject *exc = _PyImport_GetModuleAttrString("_curses", "error"); + if (exc != NULL) { + PyErr_Format(exc, "must call %s() first", funcname); + Py_DECREF(exc); + } + assert(PyErr_Occurred()); return 0; } @@ -237,7 +246,7 @@ _PyCursesStatefulCheckFunction(PyObject *module, int called, const char *funcnam if (called == TRUE) { return 1; } - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_Format(state->error, "must call %s() first", funcname); return 0; } @@ -275,7 +284,7 @@ _PyCursesStatefulCheckFunction(PyObject *module, int called, const char *funcnam /* Utility Functions */ static inline void -_PyCursesSetError(_cursesmodule_state *state, const char *funcname) +_PyCursesSetError(cursesmodule_state *state, const char *funcname) { if (funcname == NULL) { PyErr_SetString(state->error, catchall_ERR); @@ -296,7 +305,7 @@ PyCursesCheckERR(PyObject *module, int code, const char *fname) if (code != ERR) { Py_RETURN_NONE; } else { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); _PyCursesSetError(state, fname); return NULL; } @@ -308,7 +317,7 @@ PyCursesCheckERR_ForWin(PyCursesWindowObject *win, int code, const char *fname) if (code != ERR) { Py_RETURN_NONE; } else { - _cursesmodule_state *state = get_cursesmodule_state_by_win(win); + cursesmodule_state *state = get_cursesmodule_state_by_win(win); _PyCursesSetError(state, fname); return NULL; } @@ -746,7 +755,7 @@ Window_TwoArgNoReturnFunction(wresize, int, "ii;lines,columns") /* Allocation and deallocation of Window Objects */ static PyObject * -PyCursesWindow_New(_cursesmodule_state *state, +PyCursesWindow_New(cursesmodule_state *state, WINDOW *win, const char *encoding) { if (encoding == NULL) { @@ -1405,12 +1414,12 @@ _curses_window_derwin_impl(PyCursesWindowObject *self, int group_left_1, win = derwin(self->win,nlines,ncols,begin_y,begin_x); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, catchall_NULL); return NULL; } - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); return PyCursesWindow_New(state, win, NULL); } @@ -1559,7 +1568,7 @@ _curses_window_getkey_impl(PyCursesWindowObject *self, int group_right_1, /* getch() returns ERR in nodelay mode */ PyErr_CheckSignals(); if (!PyErr_Occurred()) { - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, "no input"); } return NULL; @@ -1619,7 +1628,7 @@ _curses_window_get_wch_impl(PyCursesWindowObject *self, int group_right_1, return NULL; /* get_wch() returns ERR in nodelay mode */ - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, "no input"); return NULL; } @@ -2133,7 +2142,7 @@ _curses_window_noutrefresh_impl(PyCursesWindowObject *self) #ifdef py_is_pad if (py_is_pad(self->win)) { if (!group_right_1) { - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, "noutrefresh() called for a pad " "requires 6 arguments"); @@ -2358,7 +2367,7 @@ _curses_window_refresh_impl(PyCursesWindowObject *self, int group_right_1, #ifdef py_is_pad if (py_is_pad(self->win)) { if (!group_right_1) { - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, "refresh() for a pad requires 6 arguments"); return NULL; @@ -2441,12 +2450,12 @@ _curses_window_subwin_impl(PyCursesWindowObject *self, int group_left_1, win = subwin(self->win, nlines, ncols, begin_y, begin_x); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); PyErr_SetString(state->error, catchall_NULL); return NULL; } - _cursesmodule_state *state = get_cursesmodule_state_by_win(self); + cursesmodule_state *state = get_cursesmodule_state_by_win(self); return PyCursesWindow_New(state, win, self->encoding); } @@ -2846,7 +2855,7 @@ _curses_color_content_impl(PyObject *module, int color_number) PyCursesStatefulInitialisedColor(module); if (_COLOR_CONTENT_FUNC(color_number, &r, &g, &b) == ERR) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_Format(state->error, "%s() returned ERR", Py_STRINGIFY(_COLOR_CONTENT_FUNC)); return NULL; @@ -3086,7 +3095,7 @@ _curses_getmouse_impl(PyObject *module) rtn = getmouse( &event ); if (rtn == ERR) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, "getmouse() returned ERR"); return NULL; } @@ -3181,11 +3190,11 @@ _curses_getwin(PyObject *module, PyObject *file) fseek(fp, 0, 0); win = getwin(fp); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, catchall_NULL); goto error; } - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); res = PyCursesWindow_New(state, win, NULL); error: @@ -3332,7 +3341,7 @@ _curses_init_pair_impl(PyObject *module, int pair_number, int fg, int bg) COLOR_PAIRS - 1); } else { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_Format(state->error, "%s() returned ERR", Py_STRINGIFY(_CURSES_INIT_PAIR_FUNC)); } @@ -3358,14 +3367,14 @@ _curses_initscr_impl(PyObject *module) if (curses_initscr_called) { wrefresh(stdscr); - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); return PyCursesWindow_New(state, stdscr, NULL); } win = initscr(); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, catchall_NULL); return NULL; } @@ -3462,7 +3471,7 @@ _curses_initscr_impl(PyObject *module) SetDictInt("COLS", COLS); #undef SetDictInt - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyObject *winobj = PyCursesWindow_New(state, win, NULL); if (winobj == NULL) { return NULL; @@ -3496,7 +3505,7 @@ _curses_setupterm_impl(PyObject *module, const char *term, int fd) sys_stdout = PySys_GetObject("stdout"); if (sys_stdout == NULL || sys_stdout == Py_None) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, "lost sys.stdout"); return NULL; } @@ -3517,7 +3526,7 @@ _curses_setupterm_impl(PyObject *module, const char *term, int fd) s = "setupterm: could not find terminfo database"; } - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, s); return NULL; } @@ -3835,12 +3844,12 @@ _curses_newpad_impl(PyObject *module, int nlines, int ncols) win = newpad(nlines, ncols); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, catchall_NULL); return NULL; } - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); return PyCursesWindow_New(state, win, NULL); } @@ -3876,12 +3885,12 @@ _curses_newwin_impl(PyObject *module, int nlines, int ncols, win = newwin(nlines,ncols,begin_y,begin_x); if (win == NULL) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, catchall_NULL); return NULL; } - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); return PyCursesWindow_New(state, win, NULL); } @@ -3996,7 +4005,7 @@ _curses_pair_content_impl(PyObject *module, int pair_number) COLOR_PAIRS - 1); } else { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_Format(state->error, "%s() returned ERR", Py_STRINGIFY(_CURSES_PAIR_CONTENT_FUNC)); } @@ -4327,7 +4336,7 @@ _curses_start_color_impl(PyObject *module) PyCursesStatefulInitialised(module); if (start_color() == ERR) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, "start_color() returned ERR"); return NULL; } @@ -4480,7 +4489,7 @@ _curses_tparm_impl(PyObject *module, const char *str, int i1, int i2, int i3, result = tparm((char *)str,i1,i2,i3,i4,i5,i6,i7,i8,i9); if (!result) { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, "tparm() returned NULL"); return NULL; } @@ -4682,7 +4691,7 @@ _curses_use_default_colors_impl(PyObject *module) if (code != ERR) { Py_RETURN_NONE; } else { - _cursesmodule_state *state = get_cursesmodule_state(module); + cursesmodule_state *state = get_cursesmodule_state(module); PyErr_SetString(state->error, "use_default_colors() returned ERR"); return NULL; } @@ -4763,7 +4772,7 @@ _curses_has_extended_color_support_impl(PyObject *module) /* List of functions defined in the module */ -static PyMethodDef PyCurses_methods[] = { +static PyMethodDef cursesmodule_methods[] = { _CURSES_BAUDRATE_METHODDEF _CURSES_BEEP_METHODDEF _CURSES_CAN_CHANGE_COLOR_METHODDEF @@ -4872,7 +4881,7 @@ curses_capi_start_color_called(void) } static void * -curses_capi_new(_cursesmodule_state *state) +curses_capi_new(cursesmodule_state *state) { assert(state->window_type != NULL); void **capi = (void **)PyMem_Calloc(PyCurses_API_pointers, sizeof(void *)); @@ -4892,8 +4901,9 @@ curses_capi_free(void *capi) { assert(capi != NULL); void **capi_ptr = (void **)capi; - assert(capi_ptr[0] != NULL); - Py_DECREF(capi_ptr[0]); // decref curses window type + // In free-threaded builds, capi_ptr[0] may have been already cleared + // by curses_capi_capsule_destructor(), hence the use of Py_XDECREF(). + Py_XDECREF(capi_ptr[0]); // decref curses window type PyMem_Free(capi_ptr); } @@ -4942,12 +4952,44 @@ curses_capi_capsule_new(void *capi) return capsule; } -/* Module initialization */ +/* Module initialization and cleanup functions */ + +static int +cursesmodule_traverse(PyObject *mod, visitproc visit, void *arg) +{ + cursesmodule_state *state = get_cursesmodule_state(mod); + Py_VISIT(state->error); + Py_VISIT(state->window_type); + return 0; +} + +static int +cursesmodule_clear(PyObject *mod) +{ + cursesmodule_state *state = get_cursesmodule_state(mod); + Py_CLEAR(state->error); + Py_CLEAR(state->window_type); + return 0; +} + +static void +cursesmodule_free(void *mod) +{ + (void)cursesmodule_clear((PyObject *)mod); + curses_module_loaded = 0; // allow reloading once garbage-collected +} static int cursesmodule_exec(PyObject *module) { - _cursesmodule_state *state = get_cursesmodule_state(module); + if (curses_module_loaded) { + PyErr_SetString(PyExc_ImportError, + "module 'curses' can only be loaded once per process"); + return -1; + } + curses_module_loaded = 1; + + cursesmodule_state *state = get_cursesmodule_state(module); /* Initialize object type */ state->window_type = (PyTypeObject *)PyType_FromModuleAndSpec( module, &PyCursesWindow_Type_spec, NULL); @@ -4987,7 +5029,6 @@ cursesmodule_exec(PyObject *module) return -1; } rc = PyDict_SetItemString(module_dict, "error", state->error); - Py_DECREF(state->error); if (rc < 0) { return -1; } @@ -5181,33 +5222,26 @@ cursesmodule_exec(PyObject *module) /* Initialization function for the module */ -static struct PyModuleDef _cursesmodule = { +static PyModuleDef_Slot cursesmodule_slots[] = { + {Py_mod_exec, cursesmodule_exec}, + {Py_mod_multiple_interpreters, Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED}, + {Py_mod_gil, Py_MOD_GIL_NOT_USED}, + {0, NULL} +}; + +static struct PyModuleDef cursesmodule = { PyModuleDef_HEAD_INIT, .m_name = "_curses", - .m_size = -1, - .m_methods = PyCurses_methods, + .m_size = sizeof(cursesmodule_state), + .m_methods = cursesmodule_methods, + .m_slots = cursesmodule_slots, + .m_traverse = cursesmodule_traverse, + .m_clear = cursesmodule_clear, + .m_free = cursesmodule_free }; PyMODINIT_FUNC PyInit__curses(void) { - // create the module - PyObject *mod = PyModule_Create(&_cursesmodule); - if (mod == NULL) { - goto error; - } -#ifdef Py_GIL_DISABLED - if (PyUnstable_Module_SetGIL(mod, Py_MOD_GIL_NOT_USED) < 0) { - goto error; - } -#endif - // populate the module - if (cursesmodule_exec(mod) < 0) { - goto error; - } - return mod; - -error: - Py_XDECREF(mod); - return NULL; + return PyModuleDef_Init(&cursesmodule); } diff --git a/Modules/_io/_iomodule.c b/Modules/_io/_iomodule.c index 1238e6074246d0..6622f2cabb908b 100644 --- a/Modules/_io/_iomodule.c +++ b/Modules/_io/_iomodule.c @@ -346,7 +346,7 @@ _io_open_impl(PyObject *module, PyObject *file, const char *mode, /* buffering */ if (buffering < 0) { - PyObject *res = PyObject_CallMethodNoArgs(raw, &_Py_ID(isatty)); + PyObject *res = PyObject_CallMethodNoArgs(raw, &_Py_ID(_isatty_open_only)); if (res == NULL) goto error; isatty = PyObject_IsTrue(res); diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index d9597f860b9429..f374592eb95967 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -12,9 +12,6 @@ #ifdef HAVE_SYS_TYPES_H # include #endif -#ifdef HAVE_SYS_STAT_H -# include -#endif #ifdef HAVE_IO_H # include #endif @@ -1218,6 +1215,24 @@ _io_FileIO_isatty_impl(fileio *self) return PyBool_FromLong(res); } +/* Checks whether the file is a TTY using an open-only optimization. + + TTYs are always character devices. If the interpreter knows a file is + not a character device when it would call ``isatty``, can skip that + call. Inside ``open()`` there is a fresh stat result that contains that + information. Use the stat result to skip a system call. Outside of that + context TOCTOU issues (the fd could be arbitrarily modified by + surrounding code). */ +static PyObject * +_io_FileIO_isatty_open_only(PyObject *op, PyObject *Py_UNUSED(ignored)) +{ + fileio *self = _PyFileIO_CAST(op); + if (self->stat_atopen != NULL && !S_ISCHR(self->stat_atopen->st_mode)) { + Py_RETURN_FALSE; + } + return _io_FileIO_isatty_impl(self); +} + #include "clinic/fileio.c.h" static PyMethodDef fileio_methods[] = { @@ -1234,6 +1249,7 @@ static PyMethodDef fileio_methods[] = { _IO_FILEIO_WRITABLE_METHODDEF _IO_FILEIO_FILENO_METHODDEF _IO_FILEIO_ISATTY_METHODDEF + {"_isatty_open_only", _io_FileIO_isatty_open_only, METH_NOARGS}, {"_dealloc_warn", fileio_dealloc_warn, METH_O, NULL}, {"__reduce__", _PyIOBase_cannot_pickle, METH_NOARGS}, {"__reduce_ex__", _PyIOBase_cannot_pickle, METH_O}, diff --git a/Modules/_io/winconsoleio.c b/Modules/_io/winconsoleio.c index ec5c298066a587..d7cb5abfdc0abd 100644 --- a/Modules/_io/winconsoleio.c +++ b/Modules/_io/winconsoleio.c @@ -1128,6 +1128,7 @@ static PyMethodDef winconsoleio_methods[] = { _IO__WINDOWSCONSOLEIO_WRITABLE_METHODDEF _IO__WINDOWSCONSOLEIO_FILENO_METHODDEF _IO__WINDOWSCONSOLEIO_ISATTY_METHODDEF + {"_isatty_open_only", (PyCFunction)_io__WindowsConsoleIO_isatty, METH_NOARGS}, {NULL, NULL} /* sentinel */ }; diff --git a/Modules/_localemodule.c b/Modules/_localemodule.c index 9452df492bb23b..ce77c4072a6aea 100644 --- a/Modules/_localemodule.c +++ b/Modules/_localemodule.c @@ -144,6 +144,17 @@ locale_is_ascii(const char *str) return (strlen(str) == 1 && ((unsigned char)str[0]) <= 127); } +static int +is_all_ascii(const char *str) +{ + for (; *str; str++) { + if ((unsigned char)*str > 127) { + return 0; + } + } + return 1; +} + static int locale_decode_monetary(PyObject *dict, struct lconv *lc) { @@ -478,113 +489,153 @@ _locale__getdefaultlocale_impl(PyObject *module) #endif #ifdef HAVE_LANGINFO_H -#define LANGINFO(X) {#X, X} +#define LANGINFO(X, Y) {#X, X, Y} static struct langinfo_constant{ - char* name; + const char *name; int value; + int category; } langinfo_constants[] = { /* These constants should exist on any langinfo implementation */ - LANGINFO(DAY_1), - LANGINFO(DAY_2), - LANGINFO(DAY_3), - LANGINFO(DAY_4), - LANGINFO(DAY_5), - LANGINFO(DAY_6), - LANGINFO(DAY_7), - - LANGINFO(ABDAY_1), - LANGINFO(ABDAY_2), - LANGINFO(ABDAY_3), - LANGINFO(ABDAY_4), - LANGINFO(ABDAY_5), - LANGINFO(ABDAY_6), - LANGINFO(ABDAY_7), - - LANGINFO(MON_1), - LANGINFO(MON_2), - LANGINFO(MON_3), - LANGINFO(MON_4), - LANGINFO(MON_5), - LANGINFO(MON_6), - LANGINFO(MON_7), - LANGINFO(MON_8), - LANGINFO(MON_9), - LANGINFO(MON_10), - LANGINFO(MON_11), - LANGINFO(MON_12), - - LANGINFO(ABMON_1), - LANGINFO(ABMON_2), - LANGINFO(ABMON_3), - LANGINFO(ABMON_4), - LANGINFO(ABMON_5), - LANGINFO(ABMON_6), - LANGINFO(ABMON_7), - LANGINFO(ABMON_8), - LANGINFO(ABMON_9), - LANGINFO(ABMON_10), - LANGINFO(ABMON_11), - LANGINFO(ABMON_12), + LANGINFO(DAY_1, LC_TIME), + LANGINFO(DAY_2, LC_TIME), + LANGINFO(DAY_3, LC_TIME), + LANGINFO(DAY_4, LC_TIME), + LANGINFO(DAY_5, LC_TIME), + LANGINFO(DAY_6, LC_TIME), + LANGINFO(DAY_7, LC_TIME), + + LANGINFO(ABDAY_1, LC_TIME), + LANGINFO(ABDAY_2, LC_TIME), + LANGINFO(ABDAY_3, LC_TIME), + LANGINFO(ABDAY_4, LC_TIME), + LANGINFO(ABDAY_5, LC_TIME), + LANGINFO(ABDAY_6, LC_TIME), + LANGINFO(ABDAY_7, LC_TIME), + + LANGINFO(MON_1, LC_TIME), + LANGINFO(MON_2, LC_TIME), + LANGINFO(MON_3, LC_TIME), + LANGINFO(MON_4, LC_TIME), + LANGINFO(MON_5, LC_TIME), + LANGINFO(MON_6, LC_TIME), + LANGINFO(MON_7, LC_TIME), + LANGINFO(MON_8, LC_TIME), + LANGINFO(MON_9, LC_TIME), + LANGINFO(MON_10, LC_TIME), + LANGINFO(MON_11, LC_TIME), + LANGINFO(MON_12, LC_TIME), + + LANGINFO(ABMON_1, LC_TIME), + LANGINFO(ABMON_2, LC_TIME), + LANGINFO(ABMON_3, LC_TIME), + LANGINFO(ABMON_4, LC_TIME), + LANGINFO(ABMON_5, LC_TIME), + LANGINFO(ABMON_6, LC_TIME), + LANGINFO(ABMON_7, LC_TIME), + LANGINFO(ABMON_8, LC_TIME), + LANGINFO(ABMON_9, LC_TIME), + LANGINFO(ABMON_10, LC_TIME), + LANGINFO(ABMON_11, LC_TIME), + LANGINFO(ABMON_12, LC_TIME), #ifdef RADIXCHAR /* The following are not available with glibc 2.0 */ - LANGINFO(RADIXCHAR), - LANGINFO(THOUSEP), + LANGINFO(RADIXCHAR, LC_NUMERIC), + LANGINFO(THOUSEP, LC_NUMERIC), /* YESSTR and NOSTR are deprecated in glibc, since they are a special case of message translation, which should be rather done using gettext. So we don't expose it to Python in the first place. - LANGINFO(YESSTR), - LANGINFO(NOSTR), + LANGINFO(YESSTR, LC_MESSAGES), + LANGINFO(NOSTR, LC_MESSAGES), */ - LANGINFO(CRNCYSTR), + LANGINFO(CRNCYSTR, LC_MONETARY), #endif - LANGINFO(D_T_FMT), - LANGINFO(D_FMT), - LANGINFO(T_FMT), - LANGINFO(AM_STR), - LANGINFO(PM_STR), + LANGINFO(D_T_FMT, LC_TIME), + LANGINFO(D_FMT, LC_TIME), + LANGINFO(T_FMT, LC_TIME), + LANGINFO(AM_STR, LC_TIME), + LANGINFO(PM_STR, LC_TIME), /* The following constants are available only with XPG4, but... OpenBSD doesn't have CODESET but has T_FMT_AMPM, and doesn't have a few of the others. Solution: ifdef-test them all. */ #ifdef CODESET - LANGINFO(CODESET), + LANGINFO(CODESET, LC_CTYPE), #endif #ifdef T_FMT_AMPM - LANGINFO(T_FMT_AMPM), + LANGINFO(T_FMT_AMPM, LC_TIME), #endif #ifdef ERA - LANGINFO(ERA), + LANGINFO(ERA, LC_TIME), #endif #ifdef ERA_D_FMT - LANGINFO(ERA_D_FMT), + LANGINFO(ERA_D_FMT, LC_TIME), #endif #ifdef ERA_D_T_FMT - LANGINFO(ERA_D_T_FMT), + LANGINFO(ERA_D_T_FMT, LC_TIME), #endif #ifdef ERA_T_FMT - LANGINFO(ERA_T_FMT), + LANGINFO(ERA_T_FMT, LC_TIME), #endif #ifdef ALT_DIGITS - LANGINFO(ALT_DIGITS), + LANGINFO(ALT_DIGITS, LC_TIME), #endif #ifdef YESEXPR - LANGINFO(YESEXPR), + LANGINFO(YESEXPR, LC_MESSAGES), #endif #ifdef NOEXPR - LANGINFO(NOEXPR), + LANGINFO(NOEXPR, LC_MESSAGES), #endif #ifdef _DATE_FMT /* This is not available in all glibc versions that have CODESET. */ - LANGINFO(_DATE_FMT), + LANGINFO(_DATE_FMT, LC_TIME), #endif - {0, 0} + {0, 0, 0} }; +/* Temporary make the LC_CTYPE locale to be the same as + * the locale of the specified category. */ +static int +change_locale(int category, char **oldloc) +{ + /* Keep a copy of the LC_CTYPE locale */ + *oldloc = setlocale(LC_CTYPE, NULL); + if (!*oldloc) { + PyErr_SetString(PyExc_RuntimeError, "faild to get LC_CTYPE locale"); + return -1; + } + *oldloc = _PyMem_Strdup(*oldloc); + if (!*oldloc) { + PyErr_NoMemory(); + return -1; + } + + /* Set a new locale if it is different. */ + char *loc = setlocale(category, NULL); + if (loc == NULL || strcmp(loc, *oldloc) == 0) { + PyMem_Free(*oldloc); + *oldloc = NULL; + return 0; + } + + setlocale(LC_CTYPE, loc); + return 1; +} + +/* Restore the old LC_CTYPE locale. */ +static void +restore_locale(char *oldloc) +{ + if (oldloc != NULL) { + setlocale(LC_CTYPE, oldloc); + PyMem_Free(oldloc); + } +} + /*[clinic input] _locale.nl_langinfo @@ -602,14 +653,24 @@ _locale_nl_langinfo_impl(PyObject *module, int item) /* Check whether this is a supported constant. GNU libc sometimes returns numeric values in the char* return value, which would crash PyUnicode_FromString. */ - for (i = 0; langinfo_constants[i].name; i++) + for (i = 0; langinfo_constants[i].name; i++) { if (langinfo_constants[i].value == item) { /* Check NULL as a workaround for GNU libc's returning NULL instead of an empty string for nl_langinfo(ERA). */ const char *result = nl_langinfo(item); result = result != NULL ? result : ""; - return PyUnicode_DecodeLocale(result, NULL); + char *oldloc = NULL; + if (langinfo_constants[i].category != LC_CTYPE + && !is_all_ascii(result) + && change_locale(langinfo_constants[i].category, &oldloc) < 0) + { + return NULL; + } + PyObject *unicode = PyUnicode_DecodeLocale(result, NULL); + restore_locale(oldloc); + return unicode; } + } PyErr_SetString(PyExc_ValueError, "unsupported langinfo constant"); return NULL; } diff --git a/Modules/_testlimitedcapi/unicode.c b/Modules/_testlimitedcapi/unicode.c index 2b70d09108a333..c7a23d5d1cbd71 100644 --- a/Modules/_testlimitedcapi/unicode.c +++ b/Modules/_testlimitedcapi/unicode.c @@ -1,7 +1,7 @@ #include "pyconfig.h" // Py_GIL_DISABLED #ifndef Py_GIL_DISABLED - // Need limited C API 3.13 to test PyUnicode_EqualToUTF8() -# define Py_LIMITED_API 0x030d0000 + // Need limited C API 3.14 to test PyUnicode_Equal() +# define Py_LIMITED_API 0x030e0000 #endif #include "parts.h" @@ -1837,6 +1837,23 @@ test_string_from_format(PyObject *self, PyObject *Py_UNUSED(ignored)) #undef CHECK_FORMAT_0 } + +/* Test PyUnicode_Equal() */ +static PyObject * +unicode_equal(PyObject *module, PyObject *args) +{ + PyObject *str1, *str2; + if (!PyArg_ParseTuple(args, "OO", &str1, &str2)) { + return NULL; + } + + NULLABLE(str1); + NULLABLE(str2); + RETURN_INT(PyUnicode_Equal(str1, str2)); +} + + + static PyMethodDef TestMethods[] = { {"codec_incrementalencoder", codec_incrementalencoder, METH_VARARGS}, {"codec_incrementaldecoder", codec_incrementaldecoder, METH_VARARGS}, @@ -1924,6 +1941,7 @@ static PyMethodDef TestMethods[] = { {"unicode_format", unicode_format, METH_VARARGS}, {"unicode_contains", unicode_contains, METH_VARARGS}, {"unicode_isidentifier", unicode_isidentifier, METH_O}, + {"unicode_equal", unicode_equal, METH_VARARGS}, {NULL}, }; diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index 153fe85597749d..9617f9cafe76ff 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -705,9 +705,7 @@ static PyType_Spec ThreadHandle_Type_spec = { typedef struct { PyObject_HEAD - PyThread_type_lock lock_lock; - PyObject *in_weakreflist; - char locked; /* for sanity checking */ + PyMutex lock; } lockobject; static int @@ -722,15 +720,7 @@ lock_dealloc(PyObject *op) { lockobject *self = (lockobject*)op; PyObject_GC_UnTrack(self); - if (self->in_weakreflist != NULL) { - PyObject_ClearWeakRefs((PyObject *) self); - } - if (self->lock_lock != NULL) { - /* Unlock the lock so it's safe to free it */ - if (self->locked) - PyThread_release_lock(self->lock_lock); - PyThread_free_lock(self->lock_lock); - } + PyObject_ClearWeakRefs((PyObject *) self); PyTypeObject *tp = Py_TYPE(self); tp->tp_free((PyObject*)self); Py_DECREF(tp); @@ -798,13 +788,12 @@ lock_PyThread_acquire_lock(PyObject *op, PyObject *args, PyObject *kwds) return NULL; } - PyLockStatus r = acquire_timed(self->lock_lock, timeout); + PyLockStatus r = _PyMutex_LockTimed(&self->lock, timeout, + _PY_LOCK_HANDLE_SIGNALS | _PY_LOCK_DETACH); if (r == PY_LOCK_INTR) { return NULL; } - if (r == PY_LOCK_ACQUIRED) - self->locked = 1; return PyBool_FromLong(r == PY_LOCK_ACQUIRED); } @@ -836,13 +825,11 @@ lock_PyThread_release_lock(PyObject *op, PyObject *Py_UNUSED(ignored)) { lockobject *self = (lockobject*)op; /* Sanity check: the lock must be locked */ - if (!self->locked) { + if (_PyMutex_TryUnlock(&self->lock) < 0) { PyErr_SetString(ThreadError, "release unlocked lock"); return NULL; } - self->locked = 0; - PyThread_release_lock(self->lock_lock); Py_RETURN_NONE; } @@ -870,7 +857,7 @@ static PyObject * lock_locked_lock(PyObject *op, PyObject *Py_UNUSED(ignored)) { lockobject *self = (lockobject*)op; - return PyBool_FromLong((long)self->locked); + return PyBool_FromLong(PyMutex_IsLocked(&self->lock)); } PyDoc_STRVAR(locked_doc, @@ -890,21 +877,15 @@ lock_repr(PyObject *op) { lockobject *self = (lockobject*)op; return PyUnicode_FromFormat("<%s %s object at %p>", - self->locked ? "locked" : "unlocked", Py_TYPE(self)->tp_name, self); + PyMutex_IsLocked(&self->lock) ? "locked" : "unlocked", Py_TYPE(self)->tp_name, self); } #ifdef HAVE_FORK static PyObject * lock__at_fork_reinit(PyObject *op, PyObject *Py_UNUSED(args)) { - lockobject *self = (lockobject*)op; - if (_PyThread_at_fork_reinit(&self->lock_lock) < 0) { - PyErr_SetString(ThreadError, "failed to reinitialize lock at fork"); - return NULL; - } - - self->locked = 0; - + lockobject *self = (lockobject *)op; + _PyMutex_at_fork_reinit(&self->lock); Py_RETURN_NONE; } #endif /* HAVE_FORK */ @@ -970,18 +951,12 @@ A lock is not owned by the thread that locked it; another thread may\n\ unlock it. A thread attempting to lock a lock that it has already locked\n\ will block until another thread unlocks it. Deadlocks may ensue."); -static PyMemberDef lock_type_members[] = { - {"__weaklistoffset__", Py_T_PYSSIZET, offsetof(lockobject, in_weakreflist), Py_READONLY}, - {NULL}, -}; - static PyType_Slot lock_type_slots[] = { {Py_tp_dealloc, lock_dealloc}, {Py_tp_repr, lock_repr}, {Py_tp_doc, (void *)lock_doc}, {Py_tp_methods, lock_methods}, {Py_tp_traverse, lock_traverse}, - {Py_tp_members, lock_type_members}, {Py_tp_new, lock_new}, {0, 0} }; @@ -990,7 +965,7 @@ static PyType_Spec lock_type_spec = { .name = "_thread.lock", .basicsize = sizeof(lockobject), .flags = (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | - Py_TPFLAGS_IMMUTABLETYPE), + Py_TPFLAGS_IMMUTABLETYPE | Py_TPFLAGS_MANAGED_WEAKREF), .slots = lock_type_slots, }; @@ -1340,16 +1315,7 @@ newlockobject(PyObject *module) if (self == NULL) { return NULL; } - - self->lock_lock = PyThread_allocate_lock(); - self->locked = 0; - self->in_weakreflist = NULL; - - if (self->lock_lock == NULL) { - Py_DECREF(self); - PyErr_SetString(ThreadError, "can't allocate lock"); - return NULL; - } + self->lock = (PyMutex){0}; return self; } diff --git a/Objects/codeobject.c b/Objects/codeobject.c index 6f0b3f8b9a3262..8a2f4d32b911d9 100644 --- a/Objects/codeobject.c +++ b/Objects/codeobject.c @@ -2336,6 +2336,7 @@ _PyCode_ConstantKey(PyObject *op) if (op == Py_None || op == Py_Ellipsis || PyLong_CheckExact(op) || PyUnicode_CheckExact(op) + || PySlice_Check(op) /* code_richcompare() uses _PyCode_ConstantKey() internally */ || PyCode_Check(op)) { diff --git a/Objects/exceptions.c b/Objects/exceptions.c index b3910855165494..c685481b13a93a 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -2994,46 +2994,55 @@ UnicodeEncodeError_init(PyObject *self, PyObject *args, PyObject *kwds) static PyObject * UnicodeEncodeError_str(PyObject *self) { - PyUnicodeErrorObject *uself = (PyUnicodeErrorObject *)self; + PyUnicodeErrorObject *exc = (PyUnicodeErrorObject *)self; PyObject *result = NULL; PyObject *reason_str = NULL; PyObject *encoding_str = NULL; - if (!uself->object) + if (exc->object == NULL) { /* Not properly initialized. */ return PyUnicode_FromString(""); + } /* Get reason and encoding as strings, which they might not be if they've been modified after we were constructed. */ - reason_str = PyObject_Str(uself->reason); - if (reason_str == NULL) + reason_str = PyObject_Str(exc->reason); + if (reason_str == NULL) { goto done; - encoding_str = PyObject_Str(uself->encoding); - if (encoding_str == NULL) + } + encoding_str = PyObject_Str(exc->encoding); + if (encoding_str == NULL) { goto done; + } + + Py_ssize_t len = PyUnicode_GET_LENGTH(exc->object); + Py_ssize_t start = exc->start, end = exc->end; - if (uself->start < PyUnicode_GET_LENGTH(uself->object) && uself->end == uself->start+1) { - Py_UCS4 badchar = PyUnicode_ReadChar(uself->object, uself->start); + if ((start >= 0 && start < len) && (end >= 0 && end <= len) && end == start + 1) { + Py_UCS4 badchar = PyUnicode_ReadChar(exc->object, start); const char *fmt; - if (badchar <= 0xff) + if (badchar <= 0xff) { fmt = "'%U' codec can't encode character '\\x%02x' in position %zd: %U"; - else if (badchar <= 0xffff) + } + else if (badchar <= 0xffff) { fmt = "'%U' codec can't encode character '\\u%04x' in position %zd: %U"; - else + } + else { fmt = "'%U' codec can't encode character '\\U%08x' in position %zd: %U"; + } result = PyUnicode_FromFormat( fmt, encoding_str, (int)badchar, - uself->start, + start, reason_str); } else { result = PyUnicode_FromFormat( "'%U' codec can't encode characters in position %zd-%zd: %U", encoding_str, - uself->start, - uself->end-1, + start, + end - 1, reason_str); } done: @@ -3107,41 +3116,46 @@ UnicodeDecodeError_init(PyObject *self, PyObject *args, PyObject *kwds) static PyObject * UnicodeDecodeError_str(PyObject *self) { - PyUnicodeErrorObject *uself = (PyUnicodeErrorObject *)self; + PyUnicodeErrorObject *exc = (PyUnicodeErrorObject *)self; PyObject *result = NULL; PyObject *reason_str = NULL; PyObject *encoding_str = NULL; - if (!uself->object) + if (exc->object == NULL) { /* Not properly initialized. */ return PyUnicode_FromString(""); + } /* Get reason and encoding as strings, which they might not be if they've been modified after we were constructed. */ - reason_str = PyObject_Str(uself->reason); - if (reason_str == NULL) + reason_str = PyObject_Str(exc->reason); + if (reason_str == NULL) { goto done; - encoding_str = PyObject_Str(uself->encoding); - if (encoding_str == NULL) + } + encoding_str = PyObject_Str(exc->encoding); + if (encoding_str == NULL) { goto done; + } + + Py_ssize_t len = PyBytes_GET_SIZE(exc->object); + Py_ssize_t start = exc->start, end = exc->end; - if (uself->start < PyBytes_GET_SIZE(uself->object) && uself->end == uself->start+1) { - int byte = (int)(PyBytes_AS_STRING(((PyUnicodeErrorObject *)self)->object)[uself->start]&0xff); + if ((start >= 0 && start < len) && (end >= 0 && end <= len) && end == start + 1) { + int badbyte = (int)(PyBytes_AS_STRING(exc->object)[start] & 0xff); result = PyUnicode_FromFormat( "'%U' codec can't decode byte 0x%02x in position %zd: %U", encoding_str, - byte, - uself->start, + badbyte, + start, reason_str); } else { result = PyUnicode_FromFormat( "'%U' codec can't decode bytes in position %zd-%zd: %U", encoding_str, - uself->start, - uself->end-1, - reason_str - ); + start, + end - 1, + reason_str); } done: Py_XDECREF(reason_str); @@ -3204,42 +3218,49 @@ UnicodeTranslateError_init(PyUnicodeErrorObject *self, PyObject *args, static PyObject * UnicodeTranslateError_str(PyObject *self) { - PyUnicodeErrorObject *uself = (PyUnicodeErrorObject *)self; + PyUnicodeErrorObject *exc = (PyUnicodeErrorObject *)self; PyObject *result = NULL; PyObject *reason_str = NULL; - if (!uself->object) + if (exc->object == NULL) { /* Not properly initialized. */ return PyUnicode_FromString(""); + } /* Get reason as a string, which it might not be if it's been modified after we were constructed. */ - reason_str = PyObject_Str(uself->reason); - if (reason_str == NULL) + reason_str = PyObject_Str(exc->reason); + if (reason_str == NULL) { goto done; + } + + Py_ssize_t len = PyUnicode_GET_LENGTH(exc->object); + Py_ssize_t start = exc->start, end = exc->end; - if (uself->start < PyUnicode_GET_LENGTH(uself->object) && uself->end == uself->start+1) { - Py_UCS4 badchar = PyUnicode_ReadChar(uself->object, uself->start); + if ((start >= 0 && start < len) && (end >= 0 && end <= len) && end == start + 1) { + Py_UCS4 badchar = PyUnicode_ReadChar(exc->object, start); const char *fmt; - if (badchar <= 0xff) + if (badchar <= 0xff) { fmt = "can't translate character '\\x%02x' in position %zd: %U"; - else if (badchar <= 0xffff) + } + else if (badchar <= 0xffff) { fmt = "can't translate character '\\u%04x' in position %zd: %U"; - else + } + else { fmt = "can't translate character '\\U%08x' in position %zd: %U"; + } result = PyUnicode_FromFormat( fmt, (int)badchar, - uself->start, - reason_str - ); - } else { + start, + reason_str); + } + else { result = PyUnicode_FromFormat( "can't translate characters in position %zd-%zd: %U", - uself->start, - uself->end-1, - reason_str - ); + start, + end - 1, + reason_str); } done: Py_XDECREF(reason_str); diff --git a/Objects/funcobject.c b/Objects/funcobject.c index 98e6766d0ca0d8..855d1a2eeca819 100644 --- a/Objects/funcobject.c +++ b/Objects/funcobject.c @@ -128,7 +128,7 @@ _PyFunction_FromConstructor(PyFrameConstructor *constr) op->func_annotate = NULL; op->func_typeparams = NULL; op->vectorcall = _PyFunction_Vectorcall; - op->func_version = 0; + op->func_version = FUNC_VERSION_UNSET; // NOTE: functions created via FrameConstructor do not use deferred // reference counting because they are typically not part of cycles // nor accessed by multiple threads. @@ -207,7 +207,7 @@ PyFunction_NewWithQualName(PyObject *code, PyObject *globals, PyObject *qualname op->func_annotate = NULL; op->func_typeparams = NULL; op->vectorcall = _PyFunction_Vectorcall; - op->func_version = 0; + op->func_version = FUNC_VERSION_UNSET; if ((code_obj->co_flags & CO_NESTED) == 0) { // Use deferred reference counting for top-level functions, but not // nested functions because they are more likely to capture variables, @@ -287,31 +287,59 @@ functions is running. */ +static inline struct _func_version_cache_item * +get_cache_item(PyInterpreterState *interp, uint32_t version) +{ + return interp->func_state.func_version_cache + + (version % FUNC_VERSION_CACHE_SIZE); +} + void _PyFunction_SetVersion(PyFunctionObject *func, uint32_t version) { + assert(func->func_version == FUNC_VERSION_UNSET); + assert(version >= FUNC_VERSION_FIRST_VALID); + // This should only be called from MAKE_FUNCTION. No code is specialized + // based on the version, so we do not need to stop the world to set it. + func->func_version = version; #ifndef Py_GIL_DISABLED PyInterpreterState *interp = _PyInterpreterState_GET(); - if (func->func_version != 0) { - struct _func_version_cache_item *slot = - interp->func_state.func_version_cache - + (func->func_version % FUNC_VERSION_CACHE_SIZE); - if (slot->func == func) { - slot->func = NULL; - // Leave slot->code alone, there may be use for it. - } - } + struct _func_version_cache_item *slot = get_cache_item(interp, version); + slot->func = func; + slot->code = func->func_code; #endif - func->func_version = version; +} + +static void +func_clear_version(PyInterpreterState *interp, PyFunctionObject *func) +{ + if (func->func_version < FUNC_VERSION_FIRST_VALID) { + // Version was never set or has already been cleared. + return; + } #ifndef Py_GIL_DISABLED - if (version != 0) { - struct _func_version_cache_item *slot = - interp->func_state.func_version_cache - + (version % FUNC_VERSION_CACHE_SIZE); - slot->func = func; - slot->code = func->func_code; + struct _func_version_cache_item *slot = + get_cache_item(interp, func->func_version); + if (slot->func == func) { + slot->func = NULL; + // Leave slot->code alone, there may be use for it. } #endif + func->func_version = FUNC_VERSION_CLEARED; +} + +// Called when any of the critical function attributes are changed +static void +_PyFunction_ClearVersion(PyFunctionObject *func) +{ + if (func->func_version < FUNC_VERSION_FIRST_VALID) { + // Version was never set or has already been cleared. + return; + } + PyInterpreterState *interp = _PyInterpreterState_GET(); + _PyEval_StopTheWorld(interp); + func_clear_version(interp, func); + _PyEval_StartTheWorld(interp); } void @@ -319,9 +347,7 @@ _PyFunction_ClearCodeByVersion(uint32_t version) { #ifndef Py_GIL_DISABLED PyInterpreterState *interp = _PyInterpreterState_GET(); - struct _func_version_cache_item *slot = - interp->func_state.func_version_cache - + (version % FUNC_VERSION_CACHE_SIZE); + struct _func_version_cache_item *slot = get_cache_item(interp, version); if (slot->code) { assert(PyCode_Check(slot->code)); PyCodeObject *code = (PyCodeObject *)slot->code; @@ -340,9 +366,7 @@ _PyFunction_LookupByVersion(uint32_t version, PyObject **p_code) return NULL; #else PyInterpreterState *interp = _PyInterpreterState_GET(); - struct _func_version_cache_item *slot = - interp->func_state.func_version_cache - + (version % FUNC_VERSION_CACHE_SIZE); + struct _func_version_cache_item *slot = get_cache_item(interp, version); if (slot->code) { assert(PyCode_Check(slot->code)); PyCodeObject *code = (PyCodeObject *)slot->code; @@ -431,7 +455,7 @@ PyFunction_SetDefaults(PyObject *op, PyObject *defaults) } handle_func_event(PyFunction_EVENT_MODIFY_DEFAULTS, (PyFunctionObject *) op, defaults); - _PyFunction_SetVersion((PyFunctionObject *)op, 0); + _PyFunction_ClearVersion((PyFunctionObject *)op); Py_XSETREF(((PyFunctionObject *)op)->func_defaults, defaults); return 0; } @@ -440,7 +464,7 @@ void PyFunction_SetVectorcall(PyFunctionObject *func, vectorcallfunc vectorcall) { assert(func != NULL); - _PyFunction_SetVersion(func, 0); + _PyFunction_ClearVersion(func); func->vectorcall = vectorcall; } @@ -473,7 +497,7 @@ PyFunction_SetKwDefaults(PyObject *op, PyObject *defaults) } handle_func_event(PyFunction_EVENT_MODIFY_KWDEFAULTS, (PyFunctionObject *) op, defaults); - _PyFunction_SetVersion((PyFunctionObject *)op, 0); + _PyFunction_ClearVersion((PyFunctionObject *)op); Py_XSETREF(((PyFunctionObject *)op)->func_kwdefaults, defaults); return 0; } @@ -506,7 +530,7 @@ PyFunction_SetClosure(PyObject *op, PyObject *closure) Py_TYPE(closure)->tp_name); return -1; } - _PyFunction_SetVersion((PyFunctionObject *)op, 0); + _PyFunction_ClearVersion((PyFunctionObject *)op); Py_XSETREF(((PyFunctionObject *)op)->func_closure, closure); return 0; } @@ -658,7 +682,7 @@ func_set_code(PyObject *self, PyObject *value, void *Py_UNUSED(ignored)) } handle_func_event(PyFunction_EVENT_MODIFY_CODE, op, value); - _PyFunction_SetVersion(op, 0); + _PyFunction_ClearVersion(op); Py_XSETREF(op->func_code, Py_NewRef(value)); return 0; } @@ -744,7 +768,7 @@ func_set_defaults(PyObject *self, PyObject *value, void *Py_UNUSED(ignored)) } handle_func_event(PyFunction_EVENT_MODIFY_DEFAULTS, op, value); - _PyFunction_SetVersion(op, 0); + _PyFunction_ClearVersion(op); Py_XSETREF(op->func_defaults, Py_XNewRef(value)); return 0; } @@ -787,7 +811,7 @@ func_set_kwdefaults(PyObject *self, PyObject *value, void *Py_UNUSED(ignored)) } handle_func_event(PyFunction_EVENT_MODIFY_KWDEFAULTS, op, value); - _PyFunction_SetVersion(op, 0); + _PyFunction_ClearVersion(op); Py_XSETREF(op->func_kwdefaults, Py_XNewRef(value)); return 0; } @@ -1030,7 +1054,7 @@ static int func_clear(PyObject *self) { PyFunctionObject *op = _PyFunction_CAST(self); - _PyFunction_SetVersion(op, 0); + func_clear_version(_PyInterpreterState_GET(), op); Py_CLEAR(op->func_globals); Py_CLEAR(op->func_builtins); Py_CLEAR(op->func_module); @@ -1068,7 +1092,6 @@ func_dealloc(PyObject *self) if (op->func_weakreflist != NULL) { PyObject_ClearWeakRefs((PyObject *) op); } - _PyFunction_SetVersion(op, 0); (void)func_clear((PyObject*)op); // These aren't cleared by func_clear(). Py_DECREF(op->func_code); diff --git a/Objects/longobject.c b/Objects/longobject.c index 6ca8d449bcf4a2..4e948940485730 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -4828,21 +4828,12 @@ long_divmod(PyObject *a, PyObject *b) static PyLongObject * long_invmod(PyLongObject *a, PyLongObject *n) { - PyLongObject *b, *c; - /* Should only ever be called for positive n */ assert(_PyLong_IsPositive(n)); - b = (PyLongObject *)PyLong_FromLong(1L); - if (b == NULL) { - return NULL; - } - c = (PyLongObject *)PyLong_FromLong(0L); - if (c == NULL) { - Py_DECREF(b); - return NULL; - } Py_INCREF(a); + PyLongObject *b = (PyLongObject *)Py_NewRef(_PyLong_GetOne()); + PyLongObject *c = (PyLongObject *)Py_NewRef(_PyLong_GetZero()); Py_INCREF(n); /* references now owned: a, b, c, n */ diff --git a/Objects/sliceobject.c b/Objects/sliceobject.c index 1b6d35998c2b69..4fef0af93fe095 100644 --- a/Objects/sliceobject.c +++ b/Objects/sliceobject.c @@ -343,7 +343,7 @@ Create a slice object. This is used for extended slicing (e.g. a[0:10:2])."); static void slice_dealloc(PySliceObject *r) { - _PyObject_GC_UNTRACK(r); + PyObject_GC_UnTrack(r); Py_DECREF(r->step); Py_DECREF(r->start); Py_DECREF(r->stop); diff --git a/Objects/stringlib/eq.h b/Objects/stringlib/eq.h index 2eac4baf5ca9ce..821b692f26b830 100644 --- a/Objects/stringlib/eq.h +++ b/Objects/stringlib/eq.h @@ -4,14 +4,19 @@ * unicode_eq() is called when the hash of two unicode objects is equal. */ Py_LOCAL_INLINE(int) -unicode_eq(PyObject *a, PyObject *b) +unicode_eq(PyObject *str1, PyObject *str2) { - if (PyUnicode_GET_LENGTH(a) != PyUnicode_GET_LENGTH(b)) + Py_ssize_t len = PyUnicode_GET_LENGTH(str1); + if (PyUnicode_GET_LENGTH(str2) != len) { return 0; - if (PyUnicode_GET_LENGTH(a) == 0) - return 1; - if (PyUnicode_KIND(a) != PyUnicode_KIND(b)) + } + + int kind = PyUnicode_KIND(str1); + if (PyUnicode_KIND(str2) != kind) { return 0; - return memcmp(PyUnicode_1BYTE_DATA(a), PyUnicode_1BYTE_DATA(b), - PyUnicode_GET_LENGTH(a) * PyUnicode_KIND(a)) == 0; + } + + const void *data1 = PyUnicode_DATA(str1); + const void *data2 = PyUnicode_DATA(str2); + return (memcmp(data1, data2, len * kind) == 0); } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index e9589cfe44f3bf..60d4875d3b393e 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -11001,6 +11001,24 @@ _PyUnicode_Equal(PyObject *str1, PyObject *str2) } +int +PyUnicode_Equal(PyObject *str1, PyObject *str2) +{ + if (!PyUnicode_Check(str1)) { + PyErr_Format(PyExc_TypeError, + "first argument must be str, not %T", str1); + return -1; + } + if (!PyUnicode_Check(str2)) { + PyErr_Format(PyExc_TypeError, + "second argument must be str, not %T", str2); + return -1; + } + + return _PyUnicode_Equal(str1, str2); +} + + int PyUnicode_Compare(PyObject *left, PyObject *right) { diff --git a/PC/python3dll.c b/PC/python3dll.c index 6b8208ab90bd95..9296474617e115 100755 --- a/PC/python3dll.c +++ b/PC/python3dll.c @@ -717,6 +717,7 @@ EXPORT_FUNC(PyUnicode_DecodeUTF8Stateful) EXPORT_FUNC(PyUnicode_EncodeCodePage) EXPORT_FUNC(PyUnicode_EncodeFSDefault) EXPORT_FUNC(PyUnicode_EncodeLocale) +EXPORT_FUNC(PyUnicode_Equal) EXPORT_FUNC(PyUnicode_EqualToUTF8) EXPORT_FUNC(PyUnicode_EqualToUTF8AndSize) EXPORT_FUNC(PyUnicode_Find) diff --git a/Programs/_testembed.c b/Programs/_testembed.c index ab2b2d06cca15d..ab619e32429d63 100644 --- a/Programs/_testembed.c +++ b/Programs/_testembed.c @@ -311,14 +311,36 @@ static int test_pre_initialization_api(void) _Py_EMBED_PREINIT_CHECK("Checking Py_SetProgramName\n"); Py_SetProgramName(program); + _Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized pre-initialization\n"); + if (Py_IsInitialized()) { + fprintf(stderr, "Fatal error: initialized before initialization!\n"); + return 1; + } + _Py_EMBED_PREINIT_CHECK("Initializing interpreter\n"); Py_Initialize(); + + _Py_EMBED_PREINIT_CHECK("Checking Py_IsInitialized post-initialization\n"); + if (!Py_IsInitialized()) { + fprintf(stderr, "Fatal error: not initialized after initialization!\n"); + return 1; + } + _Py_EMBED_PREINIT_CHECK("Check sys module contents\n"); - PyRun_SimpleString("import sys; " - "print('sys.executable:', sys.executable)"); + PyRun_SimpleString( + "import sys; " + "print('sys.executable:', sys.executable); " + "sys.stdout.flush(); " + ); _Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n"); Py_Finalize(); + _Py_EMBED_PREINIT_CHECK("Checking !Py_IsInitialized post-finalization\n"); + if (Py_IsInitialized()) { + fprintf(stderr, "Fatal error: still initialized after finalization!\n"); + return 1; + } + _Py_EMBED_PREINIT_CHECK("Freeing memory allocated by Py_DecodeLocale\n"); PyMem_RawFree(program); return 0; @@ -364,12 +386,15 @@ static int test_pre_initialization_sys_options(void) _Py_EMBED_PREINIT_CHECK("Initializing interpreter\n"); _testembed_Py_InitializeFromConfig(); _Py_EMBED_PREINIT_CHECK("Check sys module contents\n"); - PyRun_SimpleString("import sys; " - "print('sys.warnoptions:', sys.warnoptions); " - "print('sys._xoptions:', sys._xoptions); " - "warnings = sys.modules['warnings']; " - "latest_filters = [f[0] for f in warnings.filters[:3]]; " - "print('warnings.filters[:3]:', latest_filters)"); + PyRun_SimpleString( + "import sys; " + "print('sys.warnoptions:', sys.warnoptions); " + "print('sys._xoptions:', sys._xoptions); " + "warnings = sys.modules['warnings']; " + "latest_filters = [f[0] for f in warnings.filters[:3]]; " + "print('warnings.filters[:3]:', latest_filters); " + "sys.stdout.flush(); " + ); _Py_EMBED_PREINIT_CHECK("Finalizing interpreter\n"); Py_Finalize(); diff --git a/Python/bytecodes.c b/Python/bytecodes.c index a0edf17d747e77..228d82173e6126 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1922,9 +1922,10 @@ dummy_func( DECREF_INPUTS(); ERROR_IF(super == NULL, error); PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2); - attr = PyStackRef_FromPyObjectSteal(PyObject_GetAttr(super, name)); + PyObject *attr_o = PyObject_GetAttr(super, name); Py_DECREF(super); - ERROR_IF(PyStackRef_IsNull(attr), error); + ERROR_IF(attr_o == NULL, error); + attr = PyStackRef_FromPyObjectSteal(attr_o); null = PyStackRef_NULL; } @@ -2740,9 +2741,10 @@ dummy_func( inst(GET_ITER, (iterable -- iter)) { /* before: [obj]; after [getiter(obj)] */ - iter = PyStackRef_FromPyObjectSteal(PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable))); + PyObject *iter_o = PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable)); DECREF_INPUTS(); - ERROR_IF(PyStackRef_IsNull(iter), error); + ERROR_IF(iter_o == NULL, error); + iter = PyStackRef_FromPyObjectSteal(iter_o); } inst(GET_YIELD_FROM_ITER, (iterable -- iter)) { @@ -3052,16 +3054,18 @@ dummy_func( PyObject *owner_o = PyStackRef_AsPyObjectSteal(owner); PyObject *name = _Py_SpecialMethods[oparg].name; PyObject *self_or_null_o; - attr = PyStackRef_FromPyObjectSteal(_PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o)); - if (PyStackRef_IsNull(attr)) { + PyObject *attr_o = _PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o); + if (attr_o == NULL) { if (!_PyErr_Occurred(tstate)) { _PyErr_Format(tstate, PyExc_TypeError, _Py_SpecialMethods[oparg].error, Py_TYPE(owner_o)->tp_name); } + ERROR_IF(true, error); } - ERROR_IF(PyStackRef_IsNull(attr), error); - self_or_null = PyStackRef_FromPyObjectSteal(self_or_null_o); + attr = PyStackRef_FromPyObjectSteal(attr_o); + self_or_null = self_or_null_o == NULL ? + PyStackRef_NULL : PyStackRef_FromPyObjectSteal(self_or_null_o); } inst(WITH_EXCEPT_START, (exit_func, exit_self, lasti, unused, val -- exit_func, exit_self, lasti, unused, val, res)) { @@ -3092,9 +3096,10 @@ dummy_func( (void)lasti; // Shut up compiler warning if asserts are off PyObject *stack[5] = {NULL, PyStackRef_AsPyObjectBorrow(exit_self), exc, val_o, tb}; int has_self = !PyStackRef_IsNull(exit_self); - res = PyStackRef_FromPyObjectSteal(PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, - (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL)); - ERROR_IF(PyStackRef_IsNull(res), error); + PyObject *res_o = PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, + (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); + ERROR_IF(res_o == NULL, error); + res = PyStackRef_FromPyObjectSteal(res_o); } pseudo(SETUP_FINALLY, (-- unused), (HAS_ARG)) = { @@ -4539,9 +4544,10 @@ dummy_func( /* If value is a unicode object, then we know the result * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value_o)) { - res = PyStackRef_FromPyObjectSteal(PyObject_Format(value_o, NULL)); + PyObject *res_o = PyObject_Format(value_o, NULL); PyStackRef_CLOSE(value); - ERROR_IF(PyStackRef_IsNull(res), error); + ERROR_IF(res_o == NULL, error); + res = PyStackRef_FromPyObjectSteal(res_o); } else { res = value; diff --git a/Python/codegen.c b/Python/codegen.c index 896c30cc14952a..689d2b5124e9d3 100644 --- a/Python/codegen.c +++ b/Python/codegen.c @@ -194,6 +194,7 @@ static int codegen_visit_expr(compiler *, expr_ty); static int codegen_augassign(compiler *, stmt_ty); static int codegen_annassign(compiler *, stmt_ty); static int codegen_subscript(compiler *, expr_ty); +static int codegen_slice_two_parts(compiler *, expr_ty); static int codegen_slice(compiler *, expr_ty); static bool are_all_items_const(asdl_expr_seq *, Py_ssize_t, Py_ssize_t); @@ -5005,12 +5006,8 @@ codegen_visit_expr(compiler *c, expr_ty e) } break; case Slice_kind: - { - int n = codegen_slice(c, e); - RETURN_IF_ERROR(n); - ADDOP_I(c, loc, BUILD_SLICE, n); + RETURN_IF_ERROR(codegen_slice(c, e)); break; - } case Name_kind: return codegen_nameop(c, loc, e->v.Name.id, e->v.Name.ctx); /* child nodes of List and Tuple will have expr_context set */ @@ -5023,9 +5020,22 @@ codegen_visit_expr(compiler *c, expr_ty e) } static bool -is_two_element_slice(expr_ty s) +is_constant_slice(expr_ty s) { return s->kind == Slice_kind && + (s->v.Slice.lower == NULL || + s->v.Slice.lower->kind == Constant_kind) && + (s->v.Slice.upper == NULL || + s->v.Slice.upper->kind == Constant_kind) && + (s->v.Slice.step == NULL || + s->v.Slice.step->kind == Constant_kind); +} + +static bool +should_apply_two_element_slice_optimization(expr_ty s) +{ + return !is_constant_slice(s) && + s->kind == Slice_kind && s->v.Slice.step == NULL; } @@ -5046,8 +5056,8 @@ codegen_augassign(compiler *c, stmt_ty s) break; case Subscript_kind: VISIT(c, expr, e->v.Subscript.value); - if (is_two_element_slice(e->v.Subscript.slice)) { - RETURN_IF_ERROR(codegen_slice(c, e->v.Subscript.slice)); + if (should_apply_two_element_slice_optimization(e->v.Subscript.slice)) { + RETURN_IF_ERROR(codegen_slice_two_parts(c, e->v.Subscript.slice)); ADDOP_I(c, loc, COPY, 3); ADDOP_I(c, loc, COPY, 3); ADDOP_I(c, loc, COPY, 3); @@ -5084,7 +5094,7 @@ codegen_augassign(compiler *c, stmt_ty s) ADDOP_NAME(c, loc, STORE_ATTR, e->v.Attribute.attr, names); break; case Subscript_kind: - if (is_two_element_slice(e->v.Subscript.slice)) { + if (should_apply_two_element_slice_optimization(e->v.Subscript.slice)) { ADDOP_I(c, loc, SWAP, 4); ADDOP_I(c, loc, SWAP, 3); ADDOP_I(c, loc, SWAP, 2); @@ -5231,8 +5241,10 @@ codegen_subscript(compiler *c, expr_ty e) } VISIT(c, expr, e->v.Subscript.value); - if (is_two_element_slice(e->v.Subscript.slice) && ctx != Del) { - RETURN_IF_ERROR(codegen_slice(c, e->v.Subscript.slice)); + if (should_apply_two_element_slice_optimization(e->v.Subscript.slice) && + ctx != Del + ) { + RETURN_IF_ERROR(codegen_slice_two_parts(c, e->v.Subscript.slice)); if (ctx == Load) { ADDOP(c, loc, BINARY_SLICE); } @@ -5254,15 +5266,9 @@ codegen_subscript(compiler *c, expr_ty e) return SUCCESS; } -/* Returns the number of the values emitted, - * thus are needed to build the slice, or -1 if there is an error. */ static int -codegen_slice(compiler *c, expr_ty s) +codegen_slice_two_parts(compiler *c, expr_ty s) { - int n = 2; - assert(s->kind == Slice_kind); - - /* only handles the cases where BUILD_SLICE is emitted */ if (s->v.Slice.lower) { VISIT(c, expr, s->v.Slice.lower); } @@ -5277,11 +5283,45 @@ codegen_slice(compiler *c, expr_ty s) ADDOP_LOAD_CONST(c, LOC(s), Py_None); } + return 0; +} + +static int +codegen_slice(compiler *c, expr_ty s) +{ + int n = 2; + assert(s->kind == Slice_kind); + + if (is_constant_slice(s)) { + PyObject *start = NULL; + if (s->v.Slice.lower) { + start = s->v.Slice.lower->v.Constant.value; + } + PyObject *stop = NULL; + if (s->v.Slice.upper) { + stop = s->v.Slice.upper->v.Constant.value; + } + PyObject *step = NULL; + if (s->v.Slice.step) { + step = s->v.Slice.step->v.Constant.value; + } + PyObject *slice = PySlice_New(start, stop, step); + if (slice == NULL) { + return ERROR; + } + ADDOP_LOAD_CONST_NEW(c, LOC(s), slice); + return SUCCESS; + } + + RETURN_IF_ERROR(codegen_slice_two_parts(c, s)); + if (s->v.Slice.step) { n++; VISIT(c, expr, s->v.Slice.step); } - return n; + + ADDOP_I(c, LOC(s), BUILD_SLICE, n); + return SUCCESS; } diff --git a/Python/compile.c b/Python/compile.c index 9826d3fbbde976..d463fcde204a05 100644 --- a/Python/compile.c +++ b/Python/compile.c @@ -1534,7 +1534,7 @@ _PyCompile_CodeGen(PyObject *ast, PyObject *filename, PyCompilerFlags *pflags, _PyCompile_CodeUnitMetadata *umd = &c->u->u_metadata; -#define SET_MATADATA_INT(key, value) do { \ +#define SET_METADATA_INT(key, value) do { \ PyObject *v = PyLong_FromLong((long)value); \ if (v == NULL) goto finally; \ int res = PyDict_SetItemString(metadata, key, v); \ @@ -1542,10 +1542,10 @@ _PyCompile_CodeGen(PyObject *ast, PyObject *filename, PyCompilerFlags *pflags, if (res < 0) goto finally; \ } while (0); - SET_MATADATA_INT("argcount", umd->u_argcount); - SET_MATADATA_INT("posonlyargcount", umd->u_posonlyargcount); - SET_MATADATA_INT("kwonlyargcount", umd->u_kwonlyargcount); -#undef SET_MATADATA_INT + SET_METADATA_INT("argcount", umd->u_argcount); + SET_METADATA_INT("posonlyargcount", umd->u_posonlyargcount); + SET_METADATA_INT("kwonlyargcount", umd->u_kwonlyargcount); +#undef SET_METADATA_INT int addNone = mod->kind != Expression_kind; if (_PyCodegen_AddReturnAtEnd(c, addNone) < 0) { diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 7631ff75ecafd6..4574e183921006 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -3281,10 +3281,11 @@ iterable = stack_pointer[-1]; /* before: [obj]; after [getiter(obj)] */ _PyFrame_SetStackPointer(frame, stack_pointer); - iter = PyStackRef_FromPyObjectSteal(PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable))); + PyObject *iter_o = PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable)); stack_pointer = _PyFrame_GetStackPointer(frame); PyStackRef_CLOSE(iterable); - if (PyStackRef_IsNull(iter)) JUMP_TO_ERROR(); + if (iter_o == NULL) JUMP_TO_ERROR(); + iter = PyStackRef_FromPyObjectSteal(iter_o); stack_pointer[-1] = iter; break; } @@ -3552,24 +3553,21 @@ stack_pointer += -1; assert(WITHIN_STACK_BOUNDS()); _PyFrame_SetStackPointer(frame, stack_pointer); - attr = PyStackRef_FromPyObjectSteal(_PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o)); + PyObject *attr_o = _PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o); stack_pointer = _PyFrame_GetStackPointer(frame); - if (PyStackRef_IsNull(attr)) { + if (attr_o == NULL) { if (!_PyErr_Occurred(tstate)) { - stack_pointer[0] = attr; - stack_pointer += 1; - assert(WITHIN_STACK_BOUNDS()); _PyFrame_SetStackPointer(frame, stack_pointer); _PyErr_Format(tstate, PyExc_TypeError, _Py_SpecialMethods[oparg].error, Py_TYPE(owner_o)->tp_name); stack_pointer = _PyFrame_GetStackPointer(frame); - stack_pointer += -1; - assert(WITHIN_STACK_BOUNDS()); } + if (true) JUMP_TO_ERROR(); } - if (PyStackRef_IsNull(attr)) JUMP_TO_ERROR(); - self_or_null = PyStackRef_FromPyObjectSteal(self_or_null_o); + attr = PyStackRef_FromPyObjectSteal(attr_o); + self_or_null = self_or_null_o == NULL ? + PyStackRef_NULL : PyStackRef_FromPyObjectSteal(self_or_null_o); stack_pointer[0] = attr; stack_pointer[1] = self_or_null; stack_pointer += 2; @@ -3613,10 +3611,11 @@ PyObject *stack[5] = {NULL, PyStackRef_AsPyObjectBorrow(exit_self), exc, val_o, tb}; int has_self = !PyStackRef_IsNull(exit_self); _PyFrame_SetStackPointer(frame, stack_pointer); - res = PyStackRef_FromPyObjectSteal(PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, - (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL)); + PyObject *res_o = PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, + (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); stack_pointer = _PyFrame_GetStackPointer(frame); - if (PyStackRef_IsNull(res)) JUMP_TO_ERROR(); + if (res_o == NULL) JUMP_TO_ERROR(); + res = PyStackRef_FromPyObjectSteal(res_o); stack_pointer[0] = res; stack_pointer += 1; assert(WITHIN_STACK_BOUNDS()); @@ -5373,10 +5372,11 @@ * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value_o)) { _PyFrame_SetStackPointer(frame, stack_pointer); - res = PyStackRef_FromPyObjectSteal(PyObject_Format(value_o, NULL)); + PyObject *res_o = PyObject_Format(value_o, NULL); stack_pointer = _PyFrame_GetStackPointer(frame); PyStackRef_CLOSE(value); - if (PyStackRef_IsNull(res)) JUMP_TO_ERROR(); + if (res_o == NULL) JUMP_TO_ERROR(); + res = PyStackRef_FromPyObjectSteal(res_o); } else { res = value; diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index a9c127784edc83..e1107caa197d7a 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -3856,10 +3856,11 @@ * of format(value) is value itself. */ if (!PyUnicode_CheckExact(value_o)) { _PyFrame_SetStackPointer(frame, stack_pointer); - res = PyStackRef_FromPyObjectSteal(PyObject_Format(value_o, NULL)); + PyObject *res_o = PyObject_Format(value_o, NULL); stack_pointer = _PyFrame_GetStackPointer(frame); PyStackRef_CLOSE(value); - if (PyStackRef_IsNull(res)) goto pop_1_error; + if (res_o == NULL) goto pop_1_error; + res = PyStackRef_FromPyObjectSteal(res_o); } else { res = value; @@ -4239,10 +4240,11 @@ iterable = stack_pointer[-1]; /* before: [obj]; after [getiter(obj)] */ _PyFrame_SetStackPointer(frame, stack_pointer); - iter = PyStackRef_FromPyObjectSteal(PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable))); + PyObject *iter_o = PyObject_GetIter(PyStackRef_AsPyObjectBorrow(iterable)); stack_pointer = _PyFrame_GetStackPointer(frame); PyStackRef_CLOSE(iterable); - if (PyStackRef_IsNull(iter)) goto pop_1_error; + if (iter_o == NULL) goto pop_1_error; + iter = PyStackRef_FromPyObjectSteal(iter_o); stack_pointer[-1] = iter; DISPATCH(); } @@ -6270,29 +6272,21 @@ stack_pointer += -1; assert(WITHIN_STACK_BOUNDS()); _PyFrame_SetStackPointer(frame, stack_pointer); - attr = PyStackRef_FromPyObjectSteal(_PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o)); + PyObject *attr_o = _PyObject_LookupSpecialMethod(owner_o, name, &self_or_null_o); stack_pointer = _PyFrame_GetStackPointer(frame); - if (PyStackRef_IsNull(attr)) { + if (attr_o == NULL) { if (!_PyErr_Occurred(tstate)) { - stack_pointer[0] = attr; - stack_pointer += 1; - assert(WITHIN_STACK_BOUNDS()); _PyFrame_SetStackPointer(frame, stack_pointer); _PyErr_Format(tstate, PyExc_TypeError, _Py_SpecialMethods[oparg].error, Py_TYPE(owner_o)->tp_name); stack_pointer = _PyFrame_GetStackPointer(frame); - stack_pointer += -1; - assert(WITHIN_STACK_BOUNDS()); } + if (true) goto error; } - if (PyStackRef_IsNull(attr)) { - stack_pointer[0] = attr; - stack_pointer += 1; - assert(WITHIN_STACK_BOUNDS()); - goto error; - } - self_or_null = PyStackRef_FromPyObjectSteal(self_or_null_o); + attr = PyStackRef_FromPyObjectSteal(attr_o); + self_or_null = self_or_null_o == NULL ? + PyStackRef_NULL : PyStackRef_FromPyObjectSteal(self_or_null_o); stack_pointer[0] = attr; stack_pointer[1] = self_or_null; stack_pointer += 2; @@ -6385,10 +6379,11 @@ stack_pointer += -3; assert(WITHIN_STACK_BOUNDS()); _PyFrame_SetStackPointer(frame, stack_pointer); - attr = PyStackRef_FromPyObjectSteal(PyObject_GetAttr(super, name)); + PyObject *attr_o = PyObject_GetAttr(super, name); stack_pointer = _PyFrame_GetStackPointer(frame); Py_DECREF(super); - if (PyStackRef_IsNull(attr)) goto error; + if (attr_o == NULL) goto error; + attr = PyStackRef_FromPyObjectSteal(attr_o); null = PyStackRef_NULL; } stack_pointer[0] = attr; @@ -8139,10 +8134,11 @@ PyObject *stack[5] = {NULL, PyStackRef_AsPyObjectBorrow(exit_self), exc, val_o, tb}; int has_self = !PyStackRef_IsNull(exit_self); _PyFrame_SetStackPointer(frame, stack_pointer); - res = PyStackRef_FromPyObjectSteal(PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, - (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL)); + PyObject *res_o = PyObject_Vectorcall(exit_func_o, stack + 2 - has_self, + (3 + has_self) | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); stack_pointer = _PyFrame_GetStackPointer(frame); - if (PyStackRef_IsNull(res)) goto error; + if (res_o == NULL) goto error; + res = PyStackRef_FromPyObjectSteal(res_o); stack_pointer[0] = res; stack_pointer += 1; assert(WITHIN_STACK_BOUNDS()); diff --git a/Python/marshal.c b/Python/marshal.c index b1708a7306f9e7..3d127b4e331d0d 100644 --- a/Python/marshal.c +++ b/Python/marshal.c @@ -76,6 +76,7 @@ module marshal #define TYPE_UNKNOWN '?' #define TYPE_SET '<' #define TYPE_FROZENSET '>' +#define TYPE_SLICE ':' #define FLAG_REF '\x80' /* with a type, add obj to index */ #define TYPE_ASCII 'a' @@ -613,6 +614,13 @@ w_complex_object(PyObject *v, char flag, WFILE *p) w_pstring(view.buf, view.len, p); PyBuffer_Release(&view); } + else if (PySlice_Check(v)) { + PySliceObject *slice = (PySliceObject *)v; + W_TYPE(TYPE_SLICE, p); + w_object(slice->start, p); + w_object(slice->stop, p); + w_object(slice->step, p); + } else { W_TYPE(TYPE_UNKNOWN, p); p->error = WFERR_UNMARSHALLABLE; @@ -1534,6 +1542,32 @@ r_object(RFILE *p) retval = Py_NewRef(v); break; + case TYPE_SLICE: + { + Py_ssize_t idx = r_ref_reserve(flag, p); + PyObject *stop = NULL; + PyObject *step = NULL; + PyObject *start = r_object(p); + if (start == NULL) { + goto cleanup; + } + stop = r_object(p); + if (stop == NULL) { + goto cleanup; + } + step = r_object(p); + if (step == NULL) { + goto cleanup; + } + retval = PySlice_New(start, stop, step); + r_ref_insert(retval, idx, flag, p); + cleanup: + Py_XDECREF(start); + Py_XDECREF(stop); + Py_XDECREF(step); + break; + } + default: /* Bogus data got written, which isn't ideal. This will let you keep working and recover. */ diff --git a/Python/specialize.c b/Python/specialize.c index d8bff39511cf12..4b33a468733d6b 100644 --- a/Python/specialize.c +++ b/Python/specialize.c @@ -1599,7 +1599,7 @@ function_get_version(PyObject *o, int opcode) assert(Py_IS_TYPE(o, &PyFunction_Type)); PyFunctionObject *func = (PyFunctionObject *)o; uint32_t version = _PyFunction_GetVersionForCurrentState(func); - if (version == 0) { + if (!_PyFunction_IsVersionValid(version)) { SPECIALIZATION_FAIL(opcode, SPEC_FAIL_OUT_OF_VERSIONS); return 0; } @@ -1692,7 +1692,7 @@ _Py_Specialize_BinarySubscr( goto fail; } uint32_t version = _PyFunction_GetVersionForCurrentState(func); - if (version == 0) { + if (!_PyFunction_IsVersionValid(version)) { SPECIALIZATION_FAIL(BINARY_SUBSCR, SPEC_FAIL_OUT_OF_VERSIONS); goto fail; } @@ -1977,7 +1977,7 @@ specialize_py_call(PyFunctionObject *func, _Py_CODEUNIT *instr, int nargs, argcount = code->co_argcount; } int version = _PyFunction_GetVersionForCurrentState(func); - if (version == 0) { + if (!_PyFunction_IsVersionValid(version)) { SPECIALIZATION_FAIL(CALL, SPEC_FAIL_OUT_OF_VERSIONS); return -1; } @@ -2009,7 +2009,7 @@ specialize_py_call_kw(PyFunctionObject *func, _Py_CODEUNIT *instr, int nargs, return -1; } int version = _PyFunction_GetVersionForCurrentState(func); - if (version == 0) { + if (!_PyFunction_IsVersionValid(version)) { SPECIALIZATION_FAIL(CALL, SPEC_FAIL_OUT_OF_VERSIONS); return -1; } diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index a0be2a0a203f8c..badd7b79102310 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -358,7 +358,6 @@ Modules/_testclinic.c - TestClass - ##----------------------- ## static types -Modules/_cursesmodule.c - PyCursesWindow_Type - Modules/_datetimemodule.c - PyDateTime_DateTimeType - Modules/_datetimemodule.c - PyDateTime_DateType - Modules/_datetimemodule.c - PyDateTime_DeltaType - @@ -383,7 +382,6 @@ Modules/_tkinter.c - Tktt_Type - Modules/xxlimited_35.c - Xxo_Type - ## exception types -Modules/_cursesmodule.c - PyCursesError - Modules/_tkinter.c - Tkinter_TclError - Modules/xxlimited_35.c - ErrorObject - Modules/xxmodule.c - ErrorObject - @@ -409,6 +407,7 @@ Modules/_tkinter.c - trbInCmd - Include/datetime.h - PyDateTimeAPI - Modules/_ctypes/cfield.c _ctypes_get_fielddesc initialized - Modules/_ctypes/malloc_closure.c - _pagesize - +Modules/_cursesmodule.c - curses_module_loaded - Modules/_cursesmodule.c - curses_initscr_called - Modules/_cursesmodule.c - curses_setupterm_called - Modules/_cursesmodule.c - curses_start_color_called - @@ -423,7 +422,6 @@ Modules/readline.c - libedit_history_start - Modules/_ctypes/cfield.c - formattable - Modules/_ctypes/malloc_closure.c - free_list - -Modules/_cursesmodule.c - curses_global_state - Modules/_curses_panel.c - lop - Modules/_ssl/debughelpers.c _PySSL_keylog_callback lock - Modules/_tkinter.c - quitMainLoop - diff --git a/Tools/cases_generator/generators_common.py b/Tools/cases_generator/generators_common.py index f32a20b304c354..0bfa1a3b56fbc2 100644 --- a/Tools/cases_generator/generators_common.py +++ b/Tools/cases_generator/generators_common.py @@ -43,13 +43,13 @@ def peek(self) -> Token | None: break return self.look_ahead -ROOT = Path(__file__).parent.parent.parent -DEFAULT_INPUT = (ROOT / "Python/bytecodes.c").absolute().as_posix() +ROOT = Path(__file__).parent.parent.parent.resolve() +DEFAULT_INPUT = (ROOT / "Python/bytecodes.c").as_posix() def root_relative_path(filename: str) -> str: try: - return Path(filename).absolute().relative_to(ROOT).as_posix() + return Path(filename).resolve().relative_to(ROOT).as_posix() except ValueError: # Not relative to root, just return original path. return filename