Merge branch 'main' into capi-number-tests

Author: skirpichev

Doc/c-api/list.rst

@@ -128,6 +128,30 @@ List Objects
    list is not supported.
 
 
+.. c:function:: int PyList_Extend(PyObject *list, PyObject *iterable)
+
+   Extend *list* with the contents of *iterable*.  This is the same as
+   ``PyList_SetSlice(list, PY_SSIZE_T_MAX, PY_SSIZE_T_MAX, iterable)``
+   and analogous to ``list.extend(iterable)`` or ``list += iterable``.
+
+   Raise an exception and return ``-1`` if *list* is not a :class:`list`
+   object. Return 0 on success.
+
+   .. versionadded:: 3.13
+
+
+.. c:function:: int PyList_Clear(PyObject *list)
+
+   Remove all items from *list*.  This is the same as
+   ``PyList_SetSlice(list, 0, PY_SSIZE_T_MAX, NULL)`` and analogous to
+   ``list.clear()`` or ``del list[:]``.
+
+   Raise an exception and return ``-1`` if *list* is not a :class:`list`
+   object.  Return 0 on success.
+
+   .. versionadded:: 3.13
+
+
 .. c:function:: int PyList_Sort(PyObject *list)
 
    Sort the items of *list* in place.  Return ``0`` on success, ``-1`` on

Doc/library/glob.rst

@@ -34,9 +34,7 @@ unlike :func:`fnmatch.fnmatch` or :func:`pathlib.Path.glob`.
 For a literal match, wrap the meta-characters in brackets.
 For example, ``'[?]'`` matches the character ``'?'``.
 
-
-.. seealso::
-   The :mod:`pathlib` module offers high-level path objects.
+The :mod:`glob` module defines the following functions:
 
 
 .. function:: glob(pathname, *, root_dir=None, dir_fd=None, recursive=False, \
@@ -117,7 +115,48 @@ For example, ``'[?]'`` matches the character ``'?'``.
    .. versionadded:: 3.4
 
 
-For example, consider a directory containing the following files:
+.. function:: translate(pathname, *, recursive=False, include_hidden=False, seps=None)
+
+   Convert the given path specification to a regular expression for use with
+   :func:`re.match`. The path specification can contain shell-style wildcards.
+
+   For example:
+
+      >>> import glob, re
+      >>>
+      >>> regex = glob.translate('**/*.txt', recursive=True, include_hidden=True)
+      >>> regex
+      '(?s:(?:.+/)?[^/]*\\.txt)\\Z'
+      >>> reobj = re.compile(regex)
+      >>> reobj.match('foo/bar/baz.txt')
+      <re.Match object; span=(0, 15), match='foo/bar/baz.txt'>
+
+   Path separators and segments are meaningful to this function, unlike
+   :func:`fnmatch.translate`. By default wildcards do not match path
+   separators, and ``*`` pattern segments match precisely one path segment.
+
+   If *recursive* is true, the pattern segment "``**``" will match any number
+   of path segments. If "``**``" occurs in any position other than a full
+   pattern segment, :exc:`ValueError` is raised.
+
+   If *include_hidden* is true, wildcards can match path segments that start
+   with a dot (``.``).
+
+   A sequence of path separators may be supplied to the *seps* argument. If
+   not given, :data:`os.sep` and :data:`~os.altsep` (if available) are used.
+
+   .. seealso::
+
+     :meth:`pathlib.PurePath.match` and :meth:`pathlib.Path.glob` methods,
+     which call this function to implement pattern matching and globbing.
+
+   .. versionadded:: 3.13
+
+
+Examples
+--------
+
+Consider a directory containing the following files:
 :file:`1.gif`, :file:`2.txt`, :file:`card.gif` and a subdirectory :file:`sub`
 which contains only the file :file:`3.txt`.  :func:`glob` will produce
 the following results.  Notice how any leading components of the path are
@@ -146,6 +185,7 @@ default. For example, consider a directory containing :file:`card.gif` and
    ['.card.gif']
 
 .. seealso::
+   The :mod:`fnmatch` module offers shell-style filename (not path) expansion.
 
-   Module :mod:`fnmatch`
-      Shell-style filename (not path) expansion
+.. seealso::
+   The :mod:`pathlib` module offers high-level path objects.

Doc/library/stdtypes.rst

@@ -4755,14 +4755,17 @@ support membership tests:
 
    .. versionadded:: 3.10
 
-Keys views are set-like since their entries are unique and :term:`hashable`.  If all
-values are hashable, so that ``(key, value)`` pairs are unique and hashable,
-then the items view is also set-like.  (Values views are not treated as set-like
+Keys views are set-like since their entries are unique and :term:`hashable`.
+Items views also have set-like operations since the (key, value) pairs
+are unique and the keys are hashable.
+If all values in an items view are hashable as well,
+then the items view can interoperate with other sets.
+(Values views are not treated as set-like
 since the entries are generally not unique.)  For set-like views, all of the
 operations defined for the abstract base class :class:`collections.abc.Set` are
 available (for example, ``==``, ``<``, or ``^``).  While using set operators,
-set-like views accept any iterable as the other operand, unlike sets which only
-accept sets as the input.
+set-like views accept any iterable as the other operand,
+unlike sets which only accept sets as the input.
 
 An example of dictionary view usage::
 

Doc/whatsnew/3.13.rst

@@ -183,6 +183,13 @@ doctest
   :attr:`doctest.TestResults.skipped` attributes.
   (Contributed by Victor Stinner in :gh:`108794`.)
 
+glob
+----
+
+* Add :func:`glob.translate` function that converts a path specification with
+  shell-style wildcards to a regular expression.
+  (Contributed by Barney Gale in :gh:`72904`.)
+
 io
 --
 
@@ -1164,6 +1171,10 @@ New Features
   :c:func:`PyErr_WriteUnraisable`, but allow to customize the warning mesage.
   (Contributed by Serhiy Storchaka in :gh:`108082`.)
 
+* Add :c:func:`PyList_Extend` and :c:func:`PyList_Clear` functions: similar to
+  Python ``list.extend()`` and ``list.clear()`` methods.
+  (Contributed by Victor Stinner in :gh:`111138`.)
+
 
 Porting to Python 3.13
 ----------------------

Include/cpython/compile.h

@@ -68,15 +68,3 @@ typedef struct {
 #define PY_INVALID_STACK_EFFECT INT_MAX
 PyAPI_FUNC(int) PyCompile_OpcodeStackEffect(int opcode, int oparg);
 PyAPI_FUNC(int) PyCompile_OpcodeStackEffectWithJump(int opcode, int oparg, int jump);
-
-PyAPI_FUNC(int) PyUnstable_OpcodeIsValid(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasArg(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasConst(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasName(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasJump(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasFree(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasLocal(int opcode);
-PyAPI_FUNC(int) PyUnstable_OpcodeHasExc(int opcode);
-
-PyAPI_FUNC(PyObject*) PyUnstable_GetUnaryIntrinsicName(int index);
-PyAPI_FUNC(PyObject*) PyUnstable_GetBinaryIntrinsicName(int index);

Include/cpython/listobject.h

@@ -44,3 +44,6 @@ PyList_SET_ITEM(PyObject *op, Py_ssize_t index, PyObject *value) {
 }
 #define PyList_SET_ITEM(op, index, value) \
     PyList_SET_ITEM(_PyObject_CAST(op), (index), _PyObject_CAST(value))
+
+PyAPI_FUNC(int) PyList_Extend(PyObject *self, PyObject *iterable);
+PyAPI_FUNC(int) PyList_Clear(PyObject *self);

Include/internal/pycore_compile.h

@@ -102,6 +102,20 @@ int _PyCompile_EnsureArrayLargeEnough(
 
 int _PyCompile_ConstCacheMergeOne(PyObject *const_cache, PyObject **obj);
 
+
+// Export for '_opcode' extention module
+PyAPI_FUNC(int) _PyCompile_OpcodeIsValid(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasArg(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasConst(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasName(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasJump(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasFree(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasLocal(int opcode);
+PyAPI_FUNC(int) _PyCompile_OpcodeHasExc(int opcode);
+
+PyAPI_FUNC(PyObject*) _PyCompile_GetUnaryIntrinsicName(int index);
+PyAPI_FUNC(PyObject*) _PyCompile_GetBinaryIntrinsicName(int index);
+
 /* Access compiler internals for unit testing */
 
 // Export for '_testinternalcapi' shared extension

Lib/fnmatch.py

@@ -78,6 +78,11 @@ def translate(pat):
     """
 
     STAR = object()
+    parts = _translate(pat, STAR, '.')
+    return _join_translated_parts(parts, STAR)
+
+
+def _translate(pat, STAR, QUESTION_MARK):
     res = []
     add = res.append
     i, n = 0, len(pat)
@@ -89,7 +94,7 @@ def translate(pat):
             if (not res) or res[-1] is not STAR:
                 add(STAR)
         elif c == '?':
-            add('.')
+            add(QUESTION_MARK)
         elif c == '[':
             j = i
             if j < n and pat[j] == '!':
@@ -146,9 +151,11 @@ def translate(pat):
         else:
             add(re.escape(c))
     assert i == n
+    return res
+
 
+def _join_translated_parts(inp, STAR):
     # Deal with STARs.
-    inp = res
     res = []
     add = res.append
     i, n = 0, len(inp)

Lib/glob.py

@@ -249,3 +249,63 @@ def escape(pathname):
 
 
 _dir_open_flags = os.O_RDONLY | getattr(os, 'O_DIRECTORY', 0)
+
+
+def translate(pat, *, recursive=False, include_hidden=False, seps=None):
+    """Translate a pathname with shell wildcards to a regular expression.
+
+    If `recursive` is true, the pattern segment '**' will match any number of
+    path segments; if '**' appears outside its own segment, ValueError will be
+    raised.
+
+    If `include_hidden` is true, wildcards can match path segments beginning
+    with a dot ('.').
+
+    If a sequence of separator characters is given to `seps`, they will be
+    used to split the pattern into segments and match path separators. If not
+    given, os.path.sep and os.path.altsep (where available) are used.
+    """
+    if not seps:
+        if os.path.altsep:
+            seps = (os.path.sep, os.path.altsep)
+        else:
+            seps = os.path.sep
+    escaped_seps = ''.join(map(re.escape, seps))
+    any_sep = f'[{escaped_seps}]' if len(seps) > 1 else escaped_seps
+    not_sep = f'[^{escaped_seps}]'
+    if include_hidden:
+        one_last_segment = f'{not_sep}+'
+        one_segment = f'{one_last_segment}{any_sep}'
+        any_segments = f'(?:.+{any_sep})?'
+        any_last_segments = '.*'
+    else:
+        one_last_segment = f'[^{escaped_seps}.]{not_sep}*'
+        one_segment = f'{one_last_segment}{any_sep}'
+        any_segments = f'(?:{one_segment})*'
+        any_last_segments = f'{any_segments}(?:{one_last_segment})?'
+
+    results = []
+    parts = re.split(any_sep, pat)
+    last_part_idx = len(parts) - 1
+    for idx, part in enumerate(parts):
+        if part == '*':
+            results.append(one_segment if idx < last_part_idx else one_last_segment)
+            continue
+        if recursive:
+            if part == '**':
+                if idx < last_part_idx:
+                    if parts[idx + 1] != '**':
+                        results.append(any_segments)
+                else:
+                    results.append(any_last_segments)
+                continue
+            elif '**' in part:
+                raise ValueError("Invalid pattern: '**' can only be an entire path component")
+        if part:
+            if not include_hidden and part[0] in '*?':
+                results.append(r'(?!\.)')
+            results.extend(fnmatch._translate(part, f'{not_sep}*', not_sep))
+        if idx < last_part_idx:
+            results.append(any_sep)
+    res = ''.join(results)
+    return fr'(?s:{res})\Z'