Skip to content

Commit 514d5d7

Browse files
committed
bpo-37834: Clarify reparse point handling on Windows.
* ntpath.realpath() and nt.stat() will traverse all supported reparse points (previously was mixed) * nt.lstat() will let the OS traverse reparse points that are not name surrogates (previously would not traverse any reparse point) * nt.[l]stat() will only set S_IFLNK for symlinks (previous behaviour) * nt.readlink() will read destinations for symlinks and junction points only bpo-1311: os.path.exists('nul') now returns True on Windows * nt.stat('nul').st_mode is now S_IFCHR (previously was an error)
1 parent 75e0649 commit 514d5d7

File tree

17 files changed

+471
-232
lines changed

17 files changed

+471
-232
lines changed

Doc/library/os.rst

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1858,6 +1858,9 @@ features:
18581858
.. versionchanged:: 3.6
18591859
Accepts a :term:`path-like object` for *src* and *dst*.
18601860

1861+
.. versionchanged:: 3.8
1862+
On Windows, now opens reparse points that represent another file
1863+
(name surrogates).
18611864

18621865
.. function:: mkdir(path, mode=0o777, *, dir_fd=None)
18631866

@@ -2053,6 +2056,11 @@ features:
20532056
.. versionchanged:: 3.8
20542057
Accepts a :term:`path-like object` and a bytes object on Windows.
20552058

2059+
.. versionchanged:: 3.8
2060+
Added support for directory junctions, and changed to return the
2061+
substitution path (which typically includes ``\\?\`` prefix) rather than
2062+
the optional "print name" field that was previously returned.
2063+
20562064
.. function:: remove(path, *, dir_fd=None)
20572065

20582066
Remove (delete) the file *path*. If *path* is a directory, an
@@ -2358,6 +2366,13 @@ features:
23582366
This method can raise :exc:`OSError`, such as :exc:`PermissionError`,
23592367
but :exc:`FileNotFoundError` is caught and not raised.
23602368

2369+
.. versionchanged:: 3.8
2370+
On Windows, now returns ``True`` for directory junctions as well as
2371+
symlinks. To determine whether the entry is actually a symlink to a
2372+
directory or a directory junction, compare
2373+
``entry.stat(follow_symlinks=False).st_reparse_tag`` against
2374+
``stat.IO_REPARSE_TAG_SYMLINK`` or ``stat.IO_REPARSE_TAG_MOUNT_POINT``.
2375+
23612376
.. method:: stat(\*, follow_symlinks=True)
23622377

23632378
Return a :class:`stat_result` object for this entry. This method
@@ -2403,6 +2418,16 @@ features:
24032418
This function can support :ref:`specifying a file descriptor <path_fd>` and
24042419
:ref:`not following symlinks <follow_symlinks>`.
24052420

2421+
On Windows, passing ``follow_symlinks=False`` will disable following all
2422+
types of reparse points, including directory junctions. Otherwise, if the
2423+
operating system is unable to follow a reparse point (for example, when it
2424+
is a custom reparse point type with no filesystem support), the stat result
2425+
for the original link is returned as if ``follow_symlinks=False`` had been
2426+
specified. To obtain stat results for the final path in this case, use the
2427+
:func:`os.path.realpath` function to resolve the path name as far as
2428+
possible and call :func:`lstat` on the result. This does not apply to
2429+
dangling symlinks or junction points, which will raise the usual exceptions.
2430+
24062431
.. index:: module: stat
24072432

24082433
Example::
@@ -2427,6 +2452,14 @@ features:
24272452
.. versionchanged:: 3.6
24282453
Accepts a :term:`path-like object`.
24292454

2455+
.. versionchanged:: 3.8
2456+
On Windows, all reparse points that can be resolved by the operating
2457+
system are now followed, and passing ``follow_symlinks=False``
2458+
disables following all name surrogate reparse points. If the operating
2459+
system reaches a reparse point that it is not able to follow, *stat* now
2460+
returns the information for the original path as if
2461+
``follow_symlinks=False`` had been specified instead of raising an error.
2462+
24302463

24312464
.. class:: stat_result
24322465

@@ -2578,7 +2611,7 @@ features:
25782611

25792612
File type.
25802613

2581-
On Windows systems, the following attribute is also available:
2614+
On Windows systems, the following attributes are also available:
25822615

25832616
.. attribute:: st_file_attributes
25842617

@@ -2587,6 +2620,12 @@ features:
25872620
:c:func:`GetFileInformationByHandle`. See the ``FILE_ATTRIBUTE_*``
25882621
constants in the :mod:`stat` module.
25892622

2623+
.. attribute:: st_reparse_tag
2624+
2625+
When :attr:`st_file_attributes` has the ``FILE_ATTRIBUTE_REPARSE_POINT``
2626+
set, this field contains the tag identifying the type of reparse point.
2627+
See the ``IO_REPARSE_TAG_*`` constants in the :mod:`stat` module.
2628+
25902629
The standard module :mod:`stat` defines functions and constants that are
25912630
useful for extracting information from a :c:type:`stat` structure. (On
25922631
Windows, some items are filled with dummy values.)
@@ -2614,6 +2653,13 @@ features:
26142653
.. versionadded:: 3.7
26152654
Added the :attr:`st_fstype` member to Solaris/derivatives.
26162655

2656+
.. versionadded:: 3.8
2657+
Added the :attr:`st_reparse_tag` member on Windows.
2658+
2659+
.. versionchanged:: 3.8
2660+
On Windows, the :attr:`st_mode` member now identifies directory
2661+
junctions as links instead of directories.
2662+
26172663
.. function:: statvfs(path)
26182664

26192665
Perform a :c:func:`statvfs` system call on the given path. The return value is

Doc/library/pathlib.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -807,6 +807,13 @@ call fails (for example because the path doesn't exist).
807807
``False`` is also returned if the path doesn't exist; other errors (such
808808
as permission errors) are propagated.
809809

810+
.. versionchanged:: 3.8
811+
On Windows, now returns ``True`` for directory junctions as well as
812+
symlinks. To determine whether the path is actually a symlink to a
813+
directory or a directory junction, compare ``Path.lstat().st_reparse_tag``
814+
against ``stat.IO_REPARSE_TAG_SYMLINK`` or
815+
``stat.IO_REPARSE_TAG_MOUNT_POINT``.
816+
810817

811818
.. method:: Path.is_socket()
812819

Doc/library/shutil.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -304,6 +304,10 @@ Directory and files operations
304304
Added a symlink attack resistant version that is used automatically
305305
if platform supports fd-based functions.
306306

307+
.. versionchanged:: 3.8
308+
On Windows, will no longer delete the contents of a directory junction
309+
before removing the junction.
310+
307311
.. attribute:: rmtree.avoids_symlink_attacks
308312

309313
Indicates whether the current platform and implementation provides a

Doc/library/stat.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -425,3 +425,13 @@ for more detail on the meaning of these constants.
425425
FILE_ATTRIBUTE_VIRTUAL
426426

427427
.. versionadded:: 3.5
428+
429+
On Windows, the following constants are available for comparing against the
430+
``st_reparse_tag`` member returned by :func:`os.lstat`. These are well-known
431+
constants, but are not an exhaustive list.
432+
433+
.. data:: IO_REPARSE_TAG_SYMLINK
434+
IO_REPARSE_TAG_MOUNT_POINT
435+
IO_REPARSE_TAG_APPEXECLINK
436+
437+
.. versionadded:: 3.8

Doc/whatsnew/3.8.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -808,6 +808,19 @@ A new :func:`os.memfd_create` function was added to wrap the
808808
``memfd_create()`` syscall.
809809
(Contributed by Zackery Spytz and Christian Heimes in :issue:`26836`.)
810810

811+
On Windows, much of the manual logic for handling reparse points (symlinks)
812+
has been delegated to the operating system. Specifically, :func:`os.stat`
813+
will now traverse anything supported by the operating system, while
814+
:func:`os.lstat` will not traverse anything. The stat result now includes
815+
:attr:`stat_result.st_reparse_tag` for reparse points, and :func:`os.readlink`
816+
is now able to read directory junctions.
817+
818+
Directory results from :func:`os.scandir` on Windows will now return true for
819+
both :meth:`os.DirEntry.is_symlink` and :meth:`os.DirEntry.is_dir` when the
820+
entry is a directory junction (this would already happen for symbolic links
821+
to directories). To distinguish a directory junction from a symlink, use
822+
``stat(follow_symlinks=False).st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT``.
823+
811824

812825
os.path
813826
-------
@@ -824,6 +837,9 @@ characters or bytes unrepresentable at the OS level.
824837
environment variable and does not use :envvar:`HOME`, which is not normally set
825838
for regular user accounts.
826839

840+
:func:`~os.path.isdir` on Windows no longer returns true for a link to a
841+
non-existent directory.
842+
827843
:func:`~os.path.realpath` on Windows now resolves reparse points, including
828844
symlinks and directory junctions.
829845

@@ -912,6 +928,9 @@ format for new archives to improve portability and standards conformance,
912928
inherited from the corresponding change to the :mod:`tarfile` module.
913929
(Contributed by C.A.M. Gerlach in :issue:`30661`.)
914930

931+
:func:`shutil.rmtree` on Windows now removes directory junctions without
932+
removing their contents first.
933+
915934

916935
ssl
917936
---

Include/fileutils.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ struct _Py_stat_struct {
8484
time_t st_ctime;
8585
int st_ctime_nsec;
8686
unsigned long st_file_attributes;
87+
unsigned long st_reparse_tag;
8788
};
8889
#else
8990
# define _Py_stat_struct stat

Lib/shutil.py

Lines changed: 41 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -452,7 +452,14 @@ def _copytree(entries, src, dst, symlinks, ignore, copy_function,
452452
dstname = os.path.join(dst, srcentry.name)
453453
srcobj = srcentry if use_srcentry else srcname
454454
try:
455-
if srcentry.is_symlink():
455+
is_symlink = srcentry.is_symlink()
456+
if is_symlink and os.name == 'nt':
457+
# Special check for directory junctions, which appear as
458+
# symlinks but we want to recurse.
459+
lstat = srcentry.stat(follow_symlinks=False)
460+
if lstat.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT:
461+
is_symlink = False
462+
if is_symlink:
456463
linkto = os.readlink(srcname)
457464
if symlinks:
458465
# We can't just leave it to `copy_function` because legacy
@@ -537,6 +544,37 @@ def copytree(src, dst, symlinks=False, ignore=None, copy_function=copy2,
537544
ignore_dangling_symlinks=ignore_dangling_symlinks,
538545
dirs_exist_ok=dirs_exist_ok)
539546

547+
if hasattr(stat, 'FILE_ATTRIBUTE_REPARSE_POINT'):
548+
# Special handling for directory junctions to make them behave like
549+
# symlinks for shutil.rmtree, since in general they do not appear as
550+
# regular links.
551+
def _rmtree_isdir(entry):
552+
try:
553+
st = entry.stat(follow_symlinks=False)
554+
return (stat.S_ISDIR(st.st_mode) and not
555+
(st.st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT
556+
and st.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT))
557+
except OSError:
558+
return False
559+
560+
def _rmtree_islink(path):
561+
try:
562+
st = os.lstat(path)
563+
return (stat.S_ISLNK(st.st_mode) or
564+
(st.st_file_attributes & stat.FILE_ATTRIBUTE_REPARSE_POINT
565+
and st.st_reparse_tag == stat.IO_REPARSE_TAG_MOUNT_POINT))
566+
except OSError:
567+
return False
568+
else:
569+
def _rmtree_isdir(entry):
570+
try:
571+
return entry.is_dir(follow_symlinks=False)
572+
except OSError:
573+
return False
574+
575+
def _rmtree_islink(path):
576+
return os.path.islink(path)
577+
540578
# version vulnerable to race conditions
541579
def _rmtree_unsafe(path, onerror):
542580
try:
@@ -547,11 +585,7 @@ def _rmtree_unsafe(path, onerror):
547585
entries = []
548586
for entry in entries:
549587
fullname = entry.path
550-
try:
551-
is_dir = entry.is_dir(follow_symlinks=False)
552-
except OSError:
553-
is_dir = False
554-
if is_dir:
588+
if _rmtree_isdir(entry):
555589
try:
556590
if entry.is_symlink():
557591
# This can only happen if someone replaces
@@ -681,7 +715,7 @@ def onerror(*args):
681715
os.close(fd)
682716
else:
683717
try:
684-
if os.path.islink(path):
718+
if _rmtree_islink(path):
685719
# symlinks to directories are forbidden, see bug #1669
686720
raise OSError("Cannot call rmtree on a symbolic link")
687721
except OSError:

0 commit comments

Comments
 (0)