Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-74696: Pass root_dir to custom archivers which support it #94251

Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e5ebb8d
gh-74696: Pass root_dir to custom archivers which support it
serhiy-storchaka Jun 22, 2022
563b0f5
Update Doc/library/shutil.rst
serhiy-storchaka Jun 25, 2022
2190304
Merge branch 'main' into shutil-register_archive_format-supports_root…
merwok Sep 29, 2022
7dc8fb3
closes gh-97650: correct sphinx executable (gh-97651)
NoSuck Sep 29, 2022
679cf96
gh-96397: Document that attributes need not be identifiers (#96454)
jeff5 Sep 29, 2022
3c84af2
gh-96348: Deprecate the 3-arg signature of coroutine.throw and genera…
ofey404 Sep 30, 2022
8a0ad46
Use SyntaxError invalid range in tutorial introduction example (GH-93…
ehebert Sep 30, 2022
182755f
gh-97649: The Tools directory is no longer installed on Windows (GH-9…
zooba Sep 30, 2022
b1a9de0
gh-90989: Install Windows launcher per-user, and clarify some install…
zooba Sep 30, 2022
cc1e8e0
gh-94526: getpath_dirname() no longer encodes the path (#97645)
vstinner Sep 30, 2022
8fd0e86
bpo-35675: IDLE - separate config_key window and frame (#11427)
csabella Sep 30, 2022
d1d6b31
gh-87597: Document TimeoutExpired.stdout & .stderr types (#97685)
gpshead Sep 30, 2022
c6203f8
GH-96827: Don't touch closed loops from executor threads (#96837)
gvanrossum Sep 30, 2022
67851dc
GH-97592: Fix crash in C remove_done_callback due to evil code (#97660)
gvanrossum Sep 30, 2022
4c95e50
gh-90110: Update the c-analyzer Tool (gh-97695)
ericsnowcurrently Oct 1, 2022
fb39e7f
gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncance…
ambv Oct 1, 2022
0bb7e71
Fix capitalization of Unix in documentation (#96913)
hawkinsw Oct 1, 2022
5ffc011
gh-95588: Drop the safety claim from `ast.literal_eval` docs. (#95919)
gpshead Oct 2, 2022
e401b65
gh-97591: In `Exception.__setstate__()` acquire strong references bef…
ofey404 Oct 2, 2022
879e866
gh-95975: Move except/*/finally ref labels to more precise locations …
CAM-Gerlach Oct 2, 2022
299dd41
gh-97607: Fix content parsing in the impl-detail reST directive (#97652)
CAM-Gerlach Oct 2, 2022
551b707
[docs] Update logging cookbook with recipe for using a logger like an…
vsajip Oct 2, 2022
b37f2cd
Refactor tests.
serhiy-storchaka Oct 2, 2022
bf58b11
Merge branch 'main' into shutil-register_archive_format-supports_root…
serhiy-storchaka Oct 2, 2022
9980c8c
Apply suggestions from code review
serhiy-storchaka Oct 4, 2022
5160cb7
fix markup
merwok Oct 4, 2022
701f896
Update Doc/whatsnew/3.12.rst
serhiy-storchaka Oct 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion Doc/library/shutil.rst
Original file line number Diff line number Diff line change
Expand Up @@ -575,7 +575,8 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
.. note::

This function is not thread-safe when custom archivers registered
with :func:`register_archive_format` are used. In this case it
with :func:`register_archive_format` does not support the *root_dir*
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved
argument. In this case it
temporarily changes the current working directory of the process
to perform archiving.
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -614,12 +615,21 @@ provided. They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
Further arguments are passed as keyword arguments: *owner*, *group*,
*dry_run* and *logger* (as passed in :func:`make_archive`).

If *function* has the *supports_root_dir* attribute set to ``True``,
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved
the *root_dir* argument is passed as a keyword argument.
Otherwise the current working directory of the process is temporarily
changed before calling *function*.
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved
In this case :func:`make_archive` is not thread-safe.

If given, *extra_args* is a sequence of ``(name, value)`` pairs that will be
used as extra keywords arguments when the archiver callable is used.

*description* is used by :func:`get_archive_formats` which returns the
list of archivers. Defaults to an empty string.

.. versionchanged:: 3.12
Added support of functions supporting the *root_dir* argument.
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved


.. function:: unregister_archive_format(name)

Expand Down
9 changes: 9 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,15 @@ os
for a process with :func:`os.pidfd_open` in non-blocking mode.
(Contributed by Kumar Aditya in :gh:`93312`.)

shutil
------

* :func:`shutil.make_archive` now passes the *root_dir* argument to custom
archivers which support it.
In this case it no longer temporarily changes the current working directory
of the process to perform archiving.
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved
(Contributed by Serhiy Storchaka in :gh:`74696`.)


Optimizations
=============
Expand Down
20 changes: 11 additions & 9 deletions Lib/shutil.py
Original file line number Diff line number Diff line change
Expand Up @@ -1023,28 +1023,30 @@ def _make_zipfile(base_name, base_dir, verbose=0, dry_run=0,
zip_filename = os.path.abspath(zip_filename)
return zip_filename

_make_tarball.supports_root_dir = True
_make_zipfile.supports_root_dir = True

# Maps the name of the archive format to a tuple containing:
# * the archiving function
# * extra keyword arguments
# * description
# * does it support the root_dir argument?
_ARCHIVE_FORMATS = {
'tar': (_make_tarball, [('compress', None)],
"uncompressed tar file", True),
"uncompressed tar file"),
}

if _ZLIB_SUPPORTED:
_ARCHIVE_FORMATS['gztar'] = (_make_tarball, [('compress', 'gzip')],
"gzip'ed tar-file", True)
_ARCHIVE_FORMATS['zip'] = (_make_zipfile, [], "ZIP file", True)
"gzip'ed tar-file")
_ARCHIVE_FORMATS['zip'] = (_make_zipfile, [], "ZIP file")

if _BZ2_SUPPORTED:
_ARCHIVE_FORMATS['bztar'] = (_make_tarball, [('compress', 'bzip2')],
"bzip2'ed tar-file", True)
"bzip2'ed tar-file")

if _LZMA_SUPPORTED:
_ARCHIVE_FORMATS['xztar'] = (_make_tarball, [('compress', 'xz')],
"xz'ed tar-file", True)
"xz'ed tar-file")

def get_archive_formats():
"""Returns a list of supported formats for archiving and unarchiving.
Expand Down Expand Up @@ -1075,7 +1077,7 @@ def register_archive_format(name, function, extra_args=None, description=''):
if not isinstance(element, (tuple, list)) or len(element) !=2:
raise TypeError('extra_args elements are : (arg_name, value)')

_ARCHIVE_FORMATS[name] = (function, extra_args, description, False)
_ARCHIVE_FORMATS[name] = (function, extra_args, description)

def unregister_archive_format(name):
del _ARCHIVE_FORMATS[name]
Expand Down Expand Up @@ -1114,10 +1116,10 @@ def make_archive(base_name, format, root_dir=None, base_dir=None, verbose=0,
if base_dir is None:
base_dir = os.curdir

support_root_dir = format_info[3]
supports_root_dir = getattr(func, 'supports_root_dir', False)
save_cwd = None
if root_dir is not None:
if support_root_dir:
if supports_root_dir:
kwargs['root_dir'] = root_dir
else:
save_cwd = os.getcwd()
Expand Down
50 changes: 43 additions & 7 deletions Lib/test/test_shutil.py
Original file line number Diff line number Diff line change
Expand Up @@ -1572,26 +1572,62 @@ def test_tarfile_root_owner(self):

def test_make_archive_cwd(self):
current_dir = os.getcwd()
root_dir = self.mkdtemp()
def _breaks(*args, **kw):
def archiver(base_name, base_dir, **kw):
self.assertNotIn('root_dir', kw)
if root_dir is None:
serhiy-storchaka marked this conversation as resolved.
Show resolved Hide resolved
self.assertEqual(base_name, 'basename')
self.assertEqual(os.getcwd(), current_dir)
else:
self.assertEqual(base_name, os.path.join(current_dir, 'basename'))
self.assertEqual(os.getcwd(), root_dir)
raise RuntimeError()
dirs = []
def _chdir(path):
dirs.append(path)
orig_chdir(path)

register_archive_format('xxx', _breaks, [], 'xxx file')
register_archive_format('xxx', archiver, [], 'xxx file')
try:
root_dir = None
with support.swap_attr(os, 'chdir', _chdir) as orig_chdir:
try:
make_archive('xxx', 'xxx', root_dir=root_dir)
except Exception:
pass
with self.assertRaises(RuntimeError):
make_archive('basename', 'xxx')
self.assertEqual(os.getcwd(), current_dir)
self.assertEqual(dirs, [])

root_dir = self.mkdtemp()
with support.swap_attr(os, 'chdir', _chdir) as orig_chdir:
with self.assertRaises(RuntimeError):
make_archive('basename', 'xxx', root_dir=root_dir)
self.assertEqual(os.getcwd(), current_dir)
self.assertEqual(dirs, [root_dir, current_dir])
finally:
unregister_archive_format('xxx')

def test_make_archive_cwd_supports_root_dir(self):
current_dir = os.getcwd()
root_dir = self.mkdtemp()
def archiver(base_name, base_dir, **kw):
self.assertEqual(base_name, 'basename')
self.assertEqual(kw['root_dir'], root_dir)
self.assertEqual(os.getcwd(), current_dir)
raise RuntimeError()
archiver.supports_root_dir = True
dirs = []
def _chdir(path):
dirs.append(path)
orig_chdir(path)

register_archive_format('xxx', archiver, [], 'xxx file')
try:
with support.swap_attr(os, 'chdir', _chdir) as orig_chdir:
with self.assertRaises(RuntimeError):
make_archive('basename', 'xxx', root_dir=root_dir)
self.assertEqual(os.getcwd(), current_dir)
self.assertEqual(dirs, [])
finally:
unregister_archive_format('xxx')

def test_make_tarfile_in_curdir(self):
# Issue #21280
root_dir = self.mkdtemp()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
:func:`shutil.make_archive` now passes the *root_dir* argument to custom
archivers which support it.