Support moving across filesystems in pathlib.Path, as shutil.move() does

[LaurentMazuel]

BPO	29805
Nosy	@brettcannon, @pfmoore, @ericvsmith, @tjguk, @zware, @eryksun, @zooba

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-03-13.21:03:44.694>
labels = ['3.8', 'type-feature', 'library', '3.9', '3.10']
title = 'Support moving across filesystems in pathlib.Path, as shutil.move() does'
updated_at = <Date 2021-03-15.21:26:30.922>
user = 'https://bugs.python.org/LaurentMazuel'

bugs.python.org fields:

activity = <Date 2021-03-15.21:26:30.922>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-03-13.21:03:44.694>
creator = 'Laurent.Mazuel'
dependencies = []
files = []
hgrepos = []
issue_num = 29805
keywords = []
message_count = 6.0
messages = ['289549', '289552', '289559', '289630', '289687', '289688']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'paul.moore', 'eric.smith', 'tim.golden', 'Laurent.Mazuel', 'zach.ware', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue29805'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• GH-73991: Add pathlib.Path.move that can handle rename across FS #30650
• GH-73991: Add pathlib.Path.copy() #119058
• GH-73991: Add pathlib.Path.rmtree() #119060
• [WIP] GH-73991: Add os.copy() and friends #119079
• GH-73991: Split "Directory and file operations" section in shutil docs #119159
• GH-73991: Use same signature for shutil._rmtree_[un]safe(). #120517
• GH-73991: Add follow_symlinks argument to pathlib.Path.copy() #120519
• GH-73991: pathlib ABC tests: add DummyPath.unlink() and rmdir() #120715
• GH-73991: Add pathlib.Path.copytree() #120718
• GH-73991: Support preserving metadata in pathlib.Path.copy() #120806
• GH-73991: Support copying directory symlinks on older Windows #120807
• GH-73991: Support preserving metadata in pathlib.Path.copytree() #121438
• GH-73991: Fix "Operation not supported" on Fedora buildbot. #121444
• GH-73991: Add pathlib.Path.move() #122073
• GH-73991: Rework pathlib.Path.rmtree() into delete() #122368
• GH-73991: Rework pathlib.Path.copytree() into copy() #122369
• GH-73991: Disallow copying directory into itself via pathlib.Path.copy() #122924
• GH-73991: Prune pathlib.Path.delete() arguments #123158
• GH-73991: Add pathlib.Path.copy_into() and move_into() #123314
• GH-73991: Make pathlib.Path.delete() private. #123315
• GH-73991: Prune pathlib.Path.copy() and copy_into() arguments #123337

[LaurentMazuel]

Trying to use Pathlib and Path.replace on Windows if drive are different leads to an issue:

  File "D:\\myscript.py", line 184, in update
    client_generated_path.replace(destination_folder)
  File "c:\\program files (x86)\\python35-32\\Lib\\pathlib.py", line 1273, in replace
    self.\_accessor.replace(self, target)
  File "c:\\program files (x86)\\python35-32\\Lib\\pathlib.py", line 377, in wrapped
    return strfunc(str(pathobjA), str(pathobjB), \*args)
OSError: [WinError 17] The system cannot move the file to a different disk drive: 'C:\\\\MyFolder' -\> 'D:\\\\MyFolderNewName'

This is a known situation of os.rename, and workaround I found is to use shutil or to copy/delete manually in two steps (e.g. http://stackoverflow.com/questions/21116510/python-oserror-winerror-17-the-system-cannot-move-the-file-to-a-different-d)

When using Pathlib, it's not that easy to workaround using shutil (even if thanks to Brett Cannon now shutil accepts Path in Py3.6, not everybody has Py3.6). At least this should be documented with a recommendation for that situation. I love Pathlib and it's too bad my code becomes complicated when it was so simple :(

[LaurentMazuel]

Just to confirm, I was able to workaround it with Py3.6:

    # client_generated_path.replace(destination_folder)
    shutil.move(client_generated_path, destination_folder)

It is reasonable to think about adding a similar feature to pathlib if it doesn't support it and just special-case the drive-to-drive scenario for Windows

[eryksun]

Moving a file across volumes isn't atomic. Getting an exception in this case can be useful. There could be a "strict" keyword-only parameter that defaults to False. If it's true, then replace() won't try to move the file.

[ericvsmith]

I agree this needs to be different from replace(), due to not being atomic. That makes it an enhancement, so I'm removing 3.5.

I'm +1 on Path supporting something like shutil.move().

[brettcannon]

I also support the idea of getting something like shutil.move() into pathlib.

[brettcannon]

I should also mention that rename() (https://docs.python.org/3/library/pathlib.html#pathlib.Path.rename) and replace() (https://docs.python.org/3/library/pathlib.html#pathlib.Path.replace) already do exist, so it might be best to add a keyword-only flag to one of those for this use-case.