System.IO.Directory.Move fails between two mounts on Linux

[hsorbo]

On Linux/netcore3 System.IO.Directory.Move throws System.IO.IOException: Invalid cross-device link when trying to move a folder between two mounts/disks in Linux.

System.IO.Directory.Move("/tmp/foo","/tmp/baz"); //works
System.IO.Directory.Move("/mnt/notsamedisk/foo","/tmp/baz"); //throws

Unhandled exception. System.IO.IOException: Invalid cross-device link
   at System.IO.FileSystem.MoveDirectory(String sourceFullPath, String destFullPath)
   at System.IO.Directory.Move(String sourceDirName, String destDirName)
   at tmp.Program.Main(String[] args) in /home/easysky/tmp/Program.cs:line 9

using System;

namespace tmp
{
    class Program
    {
        static void Main(string[] args)
        {
            System.IO.Directory.Move("/datadrive/tmp/foo","/tmp/foo");
        }
    }
}

uname -a Linux Test 5.0.0-1018-azure dotnet/corefx#19~18.04.1-Ubuntu SMP Wed Aug 21 05:13:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

dotnet --version
3.0.100

[vcsjones]

Based on the documentation, this might be expected, or at least known, behavior:

IOException
An attempt was made to move a directory to a different volume.

[vcsjones]

/cc @JeremyKuhne

[danmoseley]

dotnet/corefx#40611 related?

[JeremyKuhne]

It is something we didn't put effort in to support, primarily because the code was utilizing MoveFile. I'm not opposed to changing the behavior, it would just be significant effort (particularly from the test perspective).

[hsorbo]

@vcsjones is correct, this is documented and I didn't read the documentation thoroughly.
In that sense I think this issue can be closed.

On the other hand I think there are compelling arguments for a feature request for this.
Unix heavily uses these attached volumes example ramdisk /tmp, nfs (network) or encrypted home-folder, compressed filesystem for logs, etc.

Unix-programs are normally agnostic to which volume/mount a folder belongs to.
I think Users/admins expect to be able to set up that without breaking stuff.
In that sense, making a well-beaaving unix app, its hard to rely on Directory.Move

[svick]

Would it be less effort if the fallback behavior was to run /bin/mv?

[joshudson]

I'd personally have rathered Directory.Move and File.Move just failed when trying to span volumes even on Windows. Tracking down that bug left behind by another developer was not a fun day.

(The developer assumed it was atomic, and some other process managed to get in between and read half the file--hint: file locks don't always work on network volumes being accessed by multiple computers.)

[hsorbo]

@joshudson Aren’t you just back at writing your own Directory.Move then? (And in addition you’ll need mechanisms for checking volumes files belongs to, and in case of Unix where path/volumes are configurable, do that always)

[joshudson]

@hsorbo: Better a misconfigured program failing than losing data. Detecting cross-volume is easy. Don't pass MOVEFILE_COPY_ALLOWED to MoveFileEx and check for the error code. Writing an algorithm that can handle it, now that's harder.

[hsorbo]

@joshudson getting rid of it on windows is probably not in the cards as its breaking change (maybe worth a new issue). MoveFileEx is windows API, not unix/posix (and i think file locking is uncommon thing on unixes). Making consumers implement Directory.Move because of the problem has no guarantees for data loss in concurrent programming is, in my book, not a very strong argument for not supporting this scenario.

[MichaelSimons]

I came upon a scenario that encountered this today involving Docker. Anytime Directory.Move gets invoked on a directory created in a previous Docker layer, you will hit this issue. Docker utilizes overlay mounts to union the layers of a Docker image. Modifying files in a previous Docker layer is something you typically try to avoid because it causes image bloat. That being said there are times this has to be done.

The end user scenario that hit this is dotnet tool update. See dotnet/sdk#3838 for details.

[solrevdev]

FYI this also happens on macOS.

summary

Trying to move files on a folder shared to everyone on a Microsoft Windows Server 2012R2 to a local macos folder.

I should add this happens on System.IO.Directory.Move

from "/Volumes/WindowsShareName/foldername
to "/Users/solrevdev/TempFileLocalToMacOS"

dotnet info

dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.100
 Commit:    cd82f021f4

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.14
 OS Platform: Darwin
 RID:         osx.10.14-x64
 Base Path:   /usr/local/share/dotnet/sdk/3.1.100/

Host (useful for support):
  Version: 3.1.0
  Commit:  65f04fb6db

.NET Core SDKs installed:
  3.1.100 [/usr/local/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.13 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

environment

screenfetch -n
 solrevdev@macmini
 OS: 64bit Mac OS X 10.14.6 18G95
 Kernel: x86_64 Darwin 18.7.0
 Uptime: 23d 19h 10m
 Packages: 415
 Shell: zsh 5.7.1
 Resolution: 2560x1440
 DE: Aqua
 WM: Quartz Compositor
 WM Theme: Blue (Dark)
 Disk: 404G / 525G (78%)
 CPU: Intel Core i5-4278U @ 2.60GHz
 GPU: Intel Iris
 RAM: 8856MiB / 16384MiB