Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: if rename system call fails, try ioctl(FICLONE) #41487

Open
codeaholics opened this issue Sep 18, 2020 · 6 comments
Open

os: if rename system call fails, try ioctl(FICLONE) #41487

codeaholics opened this issue Sep 18, 2020 · 6 comments
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux
Milestone

Comments

@codeaholics
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.15.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/danny/Library/Caches/go-build"
GOENV="/Users/danny/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/danny/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/danny/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15.2/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15.2/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/danny/workspace/media-sort/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/36/kj75t85j1m57cg047cr5dh4m0000gn/T/go-build628346493=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I am cross-compiling on Mac for Linux

Linux execution env: Linux nas 4.4.190.x86_64.1 #1 SMP Mon Oct 28 01:55:46 UTC 2019 x86_64 GNU/Linux

Build command: GOOS=linux GOARCH=amd64 go build -v

func move(src, dst string) error {
	err := os.Rename(src, dst)
	// cross device move
	if err != nil && strings.Contains(err.Error(), "cross-device") {
		if err := copy(src, dst); err != nil {
			return err
		}
		if err := os.Remove(src); err != nil {
			return err
		}
	}
	return nil
}

What did you expect to see?

os.Rename() to succeed because the source and target are on the same filesystem.

What did you see instead?

os.Rename() returns "invalid cross-device link" and the above code falls back to copy-and-delete. If I do an mv on the command line, it completes immediately - even for multi-gigabyte files.

@ianlancetaylor
Copy link
Contributor

os.Rename is a fairly thin wrapper around syscall.Rename, which just calls the rename system call. If possible, please run the final GNU/Linux executable under strace -f and see what the rename system call returns.

@ianlancetaylor ianlancetaylor changed the title os.Rename() returns "invalid cross-device link" even though source and dest are on same device os: Rename returns "invalid cross-device link" even though source and dest are on same device Sep 18, 2020
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 18, 2020
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Sep 18, 2020
@ianlancetaylor ianlancetaylor added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Sep 18, 2020
@codeaholics
Copy link
Author

Thanks for your quick response @ianlancetaylor. To test, I made an empty file, f, in /data/Videos on my NAS, and mv-ed it to /data/Music. Here's the strace:

execve("/bin/mv", ["mv", "f", "/data/Music"], [/* 17 vars */]) = 0
brk(0)                                  = 0x16eb000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d8000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=28370, ...}) = 0
mmap(NULL, 28370, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe1406d1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20c\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=142728, ...}) = 0
mmap(NULL, 2246896, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe140295000
mprotect(0x7fe1402b6000, 2097152, PROT_NONE) = 0
mmap(0x7fe1404b6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21000) = 0x7fe1404b6000
mmap(0x7fe1404b8000, 6384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe1404b8000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libacl.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\37\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=35288, ...}) = 0
mmap(NULL, 2130592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe14008c000
mprotect(0x7fe140094000, 2093056, PROT_NONE) = 0
mmap(0x7fe140293000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0x7fe140293000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\23\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=18640, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d0000
mmap(NULL, 2113912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13fe87000
mprotect(0x7fe13fe8b000, 2093056, PROT_NONE) = 0
mmap(0x7fe14008a000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7fe14008a000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\34\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1738176, ...}) = 0
mmap(NULL, 3844640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13fadc000
mprotect(0x7fe13fc7d000, 2097152, PROT_NONE) = 0
mmap(0x7fe13fe7d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a1000) = 0x7fe13fe7d000
mmap(0x7fe13fe83000, 14880, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe13fe83000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\27\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=448440, ...}) = 0
mmap(NULL, 2543976, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f86e000
mprotect(0x7fe13f8da000, 2097152, PROT_NONE) = 0
mmap(0x7fe13fada000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6c000) = 0x7fe13fada000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14664, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406cf000
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f66a000
mprotect(0x7fe13f66d000, 2093056, PROT_NONE) = 0
mmap(0x7fe13f86c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fe13f86c000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320n\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=137384, ...}) = 0
mmap(NULL, 2213008, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fe13f44d000
mprotect(0x7fe13f465000, 2093056, PROT_NONE) = 0
mmap(0x7fe13f664000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fe13f664000
mmap(0x7fe13f666000, 13456, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fe13f666000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406ce000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406cc000
arch_prctl(ARCH_SET_FS, 0x7fe1406cc800) = 0
mprotect(0x7fe13fe7d000, 16384, PROT_READ) = 0
mprotect(0x7fe13f664000, 4096, PROT_READ) = 0
mprotect(0x7fe13f86c000, 4096, PROT_READ) = 0
mprotect(0x7fe13fada000, 4096, PROT_READ) = 0
mprotect(0x7fe14008a000, 4096, PROT_READ) = 0
mprotect(0x7fe140293000, 4096, PROT_READ) = 0
mprotect(0x7fe1404b6000, 4096, PROT_READ) = 0
mprotect(0x61e000, 4096, PROT_READ)     = 0
mprotect(0x7fe1406da000, 4096, PROT_READ) = 0
munmap(0x7fe1406d1000, 28370)           = 0
set_tid_address(0x7fe1406ccad0)         = 23496
set_robust_list(0x7fe1406ccae0, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7fe13f4539b0, [], SA_RESTORER|SA_SIGINFO, 0x7fe13f45c890}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fe13f453a40, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fe13f45c890}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", 0x7ffc57d353f0) = -1 ENOENT (No such file or directory)
statfs("/selinux", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=965552, f_bfree=722015, f_bavail=665542, f_files=1048576, f_ffree=1032627, f_fsid={-1827525020, -625743731}, f_namelen=255, f_frsize=4096}) = 0
brk(0)                                  = 0x16eb000
brk(0x170c000)                          = 0x170c000
open("/proc/filesystems", O_RDONLY)     = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe1406d7000
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tt"..., 1024) = 370
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0x7fe1406d7000, 4096)            = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=6839968, ...}) = 0
mmap(NULL, 6839968, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fe13edc7000
close(3)                                = 0
geteuid()                               = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
stat("/data/Music", {st_mode=S_IFDIR|0777, st_size=38, ...}) = 0
lstat("f", {st_mode=S_IFREG|0666, st_size=0, ...}) = 0
lstat("/data/Music/f", 0x7ffc57d35060)  = -1 ENOENT (No such file or directory)
rename("f", "/data/Music/f")            = -1 EXDEV (Invalid cross-device link)
unlink("/data/Music/f")                 = -1 ENOENT (No such file or directory)
open("f", O_RDONLY|O_NOFOLLOW)          = 3
fstat(3, {st_mode=S_IFREG|0666, st_size=0, ...}) = 0
open("/data/Music/f", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
ioctl(4, BTRFS_IOC_CLONE, 0x3)          = 0
utimensat(4, NULL, {{1600461257, 619450219}, {1600461257, 619450219}}, 0) = 0
flistxattr(3, NULL, 0)                  = 24
flistxattr(3, "system.posix_acl_access\0", 24) = 24
open("/etc/xattr.conf", O_RDONLY)       = -1 ENOENT (No such file or directory)
fgetxattr(3, "system.posix_acl_access", 0x0, 0) = 68
fgetxattr(3, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68) = 68
fsetxattr(4, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68, 0) = 0
fgetxattr(3, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 132) = 68
fsetxattr(4, "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x07\x00b\x00\x00\x00\x02\x00\x07\x00c\x00\x00\x00\x04\x00\x07\x00\xff\xff\xff\xff\x08\x00\x07\x00b\x00\x00\x00\x08\x00\x07\x00c\x00\x00\x00\x10\x00\x06\x00\xff\xff\xff\xff \x00\x06\x00\xff\xff\xff\xff", 68, 0) = 0
close(4)                                = 0
close(3)                                = 0
lstat("/", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
newfstatat(AT_FDCWD, "f", {st_mode=S_IFREG|0666, st_size=0, ...}, AT_SYMLINK_NOFOLLOW) = 0
unlinkat(AT_FDCWD, "f", 0)              = 0
lseek(0, 0, SEEK_CUR)                   = -1 ESPIPE (Illegal seek)
close(0)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

I'm waaaay out of my depth here, but I'm guessing these lines are the important ones:

rename("f", "/data/Music/f")            = -1 EXDEV (Invalid cross-device link)
[... followed shortly by ...]
ioctl(4, BTRFS_IOC_CLONE, 0x3)          = 0

It looks like the rename call fails, and mv falls back to using a BTRFS-specific alternative.

@ianlancetaylor
Copy link
Contributor

Interesting. I was not aware of that. I'll retitle this issue.

@ianlancetaylor ianlancetaylor changed the title os: Rename returns "invalid cross-device link" even though source and dest are on same device os: if rename system call fails, try ioctl(FICLONE) Sep 18, 2020
@ianlancetaylor ianlancetaylor added help wanted and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Sep 18, 2020
@ianlancetaylor ianlancetaylor modified the milestones: Backlog, Unplanned Sep 18, 2020
@codeaholics
Copy link
Author

I've tried to follow the source for mv, and... well... I'm not a C programmer! 🤣 But ultimately, it ends up here: https://github.com/coreutils/coreutils/blob/master/src/copy.c#L405-L418, which, AFAICT, comes from here: https://github.com/coreutils/coreutils/blob/master/src/copy.c#L1459.

Ultimately, it appears to all be controlled by the defaulting of the following two flags in mv.c:
https://github.com/coreutils/coreutils/blob/master/src/mv.c#L113
https://github.com/coreutils/coreutils/blob/master/src/mv.c#L130

@codeaholics
Copy link
Author

It's interesting that BTRFS reports those two directories as being on different filesystems, but is then happy to do a CoW!

@thanm
Copy link
Contributor

thanm commented Sep 18, 2020

bflad added a commit to speakeasy-api/speakeasy that referenced this issue Sep 29, 2024
Reference: https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and
Reference: golang/go#41487

When the CLI is running on Linux, the Linux system has different mounts for home and temporary directories, and the `update` command is called (or the same functionality automatically invoked with other commands), then the CLI can return an `invalid cross-device link` error without updating the executable when using the Go standard library `os.Rename()` function. This change introduces fallback logic to catch that error and manually overwrite the executable while accounting for `ETXTBSY` errors on the running executable.

Reproduction (macOS host):

```console
$ docker run -i -t --privileged --rm ubuntu:24.04
```

Reproduction (Docker container, manual install to not need `sudo`):

```console
$ apt-get update && apt-get install -y wget unzip
$ cd /tmp
$ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip
$ unzip speakeasy_linux_arm64.zip
$ chmod a+x speakeasy
$ cp speakeasy /usr/bin/speakeasy
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link
```

Verification (macOS host, these changes):

```console
$ GOOS=linux go build .
$ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04
```

Verification (Docker container):

```console
$ cp /tmp/speakeasy /usr/bin/speakeasy
$ speakeasy --version
speakeasy version 0.0.1
linux_amd64
$ apt-get update && apt-get install -y ca-certificates
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
Updated to version v1.404.5
$ speakeasy --version
speakeasy version 1.404.5
linux_amd64
```
bflad added a commit to speakeasy-api/speakeasy that referenced this issue Sep 29, 2024
Reference: https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and
Reference: golang/go#41487

When the CLI is running on Linux, the Linux system has different mounts for home and temporary directories, and the `update` command is called (or the same functionality automatically invoked with other commands), then the CLI can return an `invalid cross-device link` error without updating the executable when using the Go standard library `os.Rename()` function. This change introduces fallback logic to catch that error and manually overwrite the executable while accounting for `ETXTBSY` errors on the running executable.

Reproduction (macOS host):

```console
$ docker run -i -t --privileged --rm ubuntu:24.04
```

Reproduction (Docker container, manual install to not need `sudo`):

```console
$ apt-get update && apt-get install -y wget unzip
$ cd /tmp
$ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip
$ unzip speakeasy_linux_arm64.zip
$ chmod a+x speakeasy
$ cp speakeasy /usr/bin/speakeasy
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link
```

Verification (macOS host, these changes):

```console
$ GOOS=linux go build .
$ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04
```

Verification (Docker container):

```console
$ cp /tmp/speakeasy /usr/bin/speakeasy
$ speakeasy --version
speakeasy version 0.0.1
linux_amd64
$ apt-get update && apt-get install -y ca-certificates
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
Updated to version v1.404.5
$ speakeasy --version
speakeasy version 1.404.5
linux_amd64
```
bflad added a commit to speakeasy-api/speakeasy that referenced this issue Sep 30, 2024
Reference:
https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and
Reference: golang/go#41487

When the CLI is running on Linux, the Linux system has different mounts
for home and temporary directories, and the `update` command is called
(or the same functionality automatically invoked with other commands),
then the CLI can return an `invalid cross-device link` error without
updating the executable when using the Go standard library `os.Rename()`
function. This change introduces fallback logic to catch that error and
manually overwrite the executable while accounting for `ETXTBSY` errors
on the running executable.

Reproduction (macOS host):

```console
$ docker run -i -t --privileged --rm ubuntu:24.04
```

Reproduction (Docker container, manual install to not need `sudo`):

```console
$ apt-get update && apt-get install -y wget unzip
$ cd /tmp
$ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip
$ unzip speakeasy_linux_arm64.zip
$ chmod a+x speakeasy
$ cp speakeasy /usr/bin/speakeasy
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link
```

Verification (macOS host, these changes):

```console
$ GOOS=linux go build .
$ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04
```

Verification (Docker container):

```console
$ cp /tmp/speakeasy /usr/bin/speakeasy
$ speakeasy --version
speakeasy version 0.0.1
linux_amd64
$ apt-get update && apt-get install -y ca-certificates
$ echo "none    /mnt/ramfs    ramfs    noauto,user,size=1024M,mode=1777    0    0" >> /etc/fstab
$ mkdir /mnt/ramfs
$ mount /mnt/ramfs
$ export TMPDIR=/mnt/ramfs
$ speakeasy update
Updated to version v1.404.5
$ speakeasy --version
speakeasy version 1.404.5
linux_amd64
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux
Projects
None yet
Development

No branches or pull requests

3 participants