-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
os: if rename system call fails, try ioctl(FICLONE) #41487
Comments
|
Thanks for your quick response @ianlancetaylor. To test, I made an empty file,
I'm waaaay out of my depth here, but I'm guessing these lines are the important ones:
It looks like the |
Interesting. I was not aware of that. I'll retitle this issue. |
I've tried to follow the source for Ultimately, it appears to all be controlled by the defaulting of the following two flags in |
It's interesting that BTRFS reports those two directories as being on different filesystems, but is then happy to do a CoW! |
Reference: https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and Reference: golang/go#41487 When the CLI is running on Linux, the Linux system has different mounts for home and temporary directories, and the `update` command is called (or the same functionality automatically invoked with other commands), then the CLI can return an `invalid cross-device link` error without updating the executable when using the Go standard library `os.Rename()` function. This change introduces fallback logic to catch that error and manually overwrite the executable while accounting for `ETXTBSY` errors on the running executable. Reproduction (macOS host): ```console $ docker run -i -t --privileged --rm ubuntu:24.04 ``` Reproduction (Docker container, manual install to not need `sudo`): ```console $ apt-get update && apt-get install -y wget unzip $ cd /tmp $ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip $ unzip speakeasy_linux_arm64.zip $ chmod a+x speakeasy $ cp speakeasy /usr/bin/speakeasy $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link ``` Verification (macOS host, these changes): ```console $ GOOS=linux go build . $ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04 ``` Verification (Docker container): ```console $ cp /tmp/speakeasy /usr/bin/speakeasy $ speakeasy --version speakeasy version 0.0.1 linux_amd64 $ apt-get update && apt-get install -y ca-certificates $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update Updated to version v1.404.5 $ speakeasy --version speakeasy version 1.404.5 linux_amd64 ```
Reference: https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and Reference: golang/go#41487 When the CLI is running on Linux, the Linux system has different mounts for home and temporary directories, and the `update` command is called (or the same functionality automatically invoked with other commands), then the CLI can return an `invalid cross-device link` error without updating the executable when using the Go standard library `os.Rename()` function. This change introduces fallback logic to catch that error and manually overwrite the executable while accounting for `ETXTBSY` errors on the running executable. Reproduction (macOS host): ```console $ docker run -i -t --privileged --rm ubuntu:24.04 ``` Reproduction (Docker container, manual install to not need `sudo`): ```console $ apt-get update && apt-get install -y wget unzip $ cd /tmp $ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip $ unzip speakeasy_linux_arm64.zip $ chmod a+x speakeasy $ cp speakeasy /usr/bin/speakeasy $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link ``` Verification (macOS host, these changes): ```console $ GOOS=linux go build . $ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04 ``` Verification (Docker container): ```console $ cp /tmp/speakeasy /usr/bin/speakeasy $ speakeasy --version speakeasy version 0.0.1 linux_amd64 $ apt-get update && apt-get install -y ca-certificates $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update Updated to version v1.404.5 $ speakeasy --version speakeasy version 1.404.5 linux_amd64 ```
Reference: https://linear.app/speakeasy/issue/SPE-4208/bug-cli-update-fails-on-linux-with-differing-filesystems-for-tmp-and Reference: golang/go#41487 When the CLI is running on Linux, the Linux system has different mounts for home and temporary directories, and the `update` command is called (or the same functionality automatically invoked with other commands), then the CLI can return an `invalid cross-device link` error without updating the executable when using the Go standard library `os.Rename()` function. This change introduces fallback logic to catch that error and manually overwrite the executable while accounting for `ETXTBSY` errors on the running executable. Reproduction (macOS host): ```console $ docker run -i -t --privileged --rm ubuntu:24.04 ``` Reproduction (Docker container, manual install to not need `sudo`): ```console $ apt-get update && apt-get install -y wget unzip $ cd /tmp $ wget https://github.com/speakeasy-api/speakeasy/releases/download/v1.404.4/speakeasy_linux_arm64.zip $ unzip speakeasy_linux_arm64.zip $ chmod a+x speakeasy $ cp speakeasy /usr/bin/speakeasy $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update failed to replace binary: rename /mnt/ramfs/speakeasy1609184771/extracted/speakeasy /usr/bin/speakeasy: invalid cross-device link ``` Verification (macOS host, these changes): ```console $ GOOS=linux go build . $ docker run -i -t --privileged --rm -v /Users/bflad/src/github.com/speakeasy-api/speakeasy/speakeasy:/tmp/speakeasy ubuntu:24.04 ``` Verification (Docker container): ```console $ cp /tmp/speakeasy /usr/bin/speakeasy $ speakeasy --version speakeasy version 0.0.1 linux_amd64 $ apt-get update && apt-get install -y ca-certificates $ echo "none /mnt/ramfs ramfs noauto,user,size=1024M,mode=1777 0 0" >> /etc/fstab $ mkdir /mnt/ramfs $ mount /mnt/ramfs $ export TMPDIR=/mnt/ramfs $ speakeasy update Updated to version v1.404.5 $ speakeasy --version speakeasy version 1.404.5 linux_amd64 ```
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I am cross-compiling on Mac for Linux
Linux execution env:
Linux nas 4.4.190.x86_64.1 #1 SMP Mon Oct 28 01:55:46 UTC 2019 x86_64 GNU/Linux
Build command:
GOOS=linux GOARCH=amd64 go build -v
What did you expect to see?
os.Rename()
to succeed because the source and target are on the same filesystem.What did you see instead?
os.Rename()
returns "invalid cross-device link" and the above code falls back to copy-and-delete. If I do anmv
on the command line, it completes immediately - even for multi-gigabyte files.The text was updated successfully, but these errors were encountered: