add rsync flag option to copy files using rsync #3143

olamilekan000 · 2025-01-23T14:56:37Z

change adds rsync as an option to the copy command

issue #2198

nirs

Do we have rsync in the guest in all cases? Minimal vm images do not have rsync.

Also adding options is bad both for users and for testing. If it is better to copy with rsync, we should detect if rsync is available in the guest and host, and use it without adding new option. Otherwise fall back to scp.

If we always have rsync, better to use it instead of scp and minimize the testing matrix.

cmd/limactl/copy.go

nirs · 2025-01-23T19:39:06Z

Also, do we have tests for code modified in this PR? if not we should not change it before adding tests. The new rsync flow should also have tests.

jandubois · 2025-01-23T20:05:15Z

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

olamilekan000 · 2025-01-23T21:34:29Z

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

AkihiroSuda · 2025-01-23T21:47:39Z

If we always have rsync, better to use it instead of scp and minimize the testing matrix.

Yes, rsync is always available on macOS and most Linux distros.
At least it is easily installable.

nirs · 2025-01-24T15:21:36Z

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

@olamilekan000, Can you describe how did you try to test and why it did not work well?

You can also join the #lima slack channel to get quicker help from other developers.

olamilekan000 · 2025-01-24T20:47:35Z

@olamilekan000 I've asked before to please test your changes locally instead of force-pushing an update every 10 minutes for 2 hours, triggering potentially costly CI runs all the time.

I'd appreciate it if you can help with steps to run locally. I did try but It didn't work as expected.

@olamilekan000, Can you describe how did you try to test and why it did not work well?

You can also join the #lima slack channel to get quicker help from other developers.

@nirs I have joined and shared my error logs on the channel.

cmd/limactl/copy.go

pkg/hostagent/hostagent.go

nirs · 2025-01-26T15:49:12Z

I think we can do this:

Use cloud-init to install rsync
- If cloud-init packages does not work for a distro, it must be done in the lima.yaml provision scripts
Replace scp with rsync

@AkihiroSuda what do you think?

hack/test-templates.sh

AkihiroSuda · 2025-01-26T23:21:01Z

Use cloud-init to install rsync

SGTM

AkihiroSuda · 2025-07-09T08:54:24Z

What's the current status of this?

olamilekan000 · 2025-08-12T17:42:10Z

What's the current status of this?

Update has been made. Currently awaiting review.

pkg/cidata/cidata.TEMPLATE.d/user-data

pkg/hostagent/hostagent.go

cmd/limactl/copy.go

hack/test-templates.sh

Signed-off-by: olalekan odukoya <odukoyaonline@gmail.com>

AkihiroSuda

Thanks

⚠️ **CAUTION: this is a major update, indicating a breaking change!** ⚠️ This MR contains the following updates: | Package | Update | Change | |---|---|---| | [lima-vm/lima](https://github.com/lima-vm/lima) | major | `v1.2.2` -> `v2.0.1` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>lima-vm/lima (lima-vm/lima)</summary> ### [`v2.0.1`](https://github.com/lima-vm/lima/releases/tag/v2.0.1) [Compare Source](lima-vm/lima@v2.0.0...v2.0.1) #### Changes - Binary release artifacts: - Fix a regression in v2.0.0 `level=fatal msg="template \"_images/<IMAGE>.yaml\" not found"` ([#4313](lima-vm/lima#4313), thanks to [@vvoland](https://github.com/vvoland)) - Misc: - pkg/networks/usernet: use `SIGINT` instead of `SIGKILL` ([#4310](lima-vm/lima#4310), thanks to [@norio-nomura](https://github.com/norio-nomura)) Full changes: <https://github.com/lima-vm/lima/milestone/64?closed=1> #### Usage ```console $ limactl create $ limactl start ... INFO[0029] READY. Run `lima` to open the shell. $ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: <https://github.com/lima-vm/lima/actions/runs/19137304035> The sha256sum of the SHA256SUMS file itself is `25ad222fa1cf91a85ef7be67664f2ba65228a5d82a39be1adbbe842096854e24` . *** Release manager: [@AkihiroSuda](https://github.com/AkihiroSuda) ### [`v2.0.0`](https://github.com/lima-vm/lima/releases/tag/v2.0.0) [Compare Source](lima-vm/lima@v1.2.2...v2.0.0) This is the second major release of Lima, featuring the support for [pluggable VM drivers](https://lima-vm.io/docs/dev/drivers/), [GPU acceleration](https://lima-vm.io/docs/config/gpu/), and [MCP](https://lima-vm.io/docs/config/ai/outside/mcp/). This release also commemorates the promotion of the project from CNCF [Sandbox](https://www.cncf.io/sandbox-projects/) to [Incubating](https://www.cncf.io/projects/) 🎉. #### Highlights - [Experimental plug-in subsystem for VM driver infrastructure](https://lima-vm.io/docs/dev/drivers/). This will help implementing third-party plugins without modifying the code base of Lima. Thanks to [GSoC 2025](https://gist.github.com/unsuman/ff31a323ecef2289bf065882726ed7f0) contributor [@unsuman](https://github.com/unsuman) . - [Experimental krunkit VM driver](https://lima-vm.io/docs/config/vmtype/krunkit/) for supporting GPU acceleration ([#4137](lima-vm/lima#4137), thanks to [@unsuman](https://github.com/unsuman)) - [Experimental integration for Model Context Protocol (MCP)](https://lima-vm.io/docs/config/ai/outside/) ([#3744](lima-vm/lima#3744)). i.e., Lima can be now used as a sandbox for AI agents such as Gemini. - Add `limactl (start|restart) --progress` flag to show the progress of provisioning ([#3846](lima-vm/lima#3846), [#3915](lima-vm/lima#3915), thanks to [@olamilekan000](https://github.com/olamilekan000) [@norio-nomura](https://github.com/norio-nomura)) - Add `limactl shell --preserve-env` flag to propagate env vars from the host to VM ([#3830](lima-vm/lima#3830), thanks to [@olamilekan000](https://github.com/olamilekan000)) #### Other notable changes - `/tmp/lima` is no longer mounted by default ([#3951](lima-vm/lima#3951)) - SSH port is no longer hard-coded to 60022 for the "default" instance ([#3780](lima-vm/lima#3780)) - Forward UDP ports by default ([#4054](lima-vm/lima#4054)) - Support CLI plugins ([#3834](lima-vm/lima#3834), [#4009](lima-vm/lima#4009), thanks to [@olamilekan000](https://github.com/olamilekan000)) - Support custom URL scheme plugins ([#3937](lima-vm/lima#3937), thanks to [@jandubois](https://github.com/jandubois)). `template://default` is now recommended to be written as `template:default`. The old form is still supported. ##### Details - VM driver infrastructure: - [Experimental plug-in subsystem for VM driver infrastructure](https://lima-vm.io/docs/dev/drivers/) ([multiple MRs](https://github.com/lima-vm/lima/pulls?q=is%3Apr+milestone%3Av2.0.0+is%3Aclosed+label%3Aarea%2Fvmdrivers), thanks to [@unsuman](https://github.com/unsuman)) - krunkit: - [Experimental krunkit VM driver](https://lima-vm.io/docs/config/vmtype/krunkit/) for supporting GPU acceleration ([#4137](lima-vm/lima#4137), thanks to [@unsuman](https://github.com/unsuman)) - VZ: - Support Rosetta AOT Caching with CDI ([#3858](lima-vm/lima#3858), thanks to [@norio-nomura](https://github.com/norio-nomura)) - Support accelerating SSH using `AF_VSOCK` ([#3979](lima-vm/lima#3979), thanks to [@norio-nomura](https://github.com/norio-nomura)) - QEMU: - Fallback to TCG when KVM is not available on Linux hosts ([#4204](lima-vm/lima#4204)) - MCP: - [Experimental integration for Model Context Protocol (MCP)](https://lima-vm.io/docs/config/ai/outside/) ([#3744](lima-vm/lima#3744)). Lima now provides MCP tools for reading, writing, and executing local files using a VM sandbox. Known to work with Google Gemini CLI. - `limactl` CLI: - Add `limactl (start|restart) --progress` flag to show the progress of provisioning ([#3846](lima-vm/lima#3846), [#3915](lima-vm/lima#3915), thanks to [@olamilekan000](https://github.com/olamilekan000) [@norio-nomura](https://github.com/norio-nomura)) - Add `limactl (create|start|edit) --port-forward` flag for static port forwarding ([#3699](lima-vm/lima#3699), thanks to [@Horiodino](https://github.com/Horiodino)). Usually not needed, but useful for instances created with `--plain`. - Add `limactl (create|start|edit) --ssh-port` flag ([#3791](lima-vm/lima#3791)) - Add `limactl (create|start|edit) --mount-only` flag ([#3947](lima-vm/lima#3947)). Similar to `--mount`, but overrides the existing mounts. Useful for mounting `$(pwd)`. - Support specifying `--set` multiple times in `limactl (create|start|edit)` ([#4197](lima-vm/lima#4197), thanks to [@AndiDog](https://github.com/AndiDog)) - Add `limactl shell --preserve-env` flag to propagate env vars from the host to VM ([#3830](lima-vm/lima#3830), thanks to [@olamilekan000](https://github.com/olamilekan000)). See also [`LIMA_SHELLENV_ALLOW`](https://lima-vm.io/docs/config/environment-variables/#lima_shellenv_allow) and [`LIMA_SHELLENV_BLOCK`](https://lima-vm.io/docs/config/environment-variables/#lima_shellenv_block). - Support CLI plugins ([#3834](lima-vm/lima#3834), [#4009](lima-vm/lima#4009), thanks to [@olamilekan000](https://github.com/olamilekan000)) - Support custom URL scheme plugins ([#3937](lima-vm/lima#3937), thanks to [@jandubois](https://github.com/jandubois)). `template://default` is now recommended to be written as `template:default`. The old form is still supported. - Add `limactl copy --backend=rsync` flag as an alternative to `scp` backend ([#3143](lima-vm/lima#3143), thanks to [@olamilekan000](https://github.com/olamilekan000)) - Add `limactl list--yq` and `limactl info --yq` flags ([#3998](lima-vm/lima#3998), thanks to [@jandubois](https://github.com/jandubois)) - Add `limactl rename OLD NEW` ([#4207](lima-vm/lima#4207)) - Deprecate `--yes` and introduce `limactl (clone|rename|edit|shell) --start` instead ([#4108](lima-vm/lima#4108), [#4285](lima-vm/lima#4285), thanks to [@Horiodino](https://github.com/Horiodino) [@nlordell](https://github.com/nlordell)) - YAML: - Migrate `cpuType` to `vmOpts.qemu` ([#3500](lima-vm/lima#3500), thanks to [@unsuman](https://github.com/unsuman)) - Add `yq` provision mode ([#3892](lima-vm/lima#3892), thanks to [@norio-nomura](https://github.com/norio-nomura)) - Prohibit relative paths in YAML ([#3950](lima-vm/lima#3950)). Relative paths were never intended to be supported, but they were accidentally allowed due to a regression in v1.1.0. The CLI command `limactl (create|start|edit) --mount DIR` still supports relative paths. - Default template: - Remove `/tmp/lima` mount ([#3951](lima-vm/lima#3951)) - Stop hardcoding SSH port 60022 ([#3780](lima-vm/lima#3780)) - Network: - Enable mDNS for vzNAT and socket\_vmnet ([#4272](lima-vm/lima#4272), thanks to [@norio-nomura](https://github.com/norio-nomura)) - Port forwarding: - Support port forwarding in plain mode ([#3699](lima-vm/lima#3699), thanks to [@Horiodino](https://github.com/Horiodino)) - Support host sockets in gRPC port forwarder ([#4008](lima-vm/lima#4008), thanks to [@norio-nomura](https://github.com/norio-nomura)) - Forward UDP ports by default ([#4054](lima-vm/lima#4054)) - Eliminated 3-second delay for detecting ports ([#4066](lima-vm/lima#4066)) - Removed iptables watcher for `sudo nerdctl run -p ...` ([#4107](lima-vm/lima#4107)). `sudo nerdctl run -p ...` now requires nerdctl v2.1.6 or later. - Improved performance of gRPC forwarder ([#4247](lima-vm/lima#4247), thanks to [@balajiv113](https://github.com/balajiv113)) - Support UDP in Kubernetes ([#4233](lima-vm/lima#4233)) - Change default of `guestIPMustBeZero` to `true` when `guestIP` is `0.0.0.0` ([#4221](lima-vm/lima#4221), thanks to [@jandubois](https://github.com/jandubois)) - Build system: - Remove `Kconfig` and `config.mk`, in favor of Makefile variables ([#3732](lima-vm/lima#3732)) - Support Fedora, RHEL, and relevant host distributions ([#4228](lima-vm/lima#4228), thanks to [@valdela1](https://github.com/valdela1)) - Templates: - `alpine`, `alpine-iso`: update to Alpine 3.22 ([#4184](lima-vm/lima#4184), [#4190](lima-vm/lima#4190), thanks to [@jandubois](https://github.com/jandubois)) - `debian`: update to Debian 13 ([#4029](lima-vm/lima#4029), thanks to [@unsuman](https://github.com/unsuman)) - `docker`, `docker-rootful`: Enable containerd image store ([#3941](lima-vm/lima#3941), thanks to [@norio-nomura](https://github.com/norio-nomura)) - `fedora`: update to Fedora 43 ([#4255](lima-vm/lima#4255)) - `opensuse`: update to openSUSE Leap 16 ([#4203](lima-vm/lima#4203)) - `oraclelinux`: update to Oracle Linux 10 ([#4236](lima-vm/lima#4236), thanks to [@valdela1](https://github.com/valdela1)) - `ubuntu`, `default`: update Ubuntu to 25.10 ([#4202](lima-vm/lima#4202)) - `k0s`: New template ([#3728](lima-vm/lima#3728), thanks to [@plandem](https://github.com/plandem)) - `experimental/ubuntu-next`: update to Ubuntu 26.04 pre-release ([#4311](lima-vm/lima#4311)) - Project: - Invite Ansuman Sahoo ([@unsuman](https://github.com/unsuman)) as a Reviewer ([#4003](lima-vm/lima#4003), thanks to [@jandubois](https://github.com/jandubois)) - Promote from CNCF Sandbox to Incubating ([#4201](lima-vm/lima#4201)) Full changes: <https://github.com/lima-vm/lima/milestone/59?closed=1> Thanks to [@AndiDog](https://github.com/AndiDog) [@Horiodino](https://github.com/Horiodino) [@afbjorklund](https://github.com/afbjorklund) [@alexandear](https://github.com/alexandear) [@ashwat287](https://github.com/ashwat287) [@balajiv113](https://github.com/balajiv113) [@bonifaido](https://github.com/bonifaido) [@dharsanb](https://github.com/dharsanb) [@gnawhleinad](https://github.com/gnawhleinad) [@iamleot](https://github.com/iamleot) [@jandubois](https://github.com/jandubois) [@kachick](https://github.com/kachick) [@muchzill4](https://github.com/muchzill4) [@ningmingxiao](https://github.com/ningmingxiao) [@nlordell](https://github.com/nlordell) [@norio-nomura](https://github.com/norio-nomura) [@olamilekan000](https://github.com/olamilekan000) [@plandem](https://github.com/plandem) [@stek29](https://github.com/stek29) [@unsuman](https://github.com/unsuman) [@valdela1](https://github.com/valdela1) [@vax-r](https://github.com/vax-r) [@vishalanarase](https://github.com/vishalanarase) [@zyfy29](https://github.com/zyfy29) #### EOL of v1.2 Lima v1.2 will continue to receive security updates and critical bug fixes until **2026-02-06** (3 months from now). See also <https://lima-vm.io/docs/releases/>. #### Usage ```console $ limactl create $ limactl start ... INFO[0029] READY. Run `lima` to open the shell. $ lima uname Linux ``` *** The binaries were built automatically on GitHub Actions. The build log is available for 90 days: <https://github.com/lima-vm/lima/actions/runs/19130682878> The sha256sum of the SHA256SUMS file itself is `112f1ef1d9850e29b4be425ca71e8b6ac686f593ff741164885b51fbd6919ca6` . *** Release manager: [@AkihiroSuda](https://github.com/AkihiroSuda) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch 6 times, most recently from a508328 to 4b57bc6 Compare January 23, 2025 17:09

nirs reviewed Jan 23, 2025

View reviewed changes

cmd/limactl/copy.go Show resolved Hide resolved

cmd/limactl/copy.go Show resolved Hide resolved

cmd/limactl/copy.go Outdated Show resolved Hide resolved

cmd/limactl/copy.go Outdated Show resolved Hide resolved

nirs requested a review from afbjorklund January 24, 2025 15:22

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch from 4b57bc6 to aa5cf22 Compare January 24, 2025 20:45

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch from aa5cf22 to 4c95f10 Compare January 25, 2025 00:27

nirs reviewed Jan 25, 2025

View reviewed changes

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch 5 times, most recently from 73fcc08 to a9bade5 Compare January 26, 2025 15:22

nirs reviewed Jan 26, 2025

View reviewed changes

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch 4 times, most recently from 1d17b78 to 7c55a22 Compare January 26, 2025 22:20

nirs reviewed Jan 26, 2025

View reviewed changes

hack/test-templates.sh Outdated Show resolved Hide resolved

AkihiroSuda linked an issue Mar 12, 2025 that may be closed by this pull request

limactl copy should use rsync instead of scp for copying a directory #2198

Closed

olamilekan000 force-pushed the add-rsycn-as-an-option-for-copying-files-btw-host-and-guest branch 2 times, most recently from 2a5decf to 5b560fd Compare August 12, 2025 00:50

olamilekan000 requested review from AkihiroSuda and nirs August 12, 2025 01:15

AkihiroSuda added this to the v2.0.0 milestone Aug 13, 2025