Using lima as an oss alternative to vagrant

[fwilhe]

Hi,

based on this pr I'd like to discuss how vagrant's license change might drive new users to lima.

I have used vagrant with the libvirt provider for quite some time, but it always worked a bit unreliable for me. Lima seems to be a much more stable environment for me, but I guess it does not have all features that vagrant has. From what I can see so far, the most important features are

• trivial ssh wrapper
• mounting pwd into the vm
• provisioning via shell scripts

All of the above worked well for me so far. I'd like to thank all lima contributors, it's so useful.

I'd like to use this discussion to learn from others if they think lima is a good alternative for vagrant and if you see any missing features.

[afbjorklund]

Something that was discussed was to extend "script", to use something like ansible?

As an example here was a Vagrantfile that I used, similar to the Lima k8s.yaml template:

https://github.com/afbjorklund/vanilla-kubernetes/blob/master/Vagrantfile

  config.vm.provision "ansible" do |ansible|
    ansible.compatibility_mode = '2.0'
    ansible.playbook = "ansible/playbook.yml"
    ansible.extra_vars = { ansible_python_interpreter:"/usr/bin/python3" }
    ansible.verbose = false
  end

It is possible that it could make to work the same, but not sure about out-of-the-box

[fwilhe]

Yes, that would be useful. I think calling ansible in a script provisioning is a workaround, but proper support would be nice.

[afbjorklund]

I'm not sure how much would actually be needed in Lima, most of it would just fork a ansible-playbook call on the host?

[afbjorklund]

Caching packages on the host was a feature that I missed, from vagrant-cachier.

• Caching packages and images on the lima host #397

  if Vagrant.has_plugin?("vagrant-cachier")
    # Configure cached packages to be shared between instances of the same base box.
    config.cache.scope = :box
  end

Again something that could be made to work with mounts, or be extended further.

[afbjorklund]

I'd like to discuss how vagrant's license change might drive new users to lima.

In all fairness, Vagrant 2.3.7 is OSS (under the MIT license)

https://github.com/hashicorp/vagrant/tree/v2.3.7

But it is very likely that Vagrant 3.x will not be, or if 2.x will be supported in 2024.

https://www.hashicorp.com/blog/toward-vagrant-3-0

[fwilhe]

They already changed the license on the main branch in the git repo. Yes, old versions stay available and maybe someone is going to fork that, but I think it is fair to say that they changed the license.

[afbjorklund]

I think the main problem for users that don't use the third-party development libvirt provider, is that VirtualBox is only available for amd64 and not for arm64. At least on the Mac that is a problem, addressed by Lima.

[AkihiroSuda]

Relevant

• limactl create: add --plain flag to disable mounts, port forwarding, containerd, etc. #1739

[obnoxxx]

Interesting discussion topic here @fwilhe !

I have been using vagrant for years, mostly on linux hosts and have only been made aware of lima recently, looking for alternatives to use on MacOS, because I have been facing problems getting vagrant to work on MacOS.

So far, after first superficial experiments lima seems quite nice but compared to vagrant's compelling ease of use, some aspects of lima seem unnecessarily complicated to me.

while lima's templates are conceptually similar to vagrant's boxes, vagrant's box interface is more obvious and self-explanatory for me.

lima's shell access to the vm is confusing to me. I appear to be the same user as on the host and pwd in the lima shell is the host user's dir when invoking limactl shell. vagrant ssh is more obvious and less confusing to me.

vagrant's convenient provisioning hooks are missing in lima, best I can tell.

The biggest advantage I see in lima so far is the fact that I don't have to struggle with virtualisation "providers"like in vagrant. Providers have been the biggest inhibitor for vagrant for me on MacOS.

configuration: while agrant's ruby-based Vagrantfiles are certainly not the simplest mechanisms on earth, they are quite powerful and I do prefer them over lima's yaml, because I am generally not a fan of yaml.

license: I don't yet quite understand the point how vagrant's license change would drive users to lima:

for me, having to chose between Apache and MIT licenses is like being stuck between a rock and a hard place.

According to me, open source software should ideally use a copyleft license like the GPL.

GPL would definitely drive me to lima.

[fwilhe]

Yes exactly right, I personally care about OSI approved open source licenses, which of those a project has is not that much of a concert to me when I 'just' use software. That's why I try to avoid shared source licenses if possible.

In my experience, lima also works more reliable compared to vagrant. I always had issues getting the vagrant libvirt provider to work reliably, with lima I have a much more solid setup, both on macos and on linux.

I know that it's not a drop in replacement, but for my needs lima is a very good solution.

[jandubois]

lima's shell access to the vm is confusing to me. I appear to be the same user as on the host and pwd in the lima shell is the host user's dir when invoking limactl shell. vagrant ssh is more obvious and less confusing to me.

It is a matter of expectations. The original goal for Lima was to provide seamless operation of containerd and nerdctl on macOS. To make it appear as if the the containers run natively on macOS the boundary between the host and the guest needs to disappear as much as possible: the filesystem and ports in the guest and on the host should be as interchangeable as possible.

This may then be confusing if you just want to run a VM and expect it to be a distinct entity. That is still possible, but the default configurations tries to hide the boundary, similar to how WSL2 does it on Windows.

[afbjorklund]

Other users showed a similar surprise reaction with the default mount of $HOME, as they were expecting $PWD to be mounted under something like /vagrant - and then have that as their working directory. So it could be made a flag, similar to --plain?

[jandubois]

So it could be made a flag, similar to --plain?

I don't think this would work because the mounts are specified in the templates and not baked into Lima itself.

The part we don't currently support is specifying $PWD in the template, mostly because the provisioning scripts are executed when you run limactl start, and when you stop and restart an instance, you may be in a different directory then when you initially created it. But we could easily add a template variable for it, and then it is up to the user or the provisioning script to deal with it, if the directory is not always the same:

mounts:
- location: "{{.PWD}}"
  mountPoint: /vagrant  # or /lima, or whatever

I think we can implement a lot of this conditional functionality with the new param mechanism, especially if we also implement the chaining of templates via basedOn, as I discussed in #2520.

[afbjorklund]

Right, I made some quick hacks to save the pwd into a file (in instance dir) for this to work. As a part of #2442