Skip to content

Conversation

jandubois
Copy link
Member

@jandubois jandubois commented Mar 4, 2025

These are data files copied to the guest filesystem. They are not scripts and are not being executed.

It is pretty similar to the write_files feature in cloud-init, but is completely implemented in our boot.sh, so works automatically with lima-init and other cloud-init alternatives.

provision:
- mode: data
  path: /etc/conf.d/example
  content: |
    VAR=VALUE
  overwrite: false
- mode: data
  path: "{{.Home}}/kubeadm.yaml"
  file: kubeadm.yaml
  owner: "{{.User}}"
  permissions: 0700

The file property works the same way it works for provision scripts and probes (during template embedding the file content is placed into content and the file property is removed, so the data is embedded in the instance directory).

@jandubois
Copy link
Member Author

@nirs This is an evolved variant of the data mode scripts we talked about on Slack a few weeks ago. Instead of providing variables of the data file locations I thought it makes more sense to allow the user to specify the target path where the file should be installed, so you don't need a cp command in a script.

Example:

cat inotify.yaml
base: template://ubuntu-24.04
provision:
- mode: data
  path: /etc/sysctl.d/99-inotify.conf
  file: sysctl.confcat sysctl.conf
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 512l tmpl copy --embed inotify.yaml -
WARN[0000] `provision[*].file` and `probes[*].file` are experimental
base: template://ubuntu-24.04
provision:
- mode: data
  path: /etc/sysctl.d/99-inotify.conf
  content: |
    fs.inotify.max_user_watches = 524288
    fs.inotify.max_user_instances = 512

@jandubois jandubois marked this pull request as ready for review March 11, 2025 21:58
@jandubois jandubois requested a review from a team March 11, 2025 21:58
Overwrite *bool `yaml:"overwrite,omitempty" json:"overwrite,omitempty" jsonschema:"nullable"`
Owner *string `yaml:"owner,omitempty" json:"owner,omitempty"`
Path *string `yaml:"path,omitempty" json:"path,omitempty"`
Permissions *string `yaml:"permissions,omitempty" json:"permissions,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these ones have to be mixed in provision:.

Can we just have files: [] in the top-level ?

Copy link
Member Author

@jandubois jandubois Mar 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disjoint set of fields has bothered me too. But on the other hand, conceptually this is very similar to provisioning scripts; the main difference being that they are not executed. This

- mode: data
  path: /etc/sysctl.d/99-inotify.conf
  content: |
    fs.inotify.max_user_watches = 524288
    fs.inotify.max_user_instances = 512

is pretty much the same as1:

- mode: system
  script: |
    cat <<EOF >/etc/sysctl.d/99-inotify.conf
    fs.inotify.max_user_watches = 524288
    fs.inotify.max_user_instances = 512
    EOF

So creating yet another top-level setting for it feels a bit unstructured.

And while not really a strong argument, creating another setting would also require a lot of code duplication for the template merging and file embedding code.

Footnotes

  1. The main advantage (with external data files) is that you can create them easily with automation. E.g. you can use a YAML or JSON marshaler to create the file, but there is no default tooling to wrap them as here-documents in a script that writes the document to a file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion from me either

Overwrite *bool `yaml:"overwrite,omitempty" json:"overwrite,omitempty" jsonschema:"nullable"`
Owner *string `yaml:"owner,omitempty" json:"owner,omitempty"`
Path *string `yaml:"path,omitempty" json:"path,omitempty"`
Permissions *string `yaml:"permissions,omitempty" json:"permissions,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be int?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had it as an int first, but that makes it hard/impossible to verify that the user specified a leading 0 to make it octal, so you wouldn't get a validation error for permissions: 5111. The string is always parsed as octal, so it doesn't matter if you specify the leading 0 or not.

I thought this could be a common cause of errors because chmod doesn't require the leading 0 either.

Footnotes

  1. I just double-checked, and 511 is the same as 0777, so you can't tell if the user meant 0511 or not.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that I didn't update the comments in default.yaml, which still claim that the leading 0 is required. That is a leftover from when the field was still an int.

Script string `yaml:"script" json:"script"`
File *LocatorWithDigest `yaml:"file,omitempty" json:"file,omitempty" jsonschema:"nullable"`
Playbook string `yaml:"playbook,omitempty" json:"playbook,omitempty"`
// The following fields must be nil unless Mode is ProvisionModeData
Copy link
Member

@AkihiroSuda AkihiroSuda Mar 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just use an embedded struct?

type ProvisionData struct{
  Content *string `...`
  // ...
}

type Provision struct {
  Mode ProvisionMode `...`
  // ...
  *ProvisionData
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, as long as it is transparent to YAML unmarshaling (I just don't know; will test). I wouldn't want to have an extra settings level for them in lima.yaml.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to break "strict" YAML mode:

time="2025-03-11T23:03:06-07:00" level=warning msg="Non-strict YAML detected; please check for typos" comment="main file \"lima.yaml\"" error="[1:26] unknown field \"path\"\n>  1 | provision: [{mode: data, path: /tmp, content: hello}]\n                                ^\n   2 | images: [{location: /}]"

It looks like go-yaml has a tag to flatten promoted fields, so I'm going to use that:

	*ProvisionData `yaml:",inline"` // Flatten fields for "strict" YAML mode

Not sure about making the embedded ProvisionData a pointer; it will just require additional checks, and validation already checks each individual field. But I'll try.

// The following fields must be nil unless Mode is ProvisionModeData
Content *string `yaml:"content,omitempty" json:"content,omitempty" jsonschema:"nullable"`
Overwrite *bool `yaml:"overwrite,omitempty" json:"overwrite,omitempty" jsonschema:"nullable"`
Owner *string `yaml:"owner,omitempty" json:"owner,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format should be clarified in a godoc comment. Does this accept numeric UIDs? Does this accept specifying groups? ("USER:GROUP")

if p.Owner != nil {
return fmt.Errorf("field `provision[%d].owner` can only be set when mode is %q", i, ProvisionModeData)
}
if p.Path != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-absolute paths should be rejected here?

These are data files copied to the guest filesystem. They are not scripts
and are not being executed.

Signed-off-by: Jan Dubois <jan.dubois@suse.com>
@jandubois
Copy link
Member Author

Sorry, I also rebased the PR, so latest changes are hard to see. What I did:

  • Added ProvisionData embedded struct
  • Added comment to Owner field
  • Add validation check that Path is absolute
  • Changed Owner default from root to root:root
  • Updated default.yaml with owner and permissions doc changes.

@jandubois jandubois requested a review from AkihiroSuda March 12, 2025 07:28
@AkihiroSuda AkihiroSuda added this to the v1.1.0 milestone Mar 12, 2025
Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@AkihiroSuda AkihiroSuda merged commit c1994c5 into lima-vm:master Mar 12, 2025
31 checks passed
@jandubois jandubois deleted the provision-data branch March 13, 2025 02:08
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request May 23, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [lima-vm/lima](https://github.com/lima-vm/lima) | minor | `v1.0.7` -> `v1.1.1` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>lima-vm/lima (lima-vm/lima)</summary>

### [`v1.1.1`](https://github.com/lima-vm/lima/releases/tag/v1.1.1)

[Compare Source](lima-vm/lima@v1.1.0...v1.1.1)

#### Changes

-   Fixed the guestagent path issues with Homebrew  ([#&#8203;3566](lima-vm/lima#3566), thanks to [@&#8203;jandubois](https://github.com/jandubois))
-   Documentation for disk management https://lima-vm.io/docs/config/disk/ ([#&#8203;3539](lima-vm/lima#3539), thanks to [@&#8203;Sonichigo](https://github.com/Sonichigo))

#### Usage

```console
[macOS]$ limactl create
[macOS]$ limactl start
...
INFO[0029] READY. Run `lima` to open the shell.

[macOS]$ lima uname
Linux
```

***

The binaries were built automatically on GitHub Actions.
The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/15178234655

The sha256sum of the SHA256SUMS file itself is `0d2d3fb073c8e26df163937dd605e9f2b9f227814a697411cc2b8879347bdd7f` .

***

Release manager: [@&#8203;AkihiroSuda](https://github.com/AkihiroSuda)

### [`v1.1.0`](https://github.com/lima-vm/lima/releases/tag/v1.1.0)

[Compare Source](lima-vm/lima@v1.0.7...v1.1.0)

> \[!IMPORTANT]
> Package maintainers should refer to "Hint for package maintainers" below

Highlights:

-   Simplification of YAMLs
-   New port forwarder implementation by default
-   DragonflyBSD hosts
-   S390X and PPC64LE guests
-   Smaller binary packages

#### Changes

-   Build system:
    -   Split `lima-additional-guestagents-*.tar.gz` from `lima-*.tar.gz` ([#&#8203;3503](lima-vm/lima#3503))
    -   Set `CONFIG_GUESTAGENT_COMPRESS=y` by default ([#&#8203;3529](lima-vm/lima#3529))
-   YAML:
    -   Support inheritance ([#&#8203;3072](lima-vm/lima#3072), thanks to [@&#8203;jandubois](https://github.com/jandubois))
    -   Deprecate provision mode `ansible` ([#&#8203;3451](lima-vm/lima#3451), thanks to [@&#8203;afbjorklund](https://github.com/afbjorklund))
    -   Add new provision mode `data` ([#&#8203;3302](lima-vm/lima#3302), thanks to [@&#8203;jandubois](https://github.com/jandubois))
-   QEMU:
    -   Support DragonflyBSD hosts ([#&#8203;3356](lima-vm/lima#3356), thanks to [@&#8203;tuxillo](https://github.com/tuxillo))
    -   Support S390X guests ([#&#8203;3319](lima-vm/lima#3319))
    -   Support PPC64LE guests ([#&#8203;3488](lima-vm/lima#3488))
    -   TCG: change the default CPU from `qemu64` (x86-64 v1) to `max` ([#&#8203;3487](lima-vm/lima#3487))
    -   Bump up the minimum QEMU version to v8.2.1 on ARM Mac. On Linux/x86\_64, QEMU v6.2.0 is still supported. ([#&#8203;3491](lima-vm/lima#3491))
-   WSL2:
    -   Lots of improvements (several MRs, thanks to [@&#8203;arixmkii](https://github.com/arixmkii))
-   Port forwarding:
    -   Enable faster gRPC implementation by default ([#&#8203;3046](lima-vm/lima#3046))
-   `limactl` CLI:
    -   Add `--yes` flag as an alias of `--tty=false` ([#&#8203;3342](lima-vm/lima#3342), thanks to [@&#8203;suryaaprakassh](https://github.com/suryaaprakassh))
    -   Support resizing disk in `limactl edit` ([#&#8203;3437](lima-vm/lima#3437), [#&#8203;3533](lima-vm/lima#3533), thanks to [@&#8203;songponssw](https://github.com/songponssw))
    -   Add `limactl disk import` command ([#&#8203;3439](lima-vm/lima#3439), thanks to [@&#8203;songponssw](https://github.com/songponssw))
-   Rootless Containers:
    -   Allow UID >= 524288 ([#&#8203;3435](lima-vm/lima#3435))
-   nerdctl:
    -   Update from v2.0.4 to [v2.1.2](https://github.com/containerd/nerdctl/releases/tag/v2.1.2). ([#&#8203;3483](lima-vm/lima#3483), [#&#8203;3534](lima-vm/lima#3534), [#&#8203;3560](lima-vm/lima#3560))
        See also the release notes of [v2.0.5](https://github.com/containerd/nerdctl/releases/tag/v2.0.5), [v2.1.0](https://github.com/containerd/nerdctl/releases/tag/v2.1.0), [v2.1.1](https://github.com/containerd/nerdctl/releases/tag/v2.1.1).
        -   nerdctl v2.1.1 included in Lima v1.1.0-rc.0 had a vulnerability of containerd CVE-2025-47290 (GHSA-cm76-qm8v-3j95). Other versions of Lima are not affected.
-   Templates:
    -   `ubuntu-25.04`: New template ([#&#8203;3445](lima-vm/lima#3445)).
        The `default` template still refers to Ubuntu 24.10, as `ubuntu-25.04` needs
        the very recent release of macOS on Intel Mac by default (see the note below).
    -   `fedora-42`: New template ([#&#8203;3434](lima-vm/lima#3434))
    -   `linuxbrew`: New template ([#&#8203;3454](lima-vm/lima#3454))
    -   `almalinux-kitten-10`: New template ([#&#8203;3084](lima-vm/lima#3084), thanks to [@&#8203;afbjorklund](https://github.com/afbjorklund))
    -   `oraclelinux-8`: Fix virtiofs support ([#&#8203;3441](lima-vm/lima#3441))

> \[!NOTE]
> On Intel Mac with `--vm-type=vz`, macOS 15.5 or later is needed to boot `ubuntu-25.04` and `fedora-42`

Full changes: https://github.com/lima-vm/lima/milestone/26?closed=1
Thanks to [@&#8203;KGB33](https://github.com/KGB33) [@&#8203;Nino-K](https://github.com/Nino-K) [@&#8203;afbjorklund](https://github.com/afbjorklund) [@&#8203;alexandear](https://github.com/alexandear) [@&#8203;apachex692](https://github.com/apachex692) [@&#8203;arixmkii](https://github.com/arixmkii) [@&#8203;cakemanny](https://github.com/cakemanny) [@&#8203;jandubois](https://github.com/jandubois) [@&#8203;jonas-peter](https://github.com/jonas-peter) [@&#8203;kairveeehh](https://github.com/kairveeehh) [@&#8203;liangyuanpeng](https://github.com/liangyuanpeng) [@&#8203;nirs](https://github.com/nirs) [@&#8203;shenki](https://github.com/shenki) [@&#8203;songponssw](https://github.com/songponssw) [@&#8203;unsuman](https://github.com/unsuman) [@&#8203;ycdzj](https://github.com/ycdzj)

#### Hint for package maintainers

Starting with v1.1, the official binary packages of Lima are split to two files for the each of the host OS and the architectures to save the disk space:

1.  `lima-<VERSION>-Darwin-arm64.tar.gz`: (Built with `make native`)

-   The core components (`bin/lima`, `bin/limactl`, `share/lima/templates`, ...)
-   The guest agent for the native architecture (`share/lima/lima-guestagent.Linux-aarch64.gz`)

2.  `lima-additional-guestagents-<VERSION>-Darwin-arm64.tar.gz`: (Built with `make additional-guestagents`)

-   The guest agents for emulating non-native architectures (`share/lima/lima-guestagent.Linux-{armv7l,ppc64le,riscv64,s390x,x86_64}.gz)`)

For compatibility reason, `make` still builds the guest agents for all the architectures by default.

Package maintainers are suggested to split their `lima` package to `lima` (`make native`) and `lima-additional-guestagents` (`make additional-guestagents`).

#### Usage

```console
[macOS]$ limactl create
[macOS]$ limactl start
...
INFO[0029] READY. Run `lima` to open the shell.

[macOS]$ lima uname
Linux
```

***

The binaries were built automatically on GitHub Actions.
The build log is available for 90 days: https://github.com/lima-vm/lima/actions/runs/15154830653

The sha256sum of the SHA256SUMS file itself is `4391505a7f833c8245497b29fec65743abb1561a275037f6d268026284883c8a` .

***

Release manager: [@&#8203;AkihiroSuda](https://github.com/AkihiroSuda)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC4yMS4wIiwidXBkYXRlZEluVmVyIjoiNDAuMjMuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90Il19-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants