Skip to content

Implement "data" provision mode #3302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 12, 2025
Merged

Implement "data" provision mode #3302

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion hack/test-templates.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ declare -A CHECKS=(
["user-v2"]=""
["mount-path-with-spaces"]=""
["provision-ansible"]=""
["provision-data"]=""
["param-env-variables"]=""
["set-user"]=""
)
Expand Down Expand Up @@ -82,6 +83,7 @@ case "$NAME" in
CHECKS["snapshot-offline"]="1"
CHECKS["mount-path-with-spaces"]="1"
CHECKS["provision-ansible"]="1"
CHECKS["provision-data"]="1"
CHECKS["param-env-variables"]="1"
CHECKS["set-user"]="1"
;;
Expand Down Expand Up @@ -194,13 +196,19 @@ if [[ -n ${CHECKS["provision-ansible"]} ]]; then
limactl shell "$NAME" test -e /tmp/ansible
fi

if [[ -n ${CHECKS["provision-data"]} ]]; then
INFO 'Testing that /etc/sysctl.d/99-inotify.conf was created successfully on provision'
limactl shell "$NAME" grep -q fs.inotify.max_user_watches /etc/sysctl.d/99-inotify.conf
fi

if [[ -n ${CHECKS["param-env-variables"]} ]]; then
INFO 'Testing that PARAM env variables are exported to all types of provisioning scripts and probes'
limactl shell "$NAME" test -e /tmp/param-boot
limactl shell "$NAME" test -e /tmp/param-dependency
limactl shell "$NAME" test -e /tmp/param-probe
limactl shell "$NAME" test -e /tmp/param-system
limactl shell "$NAME" test -e /tmp/param-user
# TODO re-enable once https://github.com/lima-vm/lima/issues/3308 is fixed
# limactl shell "$NAME" test -e /tmp/param-user
fi

if [[ -n ${CHECKS["set-user"]} ]]; then
Expand Down
13 changes: 10 additions & 3 deletions hack/test-templates/test-misc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ param:
DEPENDENCY: dependency
PROBE: probe
SYSTEM: system
USER: user
# TODO re-enable once https://github.com/lima-vm/lima/issues/3308 is fixed
# USER: user

provision:
- mode: ansible
Expand All @@ -29,8 +30,14 @@ provision:
script: "touch /tmp/param-$PARAM_DEPENDENCY"
- mode: system
script: "touch /tmp/param-$PARAM_SYSTEM"
- mode: user
script: "touch /tmp/param-$PARAM_USER"
# TODO re-enable once https://github.com/lima-vm/lima/issues/3308 is fixed
# - mode: user
# script: "touch /tmp/param-$PARAM_USER"
- mode: data
path: /etc/sysctl.d/99-inotify.conf
content: |
fs.inotify.max_user_watches = 524288
fs.inotify.max_user_instances = 512

probes:
- mode: readiness
Expand Down
25 changes: 25 additions & 0 deletions pkg/cidata/cidata.TEMPLATE.d/boot.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,31 @@ else
done
fi

# indirect variable lookup, like ${!var} in bash
deref() {
eval echo \$"$1"
}

if [ -d "${LIMA_CIDATA_MNT}"/provision.data ]; then
for f in "${LIMA_CIDATA_MNT}"/provision.data/*; do
filename=$(basename "$f")
overwrite=$(deref "LIMA_CIDATA_DATAFILE_${filename}_OVERWRITE")
owner=$(deref "LIMA_CIDATA_DATAFILE_${filename}_OWNER")
path=$(deref "LIMA_CIDATA_DATAFILE_${filename}_PATH")
permissions=$(deref "LIMA_CIDATA_DATAFILE_${filename}_PERMISSIONS")
if [ -e "$path" ] && [ "$overwrite" = "false" ]; then
INFO "Not overwriting $path"
else
INFO "Copying $f to $path"
# intermediate directories will be owned by root, regardless of OWNER setting
mkdir -p "$(dirname "$path")"
cp "$f" "$path"
chown "$owner" "$path"
chmod "$permissions" "$path"
fi
done
fi

if [ -d "${LIMA_CIDATA_MNT}"/provision.system ]; then
for f in "${LIMA_CIDATA_MNT}"/provision.system/*; do
INFO "Executing $f"
Expand Down
6 changes: 6 additions & 0 deletions pkg/cidata/cidata.TEMPLATE.d/lima.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ LIMA_CIDATA_DISK_{{$i}}_FORMAT={{$disk.Format}}
LIMA_CIDATA_DISK_{{$i}}_FSTYPE={{$disk.FSType}}
LIMA_CIDATA_DISK_{{$i}}_FSARGS={{range $j, $arg := $disk.FSArgs}}{{if $j}} {{end}}{{$arg}}{{end}}
{{- end}}
{{- range $dataFile := .DataFiles}}
LIMA_CIDATA_DATAFILE_{{$dataFile.FileName}}_OVERWRITE={{$dataFile.Overwrite}}
LIMA_CIDATA_DATAFILE_{{$dataFile.FileName}}_OWNER={{$dataFile.Owner}}
LIMA_CIDATA_DATAFILE_{{$dataFile.FileName}}_PATH={{$dataFile.Path}}
LIMA_CIDATA_DATAFILE_{{$dataFile.FileName}}_PERMISSIONS={{$dataFile.Permissions}}
{{- end}}
LIMA_CIDATA_GUEST_INSTALL_PREFIX={{ .GuestInstallPrefix }}
{{- if .Containerd.User}}
LIMA_CIDATA_CONTAINERD_USER=1
Expand Down
17 changes: 16 additions & 1 deletion pkg/cidata/cidata.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"path"
"path/filepath"
"slices"
"strconv"
"strings"
"time"

Expand Down Expand Up @@ -322,10 +323,19 @@ func templateArgs(bootScripts bool, instDir, name string, instConfig *limayaml.L

args.BootCmds = getBootCmds(instConfig.Provision)

for _, f := range instConfig.Provision {
for i, f := range instConfig.Provision {
if f.Mode == limayaml.ProvisionModeDependency && *f.SkipDefaultDependencyResolution {
args.SkipDefaultDependencyResolution = true
}
if f.Mode == limayaml.ProvisionModeData {
args.DataFiles = append(args.DataFiles, DataFile{
FileName: fmt.Sprintf("%08d", i),
Overwrite: strconv.FormatBool(*f.Overwrite),
Owner: *f.Owner,
Path: *f.Path,
Permissions: *f.Permissions,
})
}
}

return &args, nil
Expand Down Expand Up @@ -376,6 +386,11 @@ func GenerateISO9660(instDir, name string, instConfig *limayaml.LimaYAML, udpDNS
Path: fmt.Sprintf("provision.%s/%08d", f.Mode, i),
Reader: strings.NewReader(f.Script),
})
case limayaml.ProvisionModeData:
layout = append(layout, iso9660util.Entry{
Path: fmt.Sprintf("provision.%s/%08d", f.Mode, i),
Reader: strings.NewReader(*f.Content),
})
case limayaml.ProvisionModeBoot:
continue
case limayaml.ProvisionModeAnsible:
Expand Down
10 changes: 10 additions & 0 deletions pkg/cidata/template.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,15 @@ type Mount struct {
type BootCmds struct {
Lines []string
}

type DataFile struct {
FileName string
Overwrite string
Owner string
Path string
Permissions string
}

type Disk struct {
Name string
Device string
Expand Down Expand Up @@ -84,6 +93,7 @@ type TemplateArgs struct {
Env map[string]string
Param map[string]string
BootScripts bool
DataFiles []DataFile
DNSAddresses []string
CACerts CACerts
HostHomeMountPoint string
Expand Down
59 changes: 38 additions & 21 deletions pkg/limatmpl/embed.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -552,11 +552,11 @@ func (tmpl *Template) combineNetworks() {
}
}

// updateScript replaces a "file" property with the actual script and then renames the field to "script".
func (tmpl *Template) updateScript(field string, idx int, script string) {
// updateScript replaces a "file" property with the actual script and then renames the field to newName ("script" or "content").
func (tmpl *Template) updateScript(field string, idx int, newName, script string) {
entry := fmt.Sprintf("$a.%s[%d].file", field, idx)
// Assign script to the "file" field and then rename it to "script".
tmpl.expr.WriteString(fmt.Sprintf("| (%s) = %q | (%s | key) = \"script\"\n", entry, script, entry))
tmpl.expr.WriteString(fmt.Sprintf("| (%s) = %q | (%s | key) = %q\n", entry, script, entry, newName))
}

// embedAllScripts replaces all "provision" and "probes" file references with the actual script.
Expand All @@ -565,30 +565,47 @@ func (tmpl *Template) embedAllScripts(ctx context.Context, embedAll bool) error
return err
}
for i, p := range tmpl.Config.Probes {
if p.File == nil {
continue
}
warnFileIsExperimental()
// Don't overwrite existing script. This should throw an error during validation.
if p.File != nil && p.Script == "" {
warnFileIsExperimental()
isTemplate, _ := SeemsTemplateURL(p.File.URL)
if embedAll || !isTemplate {
scriptTmpl, err := Read(ctx, "", p.File.URL)
if err != nil {
return err
}
tmpl.updateScript("probes", i, string(scriptTmpl.Bytes))
if p.Script != "" {
continue
}
isTemplate, _ := SeemsTemplateURL(p.File.URL)
if embedAll || !isTemplate {
scriptTmpl, err := Read(ctx, "", p.File.URL)
if err != nil {
return err
}
tmpl.updateScript("probes", i, "script", string(scriptTmpl.Bytes))
}
}
for i, p := range tmpl.Config.Provision {
if p.File != nil && p.Script == "" {
warnFileIsExperimental()
isTemplate, _ := SeemsTemplateURL(p.File.URL)
if embedAll || !isTemplate {
scriptTmpl, err := Read(ctx, "", p.File.URL)
if err != nil {
return err
}
tmpl.updateScript("provision", i, string(scriptTmpl.Bytes))
if p.File == nil {
continue
}
warnFileIsExperimental()
newName := "script"
switch p.Mode {
case limayaml.ProvisionModeData:
newName = "content"
if p.Content != nil {
continue
}
default:
if p.Script != "" {
continue
}
}
isTemplate, _ := SeemsTemplateURL(p.File.URL)
if embedAll || !isTemplate {
scriptTmpl, err := Read(ctx, "", p.File.URL)
if err != nil {
return err
}
tmpl.updateScript("provision", i, newName, string(scriptTmpl.Bytes))
}
}
return tmpl.evalExpr()
Expand Down
12 changes: 12 additions & 0 deletions pkg/limatmpl/embed_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -339,20 +339,32 @@ networks:
provision:
# This script will be merged from an external file
- file: base1.sh # This comment will move to the "script" key
# This is just a data file
- mode: data
file: base1.sh # This comment will move to the "content" key
path: /tmp/data
`,
`
# base0.yaml is ignored
---
#!/usr/bin/env bash
echo "This is base1.sh"
`,
// TODO: the empty line after the `path` is unexpected
`
# Hi There!
provision:
# This script will be merged from an external file
- script: |- # This comment will move to the "script" key
#!/usr/bin/env bash
echo "This is base1.sh"
# This is just a data file
- mode: data
content: |- # This comment will move to the "content" key
#!/usr/bin/env bash
echo "This is base1.sh"
path: /tmp/data

# base0.yaml is ignored
`,
},
Expand Down
45 changes: 41 additions & 4 deletions pkg/limayaml/defaults.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -487,10 +487,47 @@ func FillDefault(y, d, o *LimaYAML, filePath string, warn bool) {
if provision.Mode == ProvisionModeDependency && provision.SkipDefaultDependencyResolution == nil {
provision.SkipDefaultDependencyResolution = ptr.Of(false)
}
if out, err := executeGuestTemplate(provision.Script, instDir, y.User, y.Param); err == nil {
provision.Script = out.String()
} else {
logrus.WithError(err).Warnf("Couldn't process provisioning script %q as a template", provision.Script)
if provision.Mode == ProvisionModeData {
if provision.Content == nil {
provision.Content = ptr.Of("")
} else {
if out, err := executeGuestTemplate(*provision.Content, instDir, y.User, y.Param); err == nil {
provision.Content = ptr.Of(out.String())
} else {
logrus.WithError(err).Warnf("Couldn't process data content %q as a template", *provision.Content)
}
}
if provision.Overwrite == nil {
provision.Overwrite = ptr.Of(true)
}
if provision.Owner == nil {
provision.Owner = ptr.Of("root:root")
} else {
if out, err := executeGuestTemplate(*provision.Owner, instDir, y.User, y.Param); err == nil {
provision.Owner = ptr.Of(out.String())
} else {
logrus.WithError(err).Warnf("Couldn't owner %q as a template", *provision.Owner)
}
}
// Path is required; validation will throw an error when it is nil
if provision.Path != nil {
if out, err := executeGuestTemplate(*provision.Path, instDir, y.User, y.Param); err == nil {
provision.Path = ptr.Of(out.String())
} else {
logrus.WithError(err).Warnf("Couldn't process path %q as a template", *provision.Path)
}
}
if provision.Permissions == nil {
provision.Permissions = ptr.Of("644")
}
}
// TODO Turn Script into a pointer; it is a plain string for historical reasons only
if provision.Script != "" {
if out, err := executeGuestTemplate(provision.Script, instDir, y.User, y.Param); err == nil {
provision.Script = out.String()
} else {
logrus.WithError(err).Warnf("Couldn't process provisioning script %q as a template", provision.Script)
}
}
}

Expand Down
11 changes: 11 additions & 0 deletions pkg/limayaml/limayaml.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,7 @@ const (
ProvisionModeBoot ProvisionMode = "boot"
ProvisionModeDependency ProvisionMode = "dependency"
ProvisionModeAnsible ProvisionMode = "ansible"
ProvisionModeData ProvisionMode = "data"
)

type Provision struct {
Expand All @@ -229,6 +230,16 @@ type Provision struct {
Script string `yaml:"script" json:"script"`
File *LocatorWithDigest `yaml:"file,omitempty" json:"file,omitempty" jsonschema:"nullable"`
Playbook string `yaml:"playbook,omitempty" json:"playbook,omitempty"`
// All ProvisionData fields must be nil unless Mode is ProvisionModeData
ProvisionData `yaml:",inline"` // Flatten fields for "strict" YAML mode
}

type ProvisionData struct {
Content *string `yaml:"content,omitempty" json:"content,omitempty" jsonschema:"nullable"`
Overwrite *bool `yaml:"overwrite,omitempty" json:"overwrite,omitempty" jsonschema:"nullable"`
Owner *string `yaml:"owner,omitempty" json:"owner,omitempty"` // any owner string supported by `chown`, defaults to "root:root"
Path *string `yaml:"path,omitempty" json:"path,omitempty"`
Permissions *string `yaml:"permissions,omitempty" json:"permissions,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think these ones have to be mixed in provision:.

Can we just have files: [] in the top-level ?

Copy link
Member Author

@jandubois jandubois Mar 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disjoint set of fields has bothered me too. But on the other hand, conceptually this is very similar to provisioning scripts; the main difference being that they are not executed. This

- mode: data
  path: /etc/sysctl.d/99-inotify.conf
  content: |
    fs.inotify.max_user_watches = 524288
    fs.inotify.max_user_instances = 512

is pretty much the same as1:

- mode: system
  script: |
    cat <<EOF >/etc/sysctl.d/99-inotify.conf
    fs.inotify.max_user_watches = 524288
    fs.inotify.max_user_instances = 512
    EOF

So creating yet another top-level setting for it feels a bit unstructured.

And while not really a strong argument, creating another setting would also require a lot of code duplication for the template merging and file embedding code.

Footnotes

  1. The main advantage (with external data files) is that you can create them easily with automation. E.g. you can use a YAML or JSON marshaler to create the file, but there is no default tooling to wrap them as here-documents in a script that writes the document to a file. ↩

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinion from me either

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be int?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had it as an int first, but that makes it hard/impossible to verify that the user specified a leading 0 to make it octal, so you wouldn't get a validation error for permissions: 5111. The string is always parsed as octal, so it doesn't matter if you specify the leading 0 or not.

I thought this could be a common cause of errors because chmod doesn't require the leading 0 either.

Footnotes

  1. I just double-checked, and 511 is the same as 0777, so you can't tell if the user meant 0511 or not. ↩

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that I didn't update the comments in default.yaml, which still claim that the leading 0 is required. That is a leftover from when the field was still an int.

}

type Containerd struct {
Expand Down
Loading
Loading