Debian-slim as Alpine alternative concerning disk footprint

[pr-ju]

Since ssd space is often an issue on macs but Alpine uses a different c library, which can result in poor perforce when running a python app (https://pythonspeed.com/articles/alpine-docker-python/), I would recommend supporting debian-slim as alternative. With debian-slim the footprint is ca. 82 MiB, which is larger than Alpine but still okay. Maybe Ubuntu minimal is another option here.

[afbjorklund]

You can use any image on the VM, the libc is included in the container. Performance issues with it (the VM), would be in the kernel (that is shared) - and of course in the hypervisor. The libc on the VM only affects the apps on the VM, in this case that is not python.

[pr-ju]

I just thought about the size of the vm disk, since all other images downloaded by the templates are quite big (> 200 or 300 GiB)

[afbjorklund]

I think you mean MiB, they are not that big :-) But yes, 300M for the "small" ones and 600M for the "big" ones.

When compared with something like CoreOS (1G) or OpenShift (10G), those sizes are not so big - even on the SSD

[afbjorklund]

Actually the debian images were slightly smaller (half), so if you are concerned about disk space it might be worth it.

#	/home/anders/.cache/lima/download/by-url-sha256/65a62e4461082eeb92c0623f4db7ba6b27ea625b5e91ff7ddbdca2d721302777
728M	https://cloud-images.ubuntu.com/releases/22.10/release-20230413/ubuntu-22.10-server-cloudimg-amd64.img
#	/home/anders/.cache/lima/download/by-url-sha256/37f5fc187ab151950c3e7f30e2231bedf2f8af23b24933aec9f3a2a2020fa4ba
319M	https://cloud.debian.org/images/cloud/bullseye/20230124-1270/debian-11-generic-amd64-20230124-1270.qcow2

Made an issue to include the size with the checksums, to make it easier to know the size "ahead of time" - to compare

• Provide the size of the disk image, along with the checksum and url #1586

[afbjorklund]

Note that the above sizes are without the container runtime, so that needs to be added too for a fair comparison.

#	/home/anders/.cache/lima/download/by-url-sha256/e22c9d3619acd34258d3d43e2b685bf6d4b421aa5c83f4241d96b1ee6b6fe141
233M	https://github.com/containerd/nerdctl/releases/download/v1.3.0/nerdctl-full-1.3.0-linux-amd64.tar.gz
#	/home/anders/.cache/lima/download/by-url-sha256/28137234cfb6f64a587565fb9676c1ddf73912e99520e5026ef9bca0672b7ce0
62M	https://github.com/lima-vm/alpine-lima/releases/download/v0.2.27/alpine-lima-std-3.17.0-x86_64.iso

It would be possible to create a custom debian-lima image, that included all the software and used dynamic linking.

Most likely it would end up being somewhere in-between 50-150M (alpine) and 300-600M (debian), in download ?

[afbjorklund]

Currently there is no debian packaging for nerdctl and for buildkit (as far as I know), so that could be a start.

• Help wanted: distribution packages containerd/nerdctl#1986

A workaround could be to just package the nerdctl-full binaries as debian packages, and install with apt.

8.8M	/tmp/lima/buildg_0.4.1_amd64.deb
38M	/tmp/lima/buildkit_0.11.6_amd64.deb
3.9M	/tmp/lima/bypass4netns_0.3.0_amd64.deb
33M	/tmp/lima/cni-plugins_1.3.0_amd64.deb
39M	/tmp/lima/containerd_1.7.1_amd64.deb
4.1M	/tmp/lima/containerd-fuse-overlayfs_1.0.6_amd64.deb
744K	/tmp/lima/fuse-overlayfs_1.12.0_amd64.deb
9.2M	/tmp/lima/nerdctl_1.4.0_amd64.deb
12K	/tmp/lima/nerdctl-rootless-extras_1.4.0_amd64.deb
12M	/tmp/lima/rootlesskit_1.1.0_amd64.deb
3.9M	/tmp/lima/runc_1.1.7_amd64.deb
1020K	/tmp/lima/slirp4netns_1.2.0_amd64.deb
376K	/tmp/lima/tini_0.19.0_amd64.deb
152M	total

[afbjorklund]

Maybe Ubuntu minimal is another option here.

Maintaining a special optimized Ubuntu cloud image for Lima could be a good idea, but it also needs to be maintained properly.

It would be cool if any OS vendor did a special "spin" for it, but that can be done with a custom yaml? Or we could make our own OS.

But the Debian cloud images (qcow2) are smaller

[pr-ju]

https://github.com/debuerreotype/docker-debian-artifacts/blob/4648ad5c6feb1318b8a6c31502ebde8bb18ebb5e/bullseye/slim/Dockerfile

No I was just referring to the locations of the two images you mentioned:

debian-based image (20.10) was 100M while the tinycore-based image (19.03) was around 50M

as well to the repository where the debian-slim containers are administered for the docker registry

https://github.com/debuerreotype/docker-debian-artifacts/blob/4648ad5c6feb1318b8a6c31502ebde8bb18ebb5e/bullseye/slim/

[afbjorklund]

I don't have the old images up anymore, sorry. They were just ballpark figures on what I remembered by heart.

The "docker-machine" and "boot2docker" projects are only related in spirit to Lima, the code is completely different.

[afbjorklund]

I would suggest creating a github issue, regarding the possible creation of such a debian-based image for Lima ?

Most likely, it would be a separate project (similar to https://github.com/lima-vm/alpine-lima) and need volunteers.

[pr-ju]

I would suggest creating a github issue, regarding the possible creation of such a debian-based image for Lima ?

You mean here in the project?

Most likely, it would be a separate project (similar to https://github.com/lima-vm/alpine-lima) and need volunteers.

I would even trying to support, but my current skill-set concerning the topic is not very well developed, but I'm trying to make myself familiar with it.

[afbjorklund]

There are several different aspects of this particular topic...

The first would be to change the base of your container images, to use "debian-slim" instead of "alpine" as per those articles linked. Related to that is some documentation in Lima, regarding what is the diffence between using Alpine VM and using Debian(/Ubuntu) VM. Such a documentation would have to explain musl libc vs. glibc, and openrc vs. systemd. Something like a pros and cons? Most of this can be accomplished without changing any code in the Lima project itself, and many of the topics are more generic in nature.

Then it comes to the issue if the Lima project should provide a custom version of Debian(/Ubuntu), like it provides a custom version of Alpine? For alpine-lima, it is necessary because certain features are missing from Alpine "virt". But for debian-lima, it would be more of an optimization (like that "slimming" script) since all required features are already in Debian(/Ubuntu) cloud images ("NoSource"). So it is not a given, that it is worth the extra work of maintaining a downstream distribution - just to save disk space.

So it would help to formulate, what changes you were thinking of.