Stop putting user.overlay.* into container layer

[ChristopherHX]

EDIT its this known limitation of docker 25: https://docs.docker.com/engine/release-notes/25.0/#known-limitations

To reprodce the bug caused by this attribute in docker rootless dind containers you can do

containerid="$(docker run -d --privileged --rm -it docker:dind-rootless)"
docker exec -it "$containerid" sh
DOCKER_HOST=unix:///var/run/user/1000/docker.sock docker run --rm -it  ghcr.io/catthehacker/ubuntu:act-latest-20240222
Unable to find image 'ghcr.io/catthehacker/ubuntu:act-latest-20240222' locally
act-latest-20240222: Pulling from catthehacker/ubuntu
6ddad66377a0: Already exists 
a8ffa4da65b6: Already exists 
6f031e49d16f: Already exists 
e8bedde87c3d: Already exists 
d1f12f89b682: Extracting  204.5MB/204.5MB
4ca545ee6d5d: Download complete 
docker: failed to register layer: lsetxattr user.overlay.origin /etc: operation not supported.
See 'docker run --help'.

The old image was built with buildah release included in ubuntu 22.04

The newer image ghcr.io/catthehacker/ubuntu:act-latest-20240228 is built with a patched buildah and works without issues

References

• https://gitea.com/gitea/act_runner/issues/497
• failed to register layer: Isetxattr user.overlay. impure /etc: operation not supported catthehacker/docker_images#122

It is possible that this is a bug in docker 25.x and not here, since older dind rootless images with 24.x and older are working

In the meantime I built my images with a fork of buildah using this patch, it is up to you to decide if this is a good or bad change.

[giuseppe]

LGTM

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ChristopherHX, giuseppe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [giuseppe]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rhatdan]

/lgtm

[ChristopherHX]

Thank you for reviewing / merging my patch.

I have verified that latest buildah now works as expected from my side, I'm just attaching more information to this PR for others finding this change. The PR description didn't provide steps to create such an breaking image with buildah.

These steps for old buildah releases (like buildah version 1.23.1 (image-spec 1.0.1, runtime-spec 1.0.2-dev) as of ubuntu 22.04 like GitHub Hosted Runners / release-1.34) were enough to have those user.overlay.* xattr in several places, tar command for verify, no need for docker

buildah from ubuntu:latest
buildah run ubuntu-working-container apt-get update
buildah run ubuntu-working-container apt-get install -y xattr
buildah run ubuntu-working-container sh -c "echo "TEST" > /etc/test.txt"
buildah commit ubuntu-working-container
buildah images
buildah push <shafromimages> oci:layout:test:test
tar --xattrs --xattrs-include='*' -tvvf layout/blobs/sha256/<shaoflayer-added-by-buildah>

tar output of buildah before containers/buildah@662908f

drwxr-xr-x* root/root         0 2024-03-01 17:09 usr/lib/
  x: 1 user.overlay.impure
  x: 0 user.overlay.origin

the rootless docker dind has problems applying this attribute

Some backlinks for changes on docker side

• Fail unpacking images with xattrs to filesystems without xattr support moby/moby#45464
• [25.0 backport] pkg/system: return even richer xattr errors moby/moby#47178

[GrimzEcho]

Context for others who might discover this while attempting to pull images via rootless docker.

The v25 release of Docker engine made a bugfix so that "Unpacking layers with extended attributes onto an incompatible filesystem will now fail instead of silently discarding extended attributes." (https://github.com/moby/moby/releases/tag/v25.0.0). However, rootless docker with the native overlay storage driver (docker calls it overlay2) cannot run lsetxattr.