minikube image build returning 'permission denied error' for linux(debian-rodete) + kvm & docker vm-drivers

[aaron-prindle]

In attempting to use the minikube image build command on a local dockerfile I am seeing the following:

$ minikube image build .
error checking context: 'no permission to read from '/var/lib/minikube/build/build.87560460/Dockerfile''.

This is the dockerfile I am trying to build (from skaffold/examples/microservices/base/Dockerfile):

$ cat Dockerfile 
FROM gcr.io/distroless/base
# Define GOTRACEBACK to mark this container as using the Go language runtime
# for `skaffold debug` (https://skaffold.dev/docs/workflows/debug/).
ENV GOTRACEBACK=single
CMD ["./app"]

os: linux (debian-rodete)
minikube_version: v1.23.2

$ minikube version
minikube version: v1.23.2
commit: 0a0ad764652082477c00d51d2475284b5d39ceed
$ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux rodete
Release:	rodete
Codename:	rodete

full logs running minikube image build --alsologtostderr . with kvm2 driver:
https://gist.github.com/aaron-prindle/3b15b9082ad8b5994e56e722f78a9cd6

full logs running minikube image build --alsologtostderr . with docker driver:
https://gist.github.com/aaron-prindle/d62d5c3d8a64f2e04216d427eeffab89

This occurs for me when using using both docker and KVM virtualization (--vm-driver=docker & --vm-driver=kvm2)

NOTE: My linux user id is in both kvm and libvirt groups and the kvm2 driver is working properly:

$ sudo virsh net-list --all
 Name          State    Autostart   Persistent
------------------------------------------------
 default       active   yes         yes
 mk-minikube   active   yes         yes

Any idea why I might be seeing this permission denied issue? Am I perhaps using minikube image build . incorrectly with my current env/vm-driver setup? I can see the directory referenced in the permission denied error - /var/lib/minikube/build/build.87560460/Dockerfile in the minikube vm (below output from --vm-driver=kvm2) but not the Dockerfile:

$ minikube ssh
...
$ pwd
/var/lib/minikube/build
$ ls
build.2571535838  build.2571535838.tar

[aaron-prindle]

@sharifelgamal this is the issue I referenced in my chat w/ you

[afbjorklund]

Does the Dockerfile have some mode restrictions like 640 ? Can you list the files ls -l perhaps, the tar format might preserve the owner and group from the host?

[aaron-prindle]

Yes, it has 640/-rw-r----- permissions:

$ stat Dockerfile
...
Access: (0640/-rw-r-----)
...
$ ls -l Dockerfile 
-rw-r----- 1 aprindle primarygroup 211 Jul 30 00:36 Dockerfile

Changing the file perms to 644 resolved this issue, thanks!

[afbjorklund]

Does docker build change the permissions ?

[aaron-prindle]

I'm not sure in what context/where you are referring to the files permissions. docker build works for this Dockerfile with the original permissions (why I was initially confused). The file permissions are unchanged (remain 640) after using docker build.

[afbjorklund]

No worries, I can look into it myself. The docker build transport is somewhat different, even if it is the same "tar" code being used.

"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/fileutils"

[afbjorklund]

Indeed, they replace the user and assume that it is "root" that is reading the archive.

                buildCtx, err = archive.TarWithOptions(contextDir, &archive.TarOptions{
                        ExcludePatterns: excludes,
                        ChownOpts:       &idtools.Identity{UID: 0, GID: 0},
                })

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[sharifelgamal]

There seems to be an easy workaround for this thankfully, but the UX is less than ideal, so I'll leave this open.