Description
related moby/moby#14080
Allowing exporting contents of the type=cache mounts have been asked many times in different repositories and slack.
Regular remote instruction cache does not work for cache mounts that are not tracked by cache key and are just a location on disk that can be shared by multiple builds.
Currently, the best approach to maintain this cache between nodes is to do it as part of the build. docker/buildx#244 (comment)
I don't think we should try to combine cache mounts with the remote cache backends. Usually, cache mounts are for throwaway cache and restoring it would take a similar time to just recreating it.
What we could do is to allow users to control where the cache location is on disk, in case it is not on top of the snapshots.
We can introduce a cache mount backend concept behind a go interface that different implementation can implement.
Eg. for a Dockerfile like
RUN --mount=type=cache,target=/root/.cache,id=gocache go build ...
you could invoke a build with
docker build --cache-mount id=gocache,type=volume,volume=myvolume .
In that case, the cache could use a Docker volume as a backend. I guess good drivers would be volume in Docker and bind mount from the host for non-docker. If no --cache-mount is specified, the usual snapshotter-based location is used.
From the security perspective, BuildKit API is considered secure by default for the host, so I think this would require daemon side configuration to enable what paths can be bound.
Another complexity is buildx container driver as we can't easily add new mounts to a running container atm. Possible solutions are to force these paths to be set on buildx create or do some hackery with mount propagation.