Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kaniko - 2nd destination directory is not created when using Cloud Build #2817

Open
dazdaz opened this issue Oct 26, 2023 · 2 comments
Open
Labels
area/caching For all bugs related to cache issues area/registry For all bugs having to do with pushing/pulling into registries kind/question Further information is requested priority/p1 Basic need feature compatibility with docker build. we should be working on this next. product/artifact-registry provider/gcp

Comments

@dazdaz
Copy link

dazdaz commented Oct 26, 2023

The 2nd destination folder does not create a cache folder on Google Cloud Artifact Registry. I am using the latest kaniko release.

Only the first cache directory is created from my cloudbuild.yaml.

By swapping the destinations around, I can confirm that the 2nd destination folder is not created on Artifact Registry.

I've read around the issues on GitHub and the general consensus appears to be that a 2nd destination cache is possible but it just doesn't work for me in my tests.

Is a 3rd, 4th and 5th cache directory supported with Kaniko or limited to 2 cache folders ?

Here is the relevant section in cloudbuild.yaml

Both container images are created successfully and pushed to Artifact Registry however the 2nd destination directory is not created

This is the directory structure that I see on Artifact Registry, so app2-image-cache has not been created.

app1-image
app1-image/cache
app2-image
steps:
    - name: 'gcr.io/kaniko-project/executor:latest'
      args: [
      "--destination=${_MYREGION}-docker.pkg.dev/${_PROJECT_ID}/quickstart-docker-repo/app1-image:${BUILD_ID}",
      "--destination=${_MYREGION}-docker.pkg.dev/${_PROJECT_ID}/quickstart-docker-repo/app2-image:${BUILD_ID}",
      "--cache=true",
      "--cache-ttl=8h"
      ]
@JeromeJu JeromeJu added area/caching For all bugs related to cache issues area/registry For all bugs having to do with pushing/pulling into registries product/artifact-registry priority/p1 Basic need feature compatibility with docker build. we should be working on this next. labels Oct 27, 2023
@JeromeJu JeromeJu added provider/gcp kind/question Further information is requested labels Nov 10, 2023
@JeromeJu
Copy link
Collaborator

Thanks for filing this @dazdaz , this does look like a bug within the implementations of cache being pushed to the 2nd destination as specified. (Thanks the pointers from @aaron-prindle 🙏 )
The normal use case seems to be with one destination specified. In order to find the best way solving this issue, can I ask for what's the use case for doing so for using multiple destinations for caching?

@dazdaz
Copy link
Author

dazdaz commented Nov 12, 2023

Large microservice based app, with a lot of different images requiring some form of organization through the use of more than 1 repo due to different owners designated to each repo. I'd presume that some folks would require more than 3, 4, 5 etc

Isn't this a simple case of mkdir() missing for the secondary destination or is there more to it ?

Or do you envisage a design whereby there may well be 2,3,4,5 repo's but only 1 shared container cache, if so then this could have security implications where companies would not use it, whereas individualised cache for each repo would be ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/caching For all bugs related to cache issues area/registry For all bugs having to do with pushing/pulling into registries kind/question Further information is requested priority/p1 Basic need feature compatibility with docker build. we should be working on this next. product/artifact-registry provider/gcp
Projects
None yet
Development

No branches or pull requests

2 participants