Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove helm repo update #507

Closed
wants to merge 1 commit into from

Conversation

craigfurman
Copy link
Contributor

@craigfurman craigfurman commented Feb 7, 2021

It is entirely possible I've missed something important, but I can't figure out why helm repo update ever needs to be run by tanka. As far as I can tell, tanka execs helm in 2 contexts: helm pull, as part of tk tool charts vendor, and helm template, as part of rendering jsonnet.

The helm pull invocation passes --repository-config with a freshly created temporary helm repo file. The helm repo update invocation that precedes helm pull also passes in --repository-config, with a different freshly created tempfile, that is never used again.

The helm template invocation in pkg/helm/jsonnet.go passes in a local path to the already-vendored chart, and does not interact with remote repositories.

If I'm right, removing this code speeds up tk tool charts vendor by removing an unnecessary network call, and if the correct chart versions are already present on the filesystem, no network calls will be made at all, due to the idempotence introduced in #420. This helps make CI flows that cache/vendor helm charts more reliable.

Draft in case I'm totally wrong about this call being unncessary!

Signed-off-by: Craig Furman <craig.furman89@gmail.com>
@craigfurman
Copy link
Contributor Author

@sh0rez would you mind taking a look and letting me know if this is a sensible change?

@sh0rez
Copy link
Member

sh0rez commented Feb 7, 2021

So, I've just tested this, this is what I found out:

  • On a clean system (e.g. container), tk tool charts fails with no cached repo found. (try 'helm repo update'): open /home/<user>/.cache/helm/repository/stable-index.yaml.
    This means we need to run it at least once initially
  • After adding a new repo (tk tool charts add-repo traefik <url>), it again fails because the new repo is not in the chart cache:
    no cached repo found. (try 'helm repo update'): open /home/<user>/.cache/helm/repository/traefik-index.yaml: no such file or directory
  • I suspect that when a upstream adds a new chart to their repo and we don't update the local index, we couldn't helm pull the newer version.

Because of above, I opted for running it every time back writing the first iteration of this code.
I do agree though that we run it far too often.

It should technically be possible to catch those errors (they seem to have a common format, we could analyze stderr like we do with kubectl) and run helm repo update only on demand.

Wdyt?

@craigfurman
Copy link
Contributor Author

On a clean system (e.g. container), tk tool charts fails with no cached repo found.

Which tk tool charts subcommand fails with that? I can't reproduce that in tk tool charts vendor, even with an empty local helm cache.

I'll look more deeply at add-repo, it's possible that I was misunderstanding how (multiple) helm repo config files interact with the actual cache.

@craigfurman
Copy link
Contributor Author

I'm totally wrong above, and @sh0rez is right about needing to have repository caches locally to run helm pull. I'll close this, and will try a new PR that only runs helm repo update when the local charts directory doesn't match the chartfile, avoiding a network call when vendoring helm charts in your tanka config repo.

@craigfurman craigfurman deleted the rm-helm-repo-update branch March 10, 2021 15:06
@craigfurman
Copy link
Contributor Author

Maybe #535 🤞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants