doc: document Helm deployer's IMAGE_NAME<N>, IMAGE_TAG<N>, IMAGE_DIGEST<N>

[snickell]

I can't find any documentation (other than references in issues by skaffold devs) to these. Without this, its very hard to figure out how to use the IMAGE_NAME template etc for more than a single artifact.

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[snickell]

@googlebot I signed it!

[briandealwis]

Thanks @snickell. May I ask how you use these values?

From what I can tell, these IMAGE_XXX<N> values are only supported for the Helm deployer.

The custom builder supports IMAGE_NAME and IMAGE_TAG, but doesn't support the IMAGE_DIGEST. And this isn't documented.

The Kaniko builder supports IMAGE_REPO, IMAGE_NAME, and IMAGE_TAG, but the _REPO and _NAME are different.

[codecov]

Codecov Report

Merging #6649 (fb1001b) into main (290280e) will decrease coverage by 0.49%.
The diff coverage is 53.33%.

@@            Coverage Diff             @@
##             main    #6649      +/-   ##
==========================================
- Coverage   70.48%   69.98%   -0.50%     
==========================================
  Files         515      522       +7     
  Lines       23150    23732     +582     
==========================================
+ Hits        16317    16609     +292     
- Misses       5776     6038     +262     
- Partials     1057     1085      +28     

Impacted Files	Coverage Δ
cmd/skaffold/app/cmd/flags.go	89.00% <0.00%> (-1.82%)	⬇️
cmd/skaffold/skaffold.go	0.00% <ø> (ø)
pkg/diag/recommender/container_errors.go	0.00% <0.00%> (ø)
pkg/diag/validator/pod.go	1.32% <0.00%> (ø)
pkg/skaffold/build/buildpacks/logger.go	0.00% <ø> (ø)
pkg/skaffold/build/cluster/logs.go	0.00% <ø> (-16.67%)	⬇️
cmd/skaffold/app/cmd/cmd.go	70.32% <66.66%> (-0.73%)	⬇️
pkg/diag/validator/resource.go	47.05% <66.66%> (ø)
pkg/skaffold/build/docker/docker.go	86.44% <66.66%> (-2.85%)	⬇️
pkg/skaffold/build/docker/errors.go	63.33% <80.00%> (-2.23%)	⬇️
... and 77 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 151b8d3...fb1001b. Read the comment docs.

[snickell]

@briandealwis

TL;DR While the helmStrategy does reflect the conventional recommendation for how container image references be formatted as a values.yaml JSON tree, not all upstream charts use that recommendation, making it impossible to override their values with skaffold's helm artifactOverrides system.... instead we build the values ourself using setValueTemplates support for env substitution (yuck I know, yet it fixed our world 🙏 ).

Basically, in our case, an upstream helm chart (JupyterHub: https://github.com/jupyterhub/zero-to-jupyterhub-k8s) required build artifacts to be inserted into the helm values.yaml JSON tree in a format that was both strict (they enabled JSON schema validation, which in turn prevents this from being just a warning but there's now no way to deploy our system with skaffold until fixed), and not producable by skaffold's available imageStrategys.

Upstream jupyterhub chart expects image references to be formatted exactly like this:

image:
    name: gcr.io/my-proj/my-image
    tag: sometag

Note e.g. the lack of a repository field, this is disallowed by the Jupyterhub schema, with no way to disable the schema = can't deploy.

By explicitly using gitCommit: { variant: AbbrevCommitSha } we can use setValueTemplates to construct and inject the value JSON tree in the shape required by the upstream z2jh chart:

      setValueTemplates:
        # Why not use artifactOverrides? We have to match name/tag value format required by the jupyterhub chart
        jupyterhub.singleuser.image.name: "{{.IMAGE_NAME}}"
        jupyterhub.singleuser.image.tag: "{{.IMAGE_TAG}}@{{.IMAGE_DIGEST}}"
        jupyterhub.hub.image.name: "{{.IMAGE_NAME2}}"
        jupyterhub.hub.image.tag: "{{.IMAGE_TAG2}}@{{.IMAGE_DIGEST2}}"

Our preference, not surprisingly, would have been for buildArtifacts to itself have the option to output in the format expected by our upstream chart (which we can't change), e.g. perhaps a imageStrategy: helm: { tagOnly : true } }.

[briandealwis]

@snickell that looks exactly what the helm strategy would produce:

https://skaffold.dev/docs/pipeline-stages/deployers/helm/#helm-strategy-split-repository-and-tag

[snickell]

@briandealwis haha, I kept doing doubletakes and thinking that too, but the key required by JupyterHub is called name, but skaffold's imageStrategy: helm outputs a similar (but not identical value because it includes a sha256, this is important) to repository which is schema-wise incompatible in two blocking aspects:

With a sha256 included the JupyterHub chart (aka z2jh) would require:

image:
    name: gcr.io/my-proj/my-image
    tag: sometag@sha256:b931f0d42b663066bdd09e5966cb5731d1ee59bbe1276bea5aeb7c1c5bff728f

But the skaffold helm strategy would output:

image:
    repository: gcr.io/my-proj/my-image@sha256:b931f0d42b663066bdd09e5966cb5731d1ee59bbe1276bea5aeb7c1c5bff728f
    tag: sometag

There's two fatal incompatibilities that as far as I could tell can't be worked around:

1. Wrong field name: The field is named repository and jupyterhub chart schema requires it to be name
2. Sha256 would need to be output to the tag for JupyterHub chart templates usage: even if the field name wasn't different, the JupyterHub chart template string concatenates it with the tag in their pod yaml template: {{ .Values.image.name}}:{{ .Values.image.tag }} producing gcr.io/my-proj/my-image@sha256:b931f0d42b663066bdd09e5966cb5731d1ee59bbe1276bea5aeb7c1c5bff728f:sometag which isn't a valid image reference (wont' work w/ docker pull etc, order of tag and sha256 is wrong). In essence, with the way this chart template is written, the sha256 pretty much has to be output to the tag field (because the colon is hardcoded in the string template).

[briandealwis]

Oh I missed the difference in the key names 🤦 How annoying.

The good news is that we can use Helm's post-renderer and do away with this artifactsOverride entirely. Stay tuned.

[snickell]

@briandealwis Exciting! I had a branch that used the Helm post-renderer to patch things up (but I couldn't figure out how to active post-render from within skaffold, sounds like a forthcoming feature 💯 )

[briandealwis]

@snickell the tracking issue is #6706.