Support building of multi-arch linux amd64 and arm64 container images

[cfryanr]

This PR does not by itself enable building of the Pinniped container image for arm64. It makes a few small changes that will be necessary to support complementary changes in our CI pipelines for building and pushing multi-arch (linux/amd64 and linux/arm64) container images for the Pinniped server image. This is part of the solution for #1386.

To try this locally:

1. Docker only seems to support pushing multi-arch container images to an image registry, so you will need an image registry. You could run one locally, or use a personal account on cloud-based registry like Dockerhub.
2. Create a buildx builder if you have never done so, e.g. docker buildx create --name mybuilder --bootstrap --use
3. Build and push the multi-arch image: docker buildx build --platform linux/arm64,linux/amd64 --tag your_dockerhub_username/your-image-name:your-tag --push .
4. Then you can create any arm64-based Kubernetes cluster. For example, you can create a GKE cluster with arm64 nodes.
5. Then create the yaml manifests for Pinniped. You can use ytt or you can download some from the latest GitHub release. Either way, be sure to change all the image: references to use the container image that you pushed in the steps above.
6. Install Pinniped using your yaml manifests from the previous step on the cluster that you made in the previous steps.
7. Profit!

Release note:

TBD

[codecov]

Codecov Report

Merging #1699 (826d823) into main (af7d309) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1699   +/-   ##
=======================================
  Coverage   79.16%   79.16%           
=======================================
  Files         163      163           
  Lines       15767    15767           
=======================================
  Hits        12482    12482           
+ Misses       2970     2969    -1     
- Partials      315      316    +1     

see 1 file with indirect coverage changes

[joshuatcasey]

Do we need to change the distroless image (note that using tag nonroot-amd64 is an AMD64 image)?

[cfryanr]

Do we need to change the distroless image (note that using tag nonroot-amd64 is an AMD64 image)?

Hmm. It seems like maybe it doesn't matter? GoogleContainerTools/distroless#346 (comment)

This deserves more research.

Also, https://github.com/GoogleContainerTools/distroless does not document that there is an image called gcr.io/distroless/static:nonroot anymore. Maybe we are supposed to switch to gcr.io/distroless/static-debian12:nonroot?

[cfryanr]

This deserves more research.

Ok, I think I figured this out. gcr.io/distroless/static:nonroot is a multi arch container image. It has container images for all these platforms:

❯ docker buildx imagetools inspect gcr.io/distroless/static:nonroot
Name:      gcr.io/distroless/static:nonroot
MediaType: application/vnd.oci.image.index.v1+json
Digest:    sha256:92d40eea0b5307a94f2ebee3e94095e704015fb41e35fc1fcbd1d151cc282222

Manifests:
  Name:      gcr.io/distroless/static:nonroot@sha256:1b4dbd7d38a0fd4bbaf5216a21a615d07b56747a96d3c650689cbb7fdc412b49
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/amd64

  Name:      gcr.io/distroless/static:nonroot@sha256:dcf9c9cafaa9c328eff2ceff5f6057588336b48c9b91ddc0913102b33bbce723
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/arm64/v8

  Name:      gcr.io/distroless/static:nonroot@sha256:47562e180a26a759b92483d90fa9479eccb3479d4dbe8aee6eb62a2b5b593a51
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/arm/v7

  Name:      gcr.io/distroless/static:nonroot@sha256:c0e2f22015a1f262d73a340cb5bc58637b8cc9416c20b5be6a2bdc9e90e1e3af
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/s390x

  Name:      gcr.io/distroless/static:nonroot@sha256:f33466f769690f450d113868593535e503ef0e78f219553d36200af82722648b
  MediaType: application/vnd.oci.image.manifest.v1+json
  Platform:  linux/ppc64le

When building our Dockerfile, the FROM statement will automatically choose the container image from gcr.io/distroless/static:nonroot for our build's target platform (unless you override it with the --platform option). So when we build for arm64, we should be using the arm64 version of gcr.io/distroless/static:nonroot automatically.

By using distroless/static instead of distroless/static-debianXX we can float on the latest stable version of debian, as buried in their documentation here: see https://github.com/GoogleContainerTools/distroless#base-operating-system.