Rework container image rebuilds

[lbussell]

Current state

graph TD

subgraph docker-tools repo
check-base-images["check base images pipeline \n (multiple repos)"]
subscriptions[subscriptions.json]
end

subgraph consuming["consuming repo"]
build-pipeline[build-pipeline]
end

subgraph versions["versions repo"]
image-info[image-info.json]
end

subgraph acr
public["public images"]
end

check-base-images -->|reads| subscriptions
check-base-images -->|"reads (many)"| image-info
check-base-images -->|"queues (many)"| build-pipeline
subscriptions -->|"references (many)"| consuming
build-pipeline <-->|updates and reads| image-info
build-pipeline -->|pushes images| public

Loading

Problems with current setup

• Requires a separate repo to store image-info.json metadata file. Git is in the critical path for builds.
• Only the docker-tools repo can queue base image re-bulids, and the docker-tools repo must know about all other consuming repos in order to queue their builds.
• The docker-tools repo and the consuming repos can go out-of-sync. For example, if the manifest.json or image-info formats change, then the docker-tools check-base-images pipeline can fail until all consuming repos have adopted the updated image-builder version.

Desired state

Key ideas:

• Consuming repos should own their own base image update pipeline + schedule.
• The docker-tools repo should provide sane defaults/pipeline templates for base image re-builds.
• Base image re-builds should work both in public and internal scenarios without extra work.
• There should not be a dependency on external repos or storage locations for the critical base image re-build path. image-info.json, if it still needs to exist, should not be a source of truth. It should be computed separately, strictly downstream of image build pipelines.

graph TD

subgraph consuming[consuming repo]
check-base-images["check base images pipeline \n (one repo)"]
build-pipeline[build pipeline]
end

subgraph acr
public["public images \n with labels/annotations"]
end

public -->|reads| check-base-images
public -->|reads| build-pipeline
check-base-images -->|queues| build-pipeline
build-pipeline -->|pushes images| public

Loading

Future work

Here's the work that I see required to reach this desired state:

• Use image metadata (labels, annotations) to drive automated image re-builds #1602
  • Attach metadata labels to images during build - Add image labels for base image, base image digest, and source #2163
  • Implement a new version of the getStaleImages command which reads these labels to determine which images are out of date.
  • Implement a new pipeline which runs the new command and queues a build for stale images.
• Backwards compatibility with existing image-info.json publishing
  • ImageBuilder should still synthesize image-info.json and write it to disk, at least for the time being.
  • For backwards compatibility, we need to provide some kind paved path for publishing image-info.json to a location like a git repo, OCI artifact, or storage account. This could be in the form of a pipeline extension point/parameter, a new imagebuilder command, a totally new pipeline/template, or some combination of those.