Support referencing a Layer by Digest

[baronfel]

Part of #316.

Layers for an image can come from many places - the base image, the direct content of the project being packaged, or an arbitrary registry. We should support defining new Layers to be included in the final image via MSBuild, and how those layers should be retrieved and included in the final layer.

MSBuild

A Layer reference consists of three pieces of data: the Repository the layer belongs to, the Registry the Repository is stored in, and the Digest of the layer itself. We should support defining this in MSBuild. A proposed form of this could be:

<ItemGroup>
  <ContainerLayer Include="<digest value>" Repository="<repository name>" Registry="<registry reference>" />
</ItemGroup>

For example:

<ItemGroup>
  <ContainerLayer Include="sha256:b2b4751952d24fa810a91620aee5f49a1cdf7d05b472a209920f3310f1a84bc1" Repository="dotnet/aspnet" Registry="mcr.microsoft.com" />
</ItemGroup>

These Layers would be passed into the CreateNewImage Task and included in the image.

API

The Layer class has several static factories, and the end result is making a combination of a Descriptor for the Layer and assigning a BackingFile for the layer (which can be done from the descriptor). The data from the Descriptor should be able to be retrieved by issuing a HEAD request to the v2/{repository}/{digest} endpoint - the GET at that same location returns the content while the HEAD would just give the metadata.

Once the metadata is known, the actual Registry.Push/LocalDaemon.Load commands would be responsible for downloading and moving the layer across registries. This should be something already done by our current codepaths.

Open question

• Should we add a Type metadata to the ContainerLayer to give us a pivot for future evolution?
• Should we add more kinds of ContainerLayer creation (e.g. arbitrary filesystem globs)?

[richlander]

This proposed scenario seems wasteful: dotnet/dotnet-docker#5641 (reply in thread). Maintaining a base image that only additionally contains a NuGet package seems like a bad tradeoff of effort. Ideally, SDK publish could build multiple layers, one of the app and another for any NuGet packages that should be installed, particularly tools, installed to the regular places.

[baronfel]

Chatting with @danegsta and team today, another use case came up - seamlessly providing VS debugger binaries to make the container debug experience magical would be a great help for them.

[richlander]

A lot of people install dotnet-* tools in container images. I see them install them via the tool path approach and then copy them. That works and is fine in Dockerfile land. It doesn't work well for container builds, obviously. I also wonder if the tools were available in a registry-addressible image that the Dockerfile folks might be better served, too.