Consider using containerd's GRPC interface to pull OCI data down

[nathan-joslin]

Issue and related discussion copied from internal git instance

https://github.com/containerd/containerd/blob/main/api/services/content/v1/content.proto#L231

Then use the OCI layout content dir as the cache dir to construct the bottle from.

Discussion

Commenter 1:
Alternatively we could use the k8s CRI GRPC interface (can test this with crictl pull).
/var/lib/rancher/rke2/bin/crictl -r unix:///run/k3s/containerd/containerd.sock pull zot.lion.act3-ace.ai/ktarplee/bottle:v1 actually does pull an image. That image does not show up in kubectl get nodes -o yaml but that is expected. After a crictl pul we can extract the blobs from the OCI layout dir /var/lib/rancher/rke2/agent/containerd/io.containerd.content.v1.content.

To test directly with containerd you can use the ctr command. /var/lib/rancher/rke2/bin/ctr -a /run/k3s/containerd/containerd.sock -n k8s.io image pull zot.lion.act3-ace.ai/ktarplee/bottle:v3. All the data still goes to the same directory /var/lib/rancher/rke2/agent/containerd/io.containerd.content.v1.content. We can use a namespace other than the k8s.io namespace so we do not collide with k8s but we still benefit from the same content store.

/var/lib/rancher/rke2/bin/ctr -a /run/k3s/containerd/containerd.sock -n k8s.io.fake image list
REF                                     TYPE                                       DIGEST                                                                  SIZE     PLATFORMS LABELS 
zot.lion.act3-ace.ai/ktarplee/bottle:v2 application/vnd.oci.image.manifest.v1+json sha256:5ac8ad8eaaebedc0435b61e73bf3c6d48a60bd47ab445153e883f8b1d529d08d 33.5 KiB -         -      
zot.lion.act3-ace.ai/ktarplee/bottle:v3 application/vnd.oci.image.manifest.v1+json sha256:b6d960b512aefc2702bc60851b9a5abaa23009af48188137a4f6c6371676c553 33.8 KiB -         -  

Commenter 1:
I think this is a cool idea. I proved this morning that it would work. It would need to be an optional feature of the CSI driver, but having containerd (or any CRI) pull the OCI data for the CSI bottle driver allows us to share the same cache.

The fundamental idea here is that we need to have an abstraction for each part of a bottle pull (FYI @Njoslin):

1. downloading data (via oras, CRI, containerd, something else) to an OCI layout dir
2. read data from OCI and uppack it to "mount" a bottle. I think this step could technically be done within containerd if we had a reason to do so.

Commenter 1:
Related to this is to use a P2P network to pull images like DragonFly.