Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Images imported from airgap tarballs can be deleted by kubelet GC #8781

Closed
brandond opened this issue Nov 3, 2023 Discussed in #8779 · 3 comments
Closed

Images imported from airgap tarballs can be deleted by kubelet GC #8781

brandond opened this issue Nov 3, 2023 Discussed in #8779 · 3 comments
Assignees
@brandond @aganesh-suse
Labels
priority/high
Milestone
v1.29.1+k3s1

Comments

@brandond
Copy link
Member

brandond commented Nov 3, 2023

Discussed in #8779

Originally posted by sushiMix November 3, 2023
Hello, I know there are several tickets about it but didn't find a clear view.

Accordingly to this ticket #3193 it seems working from k3s 1.24
but #7569 there is no possible way to protect images.
Accordingly to test It seems still not working.

But I have seen tickets in k8s/containerd :
containerd/containerd#6352 and containerd/containerd#6930

and did a small test:

Use ctr to load an image and add the pinning label
sudo ./ctr images label "my image" io.cri-containerd.pinned=pinned

Fill the disk using xfs_mkfile like the first ticket

The image with the pin tag is no more freed when GC is run (like the pause image :)).
I'm using k3s version v1.26.9+k3s1.

Isn't it a solution to protect some images from GC ?
@brandond
Copy link
Member Author

brandond commented Nov 3, 2023

yes, the leases that we added a while back only protected the layer content, they did not protect the tags as hoped. We should rework our protections to use the CRI pinned label.

@brandond brandond added this to the v1.28.4+k3s1 milestone Nov 3, 2023
@brandond brandond self-assigned this Nov 3, 2023
@Slowdive-Aideron
Copy link

Thank you for this. This is still very much relevant !

@brandond brandond changed the title Air gap installation and GC Images imported from airgap tarballs can be deleted by kubelet GC Dec 8, 2023
@brandond brandond mentioned this issue Dec 8, 2023
Merged
11 tasks
This was referenced Jan 9, 2024
Closed
Closed
Closed
@brandond brandond mentioned this issue Jan 17, 2024
Closed
@aganesh-suse
Copy link

aganesh-suse commented Jan 26, 2024 •
edited
Loading

Validated on master branch with version v1.29.1-rc1+k3s1

Environment Details

Infrastructure

  • Cloud
  • Hosted

Node(s) CPU architecture, OS, and Version:

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"

$ uname -m
x86_64

Note: the VM size allocation was intentionally kept small for the test at 8GB.

Cluster Configuration:

1 server/ 1 agent

Config.yaml:

token: xxxx
embedded-registry: true
disable-default-registry-endpoint: true
debug: true
write-kubeconfig-mode: "0644"

registries.yaml:

mirrors:
  docker.io:
    endpoint:
      - https://test.compute.amazonaws.com
configs:
  test.compute.amazonaws.com:
    auth:
      username: testuser
      password: password
    tls:
      ca_file: /home/ubuntu/ca.pem

Testing Steps

  1. Copy config.yaml
$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s
  1. Install k3s
curl -sfL https://get.k3s.io | sudo INSTALL_K3S_VERSION='v1.29.1-rc1+k3s1' sh -s - server
  1. Verify Cluster Status:
kubectl get nodes -o wide
kubectl get pods -A
  1. Fill up the disk with: xfs_mkfile 6g dummy
  2. Check the image labels to have "pinned" and all pinned images retained.
sudo k3s crictl image ls
sudo k3s ctr image ls

Replication Results:

  • k3s version used for replication:
$ k3s -v 
k3s version v1.29.0+k3s1 (3190a5fa)
go version go1.21.5

Before filling up the disc:

kubectl get nodes; kubectl get pods -A
ip-172-31-26-106   Ready    control-plane,master   43s   v1.29.0+k3s1
NAMESPACE      NAME                                      READY   STATUS      RESTARTS   AGE
kube-system    coredns-6799fbcd5-8q4f7                   1/1     Running     0          47s
kube-system    local-path-provisioner-84db5d44d9-v9zlk   1/1     Running     0          47s
kube-system    helm-install-traefik-crd-jhgwk            0/1     Completed   0          47s
kube-system    svclb-traefik-801305a7-l2d5b              2/2     Running     0          27s
kube-system    helm-install-traefik-cz5qh                0/1     Completed   1          47s
kube-system    traefik-f4564c4f4-6vstq                   1/1     Running     0          27s
kube-system    metrics-server-67c658944b-xltrx           1/1     Running     0          47s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-f92pv             1/1     Running     0          10s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-9lw86             1/1     Running     0          10s

$ xfs_mkfile 6g dummy; df -h 
pwrite: No space left on device
Filesystem       Size  Used Avail Use% Mounted on
/dev/root        7.6G  7.6G   76K 100% /
tmpfs            1.9G     0  1.9G   0% /dev/shm
tmpfs            770M  1.6M  769M   1% /run
tmpfs            5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p15  105M  6.1M   99M   6% /boot/efi
tmpfs            385M  4.0K  385M   1% /run/user/1000
shm               64M     0   64M   0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/06c0596060fdd43a91d2f429eea2c207829e46e976adca6538e0f603fef9ff1f/shm
shm               64M     0   64M   0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/c8ccbb5f20f647c71401893a120dcc0b940d435591a9179b134d6f7c95b5d8cf/shm
shm               64M     0   64M   0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/9c183489df329ec59c47378c2a47a29bdaac0b6e1b06e0230813b64856970026/shm
shm               64M     0   64M   0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/755eb417e4fd87871b1df71907178004d4daae42993bc66bec011f274cd55f73/shm
$ kubectl get nodes; kubectl get pods -A 
NAME               STATUS   ROLES                  AGE   VERSION
ip-172-31-26-106   Ready    control-plane,master   4m    v1.29.0+k3s1
NAMESPACE      NAME                                      READY   STATUS      RESTARTS   AGE
kube-system    coredns-6799fbcd5-8q4f7                   1/1     Running     0          4m4s
kube-system    local-path-provisioner-84db5d44d9-v9zlk   1/1     Running     0          4m4s
kube-system    helm-install-traefik-crd-jhgwk            0/1     Completed   0          4m4s
kube-system    helm-install-traefik-cz5qh                0/1     Completed   1          4m4s
kube-system    traefik-f4564c4f4-6vstq                   1/1     Running     0          3m44s
kube-system    metrics-server-67c658944b-xltrx           1/1     Running     0          4m4s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-wkcpz             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-9lw86             0/1     Completed   0          3m27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-4xss6             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-brgv7             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-dphpg             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-qbx5r             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-xv9sm             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-hh44q             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-jdfdb             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-rmfhh             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-gmcgw             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-cwqx7             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-qcnm9             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-527j2             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-2694w             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-d82h8             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-mgns8             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-tldxv             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-kpxpt             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-gzrnq             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-4qv2t             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-f92pv             0/1     Completed   0          3m27s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-ffgtr             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-4d459             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-tgf4l             0/1     Evicted     0          26s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-6kvlw             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-x4r79             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-bbf5v             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-hnqm2             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-9g85c             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-phnmn             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-x7s26             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-hwwv6             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-72dn9             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-tmpp4             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-8scpc             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-nwvj7             0/1     Evicted     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-5bf27             0/1     Pending     0          25s
pvt-reg-test   pvt-reg-test-5b7b5fc6d5-7jqrt             0/1     Pending     0          24s
kube-system    svclb-traefik-801305a7-ql5jz              0/2     Evicted     0          9s

$ sudo k3s ctr image ls 
REF                                                                                                                          TYPE                                                 DIGEST                                                                  SIZE      PLATFORMS   LABELS
docker.io/rancher/local-path-provisioner:v0.0.24                                                                             application/vnd.docker.distribution.manifest.v2+json sha256:0c39d541b3696f919b8a59098766b5407ee2ccd956070fa4d1148437e0974c00 38.6 MiB  linux/amd64 io.cri-containerd.image=managed
docker.io/rancher/mirrored-coredns-coredns:1.10.1                                                                            application/vnd.docker.distribution.manifest.v2+json sha256:b64ac4c9100dd8ba255f2a31c2ad165f062e2cf087af9b6ffc68910e90de55d8 51.1 MiB  linux/amd64 io.cri-containerd.image=managed
docker.io/rancher/mirrored-library-traefik:2.10.5                                                                            application/vnd.docker.distribution.manifest.v2+json sha256:3883ce8abfbd8d158bcc7cfa876fb2ff6b9c49221518d7c666eb4ed9022ec919 145.7 MiB linux/amd64 io.cri-containerd.image=managed
docker.io/rancher/mirrored-metrics-server:v0.6.3                                                                             application/vnd.docker.distribution.manifest.v2+json sha256:f46cf066256b7d95f1d7e428f9014f27955be2fa67ee9793f0ae9199fd354fa3 67.0 MiB  linux/amd64 io.cri-containerd.image=managed
test.compute.amazonaws.com/nginx:latest                                                                  application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
test.compute.amazonaws.com/nginx@sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:817bbe3f2e5179b5fa7dd4085c87aab1faaf9dc879e3b1d05796901a006f58ab                                                      application/vnd.docker.distribution.manifest.v2+json sha256:f46cf066256b7d95f1d7e428f9014f27955be2fa67ee9793f0ae9199fd354fa3 67.0 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:a8758716bb6aa4d90071160d27028fe4eaee7ce8166221a97d30440c8eac2be6                                                      application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:b29384aeb4b13e047448ccfd312c52b4d023abcbbaafcab174293a97821dddb0                                                      application/vnd.docker.distribution.manifest.v2+json sha256:0c39d541b3696f919b8a59098766b5407ee2ccd956070fa4d1148437e0974c00 38.6 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:cc365cbb0397b708abc260501574769de3891d756c7a2e43c3839e9126cf9aec                                                      application/vnd.docker.distribution.manifest.v2+json sha256:3883ce8abfbd8d158bcc7cfa876fb2ff6b9c49221518d7c666eb4ed9022ec919 145.7 MiB linux/amd64 io.cri-containerd.image=managed
sha256:ead0a4a53df89fd173874b46093b6e62d8c72967bbf606d672c9e8c9b601a4fc                                                      application/vnd.docker.distribution.manifest.v2+json sha256:b64ac4c9100dd8ba255f2a31c2ad165f062e2cf087af9b6ffc68910e90de55d8 51.1 MiB  linux/amd64 io.cri-containerd.image=managed

$ sudo k3s crictl image ls 
IMAGE                                                  TAG                 IMAGE ID            SIZE
docker.io/rancher/local-path-provisioner               v0.0.24             b29384aeb4b13       40.4MB
docker.io/rancher/mirrored-coredns-coredns             1.10.1              ead0a4a53df89       53.6MB
docker.io/rancher/mirrored-library-traefik             2.10.5              cc365cbb0397b       153MB
docker.io/rancher/mirrored-metrics-server              v0.6.3              817bbe3f2e517       70.3MB
test.compute.amazonaws.com/nginx   latest              a8758716bb6aa       70.5MB

Validation Results:

  • k3s version used for validation:
$ k3s -v
k3s version v1.29.1-rc1+k3s1 (d8907ce6)
go version go1.21.6

Before filling up the disc:

$ kubectl get nodes; kubectl get pods -A 
NAME               STATUS   ROLES                  AGE   VERSION
ip-172-31-18-174   Ready    control-plane,master   49s   v1.29.1-rc1+k3s1
ip-172-31-28-211   Ready    <none>                 57s   v1.29.1-rc1+k3s1
NAMESPACE      NAME                                      READY   STATUS      RESTARTS   AGE
kube-system    local-path-provisioner-84db5d44d9-2mpv4   1/1     Running     0          55s
kube-system    coredns-6799fbcd5-qkrpt                   1/1     Running     0          55s
kube-system    helm-install-traefik-crd-rqz2h            0/1     Completed   0          55s
kube-system    svclb-traefik-34db3aa5-nnwmb              2/2     Running     0          38s
kube-system    svclb-traefik-34db3aa5-b7bg6              2/2     Running     0          38s
kube-system    helm-install-traefik-h7v2w                0/1     Completed   1          55s
kube-system    traefik-f4564c4f4-v5j5j                   1/1     Running     0          38s
kube-system    metrics-server-67c658944b-ksch5           1/1     Running     0          55s
pvt-reg-test   pvt-reg-test-84b8888b89-78ckz             1/1     Running     0          11s
pvt-reg-test   pvt-reg-test-84b8888b89-nnr25             1/1     Running     0          11s

After:

xfs_mkfile 6g dummy; df -h 
pwrite: No space left on device
Filesystem       Size  Used Avail Use% Mounted on
/dev/root        7.6G  7.6G   24K 100% /
tmpfs            1.9G     0  1.9G   0% /dev/shm
tmpfs            770M  1.1M  769M   1% /run
tmpfs            5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p15  105M  6.1M   99M   6% /boot/efi
tmpfs            385M  4.0K  385M   1% /run/user/1000
shm               64M     0   64M   0% /run/k3s/containerd/io.containerd.grpc.v1.cri/sandboxes/955dd4aa73f71560bfa7ad019056339069877d710495ed40fc3295bc6f0b557f/shm
$  kubectl get nodes; kubectl get pods -A 
NAME               STATUS   ROLES                  AGE     VERSION
ip-172-31-28-211   Ready    <none>                 4m39s   v1.29.1-rc1+k3s1
ip-172-31-18-174   Ready    control-plane,master   4m31s   v1.29.1-rc1+k3s1
NAMESPACE      NAME                                      READY   STATUS      RESTARTS   AGE
kube-system    local-path-provisioner-84db5d44d9-2mpv4   1/1     Running     0          4m36s
kube-system    coredns-6799fbcd5-qkrpt                   1/1     Running     0          4m36s
kube-system    helm-install-traefik-crd-rqz2h            0/1     Completed   0          4m36s
kube-system    svclb-traefik-34db3aa5-b7bg6              2/2     Running     0          4m19s
kube-system    helm-install-traefik-h7v2w                0/1     Completed   1          4m36s
kube-system    traefik-f4564c4f4-v5j5j                   1/1     Running     0          4m19s
kube-system    metrics-server-67c658944b-ksch5           1/1     Running     0          4m36s
pvt-reg-test   pvt-reg-test-84b8888b89-nnr25             1/1     Running     0          3m52s
pvt-reg-test   pvt-reg-test-84b8888b89-qhthq             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-qsvrg             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-9c6rc             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-mtbs2             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-s829j             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-zgg8b             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-qss47             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-r57nr             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-d6rcq             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-78ckz             0/1     Completed   0          3m52s
pvt-reg-test   pvt-reg-test-84b8888b89-vkhtg             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-x92fd             0/1     Evicted     0          30s
pvt-reg-test   pvt-reg-test-84b8888b89-fcjkl             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-zn45x             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-ndjxc             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-l4nzm             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-9f7tj             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-shwnn             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-6tvqf             0/1     Evicted     0          29s
pvt-reg-test   pvt-reg-test-84b8888b89-djnjj             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-j8pmf             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-xvfv5             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-g454s             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-f4ndb             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-7hbd2             0/1     Evicted     0          28s
pvt-reg-test   pvt-reg-test-84b8888b89-5jrjd             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-5xv4p             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-czk9x             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-jnmx4             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-6fds8             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-bq2xp             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-jswls             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-27xm5             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-z96c2             0/1     Evicted     0          27s
pvt-reg-test   pvt-reg-test-84b8888b89-6swfs             1/1     Running     0          27s
kube-system    svclb-traefik-34db3aa5-87r4s              0/2     Evicted     0          13s

$ sudo k3s ctr image ls 
REF                                                                                                                             TYPE                                                 DIGEST                                                                  SIZE      PLATFORMS   LABELS
docker.io/rancher/klipper-helm:v0.8.2-build20230815                                                                             application/vnd.docker.distribution.manifest.v2+json sha256:9f6b0a352533fe34763f81f014952f0595b9bd2ad531b179767c81ef77172668 244.5 MiB linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/klipper-lb:v0.4.5                                                                                             application/vnd.docker.distribution.manifest.v2+json sha256:370ba55e8e4731a6e54d9c9ca8c99b517a2c00c8733499f0088d38bb141daeff 18.0 MiB  linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/local-path-provisioner:v0.0.24                                                                                application/vnd.docker.distribution.manifest.v2+json sha256:0c39d541b3696f919b8a59098766b5407ee2ccd956070fa4d1148437e0974c00 38.6 MiB  linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/mirrored-coredns-coredns:1.10.1                                                                               application/vnd.docker.distribution.manifest.v2+json sha256:b64ac4c9100dd8ba255f2a31c2ad165f062e2cf087af9b6ffc68910e90de55d8 51.1 MiB  linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/mirrored-library-busybox:1.36.1                                                                               application/vnd.docker.distribution.manifest.v2+json sha256:cd34073b33f0becaa9d547f9db827b55ccdd875d06459cbd5545c4c35512573d 4.3 MiB   linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/mirrored-library-traefik:2.10.5                                                                               application/vnd.docker.distribution.manifest.v2+json sha256:3883ce8abfbd8d158bcc7cfa876fb2ff6b9c49221518d7c666eb4ed9022ec919 145.7 MiB linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/mirrored-metrics-server:v0.6.3                                                                                application/vnd.docker.distribution.manifest.v2+json sha256:f46cf066256b7d95f1d7e428f9014f27955be2fa67ee9793f0ae9199fd354fa3 67.0 MiB  linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
docker.io/rancher/mirrored-pause:3.6                                                                                            application/vnd.docker.distribution.manifest.v2+json sha256:79b611631c0d19e9a975fb0a8511e5153789b4c26610d1842e9f735c57cc8b13 669.8 KiB linux/amd64 io.cattle.k3s.pinned=pinned,io.cri-containerd.image=managed,io.cri-containerd.pinned=pinned
test.compute.amazonaws.com/nginx:latest                                                                  application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
test.compute.amazonaws.com/nginx@sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:17066c233afa320bfba42ee5085e5cf767b1b2739562b40950267f416e952231                                                         application/vnd.docker.distribution.manifest.v2+json sha256:370ba55e8e4731a6e54d9c9ca8c99b517a2c00c8733499f0088d38bb141daeff 18.0 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:5f89cb8137ccbd39377d91b9d75faf4ec4ee0a2d2a3a63635535b10c69c935fa                                                         application/vnd.docker.distribution.manifest.v2+json sha256:9f6b0a352533fe34763f81f014952f0595b9bd2ad531b179767c81ef77172668 244.5 MiB linux/amd64 io.cri-containerd.image=managed
sha256:6270bb605e12e581514ada5fd5b3216f727db55dc87d5889c790e4c760683fee                                                         application/vnd.docker.distribution.manifest.v2+json sha256:79b611631c0d19e9a975fb0a8511e5153789b4c26610d1842e9f735c57cc8b13 669.8 KiB linux/amd64 io.cri-containerd.image=managed
sha256:817bbe3f2e5179b5fa7dd4085c87aab1faaf9dc879e3b1d05796901a006f58ab                                                         application/vnd.docker.distribution.manifest.v2+json sha256:f46cf066256b7d95f1d7e428f9014f27955be2fa67ee9793f0ae9199fd354fa3 67.0 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:9211bbaa0dbd68fed073065eb9f0a6ed00a75090a9235eca2554c62d1e75c58f                                                         application/vnd.docker.distribution.manifest.v2+json sha256:cd34073b33f0becaa9d547f9db827b55ccdd875d06459cbd5545c4c35512573d 4.3 MiB   linux/amd64 io.cri-containerd.image=managed
sha256:a8758716bb6aa4d90071160d27028fe4eaee7ce8166221a97d30440c8eac2be6                                                         application/vnd.docker.distribution.manifest.v2+json sha256:9ceac672f55cdad7316651fcced0d6a1ba44e87454ec86236cb3bca647e419f6 67.2 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:b29384aeb4b13e047448ccfd312c52b4d023abcbbaafcab174293a97821dddb0                                                         application/vnd.docker.distribution.manifest.v2+json sha256:0c39d541b3696f919b8a59098766b5407ee2ccd956070fa4d1148437e0974c00 38.6 MiB  linux/amd64 io.cri-containerd.image=managed
sha256:cc365cbb0397b708abc260501574769de3891d756c7a2e43c3839e9126cf9aec                                                         application/vnd.docker.distribution.manifest.v2+json sha256:3883ce8abfbd8d158bcc7cfa876fb2ff6b9c49221518d7c666eb4ed9022ec919 145.7 MiB linux/amd64 io.cri-containerd.image=managed
sha256:ead0a4a53df89fd173874b46093b6e62d8c72967bbf606d672c9e8c9b601a4fc                                                         application/vnd.docker.distribution.manifest.v2+json sha256:b64ac4c9100dd8ba255f2a31c2ad165f062e2cf087af9b6ffc68910e90de55d8 51.1 MiB  linux/amd64 io.cri-containerd.image=managed

$ sudo k3s crictl image ls 
IMAGE                                                     TAG                    IMAGE ID            SIZE
docker.io/rancher/klipper-helm                            v0.8.2-build20230815   5f89cb8137ccb       256MB
docker.io/rancher/klipper-lb                              v0.4.5                 17066c233afa3       18.9MB
docker.io/rancher/local-path-provisioner                  v0.0.24                b29384aeb4b13       40.4MB
docker.io/rancher/mirrored-coredns-coredns                1.10.1                 ead0a4a53df89       53.6MB
docker.io/rancher/mirrored-library-busybox                1.36.1                 9211bbaa0dbd6       4.49MB
docker.io/rancher/mirrored-library-traefik                2.10.5                 cc365cbb0397b       153MB
docker.io/rancher/mirrored-metrics-server                 v0.6.3                 817bbe3f2e517       70.3MB
docker.io/rancher/mirrored-pause                          3.6                    6270bb605e12e       686kB
test.compute.amazonaws.com/nginx   latest                 a8758716bb6aa       70.5MB

Additional context / logs:

@vitality411 vitality411 mentioned this issue Feb 9, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brandond brandond

@aganesh-suse aganesh-suse

Labels
priority/high
Projects
K3s Development
Archived in project
Milestone
v1.29.1+k3s1
Development

No branches or pull requests
4 participants
@brandond @Slowdive-Aideron @caroline-suse-rancher @aganesh-suse