Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add code for downloading kicbase image to cache #10918

Merged
merged 4 commits into from
Apr 26, 2021

Conversation

afbjorklund
Copy link
Collaborator

@afbjorklund afbjorklund commented Mar 24, 2021

Download the image to the cache, instead of to the daemon.

This way it also works for Podman, and not only for Docker

Left to decide:

  1. Should we store the KIC images in a separate directory
    Before they were mixed with the container runtime images

  2. Delete the old Docker-only code, only use the cache ?
    This means not using the digest, always load from files

Currently the image is 354M on the disk (gz-compressed)

Closes #10912 #7766

Download the image to the cache, instead of to the daemon.

This way it also works for Podman, and not only for Docker
@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 24, 2021
@afbjorklund afbjorklund changed the title WIP: Add code for downloading kicbase image to cache Add code for downloading kicbase image to cache Mar 30, 2021
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 30, 2021
@ilya-zuyev
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Apr 7, 2021
@minikube-pr-bot
Copy link

kvm2 Driver
error collecting results for kvm2 driver: timing run 0 with Minikube (PR 10918): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/10918/minikube start --driver=kvm2]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/10918/minikube: exec format error
docker Driver
error collecting results for docker driver: timing run 0 with Minikube (PR 10918): timing cmd: [/home/performance-monitor/.minikube/minikube-binaries/10918/minikube start --driver=docker]: starting cmd: fork/exec /home/performance-monitor/.minikube/minikube-binaries/10918/minikube: exec format error

@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Apr 7, 2021

Note that in order to make this work for Podman, we need to fix LoadFromTarball to be compatible

     case driver.Podman:
               return fmt.Errorf("not yet implemented, see issue #8426")

It would also be nice to avoid loading again, if the image is already present in the container engine...

For this to happen, we need to use something other than go-containerregistry.

Since it only supports Docker, as can be seen indirectly in the daemon API.

#8426 #8577

@afbjorklund
Copy link
Collaborator Author

But the current daemon.Write is quite stupid, so not too hard to re-implement with podman:

https://github.com/google/go-containerregistry/blob/main/pkg/v1/daemon/write.go#L57

Instead of using github.com/docker/docker/client, one could just fork a sudo podman ?

If we want to implement the lookup, we would also need something instead of daemon.Image

This is the other end, so instead of | sudo podman load it would need sudo podman save |


// Write saves the image into the daemon as the given tag.
func Write(tag name.Tag, img v1.Image) (string, error)

// Tag adds a tag to an already existent image.
func Tag(src, dest name.Tag) error

// Image provides access to an image reference from the Docker daemon,
// applying functional options to the underlying imageOpener before
// resolving the reference into a v1.Image.
func Image(ref name.Reference, options ...ImageOption) (v1.Image, error)

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 10918) |
+----------------+----------+---------------------+
| minikube start | 49.1s    | 49.8s               |
| enable ingress | 35.3s    | 35.7s               |
+----------------+----------+---------------------+

Times for minikube ingress: 35.3s 35.8s 34.3s 36.8s 34.2s
Times for minikube (PR 10918) ingress: 34.3s 36.8s 35.7s 36.2s 35.3s

Times for minikube start: 48.5s 49.7s 51.7s 47.4s 47.9s
Times for minikube (PR 10918) start: 50.6s 47.6s 47.2s 55.2s 48.2s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 10918) |
+----------------+----------+---------------------+
| minikube start | 21.5s    | 21.7s               |
| enable ingress | 29.7s    | 31.8s               |
+----------------+----------+---------------------+

Times for minikube ingress: 28.0s 29.0s 28.5s 34.0s 29.0s
Times for minikube (PR 10918) ingress: 28.9s 30.0s 36.0s 31.5s 32.5s

Times for minikube start: 23.2s 20.6s 21.6s 21.3s 20.8s
Times for minikube (PR 10918) start: 21.4s 21.6s 21.7s 21.5s 22.5s

docker driver with containerd runtime
error collecting results for docker driver: timing run 0 with minikube: timing cmd: [out/minikube addons enable ingress]: waiting for minikube: exit status 10

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [afbjorklund,medyagh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@medyagh
Copy link
Member

medyagh commented Apr 26, 2021

Looks good to me , Thank you for this PR

@medyagh medyagh merged commit d6e011b into kubernetes:master Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

--download-only should store/load kicbase on local filesystem
5 participants