Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge repositories.json after extracting preloaded tarball so that reference store isn't lost #6985

Merged
merged 8 commits into from
Mar 12, 2020

Conversation

priyawadhwa
Copy link

@priyawadhwa priyawadhwa commented Mar 10, 2020

This PR adds a new Storage type which tracks the state of the repositories.json file, or reference store. This file is used by the docker daemon to store image references, and is overwritten when the preload tarball is extracted.

The Storage type is used to keep the state of the reference store in memory before and after extraction. The two reference stores are merged and then the file is updated.

This PR also adds a unit test and integration test to test this functionality.

should fix #6981

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 10, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 10, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 10, 2020
@codecov-io
Copy link

codecov-io commented Mar 10, 2020

Codecov Report

Merging #6985 into master will increase coverage by 0.21%.
The diff coverage is 11.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6985      +/-   ##
==========================================
+ Coverage   36.92%   37.13%   +0.21%     
==========================================
  Files         144      145       +1     
  Lines        9033     9123      +90     
==========================================
+ Hits         3335     3388      +53     
- Misses       5276     5309      +33     
- Partials      422      426       +4     
Impacted Files Coverage Δ
pkg/minikube/cruntime/containerd.go 44.54% <0.00%> (ø)
pkg/minikube/cruntime/crio.go 52.04% <0.00%> (ø)
pkg/minikube/cruntime/cruntime.go 57.50% <ø> (ø)
pkg/minikube/cruntime/docker.go 29.12% <0.00%> (-5.08%) ⬇️
pkg/minikube/machine/cache_images.go 0.00% <0.00%> (ø)
pkg/minikube/docker/store.go 23.33% <23.33%> (ø)
pkg/addons/addons.go 43.02% <0.00%> (-0.51%) ⬇️
pkg/util/utils.go 29.41% <0.00%> (ø)
pkg/minikube/tunnel/loadbalancer_patcher.go 70.93% <0.00%> (ø)
... and 5 more

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@priyawadhwa
do you mind pasting the in the PR description the After this PR, replicating the steps in the issue, to show it is solving the problem?

While this PR fixes the issue for the users with a new profile, I belive if users are upgrading from an older minikube to a newer minikube, the preload will still overwrite their own docker images.

after seeing the evidence that fixes the issue, I am good merging with this PR and this is a great improvement. (both fixing the bug and also perfomance) however I would like us to create another issue to track the users that will upgrade minikube and have existing user docker files.

the least is we should warn them that if they use preload they will loose their images.

@medyagh
Copy link
Member

medyagh commented Mar 10, 2020

I test this locally and it works for me

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ ./out/minikube start
😄  minikube v1.8.1 on Darwin 10.14.6
✨  Automatically selected the hyperkit driver
🔥  Creating hyperkit VM (CPUs=2, Memory=4096MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.17.3 on Docker 19.03.6 ...
🚀  Launching Kubernetes ... 
🌟  Enabling addons: default-storageclass, storage-provisioner
⌛  Waiting for cluster to come online ...
🏄  Done! kubectl is now configured to use "minikube"

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ eval $(minikube -p minikube docker-env)

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/kube-proxy v1.17.3 ae853e93800d 4 weeks ago 116MB
k8s.gcr.io/kube-apiserver v1.17.3 90d27391b780 4 weeks ago 171MB
k8s.gcr.io/kube-controller-manager v1.17.3 b0f1517c1f4b 4 weeks ago 161MB
k8s.gcr.io/kube-scheduler v1.17.3 d109c0821a2b 4 weeks ago 94.4MB
kubernetesui/dashboard v2.0.0-beta8 eb51a3597525 3 months ago 90.8MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
kindest/kindnetd 0.5.3 aa67fec7d7ef 4 months ago 78.5MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
kubernetesui/metrics-scraper v1.0.2 3b08661dc379 4 months ago 40.1MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 2 years ago 80.8MB

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ docker build -t medya /Desktop
Sending build context to Docker daemon 23.29MB
Step 1/2 : FROM alpine
latest: Pulling from library/alpine
c9b1b535fdd9: Pull complete
Digest: sha256:ab00606a42621fb68f2ed6ad3c88be54397f981a7b70a79db3d1172b11c4367d
Status: Downloaded newer image for alpine:latest
---> e7d92cdc71fe
Step 2/2 : RUN echo "HEllo world from medya" /hello.txt
---> Running in b69d2a42b78e
HEllo world from medya /hello.txt
Removing intermediate container b69d2a42b78e
---> 8acd76830413
Successfully built 8acd76830413
Successfully tagged medya:latest
medya@
/workspace/minikube (priyawadhwa-no-overwrite-tar) $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
medya latest 8acd76830413 21 seconds ago 5.59MB
k8s.gcr.io/kube-proxy v1.17.3 ae853e93800d 4 weeks ago 116MB
k8s.gcr.io/kube-apiserver v1.17.3 90d27391b780 4 weeks ago 171MB
k8s.gcr.io/kube-controller-manager v1.17.3 b0f1517c1f4b 4 weeks ago 161MB
k8s.gcr.io/kube-scheduler v1.17.3 d109c0821a2b 4 weeks ago 94.4MB
alpine latest e7d92cdc71fe 7 weeks ago 5.59MB
kubernetesui/dashboard v2.0.0-beta8 eb51a3597525 3 months ago 90.8MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
kindest/kindnetd 0.5.3 aa67fec7d7ef 4 months ago 78.5MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
kubernetesui/metrics-scraper v1.0.2 3b08661dc379 4 months ago 40.1MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 2 years ago 80.8MB

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ ./out/minikube start
😄 minikube v1.8.1 on Darwin 10.14.6
▪ MINIKUBE_ACTIVE_DOCKERD=minikube
✨ Using the hyperkit driver based on existing profile
⌛ Reconfiguring existing host ...
🏃 Using the running hyperkit "minikube" VM ...
🐳 Preparing Kubernetes v1.17.3 on Docker 19.03.6 ...
🚀 Launching Kubernetes ...
🌟 Enabling addons: default-storageclass, storage-provisioner
🏄 Done! kubectl is now configured to use "minikube"

medya@~/workspace/minikube (priyawadhwa-no-overwrite-tar) $ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
medya latest 8acd76830413 59 seconds ago 5.59MB
k8s.gcr.io/kube-proxy v1.17.3 ae853e93800d 4 weeks ago 116MB
k8s.gcr.io/kube-controller-manager v1.17.3 b0f1517c1f4b 4 weeks ago 161MB
k8s.gcr.io/kube-apiserver v1.17.3 90d27391b780 4 weeks ago 171MB
k8s.gcr.io/kube-scheduler v1.17.3 d109c0821a2b 4 weeks ago 94.4MB
alpine latest e7d92cdc71fe 7 weeks ago 5.59MB
kubernetesui/dashboard v2.0.0-beta8 eb51a3597525 3 months ago 90.8MB
k8s.gcr.io/coredns 1.6.5 70f311871ae1 4 months ago 41.6MB
kindest/kindnetd 0.5.3 aa67fec7d7ef 4 months ago 78.5MB
k8s.gcr.io/etcd 3.4.3-0 303ce5db0e90 4 months ago 288MB
kubernetesui/metrics-scraper v1.0.2 3b08661dc379 4 months ago 40.1MB
k8s.gcr.io/pause 3.1 da86e6ba6ca1 2 years ago 742kB
gcr.io/k8s-minikube/storage-provisioner v1.8.1 4689081edb10 2 years ago 80.8MB

@tstromberg
Copy link
Contributor

tstromberg commented Mar 10, 2020

PR looks like a great optimization and approvable in its own right, but I'm not certain it fully fixes #6981

For instance, if a user has a v1.16.0 Kubernetes cluster with custom images, won't minikube start --kubernetes-version=v1.17.3 still result in docker image metadata being lost?

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 11, 2020
@medyagh
Copy link
Member

medyagh commented Mar 11, 2020

ImagesPreloaded(r.Runner, images)

should be able to handle based on different type of runtimes, docker, cri-o, containerd.

@priyawadhwa
Copy link
Author

@medyagh agreed, I'll probably do that in another PR or when I start to look at adding the other container runtimes

Copy link
Contributor

@tstromberg tstromberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks well done, just some minor nits.

@tstromberg
Copy link
Contributor

PR title could use a more accurate take

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to look at adding the o

sounds good, I am good with with PR, once @tstromberg 's comments are resolved :)

pkg/minikube/cruntime/docker.go Outdated Show resolved Hide resolved
test/integration/start_stop_delete_test.go Show resolved Hide resolved
@priyawadhwa priyawadhwa changed the title Only extract preloaded tarball if images don't exist in daemon Merge repositories.json after extracting preloaded tarball so that reference store isn't lost Mar 11, 2020
@medyagh
Copy link
Member

medyagh commented Mar 11, 2020

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Mar 11, 2020
@minikube-pr-bot
Copy link

Error: running mkcmp: exit status 1

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 11, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 11, 2020
@tstromberg
Copy link
Contributor

/ok-to-test

@minikube-pr-bot
Copy link

All Times minikube: [ 60.053562 61.222229 63.681110]
All Times Minikube (PR 6985): [ 60.112136 59.771807 62.957849]

Average minikube: 61.652300
Average Minikube (PR 6985): 60.947264

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6985) |
+----------------------+-----------+--------------------+
| minikube v           |  0.204763 |           0.197881 |
| Creating kvm2        | 40.120619 |          40.510583 |
| Preparing Kubernetes |  0.738196 |           0.721775 |
| Pulling images       |           |                    |
| Launching Kubernetes | 18.975475 |          18.308507 |
| Waiting for cluster  |  0.069116 |           0.072374 |
+----------------------+-----------+--------------------+

@medyagh
Copy link
Member

medyagh commented Mar 12, 2020

Update, that bug bellow is not related to this PR. I will create an issue for it.

~~
@tstromberg
I see an issue, please do not merge this PR till I investigate if it is related to this PR

to replicate,

./out/minikube start

do and export the vars

./out/minikube docker-env

then minikube start again
(minikube warns you to reeval the docker-env and refresh your docker-env)

then
$ docker ps
error during connect: Get https://127.0.0.1:32853/v1.40/containers/json: EOF
~~

@medyagh medyagh merged commit 980e3ab into kubernetes:master Mar 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

preload removes images built by user in docker-env
6 participants