preload causes /var conflicts with Docker Desktop File sharing #8100
Assignees
Labels
co/docker-driver
Issues related to kubernetes in container
co/preload
kind/bug
Categorizes issue or PR as related to a bug.
priority/important-soon
Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
triage/duplicate
Indicates an issue is a duplicate of other open issue.
Milestone
Comments
|
@plnordquist thanks for creaitng this issue, I am not sure what is the root cause of this, but have u tried minikube delete and start again ? I am wondering if minikube is trying to reuse the older version of container that had a different image I also see this: that makes me believe the container was not able to shutdown and it was stuck, you might want to restart docker. do you mind sharing how much RAM your docker desktop has ? |
medyagh
added
co/docker-driver
|
My docker desktop has 2GB of ram. If I factory reset my Docker Desktop, minikube can start successfully. Once I minikube stop, minikube delete, and minikube start again, it fails to start with the same container logs I posted in the initial post. I upgraded to v1.10.1 to test this again and I'm seeing the same behavior. Here's some more logs of the successful start and the failed start. I'm running Docker Desktop Edge v2.3.0.1 with the Hyper-V backend. In the failure scenario, Good Good Good Bad Bad |
|
@plnordquist minikube needs min 2GB ram, and your docker desktop has only 2 GB ram shared across all docker containers, I suggest resize your docker desktop to have at least 4 or 6GB ram and try again. plus the logs from your second start shows that the start was actually healthy ! |
|
OK I updated the ram for Docker Desktop to 6GB and that didn't work. I found another issue like this one at #7885 and then went down a rabbit hole trying to figure this out since factory resetting Docker Desktop didn't work 100% of the time. The behavior that works is if I clear the shared preloaded tarball from the file sharing configuration of Docker Desktop and then cancel the request to share the tarball to the container, minikube can start successfully since it uses scp later in the process to copy the tarball into the running container. I think this might have started in commit b509d691 since it changed the order of operations. Prior to that commit the main minikube container would have been started and the minikube volume would be initialized from the In the current version of the code beyond the commit referenced above, the Podman driver doesn't even attempt to cache the preloaded images so it avoids this issue and thus #7885 no longer applies. I've attached a set of logs that show the error that appears when I cancel the file sharing request, it is not fatal to the Good |
@plnordquist interesting!!! thank you very much for providing this amount of detail ! good detective work ! so if you accept the Docke Desktop File Sharing then it wont work ? and it works when you disable the file sharing ? I wonder what happens if u disable preload with the file sharing |
|
if you disable preload, would it work without any issues ? |
|
Yes using |
medyagh
changed the title
|
thank you @plnordquist for reporting this, this is a bug. I wonder if we can find a Non-UI way to prevent docker to asking to share the folder with Docker Desktop ? I also seen that notification on windows, whe minikube starts, docker desktop Asks to share file |
medyagh
added
kind/bug
|
Seems like the same type of race conditions, that we have with the podman driver. (see #8056) I think the best long-term solution would be to just stop mounting all of |
|
My suggestion is to move /var into a subdirectory of the volume, and then mount it somewhere. The same we do it for the ISO. This also gives us a place to fix the storage driver persistence... |
This was referenced May 14, 2020
|
here is full log |
|
@afbjorklund the current two docker run commands are these: and if you can explain how we can do it without race condition , we could implement that way |
|
@medyagh : If we use a custom mountpoint also in the first run, then the /var files will stay in image... But just like on the VM, we then need to creating the necessary symlinks or bindmounts to the mount |
|
@afbjorklund I think I'm seeing this error trying to run minikube in Cloud Shell and it seems related to #8163 as well. WDYT of changing docker's home directory? I'm not super familiar with this issue, but I'm wondering if that would resolve it. If we changed from Do you know if this would break anything? It seems to work fine testing locally on my Mac & in Cloud Shell. |
This was referenced Jun 22, 2020
|
I don't think we will use a custom mountpoint initially, but just address the race. It is likely that it is the /extractDir that is the real issue here, rather than the /var. |
|
This issue will be solved as a duplicate of #8151 since it is the same root cause. |
afbjorklund
added
the
triage/duplicate
afbjorklund
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce the issue:
minikube startwith driver set to dockerFull output of failed command:
Full output of
minikube startcommand used, if not already included:Optional: Full output of
minikube logscommand:Minikube Container logs:
The text was updated successfully, but these errors were encountered: