Docker in docker test fails with "Error while fetching server API version"

[tillahoffmann]

The core/tests/test_docker_in_docker.py::test_wait_for_logs_docker_in_docker test has started failing in CI without an obvious reason (see here for an example run). docker/docker-py#2717 may be related. @Duchadian, do you have thoughts on this?

FAILED core/tests/test_docker_in_docker.py::test_wait_for_logs_docker_in_docker - 
docker.errors.DockerException: Error while fetching server API version: 
HTTPConnectionPool(host='172.17.0.2', port=2375): Max retries exceeded with url: 
/version (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fda5372d090>: 
Failed to establish a new connection: [Errno 111] Connection refused'))

[dfioravanti]

I have the same problem on a M1 Mac. All of a sudden it started to crash. I guess it has something to do with the new version of Docker, I upgraded yesterday, until then it worked.

Edit: I can confirm that downgrading makes testcontainers work again

[tillahoffmann]

Thanks for reporting, @dfioravanti. I also observe the issue on my M1.

Edit: I can confirm that downgrading makes testcontainers work again

Which version did you use to make the docker-in-docker tests work again?

[dfioravanti]

Hey @tillahoffmann, sorry I should have been more precise. I have this problem on docker desktop. In the gitlab CI we use it still works, but that runs on linux. In any case, locally I am running 4.16.2 (95914). It was just the installer that I had laying around. No other reason. I tried to look up on the docker github page but I cannot see any report that might explain this and I did not have time yet to really start digging to see if I can find what is going on and open a proper bug report

[Duchadian]

@tillahoffmann, there's a decent number of things that could have caused this as far as I can see. I don't think the CI failing suddenly and that you and @dfioravanti are seeing issues with a new version of docker desktop are unrelated.

There's probably been some change in how the docker api behaves in the new version (could be a change in endpoint, response, tls, etc.) that the docker client can't deal with, but it's strange that it only shows up with the DinD tests. My guess is some sort of verification on the calls to the docker api, but that's nothing more than my best guess without having looked into it at all.

I'll look into this when I have the time, which will probably be after the weekend.

[gooner22]

in my case issue appeared after Docker Desktop upgrade v4.21.1 on M1.
fixed by enabling /var/run/docker.sock in Advanced settings in docker client:

Allow the default Docker socket to be used (requires password)
Creates /var/run/docker.sock which some third-party clients may use to communicate with Docker Desktop. [Learn more⁠](https://docs.docker.com/desktop/mac/permission-requirements)