Docker 19.03.13 for arm32v7 build failing

[kereis]

For arm32v7, the current version of Docker on Docker Hub is 19.03.08. Meanwhile, Docker version 19.03.13 is available (See Jenkins).

I think it shouldn't make a huge difference to either use 19.03.08 or 19.03.13, but it's a little bit inconsistent though and it could possibly mean that builds with any other newer version may fail.

[tianon]

Indeed, the failure looks like the official binaries might be no good:

Step 6/12 : RUN set -eux; 		apkArch="$(apk --print-arch)"; 	case "$apkArch" in 		x86_64) dockerArch='x86_64' ;; 		armhf) dockerArch='armel' ;; 		armv7) dockerArch='armhf' ;; 		aarch64) dockerArch='aarch64' ;; 		*) echo >&2 "error: unsupported architecture ($apkArch)"; exit 1 ;;	esac; 		if ! wget -O docker.tgz "https://download.docker.com/linux/static/${DOCKER_CHANNEL}/${dockerArch}/docker-${DOCKER_VERSION}.tgz"; then 		echo >&2 "error: failed to download 'docker-${DOCKER_VERSION}' from '${DOCKER_CHANNEL}' for '${dockerArch}'"; 		exit 1; 	fi; 		tar --extract 		--file docker.tgz 		--strip-components 1 		--directory /usr/local/bin/ 	; 	rm docker.tgz; 		dockerd --version; 	docker --version
 ---> Running in 2003701aeec2
+ apk --print-arch
+ apkArch=armv7
+ dockerArch=armhf
+ wget -O docker.tgz https://download.docker.com/linux/static/stable/armhf/docker-19.03.13.tgz
Connecting to download.docker.com (13.225.65.117:443)
saving to 'docker.tgz'
docker.tgz           100% |********************************| 53.7M  0:00:00 ETA
'docker.tgz' saved
+ tar --extract --file docker.tgz --strip-components 1 --directory /usr/local/bin/
+ rm docker.tgz
+ dockerd --version
Illegal instruction (core dumped)

😞

[djmaze]

The outdated docker version on arm32v7 also means it is currently not possible to use debian:bullseye on arm32. Any idea on how or when this can be fixed?

[tianon]

Unfortunately, this is a problem in the upstream Docker build process that's creating the arm* binaries on https://download.docker.com/linux/static/, so the fix needs to happen there. 😞

[djmaze]

Even more people seem to be affected this problem now. Any idea on who to ping to get this forward? Or can we help with this somehow?

[tianon]

@tiborvass @thaJeztah can one of you point me at the script that's building those binaries so I can help look into why they're not working? 🙏

I've been digging in https://github.com/docker/docker-ce-packaging/tree/4865d0386b312294a2a6f8dc5a0ed0bfbfff862c/static, but I'm not sure this is what's actually creating those? (It's a bit hard to track what's actually responsible here 😇)

[djmaze]

I just tried building dockerd 20.10.3 on an armv7 machine (which runs docker 19.03.14) using make setup in the moby repository. The resulting dockerd binary yields a segmentation fault when started.

I then manually installed a development environment on the armv7 host (with debian stretch & go 1.13). I then built the static binary using hack/make.sh from the repo. The resulting dockerd binary does not crash and seems to work.

It looks to me like the docker-based build process is somehow broken on armv7.