Add more verbosity to command execution when building docker image

[dmak]

It would be nice if step-by-step command execution is logged to console when complex shell scenarios are executed. Consider the following case:

$ docker-compose -f docker-compose-single-broker.yml up
...
 ---> Running in 5ce05e594e30
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/19) Installing pkgconf (1.3.10-r0)
...
(19/19) Installing jq (1.5-r5)
Executing busybox-1.27.2-r7.trigger
OK: 232 MiB in 69 packages
wget: server returned error: HTTP/1.1 404 Not Found
ERROR: Service 'kafka' failed to build

From above log it is not clear what went wrong.
After adding / changing:

• In Dockerfile:

RUN set -x && apk add --no-cache

• In download-kafka.sh:

#!/bin/sh -e

set -x

the output is the following:

 ---> Running in 5fe2c27941ec
+ apk add --no-cache bash curl jq docker
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/19) Installing pkgconf (1.3.10-r0)
...
(19/19) Installing jq (1.5-r5)
Executing busybox-1.27.2-r7.trigger
OK: 232 MiB in 69 packages
+ mkdir /opt
+ chmod a+x /tmp/broker-list.sh /tmp/create-topics.sh /tmp/download-kafka.sh /tmp/start-kafka.sh
+ mv /tmp/start-kafka.sh /tmp/broker-list.sh /tmp/create-topics.sh /usr/bin
+ sync
+ /tmp/download-kafka.sh
+ curl --stderr /dev/null https://www.apache.org/dyn/closer.cgi?as_json=1
+ jq -r .preferred
+ mirror=http://apache.mirror1.spango.com/
+ [[ -z http://apache.mirror1.spango.com/ ]]
+ url=http://apache.mirror1.spango.com/kafka/1.1.0/kafka_2.12-1.1.0.tgz
+ wget -q http://apache.mirror1.spango.com/kafka/1.1.0/kafka_2.12-1.1.0.tgz -O /tmp/kafka_2.12-1.1.0.tgz
wget: server returned error: HTTP/1.1 404 Not Found
ERROR: Service 'kafka' failed to build

Now it is clear what needs to be corrected.

Suggestion

set -x is added/used in shell scripts which could potentially break.

[sscaling]

Are you using recent code? This commit should have addressed the mirror issue.

[dmak]

Well, the report is not about the issue with Kafka version, but the possibility to identify & correct the issue. It's not so difficult to correct the version once one knows what is going on. Thanks for fixing it anyway, but this very issue is about the different matter.

[sscaling]

I'm just trying to highlight that the specific issue you are seeing should be fixed in master.

I'm assuming anyone digging this deep would know how to debug the issue. We haven't really had issues with the build process other than the mirror not hosting the version it is supposed to host. So it's just a question of whether or not everyone's build logs get cluttered to support an exceptionally small use-case or not. Even in your proxy issue use-case (as referenced in the other #487 issue), this would not help just copy&paste debug because of the wget -q command omits the resolved server IP in the logs. All I could infer is that you can't reach the mirror, and then you'd need to perform some additional debug commands anyway. So it's just a question of how much value does it add.

[dmak]

I agree that if in 99% cases docker build works smoothly, extra output will only create noise. Feel tree to close an issue – somebody can find it via search, if necessary.

[sscaling]

Thanks for closing the loop and confirming assumptions.