Docker behind proxy - setup nightmare

[RainM]

Description

I setup docker build behind proxy. I have to have proxy settings in 3 (!!!) places:

1. Docker daemon. I created file /etc/systemd/system/docker.service.d/docker_proxy.conf as it's described in documentation.
2. '--build-arg's for docker build cli run.
3. 'http_proxy' and 'https_proxy' environment variable for docker build. Without this 'docker build' cli hangs as it's impossible to connect directly. It fails after some time with error 'ERROR: failed to solve: XXX: failed to authorize: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope...": dial tcp 44.205.64.79:443: i/o timeout'.

For me proper cli is http_proxy=.... https_proxy=... docker build --build-arg http_proxy=.... --build-arg https_proxy=.... -t ...

So, proper proxy setup consists of settings in 3(!!!!) places. And it's just docker's options. After that I need to set proper apt's proxy config for installing necessary software in container image. And additional arguments for gradle for building java application.

I strictly believe current number of settings could drive of the wall anybody. I'd like to have proxy settings in 1 (ONE!) place, not in three places.

[Trolldemorted]

Thank you for creating this issue, I know many engineers that feel the same way.

To stop me from going insane I wrote the status quo down:

• daemon configuration
  • is used for image downloads caused by docker pull and docker compose pull (and similar commands)
  • can be set in two locations (/etc/systemd/system/docker.service.d/docker_proxy.conf vs /etc/docker/daemon.json)
  • current documentation is objectively wrong (reported months ago in Conflicting info on configuring HTTP/HTTPS proxies docs#17880) and nobody cares
• build configuration
  • is used for image downloads caused by docker build and docker compose build (and similar commands)
    • if you have previously pulled the image through the daemon with docker pull, you don't have to specify this
    • if you have started a build with the proxy set and cancelled it before the image download finished, you can resume the download without having the proxy set (probably because the daemon now knows about the image?)
    • this appears to be not limited to specific tags - if you have started doing a FROM python:3.7 once with the proxy set, you can do a FROM python:3.6 without the proxy set
  • can be set by invoking docker with HTTP_PROXY or https_proxy set
• build containers
  • env vars for Dockerfiles can be set through ~/.docker/config.json

I can somewhat understand that the daemon configuration and the container configuration is done separately, but why does docker build sometimes attempt to talk with external servers?!

These are the bugs that make students and juniors reconsider their carreer choices and switch to carpentry. The amount of time wasted across the globe because of this (and the incorrect documentation) must be ridiculous.

[Trolldemorted]

I totally forgot that there also is /etc/docker/config.json (at least there are open issues about it like #2738) but I can't find any documentation about that one on https://docs.docker.com/.

Does anyone know what that file should be doing nowadays? Is it still being used?

[paddy-hack]

@RainM I can understand your frustration.

I've been fighting various incarnations of corporate proxies for two decades now and have learnt a thing or two the hard way. One of the first things you have to consider is what component lives where and what does it need to access when (and in what context). Once you got that sorted out, the various settings actually make sense, believe it or not.

First the docker daemon and client do not necessarily live on the same computer. They often do, but they don't have to. That means that the client may need proxy settings to communicate with the daemon. The daemon may in turn need proxy settings to pull images from any of the various container image registries. These settings are not necessarily the same as those needed by the client to contact the daemon.

The docker build invocation contacts the daemon to make it pull a base image (if not already cached or the cache is explicitly ignored). That same build may need to fetch packages from the distribution, an NPM registry or pull code from a git repository somewhere. That may require yet another set of proxy settings. These proxy settings should not remain in the image being built because they are build environment dependent. That is, embedding the proxy settings I need at work in an image is not going to work for you when you pull that image. Worse, they may include authentication credentials 😱
The --build-arg options keep https_proxy, http_proxy and no_proxy out of the image.

Finally, there is a set of proxy settings that the image may need at run-time. For example, an image that polls a git repository for new commits in a repository on the other side of the proxy. Again, your proxy requirements aren't the same as mine so these settings should not be baked into the image (in the general case).

In a worst case scenario, these four (not three 😓) use cases may all need different settings. In a security conscious setup, you may even need to be able to prevent some of them to default to the settings for another use case, so specifying each set explicitly, though annoying, makes sense.

All that said, I personally really would like it if they all defaulted to using a single set (as long as I can override that when needed). After all, most of the time you're dealing with the daemon and client running on the same machine where you build and run your images during development.

Hope this helps.

[cajund]

Hello All,

I may have missed this in your posts - but do any of these scenarios address connecting to a specific registry behind a proxy? While I have had success getting the docker client to connect to a registry using a proxy, the proxy itself (nginx) is unable to connect to the upstream registry and returns a 400, Bad Request. The error occurs on the CONNECT step, and is consistent with both login and pull/push commands. Perhaps this is an nginx config issue, but something is making the process that docker uses different that a straight forward HTTTP request. Is there anything that makes docker "special".

Thanks.

[okanaiki]

Hi, is there a nice way to hash config.json proxy setup to hide plaintext passwords? I'm unable to find good multiplatform solution for now