Provide ENV variable for runtime user?

[snesbittsea]

Is there a reason that the runtime user - elasticsearch - is hardcoded rather than set as an environment variable with a default value of elasticsearch.

Maybe my ignorance is showing, but for development having the owner of a bind mounted config directory someone other than me seems an unnecessary aggravation and burden.

Thx!

-steve

I can certainly see an argument for making the user configurable, but we'd have to, somehow, dynamically create the user inside the container. Without that extra logic, once inside the container, there is no user that represents "me":

$ id
uid=1000(jarpy) gid=1000(jarpy) groups=1000(jarpy),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev)

$ docker run docker.elastic.co/elasticsearch/elasticsearch:5.5.0 id jarpy
id: jarpy: no such user

We explicitly create the elasticsearch user so that the Elasticsearch process has someone it can run as (particularly since running as root ranges from "strongly discouraged" to "not permitted" depending on the circumstances).

If you really want to replicate your host-system user inside the container, you could consider making a custom image. You could do something like this:

$ cat > Dockerfile <<EOF
FROM docker.elastic.co/elasticsearch/elasticsearch:5.5.0

USER root
RUN userdel elasticsearch
RUN groupadd -g ${UID} ${USER}
RUN adduser -u ${UID} -g ${UID} -d /usr/share/elasticsearch ${USER}
RUN chown -R ${USER}:${USER} .
USER ${USER}

EOF

$ docker build . -t my-es:5.5.0

_{Edit: Removed unneeded "ARG" statements from example.}

Really though, this is deliberately working around the isolation that the container environment is carefully trying to provide. I don't know anything about your use-case, forgive me, but is it worth considering simply running Elasticsearch on the host-system? That way, you'd avoid the extra complexity that the container environment introduces.

[dliappis]

for development having the owner of a bind mounted config directory

For a bind mounted config dir (say, you want to provide a customized /usr/share/elasticseach/config/elasticsearch.yml) it will be enough if the file has permissions o+r.

Observe the following example:

[testdocker ~]$ id
uid=1001(testdocker) gid=1002(testdocker) groups=1002(testdocker),1001(docker)

[testdocker ~]$ cat >es.yml
cluster.name: "testcluster"
network.host: 0.0.0.0

[testdocker ~]$ ls -la es.yml
-rw-rw-r-- 1 testdocker testdocker 28 Jul 10 11:29 es.yml

[testdocker ~]$ docker run --rm -v $PWD/es.yml:/usr/share/elasticsearch/config/elasticsearch.yml docker.elastic.co/elasticsearch/elasticsearch:5.5.0
...
[2017-07-10T08:30:02,674][INFO ][o.e.n.Node               ] [TBB-6we] started
[2017-07-10T08:30:02,774][INFO ][o.e.g.GatewayService     ] [TBB-6we] recovered [0] indices into cluster_state

The key thing to note is that you don't need to override the entire conf dir, you can mount a specific file. For entire dirs, just ensure the dir has o+r permissions as ES just needs to read from it.
On Windows, IIRC the resulting permissions from sharing a windows file should be +r for all.

For the data dir (/usr/share/elasticsearch/data), the best practice is to use a named or anonymous volume. Both will persist and the former will be reusable across container restarts through its name. I have described some best practices at the end of this comment and for the differences between named and host mounted volumes here. See also this KB from Docker. This seems to be a common issues for people and for environments where named volumes are an options (e.g. dev envs on laptops, docker-swarm etc.) we could emphasize it more in the Documentation.

[snesbittsea]

The real issue here - IMHO - is that the security requirements for production and development are significantly different. For efficient development I need to manipulate things in a way I would never allow in production. The use case goes something like this: I want to observe/test a configuration change. I edit the bind mounted elasticsearch.yml or log properties in my local development environment (IntelliJ), bounce the elasticsearch server in the container, and observe the change. Under the current scenario I am required to make the change, rebuild the container and redeploy. Not horrific, but a nuisance. Toby's solution - as he admits - is a hack that seriously compromises container security in production. The o+r bind mount option Dimitrous presents doesn't help because I need to be able to modify the elasticsearch.yml or log4j properties file so I need write privileges. The Fluentd container provides another solution. The FluentD docker container is structured such that for development I can run as any user I want, while in production it will run as a given user: fluentd.They accomplish this by defining a FLUENTD_UID environment variable and using the entrypoint program to define a user and set permissions (See https://github.com/fluent/fluentd-docker-image/blob/master/entrypoint.sh.erb) rather than embedding it in the Dockerfile. For development purposes I use a docker-compose.override.yml with my FLUENTD_UID set to my uid. In production it remains unset and reverts to the default. Thanks for the response!

…

-steve

[dliappis]

The o+r bind mount option Dimitrous presents doesn't help because I need to
be able to modify the elasticsearch.yml or log4j properties file so I need
write privileges.

@snesbittsea You are welcome! Note that you don't lose your capability of editing files locally by granting o+r permissions for the conf files (note the + i.e. add read perms for other users). I don't know which platform you are developing on, (e.g. macOS, Linux or Windows) but you can certainly set chmod 664 <the_individual_conffiles.yml>; then you will be able to edit the files on your workstation and the container still be able to read them. See the perms of the es.yml file as well as the uid/gid of the testdocker user in my comment above.

If you are bind mounting dirs, you need to remember the x bit to be able to access files under the dir. In general the unix permission model can be used to grant read privileges to the conf files inside the ES container.

[berglh]

This might be worth a new Github issue, but for my problem, I'd like to see this thread renamed as:

• Provide ENV variable for runtime user uid/gid?

There is a genuine use case for a customisable uid/gid; having a mechanism to cater for this is pretty common in Docker-land. Having not thought about it particularly hard; would it be possible to actually implement this in a secure fashion with the stock container?

~# usermod -u 2005 elasticsearch
~# groupmod -g 3000 elasticsearch

~# find / -group 2000 -exec chgrp -h elasticsearch {} \;
~# find / -user 1005 -exec chown -h elasticsearch {} \;

My containers run in a massively convoluted environment - on hosts that contain a standard operating environment out of my control that end up with hundreds of uid/gid combinations; in these cases I could end up with a conflict giving some random user or group the ability to destroy all our data.

~# lslogins | wc -l
251

@Jarpy - Dockerfile and Jenkins are my current solution to addressing this and other non-standard changes to the stock Elastic containers. Whether or not this is addressed is not the end of the world to me.

@dliappis - I agree, that using a Docker volume is an elegant solution to this problem and would remove the need for this specific requirement in the first place, however, in my specific example it's not practical for me to move 10's of TB of elasticsearch data into Docker named volumes; particularly when I haven't tested the extent of Docker named volume performance in my environment.

Just adding some ⛽ to the 🔥 🚒.

[dliappis]

Closed by #125, support will come starting with version 6.0

[fractos]

@dliappis The documentation on 6.4.3 suggests that there is still no support for this.