Set ownership on startup for es volume mounts

[stevenmccord]

• This was necessary b/c the new official docker image hard codes the uid and
  gid for the elasticsearch user, which is the owner of the elastic data.
  Once those ids change you need to set the owner. This will be helpful in case
  that changes in the future as well.

[stevenmccord]

This was a reference as I opened up a ticket and utilized this as my solution to it, just thought I would give a reference happy to update if you think it needs it, or you can just junk it, was just trying to highlight the issue. Thanks!

[stevenmccord]

Actually the one thing I don't like about this is that it forces the chown of the data, whereas there might be mounts that don't allow for that to happen like on a macOS where there isn't the elasticsearch user, so I don't really love this being in the entry script. I have it in my own script where I make it conditional then I call the main entrypoint script, which allows me some flexibility when I mount on a mac but in production I don't need to do that. Maybe there are some other thoughts on how to do this...

[ghost]

Thanks for the suggestion, and especially for the self-critique, which is a rare gift!

Something like this has been suggested before and we did, in fact, choose not to include it. My personal concern with this technique is that it creates a kind of reverse responsibility for security. If the filesystems are mounted from the host into the container, then we are asking the container process to "reach out" and mutate security state on the host. In an ideal world, that should be impossible.

Speaking of which, I'm surprised that the chown actually succeeds in this context. If the entrypoint is running as UID 1000, then it shouldn't be allowed to chown things to itself, right?

[jmschentrup]

There is another way which I have found to work. If you make a simple docker file (as below) that labels the folder as a volume, Docker takes care of assigning elasticsearch the correct permissions and does not crash on start up.

FROM docker.elastic.co/elasticsearch/elasticsearch:5.4.3
VOLUME /usr/share/elasticsearch/data

[dliappis]

@jmschentrup Thanks for writing the example above! What this actually does is create an anonymous volume, which you can inspect, once a container has started, with docker volume ls. Anonymous volumes and named volumes are summarized in this Docker KB article as well as in the Docker docs here and in particular named volumes , are what we generally recommend for persistent storage for data under /usr/share/elasticsearch/data. Both anonymous and named data volumes will be writable by the ES process without additional adjustments.

On the cli you could achieve exactly what you defined, without necessarily creating a new image with your customized Dockerfile, with docker run -v /usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:5.4.3, whereas, a named volume e.g. called esdata can be defined like docker run -v esdata:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:5.4.3. Once the image has started you can see the volume with docker volume ls, which in the latter case will be named esdata.

Both named and anonymous data volumes are different to bind mounted local dirs aka host volumes where obviously the issue of ownership is important and need to be writable by uid:gid 1000:1000.

I have also summarized the mount options at the bottom of my comment here, hopefully it may provide some additional insights.

[jmschentrup]

@dliappis I agree with that. It is as easy as running docker run -v /usr/share/elasticsearch/data to run elasticsearch as just a docker image; however, if you where to try and run the official image in kubernetes (as I was and the reason I replied back), the containers fail when you try to mount a volume to /usr/share/elasticsearch/data. It gives a error saying that elasticsearch does not have permissions to write to that folder. Using the basic custom docker image I provided fixed that problem. As someone else pointed out to me, this may be because the docker used by kubernetes is an old version, but that old docker version is the newest version supported by kubernetes.

[dliappis]

@jmschentrup Thanks for the detailed follow up! I appreciate the context of Kubernetes in your case. I had a glimpse on the first results for persistent volumes in k8s but I could not find the equivalent of named (or anonymous) volumes, so I have to presume they reference local host dirs, hence the issue you described.

One thing to watch out for, when setting VOLUME /usr/share/elasticsearch/data in the Dockerfile, is that every time a new container gets started a new volume (with a random name) will be generated, so if a specific container restarts it will have an empty dir under data/. I haven't tested precisely how this works in k8s, but this is the case with plain Docker. The VOLUME definition can be overridden, of course, with run-time options.

[dliappis]

The changes brought into 6.0.0 in #125, allowing the use of bind mounted datadirs with gid=0 or through the env var TAKE_FILE_OWNERSHIP should provide sufficient alternatives for orchestrators like k8s, on top of the existing securityContext mechanism.