Official Dockerimage in Openshift | Permission problems when accessing served certificates

[lostiniceland]

Elasticsearch version 6.7.2 Docker

OS version: Openshift 3.11 on Centos 7

Description of the problem including expected versus actual behavior:
Running Elasticsearch in Openshift doesnt work when specifing a Certificate which was provided by the platform. Openshift mounts a secret-volume containing its own CA and the YAML specifies a self-serving certificate which was signed by the Openshift-CA.

Those files are mapped to root:root.

Openshift runs each image with a generated UID (> 10000) and GUID 0.

It seems the Elasticsearch User is not running as root (which is good) but not with GUID 0, which is necessary to access those files.

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

1. Deploy official ES image as Openshift Imagestream
2. Use DeploymentConfig (or Deployment) YAML with SSL configuration
3. Check logs

Openshift-Deployment-YAML

- kind: Service
    apiVersion: v1
    metadata:
      name: test
      annotations:
        service.alpha.openshift.io/serving-cert-secret-name: cert-secret-test
    spec:
      selector:
        name: test
      ports:
        - name: elasticsearch
          port: 9200
          targetPort: 9200
      type: LoadBalancer
	  
- kind: DeploymentConfig
    apiVersion: v1
    metadata:
      name: test
    spec:
      replicas: 1
      template:
        spec:
          containers:
            - name: elasticsearch
              image: docker-registry.default.svc:5000/project/elasticsearch:6.7.2
              ports:
                - containerPort: 9200
              env:
                - name: discovery.type
                  value: single-node
                - name: ELASTIC_PASSWORD
                  value: password
                - name: xpack.ssl.key
                  value: /var/run/secrets/openshift.io/project/cert-secret/tls.key
                - name: xpack.ssl.certificate
                  value: /var/run/secrets/openshift.io/project/cert-secret/tls.crt
                - name: xpack.ssl.certificate_authorities
                  value: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt
                - name: xpack.security.transport.ssl.enabled
                  value: 'true'
                - name: xpack.security.http.ssl.enabled
                  value: 'true'
                - name: ES_JAVA_OPTS
                  value: '-Xms512m -Xmx512m'
              volumeMounts:
                - name: cert-vol-test
                  mountPath: /var/run/secrets/openshift.io/project/cert-secret
	volumes:
	   - name: cert-vol-test
             secret:
                secretName: cert-secret-test

Provide logs (if relevant):

Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt" "read")

[elasticmachine]

Pinging @elastic/es-core-infra

[lostiniceland]

I've checked the docker-entrypoint.sh used in the Elasticsearch Dockerfile and there is some logic with regards to Openshifts user/group-maping. Still this seems not to be enough.

Furthermore I am not able to connect to the container in openshift via terminal. I get the following message: Could not connect to the container. Do you have sufficient privileges?

[alpar-t]

@lostiniceland this doesn't seem to be specific to openshift.
We configure the java security manager so that it can only read config files from the config directory. Try mounting the certs into /usr/share/elasticsearch/config and see if the problem persists.

[alpar-t]

We improved those error messages in #44787

[lostiniceland]

Yes, mounting to /usr/share/elasticsearch/config/xyz worked.

Thought I had to use an additional init-container to copy the ca-certificate from another mountpoint (openshift automatically mouts the ca to /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt)