Wrong elasticsearch.keystore permissions forbid elasticsearch.service from starting

[LolloneS]

Elasticsearch version: 7.11.1

Role version: 7.13.1

JVM version (java -version):

openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)

OS version (uname -a if on a Unix-like system): CentOS 8: Linux test-machine 4.18.0-240.22.1.el8_3.x86_64 #1 SMP Thu Apr 8 19:01:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behaviour:
When installing Elasticsearch, the /etc/elasticsearch/elasticsearch.keystore file has wrong permissions (root:root), which makes Elasticsearch's service fail while trying to start. The file should belong to the elasticsearch group in order to allow Elasticsearch to access it.

Playbook:

- name: Elasticsearch with SSL/TLS enabled
  hosts: elasticsearch_prod
  roles:
    - role: elastic.elasticsearch

Provide logs from Ansible:

➜  elk-ansible ansible-playbook -i inventory.ini main.yml --ask-become-pass
BECOME password: 

PLAY [Elasticsearch with SSL/TLS enabled] ***************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set_fact] *****************************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : os-specific vars] *********************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Set fact oss_version when using es_enable_xpack] **************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Warn about deprecated es_enable_xpack variable] ***************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Set the defaults here otherwise they can't be overriden in the same play if the role is called twice] *********************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Use the oss repo and package] *********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Set the URL scheme to https if SSL/TLS is enabled] ************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Warn about deprecated  es_xpack_features variable] ************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when oss_version is true with es_version >= 7.11.0] ******************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when es_proxy_port is not defined or is blank] ***********************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when heap size is not specified when using memory lock] **************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when password is not declared when using security] *******************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when api credentials are not declared when using tls] ****************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when ssl enabled without defining a key and certificate] *************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact file_reserved_users] *********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : fail when changing users through file realm] ******************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact m_lock_enabled] **************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set fact use_system_d] ****************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : detect if we need the .deb or .rpm] ***************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : get the minor version] ****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set the package_name] *****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : generate the artifacts url] ***********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : get latest snapshot build] ************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : use the custom package url instead of the repository] *********************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : split up the snapshot url so we can create the plugin url] ****************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set base plugin url] ******************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : create es_plugins with the snapshot url] **********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : override the original es_plugins with the snapshot version] ***************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact java_state to present] *******************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact java_state to latest] ********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : RedHat - Ensure Java is installed] ****************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Get the installed java path] **********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : correct java version selected] ********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Refresh java repo] ********************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Ensure Java is installed] ****************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : register open_jdk version] ************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : refresh the java ca-certificates] *****************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact force_install to no] *********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact force_install to yes] ********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Check if the elasticsearch package is installed] **************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : unhold elasticsearch package when switching to a different package type] **************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : stop elasticsearch] *******************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Remove elasticsearch package if we are switching to a different package type] ************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Install apt-transport-https to support https APT downloads] ******************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Add Elasticsearch repository key] ********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Add elasticsearch repository] ************************************************************************************************************************************************************************
skipping: [test-machine] => (item={'repo': 'deb http://packages.elastic.co/elasticsearch/7.x/debian stable main', 'state': 'absent'}) 
skipping: [test-machine] => (item={'repo': 'deb https://artifacts.elastic.co/packages/7.x/apt stable main', 'state': 'present'}) 
skipping: [test-machine] => (item={'repo': 'deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main', 'state': 'absent'}) 

TASK [elastic.elasticsearch : Ensure optional elasticsearch group is created with the correct id.] ******************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Ensure optional elasticsearch user is created with the correct id.] *******************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Get installed elasticsearch version] *****************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - unhold elasticsearch version] ************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Ensure elasticsearch is installed] *******************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - hold elasticsearch version] **************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set_fact] *****************************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Debian - Install Elasticsearch from url] **********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact allow_downgrade to no] *******************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set fact allow_downgrade to yes] ******************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Ensure libselinux-python on CentOS 6.x] ***********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : RedHat - add Elasticsearch repo] ******************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - remove unused Elasticsearch repo] ********************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - install yum-version-lock] ****************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - check if requested elasticsearch version lock exists] ************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - lock elasticsearch version] **************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : RedHat - check if any elasticsearch version lock exists] ******************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - unlock elasticsearch version] ************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : RedHat - Remove the other elasticsearch package if switching between OSS and standard] ************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Ensure optional elasticsearch group is created with the correct id.] ******************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Ensure optional elasticsearch user is created with the correct id.] *******************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : RedHat - Install Elasticsearch] *******************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set_fact] *****************************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : RedHat - Install Elasticsearch from url] **********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Create Configuration Directory] *******************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Create PID Directory] *****************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Create Others Directories] ************************************************************************************************************************************************************************************
ok: [test-machine] => (item=/var/log/elasticsearch)
changed: [test-machine] => (item=/data)

TASK [elastic.elasticsearch : Copy Configuration File] **************************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Copy Default File] ********************************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Make sure destination dir exists] *****************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Copy specific ElasticSearch Systemd config file] **************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Copy jvm.options File] ****************************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Copy log4j2.properties File] **********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact es_plugins_reinstall to true] ************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact list_command] ****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact list_command check for x-pack] ***********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : remove x-pack plugin directory when it isn't a plugin] ********************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Check installed elasticsearch plugins] ************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact plugins_to_remove to install_plugins.stdout_lines] ***************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact plugins_to_remove to delete plugins installed but not listed in es_plugins] **************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact plugins_to_install to es_plugins] ********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact to plugins_to_install to those in es_config but not installed] ***************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Remove elasticsearch plugins] *********************************************************************************************************************************************************************************

TASK [elastic.elasticsearch : Install elasticsearch plugins] ********************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : create the keystore if it doesn't exist yet] ******************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Check if bootstrap password is set] ***************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Create Bootstrap password for elastic user] *******************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Remove keystore entries] **************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Reload keystore entries] **************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Add keystore entries] *****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set_fact] *****************************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set_fact] *****************************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Check if old users file exists] *******************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Copy the old users file from the old deprecated location] *****************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : List Users] ***************************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact users_to_remove] *************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Remove Users] *************************************************************************************************************************************************************************************************

TASK [elastic.elasticsearch : set fact users_to_add] ****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Add Users] ****************************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Set User Passwords] *******************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : set fact users_roles] *****************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Copy roles.yml File for Instance] *****************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Copy User Roles] **********************************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Copy role_mapping.yml file for instance] **********************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Set elasticsearch.keystore Permissions] ***********************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set fact es_same_keystore] ************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : set fact es_same_keystore if stores match] ********************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Ensure certificate directory exists] **************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Upload SSL/TLS keystore] **************************************************************************************************************************************************************************************
changed: [test-machine]

TASK [elastic.elasticsearch : Upload SSL/TLS truststore] ************************************************************************************************************************************************************************************
ok: [test-machine]

TASK [elastic.elasticsearch : Upload SSL/TLS key and certificate] ***************************************************************************************************************************************************************************
skipping: [test-machine] => (item=) 
skipping: [test-machine] => (item=) 

TASK [elastic.elasticsearch : Upload SSL Certificate Authority] *****************************************************************************************************************************************************************************
skipping: [test-machine]

TASK [elastic.elasticsearch : Set keystore password] ****************************************************************************************************************************************************************************************
changed: [test-machine] => (item=None)
changed: [test-machine] => (item=None)
changed: [test-machine]

TASK [elastic.elasticsearch : Set truststore password] **************************************************************************************************************************************************************************************
changed: [test-machine] => (item=None)
changed: [test-machine] => (item=None)
changed: [test-machine]

TASK [elastic.elasticsearch : Remove keystore password] *************************************************************************************************************************************************************************************
skipping: [test-machine] => (item=http) 
skipping: [test-machine] => (item=transport) 

TASK [elastic.elasticsearch : Remove truststore password] ***********************************************************************************************************************************************************************************
skipping: [test-machine] => (item=http) 
skipping: [test-machine] => (item=transport) 

TASK [elastic.elasticsearch : Set key password] *********************************************************************************************************************************************************************************************
skipping: [test-machine] => (item=None) 
skipping: [test-machine] => (item=None) 
skipping: [test-machine]

TASK [elastic.elasticsearch : Remove key password] ******************************************************************************************************************************************************************************************
skipping: [test-machine] => (item=http) 
skipping: [test-machine] => (item=transport) 

RUNNING HANDLER [elastic.elasticsearch : reload systemd configuration] **********************************************************************************************************************************************************************
ok: [test-machine]

RUNNING HANDLER [elastic.elasticsearch : restart elasticsearch] *****************************************************************************************************************************************************************************
fatal: [test-machine]: FAILED! => {"changed": false, "msg": "Unable to start service elasticsearch: Job for elasticsearch.service failed because the control process exited with error code.\nSee \"systemctl status elasticsearch.service\" and \"journalctl -xe\" for details.\n"}

NO MORE HOSTS LEFT **************************************************************************************************************************************************************************************************************************

PLAY RECAP **********************************************************************************************************************************************************************************************************************************
test-machine    : ok=37   changed=11   unreachable=0    failed=1    skipped=89   rescued=0    ignored=0   

ES Logs if relevant:

-- Logs begin at Tue 2021-06-08 13:02:17 UTC, end at Tue 2021-06-08 13:11:48 UTC. --
Jun 08 13:07:52 test-machine systemd[1]: Starting Elasticsearch...
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:79)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:209)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:31)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:80)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.Command.main(Command.java:79)
Jun 08 13:07:53 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:32)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]: Likely root cause: java.nio.file.AccessDeniedException: /etc/elasticsearch/elasticsearch.keystore
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/java.nio.file.Files.newByteChannel(Files.java:375)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at java.base/java.nio.file.Files.newByteChannel(Files.java:426)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:79)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:209)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:233)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:227)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:116)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.cli.Command.main(Command.java:79)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115)
Jun 08 13:07:55 test-machine systemd-entrypoint[52176]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81)
Jun 08 13:07:55 test-machine systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

As a matter of fact, the /etc/elasticsearch/ folder looks as follows:

total 44
-rw-rw----. 1 root          root            516 Jun  8 13:07 elasticsearch.keystore
-rw-rw----. 1 root          elasticsearch  1451 Jun  8 13:07 elasticsearch.yml
-rw-rw----. 1 root          elasticsearch  2377 Jun  8 13:07 jvm.options
drwxr-s---. 2 root          elasticsearch     6 Feb 15 13:55 jvm.options.d
-rw-r-----. 1 elasticsearch elasticsearch  3609 Jun  8 13:07 test-machine.p12
-rw-rw----. 1 root          elasticsearch 18535 Feb 15 13:52 log4j2.properties
-rw-rw----. 1 root          elasticsearch   473 Feb 15 13:52 role_mapping.yml
-rw-rw----. 1 root          elasticsearch   197 Feb 15 13:52 roles.yml
-rw-rw----. 1 root          elasticsearch     0 Feb 15 13:52 users
-rw-rw----. 1 root          elasticsearch     0 Feb 15 13:52 users_roles

I also tried manually specifying the es_group: elasticsearch variable in the playbook but the result is still the same.

Conversely, if I:

• log in to the machine
• remove the current entries from the keystore by means of bin/elasticsearch-keystore

xpack.security.http.ssl.keystore.secure_password
xpack.security.http.ssl.truststore.secure_password
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password

• add the entries again by means of bin/elasticsearch-keystore
• chown the keystore to elasticsearch:elasticsearch

the service starts flawlessly.

[jmlrt]

Hi @LolloneS,
Thanks for opening this issue.
Unfortunately we currently don't have time to investigate every tickets so please be patient.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmlrt]

still valid

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[LolloneS]

I believe this is still valid.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.