Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker compose startup works on OSX but fails on AWS Linux #5863

Closed
pszabop opened this issue Jan 14, 2023 · 1 comment
Closed

docker compose startup works on OSX but fails on AWS Linux #5863

pszabop opened this issue Jan 14, 2023 · 1 comment
Labels
bug Something isn't working untriaged

Comments

@pszabop
Copy link

pszabop commented Jan 14, 2023

Describe the bug
Using the exact same docker-compose.yml file and image, startup works on OSX, but fails on a brand new AWS instance created with the AWS wizard

The last error output is

./opensearch-docker-entrypoint.sh: line 70:   103 Killed                  "$@" "${opensearch_opts[@]}

To Reproduce
Steps to reproduce the behavior:

  1. Create an AWS Linux T2 instance using default image, install Docker and Docker Compose
  2. copy the docker compose example from the documentation
  3. run the command docker-compose up opensearch-node1 . only run one node, T2 doesn't have enough memory for two nodes
  4. Wait a few seconds and you will get the error

Do the same but on OSX, works just fine

Expected behavior
What should happen is node1 should run fine but there will be complaints about not finding node2

Plugins
Standard off the shelf from the example and :latest

Screenshots

[+] Running 1/1
 ⠿ Container opensearch-node1  Recreated                                                            0.1s
Attaching to opensearch-node1
opensearch-node1  | Enabling execution of install_demo_configuration.sh for OpenSearch Security Plugin
opensearch-node1  | **************************************************************************
opensearch-node1  | ** This tool will be deprecated in the next major release of OpenSearch **
opensearch-node1  | ** https://github.com/opensearch-project/security/issues/1755           **
opensearch-node1  | **************************************************************************
opensearch-node1  | OpenSearch Security Demo Installer
opensearch-node1  |  ** Warning: Do not use on production or public reachable systems **
opensearch-node1  | Basedir: /usr/share/opensearch
opensearch-node1  | OpenSearch install type: rpm/deb on NAME="Amazon Linux"
opensearch-node1  | OpenSearch config dir: /usr/share/opensearch/config
opensearch-node1  | OpenSearch config file: /usr/share/opensearch/config/opensearch.yml
opensearch-node1  | OpenSearch bin dir: /usr/share/opensearch/bin
opensearch-node1  | OpenSearch plugins dir: /usr/share/opensearch/plugins
opensearch-node1  | OpenSearch lib dir: /usr/share/opensearch/lib
opensearch-node1  | Detected OpenSearch Version: x-content-2.4.1
opensearch-node1  | Detected OpenSearch Security Version: 2.4.1.0
opensearch-node1  | 
opensearch-node1  | ### Success
opensearch-node1  | ### Execute this script now on all your nodes and then start all nodes
opensearch-node1  | ### OpenSearch Security will be automatically initialized.
opensearch-node1  | ### If you like to change the runtime configuration
opensearch-node1  | ### change the files in ../../../config/opensearch-security and execute:
opensearch-node1  | "/usr/share/opensearch/plugins/opensearch-security/tools/securityadmin.sh" -cd "/usr/share/opensearch/config/opensearch-security" -icl -key "/usr/share/opensearch/config/kirk-key.pem" -cert "/usr/share/opensearch/config/kirk.pem" -cacert "/usr/share/opensearch/config/root-ca.pem" -nhnv
opensearch-node1  | ### or run ./securityadmin_demo.sh
opensearch-node1  | ### To use the Security Plugin ConfigurationGUI
opensearch-node1  | ### To access your secured cluster open https://<hostname>:<HTTP port> and log in with admin/admin.
opensearch-node1  | ### (Ignore the SSL certificate warning because we installed self-signed demo certificates)
opensearch-node1  | Enabling OpenSearch Security Plugin
opensearch-node1  | Enabling execution of OPENSEARCH_HOME/bin/opensearch-performance-analyzer/performance-analyzer-agent-cli for OpenSearch Performance Analyzer Plugin
opensearch-node1  | WARNING: A terminally deprecated method in java.lang.System has been called
opensearch-node1  | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/opensearch/lib/opensearch-2.4.1.jar)
opensearch-node1  | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
opensearch-node1  | WARNING: System::setSecurityManager will be removed in a future release
opensearch-node1  | ./opensearch-docker-entrypoint.sh: line 70:   103 Killed                  "$@" "${opensearch_opts[@]}"
opensearch-node1 exited with code 137

Host/Environment (please complete the following information):
Linux 5.10.157-139.675.amzn2.x86_64 #1 SMP Thu Dec 8 01:29:11 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Docker version 20.10.17, build 100c701
Docker Compose version v2.15.1

Additional context
docker-compose install isn't done by yum on AWS linux, used instructions from here

@pszabop pszabop added bug Something isn't working untriaged labels Jan 14, 2023
@pszabop
Copy link
Author

pszabop commented Jan 15, 2023

Closing, the problem was insufficient memory.

The real problem is that in Docker this error gets swallowed. I'll open a different issue for that

[Native memory allocation (mmap) failed to map 1073741824 bytes for committing reserved memory](https://forum.opensearch.org/t/native-memory-allocation-mmap-failed-to-map-1073741824-bytes-for-committing-reserved-memory/4258)

@pszabop pszabop closed this as completed Jan 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged
Projects
None yet
Development

No branches or pull requests

1 participant