You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By default, Elasticsearch container will download the geoip lite database.
It's better to disable this behavior by default.
There's also an elastic issue about this default behavior: elastic/elasticsearch#90673. Fixed since v8.7.0 with elastic/elasticsearch#92335 .
Modifications
geoip downloading can be disabled by passing -Dingest.geoip.downloader.enabled.default=false in JVM options or passing ingest.geoip.downloader.enabled=false in the environment.
The problem with passing ingest.geoip.downloader.enabled=false in the environment is that it's not compatible with OpenSearch. OpenSearch will fail to start with a configuration validation error org.opensearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [ingest.geoip.downloader.enabled] please check that any required plugins are installed, or check the breaking changes documentation for removed settings. Some users might be using ElasticsearchContainer class together with the OpenSearch docker image.
In Elasticsearch 7.14 a feature was added which would automatically download the MaxMind GeoIP database. Specifically, it downloads the database from Google Cloud after fetching this JSON object. This is enabled by default.
We discovered that this feature was responsible for around $20,000 on our AWS bill since we upgraded last March and based on a rough approximation of traffic we were charged for it probably cost Elastic about $5,500 💸 Investigation was slow and required enabling AWS VPC logs to assess the NAT gateway traffic source.
I guess this geoip download doesn't impact the default image tag that ElasticsearchContainer sets, 7.9.2. The feature was introduced in 7.14.0. 7.14.0 release notes includes:
Thanks again, @lhotari ! The context is really useful, let's keep compatibility in case OpenSearch is used along with ElasticsearchContainer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
By default, Elasticsearch container will download the geoip lite database.
It's better to disable this behavior by default.
There's also an elastic issue about this default behavior: elastic/elasticsearch#90673. Fixed since v8.7.0 with elastic/elasticsearch#92335 .
Modifications
geoip downloading can be disabled by passing
-Dingest.geoip.downloader.enabled.default=falsein JVM options or passingingest.geoip.downloader.enabled=falsein the environment.The problem with passing
ingest.geoip.downloader.enabled=falsein the environment is that it's not compatible with OpenSearch. OpenSearch will fail to start with a configuration validation errororg.opensearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [ingest.geoip.downloader.enabled] please check that any required plugins are installed, or check the breaking changes documentation for removed settings. Some users might be using ElasticsearchContainer class together with the OpenSearch docker image.