Add support to configure Proxy to allow geoip databases update

[fludo]

When starting the elasticsearch, it attemps to update the geoip database (since 7.14).

An error occurs if it is behind a proxy. Apparently, we cannot configure any proxy to allow accessing a remote URL if behind a proxy.

The complete log is :

[2021-08-27T16:31:01,744][INFO ][o.e.i.g.GeoIpDownloader  ] [***] fetching geoip databases overview from [https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree]
[2021-08-27T16:31:11,770][ERROR][o.e.i.g.GeoIpDownloader  ] [***] exception during geoip databases update
java.net.SocketTimeoutException: Connect timed out
	at sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546) ~[?:?]
	at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333) ~[?:?]
	at java.net.Socket.connect(Socket.java:645) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:290) ~[?:?]
	at sun.net.NetworkClient.doConnect(NetworkClient.java:177) ~[?:?]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:497) ~[?:?]
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:600) ~[?:?]
	at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265) ~[?:?]
	at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:379) ~[?:?]
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:189) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1232) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1120) ~[?:?]
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:175) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1653) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1577) ~[?:?]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527) ~[?:?]
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.lambda$get$0(HttpClient.java:55) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.doPrivileged(HttpClient.java:97) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.HttpClient.get(HttpClient.java:49) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.HttpClient.getBytes(HttpClient.java:40) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.fetchDatabasesOverview(GeoIpDownloader.java:117) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:105) ~[ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:237) [ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:89) [ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:38) [ingest-geoip-7.14.0.jar:7.14.0]
	at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:40) [elasticsearch-7.14.0.jar:7.14.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [elasticsearch-7.14.0.jar:7.14.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.14.0.jar:7.14.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]

This request may be related to 75026 if ES_JAVA_OPTS could be used.

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[noliono]

I successfully use a proxy without authentication to update geoip database with ES_JAVA_OPTS with this 2 kinds of method :

• "ES_JAVA_OPTS=-Djava.net.useSystemProxies=true"
  (With the proxy system configured through https_proxy environment configured)

or

• "ES_JAVA_OPTS=-Dhttps.proxyHost=10.x.x.x -Dhttps.proxyPort=3128 -Dhttps.nonProxyHosts=localhost|127.0.0.1|10.*.*.*"

So the ES_JAVA_OPTS seems to work now.

But I would like too to use a proxy with authentication and the following ES_JAVA_OPTS didn't work :
"ES_JAVA_OPTS=-Dhttps.proxyUser=xxxx -Dhttps.proxyPassword=yyyyy -Dhttps.proxyHost=10.x.x.x -Dhttps.proxyPort=3128 -Djdk.http.auth.tunneling.disabledSchemes= -Djdk.https.auth.tunneling.disabledSchemes="

Did someone succeed or have any advise ?

[jakelandis]

It is splitting hair's but I have updated the issue to be a bug not an enhancement since the HTTP clients should be able to be configured via Elasticsearch settings. From which HTTP clients should have general support for HTTP proxy, custom trust stores, and a small handful of other common configurations.

[Bernhard-Fluehmann]

I am dealing with this problem as well and was surprised that this feature is missing. One thing that puzzled me was there is already proxy support for watcher. The strange thing about these parameters is that they are not named after watcher. Thus, with a setting like xpack.http.proxy.host in elasticsearch.yml one could assume that it is generally applied to elasticsearch http clients like geoip database download etc. and not only watcher.

[smalenfant]

The documentation should say "reverse proxy" instead of Proxy for the endpoint.
We already have xpack.http.proxy.host set and that should just work as-is.

[smalenfant]

I set a reverse proxy and it doesn't work either. The problem is that it fetches the metadata that contains endpoint to storage.googleapis.com. One would need to manipulate the metadata file to get this working.