Elasticsearch Version

Version: 8.11.1, Build: deb/6f9ff581fbcde658e6f69d6ce03050f060d1fd0c/2023-11-11T10:05:59.421038163Z, JVM: 21.0.1

Installed Plugins

No response

Java Version

bundled

OS Version

5.4.0-159-generic #176-Ubuntu SMP

Problem Description

I am trying to integrate Azure AD to Elasticsearch cluster behind a proxy. I tried the proxy parameter settings below but could not succeeded. You can find the log behind that post. It say it cannot access to microsoftonline.com but as we diagnose it is not trying over proxy setting. It is trying directly to Azure IP. If I try to ping that domain it is going over proxy. Elasticsearch is not trying that connectivity over proxy.

How I can force that to use proxy.

tested - not working:

sudo systemctl edit --full elastic-agent.service

[Service]

Environment="HTTPS_PROXY=https://my.proxy:8443/"
Environment="HTTP_PROXY=http://my.proxy:8080/"

tested - notworking :

http.proxy.host
http.proxy.port

tested - not working

https://discuss.elastic.co/t/azure-ad-sso-setting-behind-a-proxy-not-working/346654/2

Steps to Reproduce

all document is retried many times . results is same. Test it on on prem istallation

The only thing you have to try is work the ubuntu using global proxy setting that has no access to internet.

Document :
https://www.elastic.co/guide/en/cloud/current/ec-securing-clusters-saml-azure.html

Logs (if relevant)

23-11-07T20:08:45,218][DEBUG][o.e.x.s.a.s.SamlRealm ] [AZLPELKSEARCH] Initializing OpenSAML
[2023-11-07T20:08:45,734][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Processing scheduled tasks started
[2023-11-07T20:08:45,735][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Looking for scheduled tasks to process finished, took 0ms
[2023-11-07T20:08:46,032][DEBUG][o.e.x.s.a.s.SamlRealm ] [AZLPELKSEARCH] Initialized OpenSAML
[2023-11-07T20:08:46,036][DEBUG][o.e.x.c.s.SSLService ] [AZLPELKSEARCH] SSL configuration [xpack.security.authc.realms.saml.kibana-realm.ssl] is [SslConfiguration[settingPrefix=, explicitlyConfigured=false, trustConfig=JDK-trusted-certs, keyConfig=empty-key-config, verificationMode=FULL, clientAuth=REQUIRED, ciphers=[TLS_AES_256_GCM_SHA384, TLS_AES_128_GCM_SHA256, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA], supportedProtocols=[TLSv1.3, TLSv1.2, TLSv1.1]]]
[2023-11-07T20:08:46,107][DEBUG][o.a.h.c.p.RequestAddCookies] [AZLPELKSEARCH] CookieSpec selected: default
[2023-11-07T20:08:46,112][DEBUG][o.a.h.c.p.RequestAuthCache] [AZLPELKSEARCH] Auth cache not set in the context
[2023-11-07T20:08:46,113][DEBUG][o.a.h.i.c.PoolingHttpClientConnectionManager] [AZLPELKSEARCH] Connection request: [route: {s}->https://login.microsoftonline.com:443][total available: 0; route allocated: 0 of 2; total allocated: 0 of 20]
[2023-11-07T20:08:46,122][DEBUG][o.a.h.i.c.PoolingHttpClientConnectionManager] [AZLPELKSEARCH] Connection leased: [id: 0][route: {s}->https://login.microsoftonline.com:443][total available: 0; route allocated: 1 of 2; total allocated: 1 of 20]
[2023-11-07T20:08:46,123][DEBUG][o.a.h.i.e.MainClientExec ] [AZLPELKSEARCH] Opening connection {s}->https://login.microsoftonline.com:443
[2023-11-07T20:08:46,130][DEBUG][o.a.h.i.c.DefaultHttpClientConnectionOperator] [AZLPELKSEARCH] Connecting to login.microsoftonline.com/20.190.177.21:443
[2023-11-07T20:08:46,130][DEBUG][o.a.h.c.s.SSLConnectionSocketFactory] [AZLPELKSEARCH] Connecting socket to login.microsoftonline.com/20.190.177.21:443 with timeout 0
[2023-11-07T20:08:46,736][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Processing scheduled tasks started
[2023-11-07T20:08:46,736][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Looking for scheduled tasks to process finished, took 0ms
[2023-11-07T20:08:47,736][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Processing scheduled tasks started
[2023-11-07T20:08:47,737][TRACE][o.e.x.t.t.s.TransformScheduler] [AZLPELKSEARCH] Looking for scheduled tasks to process finished, took 0ms
[2023-11-07T20:08:47,999][TRACE][o.e.i.IndexingMemoryController] [AZLPELKSEARCH] total indexing heap bytes used [0b] vs indices.mem

[2023-11-03T13:35:19,218][INFO ][o.a.h.i.e.RetryExec ] [AZLPELKSEARCH] I/O exception (java.net.SocketException) caught when processing request to {s}->https://login.microsoftonline.com:443: Network is unreachable
[2023-11-03T13:35:19,218][INFO ][o.a.h.i.e.RetryExec ] [AZLPELKSEARCH] Retrying request to {s}->https://login.microsoftonline.com:443