org.elasticsearch.client.IndicesClient.exists() semantic changed

[mlyczek]

Elasticsearch Version

7.17.4

Installed Plugins

No response

Java Version

bundled

OS Version

Linux 5.4.0-1068-azure #71~18.04.1-Ubuntu

Problem Description

Since 7.16 version of the High Level Rest Client the semantics of org.elasticsearch.client.IndicesClient.exists() changed.
After changes introduced in #77864 no indices options are sent to Elasticsearch cluster when options set on the request are equal to the default options defined in GetIndexRequest.DEFAULT_INDICES_OPTIONS. I have read https://www.elastic.co/guide/en/elasticsearch/reference/7.17/migrating-7.16.html#breaking_716_hlrc_changes and I understand the workaround, but the problem is that the default options defined in GetIndexRequest.DEFAULT_INDICES_OPTIONS are different than default values in Elasticsearch itself. Therefore not sending them changes semantics of the request. And if somebody needs to set exactly such options as in GetIndexRequest.DEFAULT_INDICES_OPTIONS, then because of the if statement in

elasticsearch/client/rest-high-level/src/main/java/org/elasticsearch/client/IndicesRequestConverters.java

Line 599 in c01070d

if (GetIndexRequest.DEFAULT_INDICES_OPTIONS.equals(getIndexRequest.indicesOptions()) == false) {

it is not possible and will have no effect.

This is specifically the case for allow_no_indices option which has a default value of true in Elasticsearch, but in GetIndexRequest.DEFAULT_INDICES_OPTIONS the default value for this option is false. So when checking for existence of indices for given wildcard expression (e.g. some-indices-*) the org.elasticsearch.client.IndicesClient.exists() method always responds true even if no index exists for given wildcard expression. But before 7.16 it returned false in such case. And there is no way of setting that flag to false, to restore previous semantics of the method, without modifying some of the other indices options so that it does not equal to GetIndexRequest.DEFAULT_INDICES_OPTIONS.

Therefore I think, that the best fix would be to leave GetIndexRequest.DEFAULT_INDICES_OPTIONS as they are now and change IndicesRequestConverters so that each option is considered separately and not sent to Elasticsearch cluster only when it is equal to the default value in Elasticsearch itself.

I know that HLRC is removed in 8.x and migrating to new Java client is recommended, but a lot of systems are forced to stick with Elasticsearch in version 7.x and HLRC for various reasons (for example no support for Elasticsearch 8.x in Spring Data Elasticsearch yet) and will stay on 7.x version for some time in the future. I'd really appreciate a fix that would restore the default semantics of the method (i.e. sending allow_no_indices set to false) and am happy to provide more details or discuss it further.

Steps to Reproduce

Elasticsearch cluster state

Empty cluster without any indices

Code to run

IndicesOptions indicesOptions = IndicesOptions.fromOptions(
        /* ignoreUnavailable */ false,
        /* allowNoIndices */ false,
        /* expandToOpenIndices */ true,
        /* expandToClosedIndices */ true);

GetIndexRequest request = new GetIndexRequest("some-indices-*").indicesOptions(indicesOptions);
        
boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);

Result

After 7.16 exists == true despite the fact of setting allowNoIndices option to false in IndicesOptions.

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/clients-team (Team:Clients)

[silentrobi]

I noticed the similar issue when updating to version 7.17.18 from 7.12.3. I found a workaroud for this by adding allow_no_indices explicitely.

 var optionsBuilder = RequestOptions.DEFAULT.toBuilder();
        options.addParameter("allow_no_indices", "false");
 boolean exists =  client.indices().exists(request, optionsBuilder.build());      

[dakrone]

Closing this as we've removed the high level rest client in favor of the Java client.