6.0 Breaking changes check

[codebrain]

Breaking Changes

Aggregations changes

• Deprecated pattern element of include/exclude for terms aggregations has been removed
• Numeric to and from parameters in date_range aggregation are interpreted according to format now

Cat API changes

• Unbounded queue size in cat thread pool

Clients changes

• Java High Level REST Client

Cluster changes

• Cluster name no longer allowed in path.data

Document API changes

• version type force remove - Deleted enum value
• Upserts no longer support versions
• created field removed in the Index API
• found field removed in the Delete API - Deleted property

Indices changes

• Index templates use index_patterns instead of template
• Shadow Replicas have been removed
• Open/Close index API allows wildcard expressions that match no indices by default
• Delete a document
• Indices aliases api resolves indices expressions only against indices
• Delete index api resolves indices expressions only against indices
• Support for + has been removed in index expressions
• Translog retention

Ingest changes

• Timestamp meta-data field type has changed
• The format of the string-formatted ingest.timestamp field has changed

Java API changes

• setSource methods require XContentType
• DeleteByQueryRequest requires an explicitly set query
• InternalStats and Stats getCountAsString() method removed
• ActionRequestBuilder#execute returns ActionFuture rather than ListenableActionFuture
• Terms.Order and Histogram.Order classes replace by BucketOrder
• getTookInMillis() removed in BulkResponse, SearchResponse and TermVectorsResponse
• GetField and SearchHitField replaced by DocumentField
• Some Aggregation classes have moved packages
• Constructor for PercentileRanksAggregationBuilder has changed

Mapping changes

• Coercion of boolean fields
• The _all meta field is now disabled by default
• The include_in_all mapping parameter is now disallowed - Deleted all IncludeInAll and include_in_all references
• Unrecognized match_mapping_type options not silently ignored
• Validation of locale on date fields

Packaging changes

• Configuring custom user and group for package is no longer allowed
• path.conf is no longer a configurable setting
• CONF_DIR is no longer supported
• Default path settings are removed
• 32-bit is no longer maintained
• ES_JVM_OPTIONS is no longer supported
• ES_INCLUDE is no longer supported
• Heap dump path default

Percolator changes

• Deprecated percolator and mpercolate apis have been removed
• The percolator.map_unmapped_fields_as_string setting has been deprecated

Plugins changes

• Mapper attachments plugin
• S3 Repository plugin
• Azure Repository plugin
• GCS Repository plugin
• EC2 Discovery plugin
• Ignoring hidden folders
• ICU Analysis plugin
• Plugins should not construct Environment instances from Settings

Reindex changes

• size parameter

REST changes

• Unquoted JSON
• Duplicate Keys in JSON, CBOR, Yaml and Smile
• Content-Type Auto-detection
• Boolean API parameters
• Analyze API changes
• Support custom normalizer in Analyze API
• Indices exists API - REST API Spec still contains paramters - https://github.com/elastic/elasticsearch/blob/6.x/rest-api-spec/src/main/resources/rest-api-spec/api/indices.exists.json. Have asked for clarification on Incorrect REST API spec? elasticsearch#27681.
• timestamp and ttl in index requests
• Refresh requests with one or more shard failures return HTTP 500 response instead of 200
• Delete by Query API requires an explicit query
• DELETE document calls now implicitly create the type
• Indices information APIs
• Requests to existing endpoints with incorrect HTTP verb now return 405 responses
• Disallow using _cache and _cache_key
• IndexClosedException to return 400 status code

Scripting changes

• Groovy, JavaScript, and Python languages removed
• Native scripts removed
• File scripts removed
• Date fields now return dates
• Removed access to index internal via the _index variable
• Script Settings
• lang can no longer be specified when using a stored script as part of a request - .Lang() not allowed to be specified as parameter in 6.0 on update request. #2935
• lang can no longer be used when putting, getting, or deleting a stored script
• Stored search template apis removed

Search and Query DSL changes

• Changes to queries:
  • The collect_payloads parameter of the span_near query has been removed. Payloads will be loaded when needed.
  • Queries on boolean fields now strictly parse boolean-like values. This means only the strings "true" and "false" will be parsed into their boolean counterparts. Other strings will cause an error to be thrown.
  • The in query (a synonym for the terms query) has been removed
  • The geo_bbox query (a synonym for the geo_bounding_box query) has been removed
  • The mlt query (a synonym for the more_like_this query) has been removed.
  • The deprecated like_text, ids and docs parameters (all synonyms for like) of the more_like_this query have been removed. Also the deprecated min_word_len (a synonym for min_word_length) and max_word_len (a synonym for max_word_length) have been removed.
  • The fuzzy_match and match_fuzzy query (synonyma for the match query) have been removed
  • The terms query now always returns scores equal to 1 and is not subject to indices.query.bool.max_clause_count anymore.
  • The deprecated indices query has been removed.
  • Support for empty query objects ({ }) has been removed from the query DSL. An error is thrown whenever an empty query object is provided.
  • The deprecated minimum_number_should_match parameter in the bool query has been removed, use minimum_should_match instead.
  • The query_string query now correctly parses the maximum number of states allowed when "determinizing" a regex as max_determinized_states instead of the typo max_determined_states.
  • The query_string query no longer accepts enable_position_increment, use enable_position_increments instead.
  • For geo_distance queries, sorting, and aggregations the sloppy_arc option has been removed from the distance_type parameter. Deleted enum value
  • The geo_distance_range query, which was deprecated in 5.0, has been removed.
  • The optimize_bbox parameter has been removed from geo_distance queries.
  • The ignore_malformed and coerce parameters have been removed from geo_bounding_box, geo_polygon, and geo_distance queries.
  • The disable_coord parameter of the bool and common_terms queries has been removed. If provided, it will be ignored and issue a deprecation warning.
  • The template query has been removed. This query was deprecated since 5.0
  • The percolate query’s document_type has been deprecated. From 6.0 and later it is no longer required to specify the document_type parameter.
  • The split_on_whitespace parameter for the query_string query has been removed. If provided, it will be ignored and issue a deprecation warning. The query_string query now splits on operator only.
  • The use_dismax parameter for the query_string query has been removed. If provided, it will be ignored and issue a deprecation warning. The tie_breaker parameter must be used instead.
  • The auto_generate_phrase_queries parameter for the query_string query has been removed, use an explicit quoted query instead. If provided, it will be ignored and issue a deprecation warning.
  • The all_fields parameter for the query_string has been removed. Set default_field to *` instead. If provided, default_field will be automatically set to *
  • The index parameter in the terms filter, used to look up terms in a dedicated index is now mandatory. Previously, the index defaulted to the index the query was executed on. Now this index must be explicitly set in the request.
  • The deprecated type and slop parameter for the match query have been removed. Instead of setting the type, the match_phrase or match_phrase_prefix should be used. The slop removed from the match query but is supported for match_phrase and match_phrase_prefix.
  • The deprecated phrase_slop parameter (a synonym for the slop parameter) of the match_phrase query has been removed.
  • The deprecated query parameter (a synonym for the filter parameter) of the constant_score query has been removed.
  • The deprecated phrase_slop parameter (a synonym for the slop parameter) of the multi_match query has been removed.
  • The deprecated prefix parameter (a synonym for the value parameter) of the prefix query has been removed.
  • The deprecated le (a synonym for lte) and ge (a synonym for gte) parameter of the range query have been removed.
  • The deprecated multi term rewrite parameters constant_score_auto, constant_score_filter (synonyms for constant_score) have been removed.
• Search shards API
• Changes to the Profile API
• Scoring changes
• Fielddata on _uid
• Highlighters
• fielddata_fields
• docvalue_fields
• script_fields
• Inner hits
• Scroll
• Limit on from/size in top hits and inner hits
• Scroll queries that use the request_cache are deprecated

Settings changes

• Remove support for elasticsearch.json and elasticsearch.yaml configuration file
• Duplicate keys in configuration file
• Coercion of boolean settings
• Snapshot settings
• Store throttling settings
• Store settings
• Network settings
• Similarity settings
• Script Settings
• Discovery Settings

Stats and info changes

• Removal of throttle_time in the store stats
• FS stats no longer reports if the disk spins

[codebrain]

Indices information APIs
Previously it was possible to execute GET /_aliases,_mappings or GET /myindex/_settings,_alias by separating mulitple types of requests with commas in order to retrieve multiple types of information about one or more indices. This comma-separation for retrieving multiple pieces of information has been removed.. GET /_all can be used to retrieve all aliases, settings, and mappings for all indices. In order to retrieve only the mappings for an index, GET /myindex/_mappings (or _aliases, or _settings).

---

Does the client concat indices information requests together?

[codebrain]

Search and Query DSL changes
The deprecated type and slop parameter for the match query have been removed. Instead of setting the type, the match_phrase or match_phrase_prefix should be used. The slop removed from the match query but is supported for match_phrase and match_phrase_prefix.

---

Should it be removed from IMultiMatchQuery?

[Mpdreamz]

@codebrain

timestamp and ttl in index requests, yes these should be removed. deprecated in 2.x they were kept around in 5.x for 2.x indices that moved over.

Refresh requests with one or more shard failures return HTTP 500 response instead of 200
IndexClosedException to return 400 status code
These should both be handled with the new response deserialization handling in master.

Requests to existing endpoints with incorrect HTTP verb now return 405 responses
These should be covered by the low level client autobaking the correct verbs. None of our integration tests unearthed this either.

fielddata_fields

Yes these should still be returned by the stats API, you can no longer select them to be returned as part of the search but they can still be used to drive aggregations

[codebrain]

@Mpdreamz - Have asked about ttl parameters (and timestamp) as they still seem to be present in the REST specs: https://github.com/elastic/elasticsearch/search?utf8=%E2%9C%93&q=Expiration+time+for+the+document

[russcam]

@codebrain They should have been removed from the REST API spec. Would you like to open a PR to remove them (against elasticsearch master)? Typical flow would be to

1. fork elasticsearch to your own account
2. set up elastic/elasticsearch as upstream for the fork
3. branch from fork master
4. Make REST API changes.
5. Open PR from fork to upstream/master.

[codebrain]

Removed TTL and Timestamp from requests: elastic/elasticsearch#27888

Will need to pull & re-gen REST specs when I backport to 6.x branches.

[codebrain]

Merged ttl & timestamp removal into elasticsearch:master and backported to elasticsearch:6.x, 6.0 and 6.1

[codebrain]

The _all meta field is now disabled by default

The _all meta field is now disabled by default
On new mappings, the _all meta field that contains a copy of the text from each field is now disabled by default. The query_string and simple_query_string queries that previously used _all to search will now check if _all is enabled/disabled and switch to executing the query across all fields if _all is disabled. _all can no longer be configured for indices created with Elasticsearch version 6.0 or later.

---

Since this can be re-enabled I have left it in the codebase.

[Mpdreamz]

Deprecated percolator and mpercolate apis have been removed

Already removed as part of #2680

[Mpdreamz]

Stored search template apis removed

#2951 covers this, the crud api's have been removed but render search template API still exists.

https://www.elastic.co/guide/en/elasticsearch/reference/6.x/search-template.html

[Mpdreamz]

The index parameter in the terms filter, used to look up terms in a dedicated index is now mandatory. Previously, the index defaulted to the index the query was executed on. Now this index must be explicitly set in the request.

No action required on our end, fall back to server validation here.