[meta] 6.7.0 Release

[codebrain]

Leftover Work

New Features 6.5.0

• ML. Detectors now support custom rules that enable the user to improve machine learning results by providing some domain-specific knowledge in the form of rule. - [ML] Implement new rules design elasticsearch#31110 (@russcam)

6.6.0

Breaking changes

Cross-cluster replication:

• Change get autofollow patterns API response format NullPointerException in ConcurrentMergeScheduler elasticsearch#3620 (issue: [HLRC] Added support for CCR Get Auto Follow Pattern apis elasticsearch#36049) - breaking change in minor - implement custom json convertor on deserialize (@codebrain)

New features

Aggregations

• Median absolute deviation agg median absolute deviation agg elasticsearch#34482 (issue: Median Absolute Deviation elasticsearch#26681)

Analysis:

• Add support for inlined user dictionary in Nori Add support for inlined user dictionary in Nori elasticsearch#36123 (issue: [Feature Request][Nori ] Add custom dictionary terms on index setting. elasticsearch#35842) (@russcam)
• Add a prebuilt ICU Analyzer Add a prebuilt ICU Analyzer elasticsearch#34958 (issue: CJK analyzer tokenization issues elasticsearch#34285) (@russcam)

Index Lifecycle Management:

• Adds Index lifecycle feature Adds Index lifecycle feature elasticsearch#35193 - (see ilm* endpoints: https://github.com/elastic/elasticsearch/tree/master/x-pack/plugin/src/test/resources/rest-api-spec/api / https://www.elastic.co/guide/en/elasticsearch/reference/6.6/index-lifecycle-management-api.html) @codebrain (Implement Index Lifecycle APIs #3671)

Geo:

• Store job configuration information in the new .ml-config index [ML] Store ml job configurations in the new .ml-config index elasticsearch#36698 (issue: [ML] Metadata Migration Meta issue elasticsearch#32905) - looks like no impact on client

Search::

• Make lucene's IntervalQuery available via the Query DSL Make lucene's IntervalQuery available via the Query DSL elasticsearch#32406 (issue: Make IntervalQuery available via the Query DSL elasticsearch#29636) - Use documentation and YML from 7.0 - https://www.elastic.co/guide/en/elasticsearch/reference/7.0/query-dsl-intervals-query.html - https://github.com/elastic/elasticsearch/blob/7.0/rest-api-spec/src/main/resources/rest-api-spec/test/search/230_interval_query.yml

Enhancements

Aggregations

• Handles exists query in composite aggs #35758 - No client implementation needed.

Authentication

• Invalidate Token API enhancements - HLRC #36362 (says HLRC but adds to request/response) - @codebrain (Security Api Key functionality #3684)
• Add realm information for Authenticate API #35648
• Formal support for "password_hash" in Put User #35242 (issue: #34729) - Add password_hash parameter to PutUser #3723
• Enhance Invalidate Token API #35388 (issues: #34556, #35115)

Core

• Added wait_for_metadata_version parameter to cluster state api. #35535

Cross-cluster replication

• Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)

The last two removed CCR_AUTO_FOLLOW_POLL_INTERVAL which i think is returned on the ccr stats API's (investigate)

Bug fixes

Aggregations::

• Fix MultiValuesSourceFieldConfig toXContent [#36525] (issue: [#36474]) - no change needed, server side issue

circuit breakers::

• modify bigarrays to take name of circuit breaker [#36461] (issue: [#31435])

Core::

• Suppress CachedTimeThread in hot threads output [#35558] (issue: [#23175])

Cross-cluster replication::

• Add fatal_exception field for ccr stats in monitoring mapping [#37563]

Geo::

• Enables coerce support in WKT polygon parser [#35414] (issue: [#35059]) - Allow geo_shape types to be coerced into valid shapes. #3718

Rollup::

• Fix Rollup's metadata parser [#36791] (issue: [#36726]) - no client changes needed

Search::

• Inner hits fail to propagate doc-value format. (#36310) [#36355] (issue: [#36310]) - no client change needed, already in FieldJsonConverter
• Fix custom AUTO issue with Fuzziness#toXContent [#35807] (issue: [#33462]) - no client change needed, server fix only

Do we expose doc value format on inner hit?

Snapshot/Restore::

• Register Azure max_retries setting [#35286] - no client change needed