Skip to content

[meta] 7.6 release #4341

New issue
New issue
Closed
Closed
[meta] 7.6 release#4341
@russcam

Description

@russcam
russcam
opened on Feb 3, 2020

Preliminary release notes for 7.6.0

(Generated from asciidoc release notes on 2020-02-03)

Breaking changes

Mapping

  • Add a cluster setting to disallow loading fielddata on _id field #49166 (issues: #26472 #43599)

Breaking Java changes

Security

  • Support Client and RoleMapping in custom Realms
    #50534
    (issue:
    #48369)

Deprecations

Analysis

Authorization

  • Deprecating kibana_user and kibana_dashboard_only_user roles
    #46456

Distributed

  • Deprecate synced flush
    #50835
    (issue:
    #50776)

  • Deprecate indices without soft-deletes
    #50502

Features/Indices APIs

  • Emit warnings when index templates have multiple mappings
    #50982

Infra/Core

  • Deprecate the local parameter of /_cat/nodes
    #50499
    (issue:
    #50088) will be picked up by the generator

MULTIPLE AREA LABELS

  • Ensure we emit a warning when using the deprecated template field.
    #50831
    (issue:
    #49460)

  • Add a deprecation warning regarding allocation awareness in search
    request
    #48351
    (issue:
    #43453)

Reindex

Search

  • Update the signature of vector script functions.
    #48604

  • Deprecate the sparse_vector field type.
    #48315

New features

Aggregations

Analysis

Features/ILM+SLM

Features/Ingest

MULTIPLE AREA LABELS

  • Password Protected Keystore (Feature Branch)
    #49210

Machine Learning

Mapping

Search

Enhancements

Aggregations

  • Add reusable HistogramValue object
    #49799
    (issue:
    #49683)

  • Optimize composite aggregation based on index sorting
    #48399
    (issue:
    #48130)

Allocation

  • Auto-expand indices according to allocation filtering rules
    #48974

  • Do not cancel ongoing recovery for noop copy on broken node
    #48265
    (issue:
    #47974)

  • Quieter logging from the DiskThresholdMonitor
    #48115
    (issue:
    #48038)

  • Faster access to INITIALIZING/RELOCATING shards
    #47817
    (issues:
    #46941,
    #48579)

Analysis

Authentication

  • Add Debug/Trace logging for authentication
    #49575
    (issue:
    #49473)

Authorization

  • Increase Size and lower TTL on DLS BitSet Cache
    #50535
    (issues:
    #43669,
    #49260)

  • Add monitor_snapshot cluster privilege
    #50489
    (issue:
    #50210)

  • Remove reserved roles for code search
    #50068
    (issue:
    #49842)

  • [Code] Remove code_admin/code_user roles
    #48164

  • Resolve the role query and the number of docs lazily
    #48036

CCR

  • Improve error message when pausing index
    #48915

  • Use MultiFileTransfer in CCR remote recovery
    #44514
    (issue:
    #44468)

CRUD

  • print id detail when id is too long.
    #49433

  • Add preflight check to dynamic mapping updates
    #48817
    (issue:
    #35564)

Cluster Coordination

Distributed

  • Use retention lease in peer recovery of closed indices
    #48430
    (issue:
    #45136)

Engine

  • Do not force refresh when write indexing buffer
    #50769

  • Deleted docs disregarded for if_seq_no check
    #50526

  • Allow realtime get to read from translog
    #48843

  • Do not warm up searcher in engine constructor
    #48605
    (issue:
    #47186)

  • Add a new merge policy that interleaves old and new segments on
    force merge
    #48533
    (issue:
    #37043)

  • Refresh should not acquire readLock
    #48414
    (issue:
    #47186)

Features/ILM+SLM

Features/Ingest

Features/Java High Level REST Client

Features/Monitoring

  • Significantly Lower Monitoring HttpExport Memory Footprint
    #48854

  • Validate proxy base path at parse time
    #47912
    (issue:
    #47711)

  • Validate index name time format setting at parse time
    #47911
    (issue:
    #47711)

  • Validate monitoring header overrides at parse time
    #47848
    (issue:
    #47711)

  • Validate monitoring username at parse time
    #47821
    (issue:
    #47711)

  • Validate monitoring password at parse time
    #47740
    (issue:
    #47711)

Features/Watcher

  • Log attachment generation failures
    #50080

  • Don’t dump a stacktrace for invalid patterns when executing
    elasticse…
    #49744
    (issue:
    #49642)

Geo

Infra/Core

  • Add "did you mean" to ObjectParser
    #50938

  • Consistent case in CLI option descriptions
    #49635

  • Improve resiliency to formatting JSON in server
    #48553
    (issue:
    #48450)

  • Don’t close stderr under --quiet
    #47208
    (issue:
    #46900)

Infra/Packaging

  • Respect ES_PATH_CONF on package install
    #50158

  • Restrict support for CMS to pre-JDK 14
    #49123
    (issue:
    #46973)

  • Remove parsed JVM settings from general settings in Windows service
    daemon manager
    #49061
    (issue:
    #48796)

  • Package the JDK into jdk.app on macOS
    #48765

  • Add UBI-based Docker images
    #48710
    (issue:
    #48429)

Infra/Plugins

  • Report progress of multiple plugin installs
    #51001
    (issue:
    #50924)

  • Allow installing multiple plugins as a transaction
    #50924
    (issue:
    #50443)

Infra/Scripting

  • Scripting: ScriptFactory not required by compile
    #50344
    (issue:
    #49466)

  • Scripting: Cache script results if deterministic
    #50106
    (issue:
    #49466)

  • Scripting: Groundwork for caching script results
    #49895
    (issue:
    #49466)

  • Scripting: add available languages & contexts API
    #49652
    (issue:
    #49463) EXPERIMENTAL API

  • Scripting: fill in get contexts REST API
    #48319
    (issue:
    #47411) EXPERIMENTAL API

  • Scripting: get context names REST API
    #48026
    (issue:
    #47411) EXPERIMENTAL API

Infra/Settings

  • Add parameter to make sure that log of updating IndexSetting be more
    detailed
    #49969
    (issue:
    #49818)

  • Enable dependent settings values to be validated
    #49942

  • Do not reference values for filtered settings
    #48066

License

MULTIPLE AREA LABELS

Machine Learning

  • [ML] Add audit warning for 1000 categories found early in job
    #51146
    (issue:
    #50749)

  • [ML] Add num_top_feature_importance_values param to regression
    and classi…
    #50914

  • [ML] Implement force deleting a data frame analytics job
    #50553
    (issue:
    #48124) will be picked up by generator

  • [ML] Delete unused data frame analytics state
    #50243

  • Make each analysis report desired field mappings to be copied
    #50219
    (issue:
    #50119)

  • [ML] retry bulk indexing of state docs
    #50149
    (issue:
    #50143)

  • [ML] Persist/restore state for DFA classification
    #50040

  • [ML] Introduce randomize_seed setting for regression and
    classification
    #49990

  • Pass prediction_field_type to C++ analytics process
    #49861
    (issue:
    #49796)

  • [ML] Add optional source filtering during data frame reindexing
    #49690
    (issue:
    #49531)

  • [ML] Add default categorization analyzer definition to ML info
    #49545 PR Add categorization analyzer to defaults on ml info response #4408

  • [ML] Add graceful retry for anomaly detector result indexing
    failures
    #49508
    (issue:
    #45711)

  • Lower minimum model memory limit value from 1MB to 1kB.
    #49227
    (issue:
    #49168)

  • Throw an exception when memory usage estimation endpoint encounters
    empty data frame.
    #49143
    (issue:
    #49140)

  • Change format of MulticlassConfusionMatrix result to be more
    self-explanatory
    #48174
    (issue:
    #46735)

  • Make num_top_classes parameter’s default value equal to 2
    #48119
    (issue:
    #46735)

  • [ML] Improve model_memory_limit UX for data frame analytics jobs
    [ISSUE]
    #44699

Mapping

Network

  • Add certutil http command
    #49827

  • Do not load SSLService in plugin contructor
    #49667
    (issue:
    #44536)

  • Netty4: switch to composite cumulator
    #49478

  • Add the simple strategy to cluster settings
    #49414
    (issue:
    #49067)

  • Deprecate misconfigured SSL server config
    #49280
    (issue:
    #45892)

  • Improved diagnostics for TLS trust failures
    #48911

Percolator

  • Refactor percolator’s QueryAnalyzer to use QueryVisitors
    #49238
    (issue:
    #45639)

Ranking

  • Support search_type in Rank Evaluation API
    #48542
    (issue:
    #48503)

Recovery

  • Use peer recovery retention leases for indices without soft-deletes
    #50351
    (issues:
    #45136,
    #46959)

  • Recovery buffer size 16B smaller
    #50100

Reindex

  • Reindex sort deprecation warning take 2
    #49855
    (issue:
    #49458)

SQL

  • SQL: Add TRUNC alias for TRUNCATE
    #49571
    (issue:
    #41195)

  • SQL: binary communication implementation for drivers and the CLI
    #48261
    (issue:
    #47785)

Search

  • Add Validation for maxQueryTerms to be greater than 0 for
    MoreLikeThisQuery
    #49966
    (issue:
    #49927)

  • Optimize numeric sort on match_all queries
    #49717
    (issue:
    #48804)

  • Pre-sort shards based on the max/min value of the primary sort field
    #49092
    (issue:
    #49091)

  • Optimize sort on long field
    #48804

  • Search optimisation - add canMatch early aborts for queries on
    "_index" field
    #48681
    (issue:
    #48473)

  • #48475 Pure disjunctions should rewrite to a MatchNoneQueryBuilder
    #48557

  • Disable caching when queries are profiled
    #48195
    (issue:
    #33298)

  • BlendedTermQuery’s equals method should consider boosts
    #48193
    (issue:
    #48184)

  • Increase the number of vector dims to 2048
    #46895

Security

  • Make .async-search-* a restricted namespace
    #50294

  • Security should not reload files that haven’t changed
    #50207
    (issue:
    #50063)

Snapshot/Restore

Store

Transform

  • [Transform] Improve force stop robustness in case of an error
    #51072

  • [Transform] add actual timeout in message
    #50140

  • [Transform] automatic deletion of old checkpoints
    #49496

  • [Transform] improve error handling of script errors
    #48887
    (issue:
    #48467)

  • [ML][Transforms] add wait_for_checkpoint flag to stop
    #47935
    (issue:
    #45293)

Bug fixes

Aggregations

  • Fix format problem in composite of unmapped
    #50869
    (issue:
    #50600)

  • SingleBucket aggs need to reduce their bucket’s pipelines first
    #50103
    (issue:
    #50054)

  • Avoid precision loss in DocValueFormat.RAW#parseLong
    #49063
    (issue:
    #38692)

  • Fix ignoring missing values in min/max aggregations
    #48970
    (issue:
    #48905)

Allocation

  • Collect shard sizes for closed indices
    #50645
    (issue:
    #33888)

  • Auto-expand replicated closed indices
    #48973

  • Ignore dangling indices created in newer versions
    #48652
    (issue:
    #34264)

  • Handle negative free disk space in deciders
    #48392
    (issue:
    #48380)

Analysis

  • Throw Error on deprecated nGram and edgeNGram custom filters
    #50376
    (issue:
    #50360)

  • _analyze api does not correctly use normalizers when specified
    #48866
    (issue:
    #48650)

Audit

Authentication

  • Don’t fallback to anonymous for tokens/apikeys
    #51042
    (issue:
    #50171)

  • Populate User metadata with OpenIDConnect collections
    #50521
    (issue:
    #50250)

  • Always return 401 for not valid tokens
    #49736
    (issue:
    #38866)

  • Fix iterate-from-1 bug in smart realm order
    #49473

  • Remove unnecessary details logged for OIDC
    #48746

  • Add owner flag parameter to the rest spec
    #48500
    (issue:
    #48499) will be picked up by generator.

Authorization

  • Fix memory leak in DLS bitset cache
    #50635
    (issue:
    #49261)

  • Validate field permissions when creating a role
    #50212
    (issues:
    #46275,
    #48108)

  • Validate field permissions when creating a role
    #48108
    (issue:
    #46275)

CCR

  • CCR should auto-retry rejected execution exceptions
    #49213

CRUD

  • Block too many concurrent mapping updates
    #51038
    (issue:
    #50670)

  • Ensure meta and document field maps are never null in GetResult
    #50112
    (issue:
    #48215)

  • Replicate write actions before fsyncing them
    #49746

  • Do not mutate request on scripted upsert
    #49578
    (issue:
    #48670)

  • Fix Transport Stopped Exception
    #48930
    (issue:
    #42612)

  • Return consistent source in updates
    #48707

  • Close query cache on index service creation failure
    #48230
    (issue:
    #48186)

Cluster Coordination

  • Import replicated closed dangling indices
    #50649

  • Ignore metadata of deleted indices at start
    #48918

  • Make elasticsearch-node tools custom metadata-aware
    #48390

Discovery-Plugins

  • Make EC2 Discovery Cache Empty Seed Hosts List
    #50607
    (issue:
    #50550)

  • Make EC2 Discovery Plugin Retry Requests
    #50550
    (issue:
    #50462)

Distributed

Engine

  • Account trimAboveSeqNo in committed translog generation
    #50205
    (issue:
    #49970)

  • Greedily advance safe commit on new global checkpoint
    #48559
    (issue:
    #48532)

  • Do not ignore exception when trim unreferenced readers
    #48470

Features/Features

  • Fix X-Pack SchedulerEngine Shutdown
    #48951

Features/ILM+SLM

  • Fix SLM check for restore in progress
    #50868

  • Handle failure to retrieve ILM policy step better
    #49193
    (issue:
    #49128)

  • Don’t halt policy execution on policy trigger exception
    #49128

  • Re-read policy phase JSON when using ILM’s move-to-step API
    #48827

  • Don’t schedule SLM jobs when services have been stopped
    #48658
    (issue:
    #47749)

  • Ensure SLM stats does not block an in-place upgrade from 7.4
    #48367

  • Ensure SLM stats does not block an in-place upgrade from 7.4
    #48361

  • Add SLM support to xpack usage and info APIs
    #48096
    (issue:
    #43663)

  • Change policy_id to list type in slm.get_lifecycle
    #47766
    (issue:
    #47765) will be picked up by generator

Features/Ingest

Features/Java High Level REST Client

  • Support es7 node http publish_address format
    #49279
    (issue:
    #48950)

  • Add slices to delete and update by query in HLRC
    #48420

  • fix incorrect comparison
    #48208

  • Fix HLRC parsing of CancelTasks response
    #47017

Features/Java Low Level REST Client

  • Improve warning value extraction performance in Response
    #50208
    (issue:
    #24114)

Features/Monitoring

Geo

  • Guard against null geoBoundingBox
    #50506
    (issue:
    #50505)

  • Geo: Switch generated GeoJson type names to camel case (#50285)
    #50400
    (issue:
    #49568)

  • Geo: Switch generated WKT to upper case
    #50285
    (issue:
    #49568)

  • Fix typo when assigning null_value in GeoPointFieldMapper
    #49645

  • Fix handling of circles in legacy geo_shape queries
    #49410
    (issue:
    #49296)

  • GEO: intersects search for geo_shape return wrong result [ISSUE]
    #49017

  • Geo: improve handling of out of bounds points in linestrings
    #47939
    (issue:
    #43916)

Highlighting

  • Fix invalid break iterator highlighting on keyword field
    #49566

Infra/Core

  • Ignore virtual ethernet devices that disappear
    #51581
    (issue:
    #49914)

  • Guess root cause support unwrap
    #50525
    (issue:
    #50417)

  • Allow parsing timezone without fully provided time
    #50178
    (issue:
    #49351)

  • [Java.time] Retain prefixed date pattern in formatter
    #48703
    (issue:
    #48698)

  • Don’t drop user’s MaxDirectMemorySize flag on jdk8/windows
    #48657
    (issues:
    #44174,
    #48365)

  • Warn when MaxDirectMemorySize may be incorrect (Windows/JDK8 only
    issue)
    #48365
    (issue:
    #47384)

  • [Java.time] Calculate week of a year with ISO rules
    #48209
    (issues:
    #41670,
    #42588,
    #43275,
    #43652)

Infra/Logging

  • Slow log must use separate underlying logger for each index
    #47234
    (issue:
    #42432)

Infra/Packaging

  • Extend systemd timeout during startup
    #49784
    (issue:
    #49593)

Infra/REST API

  • Return 400 when handling invalid JSON
    #49552
    (issue:
    #49428)

  • Slash missed in indices.put_mapping url
    #49468

MULTIPLE AREA LABELS

Machine Learning

  • [ML] Validate classification dependent_variable cardinality is at
    lea… #51232

  • Do not copy mapping from dependent variable to prediction field in
    regression analysis
    #51227

  • Handle nested and aliased fields correctly when copying mapping.
    #50918
    (issue:
    #50787)

  • [ML] Fix off-by-one error in ml_classic tokenizer end offset
    #50655

  • [ML] Improve uniqueness of result document IDs
    #50644
    (issue:
    #50613)

  • [7.x] Synchronize processInStream.close() call
    #50581
    (issue:
    #49680)

  • Fix accuracy metric
    #50310
    (issue:
    #48759)

  • Fix race condition when stopping a data frame analytics jobs
    immediately after starting it
    #50276
    (issues:
    #49680,
    #50177)

  • Use query in cardinality check
    #49939

  • Make only a part of stop() method a critical section.
    #49756
    (issue:
    #49680)

  • Fix expired job results deletion audit message
    #49560
    (issue:
    #49549)

  • [ML] Apply source query on data frame analytics memory estimation
    #49517
    (issue:
    #49454)

  • Stop timing stats failure propagation
    #49495

  • [ML] Fix r_squared eval when variance is 0
    #49439

  • Blacklist a number of prediction field names.
    #49371
    (issue:
    #48808)

  • Make AnalyticsProcessManager class more robust
    #49282
    (issue:
    #49095)

  • [ML] Fixes for stop datafeed edge cases
    #49191
    (issues:
    #43670,
    #48931)

  • [ML] Avoid NPE when node load is calculated on job assignment
    #49186
    (issue:
    #49150)

  • Do not throw exceptions resulting from persisting datafeed timing
    stats.
    #49044
    (issue:
    #49032)

  • [ML] Deduplicate multi-fields for data frame analytics
    #48799
    (issues:
    #48756,
    #48770)

  • [ML] Prevent fetching multi-field from source
    #48770
    (issue:
    #48756)

Mapping

  • Ensure that field collapsing works with field aliases.
    #50722
    (issues:
    #32648,
    #50121)

  • Improve DateFieldMapper ignore_malformed handling
    #50090
    (issues:
    #46675,
    #50081)

  • Annotated text type should extend TextFieldType
    #49555
    (issue:
    #49289)

  • Ensure parameters are updated when merging flattened mappings.
    #48971
    (issue:
    #48907)

Network

  • Fix TransportMasterNodeAction not Retrying NodeClosedException
    #51325

Percolator

  • Correctly handle MSM for nested disjunctions
    #50669
    (issue:
    #50305)

  • Fix query analyzer logic for mixed conjunctions of terms and ranges
    #49803
    (issue:
    #49684)

Recovery

  • Check allocation id when failing shard on recovery
    #50656
    (issue:
    #50508)

  • Migrate peer recovery from translog to retention lease
    #49448
    (issue:
    #45136)

  • Ignore Lucene index in peer recovery if translog corrupted
    #49114

Reindex

SQL

Search

  • Fix upgrade of custom similarity
    #50851
    (issue:
    #50763)

  • Fix NPE bug inner_hits
    #50709
    (issue:
    #50539)

  • Collect results in a local list when notifying partial results
    #49828
    (issue:
    #49778)

  • Fixes a bug in interval filter serialization
    #49793
    (issue:
    #49519)

  • Correctly handle duplicates in unordered interval matching
    #49775

  • Correct rewritting of script_score query
    #48425
    (issue:
    #48081)

  • Do not throw errors on unknown types in SearchAfterBuilder
    #48147
    (issue:
    #48074)

Security

  • Always consume the body in has privileges
    #50298
    (issue:
    #50288)

Snapshot/Restore

  • Fix Overly Aggressive Request DeDuplication
    #51270
    (issue:
    #51253)

  • Guard Repository#getRepositoryData for exception throw
    #50970

  • Fix Index Deletion During Partial Snapshot Create
    #50234
    (issues:
    #50200,
    #50202)

  • Fix Index Deletion during Snapshot Finalization
    #50202
    (issues:
    #45689,
    #50200)

  • Fix RepoCleanup not Removed on Master-Failover
    #49217

  • Make FsBlobContainer Listing Resilient to Concurrent Modifications
    #49142
    (issue:
    #37581)

  • Fix SnapshotShardStatus Reporting for Failed Shard
    #48556
    (issue:
    #48526)

  • Cleanup Concurrent RepositoryData Loading
    #48329
    (issue:
    #48122)

Transform

  • [Transform] fail to start/put on missing pipeline
    #50701
    (issue:
    #50135)

  • Fix possible audit logging disappearance after rolling upgrade
    #49731
    (issue:
    #49730)

  • [Transform] do not fail checkpoint creation due to global
    checkpoint mismatch
    #48423
    (issue:
    #48379)

Upgrades

Engine

  • Upgrade to Lucene 8.4.0.
    #50518

Infra/Packaging

  • Upgrade the bundled JDK to JDK 13.0.2
    #51511

NOT CLASSIFIED

Aggregations

  • Revert "Begin moving date_histogram to offset rounding (backport of
    #50873) (#50978)"
    #51240

Authentication

  • Change password API authenticating with a bearer token [ISSUE]
    #48752

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions