Types removal in 8.0

[jpountz]

This is a meta issue that tracks things that must or should be done before 8.0 in order to finalize the removal of types.

Remaining:

• REST-layer version compatibility
• Rewrite Removal of Types doc to point to 7x branch Remove [removal-of-types] docs page, and point to 7x docs #53670
• Master nodes should refuse to join a cluster with multi-typed index templates Handling index templates with multiple mapping types #53018
• Test rolling upgrades from 7x with typed mappings and templates Create upgrade tests for typed mappings #48800
• Remove ability to reference _type in scripts via LeafDocLookup Remove deprecation warning when doc scripts refer to '_type' field #53605
• Add deprecation warning to TermsLookup#parseTermsLookup() in 7x when type is included Emit deprecation warning when TermsLookup contains a type #53731
• Remove TypeFieldMapper

Longer term:

• Merge back FieldMapper and MappedFieldType (issue: Merge FieldMapper and MappedFieldType #41058)
• Remove String type parameter from all internal code, eg. MapperService.

MUST

• Remove typed URL paths
  • RestNoopBulkAction @romseygeek (Remove types from client-benchmark-noop-api-plugin URLs #46668)
  • RestNoopSearchAction @romseygeek (Remove types from client-benchmark-noop-api-plugin URLs #46668)
  • RestMultiSearchTemplateAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestSearchTemplateAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestDeleteByQueryAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestUpdateByQueryAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestGetFieldMappingAction @colings86 (Removes typed URLs from mapping APIs #41676)
  • RestGetMappingAction @colings86 (Removes typed URLs from mapping APIs #41676)
  • RestPutMappingAction @colings86 (Removes typed URLs from mapping APIs #41676)
  • RestValidateQueryAction @romseygeek (Remove types from ValidateQuery API #46927)
  • RestBulkAction @romseygeek (Remove types from BulkRequest #46983)
  • RestDeleteAction @romseygeek (Remove type field from DocWriteRequest and associated Response objects #47671)
  • RestGetAction @romseygeek (Remove types from Get/MultiGet #46587)
  • RestGetSourceAction @romseygeek (Remove types from _source endpoint #46931)
  • RestIndexAction @romseygeek (Remove type field from DocWriteRequest and associated Response objects #47671)
  • RestMultiGetAction @romseygeek (Remove types from Get/MultiGet #46587)
  • RestMultiTermVectorsAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestTermVectorsAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestUpdateAction @romseygeek (Remove type field from DocWriteRequest and associated Response objects #47671)
  • RestCountAction @colings86 (Removes types from SearchRequest and QueryShardContext #42112)
  • RestExplainAction @romseygeek (Remove types from explain API #46926)
  • RestMultiSearchAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestSearchAction @colings86 (Removes typed endpoint from search and related APIs #41640)
  • RestGraphAction @romseygeek (Remove types from Graph API #46935)
  • RestSimulatePipelineAction @romseygeek Remove type metadata from ingest documents #50131
• Remove types from Request objects
  • SearchRequest @colings86 (Removes types from SearchRequest and QueryShardContext #42112)
    • QueryShardContext @colings86 (Removes types from SearchRequest and QueryShardContext #42112)
  • TermVectorsRequest @colings86 (Removes type from TermVectors APIs #42198)
  • ValidateQueryRequest @romseygeek (Remove types from ValidateQuery API #46927)
  • ExplainRequest @romseygeek (Remove types from explain API #46926)
  • RoutingMissingException @romseygeek (Remove types from Get/MultiGet #46587)
• Remove types from the query DSL
  • AbstractGeometryQueryBuilder @romseygeek (Remove types from GeoShapeQueryBuilder #47792)
  • TermsQueryBuilder @romseygeek (Remove types from TermsLookup #46943)
  • PercolateQueryBuilder @romseygeek (Remove types from percolate query API #46985)
• Remove support for include_type_name in index creation, mapping and template APIs. Remove include_type_name parameter from REST layer #48632
• Remove type parameters from the node client
  • prepareIndex()
  • prepareDelete()
  • prepareUpdate()
  • PutMappingRequest
• Remove _type from GET responses @romseygeek (Remove types from Get/MultiGet #46587)
• Remove _type from _search responses. @romseygeek (Remove _type from SearchHit #46942)
• Remove searches on the _type field (e.g. in terms lookup and percolator queries)
  • Remove type query Remove type query #47207
  • Disallow using the _type field in queries (such as term and wildcard). (Add NestedPathFieldMapper to store nested path information #51100)
• Remove types from watcher's IndexAction (Remove type from Watcher IndexAction #47986)
• Remove types from WatcherSearchTemplateRequest. (Remove types from WatcherSearchTemplateRequest #51112)
• Remove deprecated typed methods on the high-level REST client. Remove deprecated include-type methods from HLRC indices client #48471

SHOULD

• Remove types from TermsLookup @romseygeek (Remove types from TermsLookup #46943)

• Remove type from SourceToParse (Don't persist type information to translog #47229)

• Remove types from IndexShard.applyXOperation methods (Upgrade to Lucene 8.5.0-snapshot-c4475920b08 (#52950) #52977

• Remove code that is related to the _default_ mapping. @romseygeek (Remove default mapping #44945)

• Remove the _type mapping field? (Add NestedPathFieldMapper to store nested path information #51100)

• InternalIndexingStats only needs one stats instance Remove per-type indexing stats #47203

KNOWN BUGS/ TRAPS

• Index creation and put template requests can accept malformed mappings: Index creation requests can accept malformed mappings. #38266

[elasticmachine]

Pinging @elastic/es-search

[jtibshirani]

I came across #24362 and am wondering if we should remove the _type field in 8.0 as well.

[jpountz]

Added, thanks @jtibshirani.

[jtibshirani]

@colings86 @jpountz just a note that I created a new section for 'known traps' to list surprises we encountered during the 7.x types removal effort that are good to be aware of.