Skip to content

Dynamic date format fails to parse if multiple formats in list #41666

Closed
@yoogie

Description

@yoogie

Elasticsearch version (bin/elasticsearch --version):
Version: 7.0.0, Build: default/zip/b7e28a7/2019-04-05T22:55:32.697037Z, JVM: 1.8.0_121

Plugins installed: []

JVM version (java -version):
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

OS version (uname -a if on a Unix-like system):
win7

Description of the problem including expected versus actual behavior:
When using dynamic mapping for date fields, it is possible to index documents of various (correct) date formats to the same field. However it is not possible to index a single document with a list of fields with different formats when creating the index.

Steps to reproduce:
Using Kibana console:
Add template with dynamic template

POST _template/example_template
{
  "order": 1,
  "index_patterns": [
    "idx*"
  ],
  "settings": {
    "number_of_replicas": 0,
    "number_of_shards": 1
  },
  "mappings": {
    "dynamic_templates": [
      {
        "date_fields": {
          "match_mapping_type": "date",
          "mapping": {
            "type": "date",
            "format": "date_time||date_time_no_millis||date_hour_minute_second_fraction||date_hour_minute_second_millis"
          }
        }
      }
      ],
    "properties": {
    }
  }
}

Index documents with different format of the same field to the same index:

POST idx/_doc/1
{
  "outer_doc" : {
    "list_of_entities" : [
      {
        "at": "2019-04-30T07:20:11.01"
      }
      ]
  }
}

POST idx/_doc/2
{
  "outer_doc" : {
    "list_of_entities" : [
      {
        "at": "2019-04-30T07:20:11.011Z"
      }
      ]
  }
}

At this point it is possible to index a document with different formats in the same document (since the mapping is now created for the index):

POST idx/_doc/101
{
  "outer_doc" : {
    "list_of_entities" : [
      {
        "at": "2019-04-30T07:20:11.01"
      },
      {
        "at": "2019-04-30T07:20:11.011Z"
      }
      ]
  }
}

How ever, adding the last document to a new index is not possible:

DELETE idx

POST idx/_doc/101
{
  "outer_doc" : {
    "list_of_entities" : [
      {
        "at": "2019-04-30T07:20:11.01"
      },
      {
        "at": "2019-04-30T07:20:11.011Z"
      }
      ]
  }
}

Please include a minimal but complete recreation of the problem, including
(e.g.) index creation, mappings, settings, query etc. The easier you make for
us to reproduce it, the more likely that somebody will take the time to look at it.

  1. Add template
  2. Index document with different formats of date

Provide logs (if relevant):

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "mapper [outer_doc.list_of_entities.at] of different type, current_type [date], merged_type [text]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "mapper [outer_doc.list_of_entities.at] of different type, current_type [date], merged_type [text]"
  },
  "status": 400
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Core/Infra/CoreCore issues without another label:Search Foundations/MappingIndex mappings, including merging and defining field types>bugTeam:Search FoundationsMeta label for the Search Foundations team in Elasticsearch

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions