Different behaviour between Object and Dotted notations on ingestion

[crisdarocha]

Elasticsearch version (bin/elasticsearch --version): 7.15.1

Description of the problem including expected versus actual behavior:

Trying to avoid fields explosion using the flattened type in a dynamic template.

PUT _template/mytemplate
{
  "order": 1,
  "index_patterns": [
    "myindex-*"
  ],
  "mappings": {
    "dynamic_templates": [
      {
        "no_deep_objects": {
          "path_match": "*.*.*",
          "match_mapping_type": "object",
          "mapping": {
            "type": "flattened"
          }
        }
      }
    ]
  }
}

When ingesting a document in Object notation it works fine:

POST myindex-01/_doc/
{
  "level1": {
    "level2": {
      "level3": {
        "level4": "myvalue"
      }
    }
  }
}

When ingesting the "same" document in Dotted notation it fails:

POST myindex-01/_doc/
{
  "level1.level2.level3.level4": "myvalue"
}

Error message:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Could not dynamically add mapping for field [level1.level2.level3.level4]. Existing mapping for [level1.level2.level3] must be of type object but found [flattened]."
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Could not dynamically add mapping for field [level1.level2.level3.level4]. Existing mapping for [level1.level2.level3] must be of type object but found [flattened]."
  },
  "status" : 400
}

One would expect the same behaviour, but unfortunately it's not happening.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) index creation, mappings, settings, query etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

1. Add a template that creates a flattened type after a certain level of nesting (see above)
2. Add a document in Object notation and see it succeed
3. Add a document in Dotted notation and see it fail

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[cbuescher]

I can reproduce this, just also tried the new "_index_template" endpoint like this:

PUT _index_template/mytemplate
{
  "index_patterns": [
    "myindex-*"
  ],
  "template": {
    "mappings": {
      "dynamic_templates": [
        {
          "no_deep_objects": {
            "path_match": "*.*.*",
            "match_mapping_type": "object",
            "mapping": {
              "type": "flattened"
            }
          }
        }
      ]
    }
  },
  "priority": 1
}

But adding the first doc with the dotted notation fails in the same way.

[romseygeek]

Interesting, I think #79922 will solve this. I'll add a new test to that PR to see if it helps.

[romseygeek]

Can confirm that this will be fixed by #79922

[javanna]

@romseygeek should this be closed?

[romseygeek]

Yes, it's fixed in 8.1 by #79922