Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for additional mapping to simulate ingest API (#114742) #115284

Open
wants to merge 1 commit into
base: 8.x
Choose a base branch
from

Conversation

masseyke
Copy link
Member

This adds support for a new mapping_addition field to the body of an ingest simulate API request. This mapping addition can be used to change the mappings used for validation. The contents of mapping_addition are merged into the mapping of the index into which the data would be inserted (only for the duration of this request) just before validation.

## First we create a component template with a strict mapping with two fields:
PUT _component_template/mappings_template
{
  "template": {
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "foo": {
          "type": "keyword"
        },
        "bar": {
          "type": "boolean"
        }
      }
    }
  }
}

## Here we create an index template  pulling in the component template above
PUT _index_template/template_1
{
  "index_patterns": ["foo*"],
  "composed_of": ["mappings_template"]
}

## We can index a document here to create the index, or not. Either way the simulate call ought to work the same
POST foo-1/_doc
{
  "foo": "FOO",
  "bar": true
}

## Event though bar is defined as a boolean and there is no baz field in the strict mapping above, this
## will not blow up with validation exceptions because the "mapping_addition" modifies the definition
## of the bar field, and adds a definition of the baz field. The contents of mapping_addition take
## priority over the existing mapping (or any mappings that come from the
component_template_substitutions or index_template_substitutions objects).
POST _ingest/_simulate?pretty&index=foo-1
{
  "docs": [
    {
      "_id": "asdf",
      "_source": {
        "foo": "foo",
        "bar": "bar",
        "baz": "baz"
      }
    }
  ],
  "mapping_addition": {
      "properties": {
        "bar": {
          "type": "keyword"
        },
        "baz": {
          "type": "keyword"
        },
  }
}

@masseyke masseyke added >enhancement :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP backport v8.17.0 auto-merge labels Oct 21, 2024
Copy link
Contributor

Documentation preview:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge backport :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement v8.17.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant