geoip processor adds `_geoip_database_unavailable_*` tag to docs with missing source field

[axw]

Elasticsearch Version

8.13.0-SNAPSHOT

Installed Plugins

No response

Java Version

18

OS Version

Linux (Docker)

Problem Description

The geoip ingest processor adds a _geoip_database_unavailable_* tag field to documents when the database cannot be downloaded, even if the source field does not exist and ignore_missing is set.

Steps to Reproduce

1. Run Elasticsearch with ingest.geoip.downloader.enabled=false
2. Execute the following request in the console:

POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "geoip": {
          "field": "client.ip",
          "target_field": "client.geo",
          "ignore_missing": true
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_source": {}
    },
    {
      "_index": "index",
      "_source": {"client": {"ip": "8.8.8.8"}}
    }
  ]
}

This produces:

{
  "docs" : [
    {
      "doc" : {
        "_index" : "index",
        "_id" : "_id",
        "_source" : {
          "tags" : [
            "_geoip_database_unavailable_GeoLite2-City.mmdb"
          ]
        },
        "_ingest" : {
          "timestamp" : "2022-06-03T02:33:16.49692803Z"
        }
      }
    },
    {
      "doc" : {
        "_index" : "index",
        "_id" : "_id",
        "_source" : {
          "client" : {
            "ip" : "8.8.8.8"
          },
          "tags" : [
            "_geoip_database_unavailable_GeoLite2-City.mmdb"
          ]
        },
        "_ingest" : {
          "timestamp" : "2022-06-03T02:33:16.496934873Z"
        }
      }
    }
  ]
}

I would expect only the second document to have the tag added.

Logs (if relevant)

No response

[elasticmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[martijnvg]

The ignore_missing parameter is about ignoring the error if the field defined by the field parameter (client.ip in this case) cannot be found.

I agree that allowing to ignore a missing databases (by not adding tags) should be possible. I just wonder whether that should be done by the current ignore_missing parameter or perhaps via a new parameter (ignore_missing_database?). So that the tags field isn't added to a document if database is missing.

[axw]

@martijnvg the docs for ignore_missing state

If true and field does not exist, the processor quietly exits without modifying the document

This is not being upheld in the example above: the first document, which lacks client.ip, is being modified.

It may also be useful to have a separate flag to disable the tags addition in case of errors, but I still think either the behaviour of ignore_missing needs to change, or the docs do.

[martijnvg]

This is not being upheld in the example above: the first document, which lacks client.ip, is being modified.

I overlooked that 2 documents were provided in the simulate api.
I agree, that the first document shouldn't have the tags field.