Support case_insensitive for 'Include' of Terms Aggregation

[tommymonk]

I noticed the recent 7.10 release bringing support for case insensitivity to more parts of the DSL (61546 & 59441)

I require the ability to filter Terms Aggregation buckets in a case-insensitive way.

GET /_search
{
  "aggs": {
    "tags": {
      "terms": {
        "field": "tags",
        "include": ".*sport.*",
        "include_case_insensitive": true
      }
    }
  }
}

[elasticmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[tommymonk]

Is there any update on this ?

[tommymonk]

@cbuescher can someone take a look at this please.

[tommymonk]

If this is being ignored because there is some easy workaround, please enlighten me

[awenger63]

Hi,
Perhaps you can add this to find result, but with this workarround, the result it's not the real value ...
"script": { "source": "_value.toLowerCase()" },

[tommymonk]

Thanks @awenger63 I think that may work but then someone would raise a bug that the values are all lowercase, I will try thinking of a reliable way to fix the casing in code afterwards.

[nik9000]

Hey! I've been ignoring this one just because I can't read all the things and I figured someone else on the team was doing it. I was wrong. Anyway, would something this do what you want?

GET /_search
{
  "runtime_mappings": {
    "matching_tags": {
      "params": {
        "substring": "sport"
      },
      "script": """
for (String tag : doc.tags.values) {
  if (tag.toLowerCase(Locale.ROOT).contains(params.substring)) {
    emit(tag);
  }
}
      """
    }
  },
  "aggs": {
    "tags": {
      "terms": {
        "field": "matching_tags"
      }
    }
  }
}

Its certainly not the simplest work around because you have to lean on runtime fields and write a script instead of set a single setting but it does give you a bunch of flexibility. I think the script itself won't dominate the runtime here - I think you'll see the map flavored execution of terms doing all the work. But I think that'd be ok.

[tommymonk]

Thanks @nik9000 I'll give this workaround a go when I get a chance.

Do you think there is much chance of this proposal getting worked as considering it's lack of complexity ?

[nik9000]

I think it's the kind of thing that we'd take a contribution for but won't have time to build ourselves. At least, I don't have time and don't know anyone who does.

I think I made a mess of the runtime field definition in my comment above. Sorry. I'll have to refer you to the docs to correct my mistakes: https://www.elastic.co/guide/en/elasticsearch/reference/7.14/runtime-search-request.html . That's what i get for trying to write a request at 7:45 without enough coffee in me.