Add ability to filter query based on data tier

[dakrone]

Some use cases have the desire to query data within a certain tier (or set of tiers), for example, in the presence of a data stream or alias using ILM, query data only in the "hot" tier. (See: #47881 where users have asked for ILM supporting aliases so that queries can target a specific lifecycle of data).

It could be nice to have a general purpose query that could be used for regular searching (as well as aggregations) that allowed specifying a "tier" of data to query. This would allow a query like:

{
  "query": {
    "bool": {
      "filter": {
        "tier": "hot"
      }
    }
  },
  "aggs": {...}
}

This is especially nice when users start using searchable snapshots for their data, as it would allow bypassing indices in other tiers (such as "cold" and "frozen") without requiring any sort of download of data.

One question that may come up is "why not just use a time range filter for getting the most recent data?". This is useful when only consuming a single set of data (such as a single data stream), but if we had a first-class query for data tier searching, multiple data streams and aliases could be queried that have differing "hot" tier definitions without requiring the user to both be aware of the timing for the tier and separate the filter range based on specific index patterns. For example: searching three data streams that have data in the hot phases for 7, 14, and 21 days respectively, using tier: hot is much simpler than specifying three different range filters tied to three different data stream index names.

This also helps some of the use cases in #47881 while being accessible to both data streams and aliases.

If this is of interest, we could perform the filtering for this prior to any query execution as the tier is accessible through the index metadata and could be rewritten to exclude indices that aren't in the specified tier.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[skearns64]

++ to this concept.

If I'm understanding it right, having this would make it easy to use DLS to make roles that can only query specific data tiers? e.g. regular users can query hot/warm. power users can also query Cold/Frozen.

[dakrone]

having this would make it easy to use DLS to make roles that can only query specific data tiers?

Yes that could definitely be a use for this as well!

[jpountz]

The easiest way to make it work would be to define a _tier metadata field, similar to the _index field.

[markharwood]

Given the way the tier checking code works would this make sense to have a metadata field called node_role which can be queried for values like data_warm?

I can see arguments both ways for tier == warm or node_role == data_warm fields/queries. The former provides some abstraction over implementation details while the latter allows for other future node roles to be testable too.

[markharwood]

Started with a PR for this but struggling - not sure how the field mapper API allows me to get hold of the current DiscoveryNode in order to get hold of node role settings so I can call a tester method on DataTier similar to this one,

The alternative is perhaps for the field mapper to get hold of the index settings and test index.routing.allocation.include._tier_preference which is more of an aspiration than a guarantee of tier.

[jimczi]

You only need the node settings to call the static function you linked. You can access them through IndexSettings#getNodeSettings. I don't think we should expose the discovery node in the SearchExecutionContext, looking at the node settings should be enough if you add a static function in DataTier that returns the list of roles.

[markharwood]

You can access them through IndexSettings#getNodeSettings

I see only a subset of settings using that - it is only the settings part of what I get if I call http://localhost:9200/_nodes/MY_NODE/settings (see full output here ).
This doesn't include the node roles info provided in that fuller output.