[Discover] - Add Breakdown Field Option in Field Stats Popover

[ninoslavmiskovic]

WHY:

In Discover, users frequently need to break down data by specific fields. While this is currently possible through the histogram, there is no way to initiate a breakdown directly from the field stats popover. Adding this capability would streamline workflows, allowing users to apply breakdowns more quickly without navigating away from the field stats view.

WHAT:

Introduce an “Add Breakdown” option within the field stats popover, supporting both KQL and ES|QL:

• For KQL Users: The “Add Breakdown” would update the breakdown in the existing visualizations, making it easier to apply a field-based breakdown directly from the stats popover, without altering the main query. Below is an example of the current histogram with a breakdown of the log level.

• For ES|QL Users:

For ES|QL, it will be achieved by adding a STATS clause directly into the query. This would add a "group by". For example, if the user clicks on log.level, the query would update with :

STATS count(*) BY log.level

Note: Adding this STATS count(*) BY log.level clause would produce an aggregated chart, not a standard histogram.

• If STATS is already present in the ES|QL query, then hide the button.

Supported Field Types for Breakdown:

The breakdown functionality should support any field type that can be aggregated in Elasticsearch. This includes:

•	Keyword
•	Numeric
•	IP
•	Boolean
•	Binary

These field types are commonly used for aggregation in Elasticsearch, making them ideal for breakdowns. When a user selects one of these fields in the stats popover, the system should enable the “Add Breakdown” option, allowing for segmentation based on that field.

BENEFITS:

• Improved Efficiency: Users can quickly break down data by specific fields without needing to manually modify the query. KQL users can update visualizations easily, while ES|QL users can directly adjust queries for more complex aggregations.
• Simplified Workflow: This feature reduces the number of steps needed to apply a breakdown, making the process more intuitive. Users won’t need to switch between views, as the breakdown can be applied directly from the field stats popover.
• **Better Data Exploration:**By adding breakdowns within the field stats popover, users can easily explore different segments of their data. KQL users will benefit from quick field-based filtering, while ES|QL users can leverage the full power of advanced aggregations and statistical analysis.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[davismcphee]

@stratoula do we need inline stats to support this in ES|QL or is it possible today?

[stratoula]

I need to understand better the ER first to reply.

@ninoslavmiskovic in your screenshot and description you mention this

FROM logs-*-*
| STATS COUNT(*) BY log.level

which means I am changing the query of Discover regardless what I have in the editor atm. Which is weird and not what the users are expecting. This suggestion doesn't need inlinestats but I don't think we should do it.

What it would make sense is that when the users are clicking a field then we update the current query with a stats something like

 STATS var0 = count(geo.dest) BY geo.dest 

this makes more sense to me and is not blocked by inlinestats

If we want by clicking this to update the histogram and only (add a breakdown) this needs inlinestats but I don't like it as the main query is independent of the histogram query so the users dont get what is done. Adding a breakdown in the histogram can be accomplished when we have inlinestats from the inline editing flyout. I wouldn't add this functionality to the field list.

[ninoslavmiskovic]

when the users are clicking a field then we update the current query with a stats something like

@stratoula This is what I wanted. Similar to what we do with WHERE. Just add the right syntax to the query.

Question: when we get inline stats - could we both update the query and select the breakdown by in the histogram ?

[ninoslavmiskovic]

In dataview mode we could simply update the histogram. I guess ?!

[stratoula]

Question: when we get inline stats - could we both update the query and select the breakdown by in the histogram ?

No because stats in the initial query displays the aggregation chart and not the histogram

[ninoslavmiskovic]

@stratoula

I have changed the GH issue description to add a STATS and a note which would produce an aggregated chart instead of the histogram.

QQ:

Currently, it is possible to create a histogram in ES|QL with a break-down by:

FROM logs-*-*
| EVAL timestamp=DATE_TRUNC(1 hour, @timestamp)
| STATS count(*) BY log.level, timestamp
| RENAME timestamp as `@timestamp every 1 hour`

This creates this histogram visualization:

And also updates the "Break-down by" in the inline editor.

How would this work with INLINE stats ? What would the query be like and the UX in the inline editor ?

[stratoula]

This creates the histogram but only for the specific field, we want the users to also be able to choose among other fields this is why we want inline stats instead of stats. With that being said the query is the same but instead of stats you use inlinestats. I have an implementation here already #189630 but is blocked by inlinestats being available only on snapshot releases. And this is a step forward #190213 and initializes the histogram with the log.level but also allows the users to choose a field from the inline editing

[ninoslavmiskovic]

This creates the histogram but only for the specific field, we want the users to also be able to choose among other fields this is why we want inline stats instead of stats

Super clear ! Makes sense.

Great with the links to the issues.

@stratoula WDYT of the current specification with doing GROUP by with STATS instead ? and then when we are ready we can add the break-down with in-line stats. This would decouple the efforts like you mentioned.

[stratoula]

We are also blocked by this #191988 so we need to wait for both.

[ninoslavmiskovic]

@stratoula I added this part: If STATS is already present in the ES|QL query, then hide the button.

[ninoslavmiskovic]

The first step on ES|QL mode will be to add the break-down and @stratoula is working on this here: #186369 which is not longer blocked. After that work is finished, the break-down on field stats pop-over can be added.

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)