Skip to content

[Transform] Add data frame scripted fields for group_by fields #43152

@blaklaybul

Description

@blaklaybul

It would be useful to be able to create a group by field upon data frame creation with the use of a painless script, as in the following example:

"pivot": {
   "group_by": {
      "foo_bar": {
         "terms": {
            "script": {
               "source": "ctx._source.foo +'-' + ctx._source.bar",
               "lang": "painless"
                       } 
                    }
                }
            }
         }

This would enable multindex-style groupby operations, and having the individual fields acted on by the script available in the data frame would allow for multindex-style querying as well.

e.g. (pandas):

gb = df.groupby(['foo', 'bar']).agg({'baz': 'mean'})
gb.loc['foo_1'] -> returns all rows indexed by 'foo_1' value of 'foo'

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions