Skip to content

[BUG] Cannot pushdown a nested root as group-by key #5006

@LantaoJin

Description

@LantaoJin

Query Information

PPL Command/Query:

source=test | stats count() by rec
source=test | top rec
source=test | dedup rec

Dataset Information

Dataset/Schema Type

  • OpenTelemetry (OTEL)
  • Simple Schema for Observability (SS4O)
  • Open Cybersecurity Schema Framework (OCSF)
  • Custom (details below)

Index Mapping

{
  "mappings": {
    "properties": {
      "rec": {
        "type": "nested",
        "properties": {
          "inner": {
            "type": "integer"
          }
        }
      }
    }
  }
}

Sample Data

{"rec":{"inner": 100}}
{"rec":{"inner": 101}}
{"rec":{"inner": 101}}

Bug Description

Issue Summary:
The queries above will try to trigger an aggregation pushdown which the group-by key (bucket) is rec. Since rec is a nested root, OpenSeach DSL returns null if the bucket fields is a nested field. So it will fallback to execute in Calcite EnumerableAggregate which may have performance issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languageperformanceMake it fast!pushdownpushdown related issues

    Type

    No type

    Projects

    Status

    Not Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions