Skip to content

[BUG] Pushdown not work when aggregate on text data type with function FIRST/LAST/TAKE #5086

@LantaoJin

Description

@LantaoJin

What is the bug?
Pushdown not work when aggregate on text data type with function FIRST/LAST

How can one reproduce the bug?
Create index

PUT good/
{
  "mappings": {
    "properties": {
      "packets_str": { "type": "keyword" }
    }
  }
}
PUT bad/
{
  "mappings": {
    "properties": {
      "packets_str": { "type": "text" }
    }
  }
}

Ingest data

POST good/_bulk
{"index": {}}
{"packets_str":"10"}
{"index": {}}
{"packets_str":"20"}
{"index": {}}
{"packets_str":"30"}
POST bad/_bulk
{"index": {}}
{"packets_str":"10"}
{"index": {}}
{"packets_str":"20"}
{"index": {}}
{"packets_str":"30"}

Query

POST /_plugins/_ppl/_explain?format=yaml
{
    "query": """
    source=good | stats last(packets_str)
    """
}

returns

calcite:
  logical: |
    LogicalSystemLimit(fetch=[10000], type=[QUERY_SIZE_LIMIT])
      LogicalAggregate(group=[{}], last(packets_str)=[LAST($0)])
        LogicalProject(packets_str=[$0])
          CalciteLogicalIndexScan(table=[[OpenSearch, test]])
  physical: |
    EnumerableLimit(fetch=[10000])
      CalciteEnumerableIndexScan(table=[[OpenSearch, test]], PushDownContext=[[AGGREGATION->rel#301:LogicalAggregate.NONE.[](input=RelSubset#300,group={},last(packets_str)=LAST($0))], OpenSearchRequestBuilder(sourceBuilder={"from":0,"size":0,"timeout":"1m","aggregations":{"last(packets_str)":{"top_hits":{"from":0,"size":1,"version":false,"seq_no_primary_term":false,"explain":false,"fields":[{"field":"packets_str"}],"sort":[{"_doc":{"order":"desc"}}]}}}}, requestedTotalSize=2147483647, pageSize=null, startFrom=0)])

But

POST /_plugins/_ppl/_explain?format=yaml
{
    "query": """
    source=bad | stats last(packets_str)
    """
}

returns

calcite:
  logical: |
    LogicalSystemLimit(fetch=[10000], type=[QUERY_SIZE_LIMIT])
      LogicalAggregate(group=[{}], last(packets_str)=[LAST($0)])
        LogicalProject(packets_str=[$0])
          CalciteLogicalIndexScan(table=[[OpenSearch, test2]])
  physical: |
    EnumerableLimit(fetch=[10000])
      EnumerableAggregate(group=[{}], last(packets_str)=[LAST($0)])
        CalciteEnumerableIndexScan(table=[[OpenSearch, test2]], PushDownContext=[[PROJECT->[packets_str]], OpenSearchRequestBuilder(sourceBuilder={"from":0,"timeout":"1m","_source":{"includes":["packets_str"],"excludes":[]}}, requestedTotalSize=2147483647, pageSize=null, startFrom=0)])

What is the expected behavior?
A clear and concise description of what you expected to happen.

What is your host/environment?

  • OS: [e.g. iOS]
  • Version [e.g. 22]
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpushdownpushdown related issues

Type

No type

Projects

Status

Not Started

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions