Skip to content

[BUG] system limit from plugins.ppl.join.subsearch_maxout prevents sort push down for sort merge join #4570

@qianheng-aws

Description

@qianheng-aws

What is the bug?
system limit from plugins.ppl.join.subsearch_maxout prevents sort push down for sort merge join

How can one reproduce the bug?

  1. Create a index, e.g.:
PUT localhost:9200/test
{
    "mappings": {
        "properties": {
            "id": {
                "type": "text"
            },
            "value": {
                "type": "integer"
            }
        }
    }
}
  1. explain the physical plan for a join query:
POST localhost:9200/_plugins/_ppl/_explain?format=extended

source=test | join value test 

and get final plan:

...
EnumerableCalc(expr#0..2=[{inputs}], value=[$t1], id=[$t2])
  EnumerableLimit(fetch=[10000])
    EnumerableMergeJoin(condition=[=($0, $1)], joinType=[inner])
      CalciteEnumerableIndexScan(table=[[OpenSearch, test]], PushDownContext=[[PROJECT->[value], SORT->[{\"value\" : {\"order\" : \"asc\",\"missing\" : \"_last\"}]], OpenSearchRequestBuilder(sourceBuilder={\"from\":0,\"timeout\":\"1m\",\"_source\":{\"includes\":[\"value\"],\"excludes\":[]},\"sort\":[{\"value\":{\"order\":\"asc\",\"missing\":\"_last\"}}]}, requestedTotalSize=2147483647, pageSize=null, startFrom=0)])
      EnumerableSort(sort0=[$0], dir0=[ASC])
        CalciteEnumerableIndexScan(table=[[OpenSearch, test]], PushDownContext=[[PROJECT->[value, id], LIMIT->50000], OpenSearchRequestBuilder(sourceBuilder={\"from\":0,\"size\":50000,\"timeout\":\"1m\",\"_source\":{\"includes\":[\"value\",\"id\"],\"excludes\":[]}}, requestedTotalSize=50000, pageSize=null, startFrom=0)])
...

What is the expected behavior?
The EnumerableSort operator on the right child of join should be pushed down into scan. Although there is limit pushed down before the sort, that limit is a system limit from a guard rail configuration which should not prevent sort.

What is your host/environment?

  • OS: macos
  • Version 3.3.0-snapshot
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpushdownpushdown related issues

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions