Skip to content

Commit

Permalink
[SPARK-36822][SQL] BroadcastNestedLoopJoinExec should use all conditi…
Browse files Browse the repository at this point in the history
…on instead of non-equi condition

### What changes were proposed in this pull request?

Change `nonEquiCond` to all join condition at `JoinSelection.ExtractEquiJoinKeys` pattern.

### Why are the changes needed?

At `JoinSelection`, with `ExtractEquiJoinKeys`, we use `nonEquiCond` as the join condition. It's wrong since there should exist some equi condition.
```
Seq(joins.BroadcastNestedLoopJoinExec(
  planLater(left), planLater(right), buildSide, joinType, nonEquiCond))
```
But it's should not be a bug, since we always use the smj as the default join strategy for ExtractEquiJoinKeys.

### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

it's not a bug for now, but we should fix it in case we use this code path in future.

Closes apache#34065 from ulysses-you/join-condition.

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information
ulysses-you authored and cloud-fan committed Sep 22, 2021
1 parent 1d7d972 commit d90b479
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -262,7 +262,7 @@ abstract class SparkStrategies extends QueryPlanner[SparkPlan] {
// This join could be very slow or OOM
val buildSide = getSmallerSide(left, right)
Seq(joins.BroadcastNestedLoopJoinExec(
planLater(left), planLater(right), buildSide, joinType, nonEquiCond))
planLater(left), planLater(right), buildSide, joinType, j.condition))
}
}

Expand Down

0 comments on commit d90b479

Please sign in to comment.