Skip to content

Commit ce3b03d

Browse files
peter-tothHyukjinKwon
authored andcommitted
[SPARK-42852][SQL] Revert NamedLambdaVariable related changes from EquivalentExpressions
### What changes were proposed in this pull request? This PR reverts the follow-up PR of SPARK-41468: apache#39046 ### Why are the changes needed? These changes are not needed and actually might cause performance regression due to preventing higher order function subexpression elimination in `EquivalentExpressions`. Please find related conversation here: apache#40473 (comment) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Existing UTs. Closes apache#40475 from peter-toth/SPARK-42852-revert-namedlambdavariable-changes. Authored-by: Peter Toth <peter.toth@gmail.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
1 parent 1e7e47d commit ce3b03d

File tree

1 file changed

+2
-3
lines changed

1 file changed

+2
-3
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala

+2-3
Original file line numberDiff line numberDiff line change
@@ -144,10 +144,9 @@ class EquivalentExpressions {
144144

145145
private def supportedExpression(e: Expression) = {
146146
!e.exists {
147-
// `LambdaVariable` is usually used as a loop variable and `NamedLambdaVariable` is used in
148-
// higher-order functions, which can't be evaluated ahead of the execution.
147+
// `LambdaVariable` is usually used as a loop variable, which can't be evaluated ahead of the
148+
// loop. So we can't evaluate sub-expressions containing `LambdaVariable` at the beginning.
149149
case _: LambdaVariable => true
150-
case _: NamedLambdaVariable => true
151150

152151
// `PlanExpression` wraps query plan. To compare query plans of `PlanExpression` on executor,
153152
// can cause error like NPE.

0 commit comments

Comments
 (0)