[SPARK-28481][SQL] More expressions should extend NullIntolerant#28626
[SPARK-28481][SQL] More expressions should extend NullIntolerant#28626wangyum wants to merge 8 commits intoapache:masterfrom wangyum:SPARK-28481
Conversation
|
Test build #123041 has finished for PR 28626 at commit
|
|
retest this please. |
...st/src/test/scala/org/apache/spark/sql/catalyst/expressions/NullIntolerantCheckerSuite.scala
Outdated
Show resolved
Hide resolved
...st/src/test/scala/org/apache/spark/sql/catalyst/expressions/NullIntolerantCheckerSuite.scala
Outdated
Show resolved
Hide resolved
...st/src/test/scala/org/apache/spark/sql/catalyst/expressions/NullIntolerantCheckerSuite.scala
Outdated
Show resolved
Hide resolved
...st/src/test/scala/org/apache/spark/sql/catalyst/expressions/NullIntolerantCheckerSuite.scala
Outdated
Show resolved
Hide resolved
|
Test build #123053 has finished for PR 28626 at commit
|
|
Test build #123059 has finished for PR 28626 at commit
|
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
|
also cc: @viirya |
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Outdated
Show resolved
Hide resolved
# Conflicts: # sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
|
Test build #123105 has finished for PR 28626 at commit
|
|
retest this please |
|
Test build #123113 has finished for PR 28626 at commit
|
|
Test build #123124 has finished for PR 28626 at commit
|
sql/core/src/test/scala/org/apache/spark/sql/expressions/ExpressionInfoSuite.scala
Show resolved
Hide resolved
|
|
||
| final override def eval(input: InternalRow): Any = { | ||
| val input2 = right.eval(input) | ||
| if (input2 == null || input2 == 0) { |
There was a problem hiding this comment.
ah I see the difference now. Previously we can skip evaluating input1 if input2 is 0. Can we change it back and add comment to explain it? sorry for the back and forth!
maropu
left a comment
There was a problem hiding this comment.
Looks okay. Thanks for the updates.
|
Test build #123233 has finished for PR 28626 at commit
|
|
Test build #123237 has finished for PR 28626 at commit
|
|
thanks, merging to master! |
What changes were proposed in this pull request?
NullIntolerant.ExpressionInfoSuite) to identify whether the expression isNullIntolerant.Why are the changes needed?
Avoid skew join if the join column has many null values and can improve query performance. For examples:
Before and after this PR:
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit test.