Skip to content

Commit 2e23da2

Browse files
ulysses-youcloud-fan
authored andcommitted
[SPARK-31975][SQL] Show AnalysisException when WindowFunction is used without WindowExpression
### What changes were proposed in this pull request? Add WindowFunction check at `CheckAnalysis`. ### Why are the changes needed? Provide friendly error msg. **BEFORE** ```scala scala> sql("select rank() from values(1)").show java.lang.UnsupportedOperationException: Cannot generate code for expression: rank() ``` **AFTER** ```scala scala> sql("select rank() from values(1)").show org.apache.spark.sql.AnalysisException: Window function rank() requires an OVER clause.;; Project [rank() AS RANK()#3] +- LocalRelation [col1#2] ``` ### Does this PR introduce _any_ user-facing change? Yes, user wiill be given a better error msg. ### How was this patch tested? Pass the newly added UT. Closes #28808 from ulysses-you/SPARK-31975. Authored-by: ulysses <youxiduo@weidian.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 5d296ed commit 2e23da2

File tree

2 files changed

+16
-0
lines changed

2 files changed

+16
-0
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,11 @@ trait CheckAnalysis extends PredicateHelper {
158158
case g: GroupingID =>
159159
failAnalysis("grouping_id() can only be used with GroupingSets/Cube/Rollup")
160160

161+
case e: Expression if e.children.exists(_.isInstanceOf[WindowFunction]) &&
162+
!e.isInstanceOf[WindowExpression] =>
163+
val w = e.children.find(_.isInstanceOf[WindowFunction]).get
164+
failAnalysis(s"Window function $w requires an OVER clause.")
165+
161166
case w @ WindowExpression(AggregateExpression(_, _, true, _, _), _) =>
162167
failAnalysis(s"Distinct window functions are not supported: $w")
163168

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -884,4 +884,15 @@ class AnalysisSuite extends AnalysisTest with Matchers {
884884
Seq("Intersect can only be performed on tables with the compatible column types. " +
885885
"timestamp <> double at the second column of the second table"))
886886
}
887+
888+
test("SPARK-31975: Throw user facing error when use WindowFunction directly") {
889+
assertAnalysisError(testRelation2.select(RowNumber()),
890+
Seq("Window function row_number() requires an OVER clause."))
891+
892+
assertAnalysisError(testRelation2.select(Sum(RowNumber())),
893+
Seq("Window function row_number() requires an OVER clause."))
894+
895+
assertAnalysisError(testRelation2.select(RowNumber() + 1),
896+
Seq("Window function row_number() requires an OVER clause."))
897+
}
887898
}

0 commit comments

Comments
 (0)