Skip to content

Spark SQL test "window function: partition and order expressions" fails with scala.MatchError #1249

@andygrove

Description

@andygrove

Describe the bug

When we fix #1248 we find a new bug that causes the following Spark SQL test to fail:

- window function: partition and order expressions *** FAILED *** (75 milliseconds)
scala.MatchError: 2 (of class org.apache.spark.sql.catalyst.expressions.Literal)
  at org.apache.comet.serde.QueryPlanSerde$$anonfun$2.applyOrElse(QueryPlanSerde.scala:3158)
  at org.apache.comet.serde.QueryPlanSerde$$anonfun$2.applyOrElse(QueryPlanSerde.scala:3157)

The expression is partition by 1 order by 2

We currently assume that partition and order expressions are attribute references and do not support literals

    val partitionColumnNames = partitionSpec.collect { case a: AttributeReference =>
      a.name
    }

    val orderColumnNames = orderSpec.collect { case s: SortOrder =>
      s.child match {
        case a: AttributeReference => a.name
      }
    }

Steps to reproduce

Change default shuffle mode to jvm and run Spark SQL tests

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions