Skip to content

Regression in CASE expression since DF 44 #14277

@andygrove

Description

@andygrove

Describe the bug

I am trying to upgrade Comet to the latest DataFusion and see some queries fail.

The regression seems to have been introduced in #13953.

Example Query:

SELECT array_remove(array(_2, _3,_4), _2) from t1 where _2 is null

Query plan:

ProjectionExec: expr=[CASE WHEN Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy] IS NULL THEN NULL ELSE array_remove(make_array(Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy], Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_1@1, eval_mode: Legacy], col_2@2), Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy]) END as col_0]
  CometFilterExec: col_0@0 IS NULL
    ScanExec: source=[CometScan parquet  (unknown)], schema=[col_0: Int8, col_1: Int16, col_2: Int32]

Failure:

Invalid argument error: arguments need to have the same data type

Cause:

In the new expr_or_expr function, the following line is causing the failure:

Ok(ColumnarValue::Array(zip(&remainder, &else_, &then_value)?))

The type of else_ is ListArray and the type of then_value is NullArray

Solution:

Perhaps we just need to add a cast in here? I am going to add a repro to DataFusion and work on a fix.

To Reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions