Skip to content

[BUG] explode_outer_position doesn't match to Spark's counterpart  #7721

Closed
@sperlingxx

Description

Describe the bug
In cuDF, explode_outer_position will mark the position values of empty rows with 0. Meanwhile, the position values of empty rows are marked as null in Spark.

Steps/Code to reproduce bug
For input data like:

  • [[5,null,15], 100]
  • [null, 200]
  • [[], 300]

cuDF returns

  • [0, 5, 100]
  • [1, null, 100]
  • [2, 15, 100]
  • [0, null, 200]
  • [0, null, 300]

But Spark returns

  • [0, 5, 100]
  • [1, null, 100]
  • [2, 15, 100]
  • [null, null, 200]
  • [null, null, 300]

Metadata

Labels

bugSomething isn't workinglibcudfAffects libcudf (C++/CUDA) code.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions