Skip to content

[C++] is_null kernel does not propagate nan_is_null for nested types #50889

Description

@hadrian-reppas

Describe the bug, including details regarding any error messages, version, and platform.

b is just a dictionary-encoded copy of a, so the compute kernel should have the same output. But it dosen't because nan_is_null is ignored when looking up dictionary values. This will involve changes to SetLogicalNullBits (see #49889). Also consider updating at ArraySpan::ComputeLogicalNullCount.

>>> a = pa.array([1.0, 1.5, 1.0, math.nan])
>>> b = pc.dictionary_encode(a)
>>> pc.is_null(a, nan_is_null=True)
<pyarrow.lib.BooleanArray object at 0x129d0ab60>
[
  false,
  false,
  false,
  true
]
>>> pc.is_null(b, nan_is_null=True)
<pyarrow.lib.BooleanArray object at 0x129d0a0e0>
[
  false,
  false,
  false,
  false
]

Component(s)

C++

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions