Skip to content

DataFrameColumn: Unable to Filter by Null via ElementWiseXYZ Methods (System.NotSupportedException) #6110

Closed
@JD-Robbs

Description

@JD-Robbs

System Information (please complete the following information):

  • OS & Version: Windows 10
  • ML.NET Version: Latest
  • .NET Version: .NET 6.0

Describe the bug
When passing null into ElementWiseXYZ methods, a System.NotSupportedException is thrown: 'Specified method is not supported.'

To Reproduce
Steps to reproduce the behavior:

        var col = new DoubleDataFrameColumn("col", new double?[] {1.23, null, 2, 3 });
        var dfTest = new DataFrame(col);

        var filtered2Df = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(2));
        var filteredNullDf = dfTest.Filter(dfTest["col"].ElementwiseNotEquals(null));

Expected behavior
I expected the resulting DataFrame stored in filteredNullDf to not include the value null - much like filered2Df does not include the value 2.

The only way of making it work is something like this:

         dfTest.Filter(new PrimitiveDataFrameColumn<bool>("filter", dfTest["col"].Cast<double?>().Select(v => v != null)));

Metadata

Metadata

Assignees

No one assigned

    Labels

    Microsoft.Data.AnalysisAll DataFrame related issues and PRsP2Priority of the issue for triage purpose: Needs to be fixed at some point.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions