Skip to content

Regression while planning plans with periods in their name like tag.one: Plan("Aggregate requires at least one grouping or aggregate expression") #8223

@alamb

Description

@alamb

Describe the bug

We found a bug while upgrading IOx to use the latest datafusion

Some of our tests began failing like this:

"Optimizer rule 'push_down_projection' failed"

To Reproduce

in https://github.com/apache/arrow-datafusion/blob/8bd70d691cba1e730ec0d5fdce5cd544acf5008d/datafusion/optimizer/src/push_down_projection.rs#L608-L631

Add this test

    #[test]
    fn aggregate_with_periods() -> Result<()> {
        let schema = Schema::new(vec![Field::new("tag.one", DataType::Utf8, false)]);

        // Build a plan that looks as follows (note "tag.one" is a column named
        // "tag.one", not a column named "one" in a table named "tag"):
        //
        // Projection: tag.one
        //   Aggregate: groupBy=[], aggr=[MAX("tag.one") AS "tag.one"]
        //    TableScan
        let plan = table_scan(Some("m4"), &schema, None)?
            .aggregate(
                Vec::<Expr>::new(),
                vec![max(col(Column::new_unqualified("tag.one"))).alias("tag.one")],
            )?
            .project([col(Column::new_unqualified("tag.one"))])?
            .build()?;

        let expected = "\
        Aggregate: groupBy=[[]], aggr=[[MAX(m4.tag.one) AS tag.one]]\
        \n  TableScan: m4 projection=[tag.one]";

        assert_optimized_plan_eq(&plan, expected)
    }

The test will fail like:

thread 'push_down_projection::tests::aggregate_with_periods' panicked at datafusion/optimizer/src/push_down_projection.rs:1123:45:
failed to optimize plan: Plan("Aggregate requires at least one grouping or aggregate expression")

Expected behavior

The test should pass

Additional context

The regression was introduced in #7981

It appears to be related to columns that have periods in their names

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions