Describe the bug
We found a bug while upgrading IOx to use the latest datafusion
Some of our tests began failing like this:
"Optimizer rule 'push_down_projection' failed"
To Reproduce
in https://github.com/apache/arrow-datafusion/blob/8bd70d691cba1e730ec0d5fdce5cd544acf5008d/datafusion/optimizer/src/push_down_projection.rs#L608-L631
Add this test
#[test]
fn aggregate_with_periods() -> Result<()> {
let schema = Schema::new(vec![Field::new("tag.one", DataType::Utf8, false)]);
// Build a plan that looks as follows (note "tag.one" is a column named
// "tag.one", not a column named "one" in a table named "tag"):
//
// Projection: tag.one
// Aggregate: groupBy=[], aggr=[MAX("tag.one") AS "tag.one"]
// TableScan
let plan = table_scan(Some("m4"), &schema, None)?
.aggregate(
Vec::<Expr>::new(),
vec![max(col(Column::new_unqualified("tag.one"))).alias("tag.one")],
)?
.project([col(Column::new_unqualified("tag.one"))])?
.build()?;
let expected = "\
Aggregate: groupBy=[[]], aggr=[[MAX(m4.tag.one) AS tag.one]]\
\n TableScan: m4 projection=[tag.one]";
assert_optimized_plan_eq(&plan, expected)
}
The test will fail like:
thread 'push_down_projection::tests::aggregate_with_periods' panicked at datafusion/optimizer/src/push_down_projection.rs:1123:45:
failed to optimize plan: Plan("Aggregate requires at least one grouping or aggregate expression")
Expected behavior
The test should pass
Additional context
The regression was introduced in #7981
It appears to be related to columns that have periods in their names