Release DataFusion 52.4.0 (minor/) Release (Mar 2026)

[alamb]

Is your feature request related to a problem or challenge?

• Part of [EPIC] DataFusion Releases #19783
• Follow on to Release DataFusion 52.3.0 (minor/) Release (Mar 2026) #20681

Are several other regressions reported against 52 that we might want to backport to branch-52 that didn't make it into #20681. Let's coordinate on this ticket in case we want to create a new release

Steps:

• Update version and changelog: [branch-52] Update version to 52.4.0 and update changelog #21004
• Voting Thread: https://lists.apache.org/thread/gtdxclh1ywd89sjfz4r1pb4d5x0s5j90
• Publish to crates.io

Potential Content:

Requested Bug Fixes. Issue / requester / Backport

• [Bug] Dynamic filter leads to wrong results in some aggregation queries with >1 partitions #20267 / @alamb / [branch-52] fix: disable dynamic filter pushdown for non min/max aggregates (#20279) #20877
• Data regression in aggregations that spill #20724 / @16pierre / [branch-52] Fix duplicate group keys after hash aggregation spill (#20724) (#20858) #20917
• Incorrect ORDER BY in unparser after DF 52 upgrade #20905 / @alamb @krinart / TODO
• Regression: array_sort fails on list with inner non-nullable field #19947 / @alamb / [branch-52] fix: maintain inner list nullability for array_sort (#19948) #20878
• Duplicate unqualified name error on unions #20123 / @alamb / [branch-52] fix: Ensure columns are casted to the correct names with Unions (#20146) #20879
• Right joins with empty schema return invalid results #20669 / @alamb / [branch-52] fix: Return probe_side.len() for RightMark/Anti count(*) queries (#20710) #20881
• interval analysis error when have two filterexec that inner filter proves zero selectirity #20742 / @alamb / [branch-52] fix: interval analysis error when have two filterexec that inner filter proves zero selectivity (#20743) #20880
• SanityCheckPlan caused by Error during planning: #20194 / @alamb / [branch-52] fix: SanityCheckPlan error with window functions and NVL filter (#20231) #20931

[alamb]

I have a first pass at the potential content above

Possible other issues:

• Right joins with empty schema return invalid results #20669
• interval analysis error when have two filterexec that inner filter proves zero selectirity #20742

[alamb]

Ok, I made a bunch of backport PRs (with codex's help)

• [branch-52] fix: disable dynamic filter pushdown for non min/max aggregates (#20279) #20877
• [branch-52] fix: maintain inner list nullability for (#19948) #20878
• [branch-52] fix: Ensure columns are casted to the correct names with Unions (#20146) #20879
• [branch-52] fix: interval analysis error when have two filterexec that inner filter proves zero selectivity (#20743) #20880
• [branch-52] fix: Return probe_side.len() for RightMark/Anti count(*) queries (#20710) #20881