Release DataFusion `54.0.0` (Apr 2026 / May 2026)

[alamb]

Is your feature request related to a problem or challenge?

Previous release is #19692 (Mar, 2026) so next major release would be approximately Apr / May 2026

Steps:

• Create a branch-54 branch:
• Update version and changelog:
• Test with DataFusion Python
• Test with DataFusion Comet
• Test with DataFusion Ballista
• Test with delta.rs:
• Test vortex:
• Test with iceberg-rust:
• Test with LakeSail:
• Test with parquet viewer
• Test with datafusion-materialized-views
• Voting Thread
• Publish to crates.io
• Create ticket for next release:
• Create blog post:

Features to mention in the blog (if they make it)

• [EPIC] Simplify datetime predicates using "preimages" #19946 from @sdf-jkl (along with the related optimizations for date_trunc and date_part)
• Migrate Avro reader to arrow-avro and remove internal conversion code #17861
• feat: add support for parquet content defined chunking options #21110 from @getChan
• Feature request: Support for lateral joins #10048 lateral joins from @neilconway
• feat: sort file groups by statistics during sort pushdown (Sort pushdown phase 2) #21182 from @zhuqi-lucas
• feat: add support for parquet content defined chunking options #21110 from @kszucs
• API to register behavior for Extension Types #18223 / Basic Extension Type Registry Implementation #20312 from @tobixdev

Bugs that should be fixed

• Regression in json performance for local files #21450

Community Wishlist

• perf: push_down_filter is pathologically slow for some plans #20002
• Streaming Aggregate operator not being used in deduplication of pre-sorted Parquet files #16919
• fix(parquet): converting parquet schema with backward compatible repeated struct/primitive with provided arrow schema arrow-rs#8496

[alamb]

Here is another one that would be sweet to mention:

• Migrate Avro reader to arrow-avro and remove internal conversion code #17861 from @getChan

Another would be morsel driven scans

• Introduce morsel-driven Parquet scan #20529

[tobixdev]

Could be interessting to mention #18223 in the blog.

[alamb]

Could be interessting to mention #18223 in the blog.

Absolutely -- added to the list