Blog post about TopK filter pushdown

[alamb]

Is your feature request related to a problem or challenge?

@adriangb has a great PR to add dynamic filtering with TopK in DataFusion here:

• Add dynamic pruning filters from TopK state #15301

Describe the solution you'd like

It would be really nice to write a blog post about this optimization in DataFusion (and maybe call out how it works)

Here are some related work

• Snowflake blog: Reimagining Top-K Aggregation Optimization at Snowflake
• Dynamic Filters in Starburst,

High level points

• Study DuckDB plan Make ClickBench Q23 Go Faster #15177 (hos much faster)
• Went back and forth to design something that could be general purpose
• Did benchmarking,
• Results are amazing, etc

Describe alternatives you've considered

No response

Additional context

No response

[aaryyya]

take

[alamb]

Thanks @aaryyya -- note we'll need to actually finish the work before we can publish a blog

Of course, blog driven development does work pretty well -- it is basically we @clflushopt @scsmithr and I did with another project (wrote the outline of the blog we wanted to see and then wrote the code to make it happen 😆 )

• [BLOG] tpchgen-rs: World’s fastest open source TPCH data generator, written in Rust datafusion-site#67