Skip to content

Add filter pushdown tests #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 2, 2025

Conversation

alamb
Copy link

@alamb alamb commented Apr 2, 2025

This PR targets a PR in DataFusion

It adds basic tests for the filter pushdown physical optimizer introduced in that PR

It is largely based on the structure in influxdb_iox's query engine

https://github.com/influxdata/influxdb3_core/blob/26a30bf8d6e2b6b3f1dd905c4ec27e3db6e20d5f/iox_query/src/physical_optimizer/predicate_pushdown.rs#L24

I believe @crepereum and @NGA-TRAN were largely responsible for this code originally. To be clear, I am trying to give credit where it is due :) Any faults of this port are my own, not thiers

@alamb alamb marked this pull request as draft April 2, 2025 20:02
insta::assert_snapshot!(
OptimizationTest::new(plan, FilterPushdown{}),
@r"
OptimizationTest:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to be able to express tests like this -- I will add a few more to show what is going on

@alamb alamb force-pushed the alamb/topk-filter-tests branch from 657187c to d89a29e Compare April 2, 2025 20:08
output:
Ok:
- FilterExec: b@1 = bar
- FilterExec: a@0 = foo AND b@1 = bar
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is some interesting behavior -- the filters get pushed down but the originals are left

@alamb alamb marked this pull request as ready for review April 2, 2025 20:09
@adriangb
Copy link
Member

adriangb commented Apr 2, 2025

Great, will merge and fix / get passing / resolve conflicts!

@adriangb adriangb merged commit d423866 into pydantic:topk-filter Apr 2, 2025
22 of 27 checks passed
@alamb alamb deleted the alamb/topk-filter-tests branch April 3, 2025 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants