-
Notifications
You must be signed in to change notification settings - Fork 797
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make Parquet reader filter APIs public (#1792) #2467
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI @Ted-Jiang @thinkharderdev @viirya and @HaoYang670
This is quite exciting and represents a big step forward for the rust parquet implementation 🎉
@@ -398,11 +398,11 @@ mod tests { | |||
let mut rand = thread_rng(); | |||
for _ in 0..100 { | |||
let a_len = rand.gen_range(10..100); | |||
let a_bools: Vec<_> = (0..a_len).map(|x| rand.gen_bool(0.2)).collect(); | |||
let a_bools: Vec<_> = (0..a_len).map(|_| rand.gen_bool(0.2)).collect(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seems random that this started getting flagged by clippy 🤷 -- I suppose I am always one for a good drive by cleanup anyways 🚗
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is because the unused suppression at the module level was removed
Co-authored-by: Liang-Chi Hsieh <viirya@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice! This is an exciting enhancement!
Benchmark runs are scheduled for baseline = 0013170 and contender = 6d0ea90. 6d0ea90 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Part of #1792
Rationale for this change
There is still more work to be done in this space, but I am confident that this functionality is sufficiently mature that we can expose it and start to get feedback from the wild
What changes are included in this PR?
Makes the crate-private row filtering APIs, public
Are there any user-facing changes?