Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for bloom_filter_agg #846

Closed
Tracked by #858
andygrove opened this issue Aug 19, 2024 · 6 comments · Fixed by #987
Closed
Tracked by #858

Add support for bloom_filter_agg #846

andygrove opened this issue Aug 19, 2024 · 6 comments · Fixed by #987
Labels
enhancement New feature or request

Comments

@andygrove
Copy link
Member

andygrove commented Aug 19, 2024

What is the problem the feature request solves?

Some TPC-H queries use bloom_filter_agg, and Comet does not have a native implementation yet.

A workaround is to set spark.sql.optimizer.runtime.bloomFilter.enabled=false.

Describe the potential solution

No response

Additional context

No response

@vaibhawvipul
Copy link
Contributor

I can take this up if more details are provided :)

@andygrove
Copy link
Member Author

I can take this up if more details are provided :)

We need to implement an equivalent of Spark's org.apache.spark.sql.catalyst.expressions.aggregate.BloomFilterAggregate.

@vaibhawvipul
Copy link
Contributor

ok, taking this up.

@mbutrovich
Copy link
Contributor

Accumulating some notes for this. Here's the Spark design doc on the feature:
https://docs.google.com/document/d/16IEuyLeQlubQkH8YuVuXWKo2-grVIoDJqQpHZrE7q04/

@mbutrovich
Copy link
Contributor

It looks like we already have the filter support thanks to #179.

@viirya
Copy link
Member

viirya commented Oct 1, 2024

BloomFilterAggregate is only supported by ObjectHashAggregate or SortAggregate. But in Comet, we only support HashAggregate so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants