Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ArrayAgg Builtin in favor of UDF #11611

Merged
merged 3 commits into from
Jul 23, 2024
Merged

Conversation

jayzhan211
Copy link
Contributor

@jayzhan211 jayzhan211 commented Jul 23, 2024

Which issue does this PR close?

Closes #.
Part of #8708

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
@github-actions github-actions bot added logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt) labels Jul 23, 2024
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
@github-actions github-actions bot added the core Core DataFusion crate label Jul 23, 2024
@jayzhan211 jayzhan211 marked this pull request as ready for review July 23, 2024 02:20
if reverse_udf.name() == "ARRAY_AGG" {
// If the function is changed, we need to reverse order_by clause as well
// i.e. First(a order by b asc null first) -> Last(a order by b desc null last)
if self.fun().name() == reverse_udf.name() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess name checking is enough for now.

Introduce supports_rewrite_order_by for AggregateUDFImpl might add additional complexity without benefit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to eventually move this to a method in https://github.com/apache/datafusion/pull/11611 though I agree this is good for now. Maybe we can file a ticket to track

Copy link
Contributor Author

@jayzhan211 jayzhan211 Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alamb
I come out a better idea of adding reverse_name (pair with reverse_udf) method instead of supports_rewrite_order_by which makes more sense to me. #11629

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @jayzhan211

if reverse_udf.name() == "ARRAY_AGG" {
// If the function is changed, we need to reverse order_by clause as well
// i.e. First(a order by b asc null first) -> Last(a order by b desc null last)
if self.fun().name() == reverse_udf.name() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to eventually move this to a method in https://github.com/apache/datafusion/pull/11611 though I agree this is good for now. Maybe we can file a ticket to track

@@ -37,8 +36,6 @@ pub enum AggregateFunction {
Min,
/// Maximum
Max,
/// Aggregation into an array
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@alamb alamb changed the title Remove ArrayAgg Builtin Remove ArrayAgg Builtin in favor of UDF Jul 23, 2024
@alamb alamb merged commit fc8e7b9 into apache:main Jul 23, 2024
25 checks passed
@jayzhan211 jayzhan211 deleted the array-agg branch July 24, 2024 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate logical-expr Logical plan and expressions physical-expr Physical Expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants