Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: Switch LastValue SQL workflow to UDAF version #10062

Closed
wants to merge 2 commits into from

Conversation

jayzhan211
Copy link
Contributor

@jayzhan211 jayzhan211 commented Apr 12, 2024

Which issue does this PR close?

Closes #.

Part of #8708

Rationale for this change

After this change, LAST_VALUE goes to the registered UDAF instead of builtin one.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Signed-off-by: jayzhan211 <jayzhan211@gmail.com>
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jayzhan211 -- this looks very cool. I think we should remove the built in window function with the same PR so it is clear which versions are being used.

If we leave both in it will be quite confusing

@jayzhan211
Copy link
Contributor Author

To support replacing window first/last, there are many issues to be solved.

Including replacing first/last in convert_firsts_last rule, and support sliding accumulator for udaf, and support list instead of NUMERIC signature.

@alamb
Copy link
Contributor

alamb commented Apr 15, 2024

To support replacing window first/last, there are many issues to be solved.

I wonder if starting with simpler aggregates (like approx_distinct) might be a good idea. They are probably simpler to port and would let us work through some of the details before breaking off the larger more complex ones

first_value / last_value are likely the most complex as they are order aware and can be used as aggregates and window functions

@jayzhan211
Copy link
Contributor Author

jayzhan211 commented Apr 16, 2024

To support replacing window first/last, there are many issues to be solved.

I wonder if starting with simpler aggregates (like approx_distinct) might be a good idea. They are probably simpler to port and would let us work through some of the details before breaking off the larger more complex ones

first_value / last_value are likely the most complex as they are order aware and can be used as aggregates and window functions

While I try to deprecate first/last, I found that window function can works with UDAF without additional change.

Including replacing first/last in convert_firsts_last rule, and support sliding accumulator for udaf, and support list instead of NUMERIC signature.

The issue I mentioned is part of a need for deprecating first/last, not for window function.

This PR is pull out from #10091 to hope that I can minimize the change in that PR.

@jayzhan211 jayzhan211 closed this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants