Open
Description
Is your feature request related to a problem or challenge?
Background
There are two issues current exist, to prevent external sort to finish stably.
- A complete solution for stable and safe sort with spill #14692
- More accurate memory accounting in external sort #14748
After those issues get fixed, we should try to do more comprehensive test through:
- Enable memory limit in benchmarks and track the failed queries (@ding-young is working on it)
- Enable the fuzzer test described in Enable sort query fuzzing with limited memory #15517
- Integrate test test: add fuzz test for doing aggregation with larger than memory groups and sorting with limited memory #15727
Current Status
For the first issue: #15610 I think this PR can serve as the foundation for future optimizations. I'll merge main
and polish it over the next few weeks, and also look for someone to help with another round of review.
For the second issue, see related tasks in #16131.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response