Skip to content

[Track]: Stabilize external sort #16132

Open
@2010YOUY01

Description

@2010YOUY01

Is your feature request related to a problem or challenge?

Background

There are two issues current exist, to prevent external sort to finish stably.

  1. A complete solution for stable and safe sort with spill #14692
  2. More accurate memory accounting in external sort #14748

After those issues get fixed, we should try to do more comprehensive test through:

  1. Enable memory limit in benchmarks and track the failed queries (@ding-young is working on it)
  2. Enable the fuzzer test described in Enable sort query fuzzing with limited memory #15517
  3. Integrate test test: add fuzz test for doing aggregation with larger than memory groups and sorting with limited memory #15727

Current Status

For the first issue: #15610 I think this PR can serve as the foundation for future optimizations. I'll merge main and polish it over the next few weeks, and also look for someone to help with another round of review.

For the second issue, see related tasks in #16131.

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions