-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Is your feature request related to a problem or challenge?
This issue is trying to track the follow on tasks for #13377
- Currently record batches' memory size will be overestimated, and this issue will be (partly) fixed by Fix record batch memory size double counting #13377 After merge, other usage of
record_batch.get_array_memory_size()should be replaced (e.g. Memory counting inTopKandSort-Merge-Join) After that, more end-to-end tests for that specific operator can be added - Check whether known memory-limited query related bug can be fixed by '1'. Issues maybe related are:
External aggregation reserves more memory than actual usage #13089
Further refine the Top K sort operator #9417
Excessive memory consumption on sorting #10511
External sorting not working for (maybe only for string columns??) #12136
Some memory reservations of GroupedHashAggregateStream seem to be mis-tagged as spillable while they do not allow spilling #11390
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
haohuaijin
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request