Skip to content

improve shuffle performance#559

Merged
lihao712 merged 1 commit intomasterfrom
dev-improve-shuf
Sep 3, 2024
Merged

improve shuffle performance#559
lihao712 merged 1 commit intomasterfrom
dev-improve-shuf

Conversation

@richox
Copy link
Contributor

@richox richox commented Sep 3, 2024

  1. reuse compression buffer.
  2. prefer coalescing to interleaving when merging same partitions of sorted batches.

@lihao712 lihao712 merged commit a931b42 into master Sep 3, 2024
@cxzl25 cxzl25 deleted the dev-improve-shuf branch September 5, 2025 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants