Skip to content

Implement grace hash join when build input is bigger than the available memory #14675

Open
@SunRunAway

Description

Feature Request

Is your feature request related to a problem? Please describe:

What we already done before is using temporary disk for build side of Hash Join.

  1. But when build input is bigger enough, the memory usage of hash table may still be larger than the available memory.
  2. It spills all build input into disk and abandons the usage of memory for build input. aka, when probing, the reading of each build row is a random I/O.

Describe the feature you'd like:

By implementing GRACE hash join or hybrid hash join, we can spill both build input and probe input when memory limit is exceeded, and for each partition we can join in memory.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions