-
Notifications
You must be signed in to change notification settings - Fork 71
Closed
Labels
Type: enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed
Milestone
Description
Describe the enhancement requested
In BaseVariableWidthVector
the reallocation logic of the data buffer is to always use the "next power of 2" size instead of the actual desired size (see here). The reallocation fails if the new size exceeds the MAX_BUFFER_SIZE
which is Integer.MAX_VALUE
by default.
The issue is Integer.MAX_VALUE
is just 1 byte less than a power of 2 (231 - 1) which means that exceeding the desired capacity of 1073741824
(1GB), the reallocation would fail. It means, we are unable to get a data buffer larger than 1GB while the default maximum size is almost 2GB.
Metadata
Metadata
Assignees
Labels
Type: enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed