[C++] Allow specifying of alignment in MemoryPool's allocations

[asfimport]

For spilling, I need to create buffers that are 512-byte aligned. The task is to augment MemoryPool to allow for specifying alignment explicitly when allocating (but keep the default the same).

Reporter: Sasha Krassovsky / @save-buffer
Assignee: Sasha Krassovsky / @save-buffer

PRs and other links:

• GitHub Pull Request #14225

_{Note: This issue was originally created as ARROW-17836. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
Note that alignment is already 64 bytes by default... which is 512 bits. Does it not suffice?

[asfimport]

Weston Pace / @westonpace:
I suspect this was a typo and meant to be 512 bytes. The current spilling PR is using direct I/O and so I believe the motivation is the logical block size of the disk.

[asfimport]

Antoine Pitrou / @pitrou:
Well, the direct IO PR handles alignment internally. Is there a plan to handle it differently?

[asfimport]

Sasha Krassovsky / @save-buffer:
Hi yes sorry about that, that was a typo. I definitely meant bytes, I’ve updated the description. 

[asfimport]

Sasha Krassovsky / @save-buffer:
That Direct IO PR could probably reuse this functionality, but the motivation for this PR is for spilling in the hash join

[asfimport]

Antoine Pitrou / @pitrou:
I see, thanks.

[asfimport]

Weston Pace / @westonpace:
Yes, the direct I/O PR offers a generic filesystem interface and thus has to potentially memcpy / buffer incoming data to satisfy alignment. The difference with spilling is that we are already doing a memcpy higher up in the chain. When we spill, we take the data we need to spill and partition it.

For example, if we want to add spill to sorting, and we pretend we are sorting by date, and we've accumulated too much data we might then partition into decade sized buckets and persist to disk. Then, once all the data has arrived, we can process a single decade at a time (with the hope that one decade of data is small enough to fit in memory).

That's a rough description, and there are corner cases, but the point is we already have to do a memcpy in order to handle the partitioning (partitioning is unfortunately a rather row-oriented operation) and so we want to go ahead and satisfy the alignment requirement at that point. This way, when we are ready to spill, we don't have to worry about alignment and we can just use direct I/O without any extra memcpy.

[asfimport]

Antoine Pitrou / @pitrou:
Ok, question: is adding alignment to ArrayBuilder needed for spilling, or is it sufficient to add support in MemoryPool, BufferBuilder and PoolBuffer?

[asfimport]

Sasha Krassovsky / @save-buffer:
I added it so that I could have MakeRandomBatches specify alignment requirements so that I could test individual components of the spilling PR. 

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 14225
#14225