Sharing of memory segment

[andrii0lomakin]

Good day.
I am curious what you think about adding the wrap(MemorySegment) method into TornadoNativeArray. There is already a slice method there. But imagine that we may use a memory segment to manage memory allocations inside GPU per task and keep different types of data in continuous memory chunks provided by such an allocator. That can be useful.
There are also possible other use cases when a single memory chunk can be sliced to keep different types of data.

If you do not mind, I can provide PR for that, especially considering that it will be small.

[jjfumero]

Hi @andrii0lomakin , can you expand on this? Not sure if I follow. Some questions I have but I think we mean different things:

There is already a slice method there. But imagine that we may use a memory segment to manage memory allocations inside GPU per task

Dynamic memory allocations are not allowed on GPUs.

keep different types of data in continuous memory chunks provided by such an allocator.

Mixing types per buffer can be a challenge, or impossible for some drivers (e.g., NVIDIA). We tried to do something similar when we integrated Apache Flink with TornadoVM and we need to have separate buffers. It could be a solution to this by building a buffer of structs types. Each struct member has a different type. Do you mean this?

If you have a test case in mind feel free to share it, so it will help us to better understand the proposal.

[andrii0lomakin]

Dynamic memory allocations are not allowed on GPUs.
Sure, I am aware of it.

Mixing types per buffer can be a challenge, or impossible for some drivers (e.g., NVIDIA). We tried to do something similar when we >integrated Apache Flink with TornadoVM and we need to have separate buffers

Got it. Do you have any links that I can follow to understand the problem deeper?

[andrii0lomakin]

Or the only way is to read PTX spec?

[jjfumero]

Dynamic memory allocations are not allowed on GPUs.
Sure, I am aware of it.

Mixing types per buffer can be a challenge, or impossible for some drivers (e.g., NVIDIA). We tried to do something similar when we >integrated Apache Flink with TornadoVM and we need to have separate buffers

Got it. Do you have any links that I can follow to understand the problem deeper?

Paper: https://pure.manchester.ac.uk/ws/portalfiles/portal/233043755/MXekalaki_vldb2023.pdf

TornadoVM used to have one heap area for all buffers, and then assign different regions per buffer. We changed this model to one buffer per array recently, before integrating with Panama types. so now we can reuse and free buffers more efficiently and fine tune accesses (READ/WRITE, READ_ONLY, etc). at the cost of higher allocation rates. This approach is also used by Aparapi, and more recently, by HAT (inside Babylon).

Potentially, we want to achieve shared memory for individual buffers as well. Buffers can potentially be pinned (as in CUDA pinned memory).

Related paper: https://github.com/jjfumero/jjfumero.github.io/blob/master/files/papers/2023/jfumero-mplr2023-unified-memory.pdf