Description
https://numba.pydata.org/numba-doc/latest/cuda/external-memory.html
The CUDA Array Interface enables sharing of data between different Python libraries that access CUDA devices. However, each library manages its own memory distinctly from the others.
When multiple CUDA-aware libraries are used together, it may be preferable for Numba to defer to another library for memory management. The EMM Plugin interface facilitates this, by enabling Numba to use another CUDA-aware library for all allocations and deallocations.
An EMM Plugin is implemented by deriving from BaseCUDAMemoryManager.
class numba.cuda.BaseCUDAMemoryManager(*args, **kwargs)
- abstract memalloc(self, size)
- abstract memhostalloc(self, size, mapped, portable, wc)
- abstract mempin(self, owner, pointer, size, mapped)
- abstract initialize(self)
- abstract get_ipc_handle(self, memory)
- abstract get_memory_info(self)
- abstract reset(self)
- abstract defer_cleanup(self)
- abstract propertyinterface_version
Some external memory managers will support management of on-device memory but not host memory. For implementing EMM Plugins using one of these memory managers, a partial implementation of a plugin that implements host-side allocation and pinning is provided. To use it, derive from HostOnlyCUDAMemoryManager instead of BaseCUDAMemoryManager.
class numba.cuda.HostOnlyCUDAMemoryManager(*args, **kwargs)
- memhostalloc(self, size, mapped=False, portable=False, wc=False)
- mempin(self, owner, pointer, size, mapped=False)
- reset(self)
- defer_cleanup(self)
EMM Plugins should construct memory pointer instances that represent their allocations, for return to Numba. The appropriate memory pointer class to use in each method is:
-
MemoryPointer: returned from memalloc
-
MappedMemory: returned from memhostalloc or mempin when the host memory is mapped into the device memory space.
-
PinnedMemory: return from memhostalloc or mempin when the host memory is not mapped into the device memory space.
-
class numba.cuda.MemoryPointer(context, pointer, size, owner=None, finalizer=None)
-
class numba.cuda.cudadrv.driver.AutoFreePointer(*args, **kwargs)
-
class numba.cuda.MappedMemory(context, pointer, size, owner=None, finalizer=None)
-
class numba.cuda.PinnedMemory(context, pointer, size, owner=None, finalizer=None)
If an implementation of get_memory_info() is to provide a result, then it should return an instance of the MemoryInfo named tuple:
- class numba.cuda.MemoryInfo(free, total)
An instance of IpcHandle is required to be returned from an implementation of get_ipc_handle():
- class numba.cuda.IpcHandle(base, handle, size, source_info=None, offset=0)
The set_memory_manager() function can be used to set the memory manager at runtime. This must be called prior to the initialization of any contexts, and EMM Plugin instances are instantiated along with contexts.
It is recommended that the memory manager is set once prior to using any CUDA functionality, and left unchanged for the remainder of execution.
- numba.cuda.set_memory_manager(mm_plugin)