Implement plugin for EMM Deallocation Behavior

[1e-to]

Implement Deallocation Behavior like defer_cleanup context manager in CUDA

Numba’s internal Deallocation Behavior is designed to increase efficiency by deferring deallocations until a significant quantity are pending. It also provides a mechanism for preventing deallocations entirely during critical sections, using the defer_cleanup() context manager.

When an EMM Plugin is in use, the deallocation strategy is implemented by the EMM, and Numba’s internal deallocation mechanism is not used. The EMM Plugin could implement:

A similar strategy to the Numba deallocation behaviour, or

Something more appropriate to the plugin - for example, deallocated memory might immediately be returned to a memory pool.

The defer_cleanup context manager may behave differently with an EMM Plugin - an EMM Plugin should be accompanied by documentation of the behaviour of the defer_cleanup context manager when it is in use. For example, a pool allocator could always immediately return memory to a pool even when the context manager is in use, but could choose not to free empty pools until defer_cleanup is not in use.

[diptorupd]

The External Memory Management (EMM) is a numba-specific feature that is used by numba.cuda. At one time we had considered a similar approach for numba-dpex, but since then the SYCL USM memory manager has proven to be a better approach.

Closing as WONTFIX and unrelated to dpctl.