[SYCL] Introduce extension for asynchronous memory allocation #14800

AerialMantis · 2024-07-26T12:40:08Z

Introduce a proposed extension for an interface for memory pools and asynchronous allocation and freeing of USM memory.

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

alloc spec * Refactor the memory pool behaviour and lifetime sections into a new section, with less prescriptive wording. * Add new `get_current_size` and `get_free_size` member functions to the `memory_pool` class. * Add new `get_allocation_chunk_size` member function to the `memory_pool` class. * Remove mention of source.

* Remove `get_free_size()` * Remove `get_max_size()` * Rename `set_new_threshold()` to `increase_threshold_to()` * Rename `get_current_size()` to `get_reserved_size_current()` * Change behaviour of `increase_threshold_to()` in case of size not being greater than the current size from exception to a no-op * Add `usm::alloc` parameter to the host `memory_pool` constructor * Add `use::alloc` parameter to the `ext_oneapi_get_default_memory_pool` free function * Correct the behaviour of updating amount of free memory when calling `async_free()` to once the asynchronous free command has completed * Add device aspect; `ext_oneapi_async_memory_alloc` * Add a note to clarify the behaviour of the `read_only` property * Clarified behaviour of `async_malloc()` and `async_malloc_from_pool()` when allocating zero bytes to align with regular `malloc_*()` functions * Add `get_used_size_current()` can be used in combination with `get_reserved_size_current()`, in place of `get_free_size()` * Removed over-prescriptive wording from `async_free()` definition

* Remove `get_allocation_chunk_size()` * Add `property::memory_pool` namespace to property types * Change parameter passing conversion for SYCL objects to const ref

* Move some specified behaviour of the memory pool to implementation nodes * Introduce the term reserve to referred to the memory allocated to the memory pool * Replace wording around current and free memory with reserved and used memory * Change wording around allocation chunk size to be an implementation detail

…the sycl_ext_oneapi_async_memory_alloc extension (#16900) Implement the [sycl_ext_oneapi_async_memory_alloc](#14800) extension for asynchronous memory allocation and freeing in CUDA, for device allocated pools only. SYCL entrypoints which specify host or shared side pools, or pools created by pre-existing allocations will throw. co-authored-by: Sean Stirling <sean.stirling@codeplay.com> co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> --------- Co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> Co-authored-by: Nicolas Miller <nicolas.miller@codeplay.com>

…ry_alloc extension (#16900) Implement the [sycl_ext_oneapi_async_memory_alloc](intel/llvm#14800) extension for asynchronous memory allocation and freeing in CUDA, for device allocated pools only. SYCL entrypoints which specify host or shared side pools, or pools created by pre-existing allocations will throw. co-authored-by: Sean Stirling <sean.stirling@codeplay.com> co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> --------- Co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> Co-authored-by: Nicolas Miller <nicolas.miller@codeplay.com>

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

gmlueck

This is looking a lot better, but I have a few comments. Mostly they are wording improvements, not changes to the API.

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

* Switch memory_pool contructors to use the properties extension.

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

…the sycl_ext_oneapi_async_memory_alloc extension (#16900) Implement the [sycl_ext_oneapi_async_memory_alloc](#14800) extension for asynchronous memory allocation and freeing in CUDA, for device allocated pools only. SYCL entrypoints which specify host or shared side pools, or pools created by pre-existing allocations will throw. co-authored-by: Sean Stirling <sean.stirling@codeplay.com> co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> --------- Co-authored-by: Hugh Delaney <hugh.delaney@codeplay.com> Co-authored-by: Nicolas Miller <nicolas.miller@codeplay.com>

* Remove memory_pool constructor for host USM allocations * Remove memory_pool constructor for user provided allocations * Remove host variant of ext_oneapi_get_default_memory_pool * Update wording to state that only device USM allocations are supported * Add known issues about reintroducing host/shared USM allocations and user provided allocations in the future * Remove the read_only property as this is only used for shared USM allocations * Add throw clauses for when the device aspect is not supported

alycm · 2025-06-04T13:09:46Z

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

+
+|1
+|The APIs of this experimental extension are not versioned, so the
+ feature-test macro always has this value.


Should this extension be versioned if it is growing over time? Or are experimental specs not versioned as they're expected to change?

We generally do not version experimental extensions because they often change in non-backward-compatible ways. (Versioning doesn't help in this case.) However, there is no rule prohibiting versioning. If you think it's valuable to add a version, that's fine. If you do that, you must clearly document which APIs are added in each version.

I think it's sort of a moot issue for now, though. At this point, you only have one version which is version 1. If you decide to add new APIs later, you can decide at that time to leave the spec version at 1 (in which case the spec is unversioned) or bump the version to 2.

If experimental extensions are not generally versioned I do not have a reason to deviate from that here.

sycl/doc/extensions/proposed/sycl_ext_oneapi_async_memory_alloc.asciidoc

* Clarify wording to remove mention of shared USM.

AerialMantis requested a review from a team as a code owner July 26, 2024 12:40

AerialMantis force-pushed the add-async-memory-management-ext branch from a7a3ffe to 9425161 Compare July 26, 2024 12:41

AerialMantis changed the title ~~[SYCL] Introduce extension proposal for asynchornous memory management~~ [SYCL] Introduce extension proposal for asynchornous memory allocation Jul 26, 2024

jbrodman reviewed Jul 26, 2024

View reviewed changes