[SYCL] Add support for native root group implementation #11043

0x12CC · 2023-09-01T16:38:02Z

Add new PI APIs and relevant runtime changes to enable root groups composed of more than one work group. This changeset includes the following changes:

Add piextKernelSuggestMaxCooperativeGroupCount API to query the maximum number of work groups that can run cooperatively for a given kernel.
Add piextEnqueueCooperativeKernelLaunch API to launch kernels cooperatively.
Add MKernelIsCooperative flag to track if a kernel needs to be launched using piextEnqueueCooperativeKernelLaunch or piEnqueueKernelLaunch. Presently, this flag is only set if the kernel has the use_root_sync_key property. It's expected that other features may set this flag in the future if they require all work groups to run concurrently.
Modify the root group barrier implementation to use __spirv_ControlBarrier with Device scope rather than Workgroup scope.

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

0x12CC · 2023-09-01T16:42:05Z

@AlexeySachkov, @Pennycook, I've marked this PR as draft since the changes to sycl/plugins/level_zero/pi_level_zero.cpp and sycl/plugins/opencl/pi_opencl.cpp likely need to be split into a separate PR in oneapi-src/unified-runtime. Other changes should be ready for review so please feel free to provide early feedback.

sycl/plugins/level_zero/pi_level_zero.cpp

sycl/source/detail/scheduler/commands.cpp

Pennycook · 2023-09-01T17:14:23Z

@0x12CC - I want to make you aware of some recent changes to the specification; we renamed a few things, and moved the queries into a separate extension specification (sycl_ext_oneapi_launch_queries). I don't think you need to make these changes in this PR (since we should just be able to rename things afterwards) but I wanted to let you know.

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

…to native_root_group

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

0x12CC · 2023-09-20T21:26:52Z

Thanks for the reviews, @victor-eds, @ldrumm! I've made all of the requested changes.

JackAKirk · 2023-09-21T08:55:14Z

This looks at a glance like it would likely be compatible with cuda's grid_group and their cooperative kernel launch api, but has someone gone through and convinced themselves that the set of interfaces in runtime/UR will support corresponding cuda functionality, as well as any other backends that may have similar functionality?

0x12CC · 2023-09-21T18:16:16Z

This looks at a glance like it would likely be compatible with cuda's grid_group and their cooperative kernel launch api, but has someone gone through and convinced themselves that the set of interfaces in runtime/UR will support corresponding cuda functionality, as well as any other backends that may have similar functionality?

There's been some discussion of the relevant CUDA APIs in the PR for the UR changes: oneapi-src/unified-runtime#849.

sycl/plugins/level_zero/pi_level_zero.cpp

jandres742

Please move changes to UR files.

sycl/plugins/level_zero/pi_level_zero.cpp

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

0x12CC · 2023-09-28T16:45:09Z

@ldrumm, @jandres742, I've removed all of the changes related to UR and made the other requested changes. This PR now only includes the new PI function and the relevant changes to the SYCL runtime. The implementation for these new PI functions will be in the UR repository. Please review this updated changeset.

jandres742 · 2023-09-28T16:50:28Z

@ldrumm, @jandres742, I've removed all of the changes related to UR and made the other requested changes. This PR now only includes the new PI function and the relevant changes to the SYCL runtime. The implementation for these new PI functions will be in the UR repository. Please review this updated changeset.

thanks @0x12CC . Right, please see guidlines here https://github.com/oneapi-src/unified-runtime/pull/902/files

jandres742 · 2023-09-28T16:51:38Z

sycl/plugins/level_zero/pi_level_zero.cpp

@@ -573,6 +583,13 @@ pi_result piextKernelGetNativeHandle(pi_kernel Kernel,
  return pi2ur::piextKernelGetNativeHandle(Kernel, NativeHandle);
 }

+pi_result piextKernelSuggestMaxCooperativeGroupCount(pi_kernel Kernel,
+                                                     pi_uint32 *GroupCountRet) {
+  (void)Kernel;


this needs to be also a redirection to pi2ur, even if all that we are doing is returning 1.

sycl/include/sycl/detail/pi.h

Co-authored-by: Nikita Kornev <nikita.kornev@intel.com>

0x12CC · 2024-01-11T18:57:00Z

I've closed this PR since the branch is outdated and created #12367 to replace it. The new PR includes all of the suggestions/fixes from this PR.

bader · 2024-01-11T19:59:51Z

I've closed this PR since the branch is outdated and created #12367 to replace it. The new PR includes all of the suggestions/fixes from this PR.

I recommend updating the branch instead of opening a new PR. It's hard to track whether comments were addressed or not. It's also unclear where to continue discussions if reviews have follow-up comments.

0x12CC added 4 commits August 22, 2023 06:38

Add calls to new PI APIs for query and launch

b6caf20

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Merge branch 'sycl' into native_root_group

cc308d9

Update CGExecKernel construction in unit tests

64c4ad3

Update barrier and kernel launch

8807a89

0x12CC requested review from Pennycook and AlexeySachkov September 1, 2023 16:38

Pennycook reviewed Sep 1, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

Pennycook reviewed Sep 1, 2023

View reviewed changes

sycl/source/detail/scheduler/commands.cpp Show resolved Hide resolved

0x12CC mentioned this pull request Sep 5, 2023

Add cooperative kernels experimental feature oneapi-src/unified-runtime#849

Merged

0x12CC added 16 commits September 12, 2023 10:24

Add extension include to sycl header

4b822ce

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Merge branch 'native_root_group' of https://github.com/0x12CC/llvm in…

ba64c38

…to native_root_group

Add piextEnqueueCooperativeKernelLaunch to CL plugin

7aefca9

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Add new APIs to symbol dumps

acc527d

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Fix is_fixed_topology test case

5260f80

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Update barrier implementation

0811c57

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Merge branch 'sycl' into native_root_group

58b8161

Merge branch 'sycl' into native_root_group

8cc65c3

Add new PI APIs to CUDA and HIP plugins

4aa376a

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Update windows symbols dump

40e7a1f

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Fix CUDA and HIP PI functions

7596203

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Update group_scope<root_group> to Device

1637e89

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Update symbol dumps

0e6b172

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Add TODO for UR changes

1c6c9cf

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Disable root group test on L0

47ce8f7

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Fix assertions in root group test

881263d

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

0x12CC marked this pull request as ready for review September 14, 2023 14:59

0x12CC requested review from a team and victor-eds as code owners September 14, 2023 14:59

0x12CC added 4 commits September 20, 2023 14:04

Refactor feature test macro and checks

317b969

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Add comments for piextEnqueue*KernelLaunch parameters

3ae2361

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Add TODO for urEnqueueCooperativeKernelLaunch

d08e935

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Merge branch 'sycl' into native_root_group

af778bc

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

0x12CC requested a review from ldrumm September 20, 2023 21:25

0x12CC temporarily deployed to WindowsCILock September 20, 2023 21:48 — with GitHub Actions Inactive

0x12CC temporarily deployed to WindowsCILock September 20, 2023 22:30 — with GitHub Actions Inactive

jandres742 reviewed Sep 22, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

jandres742 suggested changes Sep 22, 2023

View reviewed changes

jandres742 reviewed Sep 22, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

jandres742 reviewed Sep 22, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

jandres742 reviewed Sep 22, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

jandres742 reviewed Sep 22, 2023

View reviewed changes

sycl/plugins/level_zero/pi_level_zero.cpp Outdated Show resolved Hide resolved

0x12CC added 2 commits September 28, 2023 09:26

Remove L0 changes from PI plugin

33917f8

Signed-off-by: Michael Aziz <michael.aziz@intel.com>

Merge branch 'sycl' into native_root_group

2cd01b0

0x12CC temporarily deployed to WindowsCILock September 28, 2023 16:45 — with GitHub Actions Inactive

0x12CC requested a review from jandres742 September 28, 2023 16:45

jandres742 reviewed Sep 28, 2023

View reviewed changes

0x12CC temporarily deployed to WindowsCILock September 28, 2023 17:14 — with GitHub Actions Inactive

KornevNikita reviewed Nov 3, 2023

View reviewed changes

sycl/include/sycl/detail/pi.h Outdated Show resolved Hide resolved

Update sycl/include/sycl/detail/pi.h

da862ac

Co-authored-by: Nikita Kornev <nikita.kornev@intel.com>

0x12CC closed this Jan 11, 2024

0x12CC deleted the native_root_group branch January 11, 2024 18:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[SYCL] Add support for native root group implementation #11043

[SYCL] Add support for native root group implementation #11043

Uh oh!

0x12CC commented Sep 1, 2023

Uh oh!

0x12CC commented Sep 1, 2023

Uh oh!

Uh oh!

Uh oh!

Pennycook commented Sep 1, 2023

Uh oh!

0x12CC commented Sep 20, 2023

Uh oh!

JackAKirk commented Sep 21, 2023

Uh oh!

0x12CC commented Sep 21, 2023

Uh oh!

Uh oh!

jandres742 left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

0x12CC commented Sep 28, 2023

Uh oh!

jandres742 commented Sep 28, 2023

Uh oh!

jandres742 Sep 28, 2023

Uh oh!

Uh oh!

0x12CC commented Jan 11, 2024

Uh oh!

bader commented Jan 11, 2024

Uh oh!

Uh oh!

[SYCL] Add support for native root group implementation #11043

[SYCL] Add support for native root group implementation #11043

Uh oh!

Conversation

0x12CC commented Sep 1, 2023

Uh oh!

0x12CC commented Sep 1, 2023

Uh oh!

Uh oh!

Uh oh!

Pennycook commented Sep 1, 2023

Uh oh!

0x12CC commented Sep 20, 2023

Uh oh!

JackAKirk commented Sep 21, 2023

Uh oh!

0x12CC commented Sep 21, 2023

Uh oh!

Uh oh!

jandres742 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

0x12CC commented Sep 28, 2023

Uh oh!

jandres742 commented Sep 28, 2023

Uh oh!

jandres742 Sep 28, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

0x12CC commented Jan 11, 2024

Uh oh!

bader commented Jan 11, 2024

Uh oh!

Uh oh!