[ESIMD] Enable esimd emulator build by default #5058

dongkyunahn-intel · 2021-12-01T04:50:24Z

This PR is to enable ESIMD_EMULATOR build by default.

PI_APIs for PI info are updated for check-in tests

- To be replaced at CM_EMU update

buildbot/configure.py

- Reducing overhead on command creation in specific case from PR#4841 - Same kernel launching flow for ESIMD_EMULATOR is in ExecCGCommand::enqueueImp() in commands.cpp

kbobrovs · 2021-12-09T04:52:00Z

sycl/plugins/esimd_emulator/pi_esimd_emulator.cpp

@@ -1324,6 +1360,7 @@ pi_result piEnqueueMemBufferReadRect(pi_queue, pi_mem, pi_bool,
 pi_result piEnqueueMemBufferWrite(pi_queue, pi_mem, pi_bool, size_t, size_t,
                                  const void *, pi_uint32, const pi_event *,
                                  pi_event *) {
+  // TODO : intel/llvm_test_suite


what does this mean - "// TODO add a test to github.com/intel/llvm-test-suite"?
Please update if so. Even though I don't see much sense in such TODO.

kbobrovs · 2021-12-09T04:55:53Z

sycl/plugins/esimd_emulator/pi_esimd_emulator.cpp

@@ -1623,7 +1674,8 @@ pi_result piextDeviceSelectBinary(pi_device, pi_device_binary *,
 pi_result piextUSMEnqueuePrefetch(pi_queue, const void *, size_t,
                                  pi_usm_migration_flags, pi_uint32,
                                  const pi_event *, pi_event *) {
-  DIE_NO_IMPLEMENTATION;
+  // NOP for prefetch


Please explain in the comment why NOP for prefetch is safe.

I'll replace it with TODO like above ones.

kbobrovs · 2021-12-09T04:57:21Z

sycl/plugins/esimd_emulator/pi_esimd_emulator.cpp

@@ -528,7 +538,7 @@ pi_result piDeviceGetInfo(pi_device Device, pi_device_info ParamName,
  case PI_DEVICE_INFO_VERSION:
    // CM_EMU release version from
    // https://github.com/intel/cm-cpu-emulation/releases
-    return ReturnValue("1.0.7-CM_EMU");
+    return ReturnValue("1.0");


this should rather be a query into libcm. at least TODO should be added

I'll check if the version number can be retrieved from CM and apply it if possible.

And, please note that the device info version format should be 'X.Y'. It is required from one of tests in intel/llvm-test-suite.

kbobrovs · 2021-12-09T04:58:45Z

sycl/source/detail/scheduler/commands.cpp

@@ -2157,10 +2157,12 @@ cl_int ExecCGCommand::enqueueImp() {
      } else {
        assert(MQueue->getPlugin().getBackend() ==
               backend::ext_intel_esimd_emulator);
+        // Dims==0 for 'single_task() - void(void) type'
+        uint32_t Dims = (Args.size() > 0) ? NDRDesc.Dims : 0;


AFAICT, single_task is not supported. Should this cause DIE_NO_IMPLEMENTATION?

'single_task()' is widely used in intel/llvm-test-suite. ESIMD/SYCL/printf.cpp uses 'single_task()'. I would like to make sure if it is going to be supported or not.

But your code implies it is not supported yet. So DIE_NO_IMPLEMENTATION should happen on attempt to use it.

DIE_NO_IMPLEMENTATION is defined in sycl/plugins/esimd_emulator/pi_esimd_emulator.cpp. Maybe execution attempts for single_task() return PI_INVALID_WORK_DIMENSION or PI_INVALID_KERNEL (or some other error code ) here until it is supported?

Are we going to support single_task() or not ESIMD?

kbobrovs · 2021-12-10T19:23:24Z

Note on running SYCL kernels:
ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

romanovvlad · 2021-12-13T08:59:37Z

Note on running SYCL kernels: ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

If we convert sycl_explicit_simd attribute to a compile time property(see #4937) and starting report something like piDeviceGetInfo(PI_SUPPORT_ESIMD) == true from plugins that support esimd execution model we can have the wanted diagnostic in the SYCL RT alongside with other checks for kernel requirements vs device capabilities.

BTW.

IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism

I believe we have avoided adding new invocations interface, could you please share a link to the line where it's defined if we have it?

kbobrovs · 2021-12-14T00:09:06Z

Note on running SYCL kernels: ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

If we convert sycl_explicit_simd attribute to a compile time property(see #4937) and starting report something like piDeviceGetInfo(PI_SUPPORT_ESIMD) == true from plugins that support esimd execution model we can have the wanted diagnostic in the SYCL RT alongside with other checks for kernel requirements vs device capabilities.

I'm not sure if #4937 will work for emulator, as device code does not go through SPIRV or even device compiler.

BTW.

IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism

I believe we have avoided adding new invocations interface, could you please share a link to the line where it's defined if we have it?

Looks like my IIRC is wrong. I see we use piEnqueueKernelLaunch.
Maybe SYCL RT could see if there is isESIMD property on device binary being passed to ESIMD emulator BE and report an error if it is missing.

- As esimd_emulator is only for ESIMD kernels, PI_APIs causing failures for non-ESIMD kernels are not going to be implemented. 'TODO' comments are removed for such PI_APIs - CMakeLists change : installtion path fix for headers imported from CM

romanovvlad · 2021-12-14T10:49:19Z

Note on running SYCL kernels: ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

If we convert sycl_explicit_simd attribute to a compile time property(see #4937) and starting report something like piDeviceGetInfo(PI_SUPPORT_ESIMD) == true from plugins that support esimd execution model we can have the wanted diagnostic in the SYCL RT alongside with other checks for kernel requirements vs device capabilities.

I'm not sure if #4937 will work for emulator, as device code does not go through SPIRV or even device compiler.

With this properties we could convert all attributes to properties, so they are known to the SYCL RT without compiler assistance.

kbobrovs · 2021-12-15T00:50:10Z

Note on running SYCL kernels: ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

If we convert sycl_explicit_simd attribute to a compile time property(see #4937) and starting report something like piDeviceGetInfo(PI_SUPPORT_ESIMD) == true from plugins that support esimd execution model we can have the wanted diagnostic in the SYCL RT alongside with other checks for kernel requirements vs device capabilities.

I'm not sure if #4937 will work for emulator, as device code does not go through SPIRV or even device compiler.

With this properties we could convert all attributes to properties, so they are known to the SYCL RT without compiler assistance.

To get properties we need device binary. Device binary is not used with the ESIMD emulator, I believe. Should we at all require presence of device binary when running on host device (ESIMD emulator shares a lot with it) ?

romanovvlad · 2021-12-15T13:49:19Z

Note on running SYCL kernels: ESIMD emulator by definition is another kind of "gpu" device, which can run only ESIMD kernels, it can't run normal SYCL kernels. We need a way to detect that ESIMD EMU plugin is asked to run a non-ESIMD kernel (i.e. sycl_explicit_simd attribute is missing for it), and issue a meaningful error. This might require extending the PI kernel invocation interface to propagate such kernel attributes/properties. IIRC, we added special new invocation interface to PI to invoke kernels bypassing the "normal" mechanism, so we could extend it further to propagate needed info. @romanovvlad, @smaslov-intel - do you agree or have other comments?

If we convert sycl_explicit_simd attribute to a compile time property(see #4937) and starting report something like piDeviceGetInfo(PI_SUPPORT_ESIMD) == true from plugins that support esimd execution model we can have the wanted diagnostic in the SYCL RT alongside with other checks for kernel requirements vs device capabilities.

I'm not sure if #4937 will work for emulator, as device code does not go through SPIRV or even device compiler.

With this properties we could convert all attributes to properties, so they are known to the SYCL RT without compiler assistance.

To get properties we need device binary. Device binary is not used with the ESIMD emulator, I believe. Should we at all require presence of device binary when running on host device (ESIMD emulator shares a lot with it) ?

I'm referring to sycl::ext::oneapi::property_list and sycl::ext::oneapi::property_value(see https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/PropertyList/SYCL_EXT_ONEAPI_property_list.asciidoc and https://github.com/intel/llvm/pull/4937/files?short_path=8b8203c#diff-8b8203c67c38c950b9c19edd0aa137e67e3683c84fc454fe2982ebb19e7a47b1)

…d_emu_build

- Using version info fetched from calling 'CreateCmDevice'

kbobrovs · 2021-12-21T02:53:05Z

I'm referring to sycl::ext::oneapi::property_list and sycl::ext::oneapi::property_value(see https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/PropertyList/SYCL_EXT_ONEAPI_property_list.asciidoc and https://github.com/intel/llvm/pull/4937/files?short_path=8b8203c#diff-8b8203c67c38c950b9c19edd0aa137e67e3683c84fc454fe2982ebb19e7a47b1)

How these would be propagated to SYCL runtime?

bader

buildbot/configure.py looks okay, but I have one question though: should we have a switch to disable ESIMD emulator build? I suppose it might be useful for developers not involved in ESIMD development.

…d_emu_build

sycl/source/handler.cpp

And, - Argument sanity check failure revised for piDevicesGet - DeviceType condition check fix

- This PR is for clearing failures observed while merging intel/llvm PR enabling esimd_emulator support by default (intel/llvm#5058) - Another PR should be created for actively loading and testing 'esimd_emulator' backend

bader · 2022-01-18T11:43:01Z

buildbot/configure.py looks okay, but I have one question though: should we have a switch to disable ESIMD emulator build? I suppose it might be useful for developers not involved in ESIMD development.

@dongkyunahn-intel, @kbobrovs, what do you think about flipping the switch instead of removing it?

dongkyunahn-intel · 2022-01-18T17:40:36Z

buildbot/configure.py looks okay, but I have one question though: should we have a switch to disable ESIMD emulator build? I suppose it might be useful for developers not involved in ESIMD development.

Do other backend types have disabling option like you suggested? If not, I don't want to have it only for ESIMD emulator.

bader · 2022-01-18T18:01:32Z

buildbot/configure.py looks okay, but I have one question though: should we have a switch to disable ESIMD emulator build? I suppose it might be useful for developers not involved in ESIMD development.

Do other backend types have disabling option like you suggested? If not, I don't want to have it only for ESIMD emulator.

HIP and CUDA backend builds are controlled by dedicated options.

- Log-in suppression - 'cpu' to 'emu' (PR intel#4728)

…d_emu_build

kbobrovs · 2022-01-18T23:35:40Z

buildbot/configure.py looks okay, but I have one question though: should we have a switch to disable ESIMD emulator build? I suppose it might be useful for developers not involved in ESIMD development.

Do other backend types have disabling option like you suggested? If not, I don't want to have it only for ESIMD emulator.

HIP and CUDA backend builds are controlled by dedicated options.

Makes sense to me too.

vladimirlaz · 2022-01-19T07:46:03Z

I am trying to build esimd_emulator locally and got the following error:

llvm/build/tools/sycl/plugins/esimd_emulator/cm-emu-prefix/src/cm-emu/cmake/../common/linux/cm_include.h:22:10: fatal error: va/va.h: No such file or directory
   22 | #include "va/va.h"
      |          ^~~~~~~~~
compilation terminated.

Does this build brings new dependency: libva-dev package to be installed on ?

The problem was resolved locally after installing the package:

apt-get install libva-dev

bader · 2022-01-19T08:32:19Z

AFAIK, libva-dev is not the only dependency. See #5212 for the full list.

bader

buildbot/configure.py changes look good to me.
We need someone from @intel/llvm-reviewers-runtime to approve.

bader · 2022-01-20T10:01:03Z

It looks like required dependencies are not installed for "shared lib" build configuration - https://github.com/intel/llvm/runs/4872502344?check_suite_focus=true.
@alexbatashev, is it possible to unify build configurations and use a docker image to set up worker's environment?

alexbatashev · 2022-01-20T10:52:38Z

@bader working on it

- This PR is for clearing failures observed while merging intel/llvm PR enabling esimd_emulator support by default (intel#5058) - Another PR should be created for actively loading and testing 'esimd_emulator' backend

dongkyunahn-intel added 2 commits November 30, 2021 11:10

Enabling ESIMD_EMU support build by default

b010840

Replacing ESIMDEmu with ESIMDCPU for legacy CM_EMU

f8ae218

- To be replaced at CM_EMU update

dongkyunahn-intel requested review from kbobrovs and vladimirlaz December 1, 2021 04:50

dongkyunahn-intel requested review from bader, smaslov-intel and a team as code owners December 1, 2021 04:50

bader changed the title ~~Enable esimd emu build~~ [ESIMD] Enable esimd emulator build by default Dec 1, 2021

bader requested changes Dec 1, 2021

View reviewed changes

buildbot/configure.py Show resolved Hide resolved

dongkyunahn-intel added 4 commits December 1, 2021 19:42

PI Device info update

1d8cb9c

Removing command line option for esimd_cpu_emulation

183a2b3

Applying reducing overhead on command creation for ESIMD_EMULATOR BE

340e5dd

- Reducing overhead on command creation in specific case from PR#4841 - Same kernel launching flow for ESIMD_EMULATOR is in ExecCGCommand::enqueueImp() in commands.cpp

ESIMD_EMULATOR plug-in update for bringing-up intel/llvm-test-suite

8ba735d

kbobrovs reviewed Dec 9, 2021

View reviewed changes

kbobrovs mentioned this pull request Dec 10, 2021

[SYCL][ESIMD][EMU] Enabling ESIMD_EMULATOR intel/llvm-test-suite#582

Closed

dongkyunahn-intel added 4 commits December 17, 2021 10:15

Merge branch 'sycl' of https://github.com/intel/llvm into enable_esim…

e01e682

…d_emu_build

Build error fix after pulling sycl branch

11155d1

clang-format fix

7efcbc0

CM-EMU Device version info composition

5471769

- Using version info fetched from calling 'CreateCmDevice'

bader previously approved these changes Dec 21, 2021

View reviewed changes

Merge branch 'sycl' of https://github.com/intel/llvm into enable_esim…

85baa6c

…d_emu_build

dongkyunahn-intel dismissed bader’s stale review via 85baa6c January 11, 2022 01:28

bader reviewed Jan 15, 2022

View reviewed changes

sycl/source/handler.cpp Outdated Show resolved Hide resolved

dongkyunahn-intel added 2 commits January 15, 2022 11:19

Updating returning platform info with NumEntries

d71a04d

And, - Argument sanity check failure revised for piDevicesGet - DeviceType condition check fix

Single-point-of-return for EnqueueKernel

6f1034e

dongkyunahn-intel mentioned this pull request Jan 16, 2022

Esimd emulator preparation intel/llvm-test-suite#744

Merged

vladimirlaz previously approved these changes Jan 17, 2022

View reviewed changes

vladimirlaz requested review from bader and kbobrovs January 17, 2022 13:57

dongkyunahn-intel added 3 commits January 18, 2022 12:23

Adding command line option for disabling ESIMD_EMULATOR build

71802cc

Updating pi_esimd_emulator build due to update in open-source CM_EMU

139a549

- Log-in suppression - 'cpu' to 'emu' (PR intel#4728)

Merge branch 'sycl' of https://github.com/intel/llvm into enable_esim…

bba4c55

…d_emu_build

dongkyunahn-intel dismissed vladimirlaz’s stale review via bba4c55 January 18, 2022 21:31

dongkyunahn-intel mentioned this pull request Jan 18, 2022

[SYCL][ESIMD][EMU] Marking ESIMD kernels for esimd_emulator backend intel/llvm-test-suite#751

Merged

bader approved these changes Jan 19, 2022

View reviewed changes

vladimirlaz approved these changes Jan 19, 2022

View reviewed changes

vladimirlaz mentioned this pull request Jan 19, 2022

[SYCL] Update LIT to support the latest BEs intel/llvm-test-suite#752

Merged

againull merged commit 50ccce9 into intel:sycl Jan 19, 2022

JackAKirk mentioned this pull request Jan 21, 2022

DPC++ build failure due to #5058 (ESIMD cm-emu configure step) #5360

Closed

[ESIMD] Enable esimd emulator build by default #5058

[ESIMD] Enable esimd emulator build by default #5058

Uh oh!

Conversation

dongkyunahn-intel commented Dec 1, 2021

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongkyunahn-intel Dec 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kbobrovs commented Dec 10, 2021

Uh oh!

romanovvlad commented Dec 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kbobrovs commented Dec 14, 2021

Uh oh!

romanovvlad commented Dec 14, 2021

Uh oh!

kbobrovs commented Dec 15, 2021

Uh oh!

romanovvlad commented Dec 15, 2021

Uh oh!

kbobrovs commented Dec 21, 2021

Uh oh!

bader left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bader commented Jan 18, 2022

Uh oh!

dongkyunahn-intel commented Jan 18, 2022

Uh oh!

bader commented Jan 18, 2022

Uh oh!

kbobrovs commented Jan 18, 2022

Uh oh!

vladimirlaz commented Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bader commented Jan 19, 2022

Uh oh!

bader left a comment

Choose a reason for hiding this comment

Uh oh!

bader commented Jan 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexbatashev commented Jan 20, 2022

Uh oh!

Uh oh!

dongkyunahn-intel Dec 21, 2021 •

edited

Loading

romanovvlad commented Dec 13, 2021 •

edited

Loading

vladimirlaz commented Jan 19, 2022 •

edited

Loading

bader commented Jan 20, 2022 •

edited

Loading