[SYCL][LIT] Auto-detect device aspects in sycl/test-e2e/lit.cfg.py

[aelovikov-intel]

No description provided.

[aelovikov-intel]

This won't be helpful before we migrate llvm-test-suite, so we can wait with merging it.
Another option would be to implement a similar change (with a follow-up cleanup of the tests) in the llvm-test-suite repo.

[bader]

Can we put SYCL code to a separate file? I think it would easier to support.

Probably we can add a separate sycl tool for detecting device features.

[aelovikov-intel]

Probably we can add a separate sycl tool for detecting device features.

I don't want to do that because such a tool must not be part of the user installation, yet we need to run these LIT tests with such user-visible layout (internally).

Can we put SYCL code to a separate file? I think it would easier to support.

Maybe, as long as it is located under test-e2e (see above reasoning) and isn't detected as a test by the LIT framework.

[bader]

I don't want to do that because such a tool must not be part of the user installation, yet we need to run these LIT tests with such user-visible layout (internally).

I don't see the problem. Can't we exclude this tool from user installation? We don't install neither LIT tool nor e2e tests and they also required running the test. Don't you think that maintaining SYCL sources in a separate file is simpler than within a string in a python script?

[bader]

Are all these commands executed for each test run?

[aelovikov-intel]

I hope not, because the code above/below this new snippet is doing the same thign, basically.

[bader]

Just from this code snippet, I don't see any optimizations caching the result of these commands. I assume we write the file, compile it and run the executable each time test is executed.

https://llvm.org/docs/CommandGuide/lit.html#test-suites

[aelovikov-intel]

From the link you've mentioned:

lit identifies test suites as directories containing lit.cfg or lit.site.cfg files

Once a test suite is discovered, its config file is loaded. Config files themselves are Python modules which will be executed.

In other words, it's done once per llvm-lit invocation. With extra print from this code I'm getting something like

$ llvm-lit -sv --param sycl_be=level_zero --param target_devices=gpu .
# The "device_features" check is executed here
...
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:293: note: Backend: ext_oneapi_level_zero
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:374: warning: CPU device not used
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:389: note: Test GPU device
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:432: warning: Accelerator device not used
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:467: note: Found llvm-spirv
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:467: note: Found llvm-link
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:483: note: Found pre-installed AOT device compiler ocloc
llvm-lit: <...>/llvm/sycl/test-e2e/lit.cfg.py:483: note: Found pre-installed AOT device compiler opencl-aot
                           -- Testing: 3 tests, 3 workers --

FAIL: SYCL :: TestSuiteMove/fail.cpp (1 of 3)
******************** TEST 'SYCL :: TestSuiteMove/fail.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';   false
--
Exit Code: 1

Command Output (stdout):
--
$ ":" "RUN: at line 1"
note: command had no output on stdout or stderr
$ "false"
note: command had no output on stdout or stderr
error: command failed with exit status: 1

--

********************
                           -- Testing: 3 tests, 3 workers --

********************
Failed Tests (1):
  SYCL :: TestSuiteMove/fail.cpp


Testing Time: 4.79s
  Passed: 2
  Failed: 1

[bader]

In other words, it's done once per llvm-lit invocation.

That's my understanding as well and it doesn't seem to be optimal.

[aelovikov-intel]

Please ignore first commit in this PR - it will be merged in separately via #8433.

[sommerlukas]

Changes to the kernel fusion tests look fine.

[steffenlarsen]

Great! 🥳

[aelovikov-intel]

SYCL :: Basic/vec_bool.cpp fails in another PR (#9135) and is unrelated. Also, possibly fixed by #9259.

[v-klochkov]

Ok for ESIMD.
I did not review all other tests, only some of them - those some are good too. Please rely on previous approval from SYCL RT if you did not do significant changes after that, or re-request Steffen's approval.