[Draft][SYCL][E2E] Detect supported aspects at config time #9121
Conversation
This commit introduces a framework to make better use of LIT's UNSUPPORTED test category by detecting supported aspects at LIT configuration time. Until now the E2E tests would exit early and PASS, which is arguably a misleading classification. The method determines aspects at configuration time by compiling and then running a program. The output of this program is a semicolon-separated list of supported aspects *on the default device* as either '1' or '0', in an order determined by the python script. The script captures this output and sets features as available where the corresponding aspect is '1'. As a proof of concept, several tests which silently pass when fp16 and fp64 are unsupported have been converted to use this system. The method is easily extendible with new aspects. Several E2E tests make use of USM shared allocations without first checking the device, so a follow-up commit would likely make that aspect a feature too.
|
I've been prototyping this locally, and thought I'd throw it up in case others think this is a useful direction for the E2E tests. It would certainly need (better) documentation it it were to land, but this is the general sketch. |
|
Unless I'm mistaken, this tries to achieve roughly the same as #8344. However, whereas this specifies the individual aspects, @aelovikov-intel's version will try to eventually use sycl-ls to get the supported aspects available (#8433). |
Oh, excellent. I'm sorry I missed that PR. It does indeed look more complete. |
|
I've re-based and force-pushed #8433 to trigger new round of testing. Will see if/what plugins have been fixed. |
|
This is redundant now |
This commit introduces a framework to make better use of LIT's UNSUPPORTED test category by detecting supported aspects at LIT configuration time. Until now the E2E tests would exit early and PASS, which is arguably a misleading classification.
The method determines aspects at configuration time by compiling and then running a program. The output of this program is a semicolon-separated list of supported aspects on the default device as either '1' or '0', in an order determined by the python script. The script captures this output and sets features as available where the corresponding aspect is '1'.
As a proof of concept, several tests which silently pass when fp16 and fp64 are unsupported have been converted to use this system.
The method is easily extendible with new aspects. Several E2E tests make use of USM shared allocations without first checking the device, so a follow-up commit would likely make that aspect a feature too.