Typical Energy Cost Reduction w/ TornadoVM

[lojo]

Hi,

this might seem like a silly question, but is there any good/reliable information out there how much energy (costs) one can save with a TornadoVM based solution vs. a pure CPU based one? Maybe a typical set of calculation intensive scenarios with TornadoVM vs pure CPU based ones.

And furthermore: How easy is it to incorporate new hardware accelerators that might not support OpenCL, CUDA etc.
IOW, how easy is it to adapt to new accelerators. Or even the upcoming CXL standard w/ custom PCI cards? Or Groq hardware?

• Artur

[kotselidis]

Hi Artur,

Regarding your first question, unfortunately it is a bit difficult to give a clear answer regarding energy savings by using TornadoVM since there are multiple factors that can affect such calculation (i.e. workload type, data sizes, type of hardware accelerator, etc.).
We have done performance analyses with multiple set of devices and we found out that even within a single kernel, the "winning" device in terms of performance may vary across data sizes (https://jjfumero.github.io/files/VEE2019_Fumero_Preprint.pdf). If we factor in the TDP of said devices then someone could do some calculations to have an estimate but again there is no certainty unless we actually run the workloads and measure the power draw from the socket directly (this is future work we plan to do).

Regarding the second question, we already support OpenCL, CUDA (PTX), and SPIR-V. If an accelerator uses a different instruction set or programming model, then a new backend will have to be added. From our experience, this can take between 6 and 18 months based on experience. The layers of TornadoVM are fairly decoupled so all changes will be isolated to the code generation phases.

Hope this helped,

Christos