Lack of a significant increase in performance

[catfoolyou]

Just used the latest build of TornadoVM to run my Java raytracer. For whatever reason, there is no significant increase of performance.

Using the same scene (a cornell box) and the same parameters

• 600x600 pixels
• 200 samples per pixel
• 50 bounces

The difference in performance is as follows
Java: 624.59 s
TornadoVM: 604.369 s

Is this because I am running on an iGPU (intel HD 520)? Should I expect to see better results on a computer with an RTX GPU?

[jjfumero]

Hi @catfoolyou . Performance depends on the application, amount of data to be transferred and the block of threads to run. Running with TornadoVM and force to use the GPU does not guarantee higher performance compared to Java.

You can inspect some metrics such as the time for data transfers (copy in/out), JIT compilation times, Graal JIT compilation, and kernel timers with the --enableProfiler console flag. This will emit a JSon entry every time an execution plan is executed. If the dominant time is the data transfers, then it will be hard to get performance. Ideally, the majority should be spent on the GPU kernel time. This flag will help to understand the behaviour of your application.

Furthermore, as you point out, if the kernel time is the dominant but it runs slow, a better GPU (e.g., NVIDIA discrete GPU) will help.

I recently made a video-tutorial about how to profile in TornadoVM: https://youtu.be/kOo07M7MXGA?si=kfiCs-xXGtfltBVy

Hope this helps.

[catfoolyou]

When I get home I'll try running the same test on an RTX 3060 and update the results.

[catfoolyou]

Just tested on a laptop with an RTX 3060 and a core i7 12800hx. Barely noticeable performance increase from 80 seconds with pure Java to 78 seconds with TornadoVM.

[PhillHenry]

I too am not seeing massive performance gains. I'm running:
tornado -jar tornado-benchmarks/target/jmhbenchmarks.jar uk.ac.manchester.tornado.benchmarks.sgemv.JMHSgemV
on an Nvidia Quadro T2000 on Git commit ed4b6ab and seeing:

Benchmark              Mode  Cnt         Score         Error  Units
JMHSgemV.sgemVJava     avgt    5  72366270.751 ± 5916807.539  ns/op
JMHSgemV.sgemVTornado  avgt    5  57583087.103 ± 2523449.341  ns/op

which doesn't seem to be a massive improvement.
I'm looking at the TornadoVM code and there seems to be quite a lot of Java collection usage. Could this be a reason for the code not being as fast as it might be?

[jjfumero]

Hi @PhillHenry , Matrix Vector is not as compute intensive as other applications. One way to go deep is to enable the profiler in TornadoVM (--enableProfiler console ) and check the data transfer time vs the GPU kernel time. My take is that the majority of the time is spent in data transfers (host to device back and forth) rather than GPU compute time. Matrix Multiplications such as sgemm and dgemm applications are more suitable for GPU compute.

One way to improve TornadoVM is to use shared memory buffers on shared memory systems and avoid data transfers.