Overhead on running dpex kernels

[AlexanderKalistratov]

Reproducer:

import dpnp
import time
import numba_dpex
from numba_dpex import Range,NdRange

@numba_dpex.kernel
def bar(a):
    a[0] = 0

a = dpnp.empty(1)
for i in range(10):
    start = time.time()
    bar[Range(1)](a)
    end = time.time()
    print(1000*(end - start))

Result:

170.96805572509766
0.9398460388183594
0.7669925689697266
0.7381439208984375
0.6814002990722656
0.7398128509521484
0.5812644958496094
0.6299018859863281
0.7305145263671875
0.7123947143554688

Expected result... about 0.1ms.

More details here: #946 (comment)

[diptorupd]

Addressed in #1236.

The new numba_dpex.experimental.kernel uses a fully native kernel dispatch mechanism that fixes the previous overhead reported in this issue.

On my local test machine, existing numba_dpex.kernel reported following performance numbers:

518.5830593109131
5.5999755859375
5.43522834777832
5.475282669067383
5.715131759643555
5.93256950378418
5.736351013183594
5.846977233886719
5.640983581542969
6.529092788696289

Using the numba_dpex.experimental.kernel from PR #1236. The following timing was captured:

236.9523048400879
0.2722740173339844
0.14162063598632812
0.16736984252929688
0.16379356384277344
0.16546249389648438
0.18072128295898438
0.13518333435058594
0.13208389282226562
0.15735626220703125