gpu: nvidia: Added support for native host task extension

[ShanoToni]

Description

Adding support for the extension ext_codeplay_enqueue_native_command LINK to documentation.

Apart from improved dependencies for native commands, the native command provides less execution overhead, improving performance.

Example:

# old host task
    total perf: min(ms):38.256 avg(ms):97.7698
# native host task
    total perf: min(ms):33.4952 avg(ms):33.7441

Results generated running on A100 with command
--eltwise --engine=gpu --mode=P --skip-impl=dpcpp:ref:any --batch=tests/benchdnn/inputs/eltwise/test_eltwise_gpu

Checklist

General

• [ x ] Do all unit and benchdnn tests (make test and make test_benchdnn_*) pass locally for each commit?
• [ x ] Have you formatted the code using clang-format?

Performance improvements

• [ x ] Have you submitted performance data that demonstrates performance improvements?

[mgouicem]

make test
disable device_cpu
enable device_gpu
enable thr_cuda
enable arch_rtx

[densamoilov]

The spec says:

The call to interopCallable must not submit any synchronous tasks to the native backend object, and it must not block waiting for any tasks to complete. The call also must not add tasks to backend objects that underly any other queue, aside from the queue that is associated with this handler. If it does any of these things, the behavior is undefined.

And we have a few places where we do explicitly synchronize, here, for example.

It seems that it may become an issue when ext_codeplay_enqueue_native_command is used.

[densamoilov]

In validation the following case hung and the process was killed by timeout: --mode-modifier=P --bnorm --engine=gpu --inplace=true ic23n"bnorm_ci_0d:1", so there seems to be an issue indeed. Given that we enable the new host task extension conditionally we should also introduce a function that should do the synchronization under the same condition.

[ShanoToni]

I have made some changes and added a utility sync_device which would synchronize only when using default host_task.
I wanted to additionally ask, what is the reason for the additional interop_task? If my understanding is correct the current situation is q.submit{ host_task { q.submit { memset/device sync } } } in the current state I do not think that the internal submit will wait on any prior events, as that would be handled by the external submit, and no following dependencies will come out from the internal submit as we synchronize the device in the host_task case, while the enqueue_native_command would resolve any dependencies.

I have removed the internal interop task and as I am not seeing failures without it, let me know if this change does not make sense/ is wrong, and I can revert it.

[densamoilov]

I don't know why there was the additional interop_task and I can't think of any reason why we may need it if we block the entire device anyway.