Error when using onnx models with GPU (CUDA 10)

[louistrouche]

System information

• Microsoft.ML.OnnxTransformer : 1.4.0

• Microsoft.ML : 1.4.0

• OnnxRuntime.Gpu (test) : 1.0.1

• CUDA/cuDNN version: : CUDA 10.0.130, cuDNN 7.6.2.

• .NET Core 2.0

Issue

I'm currently trying to use an onnx model with a GPU and I get the following exception:

System.EntryPointNotFoundException: Unable to find an entry point named 'OrtSessionOptionsAppendExecutionProvider_CUDA' in DLL 'onnxruntime'.

This exception appear when I use the ApplyOnnxModel method :
var pipeline =_mlContext.Transforms.ApplyOnnxModel( gpuDeviceId: gpuDeviceId, modelFile: onnxModelFilePath);

I have tried with and without installing the NuGet package Microsoft.ML.OnnxRuntime.Gpu (1.0.1) and I get the same exception. I have no problem using my onnx model when I use the OnnxRuntime.Gpu package directly.

It seems like even when I install the OnnxRuntime.Gpu package, it's the CPU version that is used. I would like to know if there is a way to make it work without having to build a local ML.NET OnnxTransformer package from source which depends on the OnnxRuntime.Gpu package.

Source code / logs

[harishsk]

ML.NET v1.4 works with Microsoft.ML.OnnxRuntime v0.5.1. A change was introduced in the OnnxRuntime v1.0 release which breaks the v1.4 release.

If you use ML.NET v1.5, you won't see that exception, but you still won't get GPU support.

Currently there is no GPU support for Onnx model execution via ML.NET. This requires certain changes to both ML.NET and OnnxRuntime. We are aware of the issue and are working on it. We do not have a estimated date for this support yet.

I am closing the issue for now. Please reopen if you have additional questions.