Description
I'm trying to detect objects in images by using an exported ONNX model from Custom Vision on GPU but getting the following exception:
System.EntryPointNotFoundException: 'Unable to find an entry point named 'OrtSessionOptionsAppendExecutionProvider_CUDA' in DLL 'onnxruntime'.'
Based on the documentation the ApplyOnnxModel method provides an overload for GPU:
public static Microsoft.ML.Transforms.Onnx.OnnxScoringEstimator ApplyOnnxModel (this Microsoft.ML.TransformsCatalog catalog, string modelFile, Nullable gpuDeviceId = null, bool fallbackToCpu = false);
The model works perfectly fine when using the CPU, but when the gpuDeviceID and fallbackToCpu parameters are provided it fails with the above execption. I understand that for the OnnxRuntime to support GPU you need to also install the ML.OnnxRuntime.GPU nuget package but i noticed that ML.OnnxTransformer has a dependency with ML.OnnxRuntime 0.5.1.
Enviroment:
OS Platform: Windows 10 professional build 1809
ONNX Runtime: 1.4.0 and has dependency with ML.OnnxRuntime 0.5.1, tried ML.OnnxRuntime.GPU 1.0.0 but need OnnxTransformer.
Platform: .NET Core 3.0
Visual Studio version (if applicable): 2019
CUDA/cuDNN version: CUDA 10.0.130, cuDNN 7.6.0.34
GPU model and memory: NVidia Quadro P1000
Failing code:
var pipeline = mlContext.Transforms.ResizeImages(resizing: ImageResizingEstimator.ResizingKind.Fill, outputColumnName: onnxModel.ModelInput, imageWidth: ImageSettings.imageWidth, imageHeight: ImageSettings.imageHeight, inputColumnName: nameof(ImageInputData.Image))
.Append(mlContext.Transforms.ExtractPixels(outputColumnName: onnxModel.ModelInput))
.Append(mlContext.Transforms.ApplyOnnxModel(onnxModel.ModelOutput, onnxModel.ModelInput, onnxModel.ModelPath, 1, true));