[BUG] Failed to load .onxx model

[trihardseven]

Bug Description

Fails to load model when trying to gen tags with wd3

Environment Details

• Operating System: W11
• Application Version: 2.9.8
• CUDA Version: 13.1
• 5090 (laptop) driver v591
• Python 3.11.9
• .NET Desktop Runtime 9.0.12
• Visual C++ v14

Additional Context

Exception Details
=================
Date and Time: 2026-01-21 08:01:47
Source: SmartData.Lib
Message: Failed to load model from D:\D AI\AI Tools\DatasetHelper\models\wdV3Model.onnx.
Help Link: 
HResult: -2146233079

Inner Exception Details
======================
Source: Microsoft.ML.OnnxRuntime
Message: [ErrorCode:Fail] OrtSessionOptionsAppendExecutionProvider_Cuda: Failed to load shared library
Help Link: 
HResult: -2146233088

Stack Trace
============
   at SmartData.Lib.Services.Base.BaseAIConsumer`2.LoadModelAsync() in E:\dev\DatasetHelpers\SmartData.Lib\Services\Base\BaseAIConsumer.cs:line 94
   at SmartData.Lib.Services.Base.BaseAutoTaggerService`2.LoadModelAsync() in E:\dev\DatasetHelpers\SmartData.Lib\Services\Base\BaseAutoTaggerService.cs:line 63
   at SmartData.Lib.Services.Base.BaseAutoTaggerService`2.GenerateTagsAndKeepRedundant(String inputPath, String outputPath, Boolean appendToFile, Boolean weightedCaptions) in E:\dev\DatasetHelpers\SmartData.Lib\Services\Base\BaseAutoTaggerService.cs:line 159
   at DatasetProcessor.ViewModels.GenerateTagsViewModel.CallAutoTaggerService(IAutoTaggerService autoTagger) in E:\dev\DatasetHelpers\DatasetProcessor\ViewModels\GenerateTagsViewModel.cs:line 229
   at DatasetProcessor.ViewModels.GenerateTagsViewModel.MakePredictionsAsync() in E:\dev\DatasetHelpers\DatasetProcessor\ViewModels\GenerateTagsViewModel.cs:line 170

Target Site
============
Declaring Type: SmartData.Lib.Services.Base.BaseAIConsumer`2+<LoadModelAsync>d__14[TInput,TOutput]
Method Name: MoveNext

Additional Information
======================

[trihardseven]

Tried the previous version of the program and it works fine so I'm guessing it's a problem with the onxxruntime changes

[Particle1904]

The stack trace means ONNX Runtime could not initialize CUDA at all.
I also have a 50 series currently and I tested all features related to running models because I cleaned up a lot of code related to those in this release.

I believe the issue is CUDA 13.1, even though an earlier version of the app worked, it most likely ran in CPU while silently failing when trying to append CUDA as an execution provider.

OnnxRuntime CUDA requirements

I have CUDA 12.8 installed currently on my machine; I started with 13.0 and quickly saw that there's almost 0 support for Blackwell cards (specifically in Windows) in most software out there (SageAttention, Flash-Attn, pytorch, etc). I suggest you downgrade and see if that works.

[trihardseven]

You're probably right about it running on CPU in the older version, compared to tagging with the webui extension it takes ten times longer using the same model.

I tried downgrading CUDA to 12.8 but I still get the exact same error

[Particle1904]

I'll have to investigate then, I honestly I'm a bit clueless about what the issue might be.

First things first, the v2.9.7 and v2.9.8 are 6+ months apart so its hard for me to pinpoint exactly what I changed. I'm guessing it doesn't even run for you, right?
I'll be releasing v2.9.9 in a few minutes, which carries out a few fixes unrelated to this issue and it should just fall back to CPU for now and run.

I did a bit of research and basically the Dataset Helpers is properly finding CUDA (it sees that a CUDA enabled GPU is available in the system) to use it as an Execution Provider inside the Onnx Runtime, the issue is that its failing to find a shared .dll, but it doesn't really say which one. It could be a .dll like or cudart64_12.dll or cudnn*.dll. It could be that these .dll files are missing from your system PATH.

If you are feeling like helping me figure out what is going on, I'll leave some commands that you can run from a PowerShell window to check a few things:

Command 1: Check CUDA Toolkit Installation
nvcc --version; Get-Command nvcc | Select-Object Source

Command 2: Check for the DLLs
where.exe cudart64_12.dll cudnn64_9.dll

Command 3: List files in the CUDA bin directory
Get-ChildItem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8\bin" -Filter *.dll | Select-Object Name