CUDA Not Detected in NinjaTrader 8 with TorchSharp 0.105.0.0 Despite DLLs Loaded & CPU working

[minewarriorsSchool]

I am experiencing an issue where CUDA is not detected in my NinjaTrader 8 strategy using TorchSharp 0.105.0.0, despite having copied all necessary DLLs to the bin/custom folder and manually loading them. The strategy works fine in a local .NET project, but not in NinjaTrader 8.

Setup

• Local Project: A .NET Framework 4.8 WPF application where CUDA works with torch.cuda.is_available() returning True.
• NinjaTrader 8: A trading platform where the strategy is implemented as a plugin. DLLs are copied to Documents\NinjaTrader 8\bin\Custom.

Steps Taken

1. Copied DLLs: All DLLs from the local project (e.g., torch_cuda.dll, cudnn64_9.dll, cublas64_12.dll, etc., totaling 3.43 GB) were copied to Documents\NinjaTrader 8\bin\Custom.
2. Manual DLL Loading: Used SetDllDirectory to set the DLL directory to bin/custom and manually loaded all DLLs using LoadLibrary. Console output shows successful loading of all DLLs, including torch_cuda.dll.
3. Configuration: Confirmed the process is 64-bit (Is 64-bit process: True), TorchSharp version is 0.105.0.0, and the GPU runs CUDA 12.6 (DLLs are for CUDA 12.1, which should be compatible).

Current Status

• torch.cuda.is_available() returns False, and the device count is 0.
• Attempting to create a tensor on CUDA results in the error: "Torch device type CUDA did not initialise on the current machine. Trace from LoadNativeBackend:"

Console Output Manually logging actions
`

• Loaded saved model from C:\Users\Jelle\Documents\NinjaTrader 8\models\LstmModel.bin
• Set DLL directory to C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom
• Is 64-bit process: True
• TorchSharp version: 0.105.0.0
• Loading DLLs from C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\asmjit.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\c10.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\c10_cuda.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\caffe2_nvrtc.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cublas64_12.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cublasLt64_12.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudart64_12.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_adv64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_cnn64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_engines_precompiled64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_engines_runtime_compiled64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_graph64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_heuristic64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cudnn_ops64_9.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cufft64_11.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cufftw64_11.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cupti64_2023.1.1.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\curand64_10.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cusolver64_11.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cusolverMg64_11.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\cusparse64_12.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\fbgemm.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\Google.Protobuf.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\ICSharpCode.SharpZipLib.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\libiomp5md.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\libiompstubs5md.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\libSkiaSharp.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\LibTorchSharp.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\netstandard.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\NinjaTrader.Custom.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\NinjaTrader.Vendor.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\nvJitLink_120_0.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\nvrtc-builtins64_121.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\nvrtc64_120_0.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\nvToolsExt64_1.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\SkiaSharp.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Buffers.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Memory.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Numerics.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Numerics.Vectors.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Runtime.CompilerServices.Unsafe.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.Threading.Tasks.Extensions.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\System.ValueTuple.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\torch.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\TorchSharp.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\torch_cpu.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\torch_cuda.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\torch_global_deps.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\uv.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\VendorLicensing.dll
• Successfully loaded C:\Users\Jelle\Documents\NinjaTrader 8\bin\Custom\zlibwapi.dll
• CUDA is not available.
• Error creating tensor on CUDA: Torch device type CUDA did not initialise on the current machine. Trace from LoadNativeBackend:
• Using device: cpu - Cuda available: False, device count: 0
  `

Potential Issues

• Version mismatch between TorchSharp and libtorch DLLs (e.g., cuDNN version).
• Interference from manual DLL loading with LoadLibrary.
• Environment configuration issues specific to NinjaTrader 8.

Request
I am seeking help to identify why CUDA fails to initialize in NinjaTrader 8. Specifically:

• Are there known issues with TorchSharp 0.105.0.0 and NinjaTrader 8?
• Could manual DLL loading be causing conflicts? Should I rely on SetDllDirectory or update PATH instead?
• How can I get more detailed error messages from LoadNativeBackend to debug the initialization failure?
• Does copy pasting dll's from compiled project cause issues for CUDA and not CPU? (CPU still works)

Any guidance on debugging this issue or suggestions for resolving it would be greatly appreciated.
Attachments

[minewarriorsSchool]

Extra information, no idea what I have done, but now it is also not working on CPU anymore.
Now it is saying it can not find the cpu-x64 dll.

Which is strange, cause I had that fixed yesterday.
So no idea why it is doing it now again.

[alinpahontu2912]

Hey, I think this happens because of the installed nugget packges. Can you try uninstalling all torch related packages from your project and then only install torchsharp-cpu or torchsharp-cuda ( based on what you need) ?