Issue attempting to use onnxruntime-gpu

[0RadicalLarry0]

Following your guide, I was able to build pytorch for gfx1010 after lots of hacking

But now when attempting to use a program with onnxruntime I cannot get anything to work.

I am attempting to use https://github.com/deiteris/voice-changer but onnxruntime-gpu errors with:

2025-11-15 15:51:08,579 INFO     [EmbedderManager] Loading embedder hubert_base
2025-11-15 15:51:08.879150725 [E:onnxruntime:Default, provider_bridge_ort.cc:2251 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1844 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory

2025-11-15 15:51:08.879164405 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1013 CreateExecutionProviderFactoryInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
2025-11-15 15:51:09.067604156 [E:onnxruntime:, inference_session.cc:2544 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/graph/graph_utils.cc:29 int onnxruntime::graph_utils::GetIndexFromName(const onnxruntime::Node&, const std::string&, bool) itr != node_args.end() was false. Attempting to get index by a name which does not exist:InsertedPrecisionFreeCast_/hubert/feature_projection/layer_norm/Constant_output_0for node: /hubert/feature_projection/layer_norm/Mul/SimplifiedLayerNormFusion/

2025-11-15 15:51:09,067 ERROR    [RVCr2] Failed to create pipeline.

onnxruntime-rocm doesn't work just as you said it would. I'm at a loss as to what to do.

[Efenstor]

I am getting these messages myself in many programs but usually they can be ignored, for example chaiNNer as well as many nodes is ComfyUI work with ONNX just fine and I can see using the radeontop utility that they actually utilize the GPU. In contrast to onnx-rocm (which unfortunately does not work with gfx1010) onnx-gpu is intended to be used only with NVidia GPUs, so using it with AMD here is sort of a hack. Some programs may take the lack of NVidia libraries just as a warning while others consider it critical and fail. I tried to install the NVidia drivers just to provide all the needed libraries but that does not help.

[0RadicalLarry0]

I couldn't get it to work with RVC but I found out I could just convert the onnx models to pytorch and vibe coded some slop to get it to work.

This was the furthest I could get when attempting to run the voice changer after hacking some more fixes for onnxruntime-gpu:

[VoiceChangerManager] Error when binding input: There's no data transfer registered for copying tensors from Device:[DeviceType:1 MemoryType:0 VendorId:4318 DeviceId:0 Alignment:0] to Device:[DeviceType:0 MemoryType:0 VendorId:0 DeviceId:0 Alignment:0]

I also needed these environment variables to improve performance:

export MIOPEN_FIND_MODE=FAST
export PYTORCH_ALLOC_CONF=max_split_size_mb:512

(I don't know if max_split_size_mb:512 is needed but MIOPEN_FIND_MODE=FAST speeds it up a lot)

[Efenstor]

Thanks for MIOPEN_FIND_MODE=FAST, I didn't know about this one.

After some more trying it seems that for me ONNX works only in ComfyUI and not in chaiNNer. Or may be even that is an illusion. That's weird because I clearly remember that it certainly worked, it was fast and I heard characteristic noises from the GPU. I don't know. For now I'll probably remove the section about ONNX from the guide. I have a theory that I had some part of the CUDA stack installed in my system and have uninstalled it since then but this needs investigating.