koboldcpp_hipblas.dll / koboldcpp_vulkan.dll Not Found?

[aeternitas97]

I'm getting an unhandled exception on v1.104.yr0 when attempting to load a model. This issue occurs for both the standard and "b2" executable.

I can set whatever settings I like. Upon attempting to load a model using either the Vulkan or HIP Blas backends, I get the following errors followed by a crash.

When using HIP Blas:

Loading Chat Completions Adapter: C:\Users\csman\AppData\Local\Temp\_MEI29962\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
Auto Recommended GPU Layers: 39
System: Windows 10.0.26100 AMD64 AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Detected Available GPU Memory: 20464 MB
Detected Available RAM: 48735 MB
Initializing dynamic library: koboldcpp_vulkan.dll
Traceback (most recent call last):
  File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in __init__
  File "ctypes\__init__.py", line 374, in __init__
FileNotFoundError: Could not find module 'C:\Users\csman\AppData\Local\Temp\_MEI29962\koboldcpp_vulkan.dll' (or one of its dependencies). Try using the full path with constructor syntax.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "koboldcpp.py", line 8936, in <module>
  File "koboldcpp.py", line 7910, in main
  File "koboldcpp.py", line 8305, in kcpp_main_process
  File "koboldcpp.py", line 606, in init_library
  File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in __init__
pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Users\\csman\\AppData\\Local\\Temp\\_MEI29962\\koboldcpp_vulkan.dll'. Most likely this dynlib/dll was not found when the application was frozen.
[3212] Failed to execute script 'koboldcpp' due to unhandled exception!

When using Vulkan:

Loading Chat Completions Adapter: C:\Users\csman\AppData\Local\Temp\_MEI29962\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
Auto Recommended GPU Layers: 39
System: Windows 10.0.26100 AMD64 AMD64 Family 26 Model 68 Stepping 0, AuthenticAMD
Detected Available GPU Memory: 20464 MB
Detected Available RAM: 48735 MB
Initializing dynamic library: koboldcpp_vulkan.dll
Traceback (most recent call last):
  File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in __init__
  File "ctypes\__init__.py", line 374, in __init__
FileNotFoundError: Could not find module 'C:\Users\csman\AppData\Local\Temp\_MEI29962\koboldcpp_vulkan.dll' (or one of its dependencies). Try using the full path with constructor syntax.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "koboldcpp.py", line 8936, in <module>
  File "koboldcpp.py", line 7910, in main
  File "koboldcpp.py", line 8305, in kcpp_main_process
  File "koboldcpp.py", line 606, in init_library
  File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in __init__
pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Users\\csman\\AppData\\Local\\Temp\\_MEI29962\\koboldcpp_vulkan.dll'. Most likely this dynlib/dll was not found when the application was frozen.
[3212] Failed to execute script 'koboldcpp' due to unhandled exception!

[Carrot-Jesus]

I have the same issue. Additionally, my GPU only shows up for Vulkan. I cannot test on any other formats. The GGUF I'm trying to run is MythoMax-L2-13b. I am running an AMD Radeon RX 7900 XTX GPU with 24GB of vRAM and an AMD Ryzen 7 9800X3D CPU.

[edreces]

same issue here too, it doesn't matter what model or backend, it will immediately result in a crash, same errors as well, i have an rx 6950 xt

[aeternitas97]

Not sure if this helps anyone, but I was able to build my own .exe from current source code and it worked just fine. The current release still has the issue I originally described above.

As a warning to anyone trying to build from source, the instructions are not up to date and it will require some tinkering on your end. The final .exe is HIP only, I haven't taken the time to figure out how to build with the other backends (Vulkan, CLBlast, etc.).

[kingslycamben]

Same issue here, I have a 6600xt.