Make hipBLAS CMake more similar to cuBLAS

[ardfork]

Currently hipBLAS section in CMake use a legacy workflow, the recommended one is similar to the one used by llama.cpp for cuBLAS. I copied llama.cpp cuBLAS section and adapted it to HIP.

I chose to set default architecture to native to avoid user having to compile for a lot of architectures. This supersede #4011.

[Tungsten842]

This breaks the nix build for some reason:

llama.cpp> -- Detecting HIP compiler ABI info
llama.cpp> -- Detecting HIP compiler ABI info - failed
llama.cpp> -- Check for working HIP compiler: /nix/store/dh8079rsawnih4rpmkchn100s9s5pdih-rocm-llvm-clang-wrapper-5.7.1/bin/clang++
llama.cpp> -- Check for working HIP compiler: /nix/store/dh8079rsawnih4rpmkchn100s9s5pdih-rocm-llvm-clang-wrapper-5.7.1/bin/clang++ - broken
llama.cpp> CMake Error at /nix/store/5h0akwq4cwlc3yp92i84nfgcxpv5xv79-cmake-3.26.4/share/cmake-3.26/Modules/CMakeTestHIPCompiler.cmake:63 (message):
llama.cpp>   The HIP compiler
llama.cpp>     "/nix/store/dh8079rsawnih4rpmkchn100s9s5pdih-rocm-llvm-clang-wrapper-5.7.1/bin/clang++"
llama.cpp>   is not able to compile a simple test program.
llama.cpp>   It fails with the following output:
llama.cpp>     Change Dir: /build/dclh6ll2xxq8sn0zmq5dav6b89d0vffa-source/build/CMakeFiles/CMakeScratch/TryCompile-J52vdW
llama.cpp>     Run Build Command(s):/nix/store/jyk3j4632ldxsikjq0syjlj16i12jfsx-ninja-1.11.1/bin/ninja -v cmTC_27756 && [1/2] /nix/store/dh8079rsawnih4rpmkchn100s9s5pdih-rocm-llvm-clang-wrapper-5.7.1/bin/clang++ -D__HIP_ROCclr__=1 -I/nix/store/lm7mbxnw9bfsarhy3dvcpay7z9rw0mrj-clr-5.7.1/include --cuda-host-only  --offload-arch=native -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -o CMakeFiles/cmTC_27756.dir/testHIPCompiler.hip.o  -c /build/dclh6ll2xxq8sn0zmq5dav6b89d0vffa-source/build/CMakeFiles/CMakeScratch/TryCompile-J52vdW/testHIPCompiler.hip
llama.cpp>     FAILED: CMakeFiles/cmTC_27756.dir/testHIPCompiler.hip.o
llama.cpp>     /nix/store/dh8079rsawnih4rpmkchn100s9s5pdih-rocm-llvm-clang-wrapper-5.7.1/bin/clang++ -D__HIP_ROCclr__=1 -I/nix/store/lm7mbxnw9bfsarhy3dvcpay7z9rw0mrj-clr-5.7.1/include --cuda-host-only  --offload-arch=native -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -o CMakeFiles/cmTC_27756.dir/testHIPCompiler.hip.o  -c /build/dclh6ll2xxq8sn0zmq5dav6b89d0vffa-source/build/CMakeFiles/CMakeScratch/TryCompile-J52vdW/testHIPCompiler.hip
llama.cpp>     clang++: error: cannot determine amdgcn architecture: /nix/store/8rfdqi6c2ag8s7h93kmpgas22kwnqbq5-rocm-llvm-clang-5.7.1/bin/amdgpu-arch: ; consider passing it via '--offload-arch'
llama.cpp>     clang++: error: cannot find HIP runtime; provide its path via '--rocm-path', or pass '-nogpuinc' to build without HIP runtime
llama.cpp>     ninja: build stopped: subcommand failed.
llama.cpp>   CMake will not be able to correctly generate this project.
llama.cpp> Call Stack (most recent call first):
llama.cpp>   CMakeLists.txt:365 (enable_language)
llama.cpp>
llama.cpp> -- Configuring incomplete, errors occurred!

This is currently built like this:
https://github.com/ggerganov/llama.cpp/blob/d96ca7ded77df764db797b68b4a29e34c5b56285/flake.nix#L100-L106

[ardfork]

Targets are chosen with CMAKE_HIP_ARCHITECTURES when using HIP as a language in CMake. Also, you don't need to specify CMAKE_C_COMPILER or CMAKE_CXX_COMPILER, if you still want to change compiler used for HIP, you can change CMAKE_HIP_COMPILER. Overall, I think below should work.

 "-DLLAMA_HIPBLAS=1" 
 # Build all targets supported by rocBLAS. When updating search for TARGET_LIST_ROCM 
 # in github.com/ROCmSoftwarePlatform/rocBLAS/blob/develop/CMakeLists.txt 
 # and select the line that matches the current nixpkgs version of rocBLAS. 
 "-DCMAKE_HIP_ARCHITECTURES=gfx803;gfx900;gfx906:xnack-;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102" 

[Tungsten842]

Targets are chosen with CMAKE_HIP_ARCHITECTURES when using HIP as a language in CMake. Also, you don't need to specify CMAKE_C_COMPILER or CMAKE_CXX_COMPILER, if you still want to change compiler used for HIP, you can change CMAKE_HIP_COMPILER. Overall, I think below should work.

 "-DLLAMA_HIPBLAS=1" 
 # Build all targets supported by rocBLAS. When updating search for TARGET_LIST_ROCM 
 # in github.com/ROCmSoftwarePlatform/rocBLAS/blob/develop/CMakeLists.txt 
 # and select the line that matches the current nixpkgs version of rocBLAS. 
 "-DCMAKE_HIP_ARCHITECTURES=gfx803;gfx900;gfx906:xnack-;gfx908:xnack-;gfx90a:xnack+;gfx90a:xnack-;gfx940;gfx941;gfx942;gfx1010;gfx1012;gfx1030;gfx1100;gfx1101;gfx1102" 

By building it like you suggested I get the same error as before, If i try to override the compiler with CMAKE_HIP_COMPILER it says that hipcc is not supported.
In nixos the hip runtime is not in /opt/rocm, clang doesn't find it by default and hen hipcc was used --rocm-path was automaticaly generated when calling clang. I belive that either hipcc is fixed or there should be a to change the rocm-path from cmake.

[YellowRoseCx]

the AND ${rocblas_FOUND}) section could/would potentially break the build for people who are using hipBLAS but not using AMD rocblas

[GZGavinZhao]

I have a similar PR #4813 that does basically the same thing but it has conflicts resolved and I've tried to add Nix support. Do we perhaps want to consolidate the effort into #4813? If so, can anyone guide me as to how I am supposed to test the Nix build?

[Tungsten842]

I have a similar PR #4813 that does basically the same thing but it has conflicts resolved and I've tried to add Nix support. Do we perhaps want to consolidate the effort into #4813? If so, can anyone guide me as to how I am supposed to test the Nix build?

First you install the nix package manager, after that you enable flakes and then you can just go inside the project directory and run nix build .#rocm.
Anyway I am getting a very similar error trying to build you branch.

    Change Dir: '/build/source/build/CMakeFiles/CMakeScratch/TryCompile-VKcJVX'

    Run Build Command(s): /nix/store/7p8kahc23klw4lfx1d7i5gs2547mbcj5-ninja-1.11.1/bin/ninja -v cmTC_3c0e1
    [1/2] /nix/store/6xj537dwafp618f084hkl4gvlf5sfqqa-rocm-llvm-clang-wrapper-5.7.1/bin/clang -D__HIP_ROCclr__=1 -I/nix/store>
    FAILED: CMakeFiles/cmTC_3c0e1.dir/testHIPCompiler.hip.o
    /nix/store/6xj537dwafp618f084hkl4gvlf5sfqqa-rocm-llvm-clang-wrapper-5.7.1/bin/clang -D__HIP_ROCclr__=1 -I/nix/store/68xb3>
    clang: error: cannot find HIP runtime; provide its path via '--rocm-path', or pass '-nogpuinc' to build without HIP runti>
    ninja: build stopped: subcommand failed.

[GZGavinZhao]

It needs extra environment variables:

diff --git a/.devops/nix/package.nix b/.devops/nix/package.nix
index 10c359b..0804e75 100644
--- a/.devops/nix/package.nix
+++ b/.devops/nix/package.nix
@@ -190,6 +190,11 @@ effectiveStdenv.mkDerivation (
       ++ optionals useMetalKit [ (lib.cmakeFeature "CMAKE_C_FLAGS" "-D__ARM_FEATURE_DOTPROD=1") ]
       ++ optionals useBlas [ (lib.cmakeFeature "LLAMA_BLAS_VENDOR" "OpenBLAS") ];
 
+    env = {
+      ROCM_PATH = "${rocmPackages.clr}";
+      HIP_DEVICE_LIB_PATH = "${rocmPackages.rocm-device-libs}/amdgcn/bitcode";
+    };
+
     # TODO(SomeoneSerge): It's better to add proper install targets at the CMake level,
     # if they haven't been added yet.
     postInstall = ''

Edit: I built it successfully on my end. Updated my branch in #4813 accordingly. Please see if you still get errors.

[ggerganov]

Some recent changes overlap with this PR. Open a new one if there is still stuff that you think should be merged