Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to CMakePresets.json to add ninja clang target on windows #10668

Merged

Conversation

Srihari-mcw
Copy link
Contributor

This PR adds changes to CMakePresets.json to add a ninja clang target on windows. This will help users to build llama.cpp with clang and ninja dependencies, with relative ease. The additional dependencies are listed in build.md

Build commands :

    cmake --preset x64-windows-ninja-release
    cmake --build build-x64-windows-ninja-release

Experiments were conducted to test llama.cpp with MSVC (Visual Studio Generator), Clang (Ninja) and Clang-Cl (Visual Studio Generator) and to compare the performance

Best performance was seen among the three for clang with ninja generator

Ninja Generator, Clang.exe/Clang++.exe (17.0.3)

  • Clang.exe and clang++.exe are obtained from the same path as clang-cl - C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\x64\bin
  • CMAKE Command : cmake -G "Ninja" -S . -B Windows-build/ -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang.exe -DCMAKE_CXX_COMPILER=clang++.exe
  • Ninja path : C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe and version (1.11.0)
  • Additional library depedencies are resolved with : set LIB=C:\Program Files (x86)\Windows Kits\10\Lib\10.0.22621.0\um\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\lib\x64\uwp;C:\Program Files (x86)\Windows Kits\10\Lib\10.0.22621.0\ucrt\x64 before cmake and build

Both the clang builds are done without the openmp dependencies

The tests were conducted in AMD Raphael 7600X which supports the following flags (The same were kept on in our windows tests) :
AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1|

Prompt Processing

model Compiler/Generator size params backend threads test t/s speedup Commit id
llama 7B Q4_0 MSVC Compiler/ VS Generator 3.56 GiB 6.74 B CPU 6 pp 512 48.14 ± 0.67 231f936
llama 7B Q4_0 clang-cl.exe/ VS Generator 3.56 GiB 6.74 B CPU 6 pp 512 50.23 ± 0.91 4.33% 231f936
llama 7B Q4_0 clang.exe/ Ninja Generator 3.56 GiB 6.74 B CPU 6 pp 512 55.69 ± 0.71 15.68% 231f936

Text Generation

model Compiler/Generator size params backend threads test t/s speedup Commit id
llama 7B Q4_0 MSVC Compiler/ VS Generator 3.56 GiB 6.74 B CPU 6 tg 128 14.8 ± 0.01 231f936
llama 7B Q4_0 clang-cl.exe/ VS Generator 3.56 GiB 6.74 B CPU 6 tg 128 14.95 ± 0.02 1.00% 231f936
llama 7B Q4_0 clang.exe/ Ninja Generator 3.56 GiB 6.74 B CPU 6 tg 128 14.95 ± 0.02 1.00% 231f936

OS : Windows

Tests done with Meta Llama2 7B model

@github-actions github-actions bot added documentation Improvements or additions to documentation build Compilation issues labels Dec 5, 2024
@max-krasnyansky
Copy link
Collaborator

max-krasnyansky commented Dec 5, 2024

Do you want to maybe just match what I did for the Windows on ARM64 arm64-windows-llvm ?
i.e add CMake toolchain file that uses llvm/clang and call the targets x64-windows-llvm that use Ninja.
The main performance difference is llvm/clang not Ninja.
We should just use Ninja as the default anyway. It's faster in general and cleaner output.

@Srihari-mcw
Copy link
Contributor Author

Hi @max-krasnyansky , the preset target names were changed as per the above comment. The aim is to facilitate users to build with clang on x64/x86 Platforms on windows. Thanks

Copy link
Collaborator

@max-krasnyansky max-krasnyansky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Srihari-mcw
Sorry. I approved first but then realized that there is still no toolchain file.
Just copy cmake/arm64-windows-llvm.cmake to cmake/x64-windows-llvm.cmake and update CFLAGS and things.
We probably don't need to use .exe in the compiler name either.

@Srihari-mcw
Copy link
Contributor Author

HI @max-krasnyansky ,

updates were made to use .cmake file. Thanks

docs/build.md Outdated Show resolved Hide resolved
docs/build.md Outdated Show resolved Hide resolved
@max-krasnyansky max-krasnyansky merged commit c37fb4c into ggerganov:master Dec 9, 2024
2 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
…erganov#10668)

* Update cmakepreset.json to use clang with ninja by default

* Update cmakepreset.json to add clang and ninja based configs

* Updates to build.md file

* Make updates to rename preset targets

* Update with .cmake file

* Remove additional whitespaces

* Add .cmake file for x64-windows-llvm

* Update docs/build.md

* Update docs/build.md

---------

Co-authored-by: Max Krasnyansky <max.krasnyansky@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Compilation issues documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants