Skip to content

Commit

Permalink
Update llama_cpp_quickstart.md (#10663)
Browse files Browse the repository at this point in the history
  • Loading branch information
yangw1234 authored Apr 4, 2024
1 parent 6699d86 commit ac65ab6
Showing 1 changed file with 3 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Visit the [Install IPEX-LLM on Linux with Intel GPU](https://ipex-llm.readthedoc
#### Windows
Visit the [Install IPEX-LLM on Windows with Intel GPU Guide](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html), and follow [Install Prerequisites](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html#install-prerequisites) to install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) Community Edition, latest [GPU driver](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) and Intel® oneAPI Base Toolkit 2024.0.

**Note**: IPEX-LLM backend only supports the more recent GPU drivers. Please make sure your GPU driver version is equal or newer than `31.0.101.5333`, otherwise you might find gibberish output.

### 1 Install IPEX-LLM for llama.cpp

To use `llama.cpp` with IPEX-LLM, first ensure that `ipex-llm[cpp]` is installed.
Expand Down Expand Up @@ -259,3 +261,4 @@ Log end

#### Fail to quantize model
If you encounter `main: failed to quantize model from xxx`, please make sure you have created related output directory.

0 comments on commit ac65ab6

Please sign in to comment.