Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A "SYCL error" happened when load Phi3 model to Intel GPU #11924

Open
soulyet opened this issue Aug 26, 2024 · 2 comments
Open

A "SYCL error" happened when load Phi3 model to Intel GPU #11924

soulyet opened this issue Aug 26, 2024 · 2 comments
Assignees

Comments

@soulyet
Copy link

soulyet commented Aug 26, 2024

We follow the guideline to setup IPEX-LLM[CPP] and also Intel OneAPI. but when we load Phi3 model to Intel GPU, it always have "SYCL error" happened. and we also use main.exe to load model manually, same error.
I attached ollama and llama.cpp log files:
llama.cpp_log.txt
ollama_log.txt

I'm not sure is there is any environment issue for my setup.

@rnwang04
Copy link
Contributor

Hi @soulyet ,
Q4_K has some error at your current machine, we may try to fix it later.
But for now, I think you can try Q4_0 first, which should work.

@soulyet
Copy link
Author

soulyet commented Aug 28, 2024

Hi @soulyet , Q4_K has some error at your current machine, we may try to fix it later. But for now, I think you can try Q4_0 first, which should work.

Thanks for reply.
Now I use Q4_0 which download from https://huggingface.co/SanctumAI/Phi-3-mini-4k-instruct-GGUF/tree/main for try, but met another error:
GGML_ASSERT: C:\Users\Administrator\actions-runner\cpp-release_work\llm.cpp\llm.cpp\ollama-internal\llm\llama.cpp\ggml-backend.c:100: base != NULL && "backend buffer base cannot be NULL"

log detail:
llama_log.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants