Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml : update ggml_backend_cpu_device_supports_op #10867

Merged
merged 3 commits into from
Dec 17, 2024
Merged

Conversation

ggerganov
Copy link
Owner

This fixes the following failures:

./build/bin/test-backend-ops -b CPU -o CPY

  CPY(type_src=f16,type_dst=q6_K,ne=[256,4,4,4],permute=[0,0,0,0]): OK
  CPY(type_src=f16,type_dst=q6_K,ne=[256,2,3,4],permute=[0,2,1,3]): OK
  CPY(type_src=f16,type_dst=iq2_xxs,ne=[256,4,4,4],permute=[0,0,0,0]): not supported [CPU] not supported [CPU] 
  CPY(type_src=f16,type_dst=iq2_xxs,ne=[256,2,3,4],permute=[0,2,1,3]): not supported [CPU] not supported [CPU] 
  CPY(type_src=f16,type_dst=iq2_xs,ne=[256,4,4,4],permute=[0,0,0,0]): not supported [CPU] not supported [CPU] 
  CPY(type_src=f16,type_dst=iq2_xs,ne=[256,2,3,4],permute=[0,2,1,3]): not supported [CPU] not supported [CPU] 
  CPY(type_src=f16,type_dst=iq2_s,ne=[256,4,4,4],permute=[0,0,0,0]): /llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal error
/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: /llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal error
/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal errorfatal error/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: 
/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal error/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal error
/llama.cpp/ggml/src/ggml-cpu/ggml-cpu.c:2996: fatal errorAbort trap: 6

@github-actions github-actions bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning labels Dec 17, 2024
@ggerganov
Copy link
Owner Author

@slaren Is this a valid fix, or is it better to skip these types in the tests?

@slaren
Copy link
Collaborator

slaren commented Dec 17, 2024

My understanding is that these types are effectively unusable without an imatrix, so they should be entirely disabled for quantizing in ggml_cpy. I think what needs to be fixed is ggml_backend_cpu_device_supports_op.

@ggerganov ggerganov changed the title ggml : fix cpy op for IQ-quants to use reference impl ggml : fix cpy tests requiring i-matrix quantization Dec 17, 2024
@ggerganov ggerganov changed the title ggml : fix cpy tests requiring i-matrix quantization ggml : disable cpy tests requiring i-matrix quantization Dec 17, 2024
@ggerganov ggerganov changed the title ggml : disable cpy tests requiring i-matrix quantization ggml : update ggml_backend_cpu_device_supports_op Dec 17, 2024
@ggerganov ggerganov merged commit 0006f5a into master Dec 17, 2024
51 of 55 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
* ggml : fix cpy op for IQ-quants to use reference impl

ggml-ci

* ggml : disable tests involving i-matrix quantization

* ggml : update ggml_backend_cpu_device_supports_op

ggml-ci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ggml changes relating to the ggml tensor library for machine learning testing Everything test related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants