-
Notifications
You must be signed in to change notification settings - Fork 10.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add OpenCL add kernel #5151
Add OpenCL add kernel #5151
Conversation
I get this error when building with MSVC:
|
MSVC has a limitation on string length? Ugh. That would mean it was right up at the limit before I added the add kernel. I guess I can split it into a second string. |
Yes, it caused issues before. I think that's the reason why several kernels are in different strings already. |
…h limit, disable float16 support due to bad results
No, all of the kernels in separate strings are there for templating reasons. It was pure luck that the huge string stayed in the limit so far. Can you try again? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Works with phi2 for me.
I could find a reference to the string length issue in #3973, but maybe it never got actually fixed. |
@ggerganov OpenCL is the one compute acceleration thing that can run on the oldest computers and phones. At some point, please make a great release which still has good OpenCL support before abandoning it. Of course, it is quite possible to build exotic backends with Cg or GLSL, and keeping OpenCL is a good idea for everywhere there isn't Vulkan support. GGML can 100% run on old GPUs through OpenCL 1.1; All it requires is a polyfill for one function last I checked it. What needs to be done to keep the OpenCL backend healthy? Please suggest. |
I wrote about the OpenCL backend here: #5138 It can be updated similar to the rest of the backends given community enough support. There is no plan to completely abandon it - we just have to accept that at times it can be in a broken state |
* Add OpenCL add kernel * Put add kernel into different string to stay within MSVC string length limit, disable float16 support due to bad results
* Add OpenCL add kernel * Put add kernel into different string to stay within MSVC string length limit, disable float16 support due to bad results
Fixes #5116
I took the simple route and added the GGML_OP_ADD op (by just copying and adapting the mul code). That should keep the OpenCL backend afloat until it gets superseded.