-
Notifications
You must be signed in to change notification settings - Fork 9.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SYCL] get MAX_MEM_ALLOC from device property #5270
Conversation
can confirm, this fixed #5250 |
@ggerganov @slaren seems the macos building failures are not related, could you give a review? |
I see the fault CI has nothing with changed code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@ggerganov could you add a label called "Intel GPU" like AMD and NV has, then we can quickly response to related issues/PR? |
@NeoZhangJianyu @airMeng As long as PRs make changes only to the SYCL code, you can merge at your discretion |
@ggerganov We have no merge access. Could you assign the access to us? |
You need to accept the collaborator invite |
Yes, we see it. Thank you! :) |
Unfortunately, with this fix, the token generation speed became the same as Vulkan. |
@characharm could you paste the details in #5277, your HW, SW, OS, models and performance numbers and we can see whether the performance are reasonable? |
* get max alloc size from device prop * fix macro typo
* get max alloc size from device prop * fix macro typo
Thanks remindings from @slaren and @0cc4m , fix #5250
Note:
Limited max memory allocation size will cause slight performance regression. If you are using an Intel Data Center GPU like Intel GPU Max series(codename "ponte vecchio"), I will suggest you to follow "Allocations greater than 4GB" to remove the limit.