Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rwkv6: add wkv6 support for Vulkan backend #10829

Merged
merged 7 commits into from
Dec 16, 2024
Merged

rwkv6: add wkv6 support for Vulkan backend #10829

merged 7 commits into from
Dec 16, 2024

Conversation

uniartisan
Copy link
Contributor

Make sure to read the contributing guidelines before submitting a PR

This PR aims to enhance support for RWKV6 models, it enables wkv6 at Vulkan backend.

uniartisan and others added 4 commits December 13, 2024 17:15
@uniartisan @MollySophia
rwkv_wkv6 vulkan shader
4651f5e
@MollySophia
RWKV_WKV6 Vulkan op tests passed
77fe4fd 
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
@MollySophia
Apply code format changes
60bbd4e 
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
@uniartisan
Merge branch 'master' into vulkan
64c16c4
@github-actions github-actions bot added Vulkan Issues specific to the Vulkan backend ggml changes relating to the ggml tensor library for machine learning labels Dec 14, 2024
@0cc4m 0cc4m self-requested a review December 15, 2024 11:27
@uniartisan
Copy link
Contributor Author

image
image
update with tests :)

jeffbolznv
jeffbolznv reviewed Dec 16, 2024
View reviewed changes
Copy link
Collaborator

@jeffbolznv jeffbolznv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with rwkv, but the shader code appears to correctly match CUDA.

ggml/src/ggml-vulkan/ggml-vulkan.cpp Show resolved Hide resolved
ggml/src/ggml-vulkan/ggml-vulkan.cpp Outdated Show resolved Hide resolved
ggml/src/ggml-vulkan/ggml-vulkan.cpp Outdated Show resolved Hide resolved
ggml/src/ggml-vulkan/vulkan-shaders/wkv6.comp Outdated Show resolved Hide resolved
ggml/src/ggml-vulkan/vulkan-shaders/wkv6.comp Outdated Show resolved Hide resolved
@jeffbolznv
Copy link
Collaborator

Thanks for the contribution. I'll try to test this locally tomorrow.

jeffbolznv
jeffbolznv approved these changes Dec 16, 2024
View reviewed changes
Copy link
Collaborator

@jeffbolznv jeffbolznv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes. LGTM. I ran the backend tests on RTX 4070 and they passed.

0cc4m
0cc4m approved these changes Dec 16, 2024
View reviewed changes
Copy link
Collaborator

@0cc4m 0cc4m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, great work!

@0cc4m 0cc4m merged commit 160bc03 into ggerganov:master Dec 16, 2024
48 checks passed
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Dec 20, 2024
@uniartisan @MollySophia @arthw
rwkv6: add wkv6 support for Vulkan backend (ggerganov#10829)
4e32aca 
* rwkv_wkv6 vulkan shader

* RWKV_WKV6 Vulkan op tests passed

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>

* Apply code format changes

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>

* add [[unroll]] and remove unnecessary conditions

* add uma support

* fix erros in EditorConfig Checker

---------

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
Co-authored-by: Molly Sophia <mollysophia379@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MollySophia MollySophia MollySophia left review comments

@0cc4m 0cc4m 0cc4m approved these changes

@jeffbolznv jeffbolznv jeffbolznv approved these changes

Assignees
No one assigned
Labels
ggml changes relating to the ggml tensor library for machine learning Vulkan Issues specific to the Vulkan backend
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@uniartisan @jeffbolznv @0cc4m @MollySophia