Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add ipex moe implementation to support Mixtral and PhiMoe #2707

Merged
merged 8 commits into from
Nov 18, 2024
Merged

add ipex moe implementation to support Mixtral and PhiMoe #2707

merged 8 commits into from
Nov 18, 2024

Conversation

sywangyi
Copy link
Contributor

@sywangyi
add ipex moe implementation to support Mixtral and PhiMoe
d9a8bbc 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
@sywangyi sywangyi marked this pull request as draft October 30, 2024 07:04
@sywangyi
Copy link
Contributor Author

need xpu ipex to upgrade to 2.5 as well. or else it will block tgi-xpu

@Narsil
Copy link
Collaborator

Narsil commented Nov 4, 2024

This is still marked as draft, should we review/merge or not ? The code change looks Ok.

@sywangyi
update to ipex xpu 2.5
1108051 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
@sywangyi
Copy link
Contributor Author

sywangyi commented Nov 4, 2024

It also need the upgrade of xpu ipex too. once ready, I will mark it ready for review. thanks

@sywangyi
Merge branch 'main' into moe
3d4c50f
@sywangyi
torch has xpu support in 2.5
6becab5 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
@sywangyi sywangyi marked this pull request as ready for review November 5, 2024 02:07
@sywangyi
Copy link
Contributor Author

sywangyi commented Nov 5, 2024

@Narsil please help review the PR. thanks

@sywangyi
fix oneapi basekit version
1639152 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
@sywangyi
Copy link
Contributor Author

@Narsil could you review it. the PR also upgrade xpu ipex which means to support AWQ and GPTQ

@sywangyi
Copy link
Contributor Author

@danieldk could you help?

@danieldk
Copy link
Member

Looking at it now!

@danieldk danieldk mentioned this pull request Nov 18, 2024
Closed
5 tasks
danieldk
danieldk reviewed Nov 18, 2024
View reviewed changes
Copy link
Member

@danieldk danieldk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it's nice to have fused MoE on ipex. I added some comments.

Dockerfile_intel Outdated Show resolved Hide resolved
server/text_generation_server/layers/moe/__init__.py Outdated Show resolved Hide resolved
server/text_generation_server/layers/moe/__init__.py Show resolved Hide resolved
@sywangyi @danieldk
Apply suggestions from code review
53b6f6e 
Co-authored-by: Daniël de Kok <me@github.danieldk.eu>
@danieldk danieldk merged commit a5ecd6e into huggingface:main Nov 18, 2024
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danieldk danieldk danieldk approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sywangyi @Narsil @danieldk