Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fused MOE for Mixtral #2542

Merged
merged 23 commits into from
Jan 30, 2024
Merged

Fused MOE for Mixtral #2542

Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
c33e37a
Fused MOE for Mixtral
pcmoritz Jan 22, 2024
03f5dc8
fix weight loading
pcmoritz Jan 22, 2024
2867f34
lint and cleanup
pcmoritz Jan 22, 2024
a82f73f
style
pcmoritz Jan 22, 2024
406a188
yapf
pcmoritz Jan 22, 2024
707479d
update comment
pcmoritz Jan 22, 2024
649287e
Merge branch 'main' into fused-mixtral
pcmoritz Jan 22, 2024
f283c98
update fused_moe implementation
pcmoritz Jan 23, 2024
adeb80f
optimize block sizes
pcmoritz Jan 24, 2024
d6a0b11
Merge branch 'main' into fused-mixtral
pcmoritz Jan 26, 2024
74308ba
Merge branch 'main' into fused-mixtral
pcmoritz Jan 28, 2024
23284e3
add dtype
pcmoritz Jan 29, 2024
429f51c
Merge branch 'main' into fused-mixtral
pcmoritz Jan 29, 2024
a1f0107
update
pcmoritz Jan 29, 2024
f1691ba
Merge branch 'fused-mixtral' of https://github.com/pcmoritz/vllm-publ…
pcmoritz Jan 29, 2024
ee0a490
Merge branch 'main' into fused-mixtral
WoosukKwon Jan 30, 2024
a77a2c1
Resolve merge error
WoosukKwon Jan 30, 2024
2233c2a
remove extra function
pcmoritz Jan 30, 2024
44cc7d0
update
pcmoritz Jan 30, 2024
7b076aa
move mixtral moe layer back
pcmoritz Jan 30, 2024
b4cb78c
put import back
pcmoritz Jan 30, 2024
111c1b5
yapf
pcmoritz Jan 30, 2024
d30e844
fix typo
pcmoritz Jan 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fix typo
  • Loading branch information
@pcmoritz
pcmoritz committed Jan 30, 2024
commit d30e844d46ea60805b1e8cbf0b096e256db2c7d1
1 change: 0 additions & 1 deletion vllm/model_executor/models/mixtral.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,6 @@ def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
self.w2s,
routing_weights,
selected_experts,
routing_weights,
inplace=True)

final_hidden_states = tensor_model_parallel_all_reduce(
Expand Down
Loading