Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert.py : Update to support 70B HF format model files #2427

Merged
merged 3 commits into from
Jul 27, 2023
Merged

convert.py : Update to support 70B HF format model files #2427

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e15a67d
convert.py : fix llama 2 70b conversion from Huggingface
mj-shifu Jul 27, 2023
01d16e1
convert.py : fix of type and shorter code
mj-shifu Jul 27, 2023
9442c34
convert.py : shorten and simplify permute
mj-shifu Jul 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
convert.py : shorten and simplify permute 
* idea from @KerfuffleV2
  • Loading branch information
@mj-shifu
mj-shifu committed Jul 27, 2023
commit 9442c34f4960586f745245f3dd2782b3e4fb8129
13 changes: 5 additions & 8 deletions convert.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,14 +323,11 @@ def __repr__(self) -> str:


def permute(weights: NDArray, n_head: int, n_kv_head: Optional[int] = None) -> NDArray:
if n_kv_head is None or n_head == n_kv_head:
return (weights.reshape(n_head, 2, weights.shape[0] // n_head // 2, *weights.shape[1:])
.swapaxes(1, 2)
.reshape(weights.shape))
else:
return (weights.reshape(n_head // n_kv_head, 2, weights.shape[0] * n_kv_head // n_head // 2, *weights.shape[1:])
.swapaxes(1, 2)
.reshape(weights.shape))
if n_kv_head is not None and n_head != n_kv_head:
n_head //= n_kv_head
return (weights.reshape(n_head, 2, weights.shape[0] // n_head // 2, *weights.shape[1:])
.swapaxes(1, 2)
.reshape(weights.shape))


def dequantize_q4(qvalues_pack32: NDArray, scales: NDArray, addends: Optional[NDArray], g_idx: Optional[NDArray]) -> NDArray:
Expand Down