Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Adept Persimmon 8b #3410

Merged
merged 30 commits into from
Oct 7, 2023
Merged

Support Adept Persimmon 8b #3410

merged 30 commits into from
Oct 7, 2023

Conversation

phillip-kravtsov
Copy link
Contributor

@phillip-kravtsov phillip-kravtsov commented Sep 29, 2023 •
edited
Loading

  • Adds Persimmon 8B which is, architecturally, a standard dense transformer with:
    • Q/K layernorm
    • Squared ReLU activations
    • partial RoPE
    • very large vocab size (most unused for text)

To support Partial RoPE & Squared ReLU, this PR adds concat & square kernels for metal.
I've confirmed agreement between the GGML & HF implementation up to tensor values in the last layer.

@phillip-kravtsov
Produces garbage output
7cdc3ea
@phillip-kravtsov
wip: correct tensors up to RoPE
4bcf412
@phillip-kravtsov
correct tensors thru RoPE
c9e1446
@phillip-kravtsov
Correct outputs through masked & softmax'd KQ
d1b40ef
@phillip-kravtsov
fp32 works
db2181a
@phillip-kravtsov
Rename adept->persimmon
3f31799
@phillip-kravtsov
Merge branch 'master' of github.com:phillip-kravtsov/llama.cpp into p…
720503b 
…hillip-kravtsov/support-adept-persimmon-8b
@phillip-kravtsov
Produces correct outputs
d61eed0
@phillip-kravtsov
Merge branch 'master' of github.com:ggerganov/llama.cpp into phillip-…
d0a7143 
…kravtsov/support-adept-persimmon-8b
@phillip-kravtsov
clean up convert scripts
fa92f6e
@phillip-kravtsov
remove printing logic from ggml.c
c28a6c5
@phillip-kravtsov
remove prints from llama.cpp & fix merge
47dcb9f
@phillip-kravtsov
Merge branch 'master' of github.com:ggerganov/llama.cpp into phillip-…
7473773 
…kravtsov/support-adept-persimmon-8b
@phillip-kravtsov
trivial cleanups
d904aff
@phillip-kravtsov
Add offload funcs
ec0ce97
@phillip-kravtsov
update conversion script to directly take adept artifacts rather than…
3db04db 
… .saftensors file
@phillip-kravtsov
Fix norm eps bug
f28f52c
@phillip-kravtsov
Merge branch 'master' of github.com:ggerganov/llama.cpp into phillip-…
d93cf1e 
…kravtsov/support-adept-persimmon-8b
@ggerganov ggerganov added high priority Very important issue model Model specific labels Sep 30, 2023
ggerganov
ggerganov approved these changes Oct 2, 2023
View reviewed changes
ggml-metal.m Outdated Show resolved Hide resolved
ggml-metal.m Outdated Show resolved Hide resolved
ggml-metal.m Show resolved Hide resolved
ggml-metal.m Outdated Show resolved Hide resolved
gguf-py/gguf/gguf.py Outdated Show resolved Hide resolved
gguf-py/gguf/gguf.py Outdated Show resolved Hide resolved
llama.cpp Outdated Show resolved Hide resolved
llama.cpp Outdated Show resolved Hide resolved
llama.cpp Show resolved Hide resolved
@ggerganov
Copy link
Owner

Let's resolve the CI fails and merge

cebtenzzre
cebtenzzre reviewed Oct 5, 2023
View reviewed changes
gguf-py/gguf/gguf.py Outdated Show resolved Hide resolved
@phillip-kravtsov
Merge branch 'master' of github.com:ggerganov/llama.cpp into phillip-…
5d259d3 
…kravtsov/support-adept-persimmon-8b. ggml-ci
@phillip-kravtsov phillip-kravtsov force-pushed the phillip-kravtsov/support-adept-persimmon-8b branch from 92acb44 to 5d259d3 Compare October 5, 2023 18:04
ggerganov
ggerganov reviewed Oct 6, 2023
View reviewed changes
llama.cpp Outdated Show resolved Hide resolved
@ggerganov ggerganov merged commit 0e797c2 into ggerganov:master Oct 7, 2023
35 checks passed
@slaren
Copy link
Collaborator

slaren commented Oct 7, 2023

The switches in llm_load_hparams and llama_build_graph are missing breaks, so it should be using the refact graph. Does this work currently?

@ggerganov
Copy link
Owner

@phillip-kravtsov PTAL at @slaren's comment and fix as necessary

@KerfuffleV2
Copy link
Collaborator

I got tired of seeing the compiler warning and created #3535 (not sure if there are any other issues, haven't had a chance to test it yet).

@phillip-kravtsov
Copy link
Contributor Author

Thanks for the fix @KerfuffleV2 -- that PR should be sufficient.

joelkuiper added a commit to vortext/llama.cpp that referenced this pull request Oct 12, 2023
@joelkuiper
Merge branch 'master' of github.com:ggerganov/llama.cpp into grammar-…
f7b9bf1 
…example

* 'master' of github.com:ggerganov/llama.cpp:
  py : change version of numpy requirement to 1.24.4 (ggerganov#3515)
  quantize : fail fast on write errors (ggerganov#3521)
  metal : support default.metallib load & reuse code for swift package (ggerganov#3522)
  llm : support Adept Persimmon 8B (ggerganov#3410)
  Fix for ggerganov#3454 (ggerganov#3455)
  readme : update models, cuda + ppl instructions (ggerganov#3510)
  server : docs fix default values and add n_probs (ggerganov#3506)
@leo-gan leo-gan mentioned this pull request Oct 18, 2023
Merged
@Galunid Galunid mentioned this pull request Nov 9, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cebtenzzre cebtenzzre cebtenzzre left review comments

@goerch goerch goerch left review comments

@ggerganov ggerganov ggerganov approved these changes

Assignees
No one assigned
Labels
high priority Very important issue model Model specific
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@phillip-kravtsov @ggerganov @slaren @KerfuffleV2 @goerch @cebtenzzre