Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add llama 3.1 rope scaling factors to llama conversion and inference #8676

Merged
merged 6 commits into from
Jul 27, 2024
Merged

Add llama 3.1 rope scaling factors to llama conversion and inference #8676

merged 6 commits into from
Jul 27, 2024
+40 −2

Commits on Jul 26, 2024

  1. Add llama 3.1 rope scaling factors to llama conversion and inference 

    This commit generates the rope factors on conversion and adds them to the resulting model as a tensor. At inference time, these factors are passed to the `ggml_rope_ext` rope oepration, improving results for context windows above 8192
    @jmorganca
    jmorganca committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    e6bacb4 View commit details
    Browse the repository at this point in the history

  2. Update convert_hf_to_gguf.py 

    Co-authored-by: compilade <git@compilade.net>
    @jmorganca @compilade
    jmorganca and compilade committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    24540dd View commit details
    Browse the repository at this point in the history

  3. address comments

    @jmorganca
    jmorganca committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    1a3a1b6 View commit details
    Browse the repository at this point in the history

  4. address comments

    @jmorganca
    jmorganca committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    90fd87d View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2024

  1. Update src/llama.cpp 

    Co-authored-by: compilade <git@compilade.net>
    @jmorganca @compilade
    jmorganca and compilade authored Jul 27, 2024
    Configuration menu
    Copy the full SHA
    e6d5bed View commit details
    Browse the repository at this point in the history

  2. Update convert_hf_to_gguf.py 

    Co-authored-by: compilade <git@compilade.net>
    @jmorganca @compilade
    jmorganca and compilade authored Jul 27, 2024
    Configuration menu
    Copy the full SHA
    658041d View commit details
    Browse the repository at this point in the history