Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support StarCoder model architectures #3187

Merged
merged 30 commits into from
Sep 15, 2023
Merged

feat: support StarCoder model architectures #3187

merged 30 commits into from
Sep 15, 2023

Commits on Sep 15, 2023

  1. add placeholder of starcoder in gguf / llama.cpp

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    0c5d4d8 View commit details
    Browse the repository at this point in the history

  2. support convert starcoder weights to gguf

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    eb7f0eb View commit details
    Browse the repository at this point in the history

  3. convert MQA to MHA

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    76d32cc View commit details
    Browse the repository at this point in the history

  4. fix ffn_down name

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    7e0a843 View commit details
    Browse the repository at this point in the history

  5. add LLM_ARCH_STARCODER to llama.cpp

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    7298c37 View commit details
    Browse the repository at this point in the history

  6. set head_count_kv = 1

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    166a259 View commit details
    Browse the repository at this point in the history

  7. load starcoder weight

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    57f064d View commit details
    Browse the repository at this point in the history

  8. add max_position_embeddings

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    a17ef39 View commit details
    Browse the repository at this point in the history

  9. set n_positions to max_positioin_embeddings

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    2683611 View commit details
    Browse the repository at this point in the history

  10. properly load all starcoder params

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    77c7ec1 View commit details
    Browse the repository at this point in the history

  11. fix head count kv

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    0be15e1 View commit details
    Browse the repository at this point in the history

  12. fix comments

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    dac31da View commit details
    Browse the repository at this point in the history

  13. fix vram calculation for starcoder

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    4420cff View commit details
    Browse the repository at this point in the history

  14. store mqa directly

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    ab13d07 View commit details
    Browse the repository at this point in the history

  15. add input embeddings handling

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    8bc76a2 View commit details
    Browse the repository at this point in the history

  16. add TBD

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    101c578 View commit details
    Browse the repository at this point in the history

  17. working in cpu, metal buggy

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    a1cf66e View commit details
    Browse the repository at this point in the history

  18. cleanup useless code

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    6c353dc View commit details
    Browse the repository at this point in the history

  19. metal : fix out-of-bounds access in soft_max kernels

    @ggerganov
    ggerganov committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    f82328a View commit details
    Browse the repository at this point in the history

  20. llama : make starcoder graph build more consistent with others

    @ggerganov
    ggerganov committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    92a4f86 View commit details
    Browse the repository at this point in the history

  21. Merge pull request #2 from ggerganov/support-starcoder-fix 

    Support starcoder fix
    @wsxiaoys
    wsxiaoys authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    caa7220 View commit details
    Browse the repository at this point in the history

  22. refactor: cleanup comments a bit

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    57eaa39 View commit details
    Browse the repository at this point in the history

  23. add other starcoder models: 3B, 7B, 15B

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    5ca037b View commit details
    Browse the repository at this point in the history

  24. support-mqa-directly

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    08f35c4 View commit details
    Browse the repository at this point in the history

  25. Merge pull request #3 from TabbyML/support-starcoder-mqa 

    feat: support starcoder mqa
    @wsxiaoys
    wsxiaoys authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    e1fa9dd View commit details
    Browse the repository at this point in the history

  26. fix: remove max_position_embeddings, use n_train_ctx

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    f989ba1 View commit details
    Browse the repository at this point in the history

  27. Update llama.cpp 

    Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
    @wsxiaoys @ggerganov
    wsxiaoys and ggerganov authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    bb9931c View commit details
    Browse the repository at this point in the history

  28. Update llama.cpp 

    Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
    @wsxiaoys @ggerganov
    wsxiaoys and ggerganov authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    eafcc34 View commit details
    Browse the repository at this point in the history

  29. Apply suggestions from code review 

    Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
    @wsxiaoys @ggerganov
    wsxiaoys and ggerganov authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    e30ad71 View commit details
    Browse the repository at this point in the history

  30. fix: switch to space from tab

    @wsxiaoys
    wsxiaoys committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    72a7285 View commit details
    Browse the repository at this point in the history