Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[LLM] Support block_attention/cachekv quant for llama #7649

Merged
merged 15 commits into from
Jan 10, 2024
Merged

[LLM] Support block_attention/cachekv quant for llama #7649

merged 15 commits into from
Jan 10, 2024

Commits on Dec 20, 2023

  1. support blha and cache kv quant

    @RichardWooSJTU
    RichardWooSJTU committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    c43c61a View commit details
    Browse the repository at this point in the history

  2. fix conflict

    @RichardWooSJTU
    RichardWooSJTU committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    d5ab849 View commit details
    Browse the repository at this point in the history

  3. lint

    @RichardWooSJTU
    RichardWooSJTU committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    340d6d1 View commit details
    Browse the repository at this point in the history

  4. fix unit test

    @RichardWooSJTU
    RichardWooSJTU committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    235198d View commit details
    Browse the repository at this point in the history

  5. fix infer when blha is on

    @RichardWooSJTU
    RichardWooSJTU committed Dec 20, 2023
    Configuration menu
    Copy the full SHA
    7df5824 View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2023

  1. code refine

    @RichardWooSJTU
    RichardWooSJTU committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    79e0b31 View commit details
    Browse the repository at this point in the history

  2. merge develop

    @RichardWooSJTU
    RichardWooSJTU committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    d44fe99 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. merge develop

    @RichardWooSJTU
    RichardWooSJTU committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    846e627 View commit details
    Browse the repository at this point in the history

  2. add docs and fix ops

    @RichardWooSJTU
    RichardWooSJTU committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    8dac802 View commit details
    Browse the repository at this point in the history

  3. merge blha read res in predictor

    @RichardWooSJTU
    RichardWooSJTU committed Jan 5, 2024
    Configuration menu
    Copy the full SHA
    cc0a25d View commit details
    Browse the repository at this point in the history

Commits on Jan 8, 2024

  1. finish docs

    @RichardWooSJTU
    RichardWooSJTU committed Jan 8, 2024
    Configuration menu
    Copy the full SHA
    121c920 View commit details
    Browse the repository at this point in the history

Commits on Jan 9, 2024

  1. add docs and unittest

    @RichardWooSJTU
    RichardWooSJTU committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    b8aea32 View commit details
    Browse the repository at this point in the history

  2. add unittest

    @RichardWooSJTU
    RichardWooSJTU committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    afb1dd0 View commit details
    Browse the repository at this point in the history

  3. merge develop

    @RichardWooSJTU
    RichardWooSJTU committed Jan 9, 2024
    Configuration menu
    Copy the full SHA
    ed6db33 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2024

  1. migrate read res

    @RichardWooSJTU
    RichardWooSJTU committed Jan 10, 2024
    Configuration menu
    Copy the full SHA
    8b91dc8 View commit details
    Browse the repository at this point in the history