Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Kernel][Core][WIP] Tree attention and parallel decoding #4325

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

[Kernel][Core][WIP] Tree attention and parallel decoding #4325

wants to merge 8 commits into from

Commits on Apr 16, 2024

  1. merge different seqs in seqs group in to once attention inference wit… 

    …hout implement tree attention kernel
    kavioyu committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    bdc863d View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. tree width = 2 have been tested, but there is error when width >2

    kavioyu committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    dadbed1 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. temp

    kavioyu committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    7596ad8 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. tested

    kavioyu committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    1534c5c View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. fix early stop

    kavioyu committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    a3849d1 View commit details
    Browse the repository at this point in the history

  2. fix code style

    kavioyu committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    af98a27 View commit details
    Browse the repository at this point in the history

  3. fix bug

    kavioyu committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    92ebde5 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. add duration check

    kavioyu committed May 7, 2024
    Configuration menu
    Copy the full SHA
    ce76cc7 View commit details
    Browse the repository at this point in the history