Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ModelTC/lightllm
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.0.0
Choose a base ref
...
head repository: ModelTC/lightllm
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.0.1
Choose a head ref
  • 14 commits
  • 76 files changed
  • 13 contributors

Commits on Feb 19, 2025

  1. update and fix README

    @XHPlus
    XHPlus committed Feb 19, 2025
    Configuration menu
    Copy the full SHA
    5c28b33 View commit details
    Browse the repository at this point in the history

  2. Benchclient (#740) 

    Co-authored-by: shihaobai <baishihao@sensetime.com>
    @shihaobai
    shihaobai and shihaobai authored Feb 19, 2025
    Configuration menu
    Copy the full SHA
    971936e View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2025

  1. fix pause reqs (#741)

    @shihaobai
    shihaobai authored Feb 20, 2025
    Configuration menu
    Copy the full SHA
    57bb6e1 View commit details
    Browse the repository at this point in the history

  2. add RETURN_LIST for tgi_api (#742) 

    Co-authored-by: shihaobai <baishihao@sensetime.com>
    @shihaobai
    shihaobai and shihaobai authored Feb 20, 2025
    Configuration menu
    Copy the full SHA
    9313a08 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2025

  1. fix: fix a precision bug in the context_flashattention (#743)

    @blueswhen
    blueswhen authored Feb 21, 2025
    Configuration menu
    Copy the full SHA
    00e4de7 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2025

  1. Improve the accuracy of deepseekv3 (#744)

    @hiworldwzj
    hiworldwzj authored Feb 22, 2025
    Configuration menu
    Copy the full SHA
    808d832 View commit details
    Browse the repository at this point in the history

  2. deepseekv3 bmm noquant and fix moe gemm bug. (#745) 

    Co-authored-by: shihaobai <baishihao@sensetime.com>
Co-authored-by: shihaobai <42648726+shihaobai@users.noreply.github.com>
    @hiworldwzj @shihaobai
    3 people authored Feb 22, 2025
    Configuration menu
    Copy the full SHA
    c483b1e View commit details
    Browse the repository at this point in the history

Commits on Feb 26, 2025

  1. Add Xgrammar Support (#701) 

    Change:
- change launch argument from `--simple_constraint_mode` to
`--output_constraint_mode`, now user can choose the constriant decode
backend from ['outlines', 'xgrammar']
- add `XgrammarBackend` used for xgrammar constraint decode, maybe we
should merge it with `SimpleConstraintBackend` later?
- now we adopt the same request body, the same as vLLM with
xgrammar(https://docs.vllm.ai/en/stable/serving/openai_compatible_server.html)
- user can add `guided_grammar` to pass a EBNF grammar and execute the
constraint decode
- user can add `guided_json` to pass a standard json schema and do the
constraint decode

---------

Co-authored-by: hiworldwzj <30762946+hiworldwzj@users.noreply.github.com>
    @flyinglandlord @hiworldwzj
    flyinglandlord and hiworldwzj authored Feb 26, 2025
    Configuration menu
    Copy the full SHA
    c8c892a View commit details
    Browse the repository at this point in the history

  2. fuse fp8 quant in kv copying and add flashinfer decode mla operator i… 

    …n the attention module (#737)
    @blueswhen
    blueswhen authored Feb 26, 2025
    Configuration menu
    Copy the full SHA
    d7c0a4b View commit details
    Browse the repository at this point in the history

  3. fix: add flashinfer-python in the requirements.txt (#749) 

    Co-authored-by: niushengxiao <niushengxiao@sensetime.com>
    @blueswhen
    blueswhen and niushengxiao authored Feb 26, 2025
    Configuration menu
    Copy the full SHA
    39d9a59 View commit details
    Browse the repository at this point in the history

  4. Fix tokens2 (#748)

    @SangChengC
    SangChengC authored Feb 26, 2025
    Configuration menu
    Copy the full SHA
    f6f7a41 View commit details
    Browse the repository at this point in the history

Commits on Feb 27, 2025

  1. Fix Unit-test in PR: Add xgrammar (#750)

    @flyinglandlord
    flyinglandlord authored Feb 27, 2025
    Configuration menu
    Copy the full SHA
    4a974c2 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2025

  1. add support for multinode tp (#751) 

    Co-authored-by: wufeiyang <jayfeather9@qq.com>
Co-authored-by: root <root@pt-290ac8041d114af0b1647509a5544872-master-0.pt-290ac8041d114af0b1647509a5544872.ns-devoversea-d41e68bd.svc.cluster.local>
Co-authored-by: shihaobai <baishihao@sensetime.com>
Co-authored-by: root <root@pt-511f450a52c24c2d9df9b20f0c8ebdb7-master-0.pt-511f450a52c24c2d9df9b20f0c8ebdb7.ns-devoversea-d41e68bd.svc.cluster.local>
Co-authored-by: Feiyang Wu <wufeiyang@sensetime.com>
Co-authored-by: wangzaijun <wzjhelloworld@qq.com>
Co-authored-by: hiworldwzj <30762946+hiworldwzj@users.noreply.github.com>
    @shihaobai @jayfeather9 @hiworldwzj
    8 people authored Mar 1, 2025
    Configuration menu
    Copy the full SHA
    2dbd8e5 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2025

  1. fix /tokens docs && tag v1.0.1 (#753) 

    Co-authored-by: shihaobai <baishihao@sensetime.com>
    @shihaobai
    shihaobai and shihaobai authored Mar 3, 2025
    Configuration menu
    Copy the full SHA
    b418bdf View commit details
    Browse the repository at this point in the history
Loading