Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supporting log probabilities of prompt tokens in both engine and OpenAI API server (aka echo) #959

Closed
wants to merge 40 commits into from
Closed

Supporting log probabilities of prompt tokens in both engine and OpenAI API server (aka echo) #959

Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
721c778
A runnable implementation of `echo`.
wanmok Sep 6, 2023
4abbe99
Reformatted.
wanmok Sep 6, 2023
3a9649f
Fixed pylint `R1729` and `R1721`
wanmok Sep 6, 2023
f075e8b
Fixed pylint and allowed `logprobs=0, echo=True`.
wanmok Sep 6, 2023
e419414
Fixed edge case of when to return `null` - consistent with OpenAI.
wanmok Sep 6, 2023
50b30f5
Fixed the issue of using `prompt_token_ids` while not being able to g…
wanmok Sep 6, 2023
69f9e52
Fixed the error of computing logprobs.
wanmok Sep 7, 2023
1070080
Fixed the pylint E1133
wanmok Sep 7, 2023
bd180cb
Fixed the issue of running with distributed workers.
wanmok Sep 7, 2023
46af13a
Set torch default dtype in a context manager (#971)
Yard1 Sep 7, 2023
8309110
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 7, 2023
fdc4e5a
Moved postprocessing to `_process_sequence_group_samples()`.
wanmok Sep 7, 2023
f422556
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 7, 2023
4ede60d
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 7, 2023
6c1a70c
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 8, 2023
0bbf78d
Fixed the issue with the first `top_logprobs` being None.
wanmok Sep 8, 2023
b60817d
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 8, 2023
39f8268
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 11, 2023
061ef6f
Fixed handle the batched case correctly.
wanmok Sep 13, 2023
07240ab
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 13, 2023
9b874f2
Updated to address review comments.
wanmok Sep 18, 2023
bf47666
Merge branch 'main' into add-echo
wanmok Sep 18, 2023
7cab379
Reformatted.
wanmok Sep 18, 2023
73f542c
Added test for `get_prompt_logprobs`
wanmok Sep 18, 2023
d6e8f68
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 19, 2023
8541ae0
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 21, 2023
5b63edf
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 21, 2023
1aab537
Merge branch 'main' into add-echo
wanmok Sep 22, 2023
4b7fd8e
Reformatted sampling_params.py
wanmok Sep 22, 2023
364c7aa
Merge branch 'main' into add-echo
wanmok Sep 23, 2023
c6b8d1f
Reformatted
wanmok Sep 23, 2023
67a5de0
Merge branch 'vllm-project:main' into add-echo
wanmok Sep 24, 2023
17675ec
Merge branch 'main' into add-echo
wanmok Sep 27, 2023
792f857
Merge branch 'main' into add-echo
wanmok Sep 29, 2023
ed52b8e
Merge branch 'vllm-project:main' into add-echo
wanmok Oct 1, 2023
0d643fd
Merge branch 'vllm-project:main' into add-echo
wanmok Oct 4, 2023
18f9efa
Update sampler.py
wanmok Oct 4, 2023
9967c2f
Merge branch 'vllm-project:main' into add-echo
wanmok Oct 9, 2023
a0ebd61
Merged log forward.
wanmok Oct 9, 2023
4121f86
Merge branch 'vllm-project:main' into add-echo
wanmok Oct 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'vllm-project:main' into add-echo
  • Loading branch information
@wanmok
wanmok authored Sep 21, 2023
commit 5b63edfcb1d89ccd8633d22f62aa4819c3a5c833

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.