Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Improve quickstart documentation #9256

Merged
merged 13 commits into from
Oct 25, 2024
Merged

[Doc] Improve quickstart documentation #9256

merged 13 commits into from
Oct 25, 2024

Conversation

rafvasq
Copy link
Contributor

@rafvasq rafvasq commented Oct 10, 2024 •
edited
Loading

This PR tries to improve the existing quickstart guide for newcomers to the project who want to generate something with vLLM quickly.

  • Include pre-req system requirements
  • Include installation step
  • Reorganize sections
  • Add links to help with LLM terminology
  • Improve step-by-step guide readability
  • Use Qwen2.5-1.5B-Instruct for chat/server sections

@rafvasq
Revamp quickstart doc
4e85234 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Update titles
a316761 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Uses shell consistently
c0fd6b7 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

@rafvasq rafvasq marked this pull request as ready for review October 10, 2024 17:25
@rafvasq rafvasq marked this pull request as draft October 10, 2024 17:26
@rafvasq
Remove targets
cb54ce8 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Fix hyperlink
e206a39 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Uses anonymous references
99a3d84 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Removes paragraph out of note
2a8dbee 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq rafvasq marked this pull request as ready for review October 10, 2024 18:11
ckadner
ckadner reviewed Oct 15, 2024
View reviewed changes
Copy link

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @rafvasq for improving the quickstart guide. I have a few nits and some questions. I'm new rst formatting as and some of my suggestion are likely just personal preference. Feel free to ignore those :-)

docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
docs/source/getting_started/quickstart.rst Outdated Show resolved Hide resolved
@rafvasq
Fixes typos, addresses review
b7606cd 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Addresses comments
aa09dec 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Fixes refs
d430c62 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Fixes wording
067cb01 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq rafvasq requested a review from ckadner October 16, 2024 20:43
@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 16, 2024

Thanks @ckadner, I believe I addressed everything!

@rafvasq rafvasq mentioned this pull request Oct 18, 2024
Merged
ckadner
ckadner approved these changes Oct 23, 2024
View reviewed changes
Copy link

@ckadner ckadner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks @rafvasq

@rafvasq
Changes last shell to console
b8e1531 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@simon-mo
Copy link
Collaborator

This looks great! Can you also update the chat/server section to ues https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct so that the users don't need a chat template to start with (advanced feature)?

@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 23, 2024

! Can you also update the chat/server section to ues https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct so that the users don't need a chat template to start with (advanced feature)?

Hey @simon-mo, would removing the extra note about chat templates suffice? It doesn't seem to me like a user needs to know anything about chat templates to use the guide with facebook/opt-125m as written since vLLM just uses a default quietly.

@simon-mo
Copy link
Collaborator

ah that's a good point. however my main problem with the opt model is that just generates gibberish for chat.

@rafvasq
Use qwen model instead of opt125
2f7e4dd 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@rafvasq
Copy link
Contributor Author

rafvasq commented Oct 25, 2024

Makes sense, I've updated the model for those sections

@simon-mo simon-mo merged commit 228cfbd into vllm-project:main Oct 25, 2024
27 of 30 checks passed
@rafvasq rafvasq deleted the update-quickstart-doc branch October 25, 2024 21:38
MErkinSag pushed a commit to MErkinSag/vllm that referenced this pull request Oct 26, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
f255705 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Erkin Sagiroglu <erkin@infra-aipipeline-1-at1-prox-prod-a.ipa.corp.telnyx.com>
MErkinSag pushed a commit to MErkinSag/vllm that referenced this pull request Oct 26, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
cb49053 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Erkin Sagiroglu <erkin@infra-aipipeline-1-at1-prox-prod-a.ipa.corp.telnyx.com>
cooleel pushed a commit to cooleel/vllm that referenced this pull request Oct 28, 2024
@rafvasq @cooleel
[Doc] Improve quickstart documentation (vllm-project#9256)
cf45bc9 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
cooleel pushed a commit to cooleel/vllm that referenced this pull request Oct 28, 2024
@rafvasq @cooleel
[Doc] Improve quickstart documentation (vllm-project#9256)
0d890a2 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Shanshan Wang <shanshan.wang@h2o.ai>
FerdinandZhong pushed a commit to FerdinandZhong/vllm that referenced this pull request Oct 29, 2024
@rafvasq @FerdinandZhong
[Doc] Improve quickstart documentation (vllm-project#9256)
6939553 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: qishuai <ferdinandzhong@gmail.com>
rasmith pushed a commit to rasmith/vllm that referenced this pull request Oct 30, 2024
@rafvasq @rasmith
[Doc] Improve quickstart documentation (vllm-project#9256)
859700e 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Randall Smith <Randall.Smith@amd.com>
NickLucche pushed a commit to NickLucche/vllm that referenced this pull request Oct 31, 2024
@rafvasq @NickLucche
[Doc] Improve quickstart documentation (vllm-project#9256)
fd5fefd 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
NickLucche pushed a commit to NickLucche/vllm that referenced this pull request Oct 31, 2024
@rafvasq @NickLucche
[Doc] Improve quickstart documentation (vllm-project#9256)
813b4f0 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
410f5e0 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Nov 4, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
c1bda37 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Linkun Chen <github+anyscale@lkchen.net>
sumitd2 pushed a commit to sumitd2/vllm that referenced this pull request Nov 14, 2024
@rafvasq @sumitd2
[Doc] Improve quickstart documentation (vllm-project#9256)
a7d8de7 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Sumit Dubey <sumit.dubey2@ibm.com>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
1483c01 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
mfournioux pushed a commit to mfournioux/vllm that referenced this pull request Nov 20, 2024
@rafvasq @mfournioux
[Doc] Improve quickstart documentation (vllm-project#9256)
e57148c 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Maxime Fournioux <55544262+mfournioux@users.noreply.github.com>
tlrmchlsmth pushed a commit to neuralmagic/vllm that referenced this pull request Nov 23, 2024
@rafvasq @tlrmchlsmth
[Doc] Improve quickstart documentation (vllm-project#9256)
b576263 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Signed-off-by: Tyler Michael Smith <tyler@neuralmagic.com>
sleepwalker2017 pushed a commit to sleepwalker2017/vllm that referenced this pull request Dec 13, 2024
@rafvasq
[Doc] Improve quickstart documentation (vllm-project#9256)
791d20e 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simon-mo simon-mo simon-mo approved these changes

@ckadner ckadner ckadner approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rafvasq @simon-mo @ckadner