v0.5.0 Release Tracker

[simon-mo]

ETA Monday 06/10.

This is a version bump in minor because we expect the following major features to informally enter beta stage:

• Chunked Prefill
• Speculative Decode
• FP8
• VLM

Blockers:

• [ ]

Optional:

• Support Deepseek-V2 #4650
• [Frontend][Core] Update Outlines Integration from FSM to Guide #4109

[DarkLight1337]

Until I complete the chain of PRs related to #5214, imo the dev API for VLMs is still too unstable to be considered "beta".

On the other hand, the user API should be stable once #5237 is merged.

[sasha0552]

Hi, is it possible to include the following PRs?

• [Hardware][Nvidia] Enable support for Pascal GPUs #4409
• [Frontend] Add tokenize/detokenize endpoints #5054
• [Frontend] Customizable RoPE theta #5197

[chemistry-rocks]

Yes, #4409 please.

[flexorRegev]

if you can include #5164 in the release it would be great

[DarkLight1337]

Major PRs for supporting VLMs:

• [User] OpenAI Vision API ([Frontend] Add OpenAI Vision API Support #5237)
• [User] Initial impl. LLaVA-NeXT ([Model] Initial support for LLaVA-NeXT #4199 + [Bugfix] Fix LLaVA-NeXT #5380)
• [User][Dev] Dynamic image token size for LLaVA-NeXT ([Core] Dynamic image size support for VLMs #5276)

It would be ideal if we can add at least two more VLMs outside LLaVA.

[w013nad]

What's going on with the releases? It shows up under releases but it's also not the latest release. Docker vllm/vllm-openai also hasn't been updated?

[simon-mo]

It has not been released yet. The pre-release note in GitHub is generated as part of build process. When the release is out on PyPI and docker, we will close this issue and mark the release latest on GitHub

[cyc00518]

@simon-mo Sorry for tagging you. Would you mind telling me if vllm support aarch-64 now or not?

[simon-mo]

No. Contribution welcomed.