add example & check for vLLM hosted inference server

[asamal4]

Add example & check for vLLM hosted inference server

Summary by CodeRabbit

• New Features

  • Added support for the Hosted vLLM provider, enabling model access via hosted endpoints. Requires HOSTED_VLLM_API_KEY and HOSTED_VLLM_API_BASE.

• Documentation

  • Updated provider list to include vLLM and adjusted per-provider examples with clear provider annotations (e.g., watsonx, gemini, hosted_vllm).
  • Added setup guidance for Hosted vLLM, including required environment variables.

• Chores

  • Refreshed configuration comments to reflect the expanded provider options (including gemini and hosted_vllm).

[coderabbitai]

Walkthrough

Adds hosted_vllm as a new LLM provider across docs and code. Updates README and config comments to document provider and env vars. Implements provider-specific handling in LLMManager and adds environment validation for HOSTED_VLLM_API_KEY and HOSTED_VLLM_API_BASE.

Changes

Cohort / File(s)	Summary
Docs updates README.md, config/system.yaml	Document hosted_vllm provider; add env vars HOSTED_VLLM_API_KEY and HOSTED_VLLM_API_BASE; adjust provider listings and example annotations (e.g., watsonx, gemini).
LLM Manager provider handling src/lightspeed_evaluation/core/llm/manager.py	Add _handle_hosted_vllm_provider and wire it into provider handler map; constructs model name hosted_vllm/{model} after env validation. Existing providers unchanged.
Env validation src/lightspeed_evaluation/core/system/env_validator.py	Add validate_hosted_vllm_env() checking HOSTED_VLLM_API_KEY and HOSTED_VLLM_API_BASE; register in validate_provider_env validators map.

Sequence Diagram(s)

sequenceDiagram
  participant User as User
  participant LLMManager as LLMManager
  participant Env as EnvValidator
  participant Backend as Hosted vLLM API

  User->>LLMManager: init(provider="hosted_vllm", model)
  LLMManager->>Env: validate_provider_env("hosted_vllm")
  Env-->>LLMManager: OK (HOSTED_VLLM_API_KEY, HOSTED_VLLM_API_BASE)
  LLMManager->>LLMManager: build model name "hosted_vllm/{model}"
  LLMManager->>Backend: request with constructed model and API base/key
  Backend-->>LLMManager: response
  LLMManager-->>User: return response

  rect rgba(230, 245, 255, 0.6)
  note over LLMManager,Env: New/changed path for provider "hosted_vllm"
  end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

• VladimirKadlec
• tisnik

Poem

I twitch my ears at vLLM’s call,
New keys set snug upon the wall.
I hop through handlers, neat and trim,
Env checks pass—no chances slim.
With hosted paths I lightly tread,
Carrots cached, the models fed.
Thump-thump—ship it! 🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title succinctly captures the primary change of adding example documentation and a validation check for a hosted vLLM inference server, clearly reflecting the updates to README examples, LLM manager, and environment validation logic without extraneous detail.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5a2ed1c and 8ebb792.

📒 Files selected for processing (4)

• README.md (2 hunks)
• config/system.yaml (1 hunks)
• src/lightspeed_evaluation/core/llm/manager.py (2 hunks)
• src/lightspeed_evaluation/core/system/env_validator.py (2 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

src/lightspeed_evaluation/core/system/env_validator.py (1)

src/lightspeed_evaluation/core/system/exceptions.py (1)

• LLMError (24-25)

src/lightspeed_evaluation/core/llm/manager.py (1)

src/lightspeed_evaluation/core/system/env_validator.py (1)

• validate_provider_env (74-95)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: mypy

🔇 Additional comments (7)

config/system.yaml (1)

5-5: LGTM!

Documentation update correctly reflects the new hosted_vllm provider option.

src/lightspeed_evaluation/core/system/env_validator.py (2)

8-14: LGTM!

Environment validation follows the established pattern for multi-variable providers (Azure, Watsonx). Correctly validates both API key and base URL are present.

---

84-84: LGTM!

Validator correctly registered in the provider mapping. The implementation will now validate hosted_vllm environment variables when the provider is selected.

README.md (2)

12-12: LGTM!

Feature list correctly updated to reflect vLLM support.

---

348-362: LGTM!

Documentation correctly specifies the required environment variables for hosted_vllm (HOSTED_VLLM_API_KEY and HOSTED_VLLM_API_BASE), matching the validation logic in env_validator.py. Provider labels enhance clarity.

src/lightspeed_evaluation/core/llm/manager.py (2)

33-33: LGTM!

Handler correctly registered in the provider mapping, following the established pattern.

---

50-53: LGTM!

Handler implementation follows the established pattern: validates environment variables via validate_provider_env, then returns the model name with the provider prefix. The hosted_vllm/{model} format is consistent with other custom providers (watsonx, gemini, anthropic).

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[asamal4]

@tisnik @VladimirKadlec PTAL - example for vllm based inference server
cc: @lpiwowar

[VladimirKadlec]

LGTM

[tisnik]

LGTM