Skip to content

[GuideLLM Refactor] Data pipelines rework and multimodal support #384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 14, 2025
Merged

[GuideLLM Refactor] Data pipelines rework and multimodal support #384

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
730eeb1
Initial state for datasets rework to enable multimodal and more compl…
markurtz Oct 1, 2025
dd7a4b8
[GuideLLM Refactor] Advanced Prefix Cache Controls (#382)
sjmonson Oct 3, 2025
bbca65a
Simplifications for new data pathways and reenablement of completions…
markurtz Oct 7, 2025
616ef92
Fix audio pathways so requests work
markurtz Oct 8, 2025
f862943
Finalize general refactor implementation for data pathways and enabli…
markurtz Oct 10, 2025
c65f37f
Merge branch 'features/refactor/base' into features/refactor/multimod…
markurtz Oct 13, 2025
3c646d4
runnable state for multi modal refactor
markurtz Oct 13, 2025
2b0fef8
Update src/guidellm/data/deserializers/synthetic.py
markurtz Oct 13, 2025
24f2ca3
Update src/guidellm/scheduler/worker_group.py
markurtz Oct 13, 2025
ef36af1
Fixes from review
markurtz Oct 13, 2025
b0becd5
Reenablement of flows and fixes
markurtz Oct 14, 2025
6adf793
Update src/guidellm/backends/openai.py
markurtz Oct 14, 2025
16f981c
Merge branch 'features/refactor/base' into features/refactor/multimod…
markurtz Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pylock.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4227,4 +4227,4 @@ strategy = ["inherit_metadata", "static_urls"]
requires_python = "~=3.12"

[[tool.pdm.targets]]
requires_python = ">=3.10.0,<3.12"
requires_python = ">=3.10.0,<3.12"
25 changes: 15 additions & 10 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ include = ["*"]
[tool.pdm]
distribution = true

[[tool.pdm.source]]
name = "torch"
type = "find_links"
#url = "https://download.pytorch.org/whl/cpu/torch_stable.html"
url = "https://download.pytorch.org/whl/cpu/torch/"
include_packages = ["torch"]


# ************************************************
# ********** Project Metadata **********
Expand Down Expand Up @@ -54,28 +61,26 @@ dependencies = [
"httpx[http2]<1.0.0",
"loguru",
"msgpack",
"numpy",
"numpy<2.0.0",
"pillow",
"protobuf",
"pydantic>=2.11.7",
"pydantic-settings>=2.0.0",
"pydub",
"pyyaml>=6.0.0",
"rich",
"sanic",
"transformers",
"uvloop>=0.18",
"librosa>=0.11.0",
"torch",
]

[project.optional-dependencies]
perf = [
"orjson",
"msgpack",
"msgspec",
"uvloop",
]
perf = ["orjson", "msgpack", "msgspec", "uvloop"]
recommended = [
"tiktoken>=0.11.0", # For OpenAI tokenizer
"blobfile>=3.1.0", # For OpenAI tokenizer
"tiktoken>=0.11.0", # For OpenAI tokenizer
"blobfile>=3.1.0", # For OpenAI tokenizer
]
dev = [
# build
Expand Down Expand Up @@ -118,7 +123,7 @@ dev = [
]

[dependency-groups]
dev = [ "guidellm[dev]" ]
dev = ["guidellm[dev]"]

[project.urls]
homepage = "https://github.com/vllm-project/guidellm"
Expand Down
3 changes: 3 additions & 0 deletions src/guidellm/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
import logging
import os

from datasets import config

with (
open(os.devnull, "w") as devnull, # noqa: PTH123
contextlib.redirect_stderr(devnull),
Expand All @@ -19,6 +21,7 @@
os.environ["TOKENIZERS_PARALLELISM"] = "false" # Silence warnings for tokenizers
hf_logging.set_verbosity_error()
logging.getLogger("transformers").setLevel(logging.ERROR)
config.USE_AUDIO_DECODE = False

from .logger import configure_logger, logger
from .settings import (
Expand Down
Loading
Loading