Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0 # Use the ref you want to point at
rev: v5.0.0 # Use the ref you want to point at
hooks:
- id: trailing-whitespace
- id: check-ast
Expand All @@ -15,8 +15,8 @@ repos:
- id: check-yaml
- id: check-toml

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: 'v0.6.2'
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: 'v0.9.3'
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
Expand All @@ -25,7 +25,7 @@ repos:
types_or: [python, jupyter]

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.11.1
rev: v1.14.1
hooks:
- id: mypy
entry: python3 -m mypy --config-file pyproject.toml
Expand All @@ -34,7 +34,7 @@ repos:
exclude: "tests"

- repo: https://github.com/nbQA-dev/nbQA
rev: 1.8.7
rev: 1.9.1
hooks:
- id: nbqa-ruff
args: [--fix, --exit-non-zero-on-fix]
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ vec-inf launch Meta-Llama-3.1-8B-Instruct
```
You should see an output like the following:

<img width="700" alt="launch_img" src="https://github.com/user-attachments/assets/ab658552-18b2-47e0-bf70-e539c3b898d5">
<img width="600" alt="launch_img" src="https://github.com/user-attachments/assets/ab658552-18b2-47e0-bf70-e539c3b898d5">

The model would be launched using the [default parameters](vec_inf/models/models.csv), you can override these values by providing additional parameters, use `--help` to see the full list. You can also launch your own customized model as long as the model architecture is [supported by vLLM](https://docs.vllm.ai/en/stable/models/supported_models.html), and make sure to follow the instructions below:
* Your model weights directory naming convention should follow `$MODEL_FAMILY-$MODEL_VARIANT`.
Expand Down