Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
fixes issue identified by derek (#83)
Browse files Browse the repository at this point in the history
  • Loading branch information
robertgshaw2-neuralmagic authored Feb 29, 2024
1 parent 2879d9d commit adab47a
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion examples/production_monitoring/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@ Launch Prometheus and Grafana servers with `docker compose`:
docker compose up
```

Submit some sample requests to the server:
Submit some sample requests to the server (after `pip install aiohttp`):
```bash
wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json

python3 ../../benchmarks/benchmark_serving.py \
--model mistralai/Mistral-7B-v0.1 \
--tokenizer mistralai/Mistral-7B-v0.1 \
--backend openai \
--endpoint /v1/completions \
--dataset ShareGPT_V3_unfiltered_cleaned_split.json \
--request-rate 3.0
Expand Down

0 comments on commit adab47a

Please sign in to comment.