Skip to content

Commit 8a57940

Browse files
remiminRen MinMin
and
Ren MinMin
authored
[Misc] Update benchmark_prefix_caching.py fixed example usage (#11920)
Signed-off-by: Ren MinMin <renmm6@chinaunicom.cn> Co-authored-by: Ren MinMin <renmm6@chinaunicom.cn>
1 parent 46fa98c commit 8a57940

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

benchmarks/benchmark_prefix_caching.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@
1010
--model meta-llama/Llama-2-7b-chat-hf \
1111
--enable-prefix-caching \
1212
--num-prompts 1 \
13-
--repeat-count 100
13+
--repeat-count 100 \
14+
--input-length-range 128:256
1415
1516
ShareGPT example usage:
1617
# This command samples 20 prompts with input lengths

0 commit comments

Comments
 (0)