Skip to content

Commit 706063a

Browse files
committed
update readme
1 parent e205e7f commit 706063a

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

docs/compare_with_phi-3_vision.md

+12
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,24 @@ In most benchmarks, MiniCPM-Llama3-V 2.5 achieves **better performance** compare
2525
| | Phi-3-vision-128K-Instruct | MiniCPM-Llama3-V 2.5|
2626
|:-|:----------:|:-------------------:|
2727
| Size(参数) | **4B** | 8B|
28+
| First Token Latency(首token延迟)$^1$ | L: 330ms, M: 330ms, H: 330ms | **L: 48ms, M: 145ms, H: 278ms** |
29+
| Throughtput(吞吐率)$^2$| 30 tokens/s | **41 tokens/s**|
2830
| OpenCompass 2024/05 | 53.7 | **58.8** |
2931
| OCRBench | 639.0 | **725.0**|
3032
| RealworldQA | 58.8 | **63.5**|
3133
| TextVQA | 72.2 | **76.6** |
3234
| ScienceQA| **90.8** | 89.0 |
3335
| POPE | 83.4 | **87.2** |
36+
| MathVista | 44.5 | **54.3** |
37+
| MMStar | 47.4 | **51.8** |
38+
| LLaVA Bench | 64.2 | **86.7** |
39+
| AI2D | 76.7 | **78.4** |
40+
41+
<small>
42+
1: L(ow): 448pxl, M(edium): 896pxl, H(igh): 1344pxl input images.
43+
<br>
44+
2. Evaluation environment: A800 GPU, flash-attn=2.4.3, batch-size=1.
45+
</small>
3446

3547

3648
## Multilingual Capabilities

0 commit comments

Comments
 (0)