How to use SeedBench ?

[Jiangnan-Twin-Paradises]

I can't find any link to use this model.

[tpoisonooo]

SeedBench is a benchmark for LLM, it has been committed to opencompass, see open-compass/opencompass#2020

Firstly in your opencompass_config.py (see opencompass usage)

..
    from opencompass.configs.datasets.SeedBench.seedbench_gen import seedbench_datasets as agri_datasets
..
datasets = agri_datasets + ...

Then run opencompass benchmark commmand, for example:

DATASET_SOURCE=ModelScope opencompass config/your_config.py

This would download SeedBench from modelscope and benchmark you language model.

[tpoisonooo]

After opencompass running, you will get prediction results and precision summary table, just like this:

[mengkaishil]

i cannot find SeedBench in Compass

[tpoisonooo]

i cannot find SeedBench in Compass

This PR is waiting for opencompass developer merge it. So you have to checkout branch.

[ChenZiHong-Gavin]

i cannot find SeedBench in Compass

open-compass/opencompass#2020

This PR has been merged. Now you can use opencompass to evaluate SeedBench.