Open
Description
Issue encountered
Having the ability for benchmark builders to create a dataset and a spec that will be read with lighteval.
Solution/Feature
A dataset, with the benchmark data
A python file that defines the prompt metric etc OR a yaml file if you want to use regular metrics.