Skip to content

Update prompt set file handling to support shareability flag #1414

@rogthefrog

Description

@rogthefrog

We are sharing a subset of prompts and responses with customers through BaaS. New prompt set files have been generated with a new column indicating which prompts can be included. Make Modelbench compatible with that new file format, and change the logic in BenchmarkRun.compile_annotations that filters which annotations are included in the annotation file.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions