Search results

Create 1.0 jailbreak benchmark (prompts, calibration)

Status: Open.

#1462 In mlcommons/modelbench;

· wpietri opened on Jan 20, 2026

Migrate modelbench to uv

Status: Open.

#1456 In mlcommons/modelbench;

· superdosh opened on Jan 16, 2026

Annotator logprobs

Status: Open.

#1446 In mlcommons/modelbench;

· bkorycki opened on Jan 12, 2026

Fix transient failures due to typer dependency

Status: Open.

#1444 In mlcommons/modelbench;

· bkorycki opened on Jan 8, 2026

Fix dynamic dedicated huggingface endpoint suts

bugSomething isn't workingSomething isn't working

Status: Open.

#1441 In mlcommons/modelbench;

· bkorycki opened on Jan 8, 2026

Run nemotron 3 nano 30b a3b safety and jailbreak benchmarks.

Status: Open.

#1439 In mlcommons/modelbench;

· bollacker opened on Jan 8, 2026

Ensure reasonable defult SUTOptions for modelgauge/modelplane

Status: Open.

#1433 In mlcommons/modelbench;

· superdosh opened on Jan 6, 2026

Return annotations for all items marked as shareable, not a %

Status: Open.

#1426 In mlcommons/modelbench;

· rogthefrog opened on Dec 19, 2025

Move grumblebot to #airr-alerts

choreShould do it, but it doesn't directly create valueShould do it, but it doesn't directly create value

Status: Open.

#1424 In mlcommons/modelbench;

· superdosh opened on Dec 18, 2025

update dependencies

Status: Open.

#1418 In mlcommons/modelbench;

· wpietri opened on Dec 15, 2025

Results and annotation file names should include a timestamp

choreShould do it, but it doesn't directly create valueShould do it, but it doesn't directly create value

Status: Open.

#1415 In mlcommons/modelbench;

· rogthefrog opened on Dec 12, 2025

Update prompt set file handling to support shareability flag

Status: Open.

#1414 In mlcommons/modelbench;

· rogthefrog opened on Dec 11, 2025