Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
114 Open 1,442 Closed
114 Open 1,442 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Adapting multimodal tasks for phi3.5 vision
#2909 opened Apr 14, 2025 by artemorloff Loading…
2
Audio modality: add openbmb/MiniCPM-o-2_6 model
#2908 opened Apr 14, 2025 by artemorloff Loading…
1
Add Qwen/Qwen-Audio-Chat support for audio modality
#2906 opened Apr 14, 2025 by artemorloff Loading…
2
tasks README: fix dead link
#2899 opened Apr 11, 2025 by dtrifiro Loading…
1
Longbench bugfix
#2895 opened Apr 9, 2025 by baberabb Loading…
1
enable evaluation from yaml config file
#2893 opened Apr 8, 2025 by artemorloff Loading…
4
Added AIME Support
#2892 opened Apr 8, 2025 by Zephyr271828 Loading…
2 of 3 tasks
3
Added C4 Support
#2889 opened Apr 8, 2025 by Zephyr271828 Loading…
2 of 3 tasks
1
Fix GPQA CoT n shot
#2888 opened Apr 7, 2025 by anmarques Loading…
2
Support override_generation_config in vLLM
#2882 opened Apr 5, 2025 by yeqcharlotte Loading…
1
Add device arg to model_args passed to LLM object in VLLM model class
#2879 opened Apr 4, 2025 by momentino Loading…
2
Add question suffix before the <|assistant|> tag
#2876 opened Apr 4, 2025 by TingchenFu Loading…
1
Fixed a bug that in MMLU-Pro utils.py that throw index error if one choice was removed
#2870 opened Apr 2, 2025 by sleepingcat4 Loading…
1
Fix formatting issues in XNLI tasks in Basque, Catalan, Galician and Spanish
#2869 opened Apr 2, 2025 by juliafalcao • Draft
1
allow API key in GGUFLM
#2862 opened Mar 29, 2025 by Huge Loading…
3
Feature: Add Sambanova Integration
#2859 opened Mar 28, 2025 by luisfucros Loading…
4
update gguf backend to use Chat-completion API
#2856 opened Mar 28, 2025 by falkbene Loading…
4
Fix slow gguf tests
#2846 opened Mar 26, 2025 by For-rest2005 Loading…
1
Add support for quantization_config
#2842 opened Mar 25, 2025 by jerryzh168 Loading…
11
Add simple Dockerfile and instructions
#2837 opened Mar 24, 2025 by kiersten-stokes Loading…
2
feat: Numeric bench
#2835 opened Mar 24, 2025 by Gresham429 Loading…
@baberabb
4
Add MkDocs Documentation with GitHub Actions Deployment
#2832 opened Mar 23, 2025 by Slyracoon23 Loading…
2
AfroBench: How Good are Large Language Models on African Languages?
#2825 opened Mar 20, 2025 by JessicaOjo Loading…
@baberabb
6
E3 c v3 name entity recognition
#2812 opened Mar 18, 2025 by sfarzi Loading…
3
Add new task named e3c_v3_re
#2806 opened Mar 17, 2025 by sfarzi Loading…
3
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.