Skip to content

Issues: instructlab/eval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

11 Open 33 Closed
11 Open 33 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Create XL E2E CI job CI/CD Affects CI/CD configuration enhancement New feature or request
#201 opened Dec 20, 2024 by nathan-weinberg
1
MMLU eval failing on granite-8b-starter model: 500 Server Error bug Something isn't working
#195 opened Dec 9, 2024 by relyt0925
8
Eval Maintainer nomination nomination
#194 opened Dec 6, 2024 by RobotSail
Move dynamic default values to instance time
#189 opened Nov 22, 2024 by osilkin98
Change MMLU evaluator to have float16 as the default dtype
#188 opened Nov 22, 2024 by osilkin98
Be more consistent with OOP/functional programming
#187 opened Nov 21, 2024 by RobotSail
Create design doc for LLMaJ jira
#178 opened Nov 14, 2024 by ktam3
@alimaredia
LLMaJ for evaluation of InstructLab fine-tuned LLM with and without RAG enhancement New feature or request
#177 opened Nov 14, 2024 by ktam3
1 task
@alimaredia
Evaluation of user data using Unitxt
#176 opened Nov 7, 2024 by Roni-Friedman
1
@Roni-Friedman
3
Allow usage of GPT-4 as judge model enhancement New feature or request mtbench Pertains to MTBench mtbenchbranch Pertains to MTBenchBranch
#83 opened Jul 29, 2024 by jaideepr97
@alimaredia
MTBenchEvaluator and MMLUEvaluator should be/have static methods mmlu Pertains to MMLU mtbench Pertains to MTBench
#26 opened Jun 26, 2024 by JamesKunstle
2
ProTip! Find all open issues with in progress development work with linked:pr.