Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add comparisons with teacher models #234

Closed
marco-c opened this issue Oct 25, 2023 · 1 comment
Closed

Add comparisons with teacher models #234

marco-c opened this issue Oct 25, 2023 · 1 comment
Labels
evals Issues related to model evaluations quality Improving robustness and translation quality

Comments

@marco-c
Copy link
Collaborator

marco-c commented Oct 25, 2023

In our evaluation code, we currently compare our student models against models from other vendors.
It would be nice to also compare our teacher models, so we can easily see how much we are losing between teacher and student and we can consider increasing the size of student models (maybe ship different models to desktop and mobile users).
@marco-c marco-c added quality Improving robustness and translation quality evals Issues related to model evaluations duplicate labels Oct 25, 2023
@marco-c
Copy link
Collaborator Author

marco-c commented Oct 25, 2023

Oops, I had already filed this... (#175)

@marco-c marco-c closed this as not planned Won't fix, can't repro, duplicate, stale Oct 25, 2023
@marco-c marco-c mentioned this issue Oct 25, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
evals Issues related to model evaluations quality Improving robustness and translation quality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@marco-c