Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any plans to add new models? #17

Open
ryoungj opened this issue Jul 31, 2024 · 1 comment
Open

Any plans to add new models? #17

ryoungj opened this issue Jul 31, 2024 · 1 comment

Comments

@ryoungj
Copy link

ryoungj commented Jul 31, 2024

Hi there,

Thank you for the great contributions!

There have been many new models released since the benchmark was published. Do you have any plans to include some of these recent models, such as GPT-4o, Claude-3.5, Llama-3.1 405B, Mistral Large 2, DeepSeek V2, and others? Adding results from these models could provide significant value to the community!

Thanks

@ryoungj ryoungj changed the title Any plans to add new models Any plans to add new models? Jul 31, 2024
@chang-github-00
Copy link
Collaborator

Yes, we have tested Llama3-8B/ 70B, Claude-3, and Gemini pro recently and will merge this soon. We will continue to add these important models suggested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants