Is there a open continuation of this for gpt 3.5/4/ llama3.1/o1 ? #790
manucarbonell
started this conversation in
General
Replies: 1 comment
-
there is code for training much smaller versions of gpt3, but reproducing gpt4, o1 or the full llama model was never the point, this repository is meant to be a practical reference implementation to learn from and train with on a budget. But the price to reproduce any those models would be astronomically high, so implementing these architectures is expanding the scope of the project and therefore a slow process. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Distance between open WEIGHT and frontier models looks to be stable, but open source? Any available training of gpt3.5 here or somewhere?
I see only until gpt2 here, we're getting left behind
Thanks
Beta Was this translation helpful? Give feedback.
All reactions