This repository has been archived by the owner on Oct 31, 2023. It is now read-only.
FP16/AMP training #15
Comments
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Hello, I have noticed that all scripts use full precision calculations. Are there reasons for that? Does it make substantial difference in terms of accuracy on downstream tasks? Were there some ablation studies regarding that?
I am wondering because it's easier to fit model with fp16 for university project.
Sorry if this question was answered elsewhere.
The text was updated successfully, but these errors were encountered: