Use eltype(x) everywhere, ignore typeof(η)
#151
Conversation
definitely, we also get nicer prints of the opt_state has a bonus |
|
If using the eltype of momentum is too restrictive, we could always change to some promoted eltype of various parts of input + state (e.g. momentum + param + gradient eltypes). Otherwise 👍 from me though, nice to see that this works! |
mcabbott
changed the title
eltype(momentum) everywhere, ignore typeof(η)eltype(x) everywhere, ignore typeof(η)
test/runtests.jl
Outdated
| @static if VERSION <= v"1.10-" | ||
| # In July 2023, Yota works on 1.9 but not nightly (1.11) | ||
| using Yota |
There was a problem hiding this comment.
This doesn't prevent an error from Yota not compiling on master. Could revert as it's unrelated to this PR.
There was a problem hiding this comment.
since this doesn't solve the CI problem let's remove the change from this PR
|
We removed |
|
In fact I doubt anyone types that, but the concern is things like BSON, right? |
This reverts commit abf0e13.
|
I merge this now, we can think about whether anything else wants to be ganged into a breaking release. |
|
Tagging a new breaking release now since this fixes a lot of problems and I don't see any breaking change on the horizon. |
* always convert learning rate to eltype(momentum) before use * don't parameterise rule structs, just use Float64 * fix tests like === 0.2f0 * fix a constructor * fix AdamW * fix Rprop * use a macro to define structs with default values * use T = eltype(x) * a few more structs * more structs * fix tests * doc fixes * fix docstrings * skip Yota on nightly * docstrings * breaking change, v0.3-dev * print Adam(0.01f0) without 0.009999999776482582 * Revert "skip Yota on nightly" This reverts commit abf0e13. * don't accidentally write to stdout
First commit should fix #150, by working all in Float32.
Chooses to do this by always following the momentum fromchanged tosetup, not thex, but perhaps that's messy?eltype(x)now.Since the type of
ηis now ignored, perhaps we should just remove all the type parameters? Then e.g.Adam(0) == Adam(0.0), fixes #119, closes #120. Also closes #86 (redundant).Not all rules are changed yet, RFC?