Merge pull request #954 from FluxML/decaydocs

[WIP] Decaydocs
FluxML · Dec 10, 2019 · ac4c49b · ac4c49b
2 parents f46b524 + e67f09c
commit ac4c49b
Showing 1 changed file with 3 additions and 2 deletions.
diff --git a/src/optimise/optimisers.jl b/src/optimise/optimisers.jl
@@ -444,7 +444,8 @@ end
 """
   InvDecay(γ)
 
-Applies inverse time decay to an optimiser
+Applies inverse time decay to an optimiser, i.e., the effective step size at iteration `n` is `eta / (1 + γ * n)` where `eta` is the initial step size. The wrapped optimiser's step size is not modified.
+```
 
 ## Parameters
   - gamma (γ): Defaults to `0.001`
@@ -472,7 +473,7 @@ end
 """
   ExpDecay(eta, decay, decay_step, clip)
 
-Discount the learning rate `eta` by `decay` every `decay_step` till a minimum of `clip`.
+Discount the learning rate `eta` by a multiplicative factor `decay` every `decay_step` till a minimum of `clip`.
 
 ## Parameters
   - Learning Rate (eta): Defaults to `0.001`.