You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I recently discovered this excellent repository for learning basic concepts in ML and noticed a potential problem in the implementation of dropout wrapper. In particular, this is the line of code I am confused about:
Shouldn't the gradient from a later layer also apply the mask from the one used in forward()? Otherwise, dLdy will overestimate the true gradient after dividing by the probability 1.0 - self.p. Not sure this is an issue though as I am a beginner in ML.
The text was updated successfully, but these errors were encountered:
Hi, I recently discovered this excellent repository for learning basic concepts in ML and noticed a potential problem in the implementation of dropout wrapper. In particular, this is the line of code I am confused about:
numpy-ml/numpy_ml/neural_nets/wrappers/wrappers.py
Line 231 in b0359af
Shouldn't the gradient from a later layer also apply the mask from the one used in
forward()
? Otherwise,dLdy
will overestimate the true gradient after dividing by the probability1.0 - self.p
. Not sure this is an issue though as I am a beginner in ML.The text was updated successfully, but these errors were encountered: