update docs for lrn #5913
update docs for lrn #5913
Conversation
src/operator/lrn-inl.h
Outdated
| @@ -31,11 +31,11 @@ struct LRNParam : public dmlc::Parameter<LRNParam> { | |||
| uint32_t nsize; | |||
| DMLC_DECLARE_PARAMETER(LRNParam) { | |||
| DMLC_DECLARE_FIELD(alpha).set_default(1e-4f) | |||
| .describe("value of the alpha variance scaling parameter in the normalization formula"); | |||
| .describe("Alpha variance scaling parameter in the normalization formula."); | |||
There was a problem hiding this comment.
Here "alpha is used as an adjective. I think it might be more intuitive to write:
"The variance scaling parameter alpha from the normalization formula."
Where is the normalization formula? How does the user find what alpha refers to? Should we have some guidance for where this reference "normalization formula" points to? What are your thoughts??
There was a problem hiding this comment.
Hey any luck resolving this one? Thanks!
There was a problem hiding this comment.
LRN expression is in the description of the function
src/operator/lrn-inl.h
Outdated
| DMLC_DECLARE_FIELD(beta).set_default(0.75f) | ||
| .describe("value of the beta power parameter in the normalization formula"); | ||
| .describe("Beta power parameter in the normalization formula."); |
There was a problem hiding this comment.
Same issue - let's make it consistent with however we do alpha.
src/operator/lrn-inl.h
Outdated
| DMLC_DECLARE_FIELD(knorm).set_default(2.0f) | ||
| .describe("value of the k parameter in normalization formula"); | ||
| .describe("k parameter in normalization formula."); |
src/operator/lrn.cc
Outdated
| .add_arguments(LRNParam::__FIELDS__()) | ||
| .describe("Apply convolution to input then add a bias."); | ||
| .describe(R"code(Applies Local Response Normalization to the input. |
There was a problem hiding this comment.
Let's not have this in capital letters. Just
"Applies local response normalization to the input."
The main reasons to use capital letters would be if we are introducing a proper noun or an acronym.
src/operator/lrn.cc
Outdated
| .describe("Apply convolution to input then add a bias."); | ||
| .describe(R"code(Applies Local Response Normalization to the input. | ||
|
|
||
| Normalization helps in generalization and used to prevent neurons from |
There was a problem hiding this comment.
Maybe be more descriptive here. Not all neurons can saturate. Also there is a grammatical bug on "used to" (suggests in the past, not utility).
Perhaps: "Normalization helps to prevent squashing neurons (like tanh) from saturating. It can also help for transfer learning, as the inputs for each task will exhibit the same dynamic range."
There was a problem hiding this comment.
LRN is applied on a ReLU neuron. Are ReLU neurons also considered squashing neurons ?
There was a problem hiding this comment.
what I was aiming for was to write a sentence about Normalization and then describe LRN. I will remove it if this leads to confusion.
You are right that ReLU neurons don't saturate and might turn into on negative inputs.
|
No but they don't saturate! I'm referring specifically to the comment about
normalization. Perhaps it was copied from the doc for a different
activation function that actually does saturate?
…On Thu, Apr 20, 2017 at 1:49 PM, Naveen Swamy ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In src/operator/lrn.cc
<#5913 (comment)>:
> .add_arguments(LRNParam::__FIELDS__())
-.describe("Apply convolution to input then add a bias.");
+.describe(R"code(Applies Local Response Normalization to the input.
+
+Normalization helps in generalization and used to prevent neurons from
LRN is applied on a ReLU neuron. Are ReLU neurons also considered
squashing neurons ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5913 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACR4zrbe7IwspDGuuM0ht0jeT7bLBgvyks5rx5qYgaJpZM4NC5_0>
.
|
* InstanceNorm doc modified * minor change
There was a problem hiding this comment.
Thanks @zackchase, I have made changes per your suggestions.
src/operator/lrn-inl.h
Outdated
| @@ -31,11 +31,11 @@ struct LRNParam : public dmlc::Parameter<LRNParam> { | |||
| uint32_t nsize; | |||
| DMLC_DECLARE_PARAMETER(LRNParam) { | |||
| DMLC_DECLARE_FIELD(alpha).set_default(1e-4f) | |||
| .describe("value of the alpha variance scaling parameter in the normalization formula"); | |||
| .describe("Alpha variance scaling parameter in the normalization formula."); | |||
There was a problem hiding this comment.
LRN expression is in the description of the function
* update doc for lrn * [Documentation] Ndarray instancenorm doc modified (apache#6008) * InstanceNorm doc modified * minor change * xx
@mli, @zackchase, @madjam, @Roshrini , @indhub @jiajiechen
Please review