New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

在FFN中的问题 #13

Open

LLLLLLoki opened this issue Aug 18, 2018 · 1 comment

LLLLLLoki commented Aug 18, 2018

论文中我看到的FFN是FFN（X） = ReLU(XW1)W2,为什么在代码中的_ffn_layer 中的linear 函数里会有tf.nn.convolution？只有_linear_2d 没有卷积函数其他的都有

Collaborator

Playinf commented Aug 26, 2018

3D的输入可以看作1 * 1的卷积，实际上都是线性映射，只是使用convolution避免过多的reshape操作。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment