Skip to content

Commit 9b2dc5a

Browse files
committed
fix gmlp comment
1 parent 90c755b commit 9b2dc5a

File tree

2 files changed

+9
-7
lines changed

2 files changed

+9
-7
lines changed

docs/transformers/gmlp/index.html

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -102,10 +102,12 @@ <h2>gMLP Block</h2>
102102
<div class='section-link'>
103103
<a href='#section-2'>#</a>
104104
</div>
105-
<p> <code class="highlight"><span></span><span class="n">d_model</span></code>
106-
is the dimensionality (<span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal">d</span></span></span></span>) of <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span></span></span></span> <code class="highlight"><span></span><span class="n">d_ffn</span></code>
107-
is the dimensionality of <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.07153em;">Z</span></span></span></span> <code class="highlight"><span></span><span class="n">seq_len</span></code>
108-
is the length of the token sequence (<span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathnormal">n</span></span></span></span>)</p>
105+
<ul><li><code class="highlight"><span></span><span class="n">d_model</span></code>
106+
is the dimensionality (<span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.69444em;vertical-align:0em;"></span><span class="mord mathnormal">d</span></span></span></span>) of <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.07847em;">X</span></span></span></span> </li>
107+
<li><code class="highlight"><span></span><span class="n">d_ffn</span></code>
108+
is the dimensionality of <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.68333em;vertical-align:0em;"></span><span class="mord mathnormal" style="margin-right:0.07153em;">Z</span></span></span></span> </li>
109+
<li><code class="highlight"><span></span><span class="n">seq_len</span></code>
110+
is the length of the token sequence (<span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.43056em;vertical-align:0em;"></span><span class="mord mathnormal">n</span></span></span></span>)</li></ul>
109111

110112
</div>
111113
<div class='code'>

labml_nn/transformers/gmlp/__init__.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,9 +47,9 @@ class GMLPBlock(nn.Module):
4747

4848
def __init__(self, d_model: int, d_ffn: int, seq_len: int):
4949
"""
50-
`d_model` is the dimensionality ($d$) of $X$
51-
`d_ffn` is the dimensionality of $Z$
52-
`seq_len` is the length of the token sequence ($n$)
50+
* `d_model` is the dimensionality ($d$) of $X$
51+
* `d_ffn` is the dimensionality of $Z$
52+
* `seq_len` is the length of the token sequence ($n$)
5353
"""
5454
super().__init__()
5555
# Normalization layer fro Pre-Norm

0 commit comments

Comments
 (0)