Skip to content

FTA #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 23, 2022
Merged

FTA #115

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
b8db27d
fta implementation
vpj Apr 1, 2022
30c31df
rope fix
vpj Apr 5, 2022
5ea794b
Merge branch 'master' into fta
vpj Apr 10, 2022
f64e300
fta exp
vpj May 15, 2022
c3529ee
fix
vpj May 15, 2022
2d130a0
Merge branch 'master' into fta
vpj May 15, 2022
04f8a39
docs
vpj May 23, 2022
8ebeb80
fta docs
vpj May 23, 2022
4db81ff
fta notebook
vpj May 23, 2022
2d0a591
version
vpj May 23, 2022
1c04f7d
Merge branch 'master' into fta
vpj May 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
833 changes: 833 additions & 0 deletions docs/activations/fta/experiment.html
View file
Open in desktop

Large diffs are not rendered by default.

402 changes: 402 additions & 0 deletions docs/activations/fta/index.html
View file
Open in desktop

Large diffs are not rendered by default.

23 changes: 13 additions & 10 deletions docs/activations/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,24 @@
<head>
<meta http-equiv="content-type" content="text/html;charset=utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta name="description" content=""/>
<meta name="description" content="A set of PyTorch implementations/tutorials related to neural network activations"/>

<meta name="twitter:card" content="summary"/>
<meta name="twitter:image:src" content="https://avatars1.githubusercontent.com/u/64068543?s=400&amp;v=4"/>
<meta name="twitter:title" content="__init__.py"/>
<meta name="twitter:description" content=""/>
<meta name="twitter:title" content="Neural Network Activation Functions"/>
<meta name="twitter:description" content="A set of PyTorch implementations/tutorials related to neural network activations"/>
<meta name="twitter:site" content="@labmlai"/>
<meta name="twitter:creator" content="@labmlai"/>

<meta property="og:url" content="https://nn.labml.ai/activations/index.html"/>
<meta property="og:title" content="__init__.py"/>
<meta property="og:title" content="Neural Network Activation Functions"/>
<meta property="og:image" content="https://avatars1.githubusercontent.com/u/64068543?s=400&amp;v=4"/>
<meta property="og:site_name" content="LabML Neural Networks"/>
<meta property="og:type" content="object"/>
<meta property="og:title" content="__init__.py"/>
<meta property="og:description" content=""/>
<meta property="og:title" content="Neural Network Activation Functions"/>
<meta property="og:description" content="A set of PyTorch implementations/tutorials related to neural network activations"/>

<title>__init__.py</title>
<title>Neural Network Activation Functions</title>
<link rel="shortcut icon" href="/icon.png"/>
<link rel="stylesheet" href="../pylit.css?v=1">
<link rel="canonical" href="https://nn.labml.ai/activations/index.html"/>
Expand Down Expand Up @@ -64,14 +64,17 @@
</div>
</div>
<div class='section' id='section-0'>
<div class='docs'>
<div class='docs doc-strings'>
<div class='section-link'>
<a href='#section-0'>#</a>
</div>

<h1>Neural Networks Activations</h1>
<ul><li><a href="fta/index.html">Fuzzy Tiling Activations</a> </li>
<li>🚧 <a href="swish/index.html">Swish</a></li></ul>

</div>
<div class='code'>
<div class="highlight"><pre><span class="lineno">1</span><span></span><span class="kn">from</span> <span class="nn">.swish</span> <span class="kn">import</span> <span class="n">Swish</span></pre></div>
<div class="highlight"><pre><span class="lineno">14</span><span></span><span class="kn">from</span> <span class="nn">.swish</span> <span class="kn">import</span> <span class="n">Swish</span></pre></div>
</div>
</div>
<div class='footer'>
Expand Down
2 changes: 2 additions & 0 deletions docs/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ <h4>✨ <a href="adaptive_computation/index.html">Adaptive Computation</a></h4>
<ul><li><a href="adaptive_computation/ponder_net/index.html">PonderNet</a></li></ul>
<h4>✨ <a href="uncertainty/index.html">Uncertainty</a></h4>
<ul><li><a href="uncertainty/evidence/index.html">Evidential Deep Learning to Quantify Classification Uncertainty</a></li></ul>
<h4>✨ <a href="activations/index.html">Activations</a></h4>
<ul><li><a href="activations/fta/index.html">Fuzzy Tiling Activations</a></li></ul>
<h2>Highlighted Research Paper PDFs</h2>
<ul><li><a href="https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2204.10628.pdf">Autoregressive Search Engines: Generating Substrings as Document Identifiers</a> </li>
<li><a href="https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2203.15556.pdf">Training Compute-Optimal Large Language Models</a> </li>
Expand Down
5 changes: 2 additions & 3 deletions docs/normalization/deep_norm/experiment.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,8 @@
<div class='section-link'>
<a href='#section-0'>#</a>
</div>
<h1>DeepNorm Experiment</h1>
<p><a href="https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/normalization/deep_norm/experiment.ipynb"><img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg"></a> <a href="https://app.labml.ai/run/ec8e4dacb7f311ec8d1cd37d50b05c3d"><img alt="View Run" src="https://img.shields.io/badge/labml-experiment-brightgreen"></a> <a href="https://www.comet.ml/labml/deep-norm/61d817f80ff143c8825fba4aacd431d4?experiment-tab=chart&showOutliers=true&smoothing=0&transformY=smoothing&xAxis=step"><img alt="Open In Comet" src="https://images.labml.ai/images/comet.svg?experiment=deep_norm&file=experiment"></a></p>

<h1><a href="index.html">DeepNorm</a> Experiment</h1>
<p><a href="https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/normalization/deep_norm/experiment.ipynb"><img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg"></a> <a href="https://app.labml.ai/run/ec8e4dacb7f311ec8d1cd37d50b05c3d"><img alt="View Run" src="https://img.shields.io/badge/labml-experiment-brightgreen"></a></p>
</div>
<div class='code'>
<div class="highlight"><pre><span class="lineno">15</span><span></span><span class="kn">import</span> <span class="nn">copy</span>
Expand Down
22 changes: 18 additions & 4 deletions docs/sitemap.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,21 @@

<url>
<loc>https://nn.labml.ai/activations/index.html</loc>
<lastmod>2021-01-25T16:30:00+00:00</lastmod>
<lastmod>2022-05-23T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>


<url>
<loc>https://nn.labml.ai/activations/fta/index.html</loc>
<lastmod>2022-05-23T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>


<url>
<loc>https://nn.labml.ai/activations/fta/experiment.html</loc>
<lastmod>2022-05-23T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>

Expand Down Expand Up @@ -197,7 +211,7 @@

<url>
<loc>https://nn.labml.ai/normalization/deep_norm/experiment.html</loc>
<lastmod>2022-04-23T16:30:00+00:00</lastmod>
<lastmod>2022-05-23T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>

Expand Down Expand Up @@ -295,7 +309,7 @@

<url>
<loc>https://nn.labml.ai/index.html</loc>
<lastmod>2022-05-03T16:30:00+00:00</lastmod>
<lastmod>2022-05-23T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>

Expand Down Expand Up @@ -589,7 +603,7 @@

<url>
<loc>https://nn.labml.ai/transformers/rope/index.html</loc>
<lastmod>2022-02-23T16:30:00+00:00</lastmod>
<lastmod>2022-04-05T16:30:00+00:00</lastmod>
<priority>1.00</priority>
</url>

Expand Down
Loading