Commits

Branch selector

 main

User selector

lucidrains

Datepicker

All time

Commit History

Commits on May 25, 2021

• give tiny attention to Local SGU too, appropriately also sliding window local attention

  lucidrains

  committed
  efee809

  Copy full SHA for efee809

• prepare for local attention (to be paired with local SGU)

  lucidrains

  committed
  da23bcd

  Copy full SHA for da23bcd

Commits on May 24, 2021

• update readme

  lucidrains

  committed
  39e82ec

  Copy full SHA for 39e82ec

• add causal tiny attention, for use with nonlocal SGU

  lucidrains

  committed
  68bdf0c

  Copy full SHA for 68bdf0c

Commits on May 23, 2021

• allow for customizable gating activation in SGU, fix bug with padding

  lucidrains

  committed
  18efacd

  Copy full SHA for 18efacd

Commits on May 21, 2021

• assert

  lucidrains

  committed
  bd9b68a

  Copy full SHA for bd9b68a

• cleanup

  lucidrains

  committed
  f559f96

  Copy full SHA for f559f96

• cache causal masks

  lucidrains

  committed
  90af62a

  Copy full SHA for 90af62a

• easier hyperparams

  lucidrains

  committed
  9cff5f4

  Copy full SHA for 9cff5f4

• use feedforwards for sibling block in reversible net, to save on memory

  lucidrains

  committed
  b5c2b32

  Copy full SHA for b5c2b32

• demonstrate working 16k context length

  lucidrains

  committed
  b52c61a

  Copy full SHA for b52c61a

Commits on May 20, 2021

• fix remaining bugs with axial folds

  lucidrains

  committed
  246fc35

  Copy full SHA for 246fc35

• add ability to message pass every num tokens, smoothed out with a depthwise conv at the end

  lucidrains

  committed
  ed0b718

  Copy full SHA for ed0b718

• add reversible class

  lucidrains

  committed
  b028ac8

  Copy full SHA for b028ac8

• add ability to specify different window sizes for the SGU per layer

  lucidrains

  committed
  7642e36

  Copy full SHA for 7642e36

• release

  lucidrains

  committed
  dd945c8

  Copy full SHA for dd945c8

• release

  lucidrains

  committed
  98fd5a4

  Copy full SHA for 98fd5a4

• Create python-publish.yml

  lucidrains

  authored
  d142b1d

  Copy full SHA for d142b1d

• complete sliding window SGU

  lucidrains

  committed
  a2f065d

  Copy full SHA for a2f065d

• enwik8

  lucidrains

  committed
  29f9252

  Copy full SHA for 29f9252

• first commit, without sliding window

  lucidrains

  committed
  7bc3c7f

  Copy full SHA for 7bc3c7f

• add some scaffold

  lucidrains

  committed
  ae1deea

  Copy full SHA for ae1deea

• readme

  lucidrains

  authored
  76f0fd5

  Copy full SHA for 76f0fd5

• readme

  lucidrains

  authored
  099c0ba

  Copy full SHA for 099c0ba

• Initial commit

  lucidrains

  authored
  2b41b5d

  Copy full SHA for 2b41b5d