transformer-memory

A transformer with a long-term memory mechanism Download link: https://shibamoulilahiri.github.io/gutenberg_dataset.html

Name		Name	Last commit message	Last commit date
Latest commit History 54 Commits
skip		skip
transformer-distilation		transformer-distilation
.gitignore		.gitignore
README.md		README.md
book_perplexity.ipynb		book_perplexity.ipynb
dead_memories.ipynb		dead_memories.ipynb
experiment.py		experiment.py
gpt2.ipynb		gpt2.ipynb
hopfield_attention.ipynb		hopfield_attention.ipynb
hopfield_memory.py		hopfield_memory.py
imports_notebook.py		imports_notebook.py
k_means.ipynb		k_means.ipynb
main.ipynb		main.ipynb
modeling_gpt2.py		modeling_gpt2.py
original_modeling_gpt2.py		original_modeling_gpt2.py
perplexity.py		perplexity.py
softmax.ipynb		softmax.ipynb
something.py		something.py
sweep.py		sweep.py
targets.py		targets.py
testing.ipynb		testing.ipynb
token_position.ipynb		token_position.ipynb
utils.py		utils.py
viz.py		viz.py

Provide feedback