Skip to content

README.md

Latest commit

 

History

History
33 lines (20 loc) · 1.62 KB

README.md

File metadata and controls

33 lines (20 loc) · 1.62 KB

Encoders Explained

Python PyTorch

Encoder diagram

Understand the transformer architecture by learning about encoders with detailed explanations on the architecture and a mini-project

How to use

This repository is not made to use the code inside in itself, but as a summary of differents classes and papers you can find on the internet. It is a complete guide to understand the basics, but in details, of how encoders within the Transformer architecture work and how they can be used as a standalone architecture for certain tasks.

You will find:

  1. A explanations.ipynb notebook in which you will find all the information about encoders and their code implementation.

  2. A mini-project folder in which you will find a code and a cleaner code for the implemenation of the encoder which can be called directly like a library

References

Original Paper

  • Vaswani, A., et al. (2017). "Attention Is All You Need". arXiv:1706.03762. [Paper]

Video Resources

  • Hugging Face. (2022). "Transformer: encoder". [YouTube]
  • Machine Learning Studio. "A Dive Into Multihead Attention, Self-Attention and Cross-Attention". [YouTube]
  • Machine Learning Studio. "Self-Attention Using Scaled Dot-Product Approach". [YouTube]