This repository is an independent attempt to implement a Generative Pretrained Transformer (GPT) using common Python libraries. The project draws inspiration from the explanatory video Deep Dive into LLMs like ChatGPT by Andrew Karparthy and leverages iterative insights from ChatGPT.
The goal of this project is to build a simplified version of a GPT-like model that can understand and generate natural language. It serves as an educational tool and a foundation for further research and experimentation in the field of natural language processing (NLP) and machine learning.
- Modular Design: Components are organized to allow easy experimentation with different architectures and components.
- Common Libraries: Utilizes popular Python libraries for numerical computing, data manipulation, and visualization.
- Educational Focus: Provides clear documentation and interactive examples to help understand the inner workings of transformer models.
The design of this project is heavily influenced by:
- Andrew Karparthy's Video: Deep Dive into LLMs like ChatGPT offers an in-depth look at how large language models function.
- ChatGPT Assistance: Iterative guidance and refinement from ChatGPT have been invaluable in shaping the approach and implementation details.
To get started with the project, follow these steps:
-
Clone the repository:
git clone https://github.com/valiferst/generative-pretrained-transformer.git cd generative-pretrained-transformer