Transformer-Based Models (BERT for NLP)

In this lab, we explore the use of transformer-based models, particularly BERT, for various Natural Language Processing (NLP) tasks.
The lab introduces the workflow of leveraging pre-trained BERT embeddings, fine-tuning BERT for classification, and extending the model with an autoregressive head.

---

Key Concepts

• Transformer architecture and self-attention
• Tokenization and embedding
• Fine-tuning pre-trained BERT models
• Transfer learning in NLP
• Model evaluation using real-world datasets

---

Tools & Libraries

• Python
• PyTorch
• Hugging Face Transformers
• Pandas, NumPy
• Matplotlib, scikit-learn

---

---

Author

Negin Ebrahimi

---