Skip to content
#

cosine-decay

Star

Here are 2 public repositories matching this topic...

pablo-reyes8 / implementing-gpt

Star 0

Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers

nlp transformers pytorch gpu-acceleration language-model adamw gpt2 gpt3 cosine-decay decoder-only educational-implementation
  • Updated Dec 7, 2025
  • Python

Improve this page

Add a description, image, and links to the cosine-decay topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the cosine-decay topic, visit your repo's landing page and select "manage topics."

Learn more