Free, self-paced curriculum from undergraduate foundations through PhD-level depth and Staff/Principal engineer expertise, plus a hands-on course where you build your own LLM system end to end. Free learning routes with optional paid reference books.
Study Plans
·
Course
·
Report Bug
·
Prerequisites
- Features
- Tracks
- Prerequisite Graph
- Study Plans
- Course: Build Your Own LLM System
- Role Paths
- OpenLearn: the Desktop App
- Read Offline (Single PDF)
- Interviews
- Practice
- Case Studies
- Books and Resources
- Agent Skill
- License
- Build-your-own LLM system course 12 passes from math 0 to 1 through tokenizers, attention, KV caches, a Rust serving engine, a Go gateway, durable workflows, agents, and a local Kubernetes deploy, graded by
ol checktests and runnable milestones - Free-first free learning routes, with paid books identified as optional references
- Graded depth undergraduate foundations through PhD-level research topics
- Multi-track math, algorithms, ML, systems, info theory, competitive programming
- Interview-ready 19 company-specific guides across Big Tech, quant, and frontier
- Role paths reading orders for one job across tracks, with checkable done-when criteria, in the terminal (
ol learn) and as a per-role PDF - Practice path one CLI, three exercise kinds (predict, build, reattempt), attempts cloned to a gitignored scratchpad
- Case studies real builds generalized into runnable, dependency-free implementations that test themselves
- Prerequisite graph mermaid diagram showing topic dependencies
- Single-PDF book the entire curriculum renders into one PDF in CI (build artifact + release asset)
Start with the end-to-end CS curriculum for an eight-term core, applied mathematics, specialization paths, and assessment requirements. See the source register for verified book references, access labels, and audit boundaries.
| # | Topic | Textbook | Time |
|---|---|---|---|
| 00 | Precalculus | OpenStax Precalculus 2e (free) | 2-3 weeks |
| 01 | Calculus 1 | OpenStax Calculus Vol 1 | 4 weeks |
| 02 | Calculus 2 | OpenStax Calculus Vol 2 | 4 weeks |
| 03 | Linear Algebra | Hefferon's Linear Algebra | 4 weeks |
| 04 | Calculus 3 | OpenStax Calculus Vol 3 | 4 weeks |
| 05 | Discrete Math 1 | Hammack's Book of Proof | 4 weeks |
| 06 | Discrete Math 2 | Levin's Discrete Mathematics | 4 weeks |
| 07 | Probability & Statistics | Grinstead & Snell + OpenStax Stats | 3-4 weeks |
| 08 | Matrix Calculus & Autodiff | Parr & Howard + Baydin et al. (free) | 3 weeks |
| 09 | Numerical Methods & Floating Point | Goldberg (free) + Higham | 3-4 weeks |
| 10 | Optimization | Boyd & Vandenberghe (free) | 3 weeks |
| 11 |
| # | Topic | Lang | README |
|---|---|---|---|
| 01 | Arrays & Hashing | JS | Patterns |
| 02 | Two Pointers & Sliding Window | JS | Patterns |
| 03 | Binary Search | JS | Patterns |
| 04 | Linked Lists | JS | Patterns |
| 05 | Trees | Python | Patterns |
| 06 | Graphs | JS/TS/C | Patterns |
| 07 | Dynamic Programming | JS/Python/C | Patterns |
| 08 | Greedy | JS/Python/Java | Patterns |
| 09 | Backtracking | JS/Python | Patterns |
| 10 | Math & Bit Manipulation | JS | Patterns |
| 11 | Recursion & Divide-and-Conquer | Python | Patterns |
| 15 | Probabilistic Structures | Python | Patterns |
| 16 | Systems Data Structures | C/Rust/Go | Patterns |
Core references: CLRS (4th ed.), Competitive Programmer's Handbook (free). See algorithms/README.md for full details.
| # | Topic | Textbook | Time |
|---|---|---|---|
| 01 | Statistical Learning | ISLR (free) + ESL (free) | 4-5 weeks |
| 02 | Deep Learning | Goodfellow et al. on deeplearningbook.org (free) | 6-8 weeks |
| 03 | Reinforcement Learning | Sutton & Barto (free) + Spinning Up | 4-6 weeks |
| 04 | LLM Systems & Inference | ML Systems (free) + vLLM docs (free) | 4-6 weeks |
| 05 | Foundation Models & Architectures | AI Engineering (Huyen) + aie-book (free) | 4-5 weeks |
| 06 | Neural Architectures & DL History | CS231n (free) + AlexNet/LSTM papers (free) | 3-4 weeks |
| 07 | Training & Post-Training | Chinchilla + DPO papers (free) + PyTorch/JAX docs (free) | 4-6 weeks |
| 08 | tinyllm: Build an LLM from Scratch | the course spine: chapters and course tests | course passes 1 to 11 |
| # | Topic | Primary Reference | Time |
|---|---|---|---|
| 01 | Foundations | Data Engineering Cookbook (free) + Fundamentals of Data Engineering | 2-3 weeks |
| 02 | Storage & Warehousing | DDIA Ch 3 + The Data Warehouse Toolkit (Kimball) | 3-4 weeks |
| 03 | Batch & Streaming | Streaming Systems + Spark/Kafka docs (free) | 3-4 weeks |
| 04 | Orchestration & Modeling | dbt + Airflow docs (free) | 2-3 weeks |
| 05 | Corpus Pipeline | FineWeb + datatrove (free) | course passes 3, 8 |
| # | Topic | Reference | Time |
|---|---|---|---|
| 01 | System Design | ByteByteGo + System Design Primer (free) | 4-5 weeks |
| 02 | Software Architecture | Software Architecture Patterns (O'Reilly) | 2-3 weeks |
| 03 | Cloud Native | Cloud Native DevOps with K8s + K8s docs (free) | 3-4 weeks |
| 04 | Observability | Observability Engineering + Google SRE Book (free) | 2-3 weeks |
| 05 | Incident Response & Chaos | Google SRE Book + SRE Workbook (free) | one drill per course pass |
Taught as patterns, not products — like math, once you understand the fundamental patterns (cache-aside, consistent hashing, durable execution, the worker pool, retrieve-then-rerank, the complexity ladder of authorization, cross-entropy-as-compression), every trendy tool becomes a recognizable instance.
| # | Topic | Reference | Time |
|---|---|---|---|
| 01 | Training & Frameworks | PyTorch + JAX + vLLM docs (free) | 4-6 weeks |
| 02 | RPC & Protocols | gRPC + Protocol Buffers docs (free) | 2-3 weeks |
| 03 | Streaming & SSE | HTML SSE spec + MDN SSE (free) | 1-2 weeks |
| 04 | Distributed Data & Caching | Redis + Cassandra docs (free) + DDIA | 3-4 weeks |
| 05 | Durable Orchestration & Workers | Temporal + DBOS docs (free) | 2-3 weeks |
| 06 | Coding & Design Patterns | Refactoring Guru + Mostly Adequate Guide (free) | 2-3 weeks |
| 07 | Retrieval & RAG | RAG paper + BM25 + pgvector (free) | 3-4 weeks |
| 08 | Authorization & Access Control | NIST RBAC/ABAC/NGAC + Zanzibar (free) | 2-3 weeks |
| 09 | LLM Evaluation | MacKay + HELM + Ragas (free) | 2-3 weeks |
| 10 | Edge, Realtime & On-Device Inference | llama.cpp + Mistral 7B + Mamba (free) | 2-3 weeks |
| 11 | Model Routing & Cascades | RouteLLM + FrugalGPT + LLMRouterBench (free) | 1-2 weeks |
| 12 | Gateway | OpenAI API reference + W3C Trace Context (free) | course passes 1, 7, 10 |
| 13 | Agent SDK | Building effective agents + ReAct (free) | course pass 10 |
Three distinct "distributed" problems kept apart on purpose: distributed training (parallelism — PyTorch/JAX, FSDP/ZeRO, Megatron, Ray Train; served with vLLM/TensorRT-LLM; plus embeddings and small language models), distributed data (sharding, replication, in-memory caching with Redis and cache-invalidation patterns, Cassandra, knowledge graphs via Apache AGE, OLAP vs OLTP), and distributed orchestration (durable execution and the worker pattern — Temporal/Cadence/DBOS — with distributed observability and profiling). Plus the connective tissue: gRPC and serialization, SSE token streaming, the coding/design patterns these systems are built from, production-grade retrieval (encoders, chunking, hybrid BM25 + vector + graph fusion, multimodal, metadata/permission filtering), authorization (RBAC/ABAC/NGAC and the pushdown-automata complexity ladder), evaluation (cross-entropy, perplexity, bits-per-byte, LLM-as-judge, RAG faithfulness), and edge/realtime inference (the end-to-end llama.cpp + GGUF local path, streaming speech encoders, and the efficiency architectures — sliding-window attention, GQA, MoE, Mamba/SSM — behind Mistral's models).
| Track | Reference | Time |
|---|---|---|
| Competitive Programming | CP Handbook (free) + CSES Problem Set | 6-8 weeks |
| Information Theory | Student's Guide to Coding & Info Theory + MacKay (free) | 3-4 weeks |
| Software Craftsmanship | The Pragmatic Programmer + SWE at Google (free) + project postmortems; now also diagramming, documentation, and type systems | 3-4 weeks + 4-5 weeks |
| Responsible AI | Model Cards + Datasheets + NIST AI RMF (free) | course passes 3, 9, 10, 11 |
| Infrastructure | K8s + Kafka + Lucene/Elasticsearch + Apache docs (free) | 5-7 weeks |
| Field Engineering | Discovery, sizing, performance testing engagements, POCs, migration, commercials, escalation for inference-cloud customers | 3-4 weeks |
| Case Studies | Worked builds with runnable, dependency-free implementations | 5-10 days |
graph TD
C1[Calculus 1] --> C2[Calculus 2]
C2 --> C3[Calculus 3]
LA[Linear Algebra] --> C3
D1[Discrete Math 1] --> D2[Discrete Math 2]
C2 --> PS[Prob & Stats]
D1 --> PS
D1 --> ALG[Algorithms]
D2 --> ALG
ALG --> CP[Competitive Programming]
D1 --> PL[Type Systems]
ALG --> PL
PS --> SL[Statistical Learning]
LA --> SL
SL --> DL[Deep Learning]
C3 --> DL
DL --> RL[Reinforcement Learning]
PS --> RL
DL --> FM[Foundation Models]
DL --> TPT[Training & Post-Training]
RL --> TPT
TPT --> LLM
PS --> IT[Information Theory]
LA --> IT
ALG --> SYS[Systems & Architecture]
SYS --> DE[Data Engineering]
FM --> LLM[LLM Systems & Inference]
DL --> LLM
SYS --> LLM
DE --> LLM
SYS --> INFRA[Infrastructure]
DE --> INFRA
LLM --> AIP[AI Platform Engineering]
LLM --> FE[Field Engineering]
AIP --> FE
DE --> RAI[Responsible AI]
LLM --> RAI
SYS --> AIP
DE --> AIP
See STUDY-PLAN.md for structured schedules:
- Math Foundations 16 weeks, all 7 math topics
- Algorithm Mastery 17 days, all 15 algorithm topics
- ML & AI 18-37 weeks, statistical learning through RL, LLM serving, foundation models, neural-architecture foundations, and training and post-training
- Systems & Architecture 11-15 weeks, system design through observability
- Data Engineering 10-14 weeks, foundations through orchestration and data quality
- AI Platform Engineering 25-35 weeks, patterns-first: training/frameworks, RPC, streaming, distributed data & caching, durable orchestration, coding/design patterns, retrieval & RAG, authorization, LLM evaluation, edge/realtime inference, and model routing
- Type Systems (Software Craftsmanship 07) 2-3 weeks, type systems and polymorphism from the lambda-calculus ladder through Hindley-Milner inference, variance, and generics across five languages
- Diagramming & Documentation (Software Craftsmanship 05 and 06) 2 weeks, C4 diagramming-as-code and docs-as-code
- Infrastructure 5-7 weeks, containers and Kubernetes through messaging, search, and the Apache stack (partitioned workers now live in AI Platform 05)
- Combined Path 40+ weeks, zero to Staff+ interview-ready
- PhD Research Track deep learning research + information theory + RL
One system, built end to end in Python, Rust, and Go and graded by the harness: a byte-level language model trained on your own numerics, a Rust engine streaming completions, a Go gateway, and the containers, traces, and runbooks that run it on Kubernetes. Python and Rust exchange safetensors, tokenizer.json, and fixtures at file boundaries. Optional C modules are standalone exercises with their own test binaries. Each chapter teaches one module from first principles; ol check <ID> grades your code against course tests, and each built pass ends at a milestone that runs your system. Pass 12 is currently a design plan.
| Pass | Path | Gate |
|---|---|---|
| P0 | Setup: Python and numpy, shell and make, your repo and its CI gate | MS-P0 |
| P1 | The tracer: byte bigram, Rust engine, Go gateway, kind, one trace, one drill, one ADR | MS-P1 |
| P2 | Foundations: autograd, gradient checking, and the training loop | MS-P2 |
| P3 | Tokens and data: tokenizers, text corpora, and n-gram models | MS-P3 |
| P4 | Sequence models: recurrent models, sequence-to-sequence, and beam search | MS-P4 |
| P5 | Transformer: transformer variants, fine-tuning, and model loading | MS-P5 |
| P6 | Inference and kernels: Python inference stack and optional standalone C exercises | MS-P6 |
| P7 | Serving platform: Rust engine, Go gateway, routing, load, deployment, and observability | MS-P7 |
| P8 | Durable workflows: durable data and training workflows | MS-P8 |
| P9 | Capstone training: train, evaluate, release, and serve a small model | MS-P9 |
| P10 | Agents: agent SDK, retrieval, evaluation, and usage policy | MS-P10 |
| P11 | Operate: migrations, upgrades, chaos, security, and release readiness | MS-P11 |
| P12 | Multimodal plan: planned image and audio extensions; module chapters and implementation are not built yet | planned, not built |
Start at paths/course and its system map:
practice/bin/ol doctor && practice/bin/ol course init --name <system> && practice/bin/ol learn course nextapp/ is a desktop learning platform that renders a course pack: a directory of markdown with an openlearn.json manifest naming its tracks, paths, and practice CLI. This repository is the pack built into it, and packs/languide/ (communication-first language guides) ships beside it. The app reads the same .done files ol learn writes, generates timed exercises from the chapter you are reading, and grades them against a hidden rubric with a local or hosted model.
cd app && make init && make devThe whole curriculum builds into one PDF -- every track and topic in learning order, with the C4 architecture diagrams and Mermaid diagrams rendered inline.
-
From CI: every run produces a
supersource-curriculum-pdfbuild artifact. -
From a release:
supersource-curriculum.pdfand the generatedsupersource-<path>.pdffiles are attached to each GitHub Release for every course pass inpaths/. -
Locally:
./scripts/build-book.sh # -> outputs/supersource-curriculum.pdf ./scripts/build-book.sh --path course-p01-tracer # -> outputs/supersource-course-p01-tracer.pdf ./scripts/build-book.sh --help # options (--skip-mermaid, --no-toc, ...)
Requires
pandocand a LaTeX engine (xelatex). For full fidelity also installlibrsvg(embeds the C4 SVGs) andmermaid-filter(renders Mermaid); without them the build still succeeds with diagrams shown as code.
Company-specific guides for 19 companies. See interviews/README.md.
- Big Tech & AI Labs Anthropic, Google, DeepMind, OpenAI, Meta, Apple, NVIDIA, Moonshot
- Infrastructure & Data Netflix, Amazon, Databricks, Stripe, Palantir
- Quant & Trading Jane Street, Citadel, Two Sigma, HRT, Renaissance
- Frontier SpaceX
The tracks above are the learning path: you read them. practice/ is the practice path: it tells you whether you actually know it. One CLI drives it, and everything you write goes to a gitignored .scratchpad/, never into the repo.
practice/bin/ol list # every exercise, and which you have started
practice/bin/ol start predict go 01 # clone it into .scratchpad/ and open it
practice/bin/ol check predict go 01 # exit code is the verdictThree kinds of exercise, distinguished only by what you produce:
| Kind | You produce | It is wrong when |
|---|---|---|
predict |
The exact output you expect from a snippet | Your prediction differs from what ran |
build |
An implementation from scratch | The exercise's own assertions fail |
reattempt |
A second solution to something you already solved | The original's assertions fail |
Predict-then-run is the retention drill and the place to start: 24 snippets across Go, Rust, Python, and TypeScript where you write the exact expected output before running, and the diff tells you which part of your mental model is wrong. Every snippet is checked in CI for determinism, wall time, and peak memory.
Build is 10 graded exercises per language, covering data structures, algorithms, concurrency, and language-specific idioms. Each reference implementation is standard library only and passes its own assertions in CI.
| Languages | Focus | |
|---|---|---|
| Core | Go, Rust, Python, TypeScript, C, C++ | What the curriculum targets, and what CI verifies |
| Optional | Zig, Scala, Java | Worth doing, but not in use here; excluded from the default run |
Real builds, generalized. Each takes a system designed under a deadline, strips it to the transferable concepts, and records the order it was actually assembled in. Every implementation is standard library only and runs its own tests: python <file>.py. See case-studies/README.md.
| # | Study | What it teaches | Runnable |
|---|---|---|---|
| 01 | Order Book Matching | Two orderings, two structures; earning a heap then a tick-indexed ladder via BUD | matching_engine.py |
| 02 | Grounded SQL Agent | Schema grounding measured by ablation, bounded self-correction, a deterministic safety gate | safety_gate.py |
| 03 | Exactly-Once Event API | Constraint-based dedup, atomic claims, and what at-most-once actually buys | event_api.py |
| 04 | K-Means Optimization | Initialization over iteration, and stating an optimization's regime | kmeans_ladder.py |
| 05 | Agent Evaluation Harness | Outcome, evidence, and grade kept apart; replay before grading; a checker proved by oracles and mutants | eval_harness.py |
See the source register for author/publisher evidence and the distinction between free books, course materials, and paid companions. Mathematics for Machine Learning connects the foundational math tracks to regression, PCA, and optimization.
All primary textbooks are free. Recommended (non-free) books are listed separately.
| Book | Track | Link |
|---|---|---|
| OpenStax Calculus Volumes 1-3 | Math | openstax.org |
| Book of Proof by Hammack | Math | Free PDF |
| Discrete Mathematics: An Open Introduction by Levin | Math | Free online |
| Linear Algebra by Hefferon | Math | Free PDF |
| Introduction to Probability by Grinstead & Snell | Math | Free PDF |
| Competitive Programmer's Handbook by Laaksonen | Competitive Programming | Free PDF |
| Introduction to Statistical Learning (ISLR) | ML/AI | Free online |
| Elements of Statistical Learning (ESL) | ML/AI | Free PDF |
| Deep Learning by Goodfellow, Bengio, Courville | ML/AI | Free online |
| RL: An Introduction by Sutton & Barto | ML/AI (RL) | Free online |
| OpenAI Spinning Up in Deep RL | ML/AI (RL) | Free online |
| Machine Learning Systems by Reddi | ML/AI (LLM Systems) | Free online |
| AI Engineering companion (aie-book) by Huyen | ML/AI (Foundation Models) | Free on GitHub |
| Stanford CRFM Foundation Models report | ML/AI (Foundation Models) | Free PDF |
| The Data Engineering Cookbook by Kretz | Data Engineering | Free on GitHub |
| Info Theory, Inference, & Learning by MacKay | Information Theory | Free online |
| Programming Language Foundations in Agda (PLFA) by Wadler et al. | Programming Languages | Free online |
| Software Foundations (Vol. 1-2) by Pierce et al. | Programming Languages | Free online |
| Software Engineering at Google by Winters, Manshreck, Wright | Software Craftsmanship | Free online |
| Google SRE Book | Systems | Free online |
| System Design Primer | Systems | Free on GitHub |
| The C4 model by Simon Brown | Diagramming & Documentation | Free online |
| Diátaxis documentation framework | Diagramming & Documentation | Free online |
| Google Technical Writing Courses | Diagramming & Documentation | Free online |
| Kubernetes Documentation | Infrastructure / Systems | Free online |
| Apache Kafka Documentation | Infrastructure / Data Engineering | Free online |
| Apache Lucene & Elasticsearch Guide | Infrastructure | Lucene / ES Guide |
| PyTorch / JAX / vLLM Documentation | AI Platform Engineering | PyTorch / JAX / vLLM |
| gRPC + Protocol Buffers Documentation | AI Platform Engineering | grpc.io / protobuf.dev |
| MDN Server-Sent Events + WHATWG HTML spec | AI Platform Engineering | MDN / Spec |
| Temporal / DBOS / Cassandra / Apache AGE Docs | AI Platform Engineering | Temporal / DBOS / Cassandra / AGE |
| Redis Documentation | AI Platform Engineering | Free online |
| RAG paper + BM25 review + pgvector | AI Platform Engineering | RAG / BM25 / pgvector |
| NIST RBAC/ABAC/NGAC + Google Zanzibar | AI Platform Engineering | NIST / Zanzibar |
| HELM + lm-evaluation-harness + Ragas | AI Platform Engineering | HELM / harness / Ragas |
| llama.cpp + GGUF + Ollama | AI Platform Engineering | llama.cpp / GGUF / Ollama |
| Ray (Core / Train / Serve) Documentation | AI Platform Engineering | Free online |
| NVIDIA TensorRT-LLM + Triton Inference Server | AI Platform Engineering | TensorRT-LLM / Triton |
| Mistral 7B / Mixtral / Mamba papers | AI Platform Engineering | Mistral 7B / Mixtral / Mamba |
| Book | Track | Why |
|---|---|---|
| Introduction to Algorithms (CLRS) 4th ed. | Algorithms | The definitive algorithms reference, formal proofs and correctness |
| Types and Programming Languages (TAPL) by Pierce | Programming Languages | The definitive type-systems text: lambda calculus, System F, subtyping, inference |
| The Pragmatic Programmer by Hunt & Thomas (20th Anniversary) | Software Craftsmanship | The foundational text on professional software development |
| Designing Data-Intensive Applications by Kleppmann | Systems / Data Engineering | The industry bible for distributed systems |
| Fundamentals of Data Engineering by Reis & Housley | Data Engineering | The definitive lifecycle-oriented introduction |
| The Data Warehouse Toolkit by Kimball | Data Engineering | The canonical text on dimensional modeling |
| Streaming Systems by Akidau et al. | Data Engineering | The Dataflow model that unifies batch and streaming |
| Designing Machine Learning Systems by Huyen | ML/AI | Production ML from data to deployment and serving |
| AI Engineering by Chip Huyen (2025) | ML/AI | Optional paid book; the linked official repository provides free companion materials |
| ByteByteGo System Design | Systems | Visual system design walkthroughs |
| Software Architecture Patterns by Richards | Systems | Concise pattern catalog for architecture decisions |
| Cloud Native DevOps with Kubernetes 2nd ed. | Systems | Hands-on K8s from dev through production |
| Observability Engineering by Majors et al. | Systems | Modern observability beyond the three pillars |
| Probability & Statistics for Engineering by Devore | Math | Rigorous engineering-focused probability |
| A Student's Guide to Coding and Information Theory | Information Theory | Accessible introduction with worked examples |
| Neo4j Graph Algorithms | Algorithms (Graphs) | Applied graph algorithms at scale |
This repo's conventions are available as portable agent skills in skills/.