Neural Networks: Zero to Hero
A free video course that builds neural networks and GPT-style language models from scratch in Jupyter notebookkarpathy/nn-zero-to-hero
- Stars
- 25.2k
- Forks
- 3.7k
- Licence
- MIT
- Language
- Jupyter Notebook
- Last update
- 2.2 years ago
- Type
- Learning resource
No commits in over a year — check for forks or alternatives before adopting.
What it is
This is a course on neural networks that starts from the basics. It is a series of YouTube videos in which the author writes and trains neural networks step by step, and the notebooks built in those videos are stored in this repository. Each lecture covers a topic such as backpropagation, language modeling, batch normalization, and the GPT transformer architecture.
- Work through the micrograd notebook to see backpropagation on scalar values
- Follow the makemore notebooks to build a character-level language model
- Complete the exercises listed in each lecture's video description
- Study the minBPE tokenizer code linked from the final lecture
Great for
- Learners with basic Python who want to understand how neural networks train
- People who prefer following along with video and notebooks
- Students who want to see backpropagation and GPT implemented by hand
- Developers curious how tokenizers and language models work internally
Not ideal for
- Readers who want a text-only reference, since the README points to YouTube videos for the explanations
- People with no Python experience, since lecture 1 assumes basic Python and some calculus recollection
- Anyone looking for a finished, production-ready library; the README says the course 'may grow into something more respectable' and describes itself as ongoing
How to get it
- 1
Clone and build from source
You'll need: Python · Jupyter Notebook · PyTorch (implied by the notebooks; version not stated in README)
Clone the repository
git clone https://github.com/karpathy/nn-zero-to-hero - 2
Run it
- Clone the repository and open the lectures directory
- Open the notebook for Lecture 1 (micrograd) in Jupyter
- Watch the matching YouTube lecture alongside the notebook
- Work through the exercises listed in that lecture's video description
- 3
Get the most out of it
- Pause the videos and attempt the exercises before watching the answers, as the README recommends for Lecture 5
- Use the linked Google Colab exercise for Lecture 5 if you prefer not to set up locally
- Follow the makemore lectures in order before Lecture 7 (GPT), since the README says they are assumed background
- Study the minBPE repository linked in Lecture 8 for a standalone tokenizer implementation
Downloads
Strengths
- Free, MIT-licensed material with notebooks you can run and modify
- Builds models step by step, showing the reasoning behind each part
- Covers a progression from basic backpropagation to GPT and tokenization
Trade-offs
- Explanations live mainly in videos, so the README alone gives limited detail
- The README says the course is ongoing and some topics are marked as not yet covered
- Depends on PyTorch and Jupyter, which the README does not set up in detail
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| nn-zero-to-hero (this) | 25.2k | Learning resource | intermediate | 2.2 years ago | Learners with basic Python who want to understand how neural networks train |
| micrograd | 18.6k | Library | beginner | 3 months ago | Students learning how backpropagation works under the hood |
| minGPT | 25k | Library | intermediate | 2.2 years ago | Students who want to read a working GPT implementation end to end |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.