tinygrad
A small, readable deep learning library in Python that sits between PyTorch and micrograd.
- Stars
- 33.8k
- Forks
- 4.4k
- Licence
- MIT
- Language
- Python
- Last update
- 4 days ago
- Type
- Library
What it is
tinygrad is a Python library for building and training neural networks. It provides a tensor type with automatic differentiation, a compiler that fuses operations into faster kernels, and modules for layers, optimizers and datasets. It aims to be much smaller and easier to read than PyTorch while keeping a similar API.
- Train a small neural network on MNIST using the included example script
- Compare gradients from tinygrad against PyTorch using the README's side-by-side example
- Inspect generated kernels by changing the DEBUG level when running a matmul
- Run tensor operations on supported accelerators such as CUDA, Metal or AMD
Great for
- Developers who want to read and modify how a deep learning stack works internally
- Students learning how autograd and tensor compilers function
- Researchers experimenting with low-level kernel code or new hardware backends
- People who want a PyTorch-like training loop with a small codebase
Not ideal for
- Teams needing a mature, production-proven ecosystem; the README says the library is not 1.0 yet
- Users who need the broad set of pretrained models and tooling that PyTorch offers
- Beginners looking for a high-level tutorial-first framework; the docs are external and the README is terse
How to get it
- 1
Clone and build from source
You'll need: Python 3 · pip
Clone the repository
git clone https://github.com/tinygrad/tinygrad.gitEnter the directory
cd tinygradInstall in editable mode
python3 -m pip install -e . - 2
Run it
- Install with the commands above
- Check the default device with: python3 -c "from tinygrad import Device; print(Device.DEFAULT)"
- Run the small tensor example from the README comparing gradients with PyTorch
- Try examples/beautiful_mnist.py for a full training run
- 3
Get the most out of it
- Set DEBUG=3 or DEBUG=4 when running code to see the fused kernels and generated code
- Use the Context(TRAINING=1) block to enable training mode, as in the README example
- Use TinyJit to capture and replay kernels, as described in the README comparison with JAX
- Read the docs at docs.tinygrad.org for a quick start guide
Downloads
Strengths
- The compiler and IR are visible and hackable, according to the README
- Its API is designed to resemble PyTorch, so familiar training loops carry over
- Supports many backends including CPU, CUDA, Metal, OpenCL, AMD, NV and WebGPU
Trade-offs
- The README says it is not 1.0 yet, so APIs may change
- Some features such as full vmap and pmap are not yet available, per the README comparison with JAX
- Contribution rules are strict: large diffs, docs-only changes and AI-written PRs are closed without review
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| tinygrad (this) | 33.8k | Library | intermediate | 4 days ago | Developers who want to read and modify how a deep learning stack works internally |
| micrograd | 18.6k | Library | beginner | 3 months ago | Students learning how backpropagation works under the hood |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.