Guide · Open source
How to use micrograd: install, run autograd and train a tiny net
Updated · written with AI from the project's own documentation and checked against our catalogue data
micrograd is a very small Python library that performs automatic differentiation, the calculus step that lets neural networks learn. It works on single scalar values and includes a small neural network layer with an API similar to PyTorch. This guide shows you how to install it, compute gradients, and train a small binary classifier so you can see backpropagation working.
microgradkarpathy/micrograd · ★ 18.6k · MIT Install options & downloads Quick start
- 1Install micrograd. Run pip install micrograd in a terminal with Python available.
- 2Import Value. Import Value from micrograd.engine to start building scalar computations.
- 3Run a forward pass. Combine Value objects with arithmetic operations and read the result with .data.
- 4Call backward. Call backward() on the final output to compute gradients for each input.
- 5Read the gradients. Print .grad on the inputs to see the derivative of the output with respect to each one.
- 6Open the demo notebook. Open demo.ipynb in Jupyter to see a 2-layer MLP trained as a binary classifier.
What micrograd does and who it is for
micrograd implements backpropagation (reverse-mode autodiff) over a dynamically built graph of operations. Both the engine and the neural network library are tiny, with about 100 and 50 lines of code respectively. The project describes itself as potentially useful for educational purposes.
The graph only operates over scalar values, so each neuron is broken into its individual tiny adds and multiplies. This makes it a good fit for students, developers reading a complete autograd implementation, and educators demonstrating training on small examples. It is not designed for large workloads.
- Students learning how backpropagation works under the hood
- Developers who want to read a short, complete autograd implementation
- Educators showing neural net training on small examples like binary classification
- Anyone curious how PyTorch-style APIs might be built
Install micrograd with pip
micrograd is available on PyPI. The only prerequisite named in the project is Python. Install it with pip from your terminal.
pip install microgradCompute gradients with the Value class
Import Value from micrograd.engine and wrap your numbers in it. Operations such as +, *, ** and relu() build a graph. Calling backward() on the final result fills in the .grad attribute of each input, which is the derivative of the output with respect to that input.
The example below is deliberately contrived to show a range of supported operations. The comments show the values the project reports when it runs.
- Use .data to read the forward-pass value of a node
- Use .grad after calling backward() to read the derivative
from micrograd.engine import Value
a = Value(-4.0)
b = Value(2.0)
c = a + b
d = a * b + b**3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + (b + a).relu()
d += 3 * d + (b - a).relu()
e = c - d
f = e**2
g = f / 2.0
g += 10.0 / f
print(f'{g.data:.4f}') # prints 24.7041, the outcome of this forward pass
g.backward()
print(f'{a.grad:.4f}') # prints 138.8338, i.e. the numerical value of dg/da
print(f'{b.grad:.4f}') # prints 645.5773, i.e. the numerical value of dg/dbTrain a small neural network classifier
The micrograd.nn module provides neurons, layers and a multi-layer perceptron (MLP). The demo notebook demo.ipynb walks through training a 2-layer MLP binary classifier. It uses a simple SVM-style max-margin loss and SGD for optimization.
In the demo, a 2-layer network with two 16-node hidden layers produces a decision boundary on the moon dataset. Open demo.ipynb in Jupyter to run the full training loop and see the result.
- Initialize the network from micrograd.nn
- Define a max-margin (SVM-style) binary classification loss
- Optimize with SGD
- Run demo.ipynb to see the moon dataset boundary
Visualize the computation graph
The notebook trace_graph.ipynb produces graphviz visualizations. The project's example builds a single 2D neuron and calls draw_dot on its output. Each node shows the data (left number) and the gradient (right number).
from micrograd import nn
n = nn.Neuron(2)
x = [Value(1.0), Value(-2.0)]
y = n(x)
dot = draw_dot(y)Run the tests
The unit tests use PyTorch as a reference for checking the computed gradients, so you must install PyTorch first. After that, run the tests from the project directory with pytest.
python -m pytestWhat people use it for
- Learning how backpropagation computes gradients by stepping through a small graph
- Reading a complete autograd implementation in roughly 100 lines
- Teaching neural net training with the moon dataset binary classification demo
- Visualizing a single neuron's data and gradients with graphviz via trace_graph.ipynb
Frequently asked questions
How do I install micrograd?
Install it from PyPI with pip install micrograd. The only prerequisite stated by the project is Python.
Is micrograd free to use?
micrograd is released under the MIT licence, as stated in the project's README. Check the licence file in the repository for the full terms.
Can micrograd train large models?
No. micrograd operates on scalar values one at a time, and the README does not claim performance for big workloads. It is meant to be read and understood rather than used for large-scale work.
Does micrograd support GPUs or tensors?
The README does not mention GPU or tensor-level computation. The library works on single scalar values.
What is micrograd compared to PyTorch?
micrograd is a tiny educational engine with a neural network API similar to PyTorch. PyTorch is a mature framework and is the usual choice for production machine learning pipelines.
How do I run micrograd's tests?
Install PyTorch first, because the tests use it as a reference for verifying gradients. Then run python -m pytest from the project directory.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
micrograd suits students, educators and developers who want to understand backpropagation by reading and running a very small codebase. If you need to train large models, use GPUs or tensors, or build a production pipeline, pick a mature framework such as PyTorch, JAX or tinygrad instead.
Install options & downloads