Skip to content
Get packs

micrograd

A tiny educational autograd engine and small neural net library with a PyTorch-like API.

GitHub Download source
Stars
18.6k
Forks
3k
Licence
MIT
Language
Jupyter Notebook
Last update
3 months ago
Type
Library

What it is

micrograd is a very small Python library that performs automatic differentiation, the calculus step that lets neural networks learn. It works on single scalar values, breaking every operation into tiny adds and multiplies, and it includes a small neural network layer on top with an API similar to PyTorch. The README describes it as potentially useful for educational purposes, and it is meant to be read and understood rather than used for large-scale work.

  • Compute a forward pass and gradients for a small mathematical expression
  • Train a 2-layer neural network binary classifier on the moon dataset using the included demo notebook
  • Visualize a computation graph with graphviz using the trace_graph notebook
  • Study the code of a tiny autograd engine as a learning exercise

Great for

  • Students learning how backpropagation works under the hood
  • Developers who want to read a complete autograd implementation in a short codebase
  • Educators demonstrating neural net training on small examples like binary classification
  • Anyone curious about how PyTorch-style APIs might be built

Not ideal for

  • Training large models, since it operates on scalar values one at a time and the README does not claim performance for big workloads
  • Production machine learning pipelines, where mature frameworks such as PyTorch are the usual choice
  • Users who need GPU or tensor-level computation, which the README does not mention
Step-by-step guide: how to use microgradInstall, first run, examples, tips and FAQs
micrograd vs tinygradmicrograd vs Neural Networks: Zero to Hero

How to get it

  1. 1

    Install with a package manager

    You'll need: Python

    Install from PyPI

    pip install micrograd
  2. 2

    Run it

    • Install the package with pip install micrograd
    • Run the example from the README importing Value from micrograd.engine
    • Open demo.ipynb to see a full binary classifier training run
    • Open trace_graph.ipynb to generate a graphviz visualization
  3. 3

    Get the most out of it

    • Read the engine source alongside the README example to see how gradients are accumulated
    • Use the draw_dot function from trace_graph.ipynb to inspect data and gradient values per node
    • Follow the linked microgpt gist to see the same ideas applied to a GPT-style transformer
    • Run python -m pytest after installing PyTorch to verify gradient correctness

Downloads

Source code (.zip)master branch · straight from GitHub

Strengths

  • Very small codebase, which the README says is about 100 lines for the engine and 50 for the nn library
  • Includes runnable notebooks for training and graph visualization
  • MIT licensed and easy to install with pip

Trade-offs

  • Scalar-only operations make it far too slow and limited for real-world model sizes
  • Running the tests requires installing PyTorch separately
  • The README is brief and offers little beyond the example code and notebooks

How it compares

ProjectStarsTypeLevelUpdatedBest for
micrograd (this)18.6kLibrarybeginner3 months agoStudents learning how backpropagation works under the hood
tinygrad33.8kLibraryintermediate4 days agoDevelopers who want to read and modify how a deep learning stack works internally

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.