Guide · Open source

How to use tinygrad: install, run a tensor and train a network

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

tinygrad is an open-source Python library for building and training neural networks. It provides a Tensor type with automatic differentiation, a compiler that fuses operations into faster kernels, and modules for layers, optimizers and datasets. This guide shows you how to install it from source, run a small tensor calculation, train a simple network and understand where it fits compared with PyTorch.

tinygradtinygrad/tinygrad · ★ 33.8k · MIT Install options & downloads

Quick start

1Clone the repository. Run git clone https://github.com/tinygrad/tinygrad.git to download the source code.
2Enter the directory. Run cd tinygrad to move into the project folder.
3Install in editable mode. Run python3 -m pip install -e . so changes to the source take effect without reinstalling.
4Check the default device. Run the Device.DEFAULT one-liner to see which accelerator tinygrad will use.
5Run a tensor example. Try the autograd example with Tensor and backward(), then compare it with the PyTorch version in the README.
6Train a network. Adapt the LinearNet training loop to your own data, replacing the random placeholder inputs with a real data loader.

Install tinygrad from source

The current recommended way to install tinygrad is from source. You need Python 3 and pip. Clone the repository, enter the directory and install it in editable mode.

Prerequisites: Python 3 and pip
Editable mode means changes you make to the code take effect without reinstalling

git clone https://github.com/tinygrad/tinygrad.git
cd tinygrad
python3 -m pip install -e .

Check which accelerator tinygrad will use

tinygrad supports several accelerators, including CPU, OpenCL, METAL, CUDA, AMD, NV, QCOM and WEBGPU. After installing, you can check the default device with a one-line command.

Supported backends listed in the README: OpenCL, CPU, METAL, CUDA, AMD, NV, QCOM, WEBGPU

python3 -c "from tinygrad import Device; print(Device.DEFAULT)"

Run your first tensor calculation

tinygrad uses lazy evaluation, so operations are combined before they run. The README example below computes a matrix multiply and sums the result. Setting DEBUG to 3 prints information about the work, and DEBUG to 4 shows the generated code.

Use DEBUG=4 to see the generated code instead of DEBUG=3

DEBUG=3 python3 -c "from tinygrad import Tensor;
N = 1024; a, b = Tensor.empty(N, N), Tensor.empty(N, N);
(a.reshape(N, 1, N) * b.T.reshape(1, N, N)).sum(axis=2).realize()"

Compute gradients with autograd

tinygrad computes gradients with its autograd feature. The example below builds a small calculation, calls backward() and prints the gradients. The README shows the same calculation in PyTorch so you can compare the two.

from tinygrad import Tensor

x = Tensor.eye(3).clone()  # clone to make it a buffer
y = Tensor([[2.0,0,-2.0]])
z = y.matmul(x).sum()
z.backward()

print(x.grad.tolist())  # dz/dx
print(y.grad.tolist())  # dz/dy

Train a small neural network

A neural network in tinygrad is a class with a __call__ method that uses Tensor operations. You create an optimizer from nn.optim, then run a training loop inside a Context set to TRAINING=1. The README example uses random data as a placeholder, so replace it with a real data loader for actual training.

The full MNIST example in examples/beautiful_mnist.py is described in the README as reaching 98% in about 5 seconds.

Replace the random inputs with a real MNIST data loader
See examples/beautiful_mnist.py for the complete version

from tinygrad import Tensor, nn, Context

class LinearNet:
  def __init__(self):
    self.l1 = Tensor.kaiming_uniform(784, 128)
    self.l2 = Tensor.kaiming_uniform(128, 10)
  def __call__(self, x:Tensor) -> Tensor:
    return x.flatten(1).dot(self.l1).relu().dot(self.l2)

model = LinearNet()
optim = nn.optim.Adam([model.l1, model.l2], lr=0.001)

x, y = Tensor.rand(4, 1, 28, 28), Tensor([2,4,3,7])  # replace with real mnist dataloader

with Context(TRAINING=1):
  for i in range(10):
    optim.zero_grad()
    loss = model(x).sparse_categorical_crossentropy(y).backward()
    optim.step()
    print(i, loss.item())

Know the contribution rules before opening a pull request

The README asks contributors to keep changes small and readable. Bug fixes with a regression test are welcome, and PRs that look complex or add many lines are unlikely to be reviewed. Any claimed speedup must be benchmarked, and the README says AI-written submissions will be closed without feedback.

Install the pre-commit hooks before contributing. They run the linter, mypy and a subset of the tests on each commit.

Bug fixes with a regression test are welcomed
Benchmark any speedup claim
Disclose any AI use in your contribution

python3 -m pip install -e '.[testing]' # install extra deps for testing

What people use it for

Reading and modifying how a deep learning stack works internally
Studying how autograd and tensor compilers function
Experimenting with low-level kernel code or new hardware backends
Running a PyTorch-like training loop with a small codebase

Frequently asked questions

Is tinygrad free to use?

tinygrad is released under the MIT licence, according to the project summary. The MIT licence permits use, modification and redistribution, and the repository is open source.

What do I need to install tinygrad?

The README lists Python 3 and pip as prerequisites. The recommended method is to clone the repository and install it with python3 -m pip install -e .

Is tinygrad production ready?

The README says the library is not 1.0 yet. The project is better suited to learning, research and experimentation than to teams that need a mature, production-proven ecosystem.

How does tinygrad compare to PyTorch?

tinygrad aims to be much smaller and easier to read than PyTorch while keeping a similar API. The README notes that PyTorch has a broader set of pretrained models and tooling, which tinygrad does not match.

Why does my tinygrad code not use my GPU?

The README says you can check the default accelerator with python3 -c "from tinygrad import Device; print(Device.DEFAULT)". The source does not give further troubleshooting steps, so check the documentation at docs.tinygrad.org and the Discord for help.

What are alternatives to tinygrad?

The README names PyTorch, micrograd, JAX and TVM as comparable projects. PyTorch suits teams that need a large ecosystem, micrograd suits learning autograd on a very small scale, and JAX and TVM focus on transforms and compilation.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

tinygrad suits developers, students and researchers who want to read, modify and experiment with how a deep learning stack works, and who are comfortable with a library that is not yet 1.0. If you need a mature, production-proven framework with many pretrained models and tooling, PyTorch is the better choice.

Install options & downloads