Skip to content
Get packs

nanochat

A minimal, hackable harness for training and chatting with a small GPT-style language model.

GitHub Download source
Stars
59.6k
Forks
8.4k
Licence
MIT
Language
Python
Last update
2 months ago
Type
AI model / agent

What it is

nanochat is a codebase for training a large language model end to end, covering tokenization, pretraining, finetuning, evaluation, and inference. It is designed to run on a single GPU node and is meant to be small and easy to modify. The README says you can train a GPT-2-class model for about $48 of cloud GPU time and then talk to it through a command-line chat.

  • Run the speedrun script on an 8x H100 node to train a GPT-2-capability model and chat with it
  • Run small experiments such as a depth-12 model for about 5 minutes of pretraining to test a code change
  • Monitor training metrics like val_bpb and CORE score in wandb plots
  • Study the code to learn how tokenization, pretraining, and finetuning are implemented

Great for

  • Students who want to see how a full LLM training pipeline fits together
  • Developers who want a readable, modifiable LLM training codebase
  • Researchers running small experiments on compute-optimal model sizes
  • People with access to a multi-GPU cloud node who want a reproducible GPT-2-scale training run

Not ideal for

  • People who want a polished chatbot to use day to day: the README says the speedrun model is like talking to a kindergartener
  • People without GPU access who expect strong results: the CPU/MPS path shrinks the model and the README says results will be weak
  • Anyone needing production-grade support: questions are routed to DeepWiki, Discussions, and Discord rather than a formal support channel
Step-by-step guide: how to use nanochatInstall, first run, examples, tips and FAQs
llm.c vs nanochat

How to get it

  1. 1

    Clone and build from source

    You'll need: uv (Python dependency manager) · Python · CUDA-capable GPU (or CPU/MPS for a reduced run)

    Install dependencies with GPU support

    uv sync --extra gpu

    Install dependencies for CPU or MPS

    uv sync --extra cpu

    Activate the virtual environment

    source .venv/bin/activate
  2. 2

    Run it

    • Install dependencies with uv sync using the gpu or cpu extra
    • Activate the virtual environment
    • Run bash runs/speedrun.sh on an 8x H100 node (about 1.5 hours per the README)
    • Chat with the trained model using python -m scripts.chat_cli
  3. 3

    Get the most out of it

    • Use --depth to scale the model; the single dial sets the other hyperparameters automatically
    • Lower --device-batch-size (default 32) if you run out of VRAM on GPUs with less than 80GB
    • Use the dev extra (uv sync --extra gpu --group dev) to add pytest, matplotlib, and transformers
    • Track runs with wandb using the metrics val_bpb, core_metric, and train/mfu described in the README
    • Try runs/runcpu.sh to experiment on CPU or Apple Silicon at a smaller scale

Downloads

Source code (.zip)master branch · straight from GitHub

Strengths

  • Covers the full LLM pipeline in one codebase, according to the README
  • A single --depth setting automatically sets the other hyperparameters
  • Includes a public leaderboard that tracks the wall-clock time to reach GPT-2-level capability

Trade-offs

  • Needs substantial GPU hardware for the main workflow: the README describes an 8x H100 node, and GPUs with under 80GB may need tuning
  • The README says the CPU and Apple Silicon path produces weak results and the author has not tested all non-CUDA backends
  • Cost figures are estimates tied to provider pricing at the time of writing and depend on your cloud setup

How it compares

ProjectStarsTypeLevelUpdatedBest for
nanochat (this)59.6kAI model / agentadvanced2 months agoStudents who want to see how a full LLM training pipeline fits together
nanoGPT64.4kLibraryintermediate12 months agoDevelopers who want to read a compact GPT training implementation
llm.c31.4kLearning resourceadvanced1.4 years agoDevelopers who want to see how a GPT-2 style model is trained at the lowest level

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.