Guide · Open source
How to use llm.c: build, train GPT-2 and run the tests
Updated · written with AI from the project's own documentation and checked against our catalogue data
llm.c is an open-source project by Andrej Karpathy that trains GPT-2 style language models in plain C and CUDA instead of a large framework like PyTorch. This guide walks through cloning the repo, downloading the starter files, building and running training, and checking the code with the included tests. It is written for readers who are comfortable with a terminal and want to see how a model is trained at a low level.
llm.ckarpathy/llm.c · ★ 31.4k · MIT Install options & downloads Quick start
- 1Clone the repo. Run git clone https://github.com/karpathy/llm.c and change into the llm.c folder.
- 2Download the starter pack. Make the script executable with chmod u+x ./dev/download_starter_pack.sh, then run it to fetch the model, tokenizer and dataset files.
- 3Build the fp32 GPU binary. Run make train_gpt2fp32cu. This is the README's recommended 1 GPU, fp32 path.
- 4Start training. Run ./train_gpt2fp32cu. Requires an NVIDIA GPU with CUDA.
- 5Try CPU mode if you have no GPU. Run make train_gpt2 and then OMP_NUM_THREADS=8 ./train_gpt2. Expect slow progress, as the README warns.
- 6Run the tests. Run make test_gpt2 and ./test_gpt2 and check that it prints overall okay: 1.
What llm.c is and who it is for
llm.c is focused on pretraining, in particular reproducing the GPT-2 and GPT-3 model series. The mainline code is in train_gpt2.cu, and there is a simple reference CPU fp32 implementation in one file, train_gpt2.c, which the README says is about 1,000 lines. A parallel PyTorch reference is in train_gpt2.py.
The README describes the project as a slightly tweaked nanoGPT. The author states that llm.c's maintained code is C and CUDA only, so ports to other languages are expected to live in separate repos.
- Good fit: learning how a GPT-2 style model is trained at the lowest level
- Good fit: students studying how neural network layers are implemented in C or CUDA
- Good fit: people with an NVIDIA GPU who want to reproduce GPT-2 (124M) training
- Not a fit: a ready-to-use chatbot or inference app, since the README focuses on training
What you need before you start
The README lists Python for the dev and data scripts and for train_gpt2.py, a C compiler and make, and an NVIDIA GPU with CUDA for GPU training. A GPU is optional if you only want the CPU demo, but the README is clear that CPU training will not go far.
- Python (for dev/data scripts and train_gpt2.py)
- A C compiler and make
- An NVIDIA GPU with CUDA for GPU training (optional for the CPU demo)
- Optional for flash attention: cuDNN, and for multi-GPU training: MPI and NCCL
Clone the repository and download the starter pack
Clone the repo, then run the starter pack script. It downloads .bin files that get you started: the GPT-2 124M model in fp32 and bfloat16, a debug state used in unit testing, the GPT-2 tokenizer, and the tokenized tinyshakespeare dataset.
git clone https://github.com/karpathy/llm.c
cd llm.c
chmod u+x ./dev/download_starter_pack.sh
./dev/download_starter_pack.shBuild and run the 1 GPU fp32 training
The README recommends this path if you are not training across multiple nodes, are not interested in mixed precision, and want to learn CUDA. These fp32 files are earlier checkpoints of llm.c that the README describes as simpler, more portable and possibly easier to understand.
- Build with make train_gpt2fp32cu
- Run the resulting binary ./train_gpt2fp32cu
make train_gpt2fp32cu
./train_gpt2fp32cuReproduce the artifacts manually instead of using the script
If you would rather not run the .sh script, the README says you can recreate the same .bin files by installing the Python requirements and running the data and training scripts. The tinyshakespeare script downloads and tokenizes the dataset and writes .bin files that C can read easily.
- The .bin files start with a 1024-byte header, followed by tokens stored as uint16 using the GPT-2 tokenizer
- More datasets are available in /dev/data
pip install -r requirements.txt
python dev/data/tinyshakespeare.py
python train_gpt2.pyTry the CPU demo
The CPU path is for machines without a GPU. It fine-tunes GPT-2 small (124M) to produce Shakespeare-like text rather than training from scratch. The README says that unless you have a powerful CPU and raise the OpenMP thread count, you will not get far, so treat it as a demo and reference. The command below sets OMP_NUM_THREADS to 8, and the README notes you can increase it.
- The run trains for 40 steps with AdamW, batch size 4 and context length 64, then evaluates validation loss and samples text
chmod u+x ./dev/download_starter_pack.sh
./dev/download_starter_pack.sh
make train_gpt2
OMP_NUM_THREADS=8 ./train_gpt2Run the tests
The included unit test checks that the C code agrees with the PyTorch code. On the CPU, compile and run it. The test loads the debug state file, runs a forward pass, compares logits and loss with PyTorch, then runs 10 training iterations with Adam and checks the losses match. The README says the test should print overall okay: 1.
- For GPU tests, the README gives one fp32 command and one mixed precision cuDNN command
make test_gpt2
./test_gpt2Optional: flash attention with cuDNN and multi-GPU training
Flash attention from cuDNN is disabled by default because the README says it greatly increases compile time and the code path is new. To enable it you need cuDNN installed, plus the cuDNN frontend headers, which the Makefile looks for in your home directory or the current directory. For multi-GPU training, install MPI and NCCL first, then build and launch with mpirun. The README also mentions scripts under ./scripts/ for running this.
- Enable cuDNN flash attention with USE_CUDNN=1
- Set CUDNN_FRONTEND_PATH if the frontend is stored elsewhere
- Multi-node training is covered in the README's multi-node section, which lists three supported methods
make train_gpt2cu USE_CUDNN=1
make train_gpt2cu
mpirun -np <number of GPUs> ./train_gpt2cuWhat people use it for
- Reading the C code in train_gpt2.c to see how layers and training steps are written without a framework
- Reproducing GPT-2 (124M) training on an NVIDIA GPU using the starter pack
- Fine-tuning GPT-2 small on tinyshakespeare as a small CPU demo
- Comparing the C implementation against the PyTorch reference in train_gpt2.py using the test binary
- Following the layernorm tutorial in doc/layernorm/layernorm.md to learn how one GPT-2 layer is implemented
Frequently asked questions
Is llm.c free to use?
llm.c is released under the MIT licence, according to the project's repository metadata. The MIT licence generally allows use, modification and redistribution, including in commercial work, subject to its notice terms. Check the LICENSE file in the repo for the exact text.
What hardware do I need to run llm.c?
For GPU training you need an NVIDIA GPU with CUDA, plus a C compiler, make and Python. The README says a GPU is optional for the CPU demo, but CPU training will not go far, so a GPU is strongly preferred for real training.
Can I run llm.c without a GPU?
Yes, the README includes a CPU quick start that fine-tunes GPT-2 small on tinyshakespeare. The README warns you will not get far unless your CPU is powerful and you raise the OMP_NUM_THREADS setting.
Is llm.c a chatbot I can use right away?
No. The README focuses on pretraining, and the project is a training codebase rather than a ready-to-use chatbot or inference app. It is better suited to learning and reproducing model training.
How does llm.c compare to PyTorch?
The README says llm.c is a bit faster than PyTorch Nightly, by about 7%, and that the project uses no PyTorch or cPython runtime. It also ships a parallel PyTorch reference in train_gpt2.py so you can compare results directly.
What are alternatives to llm.c?
The README and project metadata list nanoGPT, PyTorch, llama.cpp and minGPT as comparable projects. nanoGPT is the closest, since the README says llm.c is a slightly tweaked version of it. Choose PyTorch-based tools if you need the Python ecosystem, which the README says this repo does not maintain.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
llm.c is a strong choice if you want to read and run a GPT-2 style training loop written in plain C and CUDA, especially with an NVIDIA GPU. Skip it if you need a ready-to-use chatbot, practical CPU-only training, or Python-based tooling, and consider nanoGPT or PyTorch instead.
Install options & downloads