Guide · Open source
How to use llama2.c: build and run Llama 2 in pure C
Updated · written with AI from the project's own documentation and checked against our catalogue data
llama2.c is an open-source project by Andrej Karpathy that runs Llama 2 style language models with a single short C file and no external dependencies. This guide is for developers, students and curious readers who want to see how LLM inference works in plain C. By the end you will be able to clone the repository, build it, run a small pre-trained model and adjust the sampling options.
llama2.ckarpathy/llama2.c · ★ 20.2k · MIT Install options & downloads Quick start
- 1Clone the repository. Run git clone on karpathy/llama2.c and move into the llama2.c folder.
- 2Download the 15M model. Use wget to fetch stories15M.bin from the Hugging Face link in the README (about 60MB).
- 3Build and run. Run make run to compile the C code, then run ./run stories15M.bin to stream a sample.
- 4Try a bigger model. Download stories42M.bin and run it with ./run stories42M.bin for more coherent stories.
- 5Adjust sampling. Add -t, -p, -n or -i flags to change temperature, top-p, step count or prompt.
What you need before you start
The core inference program needs only a C compiler with make available. The README says the code has no dependencies, so you do not need to install a machine learning framework to run a model.
- A C compiler with make available
- git and wget, used in the commands below
- Python with pip only if you want to export Meta's Llama 2 checkpoints (the repo includes a requirements.txt for this)
Clone the repository and open the folder
Clone the repository to a folder where you keep your projects, then move into it. Everything else in this guide runs from inside this folder.
git clone https://github.com/karpathy/llama2.c.git
cd llama2.cDownload a small model and run it
The README suggests a 15M parameter model trained on the TinyStories dataset, which is roughly a 60MB download. Fetch it with wget, then compile and run the C code with make run. Running the run binary with the model file streams a sample story.
The README notes that on its author's M1 MacBook Air this runs at about 110 tokens per second. Speed will depend on your machine, and the Makefile has compile flags that can speed things up.
wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories15M.bin
make run
./run stories15M.binTry the larger 42M model
The README also offers a 42M parameter TinyStories model. It still runs at interactive rates and the README says it produces more coherent and diverse stories.
wget https://huggingface.co/karpathy/tinyllamas/resolve/main/stories42M.bin
./run stories42M.binControl the output with sampling flags
You can give the model a prompt and set options on the command line. The README's example uses temperature 0.8, 256 steps and a text prefix.
The README recommends sampling with -t 1.0 -p 0.9, which are the defaults. Top-p keeps very unlikely tokens from being sampled. To control diversity, vary either the temperature (-t between 0 and 1, with -p 0) or the top-p value (-p between 0 and 1, with -t 1), but not both.
- -t sets temperature
- -p sets top-p sampling
- -n sets the number of steps
- -i sets the prompt text
./run stories42M.bin -t 0.8 -n 256 -i "One day, Lily met a Shoggoth"Run Meta's Llama 2 models (advanced)
The architecture is identical to Meta's Llama 2, so you can load their models too. The README says you first obtain the checkpoints by following Meta's instructions, then convert them with export.py after installing the Python dependencies. The code only runs float32 inference, and the README says 13B and larger models do not currently work. A 7B export is about 26GB and the README describes it as too slow to use productively.
The README reports about 4 tokens per second for a 7B model on a Linux cloud machine with 96 threads and OpenMP. Its author said a MacBook Air M1 was closer to 30 seconds per token for the same model with make runfast.
- Install Python dependencies with pip install -r requirements.txt
- Export a checkpoint with python export.py llama2_7b.bin --meta-llama path/to/llama/model/7B
- Run the exported file with ./run llama2_7b.bin
pip install -r requirements.txt
python export.py llama2_7b.bin --meta-llama path/to/llama/model/7B
./run llama2_7b.binChat mode and Code Llama
The README says you can chat with Llama 2 chat models by exporting the chat model the same way and passing the -m chat flag. Code Llama support is described as incomplete, and the README says some hyperparameters differ so output may be a bit buggy. For Code Llama it says to build the tokenizer and pass it with -z.
./run llama2_7b_chat.bin -m chatWhat people use it for
- Watching how a Llama 2 style model generates text, one token at a time, in readable C code
- Learning the Llama 2 architecture end to end by reading run.c and the training code
- Experimenting with tiny models trained on narrow datasets such as TinyStories
- Trying a minimal inference engine with no external dependencies
Frequently asked questions
Is llama2.c free to use?
llama2.c is released under the MIT licence, according to its listing in this directory. The README does not state any fees for using it.
What do I need to install to run llama2.c?
The README lists a C compiler with make available as the main prerequisite. Python with pip is only needed to export Meta's Llama 2 checkpoints using requirements.txt.
Can llama2.c run Llama 2 7B or 13B?
The README says the code only does float32 inference and that 7B is hard to use productively. It also says 13B and larger do not currently work because of an integer overflow issue in pointer arithmetic that is yet to be fixed.
Is llama2.c good for production use?
No. The README describes the project as educational and float32-only, and says it is young and moving quickly. It is built for minimalism and learning how inference works.
How does llama2.c compare to llama.cpp?
The README says llama.cpp inspired the project. Compared to it, llama2.c hard-codes the Llama 2 architecture and uses one inference file of pure C with no dependencies, aiming for simplicity and education.
Which model should I try first?
The README suggests starting with the 15M parameter stories15M.bin model, which is about 60MB. It then suggests the 42M model for more coherent output.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use llama2.c if you want to read and run a small, dependency-free Llama 2 inference engine to learn how it works, or to experiment with tiny TinyStories-style models. Pick llama.cpp or another production-oriented runtime if you need to run 7B or larger models quickly or want a supported product.
Install options & downloads