minGPT vs train-llm-from-scratch
Which should you use? A side-by-side look at what each one does, who it suits, what it costs, how hard it is to set up, and where each falls short.
Open source · MIT
A small, readable PyTorch re-implementation of GPT for training and running language models.
Open source · MIT
A from-scratch PyTorch walkthrough for training a small language model, from raw text to text generation.
The short answer
Choose minGPT if you need…
- Students who want to read a working GPT implementation end to end
- Developers who want a small, hackable base for experimenting with transformer models
- People who want to load GPT-2 weights and generate text with PyTorch
- Researchers who prefer a minimal codebase over a large framework
Choose train-llm-from-scratch if you need…
- Students who want to learn how a transformer is built and trained step by step
- Developers with Python and PyTorch experience who want readable reference code
- Researchers interested in how SFT, reward models, DPO and GRPO are implemented from scratch
- People with no GPU who want to run a small CPU training example
Side-by-side
| minGPT | train-llm-from-scratch | |
|---|---|---|
| Type | Library | Learning resource |
| Price | Free and open source | Free and open source |
| Licence | MIT | MIT |
| GitHub stars | 25k | 13.1k |
| Setup difficulty | Intermediate | Intermediate |
| How you get it | Clone and build from source | Clone and build from source |
| Main language | Python | Python |
| Last code update | 2 years ago | 35 days ago |
What each one is
minGPT
minGPT is a compact PyTorch codebase that reimplements the GPT transformer model, covering both training and inference. The README describes it as roughly 300 lines of model code, split into a model file, a byte pair encoder, and a generic training loop, aimed at being easy to read and learn from. It is a library you import into your own Python project, with demos for tasks like addition and character-level text generation.
train-llm-from-scratch
This is a tutorial repository that builds a transformer language model in plain PyTorch and walks through the steps to train it: preparing text data, pretraining a base model, generating text, and then post-training steps such as fine-tuning and reinforcement learning. The README says it can run a small model on a laptop CPU, and larger ones on a GPU. It is aimed at people who want to see how each piece works rather than calling a ready-made library.
Best for
minGPT
- Students who want to read a working GPT implementation end to end
- Developers who want a small, hackable base for experimenting with transformer models
- People who want to load GPT-2 weights and generate text with PyTorch
- Researchers who prefer a minimal codebase over a large framework
train-llm-from-scratch
- Students who want to learn how a transformer is built and trained step by step
- Developers with Python and PyTorch experience who want readable reference code
- Researchers interested in how SFT, reward models, DPO and GRPO are implemented from scratch
- People with no GPU who want to run a small CPU training example
Strengths
minGPT
- Small enough to read in an afternoon, with the model in about 300 lines according to the README
- Clear separation between model, tokenizer, and training loop
- MIT licensed and widely referenced in notebooks, blogs, and courses
train-llm-from-scratch
- Every algorithm is written out by hand in plain PyTorch, which makes the code easier to read and study
- The README is organized as a sequence of steps with explanations and expected outputs
- Provides small smoke-test configs and a CPU path so beginners can try it without a large GPU
Not ideal for
minGPT
- People who need an actively maintained project: the README says minGPT is in a semi-archived state and points to nanoGPT for newer work
- Anyone needing distributed training or mixed precision, which the README lists as todos
- Users looking for a ready-made chatbot or polished end-user app
train-llm-from-scratch
- Anyone who needs a production-ready LLM or chat product, since the README presents this as a learning path
- People who need to fine-tune an existing pretrained model with standard libraries, because the README says it avoids trl, peft and transformers on purpose
- Readers without basic Python, neural network or PyTorch knowledge, since the README lists these as prerequisites
Trade-offs
minGPT
- Semi-archived as of January 2023; the author suggests nanoGPT for newer development
- Test coverage is described as limited, and there is no requirements.txt yet per the README
- Lacks mixed precision and distributed training, which are listed as future todos
train-llm-from-scratch
- The README says it needs a GPU for larger models, and a billion-parameter model will not fit on a T4
- Written from scratch rather than with standard libraries, so it is not a drop-in tool for production work
- The README is truncated here, so details of the full setup and later steps are not confirmed in this entry
Getting started
minGPT
Clone the repository
git clone https://github.com/karpathy/minGPT.gitEnter the directory
cd minGPTtrain-llm-from-scratch
Clone the repository
git clone https://github.com/FareedKhan-dev/train-llm-from-scratch.gitEnter the folder
cd train-llm-from-scratchFrequently asked questions
What is the difference between minGPT and train-llm-from-scratch?
minGPT: A small, readable PyTorch re-implementation of GPT for training and running language models. train-llm-from-scratch: A from-scratch PyTorch walkthrough for training a small language model, from raw text to text generation. minGPT suits students who want to read a working gpt implementation end to end, while train-llm-from-scratch suits students who want to learn how a transformer is built and trained step by step.
Are minGPT and train-llm-from-scratch free?
minGPT: Free and open source. train-llm-from-scratch: Free and open source.
Is minGPT or train-llm-from-scratch more popular?
minGPT has more GitHub stars (25k) than train-llm-from-scratch (13.1k). Popularity isn't the same as fit — check which one matches your use case above.
What are other alternatives to minGPT and train-llm-from-scratch?
Other options people compare them with include llm.c.
Keep comparing
Built from our catalogue entries for both products: open-source details come from each project's README (AI-assisted, refreshed weekly) and GitHub data; AI-tool details are editorial. Spotted something wrong? Let us know from either product's page.