Guide · Agent skill
How to use the nanoGPT Minimalist GPT Trainer agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The nanogpt skill teaches an AI agent how to work with nanoGPT, a short and readable GPT implementation created by Andrej Karpathy. It is aimed at students, engineers and educators who want to learn how GPT models work by reading and changing a small codebase. After this guide you will know how to install the skill, how to ask your agent for help with it, and what hardware and setup it needs.
nanoGPT Minimalist GPT TrainerOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the ZIP from this page and unzip it. Keep the skill folder that contains SKILL.md.
- 2Place it in your skills folder. Move the folder to ~/.claude/skills/ for all projects, or to .claude/skills/ in one project. Or upload it under Settings → Capabilities → Skills in the Claude apps.
- 3Clone nanoGPT. Get the separate nanoGPT repository, since the skill references its train.py, sample.py and data scripts.
- 4Install the Python packages. Run pip install torch numpy transformers datasets tiktoken wandb tqdm in your environment.
- 5Ask for a Shakespeare run. Describe your goal in plain language, such as training a small model on your CPU. The agent will use the skill's workflow.
- 6Generate text. Run the sample script on the output folder once training finishes, and check the output.
What does the nanogpt skill make the agent do?
An agent skill is a folder of instructions and reference notes that an AI agent loads when your request matches the skill's purpose. The nanogpt skill covers the nanoGPT workflow: preparing a dataset, training a model, sampling text from it, and fine-tuning from OpenAI's GPT-2 weights.
The skill also includes reference notes on the model structure, the training loop and the data formats, so the agent can explain what each part does.
- Walk you through preparing a dataset, such as Shakespeare text or your own text file
- Train a small model on a laptop CPU, or a 124M-parameter GPT-2 model on multiple GPUs
- Sample text from a trained model
- Fine-tune starting from OpenAI's gpt2, gpt2-medium, gpt2-large or gpt2-xl weights
- Explain the GPT block structure, multi-head attention, MLP layers, learning rate schedule and gradient accumulation
How do you install the nanogpt skill?
Download the skill ZIP from this page and unzip it. For Claude Code, place the unzipped skill folder in ~/.claude/skills/ to make it available in all your projects, or in a project's .claude/skills/ folder to make it available only in that project.
In the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.
- Personal install: ~/.claude/skills/
- Project install: .claude/skills/ inside your project folder
- Claude apps: Settings → Capabilities → Skills, then upload
Install nanoGPT's Python dependencies
The skill itself contains no code. Its examples run nanoGPT's scripts, so you also need the Python packages below and a separate clone of the nanoGPT repository, which contains train.py, sample.py and the data scripts. The skill's own files do not include that code.
- Required pip packages: torch, numpy, transformers, datasets, tiktoken, wandb, tqdm
- Optional: an NVIDIA GPU with CUDA for faster training
- Optional: a Weights & Biases account for wandb logging
- Multi-GPU setup is needed only for the GPT-2 124M reproduction
pip install torch numpy transformers datasets tiktoken wandb tqdmHow do you trigger the skill with natural requests?
You do not need a special command. Describe what you want in plain language and mention nanoGPT or GPT training, and the agent can load the skill. Be specific about your goal, your hardware and your data so the agent picks the right path.
- "Walk me through training a small nanoGPT model on Shakespeare on my CPU."
- "Help me prepare my own text file as a nanoGPT dataset."
- "Fine-tune nanoGPT starting from the gpt2 weights on a small dataset."
- "My nanoGPT run is out of CUDA memory. What should I change?"
What does a first run look like?
The quickest path is the character-level Shakespeare workflow. It prepares data, trains a small model and generates text. The skill's notes give about 5 minutes on a CPU and about 1 minute on a T4 GPU. Run these commands from your nanoGPT clone.
- Step 1 creates train.bin and val.bin from the Shakespeare text
- Step 2 trains a 6-layer, 6-head model with 384-dimensional embeddings
- Step 3 generates text from the output folder
python data/shakespeare_char/prepare.py
python train.py config/train_shakespeare_char.py
python sample.py --out_dir=out-shakespeare-charWhat about hardware and reproducing GPT-2 124M?
The GPT-2 124M reproduction is a large job. The skill lists about 4 days on 8 A100 GPUs and about 1 week on one A100, and it notes the preparation step takes about 1 hour. Start with the Shakespeare workflow unless you have that hardware and budget available.
- Shakespeare char-level: CPU about 5 minutes, T4 GPU about 1 minute, under 1GB VRAM
- GPT-2 124M: 1× A100 about 1 week, 8× A100 about 4 days, about 16GB VRAM per GPU
- GPT-2 Medium 350M: 8× A100 about 2 weeks, about 40GB VRAM per GPU
What are the safety notes?
This skill runs scripts and uses the network. The training and data-preparation scripts download and process data, and the Weights & Biases and Hugging Face parts connect to online services. Read the commands the agent proposes before you approve them, and check the data sources you use. The skill's own files are documentation and contain no code; the code comes from your separate nanoGPT clone.
- Review any script the agent asks you to run
- Be aware that wandb logging sends metrics to Weights & Biases if you enable it
- The OpenWebText source is hosted on Hugging Face
What tips help you get better results?
Most problems come from memory limits or too little training. The skill's common-issue notes suggest specific config changes. Make one change at a time so you can see what helped.
- CUDA out of memory: reduce batch_size or block_size, and raise gradient_accumulation_steps to keep the effective batch size
- Training too slow: set compile = True on PyTorch 2.0 or later, and try dtype = 'bfloat16' or 'float16'
- Poor generation quality: train for more iterations and lower the sampling temperature, for example to 0.7, with top_k sampling
- Cannot load GPT-2 weights: install transformers and check that init_from uses a valid name such as 'gpt2'
What people use it for
- Learning how a GPT block, attention and MLP layers fit together by training a tiny model and reading the code
- Teaching a hands-on lesson where students train a character-level model on Shakespeare on laptops
- Prototyping a transformer variant on a single GPU or CPU before scaling up
- Fine-tuning a GPT-2 checkpoint on a small custom text file
- Preparing a character-level dataset from your own text file
Frequently asked questions
How do I install the nanoGPT agent skill?
Download the skill ZIP from this page and unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project. In the Claude apps, you can upload it under Settings → Capabilities → Skills.
Is the nanogpt skill free?
The skill is published under the MIT licence, according to its listing. The skill files contain no code, so you still need your own compute and any cloud or Weights & Biases accounts you choose to use.
What do I need to run nanoGPT with this skill?
You need Python and the pip packages torch, numpy, transformers, datasets, tiktoken, wandb and tqdm, plus a separate clone of the nanoGPT repository. A GPU is optional, but the GPT-2 124M reproduction needs a multi-GPU setup.
Can I train nanoGPT on a CPU?
Yes. The skill's Shakespeare character-level workflow is described as about 5 minutes on a CPU, which is the easiest way to start learning the workflow.
Does nanoGPT work for production text generation?
The skill itself points users to HuggingFace Transformers or LitGPT for production use. nanoGPT is a teaching-focused codebase, so it is better for learning and experiments than for serving text.
Why does my nanoGPT training run run out of CUDA memory?
The skill suggests reducing batch_size or block_size and increasing gradient_accumulation_steps to keep the effective batch size. These changes lower memory use without changing the total training batch.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Choose this skill if you want to learn how GPT models work by reading and modifying a short codebase, or if you are teaching that material. Pick something else if you need production text generation, serving, or training large models on a limited budget, since the GPT-2 124M reproduction is costly.
Download the skill