Guide · Agent skill
How to use the Ablation Planner agent skill (ablation-planner)
Updated · written with AI from the project's own documentation and checked against our catalogue data
Ablation Planner is an agent skill that helps an AI agent plan the ablation experiments a research paper needs. It is for ML researchers and research engineers whose main results already hold up and who want a prioritized list of experiments reviewers are likely to ask for. After reading this guide you will know what the skill does, how to install it, what your project needs before you trigger it, and how to ask for it in plain language.
Ablation Plannerwanshuiyin/Auto-claude-code-research-in-sleep · MIT Download the skill Quick start
- 1Download the ZIP. Download the ablation-planner ZIP file from this page.
- 2Unzip the skill. Unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project.
- 3Configure Codex MCP. Make sure a Codex MCP server is set up so the mcp__codex__codex tools are available to your agent.
- 4Prepare project files. Check that your research contract, experiment log or CLAUDE.md exist and describe your method, results and compute.
- 5Ask for ablation planning. Request ablation planning in plain language after your main results pass the claim check, then review the plan and budget before the runs start.
What does the Ablation Planner skill make the agent do?
An ablation study removes or swaps out one part of a method to show whether that part really matters. The skill asks a Codex model, reached through an MCP connector, to act as a strict reviewer. That reviewer proposes a prioritized list of ablations, hyperparameter sweeps and design comparisons, each with an expected outcome and a suggested run order.
The agent then checks the compute budget and the code changes each ablation needs, creates configs or scripts for each one, and runs them while logging results. Codex leads the design, and the agent reviews feasibility and does the implementation.
How do you install the Ablation Planner skill?
The skill ships as a ZIP file that you can download from this page. Unzip it into one of two locations. Use ~/.claude/skills/ to make the skill available across your projects, or use the .claude/skills/ folder inside a specific project to make it available only there.
If you use the Claude apps instead, you can upload the skill under Settings → Capabilities → Skills.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into the project's .claude/skills/ folder
- Claude apps: upload under Settings → Capabilities → Skills
~/.claude/skills/
.claude/skills/What does your project need before the skill can run?
The skill needs an agent with skill support, such as Claude Code or a compatible agent. It also needs a Codex MCP server configured so that the mcp__codex__codex tools are available. Without that connector the design step cannot run.
The skill expects project files to exist. It reads idea-stage/docs/research_contract.md, EXPERIMENT_LOG.md or CLAUDE.md, and it also works with the legacy docs/research_contract.md and EXPERIMENT_TRACKER.md. You also need compute resources for the experiments. GPU server details are optional and can be written in CLAUDE.md. Weights & Biases is optional and can be used as a results source.
- Claude Code or a compatible agent with skill support
- Codex MCP server configured (mcp__codex__codex and mcp__codex__codex-reply)
- Project files such as research_contract.md, EXPERIMENT_LOG.md or CLAUDE.md
- Compute resources for the experiments
- Weights & Biases (optional, as a results source)
How do you trigger the skill with a natural request?
The skill is meant to run after your main results pass a claim check, meaning the result-to-claim step returned claim_supported = yes or partial. You can also ask for ablation planning directly. If an auto-review loop reviewer says ablations are missing, that is another reason to run it.
Ask in plain language and name the method or claim. The skill accepts a method description or claim as its argument. Examples of requests that fit the skill include the ones below.
- Plan the ablations for our new attention module, since the main results passed the claim check.
- Our reviewer said we are missing ablations. Plan the ones needed for submission.
- Design a hyperparameter sensitivity sweep and a run order for the regularization term.
- Which components of this method should I remove or replace to show they matter?
What output should you expect?
The skill produces an ablation plan with sections for component ablations, hyperparameter sensitivity, design choice comparisons, a coverage assessment, unnecessary ablations to skip, a run order and an estimated total in GPU-hours. Each ablation has a name, the question it tests, what result would show the component matters, and a priority from 1 (must-run) to 5 (nice-to-have).
After planning, the agent creates configs or scripts, smoke tests each ablation, runs them in order using descriptive names such as ablation-no-module-X, and records results in EXPERIMENT_LOG.md. Negative results are recorded too, since a component removal with no effect is a finding.
What safety points should you know about?
This skill runs scripts and uses the network. Its allowed tools include Bash, file reading and writing, and the Codex MCP tools. It can launch real GPU jobs that cost money and time, so check the compute budget and the planned run list before letting it run everything.
The skill's rules say that if total compute exceeds the budget, the agent proposes cuts and asks Codex to re-prioritize rather than dropping ablations silently. Smoke test each ablation before the full run.
- It runs scripts (Bash) and uses the network.
- It can start paid GPU jobs, so review the plan and budget first.
- Config-only ablations are run before those needing code changes.
What tips help the skill work well?
Keep your research contract, experiment log and CLAUDE.md up to date, because the skill builds its picture of the project from them. Write down your GPU setup in CLAUDE.md if you have one.
Use the skill for research code only. The workflow is written around paper reviewers and ML experiments, so it adds little for other kinds of software.
- Keep research_contract.md and EXPERIMENT_LOG.md current before planning.
- Note your GPU server details in CLAUDE.md so compute estimates are realistic.
- Run it only after main results hold up, since it adds little earlier.
What people use it for
- Listing the ablations a reviewer is likely to request before submitting a paper.
- Choosing which hyperparameters to sweep and in what order when GPU time is limited.
- Turning an auto-review loop comment about missing ablations into a concrete, runnable plan.
- Logging negative ablation results so the paper can report which components did not matter.
Frequently asked questions
Is the Ablation Planner skill free?
The skill is released under the MIT licence. This page does not list prices, and running the experiments costs compute time and money on your own resources.
What do I need to install the Ablation Planner skill?
You need Claude Code or a compatible agent with skill support, plus a Codex MCP server providing the mcp__codex__codex tools. Unzip the downloaded ZIP into ~/.claude/skills/ or a project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps.
Can the Ablation Planner skill run on its own without my project files?
No. It assumes project files such as a research contract, experiment logs and a Codex MCP server are present. It is designed to run after main results pass a claim check.
Does the skill run experiments automatically?
It can. After planning, the agent creates configs or scripts, smoke tests them and runs them in the suggested order while logging results. Because these can be real GPU jobs, review the plan and compute budget first.
What should I do if the Codex tools are not available?
Check that a Codex MCP server is configured so that mcp__codex__codex and mcp__codex__codex-reply are available to the agent. The design step depends on those tools, so the skill cannot plan ablations without them.
How does it compare with other ablation tools?
This source does not compare the skill with other tools, so this guide makes no claim about alternatives. The skill is specific to research paper workflows that use Codex as a reviewer and the wider Auto-claude-code-research-in-sleep workflow.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use Ablation Planner if you are preparing an ML paper whose main results already hold up and you want a reviewer-minded checklist of ablations with a run order. Pick something else if you do not yet have a validated main result, if you need a tool that works standalone without Codex and project files, or if your code is not research code.
Download the skill