How to use the Experiment Plan Builder agent skill (experiment-plan)

What the experiment-plan skill makes the agent do

The skill takes a refined research proposal or method idea and turns it into a claim-driven experiment roadmap. The aim is to produce a chain of claim, then evidence, then run order, rather than a long wishlist of benchmarks.

The skill works through the paper's claims first: a primary claim, an optional supporting claim, an anti-claim to rule out, and the minimum convincing evidence for each. It then lays out experiment blocks such as main anchor results, novelty isolation, simplicity checks, frontier-necessity checks where relevant, and failure analysis. Each block is marked as main paper, appendix, or cut.

• Run order in milestones: sanity stage, baseline stage, main method stage, decision stage, polish stage.
• Per-milestone compute cost, decision gates and risks.
• Must-run versus nice-to-have experiments separated.
• Output files: refine-logs/EXPERIMENT_PLAN.md and refine-logs/EXPERIMENT_TRACKER.md.

Install the experiment-plan skill

Download the ZIP from this page and unzip it. The skill is a folder containing SKILL.md. Place that folder in one of these locations:

• Personal skills folder: ~/.claude/skills/ (applies to your user account).
• Project skills folder: .claude/skills/ inside your project directory (applies to that project).
• Claude apps: upload the ZIP under Settings → Capabilities → Skills.

Check what the skill needs

The skill needs an AI agent that supports SKILL.md skills. Its files are optional inputs: if they exist, the agent reads refine-logs/FINAL_PROPOSAL.md, refine-logs/REVIEW_SUMMARY.md and refine-logs/REFINEMENT_REPORT.md, which come from the research-refine workflow. If they are missing, the agent derives the context from your prompt instead.

The skill does not run scripts. According to the listing, it uses the network.

• Required: an agent that supports SKILL.md skills.
• Optional: refine-logs/ files from research-refine.
• Runs scripts: no.
• Uses network: yes.

Trigger the skill with natural requests

You do not need a special command. Describe what you want in plain language, and mention the method or proposal you already have. The skill's description matches requests about a detailed experiment plan, ablation matrix, evaluation protocol, run order, compute budget, or paper-ready validation.

The skill can also be invoked directly with /experiment-plan followed by your request, since the SKILL.md argument placeholder accepts your text. If you want the full chain in one request, the source suggests /research-refine-pipeline.

Work through an example

Start with a proposal you have already settled. If you ran research-refine earlier, the agent will read the files in refine-logs/ automatically. Then review the claim map first, since every block should trace back to a claim. Check that the anti-claims are the ones a sceptical reviewer would raise.

When the plan is ready, the agent gives a short summary listing must-run blocks, the highest-risk assumption, and the first three runs to launch. Use that summary as your starting checklist, and open the full plan file for details.

• Read the Claim Map table in EXPERIMENT_PLAN.md before any run.
• Use EXPERIMENT_TRACKER.md to record status for each run (TODO and similar).
• Verify every compute estimate yourself before committing budget.

Safety notes and limits

The skill only plans experiments. It does not run them, and its cost estimates are guesses that you must verify. Because it uses the network, review what the agent fetches and writes before you rely on it. The skill's allowed tools include Bash, Read, Write, Edit, Grep, Glob, WebSearch and WebFetch, so check the permissions your agent asks you to grant.

The structure is built around paper claims and reviewer concerns. It is not designed for non-research projects, and it assumes a stable method. Do not use it for early brainstorming.

• Plans only; it does not execute training or evaluation.
• Cost and GPU-hour figures are estimates to verify.
• Plans assume a fixed proposal and can give false precision if the method is still changing.

Tips for better plans

Give the agent a precise problem statement and one dominant contribution. The skill prefers at most two primary claims and a compact set of up to five core blocks, so a vague prompt will produce a less focused plan.

If your method uses an LLM, VLM, diffusion or RL component, say so. The skill includes a frontier-necessity check to test whether that component is the right tool compared with a simpler alternative. If the proposal is intentionally non-frontier, state that and the skill will skip the block.

• Keep the claim count low; one main claim plus one support is the default.
• Prefer a few strong baselines to many weak ones.
• Use three seeds where variance matters and budget allows (the skill's default).