How to use the Skill Optimizer agent skill (skill-optimizer)

What an agent skill is and what this one does

An agent skill is a folder containing a SKILL.md file. The file has a small header (name, description, triggers) and a body of instructions that tells an AI agent how to handle a certain kind of task. The agent loads the skill when a request matches its triggers.

Skill Optimizer treats the body of SKILL.md as the thing to tune. It generates a starter benchmark from the skill, lets you strengthen the test tasks, then runs repeated rounds of rewriting the body. A change is kept only if it scores better on a validation set, and the run stops when the budget is reached.

It does not change the frontmatter routing settings such as triggers or brain_first, so the skill still gets invoked the same way. Bundled skills that ship with gbrain are protected from in-place rewrites unless you pass explicit flags and supply a separate held-out set.

Install the skill-optimizer skill

You can install it in one of two ways. Download the ZIP from this directory page and unzip it into ~/.claude/skills/ for a personal install, or into a project's .claude/skills/ folder to limit it to that project.

If you use the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.

Installing the skill only gives the agent the instructions. The optimization itself runs through the gbrain CLI, so that must also be installed with the skillopt command.

• Personal install: unzip into ~/.claude/skills/
• Project install: unzip into the project's .claude/skills/
• Claude apps: Settings → Capabilities → Skills, then upload

Check the requirements before you start

Three things must be in place. The gbrain CLI with the skillopt command must be installed, because the SKILL.md alone cannot run the optimization. You need an LLM provider with API access, since generating benchmarks, reflection and judging all make model calls and cost money. And the skill you want to improve must live in a writable skills directory.

Plugin-bundled skills are a read-only snapshot, so you cannot write benchmark or optimization files next to them. Copy the skill into your own skills directory first if you want to iterate on it.

• gbrain CLI with the skillopt command
• An LLM provider with API access (costs are incurred)
• A writable skills directory that contains the target skill

Trigger the skill with a natural request

The skill is invoked when you ask for one of its listed triggers. Phrases the skill recognises include the examples below. Name the skill you want to improve so the agent knows which folder to work on.

If you already know the exact command, you can run it directly. The common path for a skill with no benchmark is to generate a starter from the SKILL.md, review it, and then optimize.

• "optimize this skill"
• "make the skill better"
• "tune the skill against the benchmark"
• "run skillopt"
• "skillopt for" followed by the skill name

Walk through a first run on a skill with no benchmark

Generate the starter benchmark. One LLM call reads skills/X/SKILL.md and writes about 15 tasks, each with rule judges, to skills/X/skillopt-benchmark.jsonl. A sentinel line, # BOOTSTRAP_PENDING_REVIEW, is added at the end. You can change the count with --bootstrap-tasks, up to 50.

Review and strengthen the judges. The generated checks are described as weak drafts, often generic contains checks, loose length limits or invented headings. Add real section names, real length ceilings, and checks for tools the skill truly uses. Do not accept the draft unchanged.

Delete the sentinel line, then run the optimizer with the 1:1:1 split. The default 4:1:5 split needs roughly 50 tasks, so a 15-task starter with the default split is refused with d_sel_too_small. Use --dry-run first to preview cost.

• Step 1 generates the file and the sentinel.
• Step 2 is the human review and is not optional.
• Step 3 removes the sentinel and runs with --split 1:1:1.

Iterate on a skill you already have a benchmark for

If you already have a benchmark file, skip the bootstrap and point the optimizer at it. Add --held-out with a separate file of at least 5 benchmark-disjoint tasks to guard against overfitting, since a candidate that beats the benchmark but regresses on the held-out set is refused.

To review changes before anything is applied, add --no-mutate. This writes a proposed.md for you to read instead of rewriting SKILL.md. If a run crashes midway, resume it with its run ID.

• Use --no-mutate to write proposed.md for review only.
• Use --held-out <path> to catch benchmark overfitting.
• Use --resume <run-id> after a mid-run crash.

Understand the safety rules and what gets written

The optimizer runs a pre-flight check: the working tree must be clean unless you pass --force, the benchmark must have at least 5 validation tasks, and a cost preflight refuses runs over --max-cost-usd. Every candidate must beat the current best by a margin of 0.05 on the median of three runs before SKILL.md is rewritten.

Bundled skills need both --allow-mutate-bundled and a held-out set with at least 5 tasks, otherwise the run hard-refuses with exit code 2. Bootstrapped benchmarks require your review before optimization will use them.

Output goes under skills/<name>/. This includes the updated SKILL.md when mutation is allowed, skillopt/best.md, versioned snapshots in skillopt/versions/, skillopt/history.json as an append-only run record, and skillopt/rejected.json.

• Pre-flight: clean working tree (or --force), at least 5 validation tasks, cost cap via --max-cost-usd.
• Acceptance: median of 3 runs with a 0.05 margin.
• Bundled skills: --allow-mutate-bundled plus --held-out, or the run refuses.
• Outputs: SKILL.md, skillopt/best.md, skillopt/versions/, skillopt/history.json, skillopt/rejected.json.

Write a benchmark line by hand

Each line of the benchmark file is one JSON task with a prompt and a judge. The judge uses rule checks from this vocabulary: contains, regex, section_present, max_chars, min_citations, tool_called and tool_not_called. Rule judges are deterministic and free, but they are shallow for skills whose quality depends on sequencing, privacy, refusal boundaries or file placement, so add richer checks or an llm judge for those.

• Rule checks: contains, regex, section_present, max_chars, min_citations, tool_called, tool_not_called.
• Use at least two checks per task when writing by hand.
• For a fully hand-written benchmark, aim for about 15 realistic tasks.