Skip to content
Get packs

Guide · Open source

How to use SkillOpt: install, train and deploy best_skill.md

Updated · written with AI from the project's own documentation and checked against our catalogue data

SkillOpt is an open-source Python tool from Microsoft that improves a plain-text skill document, best_skill.md, used by a language model acting as an agent. The model's weights never change; only the instructions do. This guide explains what SkillOpt needs, how to install it from PyPI, how the training loop works, and how to decide whether it fits your task.

SkillOptmicrosoft/SkillOpt · ★ 18.8k · MIT

Quick start

  1. 1Confirm Python 3.10+. Check that your Python version is 3.10 or newer, since the package requires it.
  2. 2Install the package. Run pip install skillopt to install the release from PyPI.
  3. 3Prepare a scoreable task set. Gather labeled tasks with a score for each result, since training depends on validation scores.
  4. 4Read the documentation. Follow the versioned docs in the repository for data preparation, training and eval commands, and configuration.
  5. 5Run the training loop. Run the optimizer so it proposes add, delete, or replace edits and keeps only those that improve the held-out validation score.
  6. 6Review best_skill.md. Open the resulting skill file, check the changes, and test it against your own tasks.
  7. 7Optionally monitor with the WebUI. Install the webui extra and launch the dashboard, binding to loopback unless you add authentication.

What SkillOpt does and who it is for

SkillOpt treats the skill document as the trainable state of a frozen agent. An optimizer model turns scored rollouts (runs of the agent on tasks) into small add, delete, or replace edits to the skill. An edit is kept only if it improves a held-out validation score.

The README positions it for developers who run LLM agents on repeatable tasks, teams that want a human-readable skill file instead of fine-tuned weights, researchers reproducing the paper's training loop, and users of coding agents such as Claude Code or Codex CLI who want offline refinement through SkillOpt-Sleep.

  • Good fit: you have repeatable tasks and a way to score the agent's results.
  • Good fit: you want a deployable, human-readable skill file.
  • Not a fit: you need to change model weights, which SkillOpt does not do.
  • Not a fit: you have no labeled or scoreable task set, because training depends on validation scores.
  • Not a fit: you expect a point-and-click app. Setup uses Python and a CLI.

Check the prerequisites before you install

SkillOpt requires Python 3.10 or newer. It is installed with a package manager. You also need access to the language models you plan to use, and a labeled task set with a score for each result. The source does not list other system requirements, so check the documentation for your specific backend and benchmark.

  • Python 3.10+
  • A package manager (pip)
  • A scoreable task set with validation data

Install SkillOpt from PyPI

Install the published package with pip. The README lists this as the install command for the PyPI release. For installation details, data preparation, and configuration, the README points to the versioned documentation in the repository's docs folder.

pip install skillopt

Understand the training loop

The full training loop runs in this order: rollout, reflect, aggregate, select, update, evaluate. The agent runs on tasks (rollout), the optimizer reviews the results (reflect), and proposed edits are combined and selected. The selected edit updates the skill, and the result is evaluated on held-out validation data.

SkillOpt also keeps a rejected-edit buffer and applies an epoch-wise update. These controls are meant to keep training stable. The README notes that the deployed artifact adds zero inference-time model calls, because the skill is just text added to the agent's instructions.

  • Rollout: run the agent on tasks and score the results.
  • Reflect: the optimizer model reviews the scored runs.
  • Aggregate and select: combine candidate edits and choose which to try.
  • Update: apply a bounded add, delete, or replace edit to the skill.
  • Evaluate: keep the edit only if the held-out validation score improves.

Use the deployable best_skill.md file

The output of training is a compact best_skill.md file. The README says it is typically 300 to 2,000 tokens and runs against the unchanged target model. Because it is plain Markdown text, you can read, review, and version it like any other document.

To use it, provide the skill to your agent as its instructions. Compare results with and without the skill on your own tasks before relying on it.

  • Open best_skill.md to review what changed.
  • Keep it under version control to track improvements.
  • Measure results on your own tasks; the README reports results only on its own benchmarks.

Try SkillOpt-Sleep for coding agents

SkillOpt-Sleep is a nightly offline self-evolution engine, shipped as the skillopt-sleep CLI in release v0.2.0. The README describes it as a companion for local coding agents such as Claude Code, Codex, and Copilot. It reviews past sessions, replays recurring tasks, and consolidates validated skills behind a held-out gate.

The README notes that the main CLI keeps conservative defaults and does not expose every experiment-harness control as a flag. The integration shells for Claude Code, Codex, Copilot, and Devin live in the repository, not in the PyPI package. Read docs/sleep/README.md for how to use it.

  • Install from PyPI gives you the core package; plugin and MCP files come from the repository.
  • Read docs/sleep/README.md before running it on your own sessions.

Optional: launch the WebUI dashboard

The repository includes an optional monitoring dashboard. The README gives these commands for a development install with the webui extra. By default the server binds to loopback only, and exposing it requires authentication credentials.

  • --port defaults to 7860.
  • --host defaults to 127.0.0.1.
  • --share creates a public Gradio share link and is off by default.
  • Using --host 0.0.0.0 or --share requires --auth-user and --auth-pass, or the SKILLOPT_WEBUI_USER and SKILLOPT_WEBUI_PASS environment variables.
pip install -e ".[webui]" python -m skillopt_webui.app

What people use it for

  • Refining the instructions of a coding agent that handles the same kind of task every day.
  • Producing a readable skill file that a team can review and version instead of fine-tuned weights.
  • Reproducing the SkillOpt paper's training loop and benchmarks for research.
  • Running offline nightly skill consolidation for Claude Code or Codex sessions with SkillOpt-Sleep.

Frequently asked questions

Is SkillOpt free to use?

SkillOpt is released under the MIT licence, as stated in the README and the repository badge. The README does not list prices for the language model APIs you may need to run it, so check those providers' terms separately.

What are the requirements for SkillOpt?

SkillOpt requires Python 3.10 or newer and is installed with pip. You also need a scoreable, labeled task set, because training depends on validation scores.

How do I install SkillOpt?

Run pip install skillopt. The README lists this as the PyPI install command. For data preparation and configuration, follow the documentation in the repository's docs folder.

Does SkillOpt fine-tune or retrain the model?

No. SkillOpt explicitly does not touch model weights. It improves the text of a skill document, best_skill.md, that the unchanged model reads as instructions.

Are there alternatives to SkillOpt?

The source does not name comparable projects, so this guide cannot make a comparison. If you need to change model weights, SkillOpt is not the right tool for that job.

Will SkillOpt guarantee better results?

No. The README does not promise gains and reports results only on its own benchmarks. Measure the skill against your own tasks before relying on it.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

SkillOpt suits developers and teams who run LLM agents on repeatable, scoreable tasks and want a readable skill file that improves without retraining the model. It is a poor fit for anyone who needs weight changes, lacks labeled tasks, or wants a graphical app. Test any improvement on your own work, since the published results come from the project's own benchmarks.

Install options & downloads