Skip to content
Get packs

Model Merging with mergekit

Combine several fine-tuned AI models into one using mergekit, without retraining.

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
5 files

What it does

It teaches an AI agent how to blend the weights of several existing language models into a single new model using the mergekit tool, instead of training from scratch. The agent can write the YAML config files for methods such as linear averaging, SLERP, TIES, DARE and task arithmetic, and walk you through running the merge and using the result. It is meant for people who already have compatible fine-tuned models and want to combine their strengths.

Great for

  • ML engineers experimenting with model variants who already have fine-tuned checkpoints
  • Hobbyists comparing merge recipes on open-weight models from Hugging Face
  • Teams that want a cheaper alternative to training a new combined model

Skip it if

  • Users without technical background or without GPU/CPU capacity for multi-gigabyte models, since merges still load full model weights
  • Merging models with incompatible architectures or licences, which can produce broken or legally restricted outputs
  • Anyone expecting guaranteed benchmark gains; the skill's own claims of +5-10% are unverified and results vary

Try asking

  • “Write a mergekit YAML config that blends a math model and a chat model using SLERP with t=0.5”
  • “Explain the difference between TIES and DARE and which to use for merging five task-specific models”
  • “Help me set up mergekit on my machine and run mergekit-yaml on my config”
  • “Suggest density and weight values for a TIES merge of three Mistral-7B fine-tunes”
Step-by-step guide: how to use Model Merging with mergekitInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 5 original files, the MIT licence and an attribution note.

    model-merging.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip model-merging.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip model-merging.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Start with two models that share the same base architecture before trying multi-model methods.
  • Check each source model's licence before merging and distributing the result.
  • Review every shell command the agent proposes before running it, since the skill covers installing packages and running merge jobs on large downloads.

What's inside

  • references/coefficient-tuning.md
  • references/evaluation.md
  • references/examples.md
  • references/methods.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.