Skip to content
Get packs

Guide · Agent skill

How to use the Model Merging with mergekit agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The model-merging skill teaches an AI agent to combine the weights of several existing language models into one new model using the mergekit tool, instead of training a new model from scratch. It is aimed at people who already have compatible fine-tuned models and want to combine their strengths. After this guide you will know what the skill does, how to install it, how to ask for it in plain language, and what you need on your machine before running a merge.

Model Merging with mergekitOrchestra-Research/AI-Research-SKILLs · MIT

Quick start

  1. 1Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or your project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps.
  2. 2Install the requirements. Make sure Python with pip is available, then install mergekit, transformers and torch.
  3. 3Get access to source models. Confirm you can download the fine-tuned models you plan to combine, for example from Hugging Face, and that they share an architecture.
  4. 4Ask for a config. Describe your goal and models in plain language and ask the agent to write a YAML config for a suitable method.
  5. 5Review the YAML. Check the model names, weights, density values and dtype before running anything.
  6. 6Run the merge. Run mergekit-yaml with your config and an output folder, adding --cuda only if you have a GPU.
  7. 7Evaluate the result. Test the merged model against the source models on your own tasks and keep the recipe that works best.

What is an agent skill, and what does model-merging make the agent do?

An agent skill is a folder of instructions and supporting files that an AI agent can load when a task matches it. This skill is built around SKILL.md and a set of reference files covering methods, coefficient tuning, evaluation and examples.

With the skill loaded, the agent can write YAML config files for merge methods including linear averaging, SLERP, TIES-Merging, DARE and task arithmetic, explain the parameters, and walk you through running the merge and using the result. The skill is also described as covering production deployment strategies.

  • Linear: a weighted average of parameters, useful for combining two or more models.
  • SLERP: spherical interpolation, described for merging two models.
  • Task Arithmetic: adds weighted task vectors (fine-tuned minus base) to a base model.
  • TIES-Merging: task arithmetic plus sparsification, meant for many task-specific models.
  • DARE: randomly drops fine-tuned parameters and rescales the rest.

How do you install the model-merging skill?

Download the ZIP for this skill from this page and unzip it. For a personal setup, place the folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.

You can also upload the skill in the Claude apps under Settings → Capabilities → Skills.

  • Personal install: unzip into ~/.claude/skills/
  • Project install: unzip into the project's .claude/skills/ folder
  • Claude apps: Settings → Capabilities → Skills, then upload

What do you need installed before merging?

The skill expects Python with pip, the mergekit library (installed from GitHub or PyPI), and the transformers and torch packages. You also need access to the source model weights, for example downloads from Hugging Face. A GPU is optional; the source describes merges as able to run on CPU, but multi-gigabyte models still need enough memory to load full weights.

The skill's own install options are shown below. Check the commands against the current mergekit documentation before running them.

  • Python with pip
  • mergekit (from GitHub or PyPI)
  • transformers and torch
  • Access to source model weights, e.g. Hugging Face downloads
  • GPU optional
pip install mergekit pip install transformers torch

How do you trigger the skill with a natural request?

You do not need special syntax. Describe the outcome you want in plain language and mention the models or method if you have them. The skill's description matches requests about blending domain-specific expertise, improving on a single model, or testing model variants quickly.

Good requests name your models, the goal, and the method if you know it. If you do not know the method, ask the agent to compare options for your situation.

  • "Write a mergekit YAML config that blends my math and chat fine-tunes using SLERP."
  • "I have three fine-tuned Mistral-based models. Which merge method should I try for combining them?"
  • "Explain the density and weight parameters in a TIES config and set them for my models."
  • "Help me run a merge of these two checkpoints and test the output."

What does a merge config look like?

A mergekit config is a YAML file that names the merge method, the models, and the parameters. The source includes a simple linear merge of two models with equal weights. Note that the example uses a placeholder output path and a CUDA flag, so adjust them to your setup.

The source gives this example command for running a config. Treat it as the skill's example and confirm the flags against your installed mergekit version.

  • merge_method: the technique, such as linear, slerp, ties or task_arithmetic
  • models: each source model with its weight or density
  • dtype: for example bfloat16
  • base_model: needed for task arithmetic and TIES-style merges in the examples
merge_method: linear models: - model: mistralai/Mistral-7B-v0.1 parameters: weight: 0.5 - model: teknium/OpenHermes-2.5-Mistral-7B parameters: weight: 0.5 dtype: bfloat16 # then run: mergekit-yaml config.yml ./merged-model --cuda

What should you check before and after a merge?

The skill stresses model compatibility: merge models with the same architecture. Its own example pairs two Mistral 7B models and marks a Llama and Mistral pairing as incompatible. Also check licences, since merging can produce legally restricted outputs.

Treat the skill's gains as unproven. It mentions improvements of +5-10% on benchmarks, but those claims are not verified and results vary. Evaluate your merged model on tasks you care about, and compare it with the source models.

  • Confirm all source models share the same architecture.
  • Check each model's licence before merging and distributing the result.
  • Test the merged model against the source models on your own tasks.
  • Expect results to vary; do not assume a benchmark gain.

What people use it for

  • Blending a math-focused fine-tune with a chat fine-tune and a coding fine-tune using task arithmetic.
  • Comparing linear, SLERP and TIES recipes on the same pair of open-weight models.
  • Creating several merged variants quickly to test which blend suits a domain.
  • Layer-wise merging, such as taking different layer ranges from different source models.

Frequently asked questions

How do I install the model-merging skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or a project's .claude/skills/ folder. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.

What do I need to run model merging with mergekit?

You need Python with pip, the mergekit library, transformers and torch, plus access to the source model weights. A GPU is optional, though the source notes that multi-gigabyte models still need enough memory.

Is the model-merging skill free?

The skill is listed under the MIT licence. The source does not state any price, so check the directory page for current terms.

Which merge method should I use?

The source suggests SLERP for merging two models, TIES for many task-specific models, and task arithmetic for combining specialized skills on a base model. Linear merging suits simple combinations of similar models.

Why does my merge fail or produce broken output?

The source warns that merging models with incompatible architectures can produce broken outputs, so confirm all models share one architecture. Also check that the source models are accessible and that your memory can hold the weights.

Are merged models better than the originals?

Not guaranteed. The skill mentions +5-10% benchmark gains, but these are unverified and results vary, so test the merge against the source models on your own tasks.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

This skill suits ML engineers and hobbyists who already have compatible fine-tuned checkpoints and want to test merge recipes cheaply. It is a poor fit for users without a technical background, anyone without enough memory for multi-gigabyte weights, or anyone expecting guaranteed benchmark gains.

Download the skill

Find exactly what
you're looking for