Skip to content
Get packs

Axolotl LLM Fine-Tuning Guide

Guides an agent through fine-tuning LLMs with Axolotl: YAML configs, LoRA/QLoRA, DPO and multi-GPU setups.

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
3 files

What it does

Once installed, the agent knows the Axolotl open-source tool for training and fine-tuning large language models. Axolotl is configured through YAML files rather than custom code, and this skill gives the agent patterns for common choices: parameter-efficient methods like LoRA and QLoRA, preference-tuning methods such as DPO, KTO, ORPO and GRPO, distributed training with FSDP or DeepSpeed, and dataset formats. The skill is mostly a set of reference notes and short code snippets pulled from the official documentation, so the agent can answer questions and draft configs more accurately. It does not itself run any training.

Great for

  • ML engineers who are setting up or debugging an Axolotl fine-tuning job
  • Developers who want an agent to draft a YAML config for a specific model and dataset format
  • Teams planning multi-GPU training who need guidance on FSDP, DeepSpeed or context parallelism

Skip it if

  • Non-technical users: fine-tuning needs GPUs, dataset preparation and familiarity with ML tooling, and the skill cannot provide those.
  • Anyone expecting verified, up-to-date answers: the SKILL.md says it was auto-generated from documentation and some sections are incomplete (for example, a context_parallel_size pattern is only a bare name with no example).
  • Production decisions based solely on this skill: check the official Axolotl docs and your own test results, since the content is a scraped summary.

Try asking

  • “Write an Axolotl YAML config to LoRA fine-tune a Llama model on my instruction dataset.”
  • “Explain how context_parallel_size changes the global batch size on my 8-GPU machine.”
  • “Show me the dataset format I need for DPO training in Axolotl.”
  • “Help me enable FSDP version 2 in my Axolotl config and check the options.”
Step-by-step guide: how to use Axolotl LLM Fine-Tuning GuideInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    axolotl.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip axolotl.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip axolotl.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Ask the agent to quote the relevant reference file section so you can verify settings against the official Axolotl docs.
  • Give the agent your model name, GPU count and dataset format up front so it can tailor the YAML rather than give generic examples.
  • Treat generated configs as drafts: run a small test job before committing GPU hours to a full training run.

What's inside

  • references/dataset-formats.md
  • references/index.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.